Design of Lightweight Gated Recurrent Unit Network Model Based on Memristor

HUA Honghu; XU Jia; ZHANG Bohao; WANG Wei; LI Zhiwei; LIU Haijun

doi:10.11999/JEIT260152

Article Contents

Article Navigation > Journal of Electronics & Information Technology > 2025 >

HUA Honghu, XU Jia, ZHANG Bohao, WANG Wei, LI Zhiwei, LIU Haijun. Design of Lightweight Gated Recurrent Unit Network Model Based on Memristor[J]. Journal of Electronics & Information Technology. doi: 10.11999/JEIT260152

Citation:

HUA Honghu, XU Jia, ZHANG Bohao, WANG Wei, LI Zhiwei, LIU Haijun. Design of Lightweight Gated Recurrent Unit Network Model Based on Memristor[J]. Journal of Electronics & Information Technology. doi: 10.11999/JEIT260152

Citation:

PDF( 2370 KB)

Design of Lightweight Gated Recurrent Unit Network Model Based on Memristor

doi: 10.11999/JEIT260152 cstr: 32379.14.JEIT260152

College of Electronic Science and Technology, National University of Defense Technology, Changsha 410073, China

Funds: The National Natural Science Foundation of China (62074166, 62304254, 62104256, 62404253, U23A20322)

Received Date: 2026-02-05
Accepted Date: 2026-06-24
Rev Recd Date: 2026-06-02

Available Online: 2026-07-04

Abstract

Abstract

Objective With the slowdown of Complementary Metal-Oxide-Semiconductor (CMOS) technology scaling and the inherent memory-computation separation of von Neumann architectures, conventional computing systems face increasing challenges in processing large-scale sequential data. Memristors provide a promising solution because of their high integration density, fast switching speed, and synaptic plasticity. Memristor crossbar arrays naturally support Vector-Matrix Multiplication (VMM) in the analog domain, enabling energy-efficient in-memory computing. As a representative recurrent neural network, the Gated Recurrent Unit (GRU) has achieved excellent performance in sequential tasks such as trajectory prediction and urban sound classification. However, conventional hardware implementations of GRU networks require frequent data transfer between memory and processing units, resulting in high energy consumption and limited throughput. Although memristor-based GRU implementations improve computational efficiency, their large parameter size and high weight precision require substantial hardware resources and reduce deployment reliability on resource-constrained memristor arrays. In addition, device non-idealities, such as conductance fluctuations, further reduce inference accuracy. Existing memristor-based GRU methods generally treat weights and activations using the same quantization strategy without considering their different hardware implementation characteristics, and they provide limited robustness against device variations. This paper addresses these issues through a hardware-algorithm co-design strategy. Methods This paper proposes a lightweight memristor-based GRU network model. A 1T1R (one-transistor-one-resistor) memristor crossbar array is adopted for weight mapping and analog Multiply-Accumulate (MAC) operations. Signed weights are represented by differential pairs of positive and negative conductance matrices because memristor conductance values are inherently non-negative. A linear transformation is used to map trained network weights to memristor conductance values. To account for the different hardware implementation paths of weights and activations, a device-aware fusion quantization method based on performance analysis is proposed. Symmetric quantization is applied to weights stored in the memristor array because the zero-centered quantization range eliminates zero-point storage and simplifies write-driver circuit design. In contrast, asymmetric quantization is applied to activation values computed in peripheral circuits, thereby preserving the dynamic range and reducing quantization error. To improve robustness against memristor conductance fluctuations, weight noise training is incorporated into Quantization-Aware Training (QAT). Gaussian noise with an intensity determined by the device variation parameter is injected into quantized weights during each forward pass. This strategy acts as a regularizer that guides the model toward flatter loss minima and improves tolerance to weight perturbations. During backpropagation, the straight-through estimator updates the full-precision floating-point weights, whereas noise is dynamically resampled in every forward pass. Results and Discussions On the public UrbanSound8K dataset, the proposed full-precision lightweight memristor-based GRU network model achieves a classification accuracy of 93.94%. After applying the device-aware fusion quantization method, the 6-bit quantized model achieves 92.68% accuracy, corresponding to only a 1.26% decrease while reducing weight precision by 81.25% (Table 1). The proposed model outperforms Dilated Convolution (78.00%), LM-MFCC+GRU (92.00%), TFFS-DNN (88.74%), TFCNN (93.10%), and CL-Transformer (92.95%) under their full-precision settings (Table 2). Under noisy input conditions with Signal-to-Noise Ratios (SNRs) ranging from −10 dB to 10 dB, the 6-bit quantized model exhibits robustness comparable to or better than that of the full-precision model, demonstrating the effectiveness of the proposed device-aware fusion quantization strategy (Table 3). From the perspectives of storage, hardware resources, and device feasibility, 6-bit quantization reduces weight storage from 5.6 MB to 1.05 MB, corresponding to a compression ratio of 81.2%, while requiring only 2.8 million memristor cells under the 1T1R mapping scheme. Weight noise training also substantially improves robustness against device non-idealities. When the conductance variation reaches 14%, the classification accuracy increases from 82.97% to 91.14%. At the maximum simulated variation of 28%, the accuracy increases from 54.23% to 87.01% (Fig. 7), demonstrating improved tolerance to memristor device variations. On a self-constructed true-false trajectory dataset, the lightweight memristor-based GRU network model achieves 97.35% accuracy at full precision and 96.51% after 6-bit quantization, with only a 0.84% decrease, outperforming the Dilated Convolution baseline (Table 4). To further verify its applicability to different sequential tasks, the lightweight memristor-based GRU network model is evaluated on lithium-ion battery State-of-Charge (SOC) estimation using a public dataset. The 6-bit quantized model achieves Root Mean Square Errors (RMSEs) of 1.48%, 0.79%, and 0.74% at 0 °C, 25 °C, and 45 °C, respectively, outperforming the existing memristor-based GRU implementation. The proposed model also achieves lower RMSEs than the comparison method at all evaluated quantization precisions of 6 bits and above (Table 5). Conclusions This paper presents a lightweight memristor-based GRU network model for hardware deployment. By combining device-aware fusion quantization with weight noise training integrated into Quantization-Aware Training (QAT), the model achieves substantial memory compression while maintaining high classification accuracy and improving robustness to memristor device non-idealities. Experimental results on multiple datasets and sequential tasks demonstrate that the 6-bit quantized model preserves competitive accuracy and stable performance, providing an effective solution for deploying GRU networks on resource-constrained memristor-based edge computing platforms.
- Gated Recurrent Unit (GRU),
- Memristor,
- Network model quantization,
- Urban sound classification

FullText(HTML)

References(20)

References

[1]	STRUKOV D B, SNIDER G S, STEWART D R, et al. The missing memristor found[J]. Nature, 2008, 453(7191): 80–83. doi: 10.1038/nature06932.
[2]	LECUN Y, BENGIO Y, and HINTON G. Deep learning[J]. Nature, 2015, 521(7553): 436–444. doi: 10.1038/nature14539.
[3]	刘凇佐, 王虔, 李磊, 等. 粒子群优化的门控循环单元网络漂流浮标轨迹预测[J]. 电子与信息学报, 2024, 46(8): 3295–3304. doi: 10.11999/JEIT230945. LIU Songzuo, WANG Qian, LI Lei, et al. Gated recurrent unit network of particle swarm optimization for drifting buoy trajectory prediction[J]. Journal of Electronics & Information Technology, 2024, 46(8): 3295–3304. doi: 10.11999/JEIT230945.
[4]	WANG Jiayang, JI Xiaoyue, DONG Zhekang, et al. Circuit design of memristor-based GRU and its applications in SOC estimation[C]. 2023 IEEE International Conference on Consumer Electronics (ICCE), Las Vegas, USA, 2023: 1–5. doi: 10.1109/ICCE56470.2023.10043585.
[5]	TONG Peiwen, XU Hui, SUN Yi, et al. Lightweight and highly robust memristor-based hybrid neural networks for electroencephalogram signal processing[J]. Chinese Physics B, 2023, 32(7): 078505. doi: 10.1088/1674-1056/ac9cbc.
[6]	李源堃, 王泽, 张清天, 等. NAS4CIM: 面向忆阻器存算一体芯片的神经网络结构搜索框架[J]. 电子与信息学报, 2025, 47(12): 4948–4958. doi: 10.11999/JEIT250978. LI Yuankun, WANG Ze, ZHANG Qingtian, et al. NAS4CIM: Tailored neural network architecture search for RRAM-based compute-in-memory chips[J]. Journal of Electronics & Information Technology, 2025, 47(12): 4948–4958. doi: 10.11999/JEIT250978.
[7]	蔺海荣, 段晨星, 邓晓衡, 等. 双忆阻类脑混沌神经网络及其在IoMT数据隐私保护中应用[J]. 电子与信息学报, 2025, 47(7): 2194–2210. doi: 10.11999/JEIT241133. LIN Hairong, DUAN Chenxing, DENG Xiaoheng, et al. Dual-memristor brain-like chaotic neural network and its application in IoMT data privacy protection[J]. Journal of Electronics & Information Technology, 2025, 47(7): 2194–2210. doi: 10.11999/JEIT241133.
[8]	BALASKAS K, KARATZAS A, SAD C, et al. Hardware-aware DNN compression via diverse pruning and mixed-precision quantization[J]. IEEE Transactions on Emerging Topics in Computing, 2024, 12(4): 1079–1092. doi: 10.1109/TETC.2023.3346944.
[9]	PERRIN M, GUICQUERO W, PAILLE B, et al. Hardware-aware Bayesian neural architecture search of quantized CNNs[J]. IEEE Embedded Systems Letters, 2025, 17(1): 42–45. doi: 10.1109/LES.2024.3434379.
[10]	CHEN Junren, WU Huaqiang, GAO Bin, et al. Optimization strategy for accelerating multi-bit resistive weight programming on the RRAM array[C]. 2019 IEEE International Workshop on Future Computing (IWOFC), Hangzhou, China, 2019: 1–3. doi: 10.1109/IWOFC48002.2019.9078447.
[11]	HONG Haiqiao, DU Zhiyuan, JIANG Mingrui, et al. Memristor-based adaptive analog-to-digital conversion for efficient and accurate compute-in-memory[J]. Nature Communications, 2025, 16(1): 9749. doi: 10.1038/s41467-025-65233-w.
[12]	LI Can, WANG Zhongrui, RAO Mingyi, et al. Long short-term memory networks in memristor crossbar arrays[J]. Nature Machine Intelligence, 2019, 1(1): 49–57. doi: 10.1038/s42256-018-0001-4.
[13]	HUANG Lixing, YU Hongqi, CHEN Changlin, et al. A training strategy for improving the robustness of memristor-based binarized convolutional neural networks[J]. Semiconductor Science and Technology, 2022, 37(1): 015013. doi: 10.1088/1361-6641/ac31e3.
[14]	SUN Yi, XU Hui, WANG Chao, et al. A Ti/AlO_x/TaO_x/Pt analog synapse for memristive neural network[J]. IEEE Electron Device Letters, 2018, 39(9): 1298–1301. doi: 10.1109/LED.2018.2860053.
[15]	CHEN Yan, GUO Qian, LIANG Xinyan, et al. Environmental sound classification with dilated convolutions[J]. Applied Acoustics, 2019, 148: 123–132. doi: 10.1016/j.apacoust.2018.12.019.
[16]	PENG Ning, CHEN Aibin, ZHOU Guoxiong, et al. Environment sound classification based on visual multi-feature fusion and GRU-AWS[J]. IEEE Access, 2020, 8: 191100–191114. doi: 10.1109/ACCESS.2020.3032226.
[17]	MU Wenjie, YIN Bo, HUANG Xianqing, et al. Environmental sound classification using temporal-frequency attention based convolutional neural network[J]. Scientific Reports, 2021, 11(1): 21552. doi: 10.1038/s41598-021-01045-4.
[18]	WU Bo and ZHANG Xiaoping. Environmental sound classification via time–frequency attention and framewise self-attention-based deep neural networks[J]. IEEE Internet of Things Journal, 2022, 9(5): 3416–3428. doi: 10.1109/JIOT.2021.3098464.
[19]	CHEN Xu, WANG Mei, KAN Ruixiang, et al. Improved patch-mix transformer and contrastive learning method for sound classification in noisy environments[J]. Applied Sciences, 2024, 14(21): 9711. doi: 10.3390/app14219711.
[20]	CHEN Yanan, LUO Wei, CARTER M, et al. Organic electrode for non-aqueous potassium-ion batteries[J]. Nano Energy, 2015, 18: 205–211. doi: 10.1016/j.nanoen.2015.10.015.