Soft Error Sensitivity Analysis and Field-Hierarchical Fault-Tolerant Design for CSR-Based Sparse Neural Networks

WANG Chenxu; YANG Shanqiang; SHEN Lexiang; LI Jianfeng; XU Tianliang; LV Zhenbin

doi:10.11999/JEIT260621

Article Contents

Article Navigation > Journal of Electronics & Information Technology > 2025 >

WANG Chenxu, YANG Shanqiang, SHEN Lexiang, LI Jianfeng, XU Tianliang, LV Zhenbin. Soft Error Sensitivity Analysis and Field-Hierarchical Fault-Tolerant Design for CSR-Based Sparse Neural Networks[J]. Journal of Electronics & Information Technology. doi: 10.11999/JEIT260621

Citation:

WANG Chenxu, YANG Shanqiang, SHEN Lexiang, LI Jianfeng, XU Tianliang, LV Zhenbin. Soft Error Sensitivity Analysis and Field-Hierarchical Fault-Tolerant Design for CSR-Based Sparse Neural Networks[J]. Journal of Electronics & Information Technology. doi: 10.11999/JEIT260621

Citation:

WANG Chenxu, YANG Shanqiang, SHEN Lexiang, LI Jianfeng, XU Tianliang, LV Zhenbin. Soft Error Sensitivity Analysis and Field-Hierarchical Fault-Tolerant Design for CSR-Based Sparse Neural Networks[J]. Journal of Electronics & Information Technology. doi: 10.11999/JEIT260621

PDF( 2527 KB)

Soft Error Sensitivity Analysis and Field-Hierarchical Fault-Tolerant Design for CSR-Based Sparse Neural Networks

doi: 10.11999/JEIT260621 cstr: 32379.14.JEIT260621

1.
School of Information Science and Engineering, Harbin Institute of Technology (Weihai), Weihai 264209, China
2.
Shandong Provincial Key Laboratory of Marine Electronic Information and Intelligent Unmanned Systems, Weihai 264209, China
3.
School of Astronautics, Harbin Institute of Technology, Harbin 150001, China

Funds: The Major scientific and technological innovation projects of Shandong Province of China (2022ZLGX04), The National Natural Foundation of China (U2106202), Shandong Provincial Natural Science Foundation (ZR2023MA074)

Received Date: 2026-05-18
Accepted Date: 2026-09-14
Rev Recd Date: 2026-09-13

Available Online: 2026-09-21

Abstract

Abstract

Objective The Compressed Sparse Row (CSR) format is widely used in sparse neural network accelerators to reduce weight storage and support row-wise computation. In addition to nonzero weights, column indices and row boundaries are stored as metadata. These fields perform different functions during sparse computation and may lead to different fault consequences after a Single-Event Upset (SEU). Errors in weights directly perturb arithmetic operands, whereas errors in metadata may redirect activation accesses or alter row boundaries. Existing uniform protection schemes do not distinguish these field-dependent fault characteristics and introduce redundant protection for all stored data. Therefore, a field-specific fault analysis method and a selective protection strategy are developed to establish the relationship between field functions and fault consequences while reducing unnecessary storage and decoding overhead. Methods A software fault-injection framework is established based on CSR execution rules and validity constraints. The values, col_indices, and row_ptr fields store nonzero weights, column indices for activation access, and row boundaries, respectively (Fig. 1). For each fault case, one logical storage bit is flipped in a selected field. A corrupted column index is classified as an erroneous connection when it remains within the valid input range; otherwise, it is classified as an address failure. A corrupted internal row boundary is classified as an erroneous boundary when the range and nondecreasing-order constraints remain satisfied; otherwise, it is classified as a structural failure. Executable cases are processed through inference, and the accuracy decrease is calculated as the difference between fault-free accuracy and post-fault accuracy. The original stored value is restored after each trial to ensure independent single-bit fault injections (Fig. 2). The first and last row_ptr endpoints of each layer are treated as configuration constants, and only internal boundaries are considered in the row_ptr fault and protection spaces. LeNet-5/MNIST and VGG16/CIFAR-10 models are trained, pruned, and quantized using 8-bit signed INTeger (INT8) weights. For LeNet-5, all 22 7264 logical storage bits in five target layers are exhaustively evaluated, including 111784 bit in values, 111 784 bit in col_indices, and 3696 bit in row_ptr. For VGG16, layer-stratified quota sampling is adopted for 16 target layers to evaluate larger sparse matrices and wider metadata fields. The sampled fault space contains 4 096, 16 384, and 30 944 cases for values, col_indices, and row_ptr, respectively (Table 2). Each model is evaluated with and without injected faults using the same fixed set of 1 000 test images. Field-level failure rates are calculated based on the corresponding fault cases. Single-Error Correction and Double-Error Detection (SECDED) codes are applied for selective protection evaluation. P0 denotes the unprotected baseline; P1 protects internal row_ptr boundaries; P2 additionally protects col_indices; and P3 independently protects all three fields. PT jointly encodes values and col_indices while separately protecting row_ptr. Protection effectiveness is evaluated using the original LeNet-5 CSR data-bit fault space under the condition that each SECDED codeword contains at most one erroneous bit. Accuracy comparisons are performed on the fixed set of 218 487 executable cases under P0. Element-level encoding converts 8-bit and 16-bit data into 13-bit and 22-bit codewords, respectively, whereas 64-bit memory-word encoding converts 64-bit data into 72-bit codewords. Storage overhead is calculated relative to the unprotected data capacity of each encoding organization. Codewords are generated during model loading or memory writes, and decoding and correction are performed on the inference read path. Decoder functions are verified through C and Register-Transfer Level (RTL) co-simulation with an independent bit-level reference model. Module-level Out-Of-Context (OOC) placement and routing are performed on a Xilinx XC7Z020 device under a 10 ns reference clock period. Results and Discussions In the LeNet-5/MNIST experiments, all 111 784 values fault cases remain executable, whereas address failures in col_indices and structural failures in row_ptr account for 5.83% and 61.20% of their corresponding fault cases, respectively. The maximum accuracy decrease reaches 0.70 percentage points (pp) for executable weight faults and 10.50 pp for executable boundary faults (Table 3). The larger influence of row boundary errors results from the fact that one incorrect boundary may redistribute multiple consecutive nonzero entries between adjacent output rows. For col_indices, the aggregate address-failure rate reaches 31.52% at bit 7 across the five evaluated layers. No address failure occurs in the first fully connected layer because its 256-dimensional input range covers all possible unsigned 8-bit index values (Fig. 3). For row_ptr, bit flips at positions 8～15 always violate structural constraints and contribute 81.70% of all structural failures (Fig. 4). These results demonstrate that fault consequences are determined by the interaction between bit significance, valid index ranges, and CSR structural constraints. In the VGG16 samples, address and structural failures account for 36.89% and 75.15% of the corresponding col_indices and row_ptr cases, respectively, confirming that the field-dependent failure characteristics remain under larger sparse models and wider metadata fields. In the LeNet-5 protection evaluation, P1 eliminates all 2 262 structural failures caused by row_ptr faults, and P2 further eliminates all 6 515 address failures caused by col_indices faults. Among the evaluated configurations that remove both failure types, P2 achieves the lowest storage overhead, with additional costs of 31.35% for element-level encoding and 6.35% for 64-bit memory-word encoding. P3 and PT additionally protect values and restore baseline accuracy for all cases in the fixed executable-case set. Joint encoding reduces the element-level overhead from 62.09% for P3 to 37.50% for PT. However, under the 64-bit memory-word organization, both configurations require 12.50% overhead because each 64-bit data word is expanded with eight check bits (Fig. 5). Post-route implementation results show that P1, P2, P3, and PT require 72, 119, 166, and 147 LookUp Tables (LUTs), respectively (Table 5). The protected element-level read-path modules achieve longest internal data paths of 6.50～6.86 ns and Worst Negative Slack (WNS) values of 3.11～3.44 ns, satisfying the setup-time constraint under the 10 ns reference period. Since decoder branches operate in parallel, their delays do not accumulate along a single critical path. For the 64-bit decoder, pipelining reduces the longest internal data path from 9.38 ns to 6.49 ns, corresponding to a 30.8% reduction, while increasing the flip-flop count from 140 to 379. The pipelined decoder core has a High-Level Synthesis (HLS) scheduling latency of three cycles and an initiation interval of one cycle, allowing one new codeword to be accepted per cycle. Conclusions The fault consequences of CSR-based sparse neural networks depend on field function, bit position, and CSR validity constraints. Field-specific analysis shows that protecting row_ptr and col_indices is sufficient to eliminate structural and address failures within the evaluated LeNet-5 fault space, whereas protecting values is required to suppress accuracy degradation caused by weight errors. Full-field SECDED protection restores baseline accuracy for the fixed executable-case set. Joint codewords reduce parity overhead for element-level encoding but provide no additional storage benefit under the evaluated 64-bit memory-word organization. Selective protection reduces decoding resources compared with full-field protection, whereas pipelined decoding improves timing performance at the cost of additional registers. These results provide a basis for designing reliability-aware CSR storage protection by matching field coverage and codeword organization with application requirements and hardware constraints.
- Fault-tolerant design,
- Sparse neural networks,
- Compressed Sparse Row (CSR),
- Single-Event Upset (SEU),
- Field-selective protection

FullText(HTML)

References(30)

References

[1]	BOLCHINI C, CASSANO L, and MIELE A. Resilience of deep learning applications: A systematic literature review of analysis and hardening techniques[J]. Computer Science Review, 2024, 54: 100682. doi: 10.1016/j.cosrev.2024.100682.
[2]	GUAN Hui, NING Lin, LIN Zhen, et al. In-place zero-space memory protection for CNN[C]. The 33rd Conference on Neural Information Processing Systems, Vancouver, Canada, 2019: 515.
[3]	SANTOS F F D, PIMENTA P F, LUNARDI C, et al. Analyzing and increasing the reliability of convolutional neural networks on GPUs[J]. IEEE Transactions on Reliability, 2019, 68(2): 663–677. doi: 10.1109/TR.2018.2878387.
[4]	SUN Wenhao, ZOU Zhiwei, LIU Deng, et al. Bit-balance: Model-hardware codesign for accelerating NNs by exploiting bit-level sparsity[J]. IEEE Transactions on Computers, 2024, 73(1): 152–163. doi: 10.1109/TC.2023.3324477.
[5]	HAN Song, LIU Xingyu, MAO Huizi, et al. EIE: Efficient inference engine on compressed deep neural network[C]. The 43rd Annual International Symposium on Computer Architecture, Seoul, South Korea, 2016: 243–254. doi: 10.1109/ISCA.2016.30.
[6]	TROMMER E, WASCHNECK B, and KUMAR A. dCSR: A memory-efficient sparse matrix representation for parallel neural network inference[C]. 2021 IEEE/ACM International Conference On Computer Aided Design (ICCAD), Munich, Germany, 2021: 1–9. doi: 10.1109/ICCAD51958.2021.9643506.
[7]	ZHANG Chen, GAO Shijie, DAI Guohao, et al. Fine-grained structured sparse computing for FPGA-based AI inference[J]. IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems, 2025, 44(7): 2544–2557. doi: 10.1109/TCAD.2024.3524356.
[8]	LEE J H, PARK B, KONG J, et al. Row-wise product-based sparse matrix multiplication hardware accelerator with optimal load balancing[J]. IEEE Access, 2022, 10: 64547–64559. doi: 10.1109/ACCESS.2022.3184116.
[9]	Li Baoting, Zhang Danqing, Zhao Pengfei, et al. DQ-STP: An efficient sparse on-device training processor based on low-rank decomposition and quantization for DNN[J]. IEEE Transactions on Circuits and Systems I: Regular Papers, 2024, 71(4): 1665–1678. doi: 10.1109/TCSI.2024.3364093.
[10]	闫爱斌, 李坤, 黄正峰, 等. 两种面向宇航应用的高可靠性抗辐射加固技术静态随机存储器单元[J]. 电子与信息学报, 2024, 46(10): 4072–4080. doi: 10.11999/JEIT240082. YAN Aibin, LI Kun, HUANG Zhengfeng, et al. Two highly reliable radiation hardened by design static random access memory cells for aerospace applications[J]. Journal of Electronics & Information Technology, 2024, 46(10): 4072–4080. doi: 10.11999/JEIT240082.
[11]	柏娜, 李钢, 许耀华, 等. 应用于航空航天领域的低功耗多节点抗辐射静态随机存取存储器设计[J]. 电子与信息学报, 2025, 47(3): 850–858. doi: 10.11999/JEIT240294. BAI Na, LI Gang, XU Yaohua, et al. Low-power multi-node radiation-hardened SRAM design for aerospace applications[J]. Journal of Electronics & Information Technology, 2025, 47(3): 850–858. doi: 10.11999/JEIT240294.
[12]	蔡烁, 帅威, 胡星, 等. 面向高速读写需求的宇航级抗辐射静态随机存储器加固单元设计[J]. 电子与信息学报, 2026, 48(5): 1894–1904. doi: 10.11999/JEIT251287. CAI Shuo, SHUAI Wei, HU Xing, et al. Design of an aerospace-grade radiation-hardened SRAM cell for high-speed read/write applications[J]. Journal of Electronics & Information Technology, 2026, 48(5): 1894–1904. doi: 10.11999/JEIT251287.
[13]	WANG Haibin, WANG Yangsheng, XIAO Jianhua, et al. Impact of single-event upsets on convolutional neural networks in Xilinx Zynq FPGAs[J]. IEEE Transactions on Nuclear Science, 2021, 68(4): 394–401. doi: 10.1109/TNS.2021.3062014.
[14]	MAHMOUD A, AGGARWAL N, NOBBE A, et al. PyTorchFI: A runtime perturbation tool for DNNs[C]. The 50th Annual IEEE/IFIP International Conference on Dependable Systems and Networks Workshops, Valencia, Spain, 2020: 25–31. doi: 10.1109/DSN-W50199.2020.00014.
[15]	MAHMOUD A, HARI S K S, FLETCHER C W, et al. Optimizing selective protection for CNN resilience[C]. The 32nd IEEE International Symposium on Software Reliability Engineering, Wuhan, China, 2021: 127–138. doi: 10.1109/ISSRE52982.2021.00025.
[16]	陈子洋, 张萌, 张吉良. 一种星载在轨神经网络的容错设计方法[J]. 电子与信息学报, 2023, 45(9): 3234–3243. doi: 10.11999/JEIT230378. CHEN Ziyang, ZHANG Meng, and ZHANG Jiliang. A fault-tolerant design of spaceborne onboard neural network[J]. Journal of Electronics & Information Technology, 2023, 45(9): 3234–3243. doi: 10.11999/JEIT230378.
[17]	张青, 刘成, 刘波, 等. 容错深度学习加速器跨层优化[J]. 计算机研究与发展, 2024, 61(6): 1370–1387. doi: 10.7544/issn1000-1239.202331005. ZHANG Qing, LIU Cheng, LIU Bo, et al. Cross-layer optimization for fault-tolerant deep learning accelerators[J]. Journal of Computer Research and Development, 2024, 61(6): 1370–1387. doi: 10.7544/issn1000-1239.202331005.
[18]	LEE S S and YANG J S. Value-aware parity insertion ECC for fault-tolerant deep neural network[C]. The Design, Automation & Test in Europe Conference & Exhibition, Antwerp, Belgium, 2022: 724–729. doi: 10.23919/DATE54114.2022.9774543.
[19]	柳姗姗, 金辉, 刘思佳, 等. 面向投票类AI分类器的零冗余存储器容错设计[J]. 集成电路与嵌入式系统, 2024, 24(6): 1–8. LIU Shanshan, JIN Hui, LIU Sijia, et al. Redundancy-free error-tolerant memory design for voting-based AI classifiers[J]. Integrated Circuits and Embedded Systems, 2024, 24(6): 1–8.
[20]	TRAIOLA M, KRITIKAKOU A, and SENTIEYS O. harDNNing: A machine-learning-based framework for fault tolerance assessment and protection of DNNs[C]. The 28th IEEE European Test Symposium, Venice, Italy, 2023: 1–6. doi: 10.1109/ETS56758.2023.10174178.
[21]	ZHAO Kai, DI Sheng, LI Sihuan, et al. FT-CNN: Algorithm-based fault tolerance for convolutional neural networks[J]. IEEE Transactions on Parallel and Distributed Systems, 2021, 32(7): 1677–1689. doi: 10.1109/TPDS.2020.3043449.
[22]	HARI S K S, SULLIVAN M B, TSAI T, et al. Making convolutions resilient via algorithm-based error detection techniques[J]. IEEE Transactions on Dependable and Secure Computing, 2022, 19(4): 2546–2558. doi: 10.1109/TDSC.2021.3063083.
[23]	GAO Zhen, QI Yanmao, SHI Jinchang, et al. Detect and replace: Efficient soft error protection of FPGA-based CNN accelerators[J]. IEEE Transactions on Very Large Scale Integration (VLSI) Systems, 2025, 33(1): 66–74. doi: 10.1109/tvlsi.2024.3443834.
[24]	LECUN Y, BOTTOU L, BENGIO Y, et al. Gradient-based learning applied to document recognition[J]. Proceedings of the IEEE, 1998, 86(11): 2278–2324. doi: 10.1109/5.726791.
[25]	SIMONYAN K and ZISSERMAN A. Very deep convolutional networks for large-scale image recognition[C]. The 3rd International Conference on Learning Representations, San Diego, USA, 2015.
[26]	KRIZHEVSKY A. Learning multiple layers of features from tiny images[R]. Toronto: University of Toronto, 2009.
[27]	GOLNARI P A and MALIK S. Evaluating matrix representations for error-tolerant computing[C]. The Design, Automation & Test in Europe Conference & Exhibition (DATE), Lausanne, Switzerland, 2017: 1659–1662. doi: 10.23919/DATE.2017.7927260.
[28]	AHMED S T, HEMARAM S, and TAHOORI M B. NN-ECC: Embedding error correction codes in neural network weight memories using multi-task learning[C]. 2024 IEEE 42nd VLSI Test Symposium (VTS), Tempe, USA, 2024: 1–7, doi: 10.1109/VTS60656.2024.10538886.
[29]	PARK T, GORGIN S, KIM D, et al. PoP-ECC: Robust and flexible error correction against multi-bit upsets in DNN accelerators[C]. The 62nd ACM/IEEE Design Automation Conference, San Francisco, USA, 2025: 1–7. doi: 10.1109/DAC63849.2025.11133373.
[30]	JO M J and LEE Y S. Stegano-ECC: Enhancing DNN fault tolerance with embedded parity for important bits[J]. Journal of Systems Architecture, 2026, 171: 103651. doi: 10.1016/j.sysarc.2025.103651.