An Energy-Efficient Accelerator Design for Fully Integer Vision Transformer Inference

CHEN Ke; YI Weichao; CHEN Yanqiu; GONG Yu; YAN Chenggang; WU Bi; LIU Weiqiang

doi:10.11999/JEIT260712

Article Contents

Article Navigation > Journal of Electronics & Information Technology > 2026 >

CHEN Ke, YI Weichao, CHEN Yanqiu, GONG Yu, YAN Chenggang, WU Bi, LIU Weiqiang. An Energy-Efficient Accelerator Design for Fully Integer Vision Transformer Inference[J]. Journal of Electronics & Information Technology. doi: 10.11999/JEIT260712

Citation:

CHEN Ke, YI Weichao, CHEN Yanqiu, GONG Yu, YAN Chenggang, WU Bi, LIU Weiqiang. An Energy-Efficient Accelerator Design for Fully Integer Vision Transformer Inference[J]. Journal of Electronics & Information Technology. doi: 10.11999/JEIT260712

Citation:

PDF( 3424 KB)

An Energy-Efficient Accelerator Design for Fully Integer Vision Transformer Inference

doi: 10.11999/JEIT260712 cstr: 32379.14.JEIT260712

College of Integrated Circuits, Nanjing University of Aeronautics and Astronautics, Nanjing 211106, China

Funds: The National Natural Science Foundation of China (62425404), Basic Research Program of Jiangsu (BK20253023)

Accepted Date: 2026-11-02
Rev Recd Date: 2026-11-02

Available Online: 2026-11-19

Abstract

Abstract

Objective Vision Transformers (ViTs) have demonstrated exceptional performance in complex computer vision tasks, driving the continuous introduction of novel Transformer-based architectures, such as DeiT and Swin Transformer, into the field. However, the hardware deployment of Transformer models on edge devices is severely hindered by their massive parameter volumes and three key non-linear functions (LayerNorm, Softmax, and GELU), which entail complex mathematical operations such as reciprocals, square roots, and exponentials. Traditional compression schemes attempt to maintain algorithmic accuracy by designing complex quantization factors or employing quantization-aware training (QAT) and fine-tuning strategies, which inevitably introduce substantial computational overhead. Furthermore, existing hardware acceleration designs for these non-linear functions frequently retain power-hungry floating-point arithmetic or rely on complex iterative and piecewise linear logic, fundamentally limiting hardware energy efficiency. To overcome these limitations, this paper proposes a hardware-algorithm co-designed accelerator architecture tailored for the all-integer inference of Vision Transformers. At the software level, we introduce a unified all-integer approximation method for the computation of the non-linear functions and the subsequent requantization of layer outputs, completely eliminating hardware-unfriendly operations. At the hardware level, a dedicated non-linear computing module supporting multi-operator reuse is proposed to enhance hardware utilization. Through this deep hardware-algorithm synergy, the proposed design aims to boost hardware inference efficiency while preserving algorithmic accuracy. Methods This study first employs a Post-Training Quantization (PTQ) strategy to compress the ViT model, converting the quantization factors into a Mantissa-Exponent (ME) representation. This enables on-chip requantization alignment across layers through low-overhead multiplication and bit-shift operations. For the three critical non-linear functions—Layer Normalization, Softmax, and GELU—this paper extends the ME computational scheme. By incorporating lightweight Look-Up Tables (LUTs), computationally expensive division, exponential, and square root calculations are transformed into hardware-friendly integer multiplications, bit-shifts, and table lookups. At the hardware level, a ViT accelerator architecture supporting end-to-end on-chip inference is proposed (Fig. 2). To mitigate off-chip memory access overhead, this architecture adopts a hybrid dataflow strategy combining Input-Stationary (IS) and Output-Stationary (OS) schemes. Furthermore, by mathematically unfolding the all-integer inference processes of the three non-linear operators, we extract their common underlying arithmetic operations and design a highly reusable non-linear computing module (Fig. 3). By extensively reusing LUT resources and fundamental computing units, this module efficiently executes the all-integer inference of the aforementioned complex non-linear operators, thereby significantly enhancing the area utilization of the overall hardware architecture. Results and Discussions To evaluate the proposed all-integer inference method for the three non-linear operators, this study assesses the accuracy using the ViT model on the ImageNet-2012 dataset. In this setup, the weights and activations of all layers are quantized to INT8, and the mantissa-exponent parameters involved in both the non-linear operations and requantization alignment are uniformly configured with 8 bit data. Experimental results demonstrate that, relative to the baseline with quantized linear layers, the additional accuracy degradation introduced by the all-integer approximation method for the three non-linear functions is strictly limited to less than 1% (Table 2). This indicates that the proposed method effectively maintains algorithmic accuracy while thoroughly eliminating the overhead associated with floating-point arithmetic and division operations. To further evaluate the proposed hardware accelerator architecture, we performed an ASIC implementation and conducted synthesis and simulation using Synopsys Design Compiler (DC) at a clock frequency of 500 MHz. The proposed accelerator achieves a peak throughput of 1024 GOPS, consumes 314.5 mW of power, and occupies a compact area of only 3.08 mm². Notably, the proposed non-linear computing module supporting multi-operator reuse accounts for merely 7.28% of the power consumption and 16.99% of the area of the overall on-chip computing logic (Fig. 4). By successfully circumventing the massive overhead caused by complex mathematical operations, such as reciprocals and square roots, and dynamic sparse control logic, our pure-integer, static-pipeline architecture achieves a remarkable energy efficiency of 3.26 TOPS/W and an area efficiency of 332.47 GOPS/mm² (Table 3), demonstrating prominent advantages over existing state-of-the-art accelerators. Conclusions To address the deployment bottlenecks of Vision Transformer (ViT) models on edge hardware, specifically those imposed by requantization factor processing and complex non-linear function computations, this paper proposes a hardware-algorithm co-designed accelerator architecture tailored for all-integer inference. At the algorithmic level, by employing arithmetic multiplications, bit-shifts, and table-lookups, the proposed method thoroughly eliminates hardware-intensive operations, such as divisions and square roots, within LayerNorm, Softmax, and GELU. This effectively achieves the all-integer inference of these three non-linear functions alongside end-to-end on-chip requantization alignment. At the hardware level, the area efficiency of the accelerator architecture is significantly enhanced through the deep reuse of the underlying computational units across the three operators. Experimental results demonstrate that the proposed all-integer approximation method incurs minimal accuracy degradation without necessitating costly model retraining overhead. Operating at a frequency of 500 MHz, the proposed hardware accelerator delivers a peak throughput of 1024 GOPS. With a remarkable energy efficiency of 3.26 TOPS/W and an area efficiency of 332.47 GOPS/mm², the design exhibits competitive performance compared to existing state-of-the-art Transformer hardware solutions, thereby providing a robust and practical solution for the edge-based all-integer inference of ViT models.
- Transformer,
- Hardware accelerator,
- Approximate computing,
- Non-linear function,
- Hardware-Software Co-Design

FullText(HTML)

References(28)

References

[1]	VASWANI A, SHAZEER N, PARMAR N, et al. Attention is all you need[C]. Proceedings of the 31st International Conference on Neural Information Processing Systems, Long Beach, USA, 2017: 6000–6010.
[2]	DOSOVITSKIY A, BEYER L, KOLESNIKOV A, et al. An image is worth 16x16 words: Transformers for image recognition at scale[C]. 9th International Conference on Learning Representations, Vienna, Austria, 2021. (查阅网上资料, 未找到本条文献出版地信息, 请确认).
[3]	YUAN Zhihang, XUE Chenhao, CHEN Yiqi, et al. PTQ4ViT: Post-training quantization for vision transformers with twin uniform quantization[C]. 17th European Conference on Computer Vision - ECCV 2022, Tel Aviv, Israel, 2022: 191–207. doi: 10.1007/978-3-031-19775-8_12.
[4]	LIN Yang, ZHANG Tianyu, SUN Peiqin, et al. FQ-ViT: Post-training quantization for fully quantized vision transformer[C]. Proceedings of the Thirty-First International Joint Conference on Artificial Intelligence, Vienna, Austria, 2022: 1173–1179.
[5]	LI Zhikai, XIAO Junrui, YANG Lianwei, et al. RepQ-ViT: Scale reparameterization for post-training quantization of vision transformers[C]. 2023 IEEE/CVF International Conference on Computer Vision, Paris, France, 2023: 17181–17190. doi: 10.1109/ICCV51070.2023.01580.
[6]	卢迪, 王振发. 脉动阵列协同层融合的卷积神经网络加速器设计[J]. 电子与信息学报, 2026, 48(3): 1280–1291. doi: 10.11999/JEIT250867. LU Di and WANG Zhenfa. Design of a CNN accelerator based on systolic array collaboration with inter-layer fusion[J]. Journal of Electronics & Information Technology, 2026, 48(3): 1280–1291. doi: 10.11999/JEIT250867.
[7]	秦华标, 曹钦平. 基于FPGA的卷积神经网络硬件加速器设计[J]. 电子与信息学报, 2019, 41(11): 2599–2605. doi: 10.11999/JEIT190058. QIN Huabiao and CAO Qinping. Design of convolutional neural networks hardware acceleration based on FPGA[J]. Journal of Electronics & Information Technology, 2019, 41(11): 2599–2605. doi: 10.11999/JEIT190058.
[8]	NAG S, DATTA G, KUNDU S, et al. ViTA: A vision transformer inference accelerator for edge applications[C]. 2023 IEEE International Symposium on Circuits and Systems, Monterey, USA, 2023: 1–5. doi: 10.1109/ISCAS46773.2023.10181988.
[9]	YI C, MOH H, and HA S. Vision transformer inference on a CNN accelerator[C]. 2024 IEEE 42nd International Conference on Computer Design, Milan, Italy, 2024: 628–636. doi: 10.1109/iccd63220.2024.00101.
[10]	HUANG Mingqiang, LUO Junyi, DING Chenchen, et al. An integer-only and group-vector systolic accelerator for efficiently mapping vision transformer on edge[J]. IEEE Transactions on Circuits and Systems I: Regular Papers, 2023, 70(12): 5289–5301. doi: 10.1109/TCSI.2023.3312775.
[11]	李天阳, 张帆, 王松, 等. 基于FPGA的卷积神经网络和视觉Transformer通用加速器[J]. 电子与信息学报, 2024, 46(6): 2663–2672. doi: 10.11999/JEIT230713. LI Tianyang, ZHANG Fan, WANG Song, et al. FPGA-based unified accelerator for convolutional neural network and vision Transformer[J]. Journal of Electronics & Information Technology, 2024, 46(6): 2663–2672. doi: 10.11999/JEIT230713.
[12]	SHI Huihong, CHENG Xin, MAO Wendong, et al. P²-VIT: Power-of-two post-training quantization and acceleration for fully quantized vision transformer[J]. IEEE Transactions on Very Large Scale Integration (VLSI) Systems, 2024, 32(9): 1704–1717. doi: 10.1109/TVLSI.2024.3422684.
[13]	LIANG Yanbiao, SHI Huihong, and WANG Zhongfeng. M²-ViT: Accelerating hybrid vision transformers with two-level mixed quantization[J]. IEEE Transactions on Very Large Scale Integration (VLSI) Systems, 2025, 33(5): 1492–1496. doi: 10.1109/TVLSI.2024.3525184.
[14]	姜小波, 邓晗珂, 莫志杰, 等. 规则压缩模型和灵活架构的Transformer加速器设计[J]. 电子与信息学报, 2024, 46(3): 1079–1088. doi: 10.11999/JEIT230188. JIANG Xiaobo, DENG Hanke, MO Zhijie, et al. Design of transformer accelerator with regular compression model and flexible architecture[J]. Journal of Electronics & Information Technology, 2024, 46(3): 1079–1088. doi: 10.11999/JEIT230188.
[15]	TOUVRON H, CORD M, DOUZE M, et al. Training data-efficient image transformers & distillation through attention[C]. Proceedings of the 38th International Conference on Machine Learning, 2021: 10347–10357. (查阅网上资料, 未找到本条文献出版地信息, 请确认).
[16]	LIU Ze, LIN Yutong, CAO Yue, et al. Swin Transformer: Hierarchical vision transformer using shifted windows[C]. 2021 IEEE/CVF International Conference on Computer Vision, Montreal, Canada, 2021: 9992–10002. doi: 10.1109/ICCV48922.2021.00986.
[17]	DONG Peiyan, ZHUANG Jinming, YANG Zhuoping, et al. EQ-ViT: Algorithm-hardware co-design for end-to-end acceleration of real-time vision transformer inference on Versal ACAP architecture[J]. IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems, 2024, 43(11): 3949–3960. doi: 10.1109/TCAD.2024.3443692.
[18]	WU Zhuguanyu, ZHANG Jiayi, CHEN Jiaxin, et al. APHQ-ViT: Post-training quantization with average perturbation hessian based reconstruction for vision transformers[C]. 2025 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Nashville, USA, 2025: 9686–9695. doi: 10.1109/CVPR52734.2025.00905.
[19]	TAI Yushan, LIN Mingguang, and WU A Y A. TSPTQ-ViT: Two-scaled post-training quantization for vision transformer[C]. ICASSP 2023 - 2023 IEEE International Conference on Acoustics, Speech and Signal Processing, Rhodes Island, Greece, 2023: 1–5. doi: 10.1109/ICASSP49357.2023.10096817.
[20]	KIM S, GHOLAMI A, YAO Zhewei, et al. I-BERT: Integer-only BERT quantization[C]. Proceedings of the 38th International Conference on Machine Learning, 2021: 5506–5518. (查阅网上资料, 未找到本条文献出版地信息, 请确认).
[21]	LI Zhikai and GU Qingyi. I-ViT: Integer-only quantization for efficient vision transformer inference[C]. 2023 IEEE/CVF International Conference on Computer Vision, Paris, France, 2023: 17019–17029. doi: 10.1109/ICCV51070.2023.01565.
[22]	WU Yuchi, KUO C H, and TSUI C W. SAVE: Systolic array-based accelerator for vision transformer with efficient tiling strategy[C]. 2025 International VLSI Symposium on Technology, Systems and Applications, Hsinchu, China, 2025: 1–4. doi: 10.1109/VLSITSA64674.2025.11047111.
[23]	WANG Gang, CAI Siqi, LI Wenjie, et al. BSViT: A bit-serial vision transformer accelerator exploiting dynamic patch and weight bit-group quantization[J]. IEEE Transactions on Circuits and Systems I: Regular Papers, 2024, 71(9): 4064–4077. doi: 10.1109/TCSI.2024.3426653.
[24]	HSIUNG C L and CHANG T S. Low-power vision transformer accelerator with hardware-aware pruning and optimized dataflow[J]. IEEE Transactions on Circuits and Systems I: Regular Papers, 2026, 73(1): 360–369. doi: 10.1109/TCSI.2025.3585054.
[25]	CHEN Chunyun, LI Lantian, and SABRY ALY M M. ViTA: A highly efficient dataflow and architecture for vision transformers[C]. 2024 Design, Automation & Test in Europe Conference & Exhibition, Valencia, Spain, 2024: 1–6. doi: 10.23919/DATE58400.2024.10546565.
[26]	YOU Haoran, SUN Zhanyi, SHI Huihong, et al. ViTCoD: Vision transformer acceleration via dedicated algorithm and accelerator co-design[C]. 2023 IEEE International Symposium on High-Performance Computer Architecture, Montreal, Canada, 2023: 273–286. doi: 10.1109/HPCA56546.2023.10071027.
[27]	QIN Yubin, WANG Yang, DENG Dazheng, et al. FACT: FFN-attention co-optimized transformer architecture with eager correlation prediction[C]. Proceedings of the 50th Annual International Symposium on Computer Architecture, Orlando, USA, 2023: 22. doi: 10.1145/3579371.3589057.
[28]	WANG Yang, QIN Yubin, DENG Dazheng, et al. A 28nm 27.5TOPS/W approximate-computing-based transformer processor with asymptotic sparsity speculating and out-of-order computing[C]. 2022 IEEE International Solid-State Circuits Conference, San Francisco, USA, 2022: 1–3. doi: 10.1109/ISSCC42614.2022.9731686.