TaGate: A Transaction-aware Multi-chiplet Flow Control Framework
-
摘要: 多芯粒系统为维护全局缓存一致性而频繁地在芯粒间进行事务交互,产生了大量跨芯粒通信负载。一致性事务由多个按序执行的流量控制单元(Flow control unit, Flit)组成,当多个跨芯粒事务Flit并发时,后序Flit可能经同一芯粒接口传输,造成回传汇聚。相较于芯粒内片上网络(Network on Chip, NoC),芯粒间互连链路具有高延迟、低带宽的固有特征,无法承载回传汇聚产生的突发流量,导致事务Flit在接口处持续堆积,形成严重的拥塞热点。芯粒接口拥塞不仅会沿路由路径扩散阻碍本地NoC流量正常传输,还会使跨芯粒的一致性事务延迟持续攀升,使拥塞在多芯粒间扩散传播,导致系统全局性能恶化。然而,现有的流量控制研究多关注于链路优化,由于缺乏协议层视图,难以主动识别和干预一致性事务的流量拥塞问题。为此本文提出了TaGate框架,该框架基于事务语义感知进行跨层协同,将协议层的有序依赖关系映射到链路层流控策略,从而实现事务自适应的流量线速传输。经验证,TaGate能够有效避免回传汇聚现象,在高负载场景下消除了芯粒接口拥塞,并在维持芯粒间互连带宽的同时,本地事务平均延迟损失降低15%以上。Abstract:
Objective In multi-chiplet architectures, the substantial cross-chiplet flow generated by global cache coherence maintenance poses considerable transmission challenges. The interplay between the high-latency, low-bandwidth physical links among chiplets and the strict dependency rules of coherence protocols gives rise to the phenomenon of coherence transactions stalling readily due to backhaul aggregation. This stalling, in turn, further propagates along protocol dependency chains, resulting in multi-chiplet collaborative stagnation and system performance degradation. Traditional link-layer flow control mechanisms are designed for homogeneous packets; the underlying hardware interconnect structures generally lack transaction semantic awareness, rendering them incapable of predicting or preventing the backhaul aggregation that originates from protocol-layer causal dependencies, and unable to proactively intervene in the spread of non-deadlock blocking. Consequently, there is an urgent requirement for a multi-chiplet flow control mechanism that is transaction-aware and based on cross-layer coordination between the protocol and link layers. Methods This study proposes a transaction-aware flow gating framework for multi-chiplet architectures, termed TaGate. The fundamental objective of this approach is to instil a transaction semantic awareness and to methodically incorporate protocol-layer transaction dependencies into the link-layer flow control decision process. TaGate achieves cross-layer coordinated flow regulation by predicting backhaul flow aggregation hotspot, thereby preventing congestion at inter-chiplet interfaces. The framework comprises three core mechanisms that together form a complete chain from perception to decision-making. Firstly, a transaction semantic awareness mechanism is implemented, in which a hybrid architecture of combinational logic and a read-only lookup table(LUT) is employed to parse the critical transaction information carried in cross-chiplet flits in real time. In accordance with the characteristics of protocol dependency, the parsers employ a channel-driven selective parsing strategy. This ensures that full semantic extraction is performed only for those flit types directly relevant to transaction suspension behaviour, thereby guaranteeing low latency in semantic processing under high-concurrency scenarios. Secondly, TaGate proposes a finite-precision cross-layer transaction dependency mapping strategy, in which the transaction dependency table serves as the core data structure. To address the complexity of consistency protocols, this strategy employs a progressive reduction process comprising finite-range mapping, classification and induction, and unified encoding. This process enables the explicit mapping of complex transaction dependencies at the protocol layer into a small number of flow control constraints that can be executed at the link layer. Thirdly, an adaptive line-rate flow control algorithm has been designed to implement multi-stage dynamic flow control based on a token-bucket mechanism. The algorithm utilises protocol-layer transaction semantics and deterministic routing to predict the channels through which backhual flow will be aggregated. Subsequently, the algorithm dynamically adjusts the token generation rate based on the state of the link-layer channel buffers, thereby enabling differentiated flow injection regulation under varying load conditions. Results and Discussions The experimental results demonstrate that significant improvements are achieved by TaGate in suppressing response aggregation, preventing congestion diffusion, and preserving global performance. First, compared with the backhaul channel FIFO occupancy under balanced traffic conditions, response aggregation causes the interface to remain congested for approximately 80% of the time ( Fig. 12 ). TaGate predicts the congestion risk before the backhaul channel buffer occupancy rises and proactively reduces the injection rate, so that the peak buffer occupancy is lowered by more than 36% on average, and the average buffer occupancy is reduced by approximately 55%, compared with uncontrolled injection (Fig. 13 ). Moreover, cross-chiplet flits are temporarily stored in dedicated buffers through a physical isolation mechanism, thereby mitigating their impact on the transmission of local transaction traffic within the chiplet. After TaGate is enabled, the average transmission latency of intra-chiplet local traffic is reduced by 17.3% to 28.4% compared with uncontrolled continuous injection (Fig. 15 ), while UCIe throughput is almost unaffected (Fig. 16 ). Finally, under global mixed-traffic scenarios, TaGate maintains a high level of system throughput under heavy load, reduces the average latency of local transactions by approximately 15.7%, and incurs only about a 1.8% increase in cross-chiplet transaction latency.Conclusions This study proposes a transaction-aware, cross-layer collaborative flow gating framework designed to address interface congestion and congestion propagation issues caused by backhaul aggregation in multi-chiplet architectures. The framework achieves cross-layer coordination between the protocol and link layers through three mechanisms: a highly concurrent, low-latency transaction semantic awareness mechanism; a finite-precision cross-layer transaction dependency mapping strategy; and an adaptive line-rate flow control algorithm. In TaGate, the risk of backhaul aggregation for inter-chiplet flow is predicted at the injection decision stage, and congestion at the inter-chiplet interfaces is avoided through differentiated dynamic rate adjustment. This study reveals, at the theoretical level, the formation mechanisms and characteristics of deadlock-free blocking in multi-chiplet architectures. At the implementation level, it establishes an interaction pathway between transaction semantics awareness at the protocol layer and flow control decisions at the link layer. This provides a cross-layer collaborative solution for flow control in multi-chiplet architectures. -
Key words:
- Chiplet /
- Flow Control /
- Congestion Control /
- Cache Coherence
-
表 1 芯粒互连特征参数表
特征 片内互连 芯粒间互连 互连介质 片上金属层 基板 硅桥 中介层 互连线距 <1 μm
(BEOL金属层)100–130 μm
(C4凸点)25–55 μm
(桥内<4 μm)25–55 μm
(BEOL<2 μm)带宽 单链路带宽 2 Tb/s 256 GB/s(UCIe-S) 512 GB/s(UCIe-A) 819 GB/s(HBM3) 带宽密度 >10 TB/s(聚合) 224 GB/s/mm 10.5 Tbps/mm 819 GB/s/堆栈 单跳延迟 1–3 cycles(~1 ns) 10–100 ns
(协议栈往返)2–5 ns 2–3 ns 表 2 芯粒互连特征参数表
范围 参数类型 配置值 系统 芯粒数量 4 拓扑结构 芯粒间拓扑结构 2×2 Mesh NoC拓扑结构 4×4 Mesh 路由算法 XY维度序路由 芯粒间互连 UCIe时钟频率 1.5GHz D2D直连接口 4个/芯粒边界 接口FIFO深度 128 芯粒 NoC时钟频率 2 GHz 缓存一致性协议 AMBA CHI TaGate 注入缓冲区大小 256 flits 令牌桶深度 112(16阈值) 令牌生成速率 三级动态调整 *接口FIFO深度为回传通道的缓冲大小
*注入缓冲区在接收端专门用于缓冲Pre-Flit表 3 混合流量场景下相关测试数据
类型 无流控 TaGate流控 变化率 本地事务平均延迟 506166 ps426791 ps–15.68% 本地事务最大延迟 2013500 ps1855000 ps–7.87% 通道FIFO最大占用 128 Flits 107 Flits –16.41% 跨芯粒事务平均延迟 596496 ps609613 ps+1.75% 跨芯粒事务最大延迟 2757500 ps1994500 ps–27.67% UCIe平均带宽 82% 81% –1.00% -
[1] SCHÄTZLE F, FALQUEZ C, HO N, et al. Modeling Chiplet-to-Chiplet (C2C) communication for chiplet-based co-design[C]. Proceedings of the ISC High Performance 2025 Research Paper Proceedings (40th International Conference), Hamburg, Germany, 2025: 1–11. doi: 10.23919/ISC.2025.11018303. [2] MALLYA M B, GOEL B, and SOURDIS I. MEMPLEX: A memory system with replication and migration of data for multi-chiplet NUMA architectures[C]. Proceedings of the 39th ACM International Conference on Supercomputing (ICS '25), Salt Lake City, USA, 2025: 1219–1233. doi: 10.1145/3721145.3725776. [3] MAROÑAS M, NAVARRO A, AYGUADÉ E, et al. Mitigating the NUMA effect on task-based runtime systems[J]. The Journal of Supercomputing, 2023, 79(13): 14287–14312. doi: 10.1007/s11227-023-05164-9. [4] MUSAVI M, IRABOR E, DAS A, et al. Communication characterization of AI workloads for large-scale multi-chiplet accelerators[C]. Proceedings of 2025 IEEE International Symposium on Circuits and Systems (ISCAS), London, United Kingdom, 2025: 1–5. doi: 10.1109/ISCAS56072.2025.11043311. [5] NAGARAJAN V, SORIN D J, HILL M D, et al. A Primer on Memory Consistency and Cache Coherence[M]. 2nd ed. Cham, Switzerland: Springer, 2020: 91–106. doi: 10.1007/978-3-031-01764-3. [6] LI Weihang, GOENS A, OSWALD N, et al. Determining the minimum number of virtual networks for different coherence protocols[C]. Proceedings of 2024 ACM/IEEE 51st Annual International Symposium on Computer Architecture (ISCA), Buenos Aires, Argentina, 2024: 182–197. doi: 10.1109/ISCA59077.2024.00023. [7] MATOUSSI O. NoC performance model for efficient network latency estimation[C]. Proceedings of Design, Automation & Test in Europe Conference & Exhibition (DATE), Grenoble, France, 2021: 994–999. doi: 10.23919/DATE51398.2021.9474101. [8] BHASKAR A V and VENKATESH T G. Performance analysis of network-on-chip in many-core processors[J]. Journal of Parallel and Distributed Computing, 2021, 147: 196–208. doi: 10.1016/j.jpdc.2020.09.013. [9] LIU Yafei, LI Xiangyu, and YIN Shouyi. Review of chiplet-based design: System architecture and interconnection[J]. Science China Information Sciences, 2024, 67(10): 200401. doi: 10.1007/s11432-023-3926-8. [10] 卢美璇, 许浩博, 王颖, 等. 芯粒集成芯片架构-封装协同设计[J]. 电子与信息学报, 2025, 47(9): 2994–3009. doi: 10.11999/JEIT250626.LU Meixuan, XU Haobo, WANG Ying, et al. Co-design of architecture and packaging in chiplet[J]. Journal of Electronics & Information Technology, 2025, 47(9): 2994–3009. doi: 10.11999/JEIT250626. [11] ZHANG Jinming, CHEN Zhihong, YE Yaoyao, et al. IPDR: An inter-chiplet priority-driven deadlock resolution for 2-D/2.5-D multichiplet systems[J]. IEEE Transactions on Very Large Scale Integration (VLSI) Systems, 2025, 33(9): 2424–2437. doi: 10.1109/TVLSI.2025.3583289. [12] DAS SHARMA D. An introduction to the universal chiplet interconnect express® (UCIe®)[J]. ACM Computing Surveys, 2026, 58(13): 347. doi: 10.1145/3819235. [13] DAS SHARMA D, CHOUDHARY S, ONUFRYK P, et al. On-package memory with universal chiplet interconnect express (UCIe): A low power, high bandwidth, low latency and low cost approach[C]. Proceedings of 2025 IEEE Symposium on High-Performance Interconnects (HOTI), San Jose, USA, 2025: 87–96. doi: 10.1109/HOTI66940.2025.00027. [14] 王知非, 黄之闻, 叶天辰, 等. 面向芯粒互连的单端64 Gb/s全双工收发机设计[J]. 电子与信息学报, 2025, 47(9): 2979–2993. doi: 10.11999/JEIT250506.WANG Zhifei, HUANG Zhiwen, YE Tianchen, et al. A 64Gb/s single-ended simultaneous bi-directional transceiver for die-to-die interfaces[J]. Journal of Electronics & Information Technology, 2025, 47(9): 2979–2993. doi: 10.11999/JEIT250506. [15] AN S, OH J, and LIU Ming. Server chiplet networking[C]. Proceedings of the 24th ACM Workshop on Hot Topics in Networks (HotNets '25), UMD Campus College Park, USA, 2025: 289–299. doi: 10.1145/3772356.3772389. [16] ZHOU Shize, FAN Wenjie, LI Siyue, et al. An adaptive congestion-aware approximate communication (ACAC) scheme and implementation for network-on-chip systems[J]. Integration, 2026, 106: 102561. doi: 10.1016/j.vlsi.2025.102561. [17] MERINO A, ESCUDERO-SAHUQUILLO J, GARCIA P J, et al. A hybrid solution to provide end-to-end flow control and congestion management in high-performance interconnection networks[C]. Proceedings of 2024 IEEE 24th International Symposium on Cluster, Cloud and Internet Computing (CCGrid), Philadelphia, USA, 2024: 8–17. doi: 10.1109/CCGrid59990.2024.00011. [18] QI Xingyun, LAI Mingche, DONG Dezun, et al. PFT: A congestion avoidance method based on proactive flow throttling at endpoints[C]. Proceedings of 2021 IFIP/IEEE International Symposium on Integrated Network Management (IM), Bordeaux, France, 2021: 572–578. [19] LU Chengdi, CHEN Yuang, ZHANG Fangyu, et al. FAFC: Fast and accurate flow control in data center networks[J]. IEEE Transactions on Network and Service Management, 2026, 23: 6927–6941. doi: 10.1109/TNSM.2026.3726287. [20] 周武, 倪天明, 徐冬雨, 等. 面向芯粒互连网络的故障与拥塞联合感知自适应路由算法[J]. 电子与信息学报, 2025, 47(9): 3155–3166. doi: 10.11999/JEIT250294.ZHOU Wu, NI Tianming, XU Dongyu, et al. A joint fault and congestion-aware adaptive routing algorithm for chiplet interconnect networks[J]. Journal of Electronics & Information Technology, 2025, 47(9): 3155–3166. doi: 10.11999/JEIT250294. [21] 徐冬雨, 欧阳一鸣, 黄正峰, 等. 一种基于自适应容错链路的片上网络设计与研究[J]. 电子与信息学报, 2024, 46(11): 4092–4100. doi: 10.11999/JEIT240162.XU Dongyu, OUYANG Yiming, HUANG Zhengfeng, et al. A design of on-chip network with self-adaptive fault-tolerant link[J]. Journal of Electronics & Information Technology, 2024, 46(11): 4092–4100. doi: 10.11999/JEIT240162. [22] KONDOTH L, SHANKARAN R, SHENG Q Z, et al. A survey of adaptive routing techniques in two-dimensional network-on-chip architectures[J]. IEEE Communications Surveys & Tutorials, 2026, 28: 3195–3234. doi: 10.1109/COMST.2025.3636503. [23] 周宏伟, 陈志强, 曾坤, 等. 多芯粒网络中负载均衡的死锁解决算法[J]. 国防科技大学学报, 2025, 47(2): 146–154. doi: 10.11887/j.cn.202502013.ZHOU Hongwei, CHEN Zhiqiang, ZENG Kun, et al. Load-balanced deadlock resolution algorithm in multi-chiplet network[J]. Journal of National University of Defense Technology, 2025, 47(2): 146–154. doi: 10.11887/j.cn.202502013. [24] CHEN Zhiqiang, FU Wenwen, WANG Yongwen, et al. A deadlock-free bridge module for inter-chiplet cache-coherent communication in an open chiplet ecosystem[C]. Proceedings of 2026 IEEE International Symposium on High Performance Computer Architecture (HPCA), Sydney, Australia, 2026: 1–13. doi: 10.1109/HPCA68181.2026.11408455. [25] MAJUMDER P, KIM S, HUANG Jiayi, et al. Remote control: A simple deadlock avoidance scheme for modular systems-on-chip[J]. IEEE Transactions on Computers, 2021, 70(11): 1928–1941. doi: 10.1109/TC.2020.3029682. -
下载: