U-ST-M3：面向低开销毫米波V2I通信的不确定性感知时空多模态多假设波束预测

顾鹏; 周恩治; 范智强; 郑植; 兰萍

doi:10.11999/JEIT260670

U-ST-M3：面向低开销毫米波V2I通信的不确定性感知时空多模态多假设波束预测

doi: 10.11999/JEIT260670 cstr: 32379.14.JEIT260670

顾鹏^1,,
周恩治^2,,
范智强^1,,
郑植^3,,
兰萍^{1, 4, ,}

1.
西藏大学拉萨 850000
2.
西华大学成都 610039
3.
电子科技大学成都 611731
4.
西藏自治区边境安全与智慧治理重点实验室拉萨 850000

基金项目: 西藏自治区科技计划重点研发及转化重点专项(XZ202601ZY0163)，广东省基础与应用基础研究基金项目(2024A1515140025)，西藏大学研究生高水平人才培养计划(2026-GSP-S089)

详细信息

作者简介:
顾鹏：男，硕士生，研究方向为通信感知一体化，邮箱　gupeng@stu.xzu.edu.cn

周恩治：男，博士，助理教授，研究方向为6G通信

范智强：男，硕士生，研究方向为计算机视觉

郑植：男，博士，研究员，研究方向为智能感知与信号处理、统计与阵列信号处理

兰萍：女，硕士，副教授，研究方向为智能信息处理、计算机视觉，邮箱　lanping@xzu.edu.cn

通讯作者:
兰萍　lanping@xzu.edu.cn

中图分类号: TN929.5; TP183
计量
- 文章访问数: 18
- HTML全文浏览量: 5
- PDF下载量: 3
- 被引次数: 0
出版历程
- 修回日期: 2026-11-02
- 录用日期: 2026-11-02
- 网络出版日期: 2026-11-09

U-ST-M3: Uncertainty-Aware Spatio-Temporal Multi-Modal Multi-Hypothesis Beam Prediction for Low-Overhead mmWave V2I Communications

GU Peng^1
,,
ZHOU Enzhi^2
,,
FAN Zhiqiang^1
,,
ZHENG Zhi^3
,,
LAN Ping^{1, 4
, ,}

1.
Xizang University, Lhasa, 850000, China
2.
Xihua University, Chengdu, 610039, China
3.
University of Electronic Science and Technology of China, Chengdu, 611731, China
4.
Key Laboratory of Border Security and Intelligent Governance of the Xizang Autonomous Region, Lhasa, 850000, China

Funds: Key Research and Transformation Projects of the Science and Technology Program of the Xizang Autonomous Region(XZ202601ZY0163), Guangdong Basic and Applied Basic Research Fund Project(2024A1515140025), Xizang University Graduate High-Level Talent Training Program(2026-GSP-S089)

摘要

摘要: 毫米波车联网通信中的波束管理依赖快速而准确的波束选择，而传统基于实时信道估计或穷举扫描的方案通常会引入较高的训练与搜索开销。针对这一问题，该文研究环境辅助的多模态毫米波波束预测，并将其建模为面向低开销波束管理的候选波束生成任务。为充分利用动态车联网场景中的异构感知信息，提出一种不确定性感知的时空多模态多假设Mamba架构(Uncertainty-Aware Spatio-Temporal Multi-Modal Multi-Hypothesis Mamba, U-ST-M3)。该框架将RGB图像、激光雷达(Light Detection and Ranging, LiDAR)、雷达(Radio Detection and Ranging, Radar)、GPS以及历史射频(Radio Frequency, RF)功率统一纳入时序预测过程，通过RGB-LiDAR特征增强、GPS条件特征调制、RF感知时序编码和模态级不确定性感知融合，建模多源感知信息的时序依赖与跨模态互补性。进一步地，U-ST-M3构建多假设波束预测头，生成各假设下的波束预测得分及对应的假设评分，并从评分最高的假设中提取Top-3候选波束。为兼顾候选覆盖能力与预测偏差，设计面向基于距离的准确性评分(Distance-Based Accuracy, DBA)的学习目标，在训练中引入候选覆盖和波束索引距离相关约束。基于DeepSense 6G Scenario 32-34的实验结果表明，U-ST-M3在14.42M参数量下取得了0.7842的Overall Top-3 Accuracy和0.9012的DBA Score，其中DBA Score接近当前最优模型，Top-3指标优于现有对比方法。结合模型规模、推理时延及扫描时间的分析，所提方法能够在较小参数规模下生成具有较高覆盖率的候选波束集合，为动态车联网中的低开销毫米波波束管理提供支持。
- 毫米波波束预测 /
- 多模态学习 /
- 不确定性感知融合 /
- 时空Mamba /
- 多假设预测
Abstract: Objective Beam management in millimeter-wave (mmWave) vehicular networks requires accurate beam selection under rapidly changing propagation conditions. Vehicle motion, blockage, and multipath changes can alter the preferred beam, while frequent channel measurements and exhaustive beam sweeping introduce training and search overhead. Environmental sensing offers complementary information about the communication scene, but heterogeneous observations differ in representation and reliability. A prediction framework therefore needs to integrate their temporal and cross-modal information while accounting for variations in their contributions. This study investigates environment-assisted beam prediction for vehicle-to-infrastructure (V2I) communications and formulates it as a candidate beam generation task. Instead of requiring a single prediction to identify the optimal beam, the task aims to generate a compact candidate set that covers the optimal beam and maintains small beam-index deviations. The objective is to improve candidate quality with a relatively small model and support subsequent beam management when historical multimodal inputs are already available. Methods An uncertainty-aware spatio-temporal multimodal multi-hypothesis Mamba framework, termed U-ST-M3, is proposed for the dynamic V2I scenario (Fig. 1). Red, Green, and Blue (RGB) images, Light Detection and Ranging (LiDAR), Radio Detection and Ranging (Radar), Global Positioning System (GPS) information, and historical Radio Frequency (RF) power measurements are organized into four feature branches: a joint RGB–LiDAR branch, a Radar branch, a GPS branch, and an RF branch (Fig. 2). In the joint branch, LiDAR-derived Bird’s-Eye View (BEV) features are used to gate visual features without assuming pixel-level geometric correspondence. GPS-conditioned modulation enhances the joint and Radar representations, while the RF branch captures energy and temporal-difference information. Future RF measurements are used only for supervision and training constraints. Sample-dependent uncertainty proxies determine branch fusion weights, and a Mamba encoder models the fused historical sequence. A shared decoder generates multiple hypotheses, each with beam logits and a hypothesis score. During inference, the three highest-ranked beams from the highest-scoring hypothesis form the candidate set. Training combines DBA-oriented classification, coverage, index-distance, RF-utility, and spectral-efficiency losses with hypothesis scoring, diversity, and usage-balance terms. Results and Discussions Experiments are conducted on Scenarios 32-34 of the DeepSense 6G dataset, with 80% of the data used for training, 10% for validation, and 10% for testing. U-ST-M3 achieves an Overall DBA Score of 0.9012, close to VLM’s 0.9016, and the highest reported Overall Top-3 Accuracy of 0.7842, exceeding ICMFE by 9.62 percentage points (Tables 1-2). Its nighttime DBA scores outperform the listed baselines, while daytime performance remains weaker. The model has 14.42 million parameters, the fewest among the listed methods, and requires 12.63 billion floating-point operations. Average forward-pass latency is measured on an NVIDIA RTX 4090 GPU with a batch size of one and 32-bit floating-point (FP32) precision. The measured latency is lower than those of TII, MMFF, and ICMFE but slower than Vision-Position. Baseline accuracy and complexity values come from the literature under their respective input configurations; available latencies are measured separately. Ablations show that additional modalities or fusion components do not consistently improve DBA (Table 3). Adding the full loss to the six-hypothesis configuration raises DBA from 0.8382 to 0.8590; the final four-hypothesis model reaches 0.9012 after further tuning, so this gain cannot be isolated to hypothesis count. On the communication side, an illustrative serial scanning configuration is considered with a 15 kHz subcarrier spacing and a normal cyclic prefix, yielding an average OFDM symbol duration of approximately 71.43 μs. Assuming four measurement symbols per beam, scanning all 64 beams requires approximately 18.29 ms of cumulative measurement time. The measured candidate generation latency is lower than this estimate under the specified assumptions. This comparison is limited to model inference latency and the estimated scanning time; end-to-end beam acquisition latency is not measured. Historical input acquisition, candidate confirmation, and reporting overhead require separate consideration in practical deployment. Conclusions U-ST-M3 integrates modality-specific encoding, GPS-conditioned modulation, uncertainty-aware fusion, temporal modeling, and multi-hypothesis prediction into a unified candidate beam generation framework. The results demonstrate high Top-3 candidate coverage and competitive DBA performance with a relatively small parameter count. The reported latency and the comparison with the specified serial scanning estimate support its potential for low-overhead beam management when historical inputs are available. The framework provides a practical basis for further investigation of candidate confirmation and recovery procedures. Future work will examine daytime beam-neighborhood consistency, reduce computational cost, and evaluate acquisition and feedback overhead in an explicitly defined communication process.
- Millimeter-wave Beam Prediction /
- Multimodal Learning /
- Uncertainty-aware Fusion /
- Spatio-temporal Mamba /
- Multi-hypothesis Prediction

HTML全文

图 1 动态车联网毫米波通信系统

下载: 全尺寸图片幻灯片

图 2 所提出的U-ST-M3模型整体架构

下载: 全尺寸图片幻灯片

表 1 所提模型和其他方法在DBA Score与复杂度方面的比较

Year	Method	S32	S33	S34	Overall	Parameters	FLOPs
2022	Vision-Position^[13]	0.8804	0.8778	0.8783	0.8788	33.02M	7.14G
2023	TII^[16]	0.8909	0.8805	0.8992	0.8902	66.16M	11.42G
2023	DEQFusion^[30]	0.8925	0.8768	0.8971	0.8888	75.61M	11.47G
2023	AsyFFNet^[31]	0.8916	0.8757	0.8979	0.8884	77.32M	11.74G
2024	MMFF^[5]	0.8979	0.8804	0.8932	0.8905	74.04M	56.85G
2025	ICMFE^[24]	0.9020	0.8874	0.9074	0.8989	74.03M	11.37G
2026	VLM^[32]	0.9027	0.8916	0.9105	0.9016	75.09M	11.09G
-	U-ST-M3 (本文方案)	0.8724	0.9187	0.9125	0.9012	14.42M	12.63G

下载: 导出CSV

表 2 所提模型与其他方法在Top-3 Accuracy和推理效率方面的比较

Method	S32	S33	S34	Overall	Latency(ms)
Vision-Position^[13]	0.6599	0.6626	0.6122	0.6449	1.8327
TII^[16]	0.6700	0.6715	0.6899	0.6771	22.1193
DEQFusion^[30]	0.6882	0.6594	0.6891	0.6789	-
AsyFFNet^[31]	0.6694	0.6553	0.6891	0.6713	-
MMFF^[5]	0.6973	0.6650	0.6747	0.6790	25.9532
ICMFE^[24]	0.6995	0.6635	0.7011	0.6880	23.8991
U-ST-M3 (本文方案)	0.7391	0.8176	0.7959	0.7842	9.5059

下载: 导出CSV

表 3 消融实验结果

Method	S32	S33	S34	Overall
U-ST-M3 (本文方案)	0.8724	0.9187	0.9125	0.9012
+Full loss	0.8360	0.8687	0.8724	0.8590
+Multi-Hypothesis ($ H=6 $)	0.8117	0.8492	0.8537	0.8382
+Uncertainty-weighted fusion	0.8408	0.8962	0.8912	0.8761
+FiLM conditioning	0.8416	0.8919	0.8922	0.8752
+ Radar(all 5 modalities)	0.8433	0.8931	0.8924	0.8763
+ RGB + LiDAR (CMFE)	0.8463	0.8922	0.8920	0.8768
+GPS	0.8423	0.8985	0.8915	0.8774
RF-only baseline	0.8414	0.8963	0.8912	0.8763

下载: 导出CSV

参考文献(32)

[1]	XUE Qing, JI Chengwang, MA Shaodan, et al. A survey of beam management for mmWave and THz communications towards 6G[J]. IEEE Communications Surveys & Tutorials, 2024, 26(3): 1520–1559. doi: 10.1109/COMST.2024.3361991.
[2]	KHAN M Q, GABER A, SCHULZ P, et al. Machine learning for millimeter wave and terahertz beam management: A survey and open challenges[J]. IEEE Access, 2023, 11: 11880–11902. doi: 10.1109/ACCESS.2023.3242582.
[3]	LIU Fan, CUI Yuanhao, MASOUROS C, et al. Integrated sensing and communications: Toward dual-functional wireless networks for 6G and beyond[J]. IEEE Journal on Selected Areas in Communications, 2022, 40(6): 1728–1767. doi: 10.1109/JSAC.2022.3156632.
[4]	HENG Yuqiang and ANDREWS J G. Machine learning-assisted beam alignment for mmWave systems[J]. IEEE Transactions on Cognitive Communications and Networking, 2021, 7(4): 1142–1155. doi: 10.1109/TCCN.2021.3078147.
[5]	ZHANG Haotian, GAO Shijian, CHENG Xiang, et al. Integrated sensing and communications toward proactive beamforming in mmWave V2I via multi-modal feature fusion (MMFF)[J]. IEEE Transactions on Wireless Communications, 2024, 23(11): 15721–15735. doi: 10.1109/TWC.2024.3401686.
[6]	仲伟志, 何艺, 段洪涛, 等. 可重构智能表面辅助的V2I通信系统联合波束赋形算法[J]. 电子与信息学报, 2024, 46(8): 3117–3125. doi: 10.11999/JEIT231324. ZHONG Weizhi, HE Yi, DUAN Hongtao, et al. Joint beamforming algorithm for reconfigurable intelligent surface-aided V2I communication system[J]. Journal of Electronics & Information Technology, 2024, 46(8): 3117–3125. doi: 10.11999/JEIT231324.
[7]	MORAIS J, BCHBOODI A, PEZESHKI H, et al. Position-aided beam prediction in the real world: How useful GPS locations actually are?[C]. ICC 2023 - IEEE International Conference on Communications, Rome, Italy, 2023: 1824–1829. doi: 10.1109/ICC45041.2023.10278998.
[8]	ALKHATEEB A, CHARAN G, OSMAN T, et al. DeepSense 6G: A large-scale real-world multi-modal sensing and communication dataset[J]. IEEE Communications Magazine, 2023, 61(9): 122–128. doi: 10.1109/MCOM.006.2200730.
[9]	JIANG Shuaifeng and ALKHATEEB A. Computer vision aided beam tracking in a real-world millimeter wave deployment[C]. 2022 IEEE Globecom Workshops (GC Wkshps), Rio de Janeiro, Brazil, 2022: 142–147. doi: 10.1109/GCWkshps56602.2022.10008648.
[10]	JIANG Shuaifeng, CHARAN G, and ALKHATEEB A. LiDAR aided future beam prediction in real-world millimeter wave V2I communications[J]. IEEE Wireless Communications Letters, 2023, 12(2): 212–216. doi: 10.1109/LWC.2022.3219409.
[11]	DEMIRHAN U and ALKHATEEB A. Radar aided 6G beam prediction: Deep learning algorithms and real-world demonstration[C]. 2022 IEEE Wireless Communications and Networking Conference (WCNC), Austin, USA, 2022: 2655–2660. doi: 10.1109/WCNC51071.2022.9771564.
[12]	孙昆阳, 姚睿, 祝汉城, 等. 一种测试时间自适应的夜间图像辅助波束预测方法[J]. 电子与信息学报, 2025, 47(12): 5156–5165. doi: 10.11999/JEIT250530. SUN Kunyang, YAO Rui, ZHU Hancheng, et al. A test-time adaptive method for nighttime image-aided beam prediction[J]. Journal of Electronics & Information Technology, 2025, 47(12): 5156–5165. doi: 10.11999/JEIT250530.
[13]	CHARAN G, OSMAN T, HREDZAK A, et al. Vision-position multi-modal beam prediction using real millimeter wave datasets[C]. 2022 IEEE Wireless Communications and Networking Conference (WCNC), Austin, USA, 2022: 2727–2731. doi: 10.1109/WCNC51071.2022.9771835.
[14]	CHARAN G, HREDZAK A, STODDARD C, et al. Towards real-world 6G drone communication: Position and camera aided beam prediction[C]. 2022 IEEE Global Communications Conference (GLOBECOM), Rio de Janeiro, Brazil, 2022: 2951–2956. doi: 10.1109/GLOBECOM48099.2022.10000718.
[15]	CHARAN G, DEMIRHAN U, MORAIS J, et al. Multi-modal beam prediction challenge 2022: Towards generalization[EB/OL]. https://arxiv.org/abs/2209.07519, 2022.
[16]	TIAN Yu, ZHAO Qiyang, KHERROUBI Z E A, et al. Multimodal transformers for wireless communications: A case study in beam prediction[J]. ITU Journal on Future and Evolving Technologies, 2023, 4(3): 461–471. doi: 10.52953/JWRA8095.
[17]	SHI Binpu, LI Min, ZHAO Mingmin, et al. Multimodal deep learning empowered millimeter-wave beam prediction[C]. 2024 IEEE 99th Vehicular Technology Conference (VTC2024-Spring), Singapore, Singapore, 2024: 1–6. doi: 10.1109/VTC2024-Spring62846.2024.10683225.
[18]	TARIQ S, ARFETO B E, KHALID U, et al. Deep quantum-transformer networks for multimodal beam prediction in ISAC systems[J]. IEEE Internet of Things Journal, 2024, 11(18): 29387–29401. doi: 10.1109/JIOT.2024.3420455.
[19]	CHENG Lu, ZHANG Hongliang, DI Boya, et al. Large language models empower multimodal integrated sensing and communication[J]. IEEE Communications Magazine, 2025, 63(5): 190–197. doi: 10.1109/MCOM.004.2400281.
[20]	ZHENG Can, HE Jiguang, CAI Guofa, et al. BeamLLM: Vision-empowered mmWave beam prediction with large language models[C]. 2025 IEEE 102nd Vehicular Technology Conference (VTC2025-Fall), Chengdu, China, 2025: 1–6. doi: 10.1109/VTC2025-Fall65116.2025.11309918.
[21]	赵川斌, 许伟华, 林博, 等. 融合视觉的多模态通信感知一体化关键技术及原型验证[J]. 电子与信息学报, 2026, 48(2): 487–498. doi: 10.11999/JEIT250685. ZHAO Chuanbin, XU Weihua, LIN Bo, et al. Vision enabled multimodal integrated sensing and communications: Key technologies and prototype validation[J]. Journal of Electronics & Information Technology, 2026, 48(2): 487–498. doi: 10.11999/JEIT250685.
[22]	CHARAN G, ALRABEIAH M, OSMAN T, et al. Camera based mmWave beam prediction: Towards multi-candidate real-world scenarios[J]. IEEE Transactions on Vehicular Technology, 2025, 74(4): 5897–5913. doi: 10.1109/TVT.2024.3506948.
[23]	PARK J, AHN J H, SEO J, et al. Occlusion-aware vision-aided beam tracking for multi-user V2I mmWave networks[C]. ICC 2025 - IEEE International Conference on Communications, Montreal, Canada, 2025: 2210–2216. doi: 10.1109/ICC52391.2025.11161281.
[24]	ZHU Qihao, WANG Yu, LI Wenmei, et al. Advancing multi-modal beam prediction with cross-modal feature enhancement and dynamic fusion mechanism[J]. IEEE Transactions on Communications, 2025, 73(9): 7931–7940. doi: 10.1109/TCOMM.2025.3548021.
[25]	VUCKOVIC K, HOSSEINI S M, and RAHNAVARD N. Revisiting performance metrics for multimodal mmWave beam prediction using deep learning[C]. MILCOM 2024 - 2024 IEEE Military Communications Conference, Washington, USA, 2024: 881–887. doi: 10.1109/MILCOM61039.2024.10773747.
[26]	GU A and DAO T. Mamba: Linear-time sequence modeling with selective state spaces[C]. The First Conference on Language Modeling (COLM), Philadelphia, USA, 2024: 1–32.
[27]	ZHANG Rongsheng, ZHANG Ruichen, LU Yang, et al. Mamba for wireless communications and networking: Principles and opportunities[J]. IEEE Communications Magazine, 2026: 1–7. doi: 10.1109/MCOM.001.2500436.
[28]	廖帝灵, 赖涛, 黄海风, 等. LightMamba: 一种轻量级Mamba用于高光谱图形和激光雷达数据联合分类网络[J]. 电子与信息学报, 2025, 47(12): 4937–4947. doi: 10.11999/JEIT250981. LIAO Diling, LAI Tao, HUANG Haifeng, et al. LightMamba: A lightweight Mamba network for the joint classification of HSI and LiDAR data[J]. Journal of Electronics & Information Technology, 2025, 47(12): 4937–4947. doi: 10.11999/JEIT250981.
[29]	SHI Huaguang, JIN Kaibo, REN Xiaoquan, et al. ChannelMamba: A mamba-driven selective state-space model for channel prediction of high-mobility MIMO in 6G IoT[J]. IEEE Transactions on Wireless Communications, 2026, 25: 5291–5305. doi: 10.1109/TWC.2025.3617502.
[30]	NI Jinhong, BAI Yalong, ZHANG Wei, et al. Deep equilibrium multimodal fusion[EB/OL]. https://arxiv.org/abs/2306.16645, 2023.
[31]	LI Wei, GAO Yunhao, ZHANG Mengmeng, et al. Asymmetric feature fusion network for hyperspectral and SAR image classification[J]. IEEE Transactions on Neural Networks and Learning Systems, 2023, 34(10): 8057–8070. doi: 10.1109/TNNLS.2022.3149394.
[32]	WANG Ji, TANG Bin, XIAO Jian, et al. When vision-language model (VLM) meets beam prediction: A multimodal contrastive learning framework[J]. IEEE Transactions on Vehicular Technology, 2026, 75(7): 15113–15118. doi: 10.1109/TVT.2026.3663461.