An Anomalous Traffic Detection Method Combining Stream Data Compression and Self-Supervised Graph Learning
-
摘要: 在网络流量规模持续增长、攻击手段日趋复杂的背景下，现有异常流量检测方法面临流特征提取效率低、对标注数据依赖强以及缺乏通信关系上下文建模的问题。为此，该文提出一种融合流数据压缩与图自监督学习的异常流量检测方法SketchGNN。首先，设计基于流数据压缩的自适应特征提取机制，依据流量分布动态调整数据结构，在有限内存开销下高效提取多样化流特征，并显著提升测量精度；之后，构建基于自监督学习的图神经网络模型，通过消息传递机制学习拓扑结构以建模流量上下文，生成对正常流量波动具备鲁棒性的高质量边嵌入，最终结合无监督分类算法实现无标注数据依赖的高效异常检测。实验结果表明，SketchGNN在多种无监督分类器上的检测准确率均不低于95.2%，且相比基线方案在特征测量误差关键指标上降低至少31.0%，验证了所提方法在大规模网络异常流量检测场景中的有效性与可扩展性。Abstract:
Objective With the continuous growth of network traffic scale and the increasing sophistication of attack methods, efficient and intelligent anomalous traffic detection has become essential for protecting critical information infrastructure. However, existing detection methods still face severe challenges when deployed in large-scale networks. On one hand, the analysis of raw packet sequences and deep learning-based end-to-end models incurs substantial computation and storage overhead, which is unaffordable in line-rate processing scenarios. On the other hand, flow records are usually treated as independent samples, and the topological structure and context information of inter-host communications are ignored, resulting in the lack of a global view when distributed and correlated threats are handled. Moreover, supervised learning schemes rely heavily on large amounts of labeled data, which can hardly be satisfied in practical deployment and weakens the generalization ability against unknown threats. Therefore, it is of great significance to develop an anomalous traffic detection method that supports efficient flow feature extraction under limited resources and achieves high-precision detection without relying on labeled data. Methods In the training phase, the graph encoder takes both the original communication graph and augmented negative samples as inputs to learn edge embeddings. The encoder outputs these embeddings to a discriminator. The discriminator estimates mutual information by contrasting edge embeddings of positive and negative samples against a global graph summary. The training objective maximizes scores for positive samples while minimizing them for negative ones. Through this self-supervised optimization, the encoder parameters are refined to enhance the discriminative power of edge embeddings. This iterative process continues until convergence via gradient descent. After training, the encoder parameters are fixed. The resulting edge embeddings then serve downstream anomaly detection tasks. In the inference phase, the feature extractor converts traffic into a communication graph. The trained graph encoder then generates corresponding edge embeddings. A lightweight classifier takes these embeddings as input for end-to-end anomaly detection, producing the final classification results. Results and Discussions Comprehensive experiments are conducted on four public datasets, i.e., CAIDA, CIC-IDS2018, UNSW-NB15, and TON-IoT. For feature extraction, under identical memory configurations, the average relative error (ARE) and per-flow weighted mean relative error (WMRE) of counter-type features measured by the optimized MFSketch are reduced by 31.5% and 31.0% on average compared with the fixed-structure baseline, and those of bitmap-type features are reduced by 36.1% and 34.9%, respectively ( Fig. 4 ). Meanwhile, high throughput is maintained on datasets with different traffic skewness, and an average of about 12Mpps is reached on the CAIDA dataset (Fig. 4(c) ). For detection accuracy, SketchGNN combined with PCA, HBOS, or IF stably achieves an accuracy no lower than 95.2%, a macro-F1 no lower than 90.1, and a weighted-F1 no lower than 96.7 on CIC-IDS2018 and UNSW-NB15, which outperforms Kitsune, Whisper, Anomal-E, and TS-IDS, whose accuracy either stays below 90% or fluctuates sharply across datasets (Fig. 5 ,Fig. 6 ). For detection efficiency, HBOS provides the most stable and highest throughput among the three classifiers (Fig. 7(a) ), and the end-to-end packet-level equivalent throughput of SketchGNN reaches 640Kpps, which is about 17 times that of Kitsune (37Kpps) and is comparable in magnitude to the DPDK-accelerated Whisper (1.3Mpps) (Fig. 7(b) ). In addition, the performance variation across different datasets is kept within 3%, demonstrating robust generalization to normal traffic fluctuations and diverse flow-level anomalous behaviors.Conclusions To address the challenges of high feature-extraction overhead, limited utilization of traffic context, and heavy reliance on labeled data, this paper proposes SketchGNN, an anomaly detection framework that integrates flow data compression with self-supervised graph learning. MFSketch, a dynamically configurable sketch, efficiently extracts and accurately measures diverse flow features under limited resources. Self-supervised graph neural networks then model host communication graphs and learn representations, enabling efficient anomaly detection without labeled data. Experimental results show that MFSketch can dynamically optimize its data structure based on traffic distribution, ensuring high-throughput, high-precision feature inputs for downstream detection tasks. The edge embeddings generated via graph-based self-supervised learning achieve higher detection accuracy across various unsupervised classifiers compared to baseline methods. The future work will further explore hybrid detection mechanisms that combine Deep Packet Inspection with programmable data planes to further enhance the ability to identify anomalous traffic at the application layer. -
表 1 支持的流级统计特征类型
直接捕获的流特征 计算所得的流特征 流数量(以五元组区分) 包长均值 TCP流数量(以五元组区分) 流速率均值(以包数计) UDP流数量(以五元组区分) 流速率均值(以字节数计) 流量大小(以包数计) 流量突增比(当前计数/历史均值) 流量大小(以字节数计) 流量熵值 
下载: 导出CSV
1 MFSketch自适应配置算法
输入：窗口$ t $的输入流量，初始哈希函数数量$ d $，阈值$ {\theta }_{H} $。 输出：优化后的MFSketch。 1. $ {d}^{cur}\leftarrow d $, $ {H}_{\text{prev}}\leftarrow 0 $, $ {\text{cnt}}_{\text{stab}}\leftarrow 0 $ 2. FOR window $ t $ DO 3. $ {\mathcal{C}}_{\text{win}}\leftarrow \varnothing $ 4. $ N\leftarrow \sum \limits_{f\in {\mathcal{C}}_{\text{win}}}{\mathcal{C}}_{\text{win}}[f] $ 5. $ H(t)\leftarrow -{\sum }_{f}\frac{{\mathcal{C}}_{\text{win}}[f]}{N}\log \frac{{\mathcal{C}}_{\text{win}}[f]}{N} $ //计算当前窗口流量熵值 6. $ \Delta H(t)\leftarrow |H(t)-{H}_{\text{prev}}|/{H}_{\text{prev}} $ 7. IF $ \Delta H(t) \gt {\theta }_{H} $ THEN 8. $ {d}^{op}\leftarrow \frac{\alpha NM}{e(k\alpha N+MF_{}^{\text{res}(k)})} $ //依据定理1计算最优哈希函数
数量9. IF $ {d}^{op}\neq {d}^{\text{c}ur} $ THEN 10. $ {d}^{\text{c}ur}\leftarrow {d}^{op} $, $ {\text{cnt}}_{\text{stab}}\leftarrow 0 $ 11. ELSE 12. $ {\text{cnt}}_{\text{stab}}={\text{cnt}}_{\text{stab}}+1 $ 13. IF $ {\text{cnt}}_{\text{stab}}\geq 3 $ THEN //确认当前草图配置是否稳定 14. $ {\text{cnt}}_{\text{stab}}\leftarrow 0 $ 15. $ {H}_{\text{prev}}\leftarrow H(t) $
下载: 导出CSV
2 面向边嵌入的图自监督学习算法
输入：原始图$ G(\mathcal{V},\mathcal{E}) $及其负样本$ \hat{G} $；节点特征${\mathbf{x}}_{v}=\{1,\cdots,1\} $,
$ \forall v\in \mathcal{V} $；边特征$ \{{\mathbf{e}}_{uv},\forall uv\in \mathcal{E}\} $；图编码器层数$ L $；权重矩阵
$ {\mathbf{W}}^{l},\forall l\in \{1,\cdots,L\} $；非线性激活函数$ \sigma $；聚合函数 AGG；训练
轮数$ K $。输出：优化后的图编码器$ \mathcal{G} $。 16. FOR $ v\in V $ DO 17. $ \mathbf{h}_{v}^{0}\leftarrow {\mathbf{x}}_{v} $ 18. FOR $ l\leftarrow 1 $ to $ L $ DO // 图编码器$ \mathcal{G} $ 19. FOR $ v\in \mathcal{V} $ DO 20. $ \mathbf{h}_{\mathcal{N}(v)}^{l}\leftarrow {\text{AGG}}_{l}(\{\mathbf{e}_{uv}^{l-1},\forall u\in \mathcal{N}(v),uv\in \mathcal{E}\}) $ 21. $ \mathbf{h}_{v}^{l}\leftarrow \sigma \left({\mathbf{W}}^{l}\cdot \text{CONCAT}(\mathbf{h}_{v}^{l-1},\mathbf{h}_{\mathcal{N}(v)}^{l})\right) $ 22. $ {\mathbf{z}}_{v}=\mathbf{h}_{v}^{L} $ 23. FOR $ epoch\leftarrow 1 $ to $ K $ DO 24. FOR $ (u,v)\in \mathcal{E} $ DO 25. $ {\mathbf{z}}_{uv}\leftarrow \text{CONCAT}({\mathbf{z}}_{u},{\mathbf{z}}_{v}) $ // 原始边嵌入 26. $ {\mathbf{\hat{z}}}_{uv}\leftarrow \text{CONCAT}({\mathbf{\hat{z}}}_{u},{\mathbf{\hat{z}}}_{v}) $ // 负样本边嵌入 27. $ \mathbf{\overline{s}}=\sigma \left(\dfrac{1}{|\mathcal{E}|}\displaystyle\sum \nolimits_{(u,v)\in \mathcal{E}}{\mathbf{z}}_{uv}\right) $ // 全局图摘要 28. $ \mathcal{D}({\mathbf{z}}_{uv},\mathbf{\overline{s}})=\sigma ({\mathbf{z}}_{uv}\mathbf{W\overline{s}}) $ 29. $ \mathcal{D}({\mathbf{\hat{z}}}_{uv},\mathbf{\overline{s}})=\sigma ({\mathbf{\hat{z}}}_{uv}\mathbf{W\overline{s}}) $ 30. $ \mathcal{L} = -\dfrac{1}{2|\mathcal{E}|}\displaystyle\sum \limits_{(u,v)\in \mathcal{E}}\left[\log \mathcal{D}({z}_{uv},\overline{s}) + \log (1 - \mathcal{D}({\hat{z}}_{uv},\overline{s}))\right] $ 31. Adam 优化，更新图编码器$ \mathcal{G} $参数 32. RETURN $ \mathcal{G} $
下载: 导出CSV
表 2 各类型攻击流量检测结果(TON-IoT数据集)
攻击类型 Precision Recall 宏F1 FPR FNR backdoor 97.88% 98.94% 98.38% 2.13% 1.06% ddos 89.10% 91.72% 90.06% 10.26% 8.28% dos 82.33% 83.12% 82.07% 13.82% 16.88% injection 87.70% 91.68% 88.56% 15.17% 8.32% mitm 90.62% 94.77% 92.59% 0.51% 5.23% password 80.24% 84.15% 78.16% 31.70% 15.85% ransomware 91.21% 94.19% 91.73% 11.51% 5.81% scanning 87.78% 90.31% 88.47% 12.30% 9.69% xss 78.98% 78.87% 78.80% 14.21% 21.13% 加权均值 86.93% 89.17% 87.07% 13.80% 10.84%
下载: 导出CSV
表 5 CIC-IDS2018数据集消融实验
分类算法 原始特征 嵌入表示 ACC 宏F1 加权F1 ACC 宏F1 加权F1 PCA 91.41% 71.07% 89.52% 96.82% 92.04% 96.73% HBOS 79.38% 56.28% 80.45% 97.74% 94.26% 97.66% IF 93.29% 85.08% 93.50% 96.96% 92.40% 96.88% 均值 88.03% 70.81% 87.82% 97.17% 92.90% 97.09%
下载: 导出CSV
表 6 UNSW-NB15数据集消融实验
分类算法 原始特征 嵌入表示 ACC 宏F1 加权F1 ACC 宏F1 加权F1 PCA 94.26% 68.36% 94.71% 98.14% 88.24% 98.17% HBOS 91.95% 66.87% 93.40% 98.72% 92.75% 98.81% IF 90.13% 63.36% 92.19% 98.38% 90.08% 98.43% 均值 92.11% 66.20% 93.43% 98.41% 90.36% 98.47%
下载: 导出CSV
-
[1] 胡钰林, 喻鑫岚, 高伟, 等. 低时延工业物联网中移动边缘计算的安全性与可靠性联合优化[J]. 电子与信息学报, 2025, 47(10): 3492–3504. doi: 10.11999/JEIT250262.HU Yulin, YU Xinlan, GAO Wei, et al. Security and reliability-optimal offloading for mobile edge computing in low-latency industrial IoT[J]. Journal of Electronics & Information Technology, 2025, 47(10): 3492–3504. doi: 10.11999/JEIT250262. [2] ERLACHER F and DRESSLER F. On high-speed flow-based intrusion detection using Snort-compatible signatures[J]. IEEE Transactions on Dependable and Secure Computing, 2022, 19(1): 495–506. doi: 10.1109/TDSC.2020.2973992. [3] TUDOSI A D, GRAUR A, BALAN D G, et al. Distributed firewall traffic filtering and intrusion detection using Snort on pfSense firewalls with random forest classification[C]. 2023 46th International Conference on Telecommunications and Signal Processing (TSP), Prague, Czech Republic, 2023: 101–104. doi: 10.1109/TSP59544.2023.10197784. [4] RESENDE P A A and DRUMMOND A C. A survey of random forest based methods for intrusion detection systems[J]. ACM Computing Surveys, 2019, 51(3): 48. doi: 10.1145/3178582. [5] MIRSKY Y, DOITSHMAN T, ELOVICI Y, et al. Kitsune: An ensemble of autoencoders for online network intrusion detection[C]. 25th Annual Network and Distributed System Security Symposium, San Diego, USA, 2018https://arxiv.org/abs/1802.09089, 2018. [6] FU Chuanpu, LI Qi, SHEN Meng, et al. Frequency domain feature based robust malicious traffic detection[J]. IEEE/ACM Transactions on Networking, 2023, 31(1): 452–467. doi: 10.1109/TNET.2022.3195871. [7] KHALID M, MOHSIN A R, ALI J, et al. Optimization of recurrent neural networks for high-performance intrusion detection in network traffic[J]. Cluster Computing, 2025, 28(9): 563. doi: 10.1007/s10586-025-05240-0. [8] 顾伟, 行鸿彦, 侯天浩. 基于网络流量时空特征和自适应加权系数的异常流量检测方法[J]. 电子与信息学报, 2024, 46(6): 2647–2654. doi: 10.11999/JEIT230825.GU Wei, XING Hongyan, and HOU Tianhao. Abnormal traffic detection method based on traffic spatial-temporal features and adaptive weighting coefficients[J]. Journal of Electronics & Information Technology, 2024, 46(6): 2647–2654. doi: 10.11999/JEIT230825. [9] HSIEH K, WONG M, SEGARRA S, et al. NetVigil: Robust and low-cost anomaly detection for east-west data center security[C]. 21st USENIX Symposium on Networked Systems Design and Implementation, Santa Clara, USA, 2024: 1771–1789. [10] 尹梓诺, 陈鸿昶, 马海龙, 等. 无监督自适应抽样与改进孪生网络结合的网络流量异常检测方法[J]. 电子与信息学报, 2025, 47(7): 2211–2224. doi: 10.11999/JEIT241115.YIN Zinuo, CHEN Hongchang, MA Hailong, et al. A network traffic anomaly detection method integrating unsupervised adaptive sampling with enhanced Siamese network[J]. Journal of Electronics & Information Technology, 2025, 47(7): 2211–2224. doi: 10.11999/JEIT241115. [11] HAN Hui, YAN Zheng, JING Xuyang, et al. Applications of sketches in network traffic measurement: A survey[J]. Information Fusion, 2022, 82: 58–85. doi: 10.1016/j.inffus.2021.12.007. [12] LI Yuanpeng, NIU Xian, ZHAO Yikai, et al. TitanLog: Hierarchical and elastic logging for high-speed network data stream[J]. IEEE Transactions on Networking, 2026, 34: 1988–2003. doi: 10.1109/TON.2025.3636509. [13] YUAN Ziqi, SUN Qingyun, ZHOU Haoyi, et al. A comprehensive survey on GNN-based anomaly detection: Taxonomy, methods, and the role of large language models[J]. International Journal of Machine Learning and Cybernetics, 2025, 16(7/8): 4407–4432. doi: 10.1007/s13042-024-02516-6. [14] MA Jie, SU Wei, LI Yikun, et al. Synchronizing DDoS detection and mitigation based graph learning with programmable data plane, SDN[J]. Future Generation Computer Systems, 2024, 154: 206–218. doi: 10.1016/j.future.2023.12.033. [15] LO W W, LAYEGHY S, SARHAN M, et al. E-GraphSAGE: A graph neural network based intrusion detection system[J/OL]. https://arxiv.org/abs/2103.16329v1, 2021. doi: 10.48550/arXiv.2103.16329. [16] VELIČKOVIĆ P, FEDUS W, HAMILTON W L, et al. Deep Graph Infomax[C]. 7th International Conference on Learning Representations, New Orleans, USA, 2019. [17] CAVILLE E, LO W W, LAYEGHY S, et al. Anomal-E: A self-supervised network intrusion detection system based on graph neural networks[J]. Knowledge-Based Systems, 2022, 258: 110030. doi: 10.1016/j.knosys.2022.110030. [18] LIU Jiaqian, BASAT R B, WARDT L D, et al. DISCO: A dynamically configurable sketch framework in skewed data streams[C]. 2024 IEEE 40th International Conference on Data Engineering, Utrecht, Netherlands, 2024: 4801–4814. doi: 10.1109/ICDE60146.2024.00365. [19] NGUYEN H and KASHEF R. TS-IDS: Traffic-aware self-supervised learning for IoT network intrusion detection[J]. Knowledge-Based Systems, 2023, 279: 110966. doi: 10.1016/j.knosys.2023.110966. -
下载:
图(9) / 表(6)
计量
- 文章访问数: 99
- HTML全文浏览量: 11
- PDF下载量: 3
- 被引次数: 0
