A Semantic-Enhanced Cybersecurity Named Entity Recognition Approach Oriented to Lightweight Adaptation of Large Language Models

HU Ze; XU Tongwu; YANG Hongyu

doi:10.11999/JEIT251260

Article Contents

Article Navigation > Journal of Electronics & Information Technology > 2026 >

HU Ze, XU Tongwu, YANG Hongyu. A Semantic-Enhanced Cybersecurity Named Entity Recognition Approach Oriented to Lightweight Adaptation of Large Language Models[J]. Journal of Electronics & Information Technology. doi: 10.11999/JEIT251260

Citation:

HU Ze, XU Tongwu, YANG Hongyu. A Semantic-Enhanced Cybersecurity Named Entity Recognition Approach Oriented to Lightweight Adaptation of Large Language Models[J]. Journal of Electronics & Information Technology. doi: 10.11999/JEIT251260

Citation:

PDF( 1225 KB)

A Semantic-Enhanced Cybersecurity Named Entity Recognition Approach Oriented to Lightweight Adaptation of Large Language Models

doi: 10.11999/JEIT251260 cstr: 32379.14.JEIT251260

HU Ze¹,
XU Tongwu¹,
YANG Hongyu^{1, 2
,
,}

1.
School of Safety Science and Engineering, Civil Aviation University of China, Tianjin 300300, China
2.
School of Computer Science and Artificial Intelligence, Civil Aviation University of China, Tianjin 300300, China

Funds: The National Natural Science Foundation of China (62201576, U2433205), The Supporting Fund of the National Natural Science Foundation of China (3122023PT10)

Received Date: 2025-11-26
Accepted Date: 2026-03-05
Rev Recd Date: 2026-03-05

Available Online: 2026-03-18

Abstract

Abstract

Objective Named Entity Recognition (NER) in the field of cybersecurity is a fundamental technology supporting threat intelligence analysis, vulnerability management, and security incident response. However, this field generally faces challenges such as dense technical terms, scarce labeled data, dynamic changes in entity categories, and highly complex semantic features, which make traditional deep learning models and existing Large Language Models (LLMs) significantly inadequate in terms of domain adaptability and semantic fusion capability. To address the aforementioned key issues while also considering the need for lightweight model deployment, this paper aims to construct a cybersecurity NER approach that can enhance domain semantic representation, improve the ability to identify rare entities, and apply to low-resource environments, providing a reliable technical path for intelligent threat analysis in cybersecurity scenarios. Methods To address the complex semantic features of cybersecurity texts, this paper proposes a semantically enhanced, lightweight, and LLMs-adaptable cybersecurity NER approach. The proposed approach uses LLM2Vec to achieve bidirectional semantic reconstruction of large model decoders and combines Low-Rank Adaptation (LoRA) for low-rank fine-tuning, so as to maintain deep semantic encoding capability while significantly reducing the amount of parameter updates. To address the challenges of sparse keywords and severe noise interference in cybersecurity texts, a sparse gated attention mechanism is introduced to strengthen keyword-focused feature extraction by dynamically selecting high-contribution cybersecurity terms through global gating and sparse inference. A SecRoBERTa-based semantic enhancement component is introduced, which utilizes a domain-pre-trained model to generate similar word embeddings, optimizes feature robustness in small-sample scenarios, and alleviates the challenges of identifying out-of-vocabulary words and low-frequency terms. Finally, a masked conditional random field is employed to constrain label transitions and guarantee BIO-compliant output sequences, achieving robust and consistent entity boundary prediction. Results and Discussions Extensive experiments were conducted on two public cybersecurity datasets, DNRTI and APTNER. The proposed approach achieved an F1 score of 91.91% on DNRTI, surpassing the previous state-of-the-art model by 2.14%. On APTNER, it reached an F1 score of 80.37%, outperforming the best baseline by 2.97%. Ablation studies confirmed the contribution of each key component: the Sparse Gated Attention mechanism improved F1 by 3.57% over standard Multi-Head Attention on DNRTI; the semantic enhancement module contributed a 2.32% F1 gain; and the MCRF (Masked Conditional Random Field) layer provided a 10.63% F1 improvement over traditional CRF (Conditional Random Field). The model also demonstrated efficient training and inference characteristics, aligning with its lightweight design goals. Conclusions This paper proposes a lightweight adaptation approach based on LLMs for NER in the cybersecurity domain, which effectively addresses the limitations of existing LLMs-based NER methods in domain adaptation and rare entity recognition. By integrating LLM2Vec and LoRA for lightweight fine-tuning, a sparse gated attention mechanism for domain feature fusion, and a SecRoBERTa-based semantic enhancement component for similar word precomputation, the proposed approach achieves high performance on DNRTI and APTNER datasets. The research provides an efficient technical path for NER tasks in low-resource cybersecurity scenarios and offers strong support for downstream tasks such as automated threat intelligence analysis.
- Named Entity Recognition (NER),
- Cybersecurity,
- Large Language Models (LLMs),
- Deep learning,
- Natural language processing

FullText(HTML)

References(30)

References

[1]	陈曙东, 欧阳小叶. 命名实体识别技术综述[J]. 无线电通信技术, 2020, 46(3): 251–260. doi: 10.3969/j.issn.1003-3114.2020.03.001. CHEN Shudong and OUYANG Xiaoye. Overview of named entity recognition technology[J]. Radio Communications Technology, 2020, 46(3): 251–260. doi: 10.3969/j.issn.1003-3114.2020.03.001.
[2]	SATVAT K, GJOMEMO R, and VENKATAKRISHNAN V N. Extractor: Extracting attack behavior from threat reports[C]. Proceedings of 2021 IEEE European Symposium on Security and Privacy (EuroS&P), Vienna, Austria, 2021: 598–615. doi: 10.1109/EuroSP51992.2021.00046.
[3]	GAO Chen, ZHANG Xuan, HAN Mengting, et al. A review on cyber security named entity recognition[J]. Frontiers of Information Technology & Electronic Engineering, 2021, 22(9): 1153–1168. doi: 10.1631/FITEE.2000286.
[4]	李永斌, 刘楝, 郑杰. 一种面向特定信息领域的大模型命名实体识别方法[J]. 电子与信息学报, 2026, 48(2): 662–672. doi: 10.11999/JEIT250764. LI Yongbin, LIU Lian, and ZHENG Jie. A method for named entity recognition in military intelligence domain using large language models[J]. Journal of Electronics & Information Technology, 2026, 48(2): 662–672. doi: 10.11999/JEIT250764.
[5]	HU Chenxi, WU Tao, LIU Chunsheng, et al. Joint contrastive learning and belief rule base for named entity recognition in cybersecurity[J]. Cybersecurity, 2024, 7(1): 19. doi: 10.1186/s42400-024-00206-y.
[6]	BEHNAMGHADER P, ADLAKHA V, MOSBACH M, et al. LLM2Vec: Large language models are secretly powerful text encoders[J]. arXiv preprint arXiv: 2404.05961, 2024. doi: 10.48550/arXiv.2404.05961. (查阅网上资料,不确定文献类型及格式是否正确,请确认).
[7]	HU E J, SHEN Yelong, WALLIS P, et al. LoRA: Low-rank adaptation of large language models[C]. Proceedings of the 10th International Conference on Learning Representations, 2022. (查阅网上资料, 未找到本条文献出版地信息, 请确认).
[8]	YI Feng, JIANG Bo, WANG Lu, et al. Cybersecurity named entity recognition using multi-modal ensemble learning[J]. IEEE Access, 2020, 8: 63214–63224. doi: 10.1109/ACCESS.2020.2984582.
[9]	MA Pingchuan, JIANG Bo, LU Zhigang, et al. Cybersecurity named entity recognition using bidirectional long short-term memory with conditional random fields[J]. Tsinghua Science and Technology, 2021, 26(3): 259–265. doi: 10.26599/TST.2019.9010033.
[10]	YI Junkai, LIU Yuan, JIANG Zhongbai, et al. Text command intelligent understanding for cybersecurity testing[J]. Electronics, 2024, 13(21): 4330. doi: 10.3390/electronics13214330.
[11]	胡泽, 李文君, 杨宏宇. 基于字符表示学习与时序边界扩散的网络安全实体识别方法[J]. 电子与信息学报, 2025, 47(5): 1554–1568. doi: 10.11999/JEIT240953. HU Ze, LI Wenjun, and YANG Hongyu. A cybersecurity entity recognition approach based on character representation learning and temporal boundary diffusion[J]. Journal of Electronics & Information Technology, 2025, 47(5): 1554–1568. doi: 10.11999/JEIT240953.
[12]	ZHANG Yunlong, LIU Jingju, ZHONG Xiaofeng, et al. SecLMNER: A framework for enhanced named entity recognition in multi-source cybersecurity data using large language models[J]. Expert Systems with Applications, 2025, 271: 126651. doi: 10.1016/j.eswa.2025.126651.
[13]	ZHANG Hao, WU Tingmin, ZHU Tianqing, et al. CyberLLaMA: A fine-tuned large language model for cybersecurity named entity recognition[J]. Knowledge-Based Systems, 2025, 328: 114183. doi: 10.1016/j.knosys.2025.114183.
[14]	VASWANI A, SHAZEER N, PARMAR N, et al. Attention is all you need[C]. Proceedings of the 31st International Conference on Neural Information Processing Systems, Long Beach, USA, 2017: 6000–6010.
[15]	Hugging Face. jackaduma/SecRoBERTa[EB/OL]. https://huggingface.co/jackaduma/SecRoBERTa, 2024.
[16]	才华, 冉越, 付强, 等. 多粒度文本感知分层特征交互的视觉定位方法[J]. 电子与信息学报, 2025, 47(11): 4594–4605. doi: 10.11999/JEIT250387. CAI Hua, RAN Yue, FU Qiang, et al. Multi-granularity text perception and hierarchical feature interaction method for visual grounding[J]. Journal of Electronics & Information Technology, 2025, 47(11): 4594–4605. doi: 10.11999/JEIT250387.
[17]	姜小波, 邓晗珂, 莫志杰, 等. 规则压缩模型和灵活架构的Transformer加速器设计[J]. 电子与信息学报, 2024, 46(3): 1079–1088. doi: 10.11999/JEIT230188. JIANG Xiaobo, DENG Hanke, MO Zhijie, et al. Design of transformer accelerator with regular compression model and flexible architecture[J]. Journal of Electronics & Information Technology, 2024, 46(3): 1079–1088. doi: 10.11999/JEIT230188.
[18]	YE Tianzhu, DONG Li, XIA Yuqing, et al. Differential transformer[J]. arXiv preprint arXiv: 2410.05258, 2024. doi: 10.48550/arXiv.2410.05258. (查阅网上资料,不确定文献类型及格式是否正确,请确认).
[19]	SU Jianlin, AHMED M, LU Yu, et al. RoFormer: Enhanced transformer with rotary position embedding[J]. Neurocomputing, 2024, 568: 127063. doi: 10.1016/j.neucom.2023.127063.
[20]	LIU Peipei, LI Hong, WANG Zuoguang, et al. Multi-features based semantic augmentation networks for named entity recognition in threat intelligence[C]. Proceedings of the 26th International Conference on Pattern Recognition (ICPR), Montreal, Canada, 2022: 1557–1563. doi: 10.1109/ICPR56361.2022.9956373.
[21]	WEI Tianwen, QI Jianwei, HE Shenghuan, et al. Masked conditional random fields for sequence labeling[C]. Proceedings of 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, 2021: 2024–2035. doi: 10.18653/v1/2021.naacl-main.163.(查阅网上资料,未找到本条文献出版地信息,请确认).
[22]	WANG Xuren, LIU Xinpei, AO Shengqin, et al. DNRTI: A large-scale dataset for named entity recognition in threat intelligence[C]. Proceedings of 2020 IEEE 19th International Conference on Trust, Security and Privacy in Computing and Communications (TrustCom), Guangzhou, China, 2020: 1842–1848. doi: 10.1109/TrustCom50675.2020.00252.
[23]	WANG Xuren, HE Songheng, XIONG Zihan, et al. APTNER: A specific dataset for NER missions in cyber threat intelligence field[C]. Proceedings of 2022 IEEE 25th International Conference on Computer Supported Cooperative Work in Design (CSCWD), Hangzhou, China, 2022: 1233–1238. doi: 10.1109/CSCWD54268.2022.9776031.
[24]	CAI Yongxin, KANG Lei, LENG Tao, et al. BNER: A broad learning system-based named entity recognition method for cyber threat intelligence[C]. Proceedings of 2025 11th IEEE International Conference on Privacy Computing and Data Security (PCDS), Hakodate, Japan, 2025: 397–404. doi: 10.1109/PCDS65695.2025.00060.
[25]	DU Chao, LIU Xuhong, MIAO Lin, et al. Threat intelligence named entity recognition based on global gated feature fusion[C]. Proceedings of 2024 6th International Conference on Internet of Things, Automation and Artificial Intelligence (IoTAAI), Guangzhou, China, 2024: 618–622. doi: 10.1109/IoTAAI62601.2024.10692655.
[26]	WANG Peng and LIU Jingju. A cyber threat entity recognition method based on robust feature representation and adversarial training[C]. Proceedings of 2023 12th International Conference on Computing and Pattern Recognition, Qingdao, China, 2024: 255–259. doi: 10.1145/3633637.3633677.
[27]	CHANG Yu, WANG Gang, ZHU Peng, et al. Research on unified cyber threat intelligence entity recognition method based on multiple features[C]. Proceedings of 2023 4th International Conference on Computers and Artificial Intelligence Technology (CAIT), Macau, China, 2023: 233–240. doi: 10.1109/CAIT59945.2023.10469250.
[28]	ZHANG Yunlong, LIU Jingju, ZHONG Xiaofeng, et al. SecLMNER: A framework for enhanced named entity recognition in multi-source cybersecurity data using large language models[J]. Expert Systems with Applications, 2025, 271: 126651. doi: 10.1016/j.eswa.2025.126651. (查阅网上资料,本条文献与第12条文献重复,请确认).
[29]	孙语晨. 基于大语言模型的威胁情报信息抽取研究与实现[D]. [硕士论文], 北京邮电大学, 2025. doi: 10.26969/d.cnki.gbydu.2025.002794. SUN Yuchen. Research and implemention of threat intelligence information extraction based on large language model[D]. [Master dissertation], Beijing University of Posts and Telecommunications, 2025. doi: 10.26969/d.cnki.gbydu.2025.002794.
[30]	汪溢镭, 孙歆, 韩嘉佳, 等. 暗网高质量威胁情报获取技术与实现[J/OL]. https://doi.org/10.19678/j.issn.1000-3428.0068805, 2024. WANG Yilei, SUN Xin, HAN Jiajia, et al. Techniques and implementation of high-quality threat intelligence acquisition from the dark web[J/OL]. https://doi.org/10.19678/j.issn.1000-3428.0068805, 2024.