基于头皮脑电的游戏型脑机接口应用研究综述

何峰; 董博文; 韩锦; 李雨龙; 许敏鹏; 明东

doi:10.11999/JEIT211337

基于头皮脑电的游戏型脑机接口应用研究综述

doi: 10.11999/JEIT211337 cstr: 32379.14.JEIT211337

何峰^{1, 2, ,},
董博文¹,
韩锦²,
李雨龙¹,
许敏鹏^{1, 2},
明东^{1, 2}

1.
天津大学医学工程与转化医学研究院天津 300072
2.
天津大学精密仪器与光电子工程学院天津 300072

基金项目: 国家杰出青年科学基金(81925020)，国家优秀青年科学基金(62122059)，国家自然科学基金(81630051, 61976152)，中国科协青年人才托举工程(2018QNRC001)

详细信息

作者简介:
何峰：男，1971年生，副教授，研究方向为神经工程、生物医学信号检测与处理、嵌入式医学仪器开发

董博文：男，1998年生，硕士生，研究方向为脑-机接口

韩锦：男，1994年生，博士生，研究方向为脑-机接口

李雨龙：男，1998年生，硕士生，研究方向为脑-机接口

许敏鹏：男，1988年生，副教授，研究方向为脑-机接口、神经信号处理和神经调控

明东：男，1976年生，教授，研究方向为脑-机接口、神经再生与修复、神经仿生与智能、神经刺激与调节、神经传感与成像

通讯作者:
何峰　heaven@tju.edu.cn

中图分类号: TN911.7; TP391
计量
- 文章访问数: 2083
- HTML全文浏览量: 1667
- PDF下载量: 416
- 被引次数: 0
出版历程
- 收稿日期: 2021-11-25
- 修回日期: 2022-01-16
- 录用日期: 2022-01-24
- 网络出版日期: 2022-01-24
- 刊出日期: 2022-02-25

Advances in Application of Game Brain-Computer Interface Based on ElectroEncephaloGram

HE Feng^{1, 2
, ,},
DONG Bowen¹,
HAN Jin²,
LI Yulong¹,
XU Minpeng^{1, 2},
MING Dong^{1, 2}

1.
Academy of Medical Engineering and Translational Medicine, Tianjin University, Tianjin 300072, China
2.
School of Precision Instrument and Opto-electronics Engineering, Tianjin University, Tianjin 300072, China

Funds: The National Science Foundation for Distinguished Young Scholars (81925020), The National Science Foundation for Excellent Young Scientists (62122059), The National Natural Science Foundation of China (81630051, 61976152), Young Elite Scientist Sponsorship Program by CAST (2018QNRC001)

摘要

摘要: 游戏型脑机接口(game-BCI)是指将脑机接口与游戏结合，通过对脑电信号的识别，使用户完成对游戏的直接控制，不仅为健康人提供了新型的游戏交互方式，也为残障人士提供了新的康复治疗途径。相比于其它侵入性BCI，基于头皮脑电的BCI有无创、时间分辨率高、成本低、便携性好等优势，具有更加广泛的应用前景。该文对基于头皮脑电的游戏型脑-机接口技术进行了总结，将其按控制信号类型分为主动式、反应式、被动式以及混合范式，介绍不同类型游戏BCI的控制策略和应用场景，并对比分析了游戏BCI中不同类型脑电信号常采用的分类算法，最后讨论了该领域目前存在的问题和未来的发展方向。
- 脑机接口 /
- 头皮脑电 /
- 主动式脑机接口 /
- 反应式脑机接口 /
- 被动式脑机接口
Abstract: The game Brain-Computer Interface (game-BCI) is a novel interactive mode that the user can control and interact with the game directly by recognizing brain signals. It not only provides a new way of game interaction for healthy people, but also provides a new way of rehabilitation for the disabled. Compared with the other invasive BCIs, BCI based on scalp ElectroEncephaloGram (EEG) has a wider application prospects because there are the advantages of non-invasiveness, high time resolution, low cost, and good portability. The game-BCI technology based on scalp EEG is summarized and is divided into active, reactive, passive, and hybrid paradigms according to control signal types. Then the control strategies and application scenarios of different types of game BCI are introduced. The classification algorithms of EEG signals commonly used in game BCI are compared and analyzed. Finally, the current problems and future development directions in this filed are discussed.
- Brain-Computer Interface (BCI) /
- ElectroEncephaloGram (EEG) /
- Activate BCI /
- Reactive BCI /
- Passive BCI

HTML全文

图 1 2006-2020年游戏BCI论文发表和引用数量

下载: 全尺寸图片幻灯片

图 2 MI-BCI游戏

下载: 全尺寸图片幻灯片

图 3 SSVEP-BCI游戏

下载: 全尺寸图片幻灯片

图 4 MindGomoku游戏

下载: 全尺寸图片幻灯片

图 5 混合BCI游戏俄罗斯方块

下载: 全尺寸图片幻灯片

参考文献(54)

[1]	WOLPAW J R, DEL R, MILLÁN J, and RAMSEY N F. Brain-computer interfaces: Definitions and principles[J]. Handbook of Clinical Neurology, 2020, 168: 15–23. doi: 10.1016/B978-0-444-63934-9.00002-0
[2]	WOLPAW J R, BIRBAUMER N, HEETDERKS W J, et al. Brain-computer interface technology: A review of the first international meeting[J]. IEEE Transactions on Rehabilitation Engineering, 2000, 8(2): 164–173. doi: 10.1109/tre.2000.847807
[3]	CATTAN G. The use of brain-computer interfaces in games is not ready for the general public[J]. Frontiers in Computer Science, 2021, 3: 628773. doi: 10.3389/fcomp.2021.628773
[4]	CATTAN G, ANDREEV A, and VISINONI E. Recommendations for integrating a P300-based brain–computer interface in virtual reality environments for gaming: An update[J]. Computers, 2020, 9(4): 92. doi: 10.3390/computers9040092
[5]	MERCADO J, ESCOBEDO L, and TENTORI M. A BCI video game using neurofeedback improves the attention of children with autism[J]. Journal on Multimodal User Interfaces, 2021, 15(3): 273–281. doi: 10.1007/s12193-020-00339-7
[6]	PARK K, KIHL T, PARK S, et al. Fairy tale directed game-based training system for children with ADHD using BCI and motion sensing technologies[J]. Behaviour & Information Technology, 2019, 38(6): 564–577. doi: 10.1080/0144929X.2018.1544276
[7]	SEKHAVAT Y A. Battle of minds: A new interaction approach in BCI games through competitive reinforcement[J]. Multimedia Tools and Applications, 2020, 79(5): 3449–3464. doi: 10.1007/s11042-019-07963-w
[8]	CHEN Yuliang. Design of a game control system based on Brain-computer interface: Link to a game[C]. Proceedings of the 2021 2nd International Conference on Computing and Data Science, Stanford, USA, 2021: 255–258.
[9]	TORRES E P, TORRES E A, HERNÁNDEZ-ÁLVAREZ M, et al. EEG-based BCI emotion recognition: A survey[J]. Sensors, 2020, 20(18): 5083. doi: 10.3390/s20185083
[10]	DE CASTRO-CROS M, SEBASTIAN-ROMAGOSA M, RODRÍGUEZ-SERRANO J, et al. Effects of gamification in BCI functional rehabilitation[J]. Frontiers in Neuroscience, 2020, 14: 882. doi: 10.3389/fnins.2020.00882
[11]	SINGH A K, WANG Yukai, KING J T, et al. Extended interaction with a BCI video game changes resting-state brain activity[J]. IEEE Transactions on Cognitive and Developmental Systems, 2020, 12(4): 809–823. doi: 10.1109/TCDS.2020.2985102
[12]	ARPAIA P, ESPOSITO A, MANCINO F, et al. Active and passive brain-computer interfaces integrated with extended reality for applications in health 4.0[C]. Proceedings of the 8th International Conference on Augmented Reality, Virtual Reality and Computer Graphics, Virtual Event, 2021: 392–405.
[13]	CHMURA J, ROSING J, COLLAZOS S, et al. Classification of movement and inhibition using a hybrid BCI[J]. Frontiers in Neurorobotics, 2017, 11: 38. doi: 10.3389/fnbot.2017.00038
[14]	WEN Dong, LIANG Bingbing, ZHOU Yanhong, et al. The current research of combining multi-modal brain-computer interfaces with virtual reality[J]. IEEE Journal of Biomedical and Health Informatics, 2020, 25(9): 3278–3287. doi: 10.1109/JBHI.2020.3047836
[15]	NAYAK T, KO L W, JUNG T P, et al. Target classification in a novel SSVEP-RSVP based BCI gaming system[C]. Proceedings of 2019 IEEE International Conference on Systems, Man and Cybernetics, Bari, Italy, 2019: 4194–4198.
[16]	MENG Jiayuan, XU Minpeng, WANG Kun, et al. Separable EEG features induced by timing prediction for active brain-computer interfaces[J]. Sensors, 2020, 20(12): 3588. doi: 10.3390/s20123588
[17]	TEZZA D, CAPRIO D, GARCIA S, et al. Brain-controlled drone racing game: A qualitative analysis[C]. Proceedings of the 2nd International Conference on Human-Computer Interaction, Copenhagen, Denmark, 2020: 350–360.
[18]	COYLE D, GARCIA J, SATTI A R, et al. EEG-based continuous control of a game using a 3 channel motor imagery BCI: BCI game[C]. Proceedings of 2011 IEEE Symposium on Computational Intelligence, Cognitive Algorithms, Mind, and Brain, Paris, France, 2011: 1–7.
[19]	KARÁCSONY T, HANSEN J P, IVERSEN H K, et al. Brain computer interface for neuro-rehabilitation with deep learning classification and virtual reality feedback[C]. Proceedings of the 10th Augmented Human International Conference 2019, Reims, France, 2019: 22.
[20]	VOURVOPOULOS A, FERREIRA A, and BADIA S B I. NeuRow: An immersive VR environment for motor-imagery training with the use of brain-computer interfaces and vibrotactile feedback[C]. Proceedings of the 3rd International Conference on Physiological Computing Systems, Lisbon, Portugal, 2016: 43–53.
[21]	PUTRI F, DING Hao, GARCIA A, et al. Towards successful multi-user Brain-Computer Interface (BCI) gaming: Analysis of the EEG signatures and connectivity[C]. Proceedings of the 3rd International Conference on Computer-Human Interaction Research and Applications, Vienna, Austria, 2019: 59–65.
[22]	LI Junhua, LIU Ye, LU Zhen, et al. A competitive brain computer interface: Multi-person car racing system[C]. Proceedings of the 2013 35th Annual International Conference of the IEEE Engineering in Medicine and Biology Society, Osaka, Japan, 2013: 2200–2203.
[23]	KUMAR M K, PARAMESHACHARI B D, PRABU S, et al. Comparative analysis to identify efficient technique for interfacing BCI system[J]. IOP Conference Series:Materials Science and Engineering, 2020, 925: 012062. doi: 10.1088/1757-899X/925/1/012062
[24]	FILIZ E and ARSLAN R B. Design and implementation of steady state visual evoked potential based brain computer interface video game[C]. Proceedings of the 2020 IEEE 20th Mediterranean Electrotechnical Conference, Palermo, Italy, 2020: 335–338.
[25]	VIDAL J J. Real-time detection of brain events in EEG[J]. Proceedings of the IEEE, 1977, 65(5): 633–641. doi: 10.1109/PROC.1977.10542
[26]	PARAFITA R, PIRES G, NUNES U, et al. A spacecraft game controlled with a brain-computer interface using SSVEP with phase tagging[C]. Proceedings of the 2013 IEEE 2nd International Conference on Serious Games and Applications for Health, Vilamoura, Portugal, 2013: 1–6.
[27]	PEREZ-VALERO E, LOPEZ-GORDO M A, and VAQUERO-BLASCO M A. An attention-driven videogame based on steady-state motion visual evoked potentials[J]. Expert Systems, 2021, 38(4): e12682. doi: 10.1111/exsy.12682
[28]	CRUZ I, MOREIRA C, POEL M, et al. Kessel run-a cooperative multiplayer SSVEP BCI game[C]. Proceedings of the 9th International Conference on Intelligent Technologies for Interactive Entertainment, Funchal, Portugal, 2017: 77–95.
[29]	FARWELL L A and DONCHIN E. Talking off the top of your head: Toward a mental prosthesis utilizing event-related brain potentials[J]. Electroencephalography and Clinical Neurophysiology, 1988, 70(6): 510–523. doi: 10.1016/0013-4694(88)90149-6
[30]	FINKE A, LENHARDT A, and RITTER H. The MindGame: A P300-based brain–computer interface game[J]. Neural Networks, 2009, 22(9): 1329–1333. doi: 10.1016/j.neunet.2009.07.003
[31]	LI Man, LI Feng, PAN Jiahui, et al. The MindGomoku: An online P300 BCI game based on bayesian deep learning[J]. Sensors, 2021, 21(5): 1613. doi: 10.3390/s21051613
[32]	BAYLISS J D and BALLARD D H. Single trial P3 epoch recognition in a virtual environment[J]. Neurocomputing, 2000, 32/33: 637–642. doi: 10.1016/S0925-2312(00)00226-5
[33]	HUANG Zhihua, ZHENG Wenming, WU Yingjie, et al. Ensemble or pool: A comprehensive study on transfer learning for c-VEP BCI during interpersonal interaction[J]. Journal of Neuroscience Methods, 2020, 343: 108855. doi: 10.1016/j.jneumeth.2020.108855
[34]	PENG Faqiang and HUANG Zhihua. A c-VEP BCI system for psychological experiments[C]. Proceedings of the 2019 12th International Congress on Image and Signal Processing, BioMedical Engineering and Informatics, Suzhou, China, 2019: 1–5.
[35]	BEVERIDGE R, WILSON S, CALLAGHAN M, et al. Neurogaming with motion-onset visual evoked potentials (mVEPs): Adults versus teenagers[J]. IEEE Transactions on Neural Systems and Rehabilitation Engineering, 2019, 27(4): 572–581. doi: 10.1109/TNSRE.2019.2904260
[36]	RIECHMANN H, FINKE A, and RITTER H. Using a cVEP-based brain-computer interface to control a virtual agent[J]. IEEE Transactions on Neural Systems and Rehabilitation Engineering, 2016, 24(6): 692–699. doi: 10.1109/TNSRE.2015.2490621
[37]	FRIEDRICH E V C, SUTTIE N, SIVANATHAN A, et al. Brain–computer interface game applications for combined neurofeedback and biofeedback treatment for children on the autism spectrum[J]. Frontiers in Neuroengineering, 2014, 7: 21. doi: 10.3389/fneng.2014.00021
[38]	MUÑOZ J E, LOPEZ D S, LOPEZ J F, et al. Design and creation of a BCI videogame to train sustained attention in children with ADHD[C]. Proceedings of the 2015 10th Computing Colombian Conference, Bogota, Colombia, 2015: 194–199.
[39]	HRAMOV A E, MAKSIMENKO V A, and PISARCHIK A N. Physical principles of brain-computer interfaces and their applications for rehabilitation, robotics and control of human brain states[J]. Physics Reports, 2021, 918: 1–133. doi: 10.1016/j.physrep.2021.03.002
[40]	LI Zina, ZHANG Shuqing, and PAN Jiahui. Advances in hybrid brain-computer interfaces: Principles, design, and applications[J]. Computational Intelligence and Neuroscience, 2019, 2019: 3807670. doi: 10.1155/2019/3807670
[41]	KONEČNÝ R and LIAROKAPIS F. Foresthlon: Investigating gender experience through a hybrid BCI game[C]. Proceedings of the 3rd International Conference on Human-Computer Interaction, Virtual Event, 2021: 57–74.
[42]	XU Minpeng, HAN Jin, WANG Yijun, et al. Implementing over 100 command codes for a high-speed hybrid brain-computer interface using concurrent P300 and SSVEP features[J]. IEEE Transactions on Biomedical Engineering, 2020, 67(11): 3073–3082. doi: 10.1109/TBME.2020.2975614
[43]	WANG Zhihua, YU Yang, XU Ming, et al. Towards a hybrid BCI gaming paradigm based on motor imagery and SSVEP[J]. International Journal of Human–Computer Interaction, 2019, 35(3): 197–205. doi: 10.1080/10447318.2018.1445068
[44]	MÜHL C, GÜRKÖK H, BOS D P O, et al. Bacteria hunt: Evaluating multi-paradigm BCI interaction[J]. Journal on Multimodal User Interfaces, 2010, 4(1): 11–25. doi: 10.1007/s12193-010-0046-0
[45]	KHONG A, LIN Jiangnan, THOMAS K P, et al. BCI based multi-player 3-D game control using EEG for enhancing attention and memory[C]. Proceedings of 2014 IEEE International Conference on Systems, Man, and Cybernetics, San Diego, USA, 2014: 1847–1852.
[46]	KINNEY-LANG E, MURJI S, KELLY D, et al. Designing a flexible tool for rapid implementation of brain-computer interfaces (BCI) in game development[C]. Proceedings of the 2020 42nd Annual International Conference of the IEEE Engineering in Medicine & Biology Society, Montreal, Canada, 2020: 6078–6081.
[47]	SERRANO-BARROSO A, SIUGZDAITE R, GUERRERO-CUBERO J, et al. Detecting attention levels in ADHD children with a video game and the measurement of brain activity with a single-channel BCI headset[J]. Sensors, 2021, 21(9): 3221. doi: 10.3390/s21093221
[48]	ARRAMBIDE K, FREIMAN CORMIER L, WEHBE R R, et al. The development of "orbit": The collaborative BCI game for children with AD(H)D[C]. Proceedings of Extended Abstracts of the Annual Symposium on Computer-Human Interaction in Play Companion Extended Abstracts, Barcelona, Spain, 2019: 341–348.
[49]	MRIDHA M F, DAS S C, KABIR M M, et al. Brain-computer interface: Advancement and challenges[J]. Sensors, 2021, 21(17): 5746. doi: 10.3390/s21175746
[50]	SUBRAMANIAN R R, VARMA K Y, BALAJI K, et al. Multiplayer online car racing with BCI In VR[C]. Proceedings of the 2021 5th International Conference on Intelligent Computing and Control Systems, Madurai, India, 2021: 1835–1839.
[51]	SCOTT J A and SIMS M. Acceleration of therapeutic use of brain computer interfaces by development for gaming[C]. Proceedings of the 12th International Conference on Intelligent Technologies for Interactive Entertainment. Virtual Event, 2020: 267–281.
[52]	ZHANG Rongxiang. Virtual reality games based on brain computer interface[C]. Proceedings of 2020 International Conference on Intelligent Computing and Human-Computer Interaction, Sanya, China, 2020: 227–230.
[53]	XU Lichao, XU Minpeng, KE Yufeng, et al. Cross-dataset variability problem in EEG decoding with deep learning[J]. Frontiers in Human Neuroscience, 2020, 14: 103. doi: 10.3389/fnhum.2020.00103
[54]	XU Minpeng, XIAO Xiaolin, WANG Yijun, et al. A brain–computer interface based on miniature-event-related potentials induced by very small lateral visual stimuli[J]. IEEE Transactions on Biomedical Engineering, 2018, 65(5): 1166–1175. doi: 10.1109/TBME.2018.2799661