技术领域technical field
本发明涉及脑机接口的技术领域，具体涉及一种基于AR的SSVEP-MI控制及康复训练轮椅。The invention relates to the technical field of brain-computer interface, in particular to an AR-based SSVEP-MI control and rehabilitation training wheelchair.
背景技术Background technique
对于行动不便的人员，轮椅是一种便捷的辅助移动设备，可以作为代步工具，扩大活动区域，提高生活质量。目前，市面上常见的轮椅种类有由使用者手动推动或他人推行的手动轮椅、通过操纵杆或按钮控制的电动轮椅。然而仍有一些特殊人群无法自主使用这两种常见的轮椅。对于少部分丧失沟通能力，四肢活动能力的瘫痪者，他们无法通过上肢控制轮椅滑行，也无法通过操纵杆控制电动轮椅移动，瘫痪者只能依赖看护人员，才能进行活动，导致瘫痪者的生活区域减小，生活品质下降，失去康复热情。For people with limited mobility, wheelchairs are a convenient auxiliary mobility device, which can be used as a means of transportation to expand the activity area and improve the quality of life. At present, the common types of wheelchairs on the market include manual wheelchairs that are manually pushed by the user or pushed by others, and electric wheelchairs that are controlled by joysticks or buttons. However, there are still some special groups of people who cannot use these two common wheelchairs autonomously. For a small number of paralyzed people who have lost the ability to communicate and move their limbs, they cannot control the sliding of the wheelchair with their upper limbs, nor can they control the movement of the electric wheelchair through the joystick. Decreased, decreased quality of life, loss of enthusiasm for recovery.
以脑卒中疾病为例，卒中是严重危害健康的重大慢性非传染性疾病，具有高发病率、高致残率、高死亡率、高复发率、高经济负担5大特点。病发后存活者，神经仍会有损伤，影响人体的运动功能，严重者会完全丧失对肢体的控制能力。而大部分患者仍然有着清晰的思维，只是无法通过自身的神经回路控制肢体运动。因此，脑卒中患者往往行动不便，术后的日常生活和康复过程需要看护人员的照看，往往会对家庭产生负担，引发一些社会问题。Taking stroke as an example, stroke is a major chronic non-communicable disease that seriously endangers health. It has five characteristics: high morbidity, high disability, high mortality, high recurrence rate, and high economic burden. After the onset of the disease, the survivors will still have nerve damage, which will affect the motor function of the human body. In severe cases, they will completely lose the ability to control the limbs. Most patients still have clear thinking, but they cannot control limb movements through their own neural circuits. Therefore, stroke patients often have limited mobility, and need the care of nursing staff in their postoperative daily life and rehabilitation process, which often imposes a burden on the family and causes some social problems.
目前，脑机接口(Brain-Computer Interaction，BCI)技术不断发展，将BCI技术应用于智能医疗领域已成为一种趋势。因此，脑控轮椅应运而生，它将BCI技术应用于传统轮椅，辅助患者自主控制轮椅，提高生活质量。目前常见的脑控轮椅主要是以稳态视觉诱发电位(Steady-State Visual Evoked Potentials，SSVEP)脑机接口作为控制方法进行轮椅的控制，其准确率普遍较高，但这种方法形式单一，并且对于患者来说并没有康复运动神经的效果。运动想象(Motor Imagery，MI)脑机接口对于受损的神经有康复的作用，但其准确率并不理想，并不能作为控制信号来进行轮椅控制。At present, brain-computer interface (Brain-Computer Interaction, BCI) technology continues to develop, and it has become a trend to apply BCI technology to the field of intelligent medical care. Therefore, brain-controlled wheelchairs emerged as the times require. It applies BCI technology to traditional wheelchairs to assist patients in autonomously controlling wheelchairs and improving their quality of life. At present, the common brain-controlled wheelchairs mainly use Steady-State Visual Evoked Potentials (SSVEP) brain-computer interface as the control method to control the wheelchair. The accuracy rate is generally high, but this method has a single form and There is no effect on the rehabilitation of motor nerves for patients. Motor Imagery (MI) brain-computer interface has a rehabilitation effect on damaged nerves, but its accuracy is not ideal, and it cannot be used as a control signal for wheelchair control.
因此，需要设计新的脑控轮椅技术。Therefore, new brain-controlled wheelchair technology needs to be designed.
发明内容Contents of the invention
本发明的目的就是为了克服上述现有技术存在的缺陷而提供一种兼顾了康复训练和运动控制的需要、可靠性高的基于AR的SSVEP-MI控制及康复训练轮椅。The purpose of the present invention is to overcome the defects in the above-mentioned prior art and provide an AR-based SSVEP-MI control and rehabilitation training wheelchair that meets the needs of rehabilitation training and motion control and has high reliability.
本发明的目的可以通过以下技术方案来实现：The purpose of the present invention can be achieved through the following technical solutions:
一种基于AR的SSVEP-MI控制及康复训练轮椅，包括电动轮椅，还包括：An AR-based SSVEP-MI control and rehabilitation training wheelchair, including an electric wheelchair, also includes:
脑电采集模块，用于实时采集SSVEP脑电信号和/或MI脑电信号；EEG acquisition module, used for real-time acquisition of SSVEP EEG signals and/or MI EEG signals;
AR引导模块，用于显示以诱发SSVEP脑电信号和引导MI脑电信号的刺激引导界面；The AR guidance module is used to display the stimulation guidance interface to induce SSVEP EEG signals and guide MI EEG signals;
算法模块，用于获取所述SSVEP脑电信号和MI脑电信号，分别对所述SSVEP脑电信号和MI脑电信号进行识别，基于识别结果产生显示信息，并产生针对所述刺激引导界面的显示控制指令和/或针对所述电动轮椅的行动控制指令。The algorithm module is used to obtain the SSVEP EEG signal and the MI EEG signal, respectively identify the SSVEP EEG signal and the MI EEG signal, generate display information based on the recognition result, and generate the stimulus guidance interface Displaying control instructions and/or motion control instructions for the electric wheelchair.
作为本发明优选的技术方案，所述脑电采集模块包括64导湿电极脑电帽。As a preferred technical solution of the present invention, the EEG acquisition module includes 64 moisture-conducting electrode EEG caps.
作为本发明优选的技术方案，所述脑电采集模块通过TCP/IP通信与算法模块连接。As a preferred technical solution of the present invention, the EEG acquisition module is connected to the algorithm module through TCP/IP communication.
作为本发明优选的技术方案，所述AR引导模块通过UDP通信与算法模块连接。As a preferred technical solution of the present invention, the AR guidance module is connected with the algorithm module through UDP communication.
作为本发明优选的技术方案，所述刺激引导界面包括主界面和二级界面，其中，主界面为SSVEP刺激界面，包括五个分别代表前进、后退、左转、右转和停止的闪烁刺激块，二级界面为MI引导界面。As a preferred technical solution of the present invention, the stimulation guidance interface includes a main interface and a secondary interface, wherein the main interface is the SSVEP stimulation interface, including five flickering stimulation blocks representing forward, backward, left turn, right turn and stop , the secondary interface is the MI boot interface.
作为本发明优选的技术方案，各所述闪烁刺激块按不同频率闪烁。As a preferred technical solution of the present invention, each of the flickering stimulus blocks flickers at different frequencies.
作为本发明优选的技术方案，所述算法模块包括：As a preferred technical solution of the present invention, the algorithm module includes:
SSVEP解码单元，用于在接收到SSVEP脑电信号时，采用FBCCA算法对所述SSVEP脑电信号进行解码，获得SSVEP分类结果；The SSVEP decoding unit is used to decode the SSVEP EEG signal by using the FBCCA algorithm to obtain the SSVEP classification result when receiving the SSVEP EEG signal;
判断单元，用于对SSVEP分类结果进行判断，并基于判断结果执行相应动作，具体地：当SSVEP分类结果为前进、后退或停止时，同时产生对应的显示信息和针对所述电动轮椅的对应的行动控制指令，当SSVEP分类结果为左转或右转时，产生针对所述刺激引导界面的显示控制指令，使刺激引导界面显示为二级界面；The judging unit is used to judge the SSVEP classification result, and perform corresponding actions based on the judgment result, specifically: when the SSVEP classification result is forward, backward or stop, simultaneously generate corresponding display information and corresponding information for the electric wheelchair An action control instruction, when the SSVEP classification result is turning left or right, generating a display control instruction for the stimulation guidance interface, so that the stimulation guidance interface is displayed as a secondary interface;
MI解码单元，用于在接收到MI脑电信号时，采用CSP-LDA算法对所述MI脑电信号进行解码，获得MI分类结果，将该MI分类结果与SSVEP分类结果进行比对，若一致，则产生对应的显示信息及对应的行动控制指令，否则重复采集MI脑电信号；The MI decoding unit is used to decode the MI EEG signal using the CSP-LDA algorithm when receiving the MI EEG signal to obtain the MI classification result, and compare the MI classification result with the SSVEP classification result, if consistent , then generate corresponding display information and corresponding action control instructions, otherwise repeatedly collect MI EEG signals;
返回单元，用于产生针对所述刺激引导界面的显示控制指令，使刺激引导界面显示主界面。The return unit is configured to generate a display control instruction for the stimulation guidance interface, so that the stimulation guidance interface displays the main interface.
作为本发明优选的技术方案，所述SSVEP解码单元和MI解码单元中均包括进行信号预处理的预处理子单元，所述信号预处理包括滤波、降采样、去除眼电伪迹中的多种。As a preferred technical solution of the present invention, both the SSVEP decoding unit and the MI decoding unit include a preprocessing subunit for signal preprocessing, and the signal preprocessing includes filtering, downsampling, and removal of electrooculogram artifacts. .
作为本发明优选的技术方案，所述AR引导模块包括AR眼镜。As a preferred technical solution of the present invention, the AR guidance module includes AR glasses.
作为本发明优选的技术方案，所述电动轮椅包括通过串口方式与算法模块连接的轮椅控制器。As a preferred technical solution of the present invention, the electric wheelchair includes a wheelchair controller connected to the algorithm module through a serial port.
脑控轮椅的研究处于发展期，没有固定的模式，具有较大的潜在研究价值。与现有技术相比，本发明具有以下有益效果：The research on brain-controlled wheelchairs is in the development stage, there is no fixed model, and it has great potential research value. Compared with the prior art, the present invention has the following beneficial effects:
(1)本发明将MI和SSVEP相结合，集合两种脑机接口的优点，削弱两者的缺点，搭建了两种范式并行的多模态BCI系统，兼顾了患者康复训练和运动控制的需要，可靠性高，可扩展性强，提升了脑控轮椅的应用范围。(1) The present invention combines MI and SSVEP, integrates the advantages of the two brain-computer interfaces, weakens the shortcomings of the two, and builds a multi-modal BCI system with two parallel paradigms, taking into account the needs of patient rehabilitation training and motion control , high reliability, and strong scalability, which improves the application range of brain-controlled wheelchairs.
(2)本发明基于AR技术，将两类脑机接口范式设计成两级界面的形式，使其两者结合更加紧密，使用过程中交互更加自然友好。(2) Based on AR technology, the present invention designs two types of brain-computer interface paradigms into a two-level interface form, so that the two are more closely combined, and the interaction during use is more natural and friendly.
附图说明Description of drawings
图1为本发明的总体框架示意图；Fig. 1 is the overall framework schematic diagram of the present invention;
图2为刺激引导界面及算法运行流程图；Figure 2 is a flow chart of the stimulus guidance interface and algorithm operation;
图3为FBCCA算法流程图；Fig. 3 is the flow chart of FBCCA algorithm;
图4为CSP-LDA算法流程图；Fig. 4 is the flow chart of CSP-LDA algorithm;
图5为本发明的硬件构示意图。Fig. 5 is a schematic diagram of the hardware structure of the present invention.
具体实施方式Detailed ways
下面结合附图和具体实施例对本发明进行详细说明。本实施例以本发明技术方案为前提进行实施，给出了详细的实施方式和具体的操作过程，但本发明的保护范围不限于下述的实施例。The present invention will be described in detail below in conjunction with the accompanying drawings and specific embodiments. This embodiment is carried out on the premise of the technical solution of the present invention, and detailed implementation and specific operation process are given, but the protection scope of the present invention is not limited to the following embodiments.
如图1所示，本实施例提供一种基于AR的SSVEP-MI控制及康复训练轮椅，包括电动轮椅，还包括脑电采集模块、AR引导模块和算法模块，其中，脑电采集模块用于实时采集SSVEP脑电信号和/或MI脑电信号；AR引导模块用于显示以诱发SSVEP脑电信号和引导MI脑电信号的刺激引导界面；算法模块用于获取所述SSVEP脑电信号和MI脑电信号，分别对所述SSVEP脑电信号和MI脑电信号进行识别，基于识别结果产生显示信息，并产生针对所述刺激引导界面的显示控制指令和/或针对所述电动轮椅的行动控制指令。上述轮椅能够对两种脑电信号进行分析识别，控制轮椅的前进、后退、左转、右转，同时对患者大脑的神经功能进行康复。上述轮椅装置将SSVEP与MI融合起来，集中两种脑机接口的优点，削弱两者的缺点，实现一种对两种脑电信号进行分析识别，控制轮椅的前进、后退、左转、右转，同时对患者大脑的神经功能进行康复的新型轮椅装置，功能多样，可扩展性强，为脑控轮椅的发展提供新的思路。As shown in Figure 1, this embodiment provides an AR-based SSVEP-MI control and rehabilitation training wheelchair, including an electric wheelchair, and also includes an EEG acquisition module, an AR guidance module and an algorithm module, wherein the EEG acquisition module is used for Real-time acquisition of SSVEP EEG signals and/or MI EEG signals; the AR guidance module is used to display a stimulus guidance interface to induce SSVEP EEG signals and guide MI EEG signals; the algorithm module is used to obtain the SSVEP EEG signals and MI EEG signals, respectively identifying the SSVEP EEG signals and MI EEG signals, generating display information based on the identification results, and generating display control instructions for the stimulation guidance interface and/or action control for the electric wheelchair instruction. The above-mentioned wheelchair can analyze and identify two kinds of EEG signals, control the forward, backward, left turn, and right turn of the wheelchair, and at the same time restore the neurological function of the patient's brain. The above wheelchair device integrates SSVEP and MI, integrates the advantages of the two brain-computer interfaces, weakens the shortcomings of the two, and realizes a method of analyzing and identifying the two types of EEG signals to control the forward, backward, left and right turns of the wheelchair. , and at the same time, a new type of wheelchair device for rehabilitation of the neurological function of the patient's brain, with various functions and strong scalability, providing new ideas for the development of brain-controlled wheelchairs.
本实施例中，所述算法模块设置于一主机中，所述脑电采集模块包括64导湿电极脑电帽、信号放大器和路由器(智能同步中心)和打标器(多参数同步器)，其中，脑电帽用于接触头皮并检测微弱的电位变化，进行使用者脑电信号的实时获取，信号放大器用于增强电极检测到的信号，路由器用于传输采集的脑电数据，打标器用于硬件打标，为每一个trial的数据打上标签。路由器通过WiFi与信号放大器、打标器建立连接，打标器通过USB接口与主机建立串口通信，路由器与主机建立TCP通信。通过这些设备，主机可以获取实时的脑电信号，使脑电信号通过TCP/IP通信将信号发送到算法模块进行实时分析。In the present embodiment, the algorithm module is arranged in a host, and the EEG acquisition module includes 64 moisture-conducting electrode EEG caps, a signal amplifier and a router (intelligent synchronization center) and a marker (multi-parameter synchronizer), Among them, the EEG cap is used to contact the scalp and detect weak potential changes to obtain real-time EEG signals of the user. The signal amplifier is used to enhance the signals detected by the electrodes. The router is used to transmit the collected EEG data. For hardware marking, label the data of each trial. The router establishes a connection with the signal amplifier and marking device through WiFi, the marking device establishes serial port communication with the host computer through the USB interface, and the router establishes TCP communication with the host computer. Through these devices, the host computer can obtain real-time EEG signals, so that the EEG signals can be sent to the algorithm module through TCP/IP communication for real-time analysis.
在具体实施方式中，脑电帽具体使用博睿康64导湿电极脑电帽。In a specific embodiment, the EEG cap specifically uses the EEG cap with Boruikang 64 moisture-conducting electrodes.
本实施例中，AR引导模块通过UDP通信与算法模块连接，AR引导模块中显示的刺激引导界面用于诱发SSVEP脑电信号和引导MI脑电信号。所述刺激引导界面包括主界面和二级界面，其中，主界面为SSVEP刺激界面，包括五个分别代表前进、后退、左转、右转和停止的闪烁刺激块，二级界面为MI引导界面。在具体实施方式中，各所述闪烁刺激块按不同频率闪烁，以更好地引导使用者进行选择，并更准确确定使用者的选择。In this embodiment, the AR guidance module is connected with the algorithm module through UDP communication, and the stimulation guidance interface displayed in the AR guidance module is used to induce SSVEP EEG signals and guide MI EEG signals. The stimulation guidance interface includes a main interface and a secondary interface, wherein the main interface is the SSVEP stimulation interface, including five flickering stimulation blocks representing forward, backward, left turn, right turn and stop respectively, and the secondary interface is the MI guidance interface . In a specific implementation manner, each of the flickering stimulation blocks flickers at different frequencies to better guide the user to make a choice and to more accurately determine the user's choice.
本实施例中，算法模块包括SSVEP解码单元、判断单元、MI解码单元和返回单元，其中，SSVEP解码单元为主算法单元，MI解码单元为次级算法单元，算法的主次切换与刺激引导界面的切换保持同步运行。SSVEP解码单元用于在接收到SSVEP脑电信号时，采用FBCCA算法对所述SSVEP脑电信号进行解码，获得SSVEP分类结果；判断单元用于对SSVEP分类结果进行判断，并基于判断结果执行相应动作，具体地：当SSVEP分类结果为前进、后退或停止时，同时产生对应的显示信息和针对所述电动轮椅的对应的行动控制指令，当SSVEP分类结果为左转或右转时，产生针对所述刺激引导界面的显示控制指令，使刺激引导界面显示为二级界面；MI解码单元用于在接收到MI脑电信号时，采用CSP-LDA算法对所述MI脑电信号进行解码，获得MI分类结果，将该MI分类结果与SSVEP分类结果进行比对，若一致，则产生对应的显示信息及对应的行动控制指令，否则重复采集MI脑电信号；返回单元用于产生针对所述刺激引导界面的显示控制指令，使刺激引导界面显示主界面。In this embodiment, the algorithm module includes a SSVEP decoding unit, a judging unit, an MI decoding unit, and a return unit, wherein the SSVEP decoding unit is the main algorithm unit, and the MI decoding unit is the secondary algorithm unit, and the primary and secondary switching of the algorithm and the stimulation guide interface The handoffs run synchronously. The SSVEP decoding unit is used to decode the SSVEP EEG signal using the FBCCA algorithm to obtain the SSVEP classification result when receiving the SSVEP EEG signal; the judging unit is used to judge the SSVEP classification result and execute corresponding actions based on the judgment result , specifically: when the SSVEP classification result is forward, backward or stop, the corresponding display information and the corresponding action control instruction for the electric wheelchair will be generated at the same time; Describe the display control instructions of the stimulation guidance interface, so that the stimulation guidance interface is displayed as a secondary interface; the MI decoding unit is used to decode the MI EEG signal by using the CSP-LDA algorithm to obtain the MI EEG signal when receiving the MI EEG signal. Classification results, compare the MI classification results with the SSVEP classification results, if they are consistent, then generate corresponding display information and corresponding action control instructions, otherwise repeatedly collect MI EEG signals; the return unit is used to generate instructions for the stimulation guidance The display control instruction of the interface makes the stimulus guidance interface display the main interface.
具体地，如图2所示，AR引导模块显示刺激引导界面，先显示主界面的五个闪烁刺激块，诱发大脑产生SSVEP脑电信号，使用者注视其中一个闪烁块，将此时的脑电信号发送到算法模块中，使用SSVEP算法进行分析分类。当分类结果是前进、后退或停止时，算法模块发送结果给电动轮椅，进行相应动作，并反馈给刺激引导界面进行显示，随后进行下一轮SSVEP信号诱发；当分类结果是左转或右转时，算法模块切换为MI算法进行分析，刺激引导界面也进入二级界面对MI信号的产生进行引导，使用者根据引导进行运动想象，此时算法模块的MI算法对脑电信号进行分析分类，直至运动想象结果与分类结果一致，即分类结果正确，算法模块就会将左转或右转指令发送给电动轮椅，并将分类结果反馈给刺激引导界面进行显示，随后刺激引导界面回到SSVEP主界面，算法模块回到SSVEP处理算法主线程，接着进行下一轮SSVEP信号诱发。Specifically, as shown in Figure 2, the AR guidance module displays the stimulation guidance interface, and first displays five flickering stimulation blocks on the main interface to induce the brain to generate SSVEP EEG signals. The signal is sent to the algorithm module, and the SSVEP algorithm is used for analysis and classification. When the classification result is forward, backward or stop, the algorithm module sends the result to the electric wheelchair, performs corresponding actions, and feeds back to the stimulation guidance interface for display, and then proceeds to the next round of SSVEP signal induction; when the classification result is turning left or turning right When the algorithm module switches to the MI algorithm for analysis, the stimulus guidance interface also enters the secondary interface to guide the generation of MI signals, and the user performs motor imagination according to the guidance. At this time, the MI algorithm of the algorithm module analyzes and classifies the EEG signals. Until the motor imagery result is consistent with the classification result, that is, the classification result is correct, the algorithm module will send the left turn or right turn command to the electric wheelchair, and feed back the classification result to the stimulation guidance interface for display, and then the stimulation guidance interface returns to the SSVEP main interface, the algorithm module returns to the main thread of the SSVEP processing algorithm, and then proceeds to the next round of SSVEP signal induction.
算法模块主要包含两个部分，第一部分是分析SSVEP信号的FBCCA算法，另一部分是分析MI信号的CSP-LDA算法。其中FBCCA是主线程算法，会根据采集到的脑电信号计算出五个分类结果，当SSVEP信号分类结果为前进、后退或停止时，分类结果直接通过串口将指令发送给电动轮椅的控制器进行控制，并反馈给刺激引导界面进行显示；当SSVEP信号分类结果为左转或右转时，跳转至CSP-LDA算法对MI脑电信号进行分析，CSP-LDA算法会得到两个分类结果，正确或者错误，如果分类结果正确，则将控制指令发送给电动轮椅的控制器进行控制，如果分类结果错误，则继续进行运动想象，直至想象正确。The algorithm module mainly includes two parts. The first part is the FBCCA algorithm for analyzing SSVEP signals, and the other part is the CSP-LDA algorithm for analyzing MI signals. Among them, FBCCA is the main thread algorithm, which will calculate five classification results according to the collected EEG signals. When the SSVEP signal classification results are forward, backward or stop, the classification results will be sent directly to the controller of the electric wheelchair through the serial port. Control, and feed back to the stimulus guidance interface for display; when the SSVEP signal classification result is left or right turn, jump to the CSP-LDA algorithm to analyze the MI EEG signal, and the CSP-LDA algorithm will get two classification results, Correct or incorrect, if the classification result is correct, the control instruction will be sent to the controller of the electric wheelchair for control, if the classification result is wrong, continue to perform motor imagery until the imagination is correct.
SSVEP脑电信号利用FBCCA进行解码分析的具体过程为：The specific process of decoding and analyzing SSVEP EEG signals using FBCCA is as follows:
对SSVEP脑电信号进行分类前，要对其进行预处理，减小噪声成分。使用滤波，降采样，去基线漂移等方法。Before classifying SSVEP EEG signals, it needs to be preprocessed to reduce noise components. Use filtering, downsampling, de-baseline drift, etc.
(1)滤波：由于EEG常分布在1-100Hz的区间内，过低或过高的成分都可以视作噪声。常用的方式是使用带通滤波器和陷波器，带通滤波器主要用于滤除EEG信号中包含的高频和低频成分，陷波器主要用于滤除50Hz的工频干扰。(1) Filtering: Since EEG is often distributed in the range of 1-100Hz, components that are too low or too high can be regarded as noise. The commonly used method is to use a band-pass filter and a notch filter. The band-pass filter is mainly used to filter out high-frequency and low-frequency components contained in the EEG signal, and the notch filter is mainly used to filter out 50Hz power frequency interference.
(2)去除眼电伪迹：眼电伪迹常常存在于MI脑电信号中，由被试者的眨眼动作产生。可以使用通过独立成分分析(ICA)算法来去除，ICA算法可以将原始脑电数据分解为多个独立成分，从而去除EEG的肌电/眼电伪迹成分，但具体参数需要调节，不然容易将含有MI脑电信号的成分滤除。(2) Removal of EEG artifacts: EEG artifacts often exist in MI EEG signals, which are produced by the subject's eye blinking. It can be removed by the independent component analysis (ICA) algorithm. The ICA algorithm can decompose the original EEG data into multiple independent components, thereby removing the EMG/oculoelectric artifact components of EEG, but the specific parameters need to be adjusted, otherwise it is easy to Components containing MI EEG signals were filtered out.
(3)降采样：由于部分情况下采用的处理器计算能力弱，采样率过高会导致计算时间过长或算法出错，在这种情况下，需要对原始的脑电数据进行降采样，减少数据量与计算复杂度。但需要注意，过度降低采样率，会使信号失真，损失有效信息。(3) Downsampling: Due to the weak computing power of the processor used in some cases, too high a sampling rate will lead to long calculation time or algorithm errors. In this case, it is necessary to downsample the original EEG data to reduce Data volume and computational complexity. However, it should be noted that excessively reducing the sampling rate will distort the signal and lose effective information.
CCA算法的作用是用于测量两个变量集之间的相关性。它可以分析两组变量之间的线性关系，并给出它们的相关系数。The role of the CCA algorithm is to measure the correlation between two variable sets. It can analyze the linear relationship between two sets of variables and give their correlation coefficient.
FBCCA算法是CCA算法的改进版本，具有稳定性高，准确率高等特点。目前，FBCCA算法常用于SSVEP脑电信号处理，该算法将滤波组和CCA算法相结合，充分利用了SSVEP脑电信号的谐波特性，将基波与谐波成分结合起来，进一步提高算法的准确率。FBCCA的算法步骤如图3所示，具体步骤包括：The FBCCA algorithm is an improved version of the CCA algorithm, which has the characteristics of high stability and high accuracy. At present, the FBCCA algorithm is often used in SSVEP EEG signal processing. This algorithm combines the filter bank and the CCA algorithm, makes full use of the harmonic characteristics of the SSVEP EEG signal, and combines the fundamental wave and harmonic components to further improve the accuracy of the algorithm. Rate. The algorithm steps of FBCCA are shown in Figure 3, and the specific steps include:
(1)构造对应的子带滤波器组，将预处理过后的SSVEP脑电信号使用滤波组滤波，得到信号的子带分量。(1) Construct the corresponding subband filter bank, and use the filter bank to filter the preprocessed SSVEP EEG signal to obtain the subband component of the signal.
(2)对子带分量分别使用CCA算法，得到对应的相关系数。(2) Use the CCA algorithm for the sub-band components respectively to obtain the corresponding correlation coefficients.
(3)各子带的相关系数的平方与权重点乘并相加，得到对应频率的相关系数。取最大的相关系数所在的频率为结果。(3) The square of the correlation coefficient of each sub-band is multiplied by the weight and added to obtain the correlation coefficient of the corresponding frequency. Take the frequency where the largest correlation coefficient is located as the result.
MI脑电信号使用CSP-LDA算法进行分析的具体过程为：The specific process of MI EEG signal analysis using CSP-LDA algorithm is as follows:
MI脑电信号中有着大量噪声和伪迹，为了提高分类的准确度，必须先对原始信号进行类似对SSVEP信号的预处理操作。There are a lot of noise and artifacts in the MI EEG signal. In order to improve the classification accuracy, the original signal must be preprocessed similarly to the SSVEP signal.
CSP是一种特征提取算法，本质上是利用了矩阵的对角化，不能单独使用，需要搭配分类算法来使用，如线性判别分析(LDA)或支持向量机(SVM)。CSP的主要思想是构造一个投影矩阵，将多通道的脑电信号进行线性变换，使得变换后的信号在特定方向下方差最大或最小。再将提取出的特征向量用于后续的分类器，进行分类。CSP算法提取的是MI脑电信号的空间域特征。CSP is a feature extraction algorithm, which essentially uses the diagonalization of the matrix and cannot be used alone. It needs to be used with a classification algorithm, such as linear discriminant analysis (LDA) or support vector machine (SVM). The main idea of CSP is to construct a projection matrix to linearly transform the multi-channel EEG signal so that the transformed signal has the largest or smallest difference in a specific direction. The extracted feature vectors are then used in subsequent classifiers for classification. The CSP algorithm extracts the spatial domain features of MI EEG signals.
基于左右手MI脑电信号的建模过程，对CSP算法进行分析，解释CSP算法的流程。假设单个trial的数据是EN*T，其中N为通道数，T是采样点数。CSP的具体步骤如图4所示：Based on the modeling process of left and right hand MI EEG signals, the CSP algorithm is analyzed and the flow of the CSP algorithm is explained. Assume that the data of a single trial is EN*T , where N is the number of channels and T is the number of sampling points. The specific steps of CSP are shown in Figure 4:
(1)先计算单个trial数据的归一化后的协方差，trace(X)表示矩阵X的迹，即矩阵对角线上元素的和。(1) First calculate the normalized covariance of a single trial data, trace(X) represents the trace of the matrix X, that is, the sum of the elements on the diagonal of the matrix.
(2)分别计算左手、右手的平均协方差Cl、Cr和混合空间协方差Cc：(2) Calculate the left-handed and right-handed average covariance Cl , Cr and the mixed space covariance Cc :
其中，Cc就是实验的平均协方差矩阵。Among them, Cc is the average covariance matrix of the experiment.
(3)对平均协方差矩阵Cc进行特征值分解：(3) Perform eigenvalue decomposition on the average covariance matrix Cc :
其中，Λc是特征值对角矩阵，Uc是特征向量矩阵。Among them, Λc is a diagonal matrix of eigenvalues, and Uc is a matrix of eigenvectors.
(4)求出平均协方差矩阵Cc的白化矩阵P：(4) Find the whitening matrix P of the average covariance matrix Cc :
(5)对Cl、Cr进行白化，得到空间系数矩阵Sl、Sr：(5) Whiten Cl and Cr to obtain spatial coefficient matrices Sl and Sr :
(6)对空间系数矩阵Sl、Sr进行特征值分解：(6) Decompose the eigenvalues of the spatial coefficient matrices Sl and Sr :
(7)计算空间滤波器W：(7) Calculate the spatial filter W:
W＝(BTP)T (7)W = (BT P)T (7)
(8)使用空间滤波器W对EN*T进行滤波：(8) Use the spatial filter W to filterEN*T :
ZN*T＝WN*NEN*T (8)ZN*T ＝WN*N EN*T (8)
(9)计算特征向量f。特征向量f的特征对数需要人工选定，其最大值不能够大于EEG通道数N。实际上，就是在提取特征向量时，提取Z的前m行和后m行(2m＜N)。m即为需要人工调节的超参数：(9) Calculate the feature vector f. The characteristic logarithm of the characteristic vector f needs to be manually selected, and its maximum value cannot be greater than the number N of EEG channels. In fact, when extracting the feature vector, the first m rows and the last m rows of Z are extracted (2m<N). m is the hyperparameter that needs to be adjusted manually:
var(X)代表计算样本X的方差。var(X) stands for calculating the variance of sample X.
经过CSP特征提取后，得到的最终结果是f＝{f1,f2,…,f2m}，m是人工选择的特征对数。前m维与后m维，一方为极大值，另一方就为最小值。后续可以通过LDA等分类算法，将特征向量进行分类。After CSP feature extraction, the final result obtained is f={f1 , f2 ,...,f2m }, where m is the logarithm of manually selected features. For the front m dimension and the back m dimension, one is the maximum value, and the other is the minimum value. Subsequently, the feature vectors can be classified by classification algorithms such as LDA.
如图5所示，本实施例中，AR引导模块具体采用AR头显，包括AR眼镜；电动轮椅包括通过串口方式与算法模块连接的轮椅控制器；算法模块设置于主机中。AR头显用于显示刺激引导界面，诱发脑电信号，主机用于脑电信号分析以及运行刺激引导界面，轮椅控制器通过串口接收主机传送的控制命令对轮椅的移动进行控制。As shown in FIG. 5 , in this embodiment, the AR guidance module specifically adopts an AR head-mounted display, including AR glasses; the electric wheelchair includes a wheelchair controller connected to the algorithm module through a serial port; the algorithm module is set in the host. The AR head-mounted display is used to display the stimulation guidance interface and induce EEG signals. The host computer is used to analyze the EEG signals and run the stimulation guidance interface. The wheelchair controller receives the control commands sent by the host computer through the serial port to control the movement of the wheelchair.
上述轮椅使用时，使用者佩戴脑电帽，由刺激引导界面发送开始标志给算法模块，保证两者同步，并开始诱发或引导产生实时脑电信号，同时脑电采集模块在脑电信号中根据刺激引导界面运行进行硬件打标，保证脑电信号与界面的时间戳同步。脑电采集模块将采集到的脑电信号通过TCP发送到主机的算法模块进行分析，随后将分析结果通过UDP发送给刺激引导界面进行反馈显示，同时通过串口将分析结果发送给轮椅控制器进行轮椅的行动控制。When using the above-mentioned wheelchair, the user wears an EEG cap, and the stimulation guidance interface sends a start sign to the algorithm module to ensure that the two are synchronized, and starts to induce or guide the generation of real-time EEG signals. Stimulate and guide the operation of the interface for hardware marking to ensure that the EEG signal is synchronized with the time stamp of the interface. The EEG acquisition module sends the collected EEG signals to the algorithm module of the host computer for analysis through TCP, and then sends the analysis results to the stimulation guidance interface for feedback display through UDP, and at the same time sends the analysis results to the wheelchair controller through the serial port for wheelchair monitoring. movement control.
以上详细描述了本发明的较佳具体实施例。应当理解，本领域的普通技术人员无需创造性劳动就可以根据本发明的构思作出诸多修改和变化。因此，凡本技术领域中技术人员依本发明的构思在现有技术的基础上通过逻辑分析、推理或者有限的实验可以得到的技术方案，皆应在由权利要求书所确定的保护范围内。The preferred specific embodiments of the present invention have been described in detail above. It should be understood that those skilled in the art can make many modifications and changes according to the concept of the present invention without creative effort. Therefore, all technical solutions that can be obtained by those skilled in the art based on the concept of the present invention through logical analysis, reasoning or limited experiments on the basis of the prior art shall be within the scope of protection defined by the claims.
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310693246.0ACN116687677A (en) | 2023-06-12 | 2023-06-12 | AR-based SSVEP-MI control and rehabilitation training wheelchair |
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310693246.0ACN116687677A (en) | 2023-06-12 | 2023-06-12 | AR-based SSVEP-MI control and rehabilitation training wheelchair |
| Publication Number | Publication Date |
|---|---|
| CN116687677Atrue CN116687677A (en) | 2023-09-05 |
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310693246.0APendingCN116687677A (en) | 2023-06-12 | 2023-06-12 | AR-based SSVEP-MI control and rehabilitation training wheelchair |
| Country | Link |
|---|---|
| CN (1) | CN116687677A (en) |
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN118732845A (en)* | 2024-06-20 | 2024-10-01 | 上海大学 | A group brain-computer interface collaborative control system and method based on SSVEP |
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102309380A (en)* | 2011-09-13 | 2012-01-11 | 华南理工大学 | Intelligent wheelchair based on multimode brain-machine interface |
| CN102866775A (en)* | 2012-09-04 | 2013-01-09 | 同济大学 | System and method for controlling brain computer interface (BCI) based on multimode fusion |
| CN107315478A (en)* | 2017-07-05 | 2017-11-03 | 中国人民解放军第三军医大学 | A kind of Mental imagery upper limbs intelligent rehabilitation robot system and its training method |
| CN112114670A (en)* | 2020-09-10 | 2020-12-22 | 季华实验室 | Man-machine co-driving system based on hybrid brain-computer interface and control method thereof |
| CN112114662A (en)* | 2020-08-03 | 2020-12-22 | 西安交通大学 | Reality-augmented self-adaptive dynamic multi-scene evoked brain control method |
| CN113101021A (en)* | 2021-04-02 | 2021-07-13 | 上海交通大学 | Mechanical arm control method based on MI-SSVEP hybrid brain-computer interface |
| CN113274032A (en)* | 2021-04-29 | 2021-08-20 | 上海大学 | Cerebral apoplexy rehabilitation training system and method based on SSVEP + MI brain-computer interface |
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102309380A (en)* | 2011-09-13 | 2012-01-11 | 华南理工大学 | Intelligent wheelchair based on multimode brain-machine interface |
| CN102866775A (en)* | 2012-09-04 | 2013-01-09 | 同济大学 | System and method for controlling brain computer interface (BCI) based on multimode fusion |
| CN107315478A (en)* | 2017-07-05 | 2017-11-03 | 中国人民解放军第三军医大学 | A kind of Mental imagery upper limbs intelligent rehabilitation robot system and its training method |
| CN112114662A (en)* | 2020-08-03 | 2020-12-22 | 西安交通大学 | Reality-augmented self-adaptive dynamic multi-scene evoked brain control method |
| CN112114670A (en)* | 2020-09-10 | 2020-12-22 | 季华实验室 | Man-machine co-driving system based on hybrid brain-computer interface and control method thereof |
| CN113101021A (en)* | 2021-04-02 | 2021-07-13 | 上海交通大学 | Mechanical arm control method based on MI-SSVEP hybrid brain-computer interface |
| CN113274032A (en)* | 2021-04-29 | 2021-08-20 | 上海大学 | Cerebral apoplexy rehabilitation training system and method based on SSVEP + MI brain-computer interface |
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN118732845A (en)* | 2024-06-20 | 2024-10-01 | 上海大学 | A group brain-computer interface collaborative control system and method based on SSVEP |
| CN118732845B (en)* | 2024-06-20 | 2025-07-25 | 上海大学 | Group brain-computer interface cooperative control system and method based on SSVEP |
| Publication | Publication Date | Title |
|---|---|---|
| CN110765920B (en) | Motor imagery classification method based on convolutional neural network | |
| CN114533086B (en) | Motor imagery brain electrolysis code method based on airspace characteristic time-frequency transformation | |
| CN101219048B (en) | Method for extracting brain electrical character of imagine movement of single side podosoma | |
| CN112488002B (en) | Emotion recognition method and system based on N170 | |
| Cheng et al. | Position-independent gesture recognition using sEMG signals via canonical correlation analysis | |
| Tang et al. | Motor imagery EEG decoding based on multi-scale hybrid networks and feature enhancement | |
| CN113951900B (en) | Motor imagery intention recognition method based on multi-mode signals | |
| CN102184019B (en) | Method for audio-visual combined stimulation of brain-computer interface based on covert attention | |
| CN105956624A (en) | Motor imagery electroencephalogram classification method based on space-time-frequency optimization feature sparse representation | |
| CN103258215A (en) | Multi-lead correlation analysis electroencephalo-graph (EEG) feature extraction method | |
| CN102306303B (en) | A Feature Extraction Method of EEG Signals Based on Small Training Samples | |
| CN103425249A (en) | Electroencephalogram signal classifying and recognizing method based on regularized CSP and regularized SRC and electroencephalogram signal remote control system | |
| CN102722728A (en) | Motion image electroencephalogram classification method based on channel weighting supporting vector | |
| CN105054928A (en) | Emotion display equipment based on BCI (brain-computer interface) device electroencephalogram acquisition and analysis | |
| CN103699217A (en) | Two-dimensional cursor motion control system and method based on motor imagery and steady-state visual evoked potential | |
| CN114082169B (en) | Disabled hand soft body rehabilitation robot motor imagery identification method based on electroencephalogram signals | |
| CN113952614A (en) | Hand function rehabilitation double-closed-loop regulation and control system integrating mechanical arm and functional electrical stimulation | |
| CN106502410A (en) | Improve the transcranial electrical stimulation device of Mental imagery ability and method in brain-computer interface | |
| CN105904459A (en) | Robot control system based on eye electrical signal recognition and design method of system | |
| WO2025130992A1 (en) | Real-time artifact processing and feature extraction method and system for electroencephalogram signal | |
| CN108523883A (en) | A kind of continuous Mental imagery identifying system of left and right index finger based on actual act modeling | |
| CN116687677A (en) | AR-based SSVEP-MI control and rehabilitation training wheelchair | |
| Xie et al. | A brain controlled wheelchair based on common spatial pattern | |
| Zhang et al. | EEG recognition of motor imagery based on SVM ensemble | |
| CN117332259A (en) | Information fusion-based motor imagery brain-computer interface time-frequency combination optimization method |
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |