CN114631830A - Simplified distributed dipole model establishing and identifying method based on D-K partition - Google Patents

Abstract

Translated fromChinese

本发明公开了基于D‑K分区的简化分布式偶极子模型建立与识别方法，具体包括：首先，利用不同的带通滤波器对原始MI‑EEG进行滤波，以挑选与运动想象活动相关的最优频带；然后，对挑选出的每个子带进行脑电逆变换，将头皮EEG转换为脑皮层中的偶极子；接着，获得基于神经解剖学D‑K分区的中心偶极子，以构建简化分布式偶极子模型，将大脑皮层中心偶极子的活动视为神经动力学系统，构建4D数据表达；最后，将多频带数据表达进行融合并输入至设计好的n分支并行的nB3DCNN中，从时‑频‑空三个维度进行综合特征提取与识别。本发明体现了不同频带下，偶极子在3D空间中幅值随着时间的变化，利用少量中心偶极子反映了整个大脑皮质层由运动想象引起的神经电活动。

The invention discloses a simplified distributed dipole model establishment and identification method based on D-K partition, which specifically includes: firstly, using different band-pass filters to filter the original MI-EEG, so as to select the ones related to motor imagery activities. optimal frequency band; then, inverse EEG transformation was performed on each of the selected subbands to convert scalp EEG into dipoles in the cerebral cortex; then, central dipoles based on neuroanatomical D‑K partitions were obtained to A simplified distributed dipole model is constructed, the activity of the central dipole in the cerebral cortex is regarded as a neural dynamic system, and a 4D data representation is constructed; finally, the multi-band data representation is fused and input into the designed n-branch parallel nB3DCNN , comprehensive feature extraction and recognition are carried out from the three dimensions of time-frequency-space. The invention reflects the variation of the amplitude of the dipole in 3D space with time under different frequency bands, and uses a small number of central dipoles to reflect the neural electrical activity caused by motor imagery in the entire cerebral cortex.

Description

Translated fromChinese

基于D-K分区的简化分布式偶极子模型建立与识别方法A Simplified Distributed Dipole Model Establishment and Identification Method Based on D-K Partitioning

技术领域technical field

本发明涉及脑源成像(ESI)和三维卷积神经网络(3DCNN)，用于运动想象脑电信号(MI-EEG)识别技术领域。The invention relates to brain source imaging (ESI) and three-dimensional convolutional neural network (3DCNN), which are used in the technical field of motor imagery electroencephalogram (MI-EEG) recognition.

背景技术Background technique

基于运动想象的脑机接口技术(BCI)是神经康复研究中的重要方向，如何准确的检测大脑皮质层的变化对运动想象的解码至关重要。通过头部模型的建立和在头皮层测得的EEG信号，可以计算出具有高时空分辨率的等效电流偶极子，以表示大脑内部的神经电活动，为皮质层动力学提供了新的研究思路。然而，如果利用整个皮质层的偶极子构建特征，可能会造成信息冗余，生成特征数据成本过高；若进行感兴趣区域(ROI)的挑选，仅使用部分偶极子构建特征，又可能会造成关键信息丢失。如何挑选出最能够代表运动想象特征的偶极子，是一个值得研究的问题。Brain-computer interface (BCI) based on motor imagery is an important direction in neurorehabilitation research. How to accurately detect changes in the cerebral cortex is crucial to the decoding of motor imagery. Through the establishment of a head model and EEG signals measured in the scalp, equivalent current dipoles can be calculated with high spatiotemporal resolution to represent neural electrical activity within the brain, providing new insights into cortical dynamics Research ideas. However, if the dipoles of the entire cortical layer are used to construct features, it may cause information redundancy and the cost of generating feature data is too high. It will result in the loss of key information. How to select the dipole that can best represent the characteristics of motor imagination is a problem worthy of study.

发明内容SUMMARY OF THE INVENTION

针对上述不足之处，本发明提供一种基于D-K分区的简化分布式偶极子模型建立与识别方法。In view of the above shortcomings, the present invention provides a simplified distributed dipole model establishment and identification method based on D-K partition.

具体涉及：首先，利用不同的带通滤波器对所有通道的原始MI-EEG进行滤波，以挑选与运动想象活动相关的最优频带；然后，对挑选出的每个子带信号进行基于sLORETA的脑电逆变换，从而将头皮EEG数据转换为大脑皮层空间中的偶极子；接着，获得基于神经解剖学Desikan-Killiany(D-K)分区的中心偶极子，以构建简化分布式偶极子模型(SDDM)，将大脑皮层中心偶极子的活动视为神经动力学系统；将每个采样时刻的中心偶极子幅值赋至其3D空间坐标下，形成3D偶极子幅值矩阵，并按照时间维度堆叠成4D数据表达；最后，将多频带数据进行融合并输入至设计好的n分支并行的3DCNN(nB3DCNN)中，从时-频-空三个维度进行综合特征提取与识别。Specifically, it involves: first, filtering the raw MI-EEG of all channels with different bandpass filters to pick the optimal frequency band related to motor imagery activities; then, performing sLORETA-based brain Inverse electrical transformation to convert scalp EEG data into dipoles in cerebral cortical space; then, central dipoles based on neuroanatomical Desikan-Killiany (D-K) partitions were obtained to construct a simplified distributed dipole model ( SDDM), the activity of the central dipole in the cerebral cortex is regarded as a neural dynamic system; the central dipole amplitude at each sampling moment is assigned to its 3D spatial coordinates to form a 3D dipole amplitude matrix, and according to The time dimension is stacked into a 4D data representation; finally, the multi-band data is fused and input into the designed n-branch parallel 3DCNN (nB3DCNN), and comprehensive feature extraction and recognition are performed from the three dimensions of time-frequency-space.

具体地：specifically:

(1)通过不同的带通滤波器对所有通道的原始MI-EEG进行滤波，并计算其能量值，以挑选与运动想象活动相关的最优频带。(1) The raw MI-EEGs of all channels are filtered through different bandpass filters and their energy values are calculated to pick out the optimal frequency bands related to motor imagery activities.

(2)对每个最优子带求解脑电逆问题，将头皮EEG数据转换为大脑皮层空间中的偶极子。基于神经解剖学D-K分区获得每个区域的中心偶极子以构建SDDM，使用少量偶极子反映整个大脑皮质层中神经元群的电活动。(2) Solve the EEG inverse problem for each optimal subband, and convert the scalp EEG data into dipoles in the cerebral cortex space. The central dipole of each region was obtained based on neuroanatomical D-K partitioning to construct the SDDM, using a small number of dipoles to reflect the electrical activity of neuronal populations in the entire cerebral cortical layer.

(3)将多频带的SDDMs进行融合，构建多频带融合的数据表达，并输入至设计好的n分支并行的3DCNN(nB3DCNN)中，对其复合特征进行提取与识别。(3) Multi-band SDDMs are fused to construct multi-band fusion data representation, and input into the designed n-branch parallel 3DCNN (nB3DCNN) to extract and identify its composite features.

本发明的具体步骤如下：The concrete steps of the present invention are as follows:

Step1基于能量的频带优选。Step 1 Band optimization based on energy.

Step1.1假设原始多通道MI-EEG为E_r∈R^N×T，其中N代表头皮电极的数量，T代表采样时刻的数量。分别使用24个具有相同宽度，且能够覆盖与运动想象活动最相关频带的带通滤波器(8-9Hz,9-10Hz,…,and 31-32Hz)，对所有次实验的平均MI-EEG信号

进行滤波，以获得24个子频带EEG信号，记为E_s，其中s∈{1,2,…,24}代表所有子频带的序号。Step1.1 Suppose the original multi-channel MI-EEG is E_r ∈ R^N×T , where N represents the number of scalp electrodes and T represents the number of sampling moments. Average MI-EEG signal for all experiments using 24 bandpass filters (8-9Hz, 9-10Hz,...,and 31-32Hz) with the same width and covering the frequency band most relevant to motor imagery activity, respectively.

Filtering is performed to obtain 24 subband EEG signals, denoted Es , where s∈{1,2,_... ,24} represents the sequence numbers of all subbands.

Step1.2计算在运动准备期间

为运动准备期起始点，

为运动准备期结束点，t为采样时刻)，所有导联(c∈[1,N]，c为电极)的能量和，表示为：

接着将所有的子频带信号按照能量和进行降序排列，把前n个子频带作为最优频带，记为top b_i，i∈{1,2,…,n}。Step1.2 Calculated during exercise preparation

It is the starting point of the exercise preparation period,

is the end point of the exercise preparation period, t is the sampling time), the energy sum of all leads (c∈[1,N], c is the electrode), expressed as:

Then, all sub-band signals are arranged in descending order according to the energy sum, and the first n sub-bands are regarded as optimal frequency bands, denoted as top b_i , i∈{1,2,...,n}.

Step2简化分布式偶极子模型的构建。Step2 simplifies the construction of the distributed dipole model.

Step2.1头皮层到脑皮层的数据转换。使用sLORETA算法对每一个选取的最优子频带top b_i均进行脑电逆变换，得到所有偶极子的时间序列，可以表示为：Step2.1 Data conversion from scalp to cerebral cortex. Using the sLORETA algorithm to perform inverse EEG transformation on each selected optimal subband top b_i to obtain the time series of all dipoles, which can be expressed as:

其中，

代表在第t个采样时刻时，第k个偶极子所代表的幅值。in,

Represents the amplitude represented by the k-th dipole at the t-th sampling time.

Step2.2中心偶极子序列的选择。基于D-K分区，将大脑分为了68个神经解剖区，为每个区域选择最具有代表性的中心偶极子。这样所有偶极子就可以表示为：Step2.2 Selection of central dipole sequence. Based on the D-K partition, the brain was divided into 68 neuroanatomical regions, and the most representative central dipole was selected for each region. Then all dipoles can be expressed as:

代表在第j个区域中偶极子的时间序列，N_j表示第j个区域中偶极子的个数，且j∈{1,2,…,68}。

represents the time series of dipoles in the jth region,_Nj represents the number of dipoles in the jth region, and j∈{1,2,…,68}.

然后，将每一个区域近似看作一个立方体，选择最靠近该立方体中心的偶极子作为最相关的偶极子，称为中心偶极子

大脑皮层中所有偶极子就可以近似地用中心偶极子等效为：Then, each region is approximated as a cube, and the dipole closest to the center of the cube is chosen as the most relevant dipole, called the central dipole

All dipoles in the cerebral cortex can be approximately equivalent to the central dipole as:

x_j,y_j,z_j)代表了第j个区域中心偶极子的三维坐标，

且

分别代表了x坐标轴(y坐标轴，z坐标轴)的最大值和最小值。x_j , y_j , z_j ) represent the three-dimensional coordinates of the center dipole of the jth region,

and

They represent the maximum and minimum values of the x-axis (y-axis, z-axis), respectively.

Step2.3简化分布式偶极子模型的构建。构建简化分布式偶极子模型是为了在固定空间位置下，反映中心偶极子随着时间的变化。首先，对各区中心偶极子的3D坐标(x_j,y_j,z_j)进行平移、放大和取整等操作，在不改变所有中心偶极子相对空间位置的情况下，获得其正整数3D坐标(x_j',y_j',z_j')。然后，就得到了大脑皮质68个带有3D坐标的中心偶极子，将其称为简化分布式偶极子模型，记为SDDM，由下式表示：Step2.3 Simplify the construction of the distributed dipole model. The simplified distributed dipole model is constructed to reflect the change of the central dipole with time under a fixed spatial position. First, perform translation, enlargement and rounding operations on the 3D coordinates (x_j , y_j , z_j ) of the central dipoles in each area, and obtain their positive integers without changing the relative spatial positions of all central dipoles 3D coordinates (x_j ',y_j ',z_j '). Then, 68 central dipoles with 3D coordinates in the cerebral cortex are obtained, which are called the simplified distributed dipole model, denoted as SDDM, and expressed by the following formula:

Step3基于多频带融合的数据表达与识别。Step3 is based on data expression and recognition of multi-band fusion.

Step3.1单频带的4D幅值矩阵构建。对于每一个挑选出的子频带top b_i,i∈{1,2,…,n}，将任意时刻的中心偶极子幅值赋至其对应的3D空间位置坐标下，形成3D幅值矩阵；接着，沿着采样时刻将3D幅值矩阵进行堆叠，生成4D幅值矩阵，记为下式：Step3.1 Construction of 4D amplitude matrix of single frequency band. For each selected subband top b_i ,i∈{1,2,…,n}, assign the central dipole amplitude at any time to its corresponding 3D spatial position coordinates to form a 3D amplitude matrix ; Next, stack the 3D amplitude matrix along the sampling time to generate a 4D amplitude matrix, which is recorded as the following formula:

其中，x_m＝max(x'₁,x'₂,…,x'₆₈),y_m＝max(y'₁,y'₂,…,y'₆₈),zm＝max(z'₁,z'₂,…,z'₆₈)。因此，当前4D幅值矩阵包含了所有中心偶极子在固定3D空间位置下，幅值随着时间的变化信息。Wherein, x_m =max(x'₁ ,x'₂ ,...,x'₆₈ ),y_m =max(y'₁ ,y'₂ ,...,y'₆₈ ),zm=max(z'₁ , z'₂ ,…,z'₆₈ ). Therefore, the current 4D amplitude matrix contains the information of the amplitude variation with time of all the central dipoles in a fixed 3D spatial position.

Step3.2多频带融合的数据表达。基于4D幅值矩阵D_mi和最优子频带top b_i，构建了多频带融合的数据表达，从时-频-空三个维度展示运动想象的复合特征。将前n个能量最高子带的D_mi进行融合，构建新型数据表达，记为下式：Step3.2 Data expression of multi-band fusion. Based on the 4D magnitude matrix D_mi and the optimal sub-band top_bi , a multi-band fusion data representation is constructed, showing the composite features of motor imagery from three dimensions: time-frequency-space. The_Dmi of the first n highest-energy subbands are fused to construct a new data representation, which is recorded as the following formula:

它集成了脑源域中所有中心偶极子的时域、频域和空域信息，反映了在三维源空间中最佳子频带的分布式神经动力学变化。It integrates time-, frequency-, and spatial-domain information of all central dipoles in the brain source domain, reflecting the distributed neural dynamics of the optimal subbands in the three-dimensional source space.

Step3.3并行多分支3DCNN的设计。根据多频带融合数据表达D_R的特点，设计了n分支并行的3DCNN(nB3DCNN)用于特征提取和识别。其中分支个数n取决于最佳子频带的个数，每个分支具有相同的结构，分别由两个卷积层，一个最大池化层和一个连接层组成。数据表达中n个最具有代表性的频带，依次输入至n个分支中进行特征提取，然后展平至连接层得到输出类别。Step3.3 Design of parallel multi-branch 3DCNN. According to the characteristics of multi-band fusion data expressing DR, an n-branch parallel 3DCNN (_nB3DCNN ) is designed for feature extraction and recognition. The number of branches n depends on the number of optimal subbands, and each branch has the same structure, consisting of two convolutional layers, a max-pooling layer and a connection layer, respectively. The n most representative frequency bands in the data representation are sequentially input to the n branches for feature extraction, and then flattened to the connection layer to obtain the output category.

与现有技术相比，本发明具有以下优点：Compared with the prior art, the present invention has the following advantages:

(1)本发明应用了少量中心偶极子反映整个脑皮层的特征信息，避免了使用全部偶极子带来的特征冗余，以及ROI选择带来的大量信息丢失。(1) The present invention uses a small number of central dipoles to reflect the feature information of the entire cerebral cortex, avoiding feature redundancy caused by using all dipoles and a large amount of information loss caused by ROI selection.

(2)本发明将最优子带建立的SDDMs进行融合构建数据表达，体现了不同频带中偶极子在三维空间位置下，幅值随着时间的变化。(2) The present invention fuses the SDDMs established by the optimal sub-band to construct a data representation, which reflects the variation of the amplitude with time of the dipoles in different frequency bands in the three-dimensional space position.

(3)本发明根据多频带融合的数据表达特性设计了n分支并行的nB3DCNN，可以充分地对其中的时间、频率，和空间特征进行提取与识别，有效提高了脑电信号的识别效果。(3) The present invention designs n-branch parallel nB3DCNN according to the data expression characteristics of multi-band fusion, which can fully extract and identify the time, frequency, and spatial features therein, and effectively improve the recognition effect of EEG signals.

附图说明Description of drawings

图1为本发明的方法流程图。FIG. 1 is a flow chart of the method of the present invention.

图2为nB3DCNN网络结构图。Figure 2 is the network structure diagram of nB3DCNN.

图3为本发明的算法流程图。FIG. 3 is an algorithm flow chart of the present invention.

具体实施方式Detailed ways

本发明具体是在Windows 10(64位)操作系统下的MatlabR2014仿真环境和Tensorflow环境下进行。The present invention is specifically carried out under the MatlabR2014 simulation environment and the Tensorflow environment under the Windows 10 (64-bit) operating system.

本发明使用的数据集为BCI2000，由国际10-10标准的64导电极帽进行采集，采样频率为160Hz。该数据集记录了109名受试者的MI-EEG信号，每个受试者均进行四类运动想象任务：左拳，右拳，双拳和双脚。实验共计5s，分别为1s的静息态和4s的运动想象期。本发明随机选取10个受试者进行实验分析(S5，S6，…，S14)，其中每名受试者均进行84次实验，每类实验为21次。在本方法的消融实验中，10个受试者的样本将作为一个整体进行分析。此外，该方法还在全部109个受试者上进行了测试，以避免随机挑选受试者所带来的偶然性。所有实验均在十折交叉验证的标准下进行的。The data set used in the present invention is BCI2000, which is collected by the international 10-10 standard 64-lead electrode cap, and the sampling frequency is 160 Hz. The dataset recorded MI-EEG signals from 109 subjects, each of whom performed four categories of motor imagery tasks: left fist, right fist, double fist, and double foot. The experiment lasted for a total of 5 s, which were 1 s in the resting state and 4 s in the motor imagery period. In the present invention, 10 subjects are randomly selected for experimental analysis (S5, S6, . In the ablation experiment of this method, samples from 10 subjects will be analyzed as a whole. In addition, the method was tested on all 109 subjects to avoid random chance. All experiments were performed under the criterion of ten-fold cross-validation.

基于上述运动想象脑电数据集，本发明的方法流程图如图1所示，本发明具体实施步骤如下：Based on the above motor imagery EEG data set, the flow chart of the method of the present invention is shown in FIG. 1 , and the specific implementation steps of the present invention are as follows:

Step1基于能量的频带优选。Step 1 Band optimization based on energy.

Step1.1原始多通道MI-EEG表示为E_r∈R^64×801(64导联，801个采样时刻)，分别使用24个宽度为1，且能够覆盖与运动想象活动最相关频带的带通滤波器(8-9Hz,9-10Hz,…,and 31-32Hz)。对所有次实验的平均MI-EEG信号

进行滤波，以获得24个子频带EEG信号，记为E_s，其中s∈{1,2,…,24}代表所有子频带的序号。Step1.1 The original multi-channel MI-EEG is represented as E_r ∈ R^64×801 (64 leads, 801 sampling moments), using 24 bandpasses with a width of 1 and covering the frequency band most relevant to motor imagery activities. Filters (8-9Hz, 9-10Hz, …, and 31-32Hz). Average MI-EEG signal for all experiments

Filtering is performed to obtain 24 subband EEG signals, denoted Es , where s∈{1,2,_... ,24} represents the sequence numbers of all subbands.

Step1.2计算在运动准备期间所有导联的能量和

代表采样时刻，c∈[1,64]为导联。将所有的子频带信号按照能量和进行降序排列，把前6个子频带作为最优频带，记为top b_i，i∈{1,2,…,6}。Step1.2 Calculate the energy sum of all leads during exercise preparation

represents the sampling moment, and c∈[1,64] is the lead. Arrange all sub-band signals in descending order of energy sum, and take the first 6 sub-bands as optimal frequency bands, denoted as top b_i , i∈{1,2,...,6}.

Step2简化分布式偶极子模型的构建。Step2 simplifies the construction of the distributed dipole model.

Step2.1头皮层到脑皮层的数据转换。使用sLORETA算法对每一个选取的最优子频带top b_i均进行脑电逆变换，得到所有偶极子的时间序列，可以表示为：Step2.1 Data conversion from scalp to cerebral cortex. Using the sLORETA algorithm to perform inverse EEG transformation on each selected optimal subband top b_i to obtain the time series of all dipoles, which can be expressed as:

其中，

代表在第t个采样时刻时，第k个偶极子所代表的幅值。in,

Represents the amplitude represented by the k-th dipole at the t-th sampling time.

Step2.2中心偶极子序列的选择。基于D-K分区，将大脑分为了68个神经解剖区，为每个区域选择最具有代表性的中心偶极子。这样所有偶极子就可以表示为：Step2.2 Selection of central dipole sequence. Based on the D-K partition, the brain was divided into 68 neuroanatomical regions, and the most representative central dipole was selected for each region. Then all dipoles can be expressed as:

代表在第j个区域中偶极子的时间序列，N_j表示第j个区域中偶极子的个数，且j∈{1,2,…,68}。

represents the time series of dipoles in the jth region,_Nj represents the number of dipoles in the jth region, and j∈{1,2,…,68}.

然后，将每一个区域近似看作一个立方体，选择最靠近该立方体中心的偶极子作为最相关的偶极子，称为中心偶极子

大脑皮层中所有偶极子就可以近似地用中心偶极子等效为：Then, each region is approximated as a cube, and the dipole closest to the center of the cube is chosen as the most relevant dipole, called the central dipole

All dipoles in the cerebral cortex can be approximately equivalent to the central dipole as:

x_j,y_j,z_j)代表了第j个区域中心偶极子的三维坐标，

且

分别代表了x坐标轴(y坐标轴，z坐标轴)的最大值和最小值。x_j , y_j , z_j ) represent the three-dimensional coordinates of the center dipole of the jth region,

and

They represent the maximum and minimum values of the x-axis (y-axis, z-axis), respectively.

Step2.3简化分布式偶极子模型的构建。构建简化分布式偶极子模型是为了在固定空间位置下，反映中心偶极子随着时间的变化。首先，对各区中心偶极子的3D坐标(x_j,y_j,z_j)进行平移、放大和取整等操作，在不改变所有中心偶极子相对空间位置的情况下，获得其正整数3D坐标(x_j',y_j',z_j')。然后，就得到了大脑皮质68个带有3D坐标的中心偶极子，将其称为简化分布式偶极子模型，记为SDDM，由下式表示：Step2.3 Simplify the construction of the distributed dipole model. The simplified distributed dipole model is constructed to reflect the change of the central dipole with time under a fixed spatial position. First, perform translation, enlargement and rounding operations on the 3D coordinates (x_j , y_j , z_j ) of the central dipoles in each area, and obtain their positive integers without changing the relative spatial positions of all central dipoles 3D coordinates (x_j ',y_j ',z_j '). Then, 68 central dipoles with 3D coordinates in the cerebral cortex are obtained, which are called the simplified distributed dipole model, denoted as SDDM, and expressed by the following formula:

Step3基于多频带融合的数据表达与识别。Step3 is based on data expression and recognition of multi-band fusion.

Step3.1单频带的4D幅值矩阵构建。对于每一个挑选出的子频带top b_i,i∈{1,2,…,6}，将任意时刻的中心偶极子幅值赋至其对应的3D空间位置坐标下，形成3D幅值矩阵；接着，沿着采样时刻将3D幅值矩阵进行堆叠，生成4D幅值矩阵，记为下式：Step3.1 Construction of 4D amplitude matrix of single frequency band. For each selected subband top b_i ,i∈{1,2,…,6}, assign the central dipole amplitude at any time to its corresponding 3D spatial position coordinates to form a 3D amplitude matrix ; Next, stack the 3D amplitude matrix along the sampling time to generate a 4D amplitude matrix, which is recorded as the following formula:

D_mi∈R^{13×18×11×12} (5)D_mi ∈ R^{13×18×11×12} (5)

该4D幅值矩阵体现了所有中心偶极子在固定3D空间位置(13×18×11)下，幅值随着时间的变化(12为采样点个数)。The 4D amplitude matrix reflects the change of amplitude over time (12 is the number of sampling points) for all central dipoles in a fixed 3D spatial position (13×18×11).

Step3.2多频带融合的数据表达。将6个最优子频带的4D幅值矩阵D_mi进行融合，构建多频带融合的数据表达，记为下式：Step3.2 Data expression of multi-band fusion. The 4D amplitude matrix D_mi of the 6 optimal sub-bands is fused to construct the data representation of multi-band fusion, which is recorded as the following formula:

它集成了脑源域中所有中心偶极子的时域、频域和空域信息，反映了在三维源空间中最佳子频带的分布式神经动力学变化。It integrates time-, frequency-, and spatial-domain information of all central dipoles in the brain source domain, reflecting the distributed neural dynamics of the optimal subbands in the three-dimensional source space.

Step3.3并行多分支3DCNN的设计。根据多频带融合数据表达D_R的特点，设计了6分支并行的3DCNN(6B3DCNN)用于特征提取和识别。如图2所示，其中每个分支具有相同的结构，分别由两个卷积层，一个最大池化层和一个连接层组成。数据表达中6个最具有代表性的频带，依次输入至6个分支中进行特征提取，然后展平至连接层得到输出类别。Step3.3 Design of parallel multi-branch 3DCNN. According to the characteristics of multi-band fusion data expressing DR, a 6-branch parallel 3DCNN (_6B3DCNN ) is designed for feature extraction and recognition. As shown in Figure 2, where each branch has the same structure, it consists of two convolutional layers, a max-pooling layer and a connection layer, respectively. The 6 most representative frequency bands in the data representation are sequentially input to the 6 branches for feature extraction, and then flattened to the connection layer to obtain the output category.

对受试者S5，S6，…，S14和混合十个受试者(10S)，以及全部受试者(All)的MI-EEG数据进行上述过程处理，得到十折交叉验证平均识别正确率如表1所示。The above process is performed on the MI-EEG data of subjects S5, S6, ..., S14 and mixed ten subjects (10S), and all subjects (All), and the average recognition accuracy of ten-fold cross-validation is obtained as follows: shown in Table 1.

表1十折交叉验证正确率Table 1 The correct rate of ten-fold cross-validation

Claims (1)

1. The simplified distributed dipole model establishing and identifying method based on the D-K subarea is characterized by comprising the following steps of:

firstly, filtering original motor imagery electroencephalogram signals MI-EEG of all channels by using different band-pass filters to select an optimal frequency band related to motor imagery activities; then, performing inverse electroencephalogram transformation on each selected sub-band signal, thereby converting the scalp EEG data into dipoles in cerebral cortex space; then, central dipoles based on the neural anatomy Desikan-Killiany partition are obtained to construct a simplified distributed dipole model SDDM, and the activity of the central dipoles of the cerebral cortex is regarded as a neurodynamic system; assigning the central dipole amplitude of each sampling moment to the 3D space coordinate of the central dipole to form a 3D dipole amplitude matrix, and stacking the 3D dipole amplitude matrix into 4D data expression according to the time dimension; finally, the multi-band data expression is fused and input into a designed n-branch parallel three-dimensional convolutional neural network nB3DCNN, and comprehensive feature extraction and identification are carried out from three dimensions of time, frequency and space; the method is characterized in that:

step1 band preference based on energy;

step1.1 assumes the original multichannel MI-EEG as E_r∈R^N×TWhere N represents the number of scalp electrodes and T represents the number of sampling instants; average MI-EEG signals for all experiments using 24 band-pass filters (8-9Hz,9-10Hz, …, and 31-32Hz) of the same width and capable of covering the frequency band most relevant to motor imagery activity, respectively

To carry outFiltering to obtain 24 sub-band EEG signals denoted E_sWhere s e {1,2, …,24} represents the sequence numbers of all sub-bands;

step1.2 calculation during exercise preparation

For the starting point of the preparation period of the exercise,

for the end of the exercise preparation period, t is the sampling time, all leads (c e [1, N)]And c is the sum of the energies of the electrodes), expressed as:

then, all the sub-band signals are arranged in descending order according to the energy sum, and the first n sub-bands are taken as the optimal bands and are marked as top b_i，i∈{1,2,…,n}；

Step2 simplifies the construction of the distributed dipole model;

step2.1 data conversion from the scalp to the cerebral cortex; for each selected optimal sub-band top b_iPerforming inverse brain-electrical transformation to obtain time sequences of all dipoles, expressed as:

wherein,

representing the amplitude represented by the kth dipole at the tth sampling instant;

selecting a Step2.2 central dipole sequence; dividing the brain into 68 neuroanatomical regions based on D-K partition, and selecting the most representative central dipole for each region; thus all dipoles can be represented as:

representing the time sequence of the dipoles in the j-th region, N_jRepresents the number of dipoles in the jth region, and j ∈ {1,2, …,68 };

then, each region is approximately treated as a cube, and the dipole closest to the center of the cube is selected as the most relevant dipole, called the center dipole

All dipoles in the cerebral cortex can be approximately equated with a central dipole as:

x_j,y_j,z_j) Representing the three-dimensional coordinates of the j-th regional center dipole,

and is

Respectively representing the maximum value and the minimum value of an x coordinate axis (a y coordinate axis and a z coordinate axis);

constructing a simplified distributed dipole model by Step2.3; the simplified distributed dipole model is constructed to reflect the change of the central dipole along with time at a fixed spatial position; first, the 3D coordinates (x) of the center dipole for each region_j,y_j,z_j) The operations of translation, amplification, rounding and the like are carried out, under the condition of not changing the relative spatial positions of all the central dipoles,obtain its positive integer 3D coordinate (x)_j',y_j',z_j') to a host; then, a central dipole of cerebral cortex 68 with 3D coordinates is obtained, which is called the simplified distributed dipole model, denoted SDDM, and represented by the following equation:

step3 expresses and identifies data based on multi-band fusion;

constructing a 4D amplitude matrix of a Step3.1 single frequency band; for each selected sub-band top b_iI belongs to {1,2, …, n }, and the central dipole amplitude at any moment is assigned to the corresponding 3D space position coordinate to form a 3D amplitude matrix; then, stacking the 3D amplitude matrices along the sampling time to generate a 4D amplitude matrix, which is expressed as the following formula:

wherein x is_m＝max(x'₁,x'₂,…,x'₆₈),y_m＝max(y'₁,y'₂,…,y'₆₈),z_m＝max(z'₁,z'₂,…,z'₆₈) (ii) a The current 4D amplitude matrix comprises information of amplitude changes of all central dipoles in a fixed 3D space position along with time;

data expression of Step3.2 multiband fusion; based on 4D amplitude matrix D_miAnd an optimal sub-band top b_iConstructing a data expression of multi-band fusion, and displaying the composite characteristics of the motor imagery from three dimensions of time, frequency and space; d of the first n sub-bands with the highest energy_miFusion was performed to construct a novel data expression, which is represented by the following formula:

the novel data expression formula integrates time domain, frequency domain and space domain information of all central dipoles in a brain source domain, and reflects distributed neurodynamic change of an optimal sub-band in a three-dimensional source space;

designing Step3.3 parallel multi-branch 3 DCNN; expressing D from multi-band fused data_RThe method is characterized in that n-branch parallel 3DCNN is designed for feature extraction and identification; the number n of branches depends on the number of the optimal sub-bands, and each branch has the same structure and consists of two convolution layers, a maximum pooling layer and a connecting layer; and sequentially inputting n most representative frequency bands in the data expression into n branches for feature extraction, and then flattening the frequency bands to a connecting layer to obtain an output class.

Images