CN102499676B - EEG signal classification system and method based on efficient time series and electrode reorganization - Google Patents

Abstract

The invention discloses an effective time sequence and electrode recombination based electroencephalograph signal categorizing system and a method, which can realize the identification to three types of character expression (happiness, normality and sadness) by acquiring and analyzing the electroencephalograph signals of the human brain, and mainly comprises an electroencephalograph signal acquisition process and an electroencephalograph signal analyzing process. According to the invention, the electroencephalograph signals are acquired by stimulating different expression of a testee; the feature space of the effective electroencephalograph signals is determined by the strong energy distribution of the full field of the electroencephalograph signals; then, the PCA (Principal Component Analysis) dimension reduction is carried out to the original electroencephalograph signals corresponding to the feature space, and the electroencephalograph signals with categorizing advantages are reconstructed; and finally, a linear discriminant function categorizer is selected to for categorizing. According to the invention, during expression identification, only target features are extracted from the acquired electroencephalograph signals, then, the acquired electroencephalograph signals are categorized, and the identification result can be determined; and the identification of the electroencephalograph signals can be realized based on the character expression stimulation. In the invention, the cognition of the human being is introduced, and the advantages of objectiveness and high efficiency are provided.

Description

Translated fromChinese

基于有效时间序列和电极重组的脑电信号分类系统和方法EEG signal classification system and method based on efficient time series and electrode reorganization

技术领域technical field

本发明涉及一种脑电信号的处理和分析方法，特别是涉及一种基于有效时间序列和电极重组的脑电信号分类系统和方法。The invention relates to a method for processing and analyzing electroencephalogram signals, in particular to an electroencephalogram signal classification system and method based on effective time series and electrode reorganization.

背景技术Background technique

在当今这个人际关系日益紧密的社会，正确地识别他人的表情有重要的生存意义。这不仅可以使人们及时调节自己的行为来适应环境，而且还能有效地避免不必要的危险，有利于社会交往和环境适应。同时，对正常人的研究也可为临床诊断和治疗提供参考，用于预防和治疗工作。目前，人脸表情识别技术主要的应用领域包括人机交互、安全、机器人制造、医疗、通信和汽车领域等。In today's society with increasingly close interpersonal relationships, it is of great survival significance to correctly recognize other people's expressions. This not only enables people to adjust their behavior in time to adapt to the environment, but also effectively avoids unnecessary dangers and is conducive to social interaction and environmental adaptation. At the same time, research on normal people can also provide references for clinical diagnosis and treatment, and can be used for prevention and treatment. At present, the main application fields of facial expression recognition technology include human-computer interaction, security, robot manufacturing, medical treatment, communication and automobile fields.

在有关表情识别的文献中，主要通过图像表情识别和语音信号分析来判断表情，但这些表情评估的传统方法具有主观性，很容易被他人所否认。然而另一种可用的表情识别办法是生理脑电信号分析，它是一个更直观、有效的表情识别手段，因为表情状态本来就是由神经系统的活动来反映的。In the literature on expression recognition, expressions are mainly judged by image expression recognition and speech signal analysis, but these traditional methods of expression evaluation are subjective and easily denied by others. However, another available expression recognition method is physiological EEG signal analysis, which is a more intuitive and effective means of expression recognition, because the expression state is originally reflected by the activity of the nervous system.

发明内容Contents of the invention

本发明的目的在于针对表情刺激产生的脑电信号，提出一种基于有效时间序列和电极重组的脑电信号分类系统和方法。以避免特征的主观性，通过采用并行计算策略提高执行效率。The object of the present invention is to propose an EEG signal classification system and method based on effective time series and electrode reorganization for EEG signals generated by expression stimulation. In order to avoid the subjectivity of features, the execution efficiency is improved by adopting a parallel computing strategy.

本发明是采用以下技术手段实现的：The present invention is realized by adopting the following technical means:

一种基于有效时间序列和电极重组的脑电信号分类系统，包括：脑电信号采集模块、脑电信号预处理模块、脑电信号特征选择模块、脑电信号表情分类实施模块。An EEG signal classification system based on effective time series and electrode reorganization, including: EEG signal acquisition module, EEG signal preprocessing module, EEG signal feature selection module, EEG signal expression classification implementation module.

脑电信号信息采集模块，采集被试在高兴、中性和悲伤的不同表情刺激下的原始脑电信号，并将采集到的脑电信号传递给脑电信号预处理模块；脑电信号预处理模块将采集到的原始脑电信号进行去噪(噪声包括水平眼电和垂直眼电)，之后将纯净的脑电信号转换成全局场强送入脑电特征选择模块；脑电信号特征选择模块，通过全局场强脑电信号的峰值特征确定脑电信号的有效时间区域，在对有效时间区域上的脑电信号进行电极重组，将重组后的脑电信号降维后视为脑电信号表情识别的最终特征，并将此特征传送到脑电信号表情分类实施模块；脑电信号表情分类实施模块用经典分类算法(Fisher分类器)进行脑电信号分类。The EEG signal information acquisition module collects the original EEG signals of the subjects under the stimulation of different expressions of happiness, neutrality and sadness, and transmits the collected EEG signals to the EEG signal preprocessing module; EEG signal preprocessing The module denoises the collected original EEG signals (noise includes horizontal and vertical EEG signals), and then converts the pure EEG signals into global field strength and sends them to the EEG feature selection module; the EEG signal feature selection module , the effective time area of the EEG signal is determined by the peak characteristics of the global field strength EEG signal, and the electrode reorganization is performed on the EEG signal in the effective time area, and the reorganized EEG signal is regarded as the expression of the EEG signal after dimensionality reduction Identify the final feature, and send this feature to the EEG signal expression classification implementation module; the EEG expression classification implementation module uses a classic classification algorithm (Fisher classifier) to perform EEG signal classification.

一种基于有效时间序列和电极重组的脑电信号分类方法，包括以下步骤：A method for classifying EEG signals based on effective time series and electrode reorganization, comprising the following steps:

步骤1，受试者带上电极帽，原始脑电信号是通过66导国际脑电图学会标定的10/20法的EEG放大器进行采集，并选取所有电极位置，采集不同表情刺激过程的受试者脑电信号；Step 1. The subject puts on the electrode cap. The original EEG signal is collected through the 66-lead EEG amplifier of the 10/20 method calibrated by the International Electroencephalography Society. EEG signal;

步骤2，将采集到的脑电信号输入到预处理模块，预处理模块主要对采集到的脑电信号进行去噪，并得到脑电信号的全场强，脑电信号的全场强即为所有电极信号的叠加平均值；Step 2: Input the collected EEG signals into the preprocessing module. The preprocessing module mainly denoises the collected EEG signals and obtains the full field strength of the EEG signals. The full field strength of the EEG signals is Superimposed average of all electrode signals;

步骤3，通过特征提取模块对脑电信号全场强的分析，确定脑电信号特征选择的有效时间区域；Step 3, through the analysis of the full field strength of the EEG signal by the feature extraction module, determine the effective time zone for EEG signal feature selection;

步骤4，由于不同电极产生的脑电信号具有不同的生理学意义，因此通过特征提取模块对特征选择的脑电信号进行电极重组；Step 4, since the EEG signals generated by different electrodes have different physiological meanings, the EEG signals of feature selection are reorganized through the feature extraction module;

步骤5，为了降低脑电信号的冗余信息，对步骤4所得到的重组后的脑电信号通过主成分分析(PCA)方法进行降维；Step 5, in order to reduce the redundant information of the EEG signal, perform dimensionality reduction on the reorganized EEG signal obtained inStep 4 by a principal component analysis (PCA) method;

步骤6，对特征提取模块提取后的脑电信号使用脑电信号表情分类实施模块中的线性判别函数分类器(Fisher)进行分类学习与测试；Step 6, use the linear discriminant function classifier (Fisher) in the EEG signal expression classification implementation module to carry out classification learning and testing for the EEG signals extracted by the feature extraction module;

测试表情识别时，通过脑电信号采集模块，采集待测被试的脑电信号，将脑电信号送入脑电信号预处理模块，去除噪声后，再根据脑电信号特征提取模块计算生成被试对应的特征向量，然后将这一特征向量送入脑电信号表情分类实施模块，最后得到表情刺激的脑电信号分类结果。When testing expression recognition, collect the EEG signals of the subject to be tested through the EEG signal acquisition module, send the EEG signals to the EEG signal preprocessing module, remove the noise, and then calculate and generate the EEG signals according to the EEG signal feature extraction module. Test the corresponding eigenvector, and then send this eigenvector to the EEG signal expression classification implementation module, and finally obtain the EEG signal classification result of the expression stimulation.

脑电信号的有效时间区域的选择，根据全局场强的峰值和高能量值来确定的有效时间区域；电极重组的过程是在有效时间区域的基础上，对不同电极进行重新组合的过程；脑电信号表情分类的过程是在并行的基础上，对选择的脑电信号特征进行分类。The selection of the effective time area of the EEG signal is the effective time area determined according to the peak value of the global field strength and the high energy value; the process of electrode reorganization is the process of recombining different electrodes on the basis of the effective time area; The process of electrical signal expression classification is to classify selected EEG signal features on a parallel basis.

本发明一种基于有效时间序列和电极重组的脑电信号分类系统和方法，与现有技术相比具有以下优点：An EEG signal classification system and method based on effective time series and electrode reorganization of the present invention has the following advantages compared with the prior art:

1、与传统方法相比，本发明利用生理脑电信号，避免了特征的主观性。1. Compared with traditional methods, the present invention utilizes physiological EEG signals, avoiding the subjectivity of features.

2、本发明在步骤(3)中根据脑电信号的全场强进行特征选择是一种合理且有效的新方法。2. In the present invention, it is a reasonable and effective new method to perform feature selection according to the full field strength of the EEG signal in step (3).

3、本发明在步骤(5)中所使用的主成分分析方法是统计学习中的经典方法，在许多数值计算平台中能够找到比较成熟的实现算法。3. The principal component analysis method used in step (5) of the present invention is a classic method in statistical learning, and relatively mature implementation algorithms can be found in many numerical computing platforms.

4、本发明的主要计算量集中在步骤(6)，由于在步骤(4)会产生多种电极组合，因此步骤(6)要对每种组合下的脑电特征进行分类器训练和评价，因此可以采用并行计算策略来提高执行效率。4. The main calculation amount of the present invention is concentrated in step (6). Since multiple electrode combinations can be produced in step (4), step (6) will carry out classifier training and evaluation to the EEG features under each combination. Therefore, parallel computing strategies can be used to improve execution efficiency.

附图说明Description of drawings

图1是本发明所涉及方法全过程的流程图与系统模块划分情况；Fig. 1 is the flow chart of the whole process of the method involved in the present invention and the system module division situation;

图2是本发明所涉及采集脑电信号的实验设计流程图；Fig. 2 is the experimental design flow chart of collecting EEG signal involved in the present invention;

图3是本发明所涉及基于全场强的脑电信号图；Fig. 3 is the EEG signal diagram based on the full field strength involved in the present invention;

图4是本发明图1中“脑电信号处理”部分的具体流程。Fig. 4 is the specific flow of the "EEG signal processing" part in Fig. 1 of the present invention.

具体实施方式Detailed ways

下面结合具体实施方式对本发明做进一步的说明。The present invention will be further described below in combination with specific embodiments.

本发明在训练表情识别分类器时的步骤有如下6个步骤：The present invention has following 6 steps in the step when training facial expression recognition classifier:

首先在步骤1中根据设计好的实验进行脑电信号的采集，在试验采集的过程中，选用三类人脸表情作为刺激图片，包括高兴表情、中性表情和悲伤表情，每个表情有18种脸型，每个被试者执行408次试验，三类任务各占136次。每次试验过程如下：首先给被试者显示提示语，待被试者按下空格键后，显示一个正向或倒向的，表情为高兴、中性、悲伤三种表情之一的图片，待被试者对表情进行清楚辨别并按下相应的按键后，表示一次试验结束，具体过程如图2所示。被试数据采集自12名年龄在20-30岁的健康人。Firstly, instep 1, the EEG signal was collected according to the designed experiment. In the process of experiment collection, three types of facial expressions were selected as stimulus pictures, including happy expressions, neutral expressions and sad expressions. Each expression had 18 Each subject performed 408 trials, and each of the three types of tasks accounted for 136 trials. The process of each test is as follows: firstly, a prompt is displayed to the subject, and after the subject presses the space bar, a forward or reverse picture with one of the three expressions of happy, neutral, and sad is displayed. After the subjects clearly identified the facial expressions and pressed the corresponding button, it means that a test is over. The specific process is shown in Figure 2. The test data were collected from 12 healthy people aged 20-30.

接下来对步骤1采集到的原始脑电信号进行预处理，预处理模块包括2个步骤：Next, preprocess the original EEG signals collected instep 1. The preprocessing module includes 2 steps:

步骤2.1由于脑电信号微弱，极易受到眼电信号的影响。因此，去除脑电信号中的噪声就显得尤为重要，本发明方法中利用NeuroScan软件对采集到的脑电信号进行去噪。Step 2.1 Because the EEG signal is weak, it is easily affected by the oculoelectric signal. Therefore, it is particularly important to remove the noise in the EEG signal. In the method of the present invention, NeuroScan software is used to denoise the collected EEG signal.

步骤2.2在步骤2.1获得清洁的脑电信号EEG的基础上，通过NeuroScan软件获得66导电极原始脑电信号对应的全场强，即GFP，它是通过对各个电极的信号进行叠加平均得到的。Step 2.2 On the basis of the clean EEG signal obtained in step 2.1, the full field strength corresponding to the original EEG signal of 66 conductive electrodes, that is, GFP, is obtained by NeuroScan software, which is obtained by superimposing and averaging the signals of each electrode.

接下来步骤3根据特征提取模块对三类原始脑电信号的全场强(GFP)进行对比分析，如图3所示，结果发现全场强的峰值分布在88ms、154ms、232ms处，而350ms-650ms处为慢电位反映。因此，我们根据全场强(GFP)能量的时间分布(70-110ms，125ms-185ms，200ms-250ms，350ms-650ms)来确定原始脑电信号(EEG)特征选择的有效时间段(即有效时间区域)。Thenext step 3 is to compare and analyze the full field strength (GFP) of the three types of original EEG signals according to the feature extraction module. -650ms is the slow potential response. Therefore, we determine the effective time period (i.e., effective time area).

步骤4中，特征提取模块中我们采用启发式研究方法，启发函数是线性判别函数分类器的性能，即我们以性能优劣来分析重组的有效性。首先通过电极重组对脑电信号特征选择做进一步提取，即在步骤3所确定的有效特征区域进行重组，重组的过程是采用穷举的方式按照不同时间区域和不同电极进行排列组合。公式如下：Instep 4, we use the heuristic research method in the feature extraction module. The heuristic function is the performance of the linear discriminant function classifier, that is, we use the performance to analyze the effectiveness of the recombination. First, further extract the EEG signal feature selection through electrode reorganization, that is, reorganize the effective feature area determined instep 3. The reorganization process is to arrange and combine according to different time regions and different electrodes in an exhaustive manner. The formula is as follows:

$\mathrm{<span><span>X</span>x</span>}<span><span>=</span>=</span>\underset{\mathrm{<span><span>i</span>i</span>}}{\mathrm{<span><span>\&Sigma;</span>\&Sigma;</span>}}\underset{\mathrm{<span><span>j</span>j</span>}}{\mathrm{<span><span>\&Sigma;</span>\&Sigma;</span>}}{\mathrm{<span><span>\&alpha;</span>\&alpha;</span>}}_{\mathrm{<span><span>i</span>i</span>}}{\mathrm{<span><span>\&beta;</span>\&beta;</span>}}_{\mathrm{<span><span>j</span>j</span>}}{\mathrm{<span><span>E</span>E.</span>}}_{\mathrm{<span><span>ij</span>ij</span>}}<span><span>-</span>-</span><span><span>-</span>-</span><span><span>-</span>-</span><span><span>(</span>(</span>\mathrm{<span><span>1</span>1</span>}<span><span>)</span>)</span>$

i∈{1，…，64}，j∈{1，…，5}，i∈{1,...,64}, j∈{1,...,5},

其中，i是电极个数，j是时间序列段个数，E_ij指的是第i个电极第j个时间段的脑电信号，当α_i＝1时，表示第i个单电极的脑电信号被作为特征，当β_j＝1时，表示第j个时间区域的脑电信号被作为特征。Among them, i is the number of electrodes, j is the number of time series segments, E_ij refers to the EEG signal of the i-th electrode in the j-th time period, when α_i =1, it means the brain signal of the i-th single electrode The electrical signal is used as a feature, and when β_j =1, the EEG signal representing the jth time zone is used as a feature.

接下来步骤5对步骤4中各种的电极组合进行主成分分析方法降维。因为当脑电信号特征选择进行电极重组后，都会产生高维的脑电信号，从而影响了脑电信号的分类效率和结果，所以脑电信号的降维过程显得尤为重要。本专利把脑电信号特征降到400维。Next, in step 5, the principal component analysis method is used to reduce the dimensionality of various electrode combinations instep 4. Because when the EEG signal features are selected and the electrodes are reorganized, high-dimensional EEG signals will be generated, which will affect the classification efficiency and results of EEG signals, so the dimensionality reduction process of EEG signals is particularly important. This patent reduces the EEG signal features to 400 dimensions.

之后步骤6对降维后的脑电信号使用脑电信号表情分类实施模块中的线性判别函数分类器进行分类，由于步骤4中脑电信号重组的过程选用穷举方式，经验可知，穷举方法带来了计算空间大，复杂度高的问题，因此，在这里我们为解决这个问题引入并行计算，并行主要针对分类过程，通过并行运算从而大大提高了分类时间和速率。如图4，详细显示步骤2-6的具体流程。Then step 6 uses the linear discriminant function classifier in the EEG signal expression classification implementation module to classify the dimensionally reduced EEG signals. Since the EEG signal recombination process instep 4 uses the exhaustive method, it is known from experience that the exhaustive method It brings the problem of large computing space and high complexity. Therefore, here we introduce parallel computing to solve this problem. Parallel is mainly aimed at the classification process. Through parallel computing, the classification time and speed are greatly improved. As shown in Figure 4, the specific process of steps 2-6 is shown in detail.

本发明在测试表情识别时的步骤如下：The steps of the present invention when testing expression recognition are as follows:

通过脑电信号采集模块，采集待测被试的脑电信号(方法与上述对应步骤一致)，将脑电信号送入脑电信号预处理模块，去除噪声后，再根据脑电信号特征提取模块计算生成被试对应的特征向量(方法与上述对应步骤一致)，然后将这一特征向量送入脑电信号表情分类实施模块，最后生成基于表情刺激的分类结果。结果表明，最高识别率超过了90％，平均识别率在85％左右，能够实现对表情刺激的脑电信号识别。Through the EEG signal acquisition module, collect the EEG signal of the subject to be tested (the method is consistent with the corresponding steps above), send the EEG signal to the EEG signal preprocessing module, remove the noise, and then extract the EEG signal according to the EEG signal feature extraction module Calculate and generate the eigenvector corresponding to the subject (the method is consistent with the corresponding steps above), and then send this eigenvector to the EEG expression classification implementation module, and finally generate the classification result based on the expression stimulus. The results show that the highest recognition rate exceeds 90%, and the average recognition rate is about 85%, which can realize the recognition of EEG signals for expression stimuli.

Claims (5)

Translated fromChinese

1.一种基于有效时间序列和电极重组的脑电信号分类系统，包括：脑电信号采集模块、脑电信号预处理模块、脑电信号特征提取模块、脑电信号表情分类实施模块；其特征在于：所述的脑电信号采集模块，主要用来采集不同表情刺激下的原始脑电信号，并将采集到的原始脑电信号传递给脑电信号预处理模块；1. An EEG signal classification system based on effective time series and electrode reorganization, comprising: an EEG signal acquisition module, an EEG signal preprocessing module, an EEG signal feature extraction module, and an EEG signal expression classification implementation module; its characteristics In that: the EEG signal collection module is mainly used to collect the original EEG signals stimulated by different expressions, and transmit the collected original EEG signals to the EEG signal preprocessing module;所述的脑电信号预处理模块将采集到的原始脑电信号进行去噪，之后将纯净的脑电信号转换成全局场强送入特征提取模块；The EEG signal preprocessing module denoises the collected original EEG signal, and then converts the pure EEG signal into a global field strength and sends it to the feature extraction module;所述的全局场强即为所有电极脑电信号的叠加平均值；The global field strength is the superimposed average value of all electrode EEG signals;所述的脑电信号特征提取模块，通过脑电信号全局场强的峰值和能量的时间分布来确定脑电信号的有效时间区域，再对有效时间区域上的脑电信号进行电极重组，重组的过程是采用穷举的方式按照不同时间区域和不同电极进行排列组合，将重组后的脑电信号视为脑电信号表情识别的特征，并将特征传送到脑电信号表情分类实施模块；The EEG signal feature extraction module determines the effective time area of the EEG signal through the peak value of the global field strength of the EEG signal and the time distribution of energy, and then performs electrode reorganization on the EEG signal in the effective time area, and the reorganized The process is to arrange and combine according to different time regions and different electrodes in an exhaustive manner, regard the reorganized EEG signals as the features of EEG expression recognition, and transmit the features to the EEG expression classification implementation module;所述的脑电信号表情分类实施模块用经典分类算法进行脑电信号分类。The EEG expression classification implementing module uses classical classification algorithms to classify EEG signals.2.根据权利要求1所述的基于有效时间序列和电极重组的脑电信号分类系统，其特征在于：所述的经典分类算法采用Fisher分类器。2. The EEG signal classification system based on effective time series and electrode recombination according to claim 1, characterized in that: the classical classification algorithm adopts Fisher classifier.3.一种基于有效时间序列和电极重组的脑电信号分类方法，其特征在于：包括以下步骤：3. a kind of EEG signal classification method based on effective time series and electrode reorganization, it is characterized in that: comprise the following steps:步骤1，受试者带上电极帽，原始脑电信号是通过放大器进行采集，并选取所有电极位置，采集不同表情刺激过程的受试者脑电信号；Step 1, the subject puts on the electrode cap, the original EEG signal is collected through the amplifier, and all electrode positions are selected to collect the EEG signal of the subject during different expression stimulation processes;步骤2，将经过脑电信号预处理模块处理后的脑电信号输入到脑电信号特征提取模块，预处理模块包括对采集到的脑电信号进行去噪，并得到脑电信号的全局场强；Step 2, input the EEG signal processed by the EEG signal preprocessing module to the EEG signal feature extraction module, the preprocessing module includes denoising the collected EEG signal, and obtains the global field strength of the EEG signal ;步骤3，根据特征提取模块对脑电信号全局场强的分析，确定脑电信号特征提取的有效时间区域；Step 3, according to the analysis of the global field strength of the EEG signal by the feature extraction module, determine the effective time region of the EEG signal feature extraction;步骤4，根据特征提取模块对特征选择的脑电信号进行电极重组；Step 4, performing electrode reorganization on the EEG signal of feature selection according to the feature extraction module;步骤5，对步骤4所得到的重组后的脑电信号通过主成分分析方法进行降维；Step 5, performing dimensionality reduction on the reorganized EEG signal obtained in step 4 by principal component analysis;步骤6，对特征提取模块提取后的脑电信号使用脑电信号表情分类实施模块中的线性判别函数分类器进行分类学习与测试最后得到分类结果。Step 6: Carry out classification learning and testing on the EEG signals extracted by the feature extraction module using the linear discriminant function classifier in the EEG signal expression classification implementation module, and finally obtain the classification results.4.根据权利要求3所述的基于有效时间序列和电极重组的脑电信号分类方法，所述的放大器为66导国际脑电图学会标定的10/20法的EEG放大器。4. The EEG signal classification method based on effective time series and electrode reorganization according to claim 3, wherein the amplifier is a 66-lead EEG amplifier of the 10/20 method calibrated by the International Electroencephalography Society.5.根据权利要求3所述的基于有效时间序列和电极重组的脑电信号分类方法，其特征在于：5. the EEG classification method based on effective time series and electrode reorganization according to claim 3, is characterized in that:所述脑电信号的有效时间区域的选择，是指根据全局场强的峰值和高能量值来确定的有效时间区域；The selection of the effective time region of the EEG signal refers to the effective time region determined according to the peak value of the global field strength and the high energy value;所述电极重组的过程是在有效时间区域的基础上，采用穷举的方式按照不同时间区域和不同电极进行排列组合的过程；The process of electrode reorganization is based on the effective time zone, and adopts an exhaustive method to arrange and combine according to different time zones and different electrodes;所述脑电信号表情分类的过程是在并行的基础上，对选择的脑电信号特征进行分类。The process of classifying the expression of the EEG signal is to classify the selected EEG signal features on a parallel basis.

Images