CN111110230A

Movatterモバイル変換

Info

Publication number: CN111110230A
Application number: CN202010021303.7A
Authority: CN
Inventors: 吴晓光; 何思凡; 谢平; 陈晓玲; 张宁宁; 郝慎才; 李小俚; 江国乾
Original assignee: Yanshan University
Current assignee: Yanshan University
Priority date: 2020-01-09
Filing date: 2020-01-09
Publication date: 2020-05-08

Abstract

Description

Motor imagery electroencephalogram feature enhancement method and system

Technical Field

The invention relates to the field of electroencephalogram signal enhancement, in particular to a motor imagery electroencephalogram feature enhancement method and system.

Background

"Yaolin, stimulation-assisted electroencephalogram signal enhancement method and hybrid brain-computer interface [ D ]. 2015". The article proposes an experimental paradigm based on the multi-modal mixing of selective sensation and imagination movement, and is a no-training method; the exercise illusion is introduced into a proposed stimulation auxiliary paradigm, and the proprioception of virtual exercise is enhanced from the perspective of stimulating muscle vibration to generate the exercise illusion. The article relates to a plurality of experimental paradigm stimulations of motor imagery electroencephalogram signals, but does not relate to an effective action feedback part, and the intuition is not strong.

China, CN201210324584.9[ P ].2013-01-09 said invention provides a brain-computer interface control system and method based on multi-mode fusion, said system includes brain-electrical stimulation and feedback module for inducing SSVEP steady-state visual evoked potential (SSVEP) steady-state vision and MI Motor Imagery (MI). The method combines SSVEP and MI electroencephalogram signals, reduces the low information transmission rate under a single motor imagery mode brain-computer interface, does not relate to a multi-mode neural feedback enhancement paradigm, and has certain one-sidedness.

"Yana. audio-visual evoked brain electrical signal research [ D ]. Shanxi university, 2012" this article has studied the experimental paradigm of audio-visual synchronous evoked as well as the evoked brain electrical signal extraction and classification identification method. But the feedback condition in the paradigm training is not involved, the activation degree of different brain areas under different stimuli is not involved, and the assistance to spontaneous electroencephalogram enhancement is not large.

Disclosure of Invention

The invention aims to provide a motor imagery electroencephalogram characteristic enhancement method and a motor imagery electroencephalogram characteristic enhancement system, and aims to solve the problems that an existing electroencephalogram signal enhancement method is not strong in intuition, is unilateral and low in assistance of enhancing spontaneous electroencephalograms, so that the electroencephalograms are not obviously enhanced, and the identification rate of effective electroencephalograms of a scalp is low.

In order to achieve the purpose, the invention provides the following scheme:

a motor imagery electroencephalogram feature enhancement method, comprising:

acquiring a multi-mode neural feedback training motor imagery electroencephalogram characteristic enhancement paradigm; the multi-mode nerve feedback training motor imagery electroencephalogram characteristic enhancement paradigm is a stimulation mode of multiple stimulation sources;

acquiring multichannel electroencephalogram signals corresponding to a parietal motor cortex and an occipital visual cortex based on the multi-mode neural feedback training motor imagery electroencephalogram characteristic enhancement paradigm;

preprocessing the electroencephalogram signals, and determining the preprocessed electroencephalogram signals;

extracting dynamic and coupling multilevel characteristics of the preprocessed electroencephalogram signal, and determining the dynamic characteristics and the coupling characteristics of the preprocessed electroencephalogram signal; the dynamic characteristics comprise time domain characteristics, frequency domain characteristics and space domain characteristics of the electroencephalogram signals; the coupling characteristic is a mutual information characteristic among a plurality of the preprocessed electroencephalogram signals;

and displaying the characteristic numerical value of the motor imagery electroencephalogram signal before enhancement and the characteristic numerical value of the motor imagery electroencephalogram signal after enhancement according to the dynamic characteristic and the coupling characteristic.

Optionally, the preprocessing is performed on the electroencephalogram signal, and the determining of the preprocessed electroencephalogram signal specifically includes:

eliminating the 50Hz power frequency of the electroencephalogram signal by using a self-adaptive notch filter, and determining the electroencephalogram signal after eliminating the power frequency;

eliminating the baseline drift of the EEG signal subjected to power frequency elimination by using a high-pass filter, and determining the EEG signal subjected to baseline drift elimination;

and performing band-pass filtering processing on the electroencephalogram signal with the baseline drift eliminated, and determining the preprocessed electroencephalogram signal.

Alternatively to this, the first and second parts may,said pair ofThe method comprises the following steps of carrying out dynamic and coupling multilevel feature extraction on the preprocessed electroencephalogram signal, and determining the dynamic feature and the coupling feature of the preprocessed electroencephalogram signal, wherein the method specifically comprises the following steps:

extracting the time domain characteristics of the preprocessed electroencephalogram signals from the time domain by utilizing an autoregressive model (AR) coefficient;

extracting the frequency domain characteristics of the preprocessed electroencephalogram signals from a frequency domain by utilizing a wavelet packet energy spectrum;

extracting the spatial domain characteristics of the preprocessed electroencephalogram signals from a spatial domain by utilizing a common spatial mode CSP analysis algorithm;

and extracting mutual information characteristics of the preprocessed electroencephalogram signals by using a mutual information algorithm.

Optionally, the extracting, from the time domain, the time domain feature of the preprocessed electroencephalogram signal by using the autoregressive model coefficient specifically includes:

using formulas

Extracting the time domain characteristics of the preprocessed electroencephalogram signals from the time domain; wherein x (n) is the nth time domain feature value; w (n) is the input preprocessed brain electrical signal; p is the order of the AR model; a is a matrix containing the main information of the preprocessed electroencephalogram signals; x (n-k) is the k-th number indicated as x (n) preceding; k is 1,2, 3.. p; n is an integer.

Optionally, the extracting, from a frequency domain, the frequency domain feature of the preprocessed electroencephalogram signal by using the wavelet packet energy spectrum specifically includes:

performing wavelet packet transformation processing on the preprocessed electroencephalogram signals, and determining sub-band signals subjected to the wavelet packet transformation processing;

acquiring energy of each sub-band signal;

normalizing the energy of each sub-band to determine the normalized energy of each sub-band signal;

constructing a characteristic vector by taking the proportion of the normalized energy of each sub-band signal in the total energy as a characteristic parameter; the feature vectors are frequency domain features.

Optionally, the extracting, by using a common spatial mode CSP analysis algorithm, spatial domain features of the preprocessed electroencephalogram signal from a spatial domain specifically includes:

using formulas

Extracting the spatial domain characteristics of the preprocessed electroencephalogram signals from a spatial domain; wherein f is_pIs a spatial domain feature; z_pAs CSP components; z_iIs the second CSP component; i is 1,2, 3.

Optionally, the extracting, by using a mutual information algorithm, mutual information features of the preprocessed electroencephalogram signal specifically includes:

by using

Extracting after the pretreatmentThe mutual information of the brain electrical signals; wherein I (x, y) is mutual information; x and y are arbitrary two N-lead electroencephalogram signals with the length of L, and are used for measuring the linear or nonlinear relation of the multichannel electroencephalogram signals, wherein N is the number of electroencephalogram signal channels, and L is the signal length; p (x, y) is the joint probability density of x and y, p_x(x) An edge probability density of x; p is a radical of_y(y) an edge probability density of y; h (y) is the entropy of y, and the uncertainty of y is measured; h (y | x) is the uncertainty of y given x.

A motor imagery electroencephalogram feature enhancement system, comprising:

the enhancement normal form acquisition module is used for acquiring a multi-mode neural feedback training motor imagery electroencephalogram feature enhancement normal form; the multi-mode nerve feedback training motor imagery electroencephalogram characteristic enhancement paradigm is a stimulation mode of multiple stimulation sources;

the multi-channel electroencephalogram signal acquisition module is used for acquiring multi-channel electroencephalogram signals of a parietal region motor cortex and an occipital region visual cortex pair based on the multi-mode neural feedback training motor imagery electroencephalogram characteristic enhancement paradigm;

the preprocessing module is used for preprocessing the electroencephalogram signals and determining the preprocessed electroencephalogram signals;

the dynamic characteristic and coupling characteristic determining module is used for extracting dynamic and coupling multilevel characteristics of the preprocessed electroencephalogram signal and determining the dynamic characteristic and the coupling characteristic of the preprocessed electroencephalogram signal; the dynamic characteristics comprise time domain characteristics, frequency domain characteristics and space domain characteristics of the electroencephalogram signals; the coupling characteristic is a mutual information characteristic among a plurality of the preprocessed electroencephalogram signals;

and the enhancement display module is used for displaying the characteristic numerical value of the motor imagery electroencephalogram signal before enhancement and the characteristic numerical value of the motor imagery electroencephalogram signal after enhancement according to the dynamic characteristic and the coupling characteristic.

Optionally, the preprocessing module specifically includes:

the power frequency elimination unit is used for eliminating the power frequency of the electroencephalogram signal by using the self-adaptive notch filter and determining the electroencephalogram signal after the power frequency is eliminated;

the base line drift elimination unit is used for eliminating the base line drift of the electroencephalogram signal after the power frequency is eliminated by utilizing a high-pass filter and determining the electroencephalogram signal with the base line drift eliminated;

and the band-pass filtering processing unit is used for carrying out band-pass filtering processing on the electroencephalogram signal with the baseline drift eliminated and determining the preprocessed electroencephalogram signal.

Optionally, the dynamic characteristic and coupling characteristic determining module specifically includes:

the time domain feature extraction unit is used for extracting the time domain features of the preprocessed electroencephalogram signals from the time domain by utilizing autoregressive model coefficients;

the frequency domain characteristic extraction unit is used for extracting the frequency domain characteristics of the preprocessed electroencephalogram signals from a frequency domain by utilizing a wavelet packet energy spectrum;

the spatial domain feature extraction unit is used for extracting spatial domain features of the preprocessed electroencephalogram signals from a spatial domain by utilizing a common spatial mode analysis algorithm;

and the mutual information extraction unit is used for extracting the mutual information characteristics of the preprocessed electroencephalogram signals by using a mutual information algorithm.

According to the specific embodiment provided by the invention, the invention discloses the following technical effects: the invention provides a motor imagery electroencephalogram characteristic enhancement method and a system, wherein the brain influences and mobilizes the energy activity of a brain motor imagery area through the activity of a brain visual area and a brain movement area in the process of carrying out upper limb motor imagery, and further enhances the motor imagery effect through nerve feedback, so that the method has comprehensiveness and scientificity; and the dynamic characteristics and the coupling characteristics of the preprocessed electroencephalogram signals are extracted through the dynamic and coupling multi-level characteristics, the enhancement effect of the electroencephalogram signals after the multi-mode neural feedback enhancement paradigm training is comprehensively and quantitatively evaluated, and the identification rate of the effective signals of the scalp electroencephalogram is improved.

Drawings

In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings needed to be used in the embodiments will be briefly described below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and it is obvious for those skilled in the art to obtain other drawings without inventive exercise.

FIG. 1 is a flow chart of a motor imagery electroencephalogram feature enhancement method provided by the present invention;

FIG. 2 is a flow chart of the working principle provided by the present invention;

FIG. 3 is a brain map of MI-EEG under the same instruction and different stimulation tasks provided by the present invention; FIG. 3(a) is a brain map of MI-EEG provided by the present invention without stimulation; FIG. 3(b) is a brain map of MI-EEG under visual stimuli provided by the present invention; FIG. 3(c) is a brain map of MI-EEG under motion stimulation provided by the present invention; FIG. 3(d) is a brain map of MI-EEG under multi-modal stimulation as provided by the present invention;

FIG. 4 is a diagram of an experimental interface provided by the present invention; FIG. 4(a) is an experimental interface diagram of two experimental modes of simple motor imagery and motion execution feedback provided by the present invention; FIG. 4(b) is an experimental interface diagram for two experimental tasks of observing picture motor imagery and multi-mode stimulation provided by the present invention;

FIG. 5 is a single set of experimental process diagrams provided by the present invention;

FIG. 6 is a diagram of the International Standard 10-20 brain electrical electrode distribution and selected brain electrical channel (marked in gray) provided by the present invention;

FIG. 7 is a schematic diagram of the electroencephalogram signals before and after preprocessing provided by the present invention; FIG. 7a is a waveform diagram of an original electroencephalogram signal provided by the present invention; FIG. 7b is a waveform diagram of the pre-processed electroencephalogram signal provided by the present invention; FIG. 7c is a diagram of an original EEG signal provided by the present invention; FIG. 7d is a diagram of a preprocessed initial electroencephalogram signal spectrogram;

FIG. 8 is a comparison of motor imagery EEG signatures before and after training provided by the present invention; fig. 8a is a comparison graph of integrated values provided by the present invention, fig. 8b is a comparison graph of wavelet energy spectrum-alpha frequency band provided by the present invention, fig. 8c is a comparison graph of wavelet energy spectrum-beta frequency band provided by the present invention, fig. 8d is a comparison graph of CSP provided by the present invention, fig. 8e is a comparison graph of mutual information provided by the present invention, and fig. 8f is a comparison graph of AR model coefficients provided by the present invention.

Detailed Description

The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.

The invention aims to provide a motor imagery electroencephalogram characteristic enhancement method and system, which can improve the identification rate of effective signals of scalp electroencephalograms.

All other embodiments obtained with the creative effort

In order to make the aforementioned objects, features and advantages of the present invention comprehensible, embodiments accompanied with figures are described in further detail below.

Fig. 1 is a flowchart of a motor imagery electroencephalogram feature enhancement method provided by the present invention, and as shown in fig. 1, the motor imagery electroencephalogram feature enhancement method includes:

step 101: acquiring a multi-mode neural feedback training motor imagery electroencephalogram characteristic enhancement paradigm; the multi-mode neural feedback training motor imagery electroencephalogram characteristic enhancement paradigm is a stimulation mode of multiple stimulation sources. The stimulation source is a single vision stimulation task, a single action execution feedback task and a multi-mode stimulation task integrating two single auxiliary modes.

Step 102: and acquiring multichannel electroencephalogram signals of the parietal region motor cortex and the occipital region visual cortex.

Step 103: and preprocessing the electroencephalogram signals, and determining the preprocessed electroencephalogram signals.

Step 104: extracting dynamic and coupling multilevel characteristics of the preprocessed electroencephalogram signal, and determining the dynamic characteristics and the coupling characteristics of the preprocessed electroencephalogram signal; the dynamic characteristics comprise time domain characteristics, frequency domain characteristics and space domain characteristics of the electroencephalogram signals; the coupling characteristic is a mutual information characteristic among the plurality of preprocessed electroencephalogram signals.

Based on dynamic signal analysis and machine learning theory, the preprocessed electroencephalogram signals are characterized by the multi-domain dynamic characteristics of energy characteristics, local frequency band energy characteristics, space distribution energy characteristics and the like of multi-channel motor imagery electroencephalogram signals from time domain, frequency domain and space domain by using AR coefficients, wavelet packet energy spectrums and CSP analysis algorithms, the electroencephalogram signal intensity under different stimuli is analyzed, and mutual information coupling characteristics of the electroencephalogram signals are extracted based on coupling intensities among channels and different brain areas and time-varying nonlinear characteristics of synchronous expression under different motion modes.

A. Coefficient of autoregressive model

The brain electrical signal is a non-stationary signal, but can be approximated as a stationary signal in a short time, and thus can be analyzed using an Autoregressive model (AR). The AR model is described as follows:

where x (n) represents the output signal of the system, w (n) represents the input signal of the system, and p is the order of the model. The matrix a contains the main information of the electroencephalogram signal.

B. Wavelet packet energy spectrum

Assuming a sequence S for each subband signal after wavelet packet conversion_i,j,kI k |, 1,2, …, L }. Wherein: i is the number of decomposition layers; j is the number of decomposed frequency bands, j is 1,2, …,2ⁱ-1; and L is the sample length of each frequency band sequence of the wavelet packet decomposition. Setting the total energy of the full frequency band as 1, and normalizing the energy of each sub-frequency band, wherein the normalized energy of each sub-frequency band signal is as follows:

the proportion of the energy of each frequency band in the total energy is taken as a characteristic parameterNumber structure feature vector

The eigenvector is defined as the wavelet packet energy spectrum.

Motor imagery can activate the cortex of the contralateral sensory motor area of the brain and promote metabolism and increased blood flow in this area, mainly manifested by changes in alpha (8-12Hz) waves and beta (13-30Hz) waves.

C. Common spatial pattern analysis

Common Spatial Pattern (CSP): the spatial domain filtering feature extraction algorithm under two classification tasks can extract spatial distribution components of each class from multi-channel brain-computer interface data.

An N x T matrix A is defined to represent the original brain electrical signal data segments, where N represents the number of electrodes, i.e., spatial leads, and T represents the number of sampling points per channel. For two types of data, such as left and right hand imagination data, in order to make the decomposition matrix obtained by the CSP algorithm more effective, the spatial covariance can be obtained by calculating the covariance average of each type of electroencephalogram data. For the eigenvector matrix B, when one class has the largest eigenvalue, it can be ensured that the other class has the smallest eigenvalue, and the projection matrix can be obtained, that is, the optimal spatial filter is:

W＝B'P (3)

a new EEG data segment can be projected to the matrix to obtain:

Z＝WA (4)

finally, taking variance of the rows of the generated new signal, and then carrying out logarithm and normalization processing as characteristics, wherein the purpose of log transformation is to approximate the data to be evenly distributed, and the following can be obtained:

in the formula Z_pIs a CSP component.

D. Mutual information

The larger the mutual information value is, the more relevant the two signals are, otherwise, the less relevant the two signals are. The calculation formula of mutual information of two random variables x and y is as follows:

where H (y) represents the entropy of y, the uncertainty of y can be measured, and H (y | x) represents the uncertainty of y given x.

Step 105: and displaying the characteristic numerical value of the motor imagery electroencephalogram signal before enhancement and the characteristic numerical value of the motor imagery electroencephalogram signal after enhancement according to the dynamic characteristic and the coupling characteristic.

By adopting the motor imagery electroencephalogram characteristic enhancement analysis method based on multi-mode neural feedback training, on one hand, the multi-mode training paradigm is more obvious, accurate and effective to activate the corresponding brain area, and the usability and accuracy of experimental data are well guaranteed; on the other hand, the extracted dynamic and coupled multi-level features have higher accuracy and reliability as a means for quantitatively evaluating the electroencephalogram feature enhancement effect.

Wherein, the nerve feedback is as follows: the brain nerve feedback is to generate a micro current in the operation process of the brain, the micro current is linked with a computer through an electroencephalogram acquisition device, the current brain wave activity state of a tested person is detected, and the computer gives feedback in a certain visual sense, sound and other modes.

Scalp electroencephalogram: brain electrical signals of brain scalp collected by adopting a non-invasive electrode; motor imagery electroencephalogram: without actual limb behaviors, the brain will be used to imagine the limb actions to generate endogenous spontaneous electroencephalogram.

Enhancing the electroencephalogram signals: because the electroencephalogram signal is very weak, the electroencephalogram signal has the characteristics of low spatial resolution, non-stationarity and the like, and the characteristics of the electroencephalogram signal are improved by improving an experimental paradigm, optimizing an algorithm and the like.

And (3) feature enhancement: the active mind of the brain is enhanced by a paradigm induction mind enhancement method, and then spontaneous electroencephalogram signals which can embody the subjective mind of the brain are obtained. After the examinee is trained, the characteristic information carried in the electroencephalogram signals is more obvious, and subjective intention can be more easily identified.

The motor imagery electroencephalogram feature enhancement method provided by the invention is a motor imagery electroencephalogram feature enhancement analysis method based on multi-mode neural feedback training, and the brain designs multi-mode neural feedback training integrating visual stimulation and action execution in the process of carrying out upper limb motor imagery, so that the autonomous idea is enhanced; the method comprises the steps of collecting multichannel electroencephalogram signals of a parietal region and a occipital region, removing 50Hz power frequency by using an adaptive filter, removing baseline drift by using a high-pass filter, extracting 5-32Hz frequency bands by using band-pass filtering, preprocessing, extracting dynamic characteristics of EEG from time domain, frequency domain and space domain respectively by combining an Autoregressive model coefficient (AR), a wavelet packet energy spectrum and a common space mode CSP analysis algorithm, extracting synchronous coupling characteristics of the EEG, completing synchronous coupling of the EEG and multi-level fusion characteristic index extraction of multi-domain dynamic characteristics, taking the index as an index for evaluating characteristic enhancement of multi-mode MI-neural feedback before and after training, and analyzing the superiority of a multi-mode characteristic enhancement mechanism in a contrast way, wherein the superiority is shown in figure 2.

It is known that there is a certain correlation between brain regions, which activate the cortex of the contralateral sensory motor region and the parietal lobe region of the brain and promote the metabolism and blood flow increase of the regions when a subject performs motor imagery, activate the energy change of the occipital region when a subject gives visual stimulation, and enhance the energy of the brain somatic motor region when a subject performs an action; based on the existing research, when the brain is stimulated by external, the metabolism and blood flow of the brain area are increased, certain energy fluctuation is shown, when the external is not stimulated, the brain is in a resting or inert state, and the area and the degree of energy activation can be clearly observed by drawing a brain map.

As shown in fig. 3, the exercise imagination of right-handed boxing is required, when the outside is not stimulated, the brain area energy is activated to a relatively weak degree, and the remaining three brain maps are visual stimulation, motion execution stimulation and comprehensive stimulation, so that the energy activation degree is relatively strong and the position is more accurate.

The training can consciously make the testee generate physiological reaction, thereby enhancing the brain electrical signal activity of the testee. In the experiment, the situation that the subject who participates in the motor imagery experiment for the first time often cannot complete imagery smoothly and cannot concentrate attention is found, so that the problems that the acquired MI-EEG has signal errors and effective features cannot be extracted are caused. However, the subject who has undergone several motor imagery tests does not have the above-mentioned problems, and the characteristic phenomenon of the signal is more prominent. That is, the activation degree of brain is different between the trained and untrained subjects in the motor imagery, and the MI-EEG of the trained subject has more stability and effectiveness and the characteristics are easier to extract.

Based on the exploration, a multi-mode neural feedback training motor imagery electroencephalogram characteristic enhancement paradigm is provided; the characteristic enhancement is to enhance the brain active idea through a paradigm induced idea enhancement method so as to obtain a spontaneous electroencephalogram signal which can reflect the brain active idea better, and after a subject is trained, characteristic information carried in the electroencephalogram signal is more obvious, so that subjective intention can be identified more easily.

The paradigm sets pure imagination, single visual stimulus training, single action execution feedback, and multi-mode neurofeedback to train motor imagery. Setting an experimental group and a control group, and alternately arranging the three stimulation training modes and the simple imagination, so that the electroencephalogram signals before and after each stimulation mode can be conveniently compared, and the method is used for demonstrating the effectiveness of the multi-mode neural feedback enhanced electroencephalogram characteristic effect; meanwhile, a plurality of groups of comparison tests can be set, and the arrangement sequence of the stimulation methods of each group of tests needs to be different.

Firstly, an experimental platform for training spontaneous electroencephalogram enhancement of motor imagery, observation pictures, motor execution and multi-mode stimulation is built, as shown in fig. 4, fig. 4(a) is an experimental interface diagram of two experimental modes of pure motor imagery and action execution feedback provided by the invention, fig. 4(b) is an experimental interface diagram of two experimental tasks of observing the picture motor imagery and multi-mode stimulation provided by the invention, and a testee completes actions of executing or imagining a left hand and a right hand to make a fist corresponding to a stimulation paradigm according to an arrow of the interface or the direction of a picture prompt.

Designing an experimental scheme:

the electroencephalogram signals of 24 tested subjects are collected in the experiment, all the 24 tested subjects are 20-25 years old students (male: female: 3:1) and are right-handed, and all the tested subjects do not participate in the brain-computer interface experiment of motor imagery. The experimental procedure is shown in FIG. 5.

In the experiment, the subject is required to sit on a comfortable chair about 1m away from the screen, the hands and feet are relaxed, the limbs are kept still as much as possible, eye movements such as blinking and the like are avoided as much as possible when the limbs are imagined to move, table 1 is an experimental paradigm setting provided by the invention, and the experimental paradigm setting is shown in table 1.

TABLE 1

Serial number	Experimentalgroup	Control group	1	Control group 2	Number ofexperimental groups
						1	Simple motor imagery	Simple motor imagery	Simple motor imagery	2
2	Multi-mode stimulation paradigm	Action execution feedback	Motion image of observation picture	4
					3	Simple motor imagery	Simple motor imagery	Simple motor imagery	2

An experimental group, acontrol group 1 and a control group 2 are designed, the experimental contents are shown in table 1, a group of experiments are set for 12 motor imagination times, a left instruction and a right instruction respectively appear 6 times at random, namely, a left hand fist and a right hand fist are respectively imagined for 6 times, the fist holding lasts for 4s every time, and the middle rest of the two motor imagination times is 3 s; one group of experiments is about 1 minute and 30 seconds, 8 groups are needed in the whole experiment, the rest is 1 minute between the same task groups, the rest is 3 minutes in different task groups, and the whole course is about 23 minutes.

The simple motor imagery requires the testee to imagine the fist making action of the left hand or the right hand in the corresponding direction according to the arrow direction displayed at the lower left corner in the experimental interface shown in fig. 4 a; observing the picture movement imagination, and requiring the testee to imagine the fist making action of the left hand or the right hand in the corresponding direction according to the orientation of the picture figure displayed in the center of the experimental interface shown in the figure 4 b; action execution feedback, requiring the testee to make a fist making action of the left hand or the right hand in the corresponding direction according to the arrow direction displayed at the lower left corner in the experimental interface shown in fig. 4 a; the multi-mode stimulation paradigm, i.e., feedback is performed in conjunction with viewing pictures and actions, requiring the subject to make a left or right handed punch making motion in a corresponding direction according to the orientation of the picture character displayed in the center of the experimental interface shown in fig. 4 b.

MI-EEG data acquisition based on a multimodal neurofeedback training paradigm. According to the embodiment of the invention, neuroacle 64 channel electroencephalogram acquisition equipment is used for acquiring multichannel electroencephalogram signals of a subject after multi-mode neural feedback enhancement paradigm training, and the electrode position adopts an international 10-20 lead positioning standard. FIG. 6 is a diagram of the International 10-20 systems Standard (Synamp2, Compounds incorporated, Charlotte, NC, USA) EEG electrode distribution provided by the present invention, with grey color showing the brain region channels selected in this experimental paradigm; collecting 21-channel electroencephalogram signals of a motor cortex of a top area and 8-channel electroencephalogram signals of a visual cortex of a occipital area of a subject; the sampling frequency was 1000 Hz.

A multilevel fusion characteristic index method for comprehensively acquiring MI-EEG synchronous coupling and multi-domain dynamic characteristics is used for comprehensively and quantitatively evaluating and analyzing the EEG characteristic enhancement effect.

The invention is further illustrated with reference to the following figures.

The complete electroencephalogram signal processing process comprises preprocessing, feature extraction and mode classification. Preprocessing is carried out before feature extraction is carried out, and the signal-to-noise ratio of signals is improved. The electroencephalogram signal preprocessing process comprises the following steps: and removing 50Hz power frequency interference of the acquired electroencephalogram signals by using a self-adaptive filter, removing baseline drift by using a high-pass filter, and finally performing 1-32Hz band-pass filtering by using an FIR digital filter. The electroencephalogram signals before and after preprocessing are respectively shown in fig. 7a, c, b and d.

The embodiment of the invention extracts the characteristics of the electroencephalogram signals acquired by utilizing the multi-mode experimental paradigm, enumerates the analysis of the acquired pure motor imagery EEG of 3 testees before and after the multi-mode stimulation paradigm training, and respectively explains the characteristics in the aspects of integral values, AR model coefficients, wavelet energy spectrums, CSP (chip size scale) and mutual information coupling characteristics.

As shown in fig. 8, the number of the star in the figure represents the pre-training motor imagery EEG feature, the square represents the post-training motor imagery EEG feature, and the abscissa is the experimental trial. Fig. 8a is an integrated value comparison diagram provided by the present invention, fig. 8b is a wavelet energy spectrum-alpha frequency band comparison diagram provided by the present invention, fig. 8c is a wavelet energy spectrum-beta frequency band comparison diagram provided by the present invention, fig. 8d is a CSP comparison diagram provided by the present invention, fig. 8e is a mutual information comparison diagram provided by the present invention, fig. 8f is an AR model coefficient comparison diagram provided by the present invention, and table 2 is an expectation and variance table of motor imagery electroencephalogram signal characteristics before and after training provided by the present invention, as shown in table 2.

TABLE 2

The invention utilizes t test with confidence coefficient of 0.95 to carry out statistical analysis on the characteristics of the electroencephalogram signals. The results show that the EEG after the multi-mode neural feedback training shows enhancement of different degrees on the extracted time domain characteristics (integral value and AR model coefficient), frequency domain characteristics (wavelet energy spectrum alpha frequency band and beta frequency band), space domain Characteristics (CSP) and coupling characteristics (mutual information).

The invention also comprises a motor imagery electroencephalogram characteristic enhancement system corresponding to the enhancement method, wherein the enhancement system comprises:

the enhancement normal form acquisition module is used for acquiring a multi-mode neural feedback training motor imagery electroencephalogram feature enhancement normal form; the multi-mode neural feedback training motor imagery electroencephalogram characteristic enhancement paradigm is a stimulation mode of multiple stimulation sources.

And the multi-channel electroencephalogram signal acquisition module is used for acquiring multi-channel electroencephalogram signals of the parietal region motor cortex and the occipital region visual cortex pair based on the multi-mode neural feedback training motor imagery electroencephalogram characteristic enhancement paradigm.

And the preprocessing module is used for preprocessing the electroencephalogram signals and determining the preprocessed electroencephalogram signals.

The dynamic characteristic and coupling characteristic determining module is used for extracting dynamic and coupling multilevel characteristics of the preprocessed electroencephalogram signal and determining the dynamic characteristic and the coupling characteristic of the preprocessed electroencephalogram signal; the dynamic characteristics comprise time domain characteristics, frequency domain characteristics and space domain characteristics of the electroencephalogram signals; the coupling characteristic is a mutual information characteristic among the plurality of preprocessed electroencephalogram signals.

(1) A multi-mode neural feedback training motor imagery electroencephalogram characteristic enhancement paradigm. The method is characterized in that single stimulation and multi-mode neural feedback training are set, and the autonomic idea is enhanced. Specifically, in the process of carrying out upper limb motor imagery by the brain, the motor imagery is trained by setting simple imagery, single visual stimulation training, single action execution feedback and multi-mode neural feedback, three stimulation training modes and the simple imagery are arranged alternately, and an experimental group and a control group are set for carrying out a comparison experiment.

(2) A multi-level fusion characteristic index method for comprehensively acquiring MI-EEG synchronous coupling and multi-domain dynamic characteristics. The method is characterized in that dynamic characteristics of the electroencephalogram signals are extracted from a time domain, a frequency domain and a space domain respectively by using multi-mode neural feedback enhancement training paradigm and then combined with the coupling characteristics of the electroencephalogram signals, and brain area energy fluctuation before and after multi-mode neural feedback enhancement training is comprehensively and reliably displayed through a brain map and specific results and is used as a characteristic index for quantitatively evaluating electroencephalogram characteristic enhancement before and after training.

The invention enhances motor imagery electroencephalogram signals, is used in the field of brain-computer interfaces, extracts spontaneous electroencephalogram signals with high precision and high recognition rate to perform multi-classification behavior analysis, converts the spontaneous electroencephalogram signals into control instructions, and can be used for controlling external equipment such as mechanical arms, wheelchairs and the like more accurately.

In a specific actual life, the enhancement method and the enhancement system provided by the invention can realize the following operations: (1) the wheelchair and the mechanical arm can be controlled by the patient with physical disability but clear head to facilitate life, and meanwhile, the brain-controlled rehabilitation assistive device can be independently operated to realize the significance of active rehabilitation of the patient; (2) in intelligent driving, left and right steering lamps of a vehicle can be controlled by collecting and identifying two classified motor imagery electroencephalogram signals of a driver; (3) the intelligent trolley can also realize the forward, backward, left turn, right turn and the like by identifying the multi-classification behaviors of the testee.

According to the method, the active mind of the brain is enhanced by a paradigm induced mind enhancement method, so that a spontaneous electroencephalogram signal which can embody the subjective mind of the brain is obtained; after the examinee is trained, the characteristic information carried in the electroencephalogram signal of the examinee is more obvious, the subjective intention is more easily recognized, and then the characteristics of the signal are respectively extracted from time, frequency, space and coupling angles through the set multi-level characteristic extraction module, namely: multi-domain and multi-level high-dimensional characteristics.

A great deal of research shows that the nerve feedback training can generate the function of autonomic regulation on the movement cortex area of the human body, and the main theoretical basis is that the sensory-motor rhythm has special function in nerve regulation. At present, a large amount of neural feedback training is applied to various brain function training technologies such as creativity training, attention training, memory training and the like, theoretical support and technical support are provided for the neural feedback system to be applied to the autonomic idea enhancement of the human brain, and the neural feedback training based on the multiple modes can enable a subject to be stimulated from multiple aspects in the training process, so that the training effect is more efficient.

The embodiments in the present description are described in a progressive manner, each embodiment focuses on differences from other embodiments, and the same and similar parts among the embodiments are referred to each other. For the system disclosed by the embodiment, the description is relatively simple because the system corresponds to the method disclosed by the embodiment, and the relevant points can be referred to the method part for description.

The principles and embodiments of the present invention have been described herein using specific examples, which are provided only to help understand the method and the core concept of the present invention; meanwhile, for a person skilled in the art, according to the idea of the present invention, the specific embodiments and the application range may be changed. In view of the above, the present disclosure should not be construed as limiting the invention.

Claims

1. A motor imagery electroencephalogram feature enhancement method is characterized by comprising the following steps:

acquiring multichannel electroencephalogram signals of a parietal region motor cortex and an occipital region visual cortex pair based on the multi-mode neural feedback training motor imagery electroencephalogram characteristic enhancement paradigm;

2. The motor imagery electroencephalogram feature enhancement method according to claim 1, wherein the preprocessing of the electroencephalogram signal is performed to determine the preprocessed electroencephalogram signal, and specifically comprises:

eliminating the power frequency of the electroencephalogram signal by using a self-adaptive notch filter, and determining the electroencephalogram signal after eliminating the power frequency;

3. The motor imagery electroencephalogram feature enhancement method according to claim 1, wherein the performing dynamic and coupling multilevel feature extraction on the preprocessed electroencephalogram signal to determine dynamic features and coupling features of the preprocessed electroencephalogram signal specifically comprises:

4. The motor imagery electroencephalogram feature enhancement method according to claim 3, wherein the extracting the time domain features of the preprocessed electroencephalogram signal from the time domain using an autoregressive model system specifically comprises:

using formulas

5. The motor imagery electroencephalogram feature enhancement method according to claim 4, wherein the extracting the frequency domain features of the preprocessed electroencephalogram signal from a frequency domain using a wavelet packet energy spectrum specifically comprises:

acquiring energy of each sub-band signal;

6. The motor imagery electroencephalogram feature enhancement method according to claim 5, wherein the extracting spatial domain features of the preprocessed electroencephalogram signal from a spatial domain using a common spatial mode CSP analysis algorithm specifically comprises:

using formulas

7. The motor imagery electroencephalogram feature enhancement method according to claim 3, wherein the extracting mutual information of the preprocessed electroencephalogram signals by using a mutual information algorithm specifically comprises:

by using

Extracting mutual information of the preprocessed electroencephalogram signals; wherein I (x, y) is mutual information; x and y are arbitrary two N-lead electroencephalogram signals with the length of L, and are used for measuring the linear or nonlinear relation of the multichannel electroencephalogram signals, wherein N is the number of electroencephalogram signal channels, and L is the signal length; p (x, y) is the joint probability density of x and y, p_x(x) An edge probability density of x; p is a radical of_y(y) an edge probability density of y; h (y) is the entropy of y, and the uncertainty of y is measured; h (y | x) is the uncertainty of y given x.

8. A motor imagery electroencephalogram feature enhancement system, comprising:

9. The motor imagery electroencephalogram feature enhancement system of claim 8, wherein the preprocessing module specifically comprises:

10. The motor imagery electroencephalogram feature enhancement system of claim 8, wherein the dynamic feature and coupling feature determination module specifically comprises:

the time domain feature extraction unit is used for extracting the time domain features of the preprocessed electroencephalogram signals from the time domain by utilizing the AR coefficients of the autoregressive model;

the spatial domain feature extraction unit is used for extracting spatial domain features of the preprocessed electroencephalogram signals from a spatial domain by utilizing a common spatial mode CSP analysis algorithm;