CN115429289A - Brain-computer interface training data amplification method, device, medium and electronic equipment - Google Patents

Abstract

Translated fromChinese

本申请提供一种脑机接口训练数据扩增方法、装置、介质及电子设备。该脑机接口训练数据扩增方法，包括：获取待处理脑电信号集包括的原始信号子集；其中，每获取一个原始信号子集，针对获取的原始信号子集执行以下操作，以得到用以构成扩增脑电信号集的扩增信号子集：基于获取的原始信号子集，确定与获取的原始信号子集对应的脑电信号单元平均值；基于预设的参考矩阵和脑电信号单元平均值，得到目标源混叠矩阵；基于目标源混叠矩阵和参考矩阵，得到重构源信号；基于重构源信号、随机噪声信号集、预设的扩增试次数和获取的原始信号子集，得到与获取的原始信号子集对应的扩增信号子集。该方法能提高脑机接口在小样本情况下的识别性能。

The present application provides a brain-computer interface training data amplification method, device, medium and electronic equipment. The brain-computer interface training data augmentation method includes: obtaining a subset of original signals included in the set of EEG signals to be processed; wherein, each time a subset of original signals is obtained, the following operations are performed on the obtained subset of original signals to obtain To form the amplified signal subset of the amplified EEG signal set: based on the acquired original signal subset, determine the average value of the EEG signal unit corresponding to the acquired original signal subset; based on the preset reference matrix and the EEG signal The unit average value is used to obtain the target source aliasing matrix; based on the target source aliasing matrix and the reference matrix, the reconstructed source signal is obtained; based on the reconstructed source signal, random noise signal set, the preset number of amplification trials and the acquired original signal Subset to obtain the amplified signal subset corresponding to the acquired original signal subset. This method can improve the recognition performance of the brain-computer interface in the case of small samples.

Description

Translated fromChinese

一种脑机接口训练数据扩增方法、装置、介质及电子设备A brain-computer interface training data amplification method, device, medium and electronic equipment

技术领域technical field

本申请涉及脑机接口技术领域，特别是涉及一种脑机接口训练数据扩增方法、装置、介质及电子设备。The present application relates to the technical field of brain-computer interface, in particular to a brain-computer interface training data amplification method, device, medium and electronic equipment.

背景技术Background technique

脑机接口(brain-computer interface,BCI)为大脑与外部环境之间提供了一条无需依赖周围神经的直接通讯通路，能够替代、修复、增强、补充或改善正常的神经系统功能。典型的脑机接口系统通过检测特定的神经活动来解码脑信息，并将其转化为可输出的机器指令，从而实现大脑意图的直接表达。The brain-computer interface (BCI) provides a direct communication pathway between the brain and the external environment without relying on peripheral nerves, and can replace, repair, enhance, supplement or improve normal nervous system functions. A typical brain-computer interface system decodes brain information by detecting specific neural activities and converts it into outputable machine instructions, thereby realizing the direct expression of brain intentions.

相较于功能性近红外成像、脑磁图、皮层脑电等大脑信息获取方式来说，脑电(electroencephalogram,EEG)信号具有无创伤、价格低廉和高时间分辨率的优势，因此被广泛应用于脑机接口领域。然而，受限于脑电信号的跨模态、跨个体、跨时间等变异性，当前脑机接口解码算法大多采用个体校准模式，即在每次使用系统前均需要用户重新采集训练信号来构建符合当前脑电特点的分类模型。Compared with functional near-infrared imaging, magnetoencephalography, cortical EEG and other brain information acquisition methods, electroencephalogram (EEG) signals have the advantages of non-invasive, low cost and high temporal resolution, so they are widely used in the field of brain-computer interface. However, limited by the cross-modal, cross-individual, and cross-time variability of EEG signals, most of the current brain-computer interface decoding algorithms use the individual calibration mode, that is, before each use of the system, the user needs to re-acquire training signals to build A classification model that conforms to the current EEG characteristics.

相关技术的脑机接口识别方案，是通过实验采集来获得用于校准的EEG信号数据。这种方式耗时长，舒适性差，需要投入的人力成本高，因此很难获得足够的校准数据来训练分类模型。脑电信号解码过程中面临的数据不足问题，容易导致脑电信号识别的准确率较低，阻碍了脑机接口系统走向实际应用。The brain-computer interface recognition scheme of the related art obtains EEG signal data for calibration through experimental collection. This method takes a long time, is not comfortable, and requires high labor costs, so it is difficult to obtain enough calibration data to train the classification model. The lack of data in the process of EEG signal decoding can easily lead to low accuracy of EEG signal recognition, which hinders the practical application of brain-computer interface systems.

发明内容Contents of the invention

本申请实施例提供了一种脑机接口训练数据扩增方法、装置、介质及电子设备，旨在解决现有的脑电信号解码过程中面临的数据不足问题，提高脑机接口在小样本情况下的识别性能，从而降低系统的校准负担。The embodiment of the present application provides a brain-computer interface training data amplification method, device, medium, and electronic equipment, aiming to solve the problem of insufficient data in the existing EEG signal decoding process and improve the performance of the brain-computer interface in the case of small samples. Lower recognition performance, thereby reducing the calibration burden of the system.

第一方面，本申请实施例提供一种脑机接口训练数据扩增方法，包括：In the first aspect, the embodiment of the present application provides a brain-computer interface training data amplification method, including:

获取待处理脑电信号集包括的原始信号子集；Obtain a subset of original signals included in the EEG signal set to be processed;

其中，每获取一个所述原始信号子集，针对获取的所述原始信号子集执行以下操作，以得到用以构成扩增脑电信号集的扩增信号子集：Wherein, each time a subset of the original signal is acquired, the following operations are performed on the acquired subset of the original signal to obtain an amplified signal subset used to form an amplified EEG signal set:

基于获取的所述原始信号子集，确定与获取的所述原始信号子集对应的脑电信号单元平均值；任一所述原始信号子集包括数量为实验试次数的脑电信号单元；Based on the obtained subset of original signals, determine the average value of the EEG signal units corresponding to the acquired subset of original signals; any of the subsets of original signals includes EEG signal units whose number is the number of experimental trials;

基于预设的参考矩阵和所述脑电信号单元平均值，得到目标源混叠矩阵；所述目标源混叠矩阵是满足使预设的目标函数取最小值的源混叠矩阵；Based on the preset reference matrix and the average value of the EEG signal unit, a target source aliasing matrix is obtained; the target source aliasing matrix is a source aliasing matrix that satisfies the minimum value of the preset target function;

基于所述目标源混叠矩阵和所述参考矩阵，得到重构源信号；obtaining a reconstructed source signal based on the target source aliasing matrix and the reference matrix;

基于所述重构源信号、随机噪声信号集、预设的扩增试次数和获取的所述原始信号子集，得到与获取的所述原始信号子集对应的所述扩增信号子集。The amplified signal subset corresponding to the acquired original signal subset is obtained based on the reconstructed source signal, the random noise signal set, the preset number of amplification trials, and the acquired original signal subset.

本申请实施例提供的脑机接口训练数据扩增方法，能够通过获取待处理脑电信号集包括的原始信号子集，并基于参考矩阵、随机噪声信号集和与获取的所述原始信号子集对应的脑电信号单元平均值进行脑电信号重构，获得符合脑电特性的扩增信号，从而实现对脑机接口训练数据中的待处理脑电信号集进行高效的数据扩增，提高脑机接口在小样本情况下的识别性能，从而降低系统的校准负担。The brain-computer interface training data amplification method provided in the embodiment of the present application can obtain the original signal subset included in the EEG signal set to be processed, and based on the reference matrix, the random noise signal set and the acquired original signal subset The average value of the corresponding EEG signal unit is used to reconstruct the EEG signal to obtain an amplified signal that conforms to the EEG characteristics, so as to realize efficient data amplification of the EEG signal set to be processed in the brain-computer interface training data and improve the performance of the brain. The recognition performance of the computer interface in the case of small samples, thereby reducing the calibration burden of the system.

在一种可能的实现方式中，所述基于获取的所述原始信号子集，确定与获取的所述原始信号子集对应的脑电信号单元平均值，包括：In a possible implementation manner, the determining the average value of EEG signal units corresponding to the acquired original signal subset based on the acquired original signal subset includes:

将获取的所述原始信号子集包括的所述脑电信号单元求和，得到与获取的所述原始信号子集对应的第一脑电信号；summing the EEG signal units included in the acquired subset of original signals to obtain a first EEG signal corresponding to the acquired subset of original signals;

将所述第一脑电信号与所述实验试次数求商，得到与获取的所述原始信号子集对应的脑电信号单元平均值。Quotienting the first EEG signal and the number of experimental trials to obtain an average value of EEG signal units corresponding to the acquired subset of original signals.

该实施例提供的脑机接口训练数据扩增方法，将获取的所述原始信号子集包括的所述脑电信号单元求和，得到与获取的所述原始信号子集对应的第一脑电信号；将所述第一脑电信号与所述实验试次数求商，得到与获取的所述原始信号子集对应的脑电信号单元平均值。该方法，可以通过对原始信号子集进行平均，有效减小无关的背景噪声。The brain-computer interface training data amplification method provided in this embodiment sums the EEG signal units included in the acquired original signal subset to obtain the first EEG signal corresponding to the acquired original signal subset signal; calculating the quotient of the first EEG signal and the number of experimental trials to obtain an average value of EEG signal units corresponding to the acquired subset of original signals. This method can effectively reduce irrelevant background noise by averaging a subset of the original signal.

在一种可能的实现方式中，所述目标函数为第一偏差矩阵的弗罗贝尼乌斯Frobenius范数；所述第一偏差矩阵为通过将源混叠矩阵与所述参考矩阵进行矩阵乘法，再与所述脑电信号单元平均值作差得到的。In a possible implementation, the objective function is the Frobenius norm of the first deviation matrix; the first deviation matrix is obtained by performing matrix multiplication of the source aliasing matrix and the reference matrix , and then obtained by making a difference with the average value of the EEG signal unit.

该实施例提供的方法，所述目标函数为第一偏差矩阵的弗罗贝尼乌斯Frobenius范数；所述第一偏差矩阵为通过将源混叠矩阵与所述参考矩阵进行矩阵乘法，再与所述脑电信号单元平均值作差得到的。该方法利用最小二乘法求解源混叠矩阵，计算简单，易于实现，可以减少脑机接口训练数据扩增过程的计算量，提升脑机接口的校准效率。In the method provided by this embodiment, the objective function is the Frobenius norm of the first deviation matrix; the first deviation matrix is obtained by performing matrix multiplication between the source aliasing matrix and the reference matrix, and then obtained by making a difference from the average value of the EEG signal unit. This method uses the least squares method to solve the source aliasing matrix, which is simple to calculate and easy to implement. It can reduce the calculation amount of the brain-computer interface training data amplification process and improve the calibration efficiency of the brain-computer interface.

在一种可能的实现方式中，所述目标函数为将第二偏差矩阵的Frobenius范数，与源混叠矩阵的正则化范数进行求和得到的；所述第二偏差矩阵为通过将源混叠矩阵与所述参考矩阵进行矩阵乘法，再与所述脑电信号单元平均值作差得到的。In a possible implementation, the objective function is obtained by summing the Frobenius norm of the second deviation matrix and the regularization norm of the source aliasing matrix; the second deviation matrix is obtained by combining the source The aliasing matrix is obtained by performing matrix multiplication with the reference matrix, and then making a difference with the average value of the EEG signal unit.

该实施例提供的方法，所述目标函数为将第二偏差矩阵的Frobenius范数，与源混叠矩阵的正则化范数进行求和得到的；所述第二偏差矩阵为通过将源混叠矩阵与所述参考矩阵进行矩阵乘法，再与所述脑电信号单元平均值作差得到的。该方法，在目标函数中加入正则化范数，可以对源混叠矩阵进行约束，使得参数稀疏化，增强求解源混叠矩阵的过程中数值的稳定性，提升对脑机接口训练数据中的待处理脑电信号集进行数据扩增得到的扩增数据的有效性。In the method provided in this embodiment, the objective function is obtained by summing the Frobenius norm of the second deviation matrix and the regularization norm of the source aliasing matrix; the second deviation matrix is obtained by mixing the source The matrix is obtained by matrix multiplication with the reference matrix, and then the difference with the average value of the EEG signal unit. In this method, adding a regularization norm to the objective function can constrain the source aliasing matrix, make the parameters sparse, enhance the numerical stability in the process of solving the source aliasing matrix, and improve the accuracy of the brain-computer interface training data. Validity of the augmented data obtained by data augmentation of the EEG signal set to be processed.

在一种可能的实现方式中，所述基于所述目标源混叠矩阵和所述参考矩阵，得到重构源信号，包括：In a possible implementation manner, the obtaining the reconstructed source signal based on the target source aliasing matrix and the reference matrix includes:

将所述目标源混叠矩阵与所述参考矩阵进行矩阵乘法，得到所述重构源信号。performing matrix multiplication on the target source aliasing matrix and the reference matrix to obtain the reconstructed source signal.

该实施例提供的方法，将所述目标源混叠矩阵与所述参考矩阵进行矩阵乘法，得到所述重构源信号。该方法，将所述目标源混叠矩阵与所述参考矩阵进行矩阵乘法，得到重构源信号，可以重建与当前训练任务相关的源信号，该源信号可以用于后续的信号扩增过程，能够实现对脑机接口训练数据中的待处理脑电信号集进行高效的数据扩增。In the method provided in this embodiment, matrix multiplication is performed on the target source aliasing matrix and the reference matrix to obtain the reconstructed source signal. In this method, the target source aliasing matrix is matrix multiplied by the reference matrix to obtain a reconstructed source signal, which can reconstruct a source signal related to the current training task, and the source signal can be used in a subsequent signal amplification process, Efficient data amplification can be realized for the EEG signal set to be processed in the brain-computer interface training data.

在一种可能的实现方式中，所述基于所述重构源信号、随机噪声信号集、预设的扩增试次数和获取的所述原始信号子集，得到与获取的所述原始信号子集对应的所述扩增信号子集，包括：In a possible implementation manner, based on the reconstructed source signal, the random noise signal set, the preset number of amplification trials, and the acquired original signal subset, the obtained original signal subset The subset of amplified signals corresponding to the set includes:

基于所述扩增试次数和预设规则，生成随机噪声信号集；所述随机噪声信号集包含随机噪声信号单元的数量为所述扩增试次数；Based on the number of amplification trials and preset rules, a random noise signal set is generated; the number of random noise signal units contained in the random noise signal set is the number of amplification trials;

从所述随机噪声信号集中逐一获取所述随机噪声信号单元，每获取一个随机噪声信号单元，执行第一操作，以得到与获取的所述原始信号子集对应的一个扩增信号单元；所述第一操作，包括将获取的所述随机噪声信号单元，与所述重构源信号求和；Acquiring the random noise signal units one by one from the random noise signal set, and performing a first operation for each random noise signal unit acquired to obtain an amplified signal unit corresponding to the acquired original signal subset; the A first operation comprising summing the acquired random noise signal unit with the reconstructed source signal;

将得到的各所述扩增信号单元，作为新的脑电信号单元加入到获取的所述原始信号子集，得到与获取的所述原始信号子集对应的所述扩增信号子集。The obtained amplified signal units are added as new EEG signal units to the acquired original signal subset, to obtain the amplified signal subset corresponding to the acquired original signal subset.

该实施例提供的方法，基于所述扩增试次数和预设规则，生成随机噪声信号集；将获取的所述随机噪声信号单元与所述重构源信号求和，得到扩增信号单元；将得到的各所述扩增信号单元，作为新的脑电信号单元加入到获取的所述原始信号子集，得到与获取的所述原始信号子集对应的所述扩增信号子集。The method provided in this embodiment generates a random noise signal set based on the number of amplification trials and preset rules; sums the obtained random noise signal unit and the reconstructed source signal to obtain an amplified signal unit; The obtained amplified signal units are added as new EEG signal units to the acquired original signal subset, to obtain the amplified signal subset corresponding to the acquired original signal subset.

在一种可能的实现方式中，所述方法还包括：In a possible implementation, the method further includes:

基于所述扩增脑电信号集，训练所述脑机接口的脑电信号识别模型，得到训练后的所述脑电信号识别模型；Based on the amplified EEG signal set, train the EEG signal recognition model of the brain-computer interface to obtain the trained EEG signal recognition model;

获取待识别脑电信号，并利用训练后的所述脑电信号识别模型对所述待识别脑电信号进行识别，以得到脑电信号识别结果。Acquiring the electroencephalogram signal to be identified, and using the trained electroencephalogram signal identification model to identify the electroencephalogram signal to be identified, so as to obtain an electroencephalogram signal identification result.

该实施例提供的方法，还包括：基于所述扩增脑电信号集，训练所述脑机接口的脑电信号识别模型，得到训练后的所述脑电信号识别模型；获取待识别脑电信号，并利用训练后的所述脑电信号识别模型对所述待识别脑电信号进行识别，以得到脑电信号识别结果。该方法，实现对脑机接口训练数据中的待处理脑电信号集进行高效的数据扩增，并根据数据扩增得到的扩增脑电信号集进行脑电信号识别模型的训练，可以提高脑机接口在小样本情况下的识别性能，从而降低系统的校准负担。The method provided in this embodiment further includes: based on the amplified EEG signal set, training the EEG signal recognition model of the brain-computer interface to obtain the trained EEG signal recognition model; obtaining the EEG signal to be recognized signal, and using the trained EEG signal recognition model to identify the EEG signal to be identified, so as to obtain an EEG signal identification result. This method realizes the efficient data amplification of the EEG signal set to be processed in the brain-computer interface training data, and conducts the training of the EEG signal recognition model according to the amplified EEG signal set obtained by data amplification, which can improve the performance of the brain. The recognition performance of the computer interface in the case of small samples, thereby reducing the calibration burden of the system.

第二方面，本申请实施例提供了一种脑机接口训练数据扩增装置，包括：In the second aspect, the embodiment of the present application provides a brain-computer interface training data amplification device, including:

数据准备模块，用于获取待处理脑电信号集包括的原始信号子集；The data preparation module is used to obtain the original signal subset included in the EEG signal set to be processed;

数据扩增模块，用于在所述数据准备模块每获取一个所述原始信号子集时，针对获取的所述原始信号子集执行以下操作，以得到用以构成扩增脑电信号集的扩增信号子集：The data amplification module is configured to perform the following operations on the acquired original signal subset each time the data preparation module acquires one of the original signal subsets, so as to obtain the expanded EEG signal set used to form Added subset of signals:

基于获取的所述原始信号子集，确定与获取的所述原始信号子集对应的脑电信号单元平均值；任一所述原始信号子集包括数量为实验试次数的脑电信号单元；基于预设的参考矩阵和所述脑电信号单元平均值，得到目标源混叠矩阵；所述目标源混叠矩阵是满足使预设的目标函数取最小值的源混叠矩阵；基于所述目标源混叠矩阵和所述参考矩阵，得到重构源信号；基于所述重构源信号、随机噪声信号集、预设的扩增试次数和获取的所述原始信号子集，得到与获取的所述原始信号子集对应的所述扩增信号子集。Based on the obtained subset of original signals, determine the average value of the EEG signal units corresponding to the acquired subset of original signals; any of the subsets of original signals includes EEG signal units whose number is the number of experimental trials; based on The preset reference matrix and the average value of the EEG signal unit are used to obtain the target source aliasing matrix; the target source aliasing matrix is a source aliasing matrix that satisfies the preset objective function to take the minimum value; based on the target The source aliasing matrix and the reference matrix are used to obtain the reconstructed source signal; based on the reconstructed source signal, the random noise signal set, the preset number of amplification trials and the obtained subset of the original signal, the obtained and obtained The subset of amplified signals corresponds to the subset of original signals.

在一种可能的实现方式中，所述数据扩增模块，具体用于：In a possible implementation manner, the data amplification module is specifically used for:

将获取的所述原始信号子集包括的所述脑电信号单元求和，得到与获取的所述原始信号子集对应的第一脑电信号；summing the EEG signal units included in the acquired subset of original signals to obtain a first EEG signal corresponding to the acquired subset of original signals;

将所述第一脑电信号与所述实验试次数求商，得到与获取的所述原始信号子集对应的脑电信号单元平均值。Quotienting the first EEG signal and the number of experimental trials to obtain an average value of EEG signal units corresponding to the acquired subset of original signals.

在一种可能的实现方式中，所述目标函数为第一偏差矩阵的弗罗贝尼乌斯Frobenius范数；所述第一偏差矩阵为通过将源混叠矩阵与所述参考矩阵进行矩阵乘法，再与所述脑电信号单元平均值作差得到的。In a possible implementation, the objective function is the Frobenius norm of the first deviation matrix; the first deviation matrix is obtained by performing matrix multiplication of the source aliasing matrix and the reference matrix , and then obtained by making a difference with the average value of the EEG signal unit.

在一种可能的实现方式中，所述目标函数为将第二偏差矩阵的Frobenius范数，与源混叠矩阵的正则化范数进行求和得到的；所述第二偏差矩阵为通过将源混叠矩阵与所述参考矩阵进行矩阵乘法，再与所述脑电信号单元平均值作差得到的。In a possible implementation, the objective function is obtained by summing the Frobenius norm of the second deviation matrix and the regularization norm of the source aliasing matrix; the second deviation matrix is obtained by combining the source The aliasing matrix is obtained by performing matrix multiplication with the reference matrix, and then making a difference with the average value of the EEG signal unit.

在一种可能的实现方式中，所述数据扩增模块，具体用于：In a possible implementation manner, the data amplification module is specifically used for:

将所述目标源混叠矩阵与所述参考矩阵进行矩阵乘法，得到所述重构源信号。performing matrix multiplication on the target source aliasing matrix and the reference matrix to obtain the reconstructed source signal.

在一种可能的实现方式中，所述数据扩增模块，具体用于：In a possible implementation manner, the data amplification module is specifically used for:

基于所述扩增试次数和预设规则，生成随机噪声信号集；所述随机噪声信号集包含随机噪声信号单元的数量为所述扩增试次数；Based on the number of amplification trials and preset rules, a random noise signal set is generated; the number of random noise signal units contained in the random noise signal set is the number of amplification trials;

从所述随机噪声信号集中逐一获取所述随机噪声信号单元，每获取一个随机噪声信号单元，执行第一操作，以得到与获取的所述原始信号子集对应的一个扩增信号单元；所述第一操作，包括将获取的所述随机噪声信号单元，与所述重构源信号求和；Obtain the random noise signal units one by one from the random noise signal set, and perform a first operation for each random noise signal unit obtained to obtain an amplified signal unit corresponding to the obtained original signal subset; the A first operation comprising summing the acquired random noise signal unit with the reconstructed source signal;

将得到的各所述扩增信号单元，作为新的脑电信号单元加入到获取的所述原始信号子集，得到与获取的所述原始信号子集对应的所述扩增信号子集。The obtained amplified signal units are added as new EEG signal units to the acquired original signal subset, to obtain the amplified signal subset corresponding to the acquired original signal subset.

在一种可能的实现方式中，所述装置还包括：In a possible implementation manner, the device further includes:

脑电信号识别模块，用于基于所述扩增脑电信号集，训练所述脑机接口的脑电信号识别模型，得到训练后的所述脑电信号识别模型；获取待识别脑电信号，并利用训练后的所述脑电信号识别模型对所述待识别脑电信号进行识别，以得到脑电信号识别结果。The EEG signal recognition module is used to train the EEG signal recognition model of the brain-computer interface based on the amplified EEG signal set, and obtain the trained EEG signal recognition model; obtain the EEG signal to be recognized, And using the trained EEG signal identification model to identify the EEG signal to be identified, so as to obtain an EEG signal identification result.

第三方面，本申请实施例提供了一种计算机可读存储介质，所述计算机可读存储介质内存储有计算机程序，所述计算机程序被处理器执行时，实现第一方面任一项所述的方法。In a third aspect, an embodiment of the present application provides a computer-readable storage medium, where a computer program is stored in the computer-readable storage medium, and when the computer program is executed by a processor, it can realize any one of the first aspect. Methods.

第四方面，本申请实施例提供了一种电子设备，包括存储器和处理器，所述存储器上存储有可在所述处理器上运行的计算机程序，当所述计算机程序被所述处理器执行时，实现第一方面任一项所述的方法。In a fourth aspect, the embodiment of the present application provides an electronic device, including a memory and a processor, the memory stores a computer program that can run on the processor, when the computer program is executed by the processor , implement the method described in any one of the first aspect.

第五方面，本申请实施例提供了一种计算机程序产品，其包括计算机指令，所述计算机指令存储在计算机可读存储介质中；当计算机设备的处理器从所述计算机可读存储介质读取所述计算机指令时，所述处理器执行该计算机指令，使得所述计算机设备执行第一方面任一项所述的方法的步骤。In the fifth aspect, the embodiment of the present application provides a computer program product, which includes computer instructions, and the computer instructions are stored in a computer-readable storage medium; when a processor of a computer device reads from the computer-readable storage medium When the computer instructions are used, the processor executes the computer instructions, so that the computer device executes the steps of the method described in any one of the first aspect.

第二方面至第五方面任意一种实现方式所带来的技术效果可参见第一方面的实现方式所带来的技术效果，此处不再赘述。For the technical effects brought by any one of the implementations from the second aspect to the fifth aspect, please refer to the technical effects brought by the implementation of the first aspect, which will not be repeated here.

附图说明Description of drawings

为了更清楚地说明本申请实施例中的技术方案，下面将对实施例描述中所需要使用的附图作简要介绍，显而易见地，下面描述中的附图仅仅是本申请的一些实施例，对于本领域的普通技术人员来讲，在不付出创造性劳动性的前提下，还可以根据这些附图获得其他的附图。In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following will briefly introduce the drawings that need to be used in the description of the embodiments. Obviously, the drawings in the following description are only some embodiments of the present application. For Those of ordinary skill in the art can also obtain other drawings based on these drawings without any creative effort.

图1为本申请实施例提供的一种脑机接口训练数据扩增方法的流程示意图；FIG. 1 is a schematic flowchart of a brain-computer interface training data amplification method provided by an embodiment of the present application;

图2为本申请实施例提供的一种脑机接口训练数据扩增方法的确定脑电信号单元平均值的流程示意图；Fig. 2 is a schematic flow chart of determining the average value of EEG signal units in a brain-computer interface training data amplification method provided by an embodiment of the present application;

图3为本申请实施例提供的一种脑机接口训练数据扩增方法的得到扩增信号子集的流程示意图；FIG. 3 is a schematic flow diagram of obtaining a subset of amplified signals in a brain-computer interface training data amplification method provided in an embodiment of the present application;

图4为本申请实施例提供的另一种脑机接口训练数据扩增方法的流程示意图；FIG. 4 is a schematic flowchart of another brain-computer interface training data amplification method provided by the embodiment of the present application;

图5为本申请实施例提供的一种脑机接口训练数据扩增装置的结构框图；FIG. 5 is a structural block diagram of a brain-computer interface training data amplification device provided by an embodiment of the present application;

图6为本申请实施例提供的另一种脑机接口训练数据扩增装置的结构框图；Fig. 6 is a structural block diagram of another brain-computer interface training data amplification device provided by the embodiment of the present application;

图7为本申请实施例提供的一种电子设备的结构框图。FIG. 7 is a structural block diagram of an electronic device provided by an embodiment of the present application.

具体实施方式detailed description

为了使本申请的目的、技术方案和优点更加清楚，下面将结合附图对本申请作进一步地详细描述，显然，所描述的实施例仅仅是本申请一部分实施例，而不是全部的实施例。基于本申请中的实施例，本领域普通技术人员在没有做出创造性劳动前提下所获得的所有其它实施例，都属于本申请保护的范围。In order to make the purpose, technical solution and advantages of the application clearer, the application will be further described in detail below in conjunction with the accompanying drawings. Apparently, the described embodiments are only some of the embodiments of the application, not all of them. Based on the embodiments in this application, all other embodiments obtained by persons of ordinary skill in the art without making creative efforts belong to the scope of protection of this application.

脑机接口(brain-computer interface,BCI)为大脑与外部环境之间提供了一条无需依赖周围神经的直接通讯通路，能够替代、修复、增强、补充或改善正常的神经系统功能。典型的脑机接口系统通过检测特定的神经活动来解码脑信息，并将其转化为可输出的机器指令，从而实现大脑意图的直接表达。The brain-computer interface (BCI) provides a direct communication pathway between the brain and the external environment without relying on peripheral nerves, and can replace, repair, enhance, supplement or improve normal nervous system functions. A typical brain-computer interface system decodes brain information by detecting specific neural activities and converts it into outputable machine instructions, thereby realizing the direct expression of brain intentions.

相较于功能性近红外成像、脑磁图、皮层脑电等大脑信息获取方式来说，脑电(electroencephalogram,EEG)信号具有无创伤、价格低廉和高时间分辨率的优势，因此被广泛应用于脑机接口领域。然而，受限于脑电信号的跨模态、跨个体、跨时间等变异性，当前脑机接口解码算法大多采用个体校准模式，即在每次使用系统前均需要用户重新采集训练信号来构建符合当前脑电特点的分类模型。Compared with functional near-infrared imaging, magnetoencephalography, cortical EEG and other brain information acquisition methods, electroencephalogram (EEG) signals have the advantages of non-invasive, low cost and high temporal resolution, so they are widely used in the field of brain-computer interface. However, limited by the cross-modal, cross-individual, and cross-time variability of EEG signals, most of the current brain-computer interface decoding algorithms use the individual calibration mode, that is, before each use of the system, the user needs to re-acquire training signals to build A classification model that conforms to the current EEG characteristics.

相关技术的脑机接口识别方案，是通过实验采集来获得用于校准的EEG信号数据。这种方式耗时长，舒适性差，需要投入的人力成本高，因此很难获得足够的校准数据来训练分类模型。脑电信号解码过程中面临的数据不足问题，容易导致脑电信号识别的准确率较低，阻碍了脑机接口系统走向实际应用。The brain-computer interface recognition scheme of the related art obtains EEG signal data for calibration through experimental collection. This method takes a long time, is not comfortable, and requires high labor costs, so it is difficult to obtain enough calibration data to train the classification model. The lack of data in the process of EEG signal decoding can easily lead to low accuracy of EEG signal recognition, which hinders the practical application of brain-computer interface systems.

基于此，本申请实施例提供一种脑机接口训练数据扩增方法、装置、介质及电子设备。其中，该脑机接口训练数据扩增方法，包括：获取待处理脑电信号集包括的原始信号子集；其中，每获取一个原始信号子集，针对获取的原始信号子集执行以下操作，以得到用以构成扩增脑电信号集的扩增信号子集：基于获取的原始信号子集，确定与获取的原始信号子集对应的脑电信号单元平均值，其中任一原始信号子集包括数量为实验试次数的脑电信号单元；基于预设的参考矩阵和脑电信号单元平均值，得到目标源混叠矩阵；目标源混叠矩阵是满足使预设的目标函数取最小值的源混叠矩阵；基于目标源混叠矩阵和参考矩阵，得到重构源信号；基于重构源信号、随机噪声信号集、预设的扩增试次数和获取的原始信号子集，得到与获取的原始信号子集对应的扩增信号子集。该脑机接口训练数据扩增方法能够通过获取待处理脑电信号集包括的原始信号子集，并基于参考矩阵、随机噪声信号集和与获取的原始信号子集对应的脑电信号单元平均值进行脑电信号重构，获得符合脑电特性的扩增信号，从而实现对脑机接口训练数据中的待处理脑电信号集进行高效的数据扩增，可以提高脑机接口在小样本情况下的识别性能，从而降低系统的校准负担。Based on this, the embodiments of the present application provide a brain-computer interface training data augmentation method, device, medium, and electronic equipment. Wherein, the brain-computer interface training data amplification method includes: obtaining a subset of original signals included in the EEG signal set to be processed; wherein, each time a subset of original signals is obtained, the following operations are performed on the acquired subset of original signals, to Obtain the amplified signal subset used to form the amplified EEG signal set: based on the acquired original signal subset, determine the average value of the EEG signal unit corresponding to the acquired original signal subset, wherein any original signal subset includes The number of EEG signal units is the number of experimental trials; based on the preset reference matrix and the average value of EEG signal units, the target source aliasing matrix is obtained; the target source aliasing matrix is the source that satisfies the preset objective function to take the minimum value The aliasing matrix; based on the target source aliasing matrix and the reference matrix, the reconstructed source signal is obtained; based on the reconstructed source signal, the random noise signal set, the preset number of amplification trials and the acquired original signal subset, the obtained and acquired The subset of amplified signals corresponding to the subset of raw signals. The brain-computer interface training data amplification method can obtain the original signal subset included in the EEG signal set to be processed, and based on the reference matrix, the random noise signal set and the average value of the EEG signal unit corresponding to the acquired original signal subset Perform EEG signal reconstruction to obtain amplified signals that conform to EEG characteristics, so as to realize efficient data amplification of the EEG signal set to be processed in the brain-computer interface training data, which can improve the performance of the brain-computer interface in the case of small samples. recognition performance, thereby reducing the calibration burden of the system.

为了使本申请实施例的发明目的、技术方案和优点更加清楚，下面将结合附图对本申请作进一步地详细描述，显然，所描述的实施例仅仅是本申请一部份实施例，而不是全部的实施例。基于本申请中的实施例，本领域普通技术人员在没有做出创造性劳动前提下所获得的所有其它实施例，都属于本申请保护的范围。In order to make the invention objectives, technical solutions and advantages of the embodiments of the application clearer, the application will be further described in detail below in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the application, not all of them. the embodiment. Based on the embodiments in this application, all other embodiments obtained by persons of ordinary skill in the art without making creative efforts belong to the scope of protection of this application.

下面对本申请实施例提供的脑机接口训练数据扩增方法进行进一步的解释说明。本申请提供的脑机接口训练数据扩增方法，如图1所示，包括以下步骤：The method for augmenting brain-computer interface training data provided in the embodiment of the present application will be further explained below. The brain-computer interface training data amplification method provided in this application, as shown in Figure 1, includes the following steps:

步骤S101，获取待处理脑电信号集包括的原始信号子集。Step S101, acquiring a subset of original signals included in the EEG signal set to be processed.

其中，每获取一个原始信号子集，针对获取的原始信号子集执行以下的步骤的操作，以得到用以构成扩增脑电信号集的扩增信号子集。Wherein, each time an original signal subset is acquired, the following steps are performed on the acquired original signal subset, so as to obtain the amplified signal subset used to form the amplified EEG signal set.

具体实施时，待处理脑电信号集可以是输入的脑机接口的预处理后信号；待处理脑电信号集包括的原始信号子集，可以是从输入的脑机接口的预处理后信号中确定的一个类别的训练数据。During specific implementation, the EEG signal set to be processed can be the preprocessed signal of the input brain-computer interface; the original signal subset included in the EEG signal set to be processed can be the preprocessed signal from the input brain-computer interface Identify a class of training data.

令输入的预处理后信号表示为一个四维张量

其中，N_c代表脑电信号的导联数量，N_s代表脑电信号的采样点数，N_t代表实验试次数，表征相同类别下重复实验采集的试次数量，N_f代表总的类别数量，

代表实数集。获取待处理脑电信号集包括的原始信号子集，可以是从四维张量

中，确定第n个类别下的训练信号，表示为

下面以待处理脑电信号集是

获取的原始信号子集是

为例，进行说明。Let the input preprocessed signal be expressed as a four-dimensional tensor

Among them, N_c represents the number of leads of EEG signals, N_s represents the number of sampling points of EEG signals, N_t represents the number of experimental trials, representing the number of trials collected in repeated experiments under the same category, N_f represents the total number of categories,

represents the set of real numbers. Obtain the original signal subset included in the EEG signal set to be processed, which can be from a four-dimensional tensor

, determine the training signal under the nth category, expressed as

The following set of EEG signals to be processed is

The subset of raw signals obtained is

As an example, for explanation.

步骤S102，基于获取的原始信号子集，确定与获取的原始信号子集对应的脑电信号单元平均值；任一原始信号子集包括数量为实验试次数的脑电信号单元。Step S102, based on the acquired original signal subset, determine the average value of EEG signal units corresponding to the acquired original signal subset; any original signal subset includes EEG signal units whose number is the number of experimental trials.

在一种可选的实施例中，基于获取的原始信号子集，确定与获取的原始信号子集对应的脑电信号单元平均值的过程，如图2所示，可以包括以下步骤：In an optional embodiment, based on the acquired original signal subset, the process of determining the average value of the EEG signal unit corresponding to the acquired original signal subset, as shown in Figure 2, may include the following steps:

步骤S201，将获取的原始信号子集包括的脑电信号单元求和，得到与获取的原始信号子集对应的第一脑电信号。Step S201, summing the EEG signal units included in the acquired original signal subset to obtain a first EEG signal corresponding to the acquired original signal subset.

示例性地，基于获取的原始信号子集

确定与获取的原始信号子集

对应的脑电信号单元平均值的过程，可以是将获取的原始信号子集

包括的脑电信号单元

求和，得到与获取的原始信号子集对应的第一脑电信号

Exemplarily, based on the obtained subset of raw signals

Determining and acquiring a subset of the original signal

The process of corresponding to the average value of the EEG signal unit can be a subset of the original signal that will be acquired

EEG unit included

Summing to obtain the first EEG signal corresponding to the acquired subset of original signals

步骤S202，将第一脑电信号与实验试次数求商，得到与获取的原始信号子集对应的脑电信号单元平均值。Step S202, calculating the quotient of the first EEG signal and the number of experimental trials to obtain the average value of the EEG signal unit corresponding to the acquired original signal subset.

示例性地，将第一脑电信号

与实验试次数N_t求商，得到与获取的原始信号子集

对应的脑电信号单元平均值

Exemplarily, the first EEG signal

Compute the quotient with the number of experimental trials N_t , and obtain the original signal subset obtained with

Corresponding average value of EEG signal unit

本申请的实施例中，确定第n个类别下的训练信号

之后，可以依据下面的公式计算第n个类别的下的脑电信号单元平均值

In the embodiment of the present application, the training signal under the nth category is determined

After that, the average value of the EEG signal unit under the nth category can be calculated according to the following formula

其中，N_t为实验试次数，表征相同类别下重复实验采集的试次数量。Among them, N_t is the number of experimental trials, representing the number of trials collected in repeated experiments under the same category.

步骤S103，基于预设的参考矩阵和脑电信号单元平均值，得到目标源混叠矩阵；目标源混叠矩阵是满足使预设的目标函数取最小值的源混叠矩阵。Step S103, based on the preset reference matrix and the average value of the EEG signal unit, the target source aliasing matrix is obtained; the target source aliasing matrix is a source aliasing matrix satisfying the minimum value of the preset target function.

本申请的一种实施例中，预设的参考矩阵可以为第n个类别下已知频率f_n的正弦和余弦序列

如下式所示：In an embodiment of the present application, the preset reference matrix may be a sine and cosine sequence of a known frequency f_n in the nth category

As shown in the following formula:

其中，N_h代表参考矩阵所包含的谐波次数；Among them, N_h represents the harmonic order contained in the reference matrix;

F_s代表信号的采样率。F_s represents the sampling rate of the signal.

具体实施时，基于以上的预设的参考矩阵Y_n和脑电信号单元平均值

得到目标源混叠矩阵

目标源混叠矩阵

是满足使预设的目标函数取最小值的源混叠矩阵。During specific implementation, based on the above preset reference matrix Y_n and the average value of the EEG signal unit

Get the target source aliasing matrix

target source aliasing matrix

is the source aliasing matrix that satisfies the minimum value of the preset objective function.

本申请的一种实施例中，目标函数为第一偏差矩阵的弗罗贝尼乌斯Frobenius范数；第一偏差矩阵为通过将源混叠矩阵与参考矩阵进行矩阵乘法，再与脑电信号单元平均值作差得到的。In one embodiment of the present application, the objective function is the Frobenius norm of the first deviation matrix; obtained by subtracting the unit mean.

示例性地，第一偏差矩阵，是通过将源混叠矩阵Φ与参考矩阵Y_n进行矩阵乘法，得到ΦY_n，再将得到的ΦY_n与脑电信号单元平均值

作差得到的。目标函数为第一偏差矩阵的弗罗贝尼乌斯Frobenius范数。Exemplarily, the first deviation matrix is obtained by matrix multiplication of the source aliasing matrix Φ and the reference matrix Y_n to obtain ΦY_n , and then the obtained ΦY_n and the average value of the EEG signal unit

Got it by doing bad things. The objective function is the Frobenius norm of the first deviation matrix.

该实施例中，可以依据下面的公式，估计源混叠矩阵

确定满足使预设的目标函数取最小值的源混叠矩阵，以得到目标源混叠矩阵。In this embodiment, the source aliasing matrix can be estimated according to the following formula

A source aliasing matrix that satisfies the minimum value of the preset objective function is determined to obtain a target source aliasing matrix.

其中，argmin函数用于搜索使目标函数最小的变量值；Among them, the argmin function is used to search for the variable value that minimizes the objective function;

‖‖_F代表矩阵的Frobenius范数；‖‖_F represents the Frobenius norm of the matrix;

为脑电信号单元平均值；

is the average value of the EEG signal unit;

为目标源混叠矩阵，表征对混叠矩阵Φ的估计。

is the target source aliasing matrix, representing the estimation of the aliasing matrix Φ.

需要说明的是，本申请的实施例只是以正余弦序列为例对参考矩阵Y_n进行解释性说明，但参考矩阵的构建包括但不限于上述形式，应依据不同类别的脑电信号特点来确定，本申请不做具体限定。It should be noted that the embodiment of the present application only uses the sine-cosine sequence as an example to explain the reference matrix Y_n , but the construction of the reference matrix includes but is not limited to the above-mentioned forms, which should be determined according to the characteristics of different types of EEG signals , which is not specifically limited in this application.

本申请的一种实施例中，目标函数为将第二偏差矩阵的Frobenius范数，与源混叠矩阵的正则化范数进行求和得到的；第二偏差矩阵为通过将源混叠矩阵与参考矩阵进行矩阵乘法，再与脑电信号单元平均值作差得到的。In one embodiment of the present application, the objective function is obtained by summing the Frobenius norm of the second deviation matrix and the regularization norm of the source aliasing matrix; the second deviation matrix is obtained by combining the source aliasing matrix with The reference matrix is multiplied by matrix, and then obtained by making a difference with the average value of the EEG signal unit.

示例性地，第二偏差矩阵，是通过将源混叠矩阵Φ与参考矩阵Y_n进行矩阵乘法，得到ΦY_n，再将得到的ΦY_n与脑电信号单元平均值

作差得到的。目标函数为第二偏差矩阵的弗罗贝尼乌斯Frobenius范数，与源混叠矩阵的正则化范数进行求和得到的。Exemplarily, the second deviation matrix is obtained by multiplying the source aliasing matrix Φ and the reference matrix Y_n to obtain ΦY_n , and then combining the obtained ΦY_n with the average value of the EEG signal unit

Got it by doing bad things. The objective function is obtained by summing the Frobenius norm of the second bias matrix and the regularization norm of the source aliasing matrix.

该实施例中，假定第二偏差矩阵为

正则化范数是L1范数，可以依据下面的公式，估计源混叠矩阵

确定满足使预设的目标函数取最小值的源混叠矩阵，以得到目标源混叠矩阵。In this embodiment, it is assumed that the second deviation matrix is

The regularization norm is the L1 norm, and the source aliasing matrix can be estimated according to the following formula

A source aliasing matrix that satisfies the minimum value of the preset objective function is determined to obtain a target source aliasing matrix.

其中，argmin函数用于搜索使目标函数最小的变量值；Among them, the argmin function is used to search for the variable value that minimizes the objective function;

‖‖_F代表矩阵的Frobenius范数；‖‖_F represents the Frobenius norm of the matrix;

‖‖₁代表矩阵的L1范数；‖‖₁ represents the L1 norm of the matrix;

为脑电信号单元平均值；

is the average value of the EEG signal unit;

为目标源混叠矩阵，表征对混叠矩阵Φ的估计。

is the target source aliasing matrix, representing the estimation of the aliasing matrix Φ.

为目标函数。

is the objective function.

需要说明的是，本申请的实施例只是以L1范数为例对目标函数包括的正则化范数进行解释性说明，但目标函数的构建包括但不限于上述形式，应依据不同类别的脑电信号特点来确定，本申请不做具体限定。例如，在其他一些实施例中，正则化范数还可以是L2正则化范数。It should be noted that the embodiment of the present application only uses the L1 norm as an example to explain the regularization norm included in the objective function, but the construction of the objective function includes but is not limited to the above-mentioned forms, and should be based on different types of EEG Signal characteristics are determined, and this application does not make specific limitations. For example, in some other embodiments, the regularization norm may also be an L2 regularization norm.

步骤S104，基于目标源混叠矩阵和参考矩阵，得到重构源信号。Step S104, obtain the reconstructed source signal based on the target source aliasing matrix and the reference matrix.

在一种可选的实施例中，基于目标源混叠矩阵和参考矩阵，得到重构源信号，具体为将目标源混叠矩阵与参考矩阵进行矩阵乘法，得到重构源信号。In an optional embodiment, the reconstructed source signal is obtained based on the target source aliasing matrix and the reference matrix, specifically performing matrix multiplication between the target source aliasing matrix and the reference matrix to obtain the reconstructed source signal.

示例性地，基于确定的参考矩阵Y_n和目标源混叠矩阵

通过以下公式计算重建的重构源信号

Exemplarily, based on the determined reference matrix Y_n and target source aliasing matrix

The reconstructed reconstructed source signal is calculated by the following formula

其中，in,

Y_n为参考矩阵；Y_n is a reference matrix;

为目标源混叠矩阵。

is the destination source aliasing matrix.

步骤S105，基于重构源信号、随机噪声信号集、预设的扩增试次数和获取的原始信号子集，得到与获取的原始信号子集对应的扩增信号子集。Step S105, based on the reconstructed source signal, the random noise signal set, the preset number of amplification trials, and the acquired original signal subset, obtain an amplified signal subset corresponding to the acquired original signal subset.

在一种可选的实施例中，基于重构源信号、随机噪声信号集、预设的扩增试次数和获取的原始信号子集，得到与获取的原始信号子集对应的扩增信号子集的过程，如图3所示，可以通过以下步骤实现：In an optional embodiment, based on the reconstructed source signal, the random noise signal set, the preset number of amplification trials and the acquired original signal subset, the amplified semaphore corresponding to the acquired original signal subset is obtained The process of set, as shown in Figure 3, can be realized through the following steps:

步骤S301，基于扩增试次数和预设规则，生成随机噪声信号集；随机噪声信号集包含随机噪声信号单元的数量为扩增试次数。Step S301, based on the number of amplification trials and preset rules, a random noise signal set is generated; the number of random noise signal units contained in the random noise signal set is the number of amplification trials.

本申请的一些实施例中，预设规则可以是用于生成满足均值为0，协方差为Σ的多元高斯分布的随机噪声信号集的规则，该随机噪声信号集包含随机噪声信号单元的数量为扩增试次数。In some embodiments of the present application, the preset rule may be a rule for generating a random noise signal set satisfying a mean value of 0 and a multivariate Gaussian distribution with a covariance of Σ, the random noise signal set includes a random noise signal unit whose number is Increase the number of trials.

示例性地，基于扩增试次数和预设规则，生成随机噪声信号集，可以是基于扩增试次数N_a和预设规则，生成随机噪声信号集

其中N_k满足均值为0，协方差为Σ的多元高斯分布。Exemplarily, based on the number of amplification trials and preset rules, a random noise signal set is generated, which may be based on the number of amplification trials N_a and preset rules to generate a random noise signal set

Among them, N_k satisfies the multivariate Gaussian distribution with mean value 0 and covariance Σ.

步骤S302，从随机噪声信号集中逐一获取随机噪声信号单元，每获取一个随机噪声信号单元，执行第一操作，以得到与获取的原始信号子集对应的一个扩增信号单元；第一操作，包括将获取的随机噪声信号单元，与重构源信号求和。In step S302, random noise signal units are acquired one by one from the random noise signal set, and each time a random noise signal unit is acquired, a first operation is performed to obtain an amplified signal unit corresponding to the acquired original signal subset; the first operation includes Sum the acquired random noise signal units with the reconstructed source signal.

示例性地，从随机噪声信号集

中逐一获取随机噪声信号单元

每获取一个随机噪声信号单元N_k，执行第一操作，以得到与获取的原始信号子集X_n对应的一个扩增信号单元

第一操作，包括将获取的随机噪声信号单元N_k，与重构源信号S_n求和。Exemplarily, from a random noise signal set

Get random noise signal units one by one in

Every time a random noise signal unit N_k is acquired, the first operation is performed to obtain an amplified signal unit corresponding to the acquired original signal subset X_n

The first operation consists in summing the acquired random noise signal unit N_k with the reconstructed source signal S_n .

也即，与获取的原始信号子集对应的扩增信号单元

可以通过下式确定：That is, the amplified signal unit corresponding to the acquired original signal subset

It can be determined by the following formula:

其中，

代表第n个类别下的第k个扩增信号，in,

Represents the kth amplification signal under the nth category,

S_n代表第n个类别下的重构源信号，S_n represents the reconstructed source signal under the nth category,

N_a代表扩增试次数。N_a represents the number of amplification trials.

步骤S303，将得到的各扩增信号单元，作为新的脑电信号单元加入到获取的原始信号子集，得到与获取的原始信号子集对应的扩增信号子集。Step S303, adding each obtained amplified signal unit as a new EEG signal unit to the acquired original signal subset to obtain an amplified signal subset corresponding to the acquired original signal subset.

示例性地，将得到的各扩增信号单元

作为新的脑电信号单元加入到获取的原始信号子集

得到与获取的原始信号子集

对应的扩增信号子集

Exemplarily, each amplified signal unit obtained

Added to the acquired original signal subset as a new EEG signal unit

get and acquire a subset of the original signal

Corresponding subset of amplified signals

通过对待处理脑电信号集

包含的N_f个类别的原始信号子集

按类别逐一进行上述的信号扩增，可以得到每个原始信号子集对应的扩增信号子集，进而得到与待处理脑电信号集

对应的扩增脑电信号集

Through the EEG signal set to be processed

A subset of the original signal consisting of N_f categories

By performing the above-mentioned signal amplification one by one by category, the amplified signal subset corresponding to each original signal subset can be obtained, and then the EEG signal set to be processed can be obtained.

Corresponding amplified EEG signal set

本申请的实施例还提供另一种脑机接口训练数据扩增方法。如图4所示，包括以下步骤：Embodiments of the present application also provide another brain-computer interface training data augmentation method. As shown in Figure 4, the following steps are included:

步骤S401，获取待处理脑电信号集包括的原始信号子集。Step S401, acquiring a subset of original signals included in the EEG signal set to be processed.

其中，每获取一个原始信号子集，针对获取的原始信号子集执行以下的步骤S402～S405的操作，以得到用以构成扩增脑电信号集的扩增信号子集。Wherein, each time an original signal subset is acquired, the following steps S402-S405 are performed on the acquired original signal subset, so as to obtain the amplified signal subset used to form the amplified EEG signal set.

具体实施时，待处理脑电信号集可以是输入的脑机接口的预处理后信号；待处理脑电信号集包括的原始信号子集，可以是从输入的脑机接口的预处理后信号中确定的一个类别的训练数据。During specific implementation, the EEG signal set to be processed can be the preprocessed signal of the input brain-computer interface; the original signal subset included in the EEG signal set to be processed can be the preprocessed signal from the input brain-computer interface Identify a class of training data.

令输入的预处理后信号表示为一个四维张量

其中，N_c代表脑电信号的导联数量，N_s代表脑电信号的采样点数，N_t代表实验试次数，表征相同类别下重复实验采集的试次数量，N_f代表总的类别数量，

代表实数集。获取待处理脑电信号集包括的原始信号子集，可以是从四维张量

中，确定第n个类别下的训练信号，表示为

下面以待处理脑电信号集是

获取的原始信号子集是

为例，进行说明。每获取一个原始信号子集X_n，针对获取的原始信号子集X_n执行以下的步骤S402～S405的操作，以得到与获取的原始信号子集X_n对应的扩增信号子集

从而得到与待处理脑电信号集

对应的扩增脑电信号集

Let the input preprocessed signal be expressed as a four-dimensional tensor

Among them, N_c represents the number of leads of EEG signals, N_s represents the number of sampling points of EEG signals, N_t represents the number of experimental trials, representing the number of trials collected in repeated experiments under the same category, N_f represents the total number of categories,

represents the set of real numbers. Obtain the original signal subset included in the EEG signal set to be processed, which can be from a four-dimensional tensor

, determine the training signal under the nth category, expressed as

The following set of EEG signals to be processed is

The subset of raw signals obtained is

As an example, for explanation. Each time an original signal subset X_n is acquired, the following steps S402-S405 are performed on the acquired original signal subset X_n to obtain an amplified signal subset corresponding to the acquired original signal subset X_n

So as to get the EEG signal set to be processed

Corresponding amplified EEG signal set

步骤S402，基于获取的原始信号子集，确定与获取的原始信号子集对应的脑电信号单元平均值；任一原始信号子集包括数量为实验试次数的脑电信号单元。Step S402, based on the acquired original signal subset, determine the average value of EEG signal units corresponding to the acquired original signal subset; any original signal subset includes EEG signal units whose number is the number of experimental trials.

步骤S403，基于预设的参考矩阵和脑电信号单元平均值，得到目标源混叠矩阵；目标源混叠矩阵是满足使预设的目标函数取最小值的源混叠矩阵。Step S403, based on the preset reference matrix and the average value of the EEG signal unit, the target source aliasing matrix is obtained; the target source aliasing matrix is a source aliasing matrix satisfying the minimum value of the preset target function.

步骤S404，基于目标源混叠矩阵和参考矩阵，得到重构源信号。Step S404, obtain the reconstructed source signal based on the target source aliasing matrix and the reference matrix.

步骤S405，基于重构源信号、随机噪声信号集、预设的扩增试次数和获取的原始信号子集，得到与获取的原始信号子集对应的扩增信号子集。Step S405, based on the reconstructed source signal, the random noise signal set, the preset number of amplification trials, and the acquired original signal subset, obtain an amplified signal subset corresponding to the acquired original signal subset.

步骤S406，基于扩增脑电信号集，训练脑机接口的脑电信号识别模型，得到训练后的脑电信号识别模型。Step S406, based on the amplified EEG signal set, train the EEG signal recognition model of the brain-computer interface, and obtain the trained EEG signal recognition model.

示例性地，基于扩增脑电信号集

训练脑机接口的脑电信号识别模型，得到训练后的脑电信号识别模型。Exemplarily, based on the augmented EEG signal set

The EEG signal recognition model of the brain-computer interface is trained to obtain the trained EEG signal recognition model.

步骤S407，获取待识别脑电信号，并利用训练后的脑电信号识别模型对待识别脑电信号进行识别，以得到脑电信号识别结果。Step S407, acquiring the EEG signal to be recognized, and using the trained EEG signal recognition model to recognize the EEG signal to be recognized, so as to obtain the EEG signal recognition result.

该实施例的方法，实现对脑机接口训练数据中的待处理脑电信号集进行高效的数据扩增，并根据数据扩增得到的扩增脑电信号集进行脑电信号识别模型的训练，可以提高脑机接口在小样本情况下的识别性能，从而降低系统的校准负担。The method of this embodiment realizes the efficient data amplification of the EEG signal set to be processed in the brain-computer interface training data, and performs the training of the EEG signal recognition model according to the amplified EEG signal set obtained by the data amplification, It can improve the recognition performance of the brain-computer interface in the case of small samples, thereby reducing the calibration burden of the system.

在本申请的一种实施例中，脑机接口为频-相编码脑机接口。如表1所示，该频-相编码脑机接口的每个指令由不同频率、不同相位的闪烁刺激进行编码，其诱发的脑电信号包含对应频率的基波和若干谐波成分。In one embodiment of the present application, the brain-computer interface is a frequency-phase coded brain-computer interface. As shown in Table 1, each instruction of the frequency-phase encoded brain-computer interface is encoded by flickering stimuli of different frequencies and phases, and the evoked EEG signals contain fundamental waves of corresponding frequencies and several harmonic components.

表1Table 1

该实施例中，应用滤波器组任务相关成分分析(filter bank task relatedcomponent analysis,FBTRCA)算法对脑电信号进行分类识别，其中滤波器组分析要求将信号按不同子带对信号进行滤波，从而充分利用脑电信号中的谐波信息。该频-相编码脑机接口的脑机接口训练数据扩增方法，包括以下步骤：In this embodiment, the filter bank task related component analysis (FBTRCA) algorithm is used to classify and identify EEG signals, wherein the filter bank analysis requires the signal to be filtered according to different subbands, so as to fully Harmonic information in EEG signals is exploited. The brain-computer interface training data amplification method of the frequency-phase encoding brain-computer interface comprises the following steps:

步骤A01，获取待处理脑电信号集包括的原始信号子集。Step A01, obtaining a subset of original signals included in the EEG signal set to be processed.

其中，每获取一个原始信号子集，针对获取的原始信号子集执行以下的步骤A02～A05的操作，以得到用以构成扩增脑电信号集的扩增信号子集。Wherein, each time an original signal subset is acquired, the following operations of steps A02 to A05 are performed on the acquired original signal subset, so as to obtain the amplified signal subset used to form the amplified EEG signal set.

具体实施时，频-相编码脑机接口的脑电信号先进行滤波器组分解，得到与多个子带对应的训练信号，每个子带对应的训练信号为一个待处理脑电信号集。During specific implementation, the EEG signals of the frequency-phase encoded BCI are decomposed by a filter bank to obtain training signals corresponding to multiple sub-bands, and the training signals corresponding to each sub-band are a set of EEG signals to be processed.

示例性地，令本实施例中输入的预处理后脑电信号包含Pz,PO5,PO3,POz,PO4,PO6,O1,Oz,和O2共9个导联，信号采样率为F_s为250Hz，信号时长为1s，每个事件各采集5个试次作为训练数据。因此，该脑电信号可以表示为一个四维张量

其中N_c为9，N_s为250，N_t为5，N_f为40，

代表实数集。Exemplarily, let the preprocessed EEG signal input in this embodiment include a total of 9 leads Pz, PO5, PO3, POz, PO4, PO6, O1, Oz, and O2, and the signal sampling rate_Fs is 250 Hz, The signal duration is 1s, and 5 trials are collected for each event as training data. Therefore, the EEG signal can be expressed as a four-dimensional tensor

where N_c is 9, N_s is 250, N_t is 5, N_f is 40,

represents the set of real numbers.

对脑电信号χ’进行滤波器组分解，假定滤波器子带数量N_b设置为5，带通滤波器上限截止频率均为90Hz，下限截止频率分别为8Hz，16Hz，24Hz，32Hz和40Hz。经过分解后，获得不同子带下训练信号

作为待处理脑电信号集。逐一选取各个子带的训练信号χ’^m，每选取一个子带的训练信号χ’^m，获取选取的训练信号χ’^m包括的原始信号子集

也即第m个子带、第n个类别的训练信号。每获取一个原始信号子集

针对获取的原始信号子集

执行以下的步骤A02～A05的操作，以得到用以构成扩增脑电信号集的扩增信号子集。The EEG signal χ' is decomposed into a filter bank, assuming that the number of filter subbands N_b is set to 5, the upper limit cutoff frequency of the bandpass filter is 90Hz, and the lower limit cutoff frequency is 8Hz, 16Hz, 24Hz, 32Hz and 40Hz respectively. After decomposition, the training signals under different sub-bands are obtained

as a set of EEG signals to be processed. Select the training signal χ'^m of each sub-band one by one, and each time a training signal χ'^{m of a sub-band is selected, the original signal subset included in the selected training signal χ' mis} obtained

That is, the training signal of the mth subband and the nth category. Each acquisition of a subset of the original signal

For the subset of raw signals acquired

The following steps A02-A05 are performed to obtain the amplified signal subset used to form the amplified EEG signal set.

步骤A02，基于获取的原始信号子集，确定与获取的原始信号子集对应的脑电信号单元平均值；任一原始信号子集包括数量为实验试次数的脑电信号单元。Step A02, based on the acquired original signal subset, determine the average value of EEG signal units corresponding to the acquired original signal subset; any original signal subset includes EEG signal units whose number is the number of experimental trials.

示例性地，获取选取的训练信号χ’^m包括的原始信号子集

之后，确定第m个子带为当前子带，将N_t个试次数据进行叠加平均，获得第n个类别、第m个子带下的脑电信号单元平均值

Exemplarily, the original signal subset included in the selected training signal x'^m is obtained

After that, the mth subband is determined as the current subband, and the N_t trial data are superimposed and averaged to obtain the average value of the EEG signal unit under the nth category and the mth subband

步骤A03，基于预设的参考矩阵和脑电信号单元平均值，得到目标源混叠矩阵；目标源混叠矩阵是满足使预设的目标函数取最小值的源混叠矩阵。In step A03, a target source aliasing matrix is obtained based on the preset reference matrix and the average value of the EEG signal unit; the target source aliasing matrix is a source aliasing matrix satisfying the minimum value of the preset target function.

具体实施时，该实施例中不同类别的脑电信号包含不同刺激频率的基波和谐波成分，因此，参考矩阵的构建可以由对应的频率的正余弦序列构成。由于应用了滤波器组分解，因此谐波次数的范围可以根据刺激频率f_n与不同子带的下限截止频率

共同决定，参考矩阵如下式所示：During specific implementation, different types of EEG signals in this embodiment contain fundamental and harmonic components of different stimulation frequencies, therefore, the construction of the reference matrix may be composed of sine and cosine sequences of corresponding frequencies. Due to the applied filter bank decomposition, the range of harmonic orders can be adjusted according to the stimulus frequency_fn with the lower cut-off frequency of different subbands

Co-determined, the reference matrix is as follows:

其中，N_he为最大谐波次数，此处设置为5；Among them, N_he is the maximum harmonic order, which is set to 5 here;

N_hs为最小谐波次数，是满足

的最小整数。N_hs is the minimum harmonic order, which is satisfied

The smallest integer of .

根据参考矩阵

和步骤A02中获得的脑电信号单元平均值

求解目标函数，获得目标源混叠矩阵

目标源混叠矩阵

可以是通过下面的公式得到的：According to the reference matrix

and the average value of the EEG signal unit obtained in step A02

Solve the objective function to obtain the target-source aliasing matrix

target source aliasing matrix

It can be obtained by the following formula:

其中，argmin函数用于搜索使目标函数最小的变量值；Among them, the argmin function is used to search for the variable value that minimizes the objective function;

‖‖_F代表矩阵的Frobenius范数；‖‖_F represents the Frobenius norm of the matrix;

为第n个类别、第m个子带下的脑电信号单元平均值；

is the average value of the EEG signal unit under the nth category and the mth subband;

为目标源混叠矩阵，表征对混叠矩阵Φ的估计。

is the target source aliasing matrix, representing the estimation of the aliasing matrix Φ.

步骤A04，基于目标源混叠矩阵和参考矩阵，得到重构源信号。Step A04, obtain the reconstructed source signal based on the target source aliasing matrix and the reference matrix.

示例性地，依据参考矩阵

和目标源混叠矩阵

计算第n个类别、第m个子带下的重构源信号

重构源信号

可以通过下式确定：Exemplarily, according to the reference matrix

and target source aliasing matrix

Calculate the reconstructed source signal under the nth category and the mth subband

Reconstruct the source signal

It can be determined by the following formula:

步骤A05，基于重构源信号、随机噪声信号集、预设的扩增试次数和获取的原始信号子集，得到与获取的原始信号子集对应的扩增信号子集。Step A05, based on the reconstructed source signal, the random noise signal set, the preset number of amplification trials and the acquired original signal subset, obtain an amplified signal subset corresponding to the acquired original signal subset.

具体实施时，向重构源信号

中加入随机噪声，获得与获取的原始信号子集

对应的N_a个扩增信号单元，将得到的各扩增信号单元，作为新的脑电信号单元加入到获取的原始信号子集

得到与获取的原始信号子集

对应的扩增信号子集

其中，m＝1,2,…N_b。During specific implementation, signal to the reconstructed source

Adding random noise to get a subset of the original signal obtained with

Corresponding N_a amplified signal units, the obtained amplified signal units are added to the obtained original signal subset as new EEG signal units

get and acquire a subset of the original signal

Corresponding subset of amplified signals

Wherein, m=1, 2, . . . N_b .

步骤A06，基于扩增脑电信号集，训练脑机接口的脑电信号识别模型，得到训练后的脑电信号识别模型。Step A06, based on the amplified EEG signal set, train the EEG signal recognition model of the brain-computer interface, and obtain the trained EEG signal recognition model.

示例性地，利用不同子带下的新训练数据γ^m求解FBTRCA算法的模型参数，例如空间滤波器等，用于后续脑机接口系统的分类识别。Exemplarily, the new training data γ^m under different sub-bands is used to solve the model parameters of the FBTRCA algorithm, such as spatial filters, etc., for the subsequent classification and identification of the brain-computer interface system.

步骤A07，获取待识别脑电信号，并利用训练后的脑电信号识别模型对待识别脑电信号进行识别，以得到脑电信号识别结果。Step A07, acquiring the EEG signal to be recognized, and using the trained EEG signal recognition model to recognize the EEG signal to be recognized, so as to obtain the EEG signal recognition result.

在本申请的另外一种实施例中，脑机接口为频-空编码脑机接口。如表2所示，该频-空编码脑机接口的每个指令由不同空间位置的两个不同目标频率进行联合编码，其诱发的脑电信号同时包含两个频率下的基波及谐波成分，且不同频率具有明显的空间分布差异。In another embodiment of the present application, the brain-computer interface is a frequency-space coded brain-computer interface. As shown in Table 2, each command of the frequency-space coded BCI is jointly coded by two different target frequencies at different spatial positions, and the EEG signal induced by it contains both fundamental and harmonic components at the two frequencies , and different frequencies have obvious spatial distribution differences.

表2Table 2

该实施例中，在本实施例中，应用EEG-net神经网络算法对脑电信号进行分类识别。该频-空编码脑机接口的脑机接口训练数据扩增方法，包括以下步骤：In this embodiment, in this embodiment, the EEG-net neural network algorithm is used to classify and identify EEG signals. The brain-computer interface training data amplification method of the frequency-space coding brain-computer interface comprises the following steps:

步骤B01，获取待处理脑电信号集包括的原始信号子集。Step B01, obtaining a subset of original signals included in the EEG signal set to be processed.

其中，每获取一个原始信号子集，针对获取的原始信号子集执行以下的步骤B02～B05的操作，以得到用以构成扩增脑电信号集的扩增信号子集。Wherein, each time an original signal subset is acquired, the following operations of steps B02 to B05 are performed on the acquired original signal subset, so as to obtain the amplified signal subset used to form the amplified EEG signal set.

示例性地，令本实施例中输入的预处理后脑电信号包含Pz,P1,P2,P3,P4,P5,P6,P7,P8,PO3,PO4,PO5,PO6,PO7,PO8,CB1,CB2,POz,O1,Oz和O2共21个导联，信号采样率为F_s为250Hz，信号时长为0.5s，每个事件各采集20个试次作为训练数据。因此，该脑电信号可以表示为一个四维张量

其中N_c为21；N_s为250；N_t为20；N_f为15，表征总的类别数量；

代表实数集。将四维张量χ”作为待处理脑电信号集。逐一获取待处理脑电信号集χ”包括的原始信号子集χ”_n，其中，每个原始信号子集χ”_n是一个类别的训练信号。每获取一个原始信号子集χ”_n，针对获取的原始信号子集χ”_n执行以下的步骤B02～B05的操作，以得到用以构成扩增脑电信号集的扩增信号子集。Exemplarily, let the preprocessed EEG signal input in this embodiment include Pz, P1, P2, P3, P4, P5, P6, P7, P8, PO3, PO4, PO5, PO6, PO7, PO8, CB1, CB2 , POz, O1, Oz and O2 have a total of 21 leads, the signal sampling rate F_s is 250Hz, the signal duration is 0.5s, and 20 trials are collected for each event as training data. Therefore, the EEG signal can be expressed as a four-dimensional tensor

Among them, N_c is 21; N_s is 250; N_t is 20; N_f is 15, representing the total number of categories;

represents the set of real numbers. The four-dimensional tensor χ" is used as the EEG signal set to be processed. The original signal subsets χ"_n included in the EEG signal set χ" to be processed are obtained one by one, wherein each original signal subset χ"_n is a category of training Signal. Each time an original signal subset χ"_n is obtained, the following steps B02-B05 are performed on the obtained original signal subset χ"_n to obtain the amplified signal subset used to form the amplified EEG signal set.

步骤B02，基于获取的原始信号子集，确定与获取的原始信号子集对应的脑电信号单元平均值；任一原始信号子集包括数量为实验试次数的脑电信号单元。Step B02, based on the acquired original signal subset, determine the average value of EEG signal units corresponding to the acquired original signal subset; any original signal subset includes EEG signal units whose number is the number of experimental trials.

示例性地，针对获取的第n个类别的原始信号子集χ”_n，将N_t个试次数据进行叠加平均，获得第n个类别的脑电信号单元平均值

Exemplarily, for the acquired original signal subset χ"_n of the nth category, N_t trial data are superimposed and averaged to obtain the average value of the EEG signal unit of the nth category

步骤B03，基于预设的参考矩阵和脑电信号单元平均值，得到目标源混叠矩阵；目标源混叠矩阵是满足使预设的目标函数取最小值的源混叠矩阵。Step B03, based on the preset reference matrix and the average value of the EEG signal unit, the target source aliasing matrix is obtained; the target source aliasing matrix is a source aliasing matrix satisfying the minimum value of the preset target function.

具体实施时，该实施例中不同类别的脑电信号由两个不同空间位置的闪烁刺激共同诱发，因此，参考矩阵的构建可以是同时包含所有频率的正余弦序列。令第n个类别下的左、右频率分别为f_n,1和f_n,2，则参考矩阵

可以表示为下式(1)、式(2)：During specific implementation, different types of EEG signals in this embodiment are jointly evoked by flicker stimuli at two different spatial locations, therefore, the construction of the reference matrix can be a sinusoidal sequence that simultaneously includes all frequencies. Let the left and right frequencies under the nth category be f_n,1 and f_n,2 respectively, then the reference matrix

It can be expressed as the following formula (1), formula (2):

其中，N_h代表参考矩阵所包含的谐波次数。Among them, N_h represents the harmonic order contained in the reference matrix.

根据参考矩阵Y_n"和脑电信号单元平均值

求解目标函数，获得目标源混叠矩阵

目标源混叠矩阵

可以是通过下面的公式得到的：According to the reference matrix Y_n " and the average value of the EEG signal unit

Solve the objective function to obtain the target-source aliasing matrix

target source aliasing matrix

It can be obtained by the following formula:

其中，argmin函数用于搜索使目标函数最小的变量值；Among them, the argmin function is used to search for the variable value that minimizes the objective function;

‖‖_F代表矩阵的Frobenius范数；‖‖_F represents the Frobenius norm of the matrix;

‖‖₁代表矩阵的L1范数；‖‖₁ represents the L1 norm of the matrix;

为第n个类别的脑电信号单元平均值；

is the average value of the EEG signal unit of the nth category;

Y_n"为参考矩阵；Y_n " is a reference matrix;

为目标源混叠矩阵，表征对源混叠矩阵Φ的估计。

is the target source aliasing matrix, representing the estimate of the source aliasing matrix Φ.

步骤B04，基于目标源混叠矩阵和参考矩阵，得到重构源信号。Step B04, obtain the reconstructed source signal based on the target source aliasing matrix and the reference matrix.

示例性地，依据参考矩阵Y_n"和目标源混叠矩阵

计算第n个类别的重构源信号S”_n。重构源信号S”_n可以通过下式确定：Exemplarily, according to the reference matrix Y_n " and the target source aliasing matrix

Calculate the reconstructed source signal S"_n of the nth category. The reconstructed source signal S"_n can be determined by the following formula:

步骤B05，基于重构源信号、随机噪声信号集、预设的扩增试次数和获取的原始信号子集，得到与获取的原始信号子集对应的扩增信号子集。Step B05, based on the reconstructed source signal, the random noise signal set, the preset number of amplification trials and the acquired original signal subset, obtain the amplified signal subset corresponding to the acquired original signal subset.

具体实施时，向重构源信号S”_n中加入随机噪声，获得与获取的原始信号子集χ”_n对应的N_a个扩增信号单元，将得到的各扩增信号单元，作为新的脑电信号单元加入到获取的原始信号子集χ”_n，得到与获取的原始信号子集χ”_n对应的扩增信号子集

During specific implementation, random noise is added to the reconstructed source signal S"_n to obtain Na amplified signal units corresponding to the acquired original signal subset χ"_n , and each obtained amplified signal unit is used as_a new The EEG signal unit is added to the acquired original signal subset χ"_n to obtain the amplified signal subset corresponding to the acquired original signal subset χ"_n

步骤B06，基于扩增脑电信号集，训练脑机接口的脑电信号识别模型，得到训练后的脑电信号识别模型。Step B06, based on the amplified EEG signal set, train the EEG signal recognition model of the brain-computer interface, and obtain the trained EEG signal recognition model.

示例性地，利用扩增信号子集γ”求解EEG-net网络的模型参数，用于后续脑机接口系统的分类识别。Exemplarily, the model parameters of the EEG-net network are solved by using the amplified signal subset γ", which is used for the classification and identification of the subsequent brain-computer interface system.

步骤B07，获取待识别脑电信号，并利用训练后的脑电信号识别模型对待识别脑电信号进行识别，以得到脑电信号识别结果。Step B07, acquiring the EEG signal to be recognized, and using the trained EEG signal recognition model to recognize the EEG signal to be recognized, so as to obtain the EEG signal recognition result.

上述实施例提供的脑机接口训练数据扩增方法，能够通过获取待处理脑电信号集包括的原始信号子集，并基于参考矩阵、随机噪声信号集和与获取的原始信号子集对应的脑电信号单元平均值进行脑电信号重构，获得符合脑电特性的扩增信号，从而实现对脑机接口训练数据中的待处理脑电信号集进行高效的数据扩增，提高脑机接口在小样本情况下的识别性能，从而降低系统的校准负担。The brain-computer interface training data amplification method provided by the above-mentioned embodiments can obtain the original signal subset included in the EEG signal set to be processed, and based on the reference matrix, the random noise signal set, and the brain data corresponding to the acquired original signal subset. The average value of the electrical signal unit is used to reconstruct the EEG signal to obtain an amplified signal that conforms to the EEG characteristics, thereby realizing efficient data amplification of the EEG signal set to be processed in the brain-computer interface training data, and improving the performance of the brain-computer interface. Recognition performance in the case of small samples, thereby reducing the calibration burden of the system.

基于同一发明构思，本申请实施例中还提供了一种脑机接口训练数据扩增装置。由于该装置是本申请实施例脑机接口训练数据扩增方法对应的装置，并且该装置解决问题的原理与该方法相似，因此该装置的实施可以参见上述方法的实施，重复之处不再赘述。Based on the same inventive concept, an embodiment of the present application also provides a brain-computer interface training data amplification device. Since this device is the device corresponding to the brain-computer interface training data amplification method of the embodiment of the present application, and the principle of solving the problem of the device is similar to the method, so the implementation of the device can refer to the implementation of the above method, and the repetition will not be repeated. .

图5示出了本申请实施例提供的一种脑机接口训练数据扩增装置的结构示意图，该脑机接口训练数据扩增装置，如图5所示，包括：数据准备模块501和数据扩增模块502。Fig. 5 shows a schematic structural diagram of a brain-computer interface training data amplification device provided by an embodiment of the present application. The brain-computer interface training data amplification device, as shown in Fig. 5 , includes: adata preparation module 501 and a dataexpansion Add module 502.

其中，数据准备模块501，用于获取待处理脑电信号集包括的原始信号子集；Wherein, thedata preparation module 501 is used to obtain the original signal subset included in the EEG signal set to be processed;

数据扩增模块502，用于在数据准备模块每获取一个原始信号子集时，针对获取的原始信号子集执行以下操作，以得到用以构成扩增脑电信号集的扩增信号子集：Thedata amplification module 502 is configured to perform the following operations on the acquired original signal subset each time the data preparation module acquires an original signal subset, so as to obtain the amplified signal subset used to form the amplified EEG signal set:

基于获取的原始信号子集，确定与获取的原始信号子集对应的脑电信号单元平均值；任一原始信号子集包括数量为实验试次数的脑电信号单元；基于预设的参考矩阵和脑电信号单元平均值，得到目标源混叠矩阵；目标源混叠矩阵是满足使预设的目标函数取最小值的源混叠矩阵；基于目标源混叠矩阵和参考矩阵，得到重构源信号；基于重构源信号、随机噪声信号集、预设的扩增试次数和获取的原始信号子集，得到与获取的原始信号子集对应的扩增信号子集。Based on the acquired original signal subset, determine the average value of the EEG signal unit corresponding to the acquired original signal subset; any original signal subset includes the number of EEG signal units whose number is the number of experimental trials; based on the preset reference matrix and The average value of the EEG signal unit is used to obtain the target source aliasing matrix; the target source aliasing matrix is a source aliasing matrix that satisfies the minimum value of the preset objective function; based on the target source aliasing matrix and the reference matrix, the reconstructed source signal; based on the reconstructed source signal, the random noise signal set, the preset number of amplification trials and the acquired original signal subset, an amplified signal subset corresponding to the acquired original signal subset is obtained.

在一种可能的实现方式中，数据扩增模块，具体用于：In a possible implementation manner, the data augmentation module is specifically used for:

将获取的原始信号子集包括的脑电信号单元求和，得到与获取的原始信号子集对应的第一脑电信号；summing the EEG signal units included in the acquired original signal subset to obtain the first EEG signal corresponding to the acquired original signal subset;

将第一脑电信号与实验试次数求商，得到与获取的原始信号子集对应的脑电信号单元平均值。The quotient of the first EEG signal and the number of experimental trials is obtained to obtain the average value of the EEG signal unit corresponding to the acquired original signal subset.

在一种可能的实现方式中，目标函数为第一偏差矩阵的弗罗贝尼乌斯Frobenius范数；第一偏差矩阵为通过将源混叠矩阵与参考矩阵进行矩阵乘法，再与脑电信号单元平均值作差得到的。In a possible implementation, the objective function is the Frobenius norm of the first deviation matrix; the first deviation matrix is the matrix multiplication of the source aliasing matrix and the reference matrix, and then the EEG signal obtained by subtracting the unit mean.

在一种可能的实现方式中，目标函数为将第二偏差矩阵的Frobenius范数，与源混叠矩阵的正则化范数进行求和得到的；第二偏差矩阵为通过将源混叠矩阵与参考矩阵进行矩阵乘法，再与脑电信号单元平均值作差得到的。In a possible implementation, the objective function is obtained by summing the Frobenius norm of the second deviation matrix and the regularization norm of the source aliasing matrix; the second deviation matrix is obtained by combining the source aliasing matrix with The reference matrix is multiplied by matrix, and then obtained by making a difference with the average value of the EEG signal unit.

在一种可能的实现方式中，基于目标源混叠矩阵和参考矩阵，得到重构源信号，包括：In a possible implementation, the reconstructed source signal is obtained based on the target source aliasing matrix and the reference matrix, including:

将目标源混叠矩阵与参考矩阵进行矩阵乘法，得到重构源信号。Perform matrix multiplication between the target source aliasing matrix and the reference matrix to obtain the reconstructed source signal.

在一种可能的实现方式中，基于重构源信号、随机噪声信号集、预设的扩增试次数和获取的原始信号子集，得到与获取的原始信号子集对应的扩增信号子集，包括：In a possible implementation, based on the reconstructed source signal, the random noise signal set, the preset number of amplification trials and the acquired original signal subset, the amplified signal subset corresponding to the acquired original signal subset is obtained ,include:

基于扩增试次数和预设规则，生成随机噪声信号集；随机噪声信号集包含随机噪声信号单元的数量为扩增试次数；Based on the number of amplification trials and preset rules, a random noise signal set is generated; the number of random noise signal units contained in the random noise signal set is the number of amplification trials;

从随机噪声信号集中逐一获取随机噪声信号单元，每获取一个随机噪声信号单元，执行第一操作，以得到与获取的原始信号子集对应的一个扩增信号单元；第一操作，包括将获取的随机噪声信号单元，与重构源信号求和；Obtain random noise signal units one by one from the random noise signal set, and perform the first operation for each random noise signal unit to obtain an amplified signal unit corresponding to the acquired original signal subset; the first operation includes the acquired A random noise signal unit, summed with the reconstructed source signal;

将得到的各扩增信号单元，作为新的脑电信号单元加入到获取的原始信号子集，得到与获取的原始信号子集对应的扩增信号子集。The obtained amplified signal units are added as new EEG signal units to the acquired original signal subset to obtain an amplified signal subset corresponding to the acquired original signal subset.

在一种可能的实现方式中，如图6所示，装置还包括：In a possible implementation, as shown in FIG. 6, the device further includes:

脑电信号识别模块601，用于基于扩增脑电信号集，训练脑机接口的脑电信号识别模型，得到训练后的脑电信号识别模型；获取待识别脑电信号，并利用训练后的脑电信号识别模型对待识别脑电信号进行识别，以得到脑电信号识别结果。The EEGsignal recognition module 601 is used to train the EEG signal recognition model of the brain-computer interface based on the amplified EEG signal set, and obtain the trained EEG signal recognition model; obtain the EEG signal to be recognized, and use the trained EEG signal The EEG signal identification model identifies the EEG signal to be identified to obtain an EEG signal identification result.

与上述方法实施例基于同一发明构思，本申请实施例中还提供了一种电子设备。该电子设备可以用于脑机接口训练数据扩增。在一种实施例中，该电子设备可以是服务器或其他电子设备。在该实施例中，电子设备的结构可以如图7所示，包括存储器701，通讯模块703以及一个或多个处理器702。Based on the same inventive concept as the foregoing method embodiments, an electronic device is also provided in the embodiments of the present application. The electronic device can be used for brain-computer interface training data amplification. In one embodiment, the electronic device may be a server or other electronic device. In this embodiment, the structure of the electronic device may be as shown in FIG. 7 , including amemory 701 , acommunication module 703 and one ormore processors 702 .

存储器701，用于存储处理器702执行的计算机程序。存储器701可主要包括存储程序区和存储数据区，其中，存储程序区可存储操作系统，以及运行即时通讯功能所需的程序等；存储数据区可存储各种即时通讯信息和操作指令集等。Thememory 701 is used for storing computer programs executed by theprocessor 702 . Thememory 701 may mainly include a program storage area and a data storage area, wherein the program storage area may store operating systems and programs required for running instant messaging functions, etc.; the data storage area may store various instant messaging information and operating instruction sets, etc.

存储器701可以是易失性存储器(volatile memory)，例如随机存取存储器(random-access memory，RAM)；存储器701也可以是非易失性存储器(non-volatilememory)，例如只读存储器，快闪存储器(flash memory)，硬盘(hard disk drive，HDD)或固态硬盘(solid-state drive，SSD)、或者存储器701是能够用于携带或存储具有指令或数据结构形式的期望的程序代码并能够由计算机存取的任何其他介质，但不限于此。存储器701可以是上述存储器的组合。Thememory 701 can be a volatile memory (volatile memory), such as a random-access memory (random-access memory, RAM); thememory 701 can also be a non-volatile memory (non-volatile memory), such as a read-only memory, a flash memory (flash memory), hard disk (hard disk drive, HDD) or solid-state drive (solid-state drive, SSD), or thememory 701 can be used to carry or store desired program codes in the form of instructions or data structures and can be used by the computer Any other medium accessed, but not limited to. Thememory 701 may be a combination of the above-mentioned memories.

处理器702，可以包括一个或多个中央处理单元(central processing unit，CPU)或者为数字处理单元等等。处理器702，用于调用存储器701中存储的计算机程序时实现上述脑机接口训练数据扩增方法。Theprocessor 702 may include one or more central processing units (central processing unit, CPU) or be a digital processing unit or the like. Theprocessor 702 is configured to implement the above-mentioned brain-computer interface training data augmentation method when calling the computer program stored in thememory 701 .

通讯模块703用于与终端设备或其他服务器进行通信。Thecommunication module 703 is used for communicating with terminal devices or other servers.

本申请实施例中不限定上述存储器701、通讯模块703和处理器702之间的具体连接介质。本申请实施例在图7中以存储器701和处理器702之间通过总线704连接，总线704在图7中以粗线表示，其它部件之间的连接方式，仅是进行示意性说明，并不引以为限。总线704可以分为地址总线、数据总线、控制总线等。为便于表示，图7中仅用一条粗线表示，但并不表示仅有一根总线或一种类型的总线。In this embodiment of the present application, the specific connection medium among thememory 701, thecommunication module 703, and theprocessor 702 is not limited. In the embodiment of the present application, in FIG. 7, thememory 701 and theprocessor 702 are connected through a bus 704. The bus 704 is represented by a thick line in FIG. As far as possible. The bus 704 can be divided into address bus, data bus, control bus and so on. For ease of representation, only one thick line is used in FIG. 7 , but it does not mean that there is only one bus or one type of bus.

本申请实施例还提供了一种计算机可读存储介质，计算机可读存储介质中存储有计算机可执行指令，计算机可执行指令用于实现本申请任一实施例的脑机接口训练数据扩增方法。The embodiment of the present application also provides a computer-readable storage medium, in which computer-executable instructions are stored, and the computer-executable instructions are used to implement the brain-computer interface training data amplification method in any embodiment of the present application .

根据本申请的一个方面，提供了一种计算机程序产品或计算机程序，该计算机程序产品或计算机程序包括计算机指令，该计算机指令存储在计算机可读存储介质中。计算机设备的处理器从计算机可读存储介质读取该计算机指令，处理器执行该计算机指令，使得该计算机设备执行上述实施例中的脑机接口训练数据扩增方法。所述程序产品可以采用一个或多个可读介质的任意组合。可读介质可以是可读信号介质或者可读存储介质。可读存储介质例如可以是——但不限于——电、磁、光、电磁、红外线、或半导体的系统、装置或器件，或者任意以上的组合。可读存储介质的更具体的例子(非穷举的列表)包括：具有一个或多个导线的电连接、便携式盘、硬盘、随机存取存储器(RAM)、只读存储器(ROM)、可擦式可编程只读存储器(EPROM或闪存)、光纤、便携式紧凑盘只读存储器(CD-ROM)、光存储器件、磁存储器件、或者上述的任意合适的组合。According to an aspect of the present application there is provided a computer program product or computer program comprising computer instructions stored in a computer readable storage medium. The processor of the computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device executes the brain-computer interface training data augmentation method in the above-mentioned embodiments. The program product may reside on any combination of one or more readable media. The readable medium may be a readable signal medium or a readable storage medium. A readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or device, or any combination thereof. More specific examples (non-exhaustive list) of readable storage media include: electrical connection with one or more conductors, portable disk, hard disk, random access memory (RAM), read only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

以上所述，仅为本申请的具体实施方式，但本申请的保护范围并不局限于此，任何熟悉本技术领域的技术人员在本申请揭露的技术范围内，可轻易想到变化或替换，都应涵盖在本申请的保护范围之内。The above is only a specific implementation of the application, but the scope of protection of the application is not limited thereto. Anyone familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed in the application. Should be covered within the protection scope of this application.

Claims (10)

1. A brain-computer interface training data amplification method is characterized by comprising the following steps:

acquiring an original signal subset included in an electroencephalogram signal set to be processed;

wherein, each time one of the original signal subsets is acquired, the following operations are performed on the acquired original signal subsets to obtain an amplified signal subset used for constituting an amplified electroencephalogram signal set:

determining an electroencephalogram signal unit average value corresponding to the acquired original signal subset based on the acquired original signal subset; any original signal subset comprises electroencephalogram signal units with the number of experimental test times;

obtaining a target source aliasing matrix based on a preset reference matrix and the electroencephalogram signal unit average value; the target source aliasing matrix is a source aliasing matrix which enables a preset target function to take the minimum value;

obtaining a reconstructed source signal based on the target source aliasing matrix and the reference matrix;

and obtaining the amplification signal subset corresponding to the obtained original signal subset based on the reconstructed source signal, the random noise signal set, the preset amplification test times and the obtained original signal subset.

2. The method of claim 1, wherein said determining, based on said acquired subset of raw signals, a brain electrical signal unit mean corresponding to said acquired subset of raw signals comprises:

summing the electroencephalogram signal units included in the acquired original signal subset to obtain a first electroencephalogram signal corresponding to the acquired original signal subset;

and obtaining the average value of the electroencephalogram signal units corresponding to the acquired original signal subset by taking the quotient of the first electroencephalogram signal and the experimental trial times.

3. The method of claim 1, wherein the objective function is a Frobenius norm of the first deviation matrix; the first deviation matrix is obtained by performing matrix multiplication on a source aliasing matrix and the reference matrix and then performing difference on the average value of the electroencephalogram signal unit.

4. The method of claim 1, wherein the objective function is obtained by summing a Frobenius norm of the second bias matrix with a regularized norm of the source aliasing matrix; the second deviation matrix is obtained by performing matrix multiplication on a source aliasing matrix and the reference matrix and then performing difference on the EEG unit average value.

5. The method of claim 1, wherein deriving a reconstructed source signal based on the target source aliasing matrix and the reference matrix comprises:

and performing matrix multiplication on the target source aliasing matrix and the reference matrix to obtain the reconstructed source signal.

6. The method of claim 1, wherein obtaining the subset of amplified signals corresponding to the obtained subset of original signals based on the reconstructed source signal, a set of random noise signals, a preset number of amplification tests, and the obtained subset of original signals comprises:

generating a random noise signal set based on the amplification test times and a preset rule; the random noise signal set comprises the number of random noise signal units which is the number of times of the amplification test;

acquiring the random noise signal units from the random noise signal set one by one, and executing a first operation when acquiring one random noise signal unit to obtain an amplification signal unit corresponding to the acquired original signal subset; the first operation comprises summing the acquired random noise signal unit with the reconstructed source signal;

and adding each obtained amplification signal unit as a new electroencephalogram signal unit into the obtained original signal subset to obtain the amplification signal subset corresponding to the obtained original signal subset.

7. The method of claim 1, further comprising:

training an electroencephalogram recognition model of the brain-computer interface based on the amplified electroencephalogram signal set to obtain the trained electroencephalogram recognition model;

acquiring an electroencephalogram signal to be recognized, and recognizing the electroencephalogram signal to be recognized by using the trained electroencephalogram signal recognition model to obtain an electroencephalogram signal recognition result.

8. An apparatus for augmenting brain-computer interface training data, comprising:

the data preparation module is used for acquiring an original signal subset included in the electroencephalogram signal set to be processed;

a data amplification module, configured to, when the data preparation module acquires each original signal subset, perform the following operations on the acquired original signal subset to obtain an amplified signal subset used to form an amplified electroencephalogram signal set:

determining an electroencephalogram signal unit average value corresponding to the acquired original signal subset based on the acquired original signal subset; any original signal subset comprises electroencephalogram units with the number of experimental trial times; obtaining a target source aliasing matrix based on a preset reference matrix and the electroencephalogram signal unit average value; the target source aliasing matrix is a source aliasing matrix which enables a preset target function to take the minimum value; obtaining a reconstructed source signal based on the target source aliasing matrix and the reference matrix; and obtaining the amplification signal subset corresponding to the obtained original signal subset based on the reconstructed source signal, the random noise signal set, the preset amplification test times and the obtained original signal subset.

9. A computer-readable storage medium, wherein a computer program is stored in the computer-readable storage medium; the computer program, when executed by a processor, implements the method of any one of claims 1 to 7.

10. An electronic device comprising a memory and a processor, the memory having stored thereon a computer program operable on the processor, the computer program, when executed by the processor, implementing the method of any of claims 1-7.

Images