CN110710978A - A Multimodal Immersive Synchronous Acquisition System Based on Eye Tracking-Brain Function Activity Detection - Google Patents

Abstract

Translated fromChinese

本发明公开了一种基于眼动追踪‑脑功能活动检测的多模态沉浸式同步采集系统，包括VR显示设备，VR显示设备内置有显示模块、眼动采集模块、脑血氧采集模块和通信模块，眼动采集模块用于采集被试眼部影像，脑血氧采集模块用于采集被试头部的脑血氧信号，VR显示设备及眼动采集模块、脑血氧采集模块分别通过通信模块与外接终端设备实现数据传输和时钟同步。本发明在整个采集系统中采用带有虚拟沉浸式体验的VR显示设备，整个测试与数据采集过程可以完全避免外界对被试人员的干扰，同时本发明兼顾了眼动和脑血氧信号同步采集，通过通信模块直接与外接终端设备进行数据传输，采用时钟同步方式解决了实时性特征和多模态信号的同步性问题。

The invention discloses a multi-modal immersive synchronous acquisition system based on eye tracking-brain function activity detection, comprising a VR display device. The VR display device has a built-in display module, an eye movement acquisition module, a cerebral blood oxygen acquisition module and a communication module. Module, the eye movement acquisition module is used to collect the subject's eye image, the cerebral blood oxygen acquisition module is used to collect the cerebral blood oxygen signal of the subject's head, and the VR display device, the eye movement acquisition module and the cerebral blood oxygen acquisition module respectively communicate with each other. The module realizes data transmission and clock synchronization with external terminal equipment. In the present invention, a VR display device with virtual immersive experience is used in the entire acquisition system, and the entire test and data acquisition process can completely avoid external interference to the subjects. , through the communication module directly with the external terminal equipment for data transmission, using the clock synchronization method to solve the real-time characteristics and multi-modal signal synchronization problems.

Description

Translated fromChinese

一种基于眼动追踪-脑功能活动检测的多模态沉浸式同步采集系统A multimodal immersive synchronous acquisition based on eye tracking-brain functional activity detectionset system

技术领域technical field

本发明涉及认知神经科学技术领域，具体涉及一种基于眼动追踪-脑功能活动检测的多模态沉浸式同步采集装置。The invention relates to the technical field of cognitive neuroscience, in particular to a multi-modal immersive synchronous acquisition device based on eye movement tracking-brain function activity detection.

背景技术Background technique

脑电图(electroencephalogram,EEG)是通过电极记录下来的脑细胞群的自发性、节律性电活动，它包含了大量的生理和病理信息，是神经机能检查方法之一。脑电图反映了大脑组织的电活动和各种功能状态，其基本特征包括振幅、周期、相位等等，特定位置的脑电信号还可以反映认知能力的情况。眼动追踪技术是通过眼动测量设备测量人眼瞳孔的运动情况，进而估计人视线方向和位置的技术。通过眼动追踪技术可以实现多种应用，比如用于汽车上，检测驾驶员的疲劳情况；在认知领域，通过追踪用户的视线分析用户的心理活动。Electroencephalogram (EEG) is the spontaneous and rhythmic electrical activity of brain cell groups recorded by electrodes. It contains a large amount of physiological and pathological information and is one of the methods of neurological examination. EEG reflects the electrical activity and various functional states of brain tissue, and its basic characteristics include amplitude, cycle, phase, etc., and EEG signals at specific locations can also reflect cognitive abilities. Eye tracking technology is a technology that measures the movement of the pupil of the human eye through an eye movement measurement device, and then estimates the direction and position of the person's gaze. Eye tracking technology can achieve a variety of applications, such as being used in cars to detect driver fatigue; in the field of cognition, users’ psychological activities can be analyzed by tracking the user’s gaze.

现有技术中仅仅通过单一的某一技术手段获取与脑功能相关的参数采集，比如在测试时用户浏览待评估页面，利用眼动数据采集设备采集用户注视待评估页面的注视位置和在每个注视位置进行注视时所使用的注视时长，可以通过注视坐标的形式或者注视轮廓的形式记录用户的注视位置。在浏览待评估页面时会受到外界干扰，影响了对脑功能检测的准确性。In the prior art, only a single technical means is used to obtain parameter collection related to brain function. For example, during the test, the user browses the page to be evaluated, and the eye movement data collection device is used to collect the gaze position of the user looking at the page to be evaluated and the information on each page. The gaze duration used when the gaze position is fixed, and the gaze position of the user can be recorded in the form of gaze coordinates or in the form of gaze contours. When browsing the page to be evaluated, there will be external interference, which affects the accuracy of brain function detection.

为了全面对人的认知神经进行多参数检测，亟需一种可以结合现代脑电技术和眼动追踪技术实现对人的脑功能和认知功能的多类型参数采集与多模态检测，为脑功能分析提供更全面的参数。In order to comprehensively perform multi-parameter detection on human cognitive nerves, it is urgent to realize multi-type parameter acquisition and multi-modal detection of human brain function and cognitive function by combining modern EEG technology and eye tracking technology. Brain function analysis provides more comprehensive parameters.

发明内容SUMMARY OF THE INVENTION

本发明针对现有在对涉及认知神经参数进行数据采集时存在干扰，影响脑功能分析检测准确性的技术问题，本发明提供了一种基于眼动追踪-脑功能活动检测的多模态沉浸式同步采集系统。Aiming at the existing technical problem that there is interference in data collection involving cognitive nerve parameters, which affects the accuracy of brain function analysis and detection, the present invention provides a multimodal immersion method based on eye tracking and brain function activity detection. synchronous acquisition system.

本发明采用如下技术方案：The present invention adopts following technical scheme:

一种基于眼动追踪-脑功能活动检测的多模态沉浸式同步采集系统，其特征在于，所述系统包括佩戴在头部的VR显示设备，所述VR显示设备内置有显示模块、眼动采集模块、脑血氧采集模块和通信模块，所述眼动采集模块用于采集被试眼部影像，所述脑血氧采集模块用于采集被试头部的脑血氧信号，所述VR显示设备及所述的眼动采集模块、脑血氧采集模块分别通过所述的通信模块与外接终端设备实现数据传输和时钟同步。A multi-modal immersive synchronous acquisition system based on eye tracking-brain function activity detection, characterized in that the system includes a VR display device worn on the head, the VR display device has a built-in display module, eye movement A collection module, a cerebral blood oxygen collection module and a communication module, the eye movement collection module is used to collect the subject's eye image, the cerebral blood oxygen collection module is used to collect the cerebral blood oxygen signal of the subject's head, the VR The display device, the eye movement acquisition module and the cerebral blood oxygen acquisition module respectively realize data transmission and clock synchronization with the external terminal equipment through the communication module.

所述眼动采集模块包括多个近红外光灯和近红外摄像头，多个所述近红外光灯环绕所述显示模块的边缘区域形成近红外光阵列，所述近红外摄像头位于所述VR显示设备的眼底位置。The eye movement collection module includes a plurality of near-infrared lights and a near-infrared camera, a plurality of the near-infrared lights surround the edge area of the display module to form a near-infrared light array, and the near-infrared cameras are located in the VR display. Fundus position of the device.

所述脑血氧采集模块包括设置于所述VR显示设备前额位置的探头光源模块和探头感光点；所述探头光源模块用于向被试皮肤照射红外线，所述探头感光点接收所述探头光源模块照射人体组织在不同角度的反射光，并将光信号转为电信号，并通过所述通信模块将电信号传输到外接终端设备进行处理。The cerebral blood oxygen acquisition module includes a probe light source module and a probe photosensitive point arranged at the forehead position of the VR display device; the probe light source module is used for irradiating infrared rays to the subject's skin, and the probe photosensitive point receives the probe light source The module illuminates the reflected light of human tissue at different angles, converts the optical signal into an electrical signal, and transmits the electrical signal to an external terminal device for processing through the communication module.

所述VR显示设备的显示模块上方还设有用于采集前额区域脑电信号的脑电传感器，所述脑电传感器通过所述通信模块将所采集的脑电信号传输到外接终端设备进行处理。An EEG sensor for collecting EEG signals in the forehead region is further provided above the display module of the VR display device, and the EEG sensor transmits the collected EEG signals to an external terminal device for processing through the communication module.

所述探头光源发射红光和红外光，其发光波段为600～900nm。The probe light source emits red light and infrared light, and its emission wavelength range is 600-900 nm.

所述系统还包括脑电采集装置，其与所述VR显示设备连接，用于采集被试头部的脑电信号，所述脑电采集装置通过所述通信模块与外接终端设备实现数据传输。The system further includes an EEG acquisition device, which is connected to the VR display device and used to collect the EEG signals of the subject's head, and the EEG acquisition device realizes data transmission with an external terminal device through the communication module.

所述脑电采集装置为帽式结构，其包括多个电极和与多个所述电极连接的信号处理器，所述电极采集被试的脑电信息，并通过所述信号处理器对所述脑电信息进行硬件滤波、信号增强和模数转换，所述信号处理器通过所述通信模块与外接终端设备间数据传输。The EEG acquisition device is a cap-type structure, which includes a plurality of electrodes and a signal processor connected with the plurality of electrodes, the electrodes collect the EEG information of the subject, and the signal processor is used to analyze the EEG information of the subject. The EEG information is subjected to hardware filtering, signal enhancement and analog-to-digital conversion, and the signal processor transmits data between the communication module and an external terminal device.

所述电极为AgCl电极，其分布于人的头部特定区域，用于采集被试脑电信号。The electrodes are AgCl electrodes, which are distributed in a specific area of the human head and are used to collect the subject's EEG signals.

所述通信模块为与有线USB端口数据进行数据传输和时钟同步的通用数据接口。The communication module is a general data interface for data transmission and clock synchronization with wired USB port data.

所述通信模块为蓝牙通信模块，其与外接终端设备间可以实现无线数据传输和时钟同步。The communication module is a Bluetooth communication module, which can realize wireless data transmission and clock synchronization with an external terminal device.

本发明技术方案，具有如下优点：The technical scheme of the present invention has the following advantages:

A.本发明在整个采集系统中采用带有虚拟沉浸式体验的VR显示设备，整个测试与数据采集过程可以完全避免外界对被试人员的干扰，同时本发明兼顾了眼动和脑血氧信号同步采集，通过通信模块直接与外接终端设备进行数据传输，采用时钟同步方式解决了实时性特征和多模态信号的同步性问题。A. The present invention adopts a VR display device with virtual immersive experience in the entire acquisition system, and the entire testing and data acquisition process can completely avoid external interference to the subjects, while the present invention takes into account both eye movement and cerebral blood oxygen signals Synchronous acquisition, data transmission directly with external terminal equipment through the communication module, using the clock synchronization method to solve the real-time characteristics and synchronization problems of multi-modal signals.

B.本发明通过对脑电信号、眼动信号、脑血氧信号几个方面的数据采集，外接设备可以通过所获取到的多种脑功能信号，给脑功能分析提供更全面的参数，使对人的认知作出更为准确的评估。B. The present invention provides more comprehensive parameters for brain function analysis by collecting data from several aspects of brain electrical signals, eye movement signals, and cerebral blood oxygen signals, and the external device can provide more comprehensive parameters for brain function analysis through the obtained various brain function signals. A more accurate assessment of human cognition.

C.本发明将眼动采集模块、脑血氧采集模块以及脑电采集装置与VR显示设备集成在一起，通过无线或有线数据传输方式将各种类型的采集信号同步传输至外接终端设备进行数据分析，使整个检测更加便利，效率更高。C. The present invention integrates the eye movement acquisition module, the cerebral blood oxygen acquisition module and the EEG acquisition device with the VR display device, and transmits various types of acquisition signals synchronously to the external terminal equipment for data transmission through wireless or wired data transmission. Analysis makes the whole detection more convenient and more efficient.

附图说明Description of drawings

为了更清楚地说明本发明具体实施方式，下面将对具体实施方式中所需要使用的附图作简单地介绍，显而易见地，下面描述中的附图是本发明的一些实施方式，对于本领域普通技术人员来讲，在不付出创造性劳动的前提下，还可以根据这些附图获得其他的附图。In order to illustrate the specific embodiments of the present invention more clearly, the following will briefly introduce the accompanying drawings used in the specific embodiments. As far as technical personnel are concerned, other drawings can also be obtained based on these drawings without any creative effort.

图1是本发明提供的同步采集系统结构示意图；1 is a schematic structural diagram of a synchronous acquisition system provided by the present invention;

图2为脑电采集装置结构示意图；Figure 2 is a schematic structural diagram of an EEG acquisition device;

图3是本发明所提供的系统物理拓扑图。Fig. 3 is a system physical topology diagram provided by the present invention.

图中标识如下：The figure is marked as follows:

1-VR显示设备；11-显示模块，12-眼动采集模块，121-近红外光灯， 122-近红外摄像头，13-脑血氧采集模块，131-探头光源模块，132-探头感光点；1-VR display device; 11-display module, 12-eye movement acquisition module, 121-near infrared light, 122-near infrared camera, 13-cerebral blood oxygen acquisition module, 131-probe light source module, 132-probe photosensitive point ;

2-脑电采集装置，21-电极，22-信号处理器；2- EEG acquisition device, 21- electrode, 22- signal processor;

3-脑电传感器。3- EEG sensor.

具体实施方式Detailed ways

下面将结合附图对本发明的技术方案进行清楚、完整地描述，显然，所描述的实施例是本发明一部分实施例，而不是全部的实施例。基于本发明中的实施例，本领域普通技术人员在没有做出创造性劳动前提下所获得的所有其他实施例，都属于本发明保护的范围。The technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are a part of the embodiments of the present invention, but not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

如图3所示，本发明提供了一种基于眼动追踪-脑功能活动检测的多模态沉浸式同步采集系统，包括VR显示设备1，VR显示设备1内置有显示模块11、眼动采集模块12、脑血氧采集模块13和通信模块，眼动采集模块12用于采集被试眼部影像，脑血氧采集模块13用于采集被试头部的脑血氧信号，VR显示设备1及眼动采集模块12、脑血氧采集模块13分别通过通信模块与外接终端设备实现数据传输和时钟同步。本发明在整个采集系统中采用带有虚拟沉浸式体验的VR显示设备，被试人员通过佩戴在眼睛上的虚拟现实呈现设备，被试通过内置的彩色显示模块观看视频或图片等刺激素材，整个测试与数据采集过程可以完全避免外界对被试人员的干扰；本发明还兼顾了眼动和脑血氧信号的同步采集，通信模块直接与外接终端设备(比如电脑)进行数据传输，通过采用时钟同步方式解决了实时性特征和多模态信号的同步性问题，使各种类型的数据实现同步传输。As shown in FIG. 3 , the present invention provides a multi-modal immersive synchronous acquisition system based on eye tracking-brain function activity detection, including a VR display device 1, and the VR display device 1 has a built-indisplay module 11, an eye movementacquisition system Module 12, cerebral bloodoxygen acquisition module 13 and communication module, the eyemovement acquisition module 12 is used to collect the subject's eye image, the cerebral bloodoxygen acquisition module 13 is used to collect the cerebral blood oxygen signal of the subject's head, VR display device 1 The eyemovement acquisition module 12 and the cerebral bloodoxygen acquisition module 13 respectively realize data transmission and clock synchronization with the external terminal equipment through the communication module. The present invention adopts VR display equipment with virtual immersive experience in the whole acquisition system. The subjects watch the stimulation materials such as videos or pictures through the built-in color display module through the virtual reality presentation equipment worn on the eyes. The process of testing and data collection can completely avoid external interference to the subjects; the present invention also takes into account the synchronous collection of eye movement and cerebral blood oxygen signals, and the communication module directly performs data transmission with external terminal equipment (such as a computer), by using a clock The synchronization method solves the problem of real-time characteristics and synchronization of multi-modal signals, so that various types of data can be transmitted synchronously.

这里的眼动采集模块12优选为包括多个近红外光灯121和近红外摄像头122，多个近红外光灯121环绕显示模块11的边缘区域形成近红外光阵列，近红外摄像头122位于VR显示设备1的眼底位置，通过通信模块将采集到的原始眼部影像传输电脑进行处理。因为在佩戴VR显示设备时，眼部区域是比较暗，需要额外对眼部区域进行照明来采集瞳孔信息。近红外光的特点是其为不可见光，不会对人眼产生影响。同时用户所视的屏幕距离眼部区域很近，屏幕的亮度会干扰眼部图像的采集，采用近红外摄像头22 将可见光波段全部过滤掉，只采集近红外光灯所照明的信息。The eyemovement acquisition module 12 here preferably includes a plurality of near-infrared light lamps 121 and a near-infrared camera 122 , and the plurality of near-infrared light lamps 121 surround the edge area of thedisplay module 11 to form a near-infrared light array, and the near-infrared camera 122 is located in the VR display. The fundus position of the device 1 is transmitted to the computer through the communication module to transmit the collected original eye image for processing. Because the eye area is relatively dark when wearing a VR display device, it is necessary to additionally illuminate the eye area to collect pupil information. The characteristic of near-infrared light is that it is invisible light and will not affect the human eye. At the same time, the screen viewed by the user is very close to the eye area, and the brightness of the screen will interfere with the collection of eye images. The near-infrared camera 22 is used to filter out all visible light bands, and only the information illuminated by the near-infrared light is collected.

这里的通信模块可以为通用数据接口，兼容不同数据格式，可以实现有线USB端口数据传输和时钟同步。当然，还可以为蓝牙通信模块，可以实现无线数据传输和时钟同步。通信模块兼容常用通信协议，同步功能可以靠内置时钟进行，也可以由外接终端设备控制进行。The communication module here can be a general data interface, compatible with different data formats, and can realize wired USB port data transmission and clock synchronization. Of course, it can also be a Bluetooth communication module, which can realize wireless data transmission and clock synchronization. The communication module is compatible with common communication protocols, and the synchronization function can be performed by the built-in clock or controlled by an external terminal device.

如图1所示，其中的脑血氧采集模块13包括设置于VR显示设备1前额位置的探头光源模块131和探头感光点132；探头光源模块131用于向被试的前额皮肤位置照射红外线和近红外线，其中的发光波段为600-900nm，，探头感光点132接收照射至被试皮肤上的反射光，并将光信号转化为电信号，通过通信模块将所得到的电信号传输到外接终端设备进行处理。当然，如图1所示，还可以在VR显示设备的显示模块11上方设有用于采集前额区域脑电信号的脑电传感器3，脑电传感器3通过通信模块将所采集的脑电信号传输到外接终端设备进行处理。As shown in FIG. 1 , the cerebral bloodoxygen acquisition module 13 includes a probelight source module 131 and a probephotosensitive point 132 arranged at the forehead position of the VR display device 1; the probelight source module 131 is used to irradiate infrared rays and Near-infrared rays, in which the light-emitting band is 600-900 nm, the probephotosensitive point 132 receives the reflected light irradiated on the subject's skin, converts the optical signal into an electrical signal, and transmits the obtained electrical signal to the external terminal through the communication module equipment for processing. Of course, as shown in FIG. 1 , an EEG sensor 3 for collecting EEG signals in the forehead region can also be provided above thedisplay module 11 of the VR display device, and the EEG sensor 3 transmits the collected EEG signals to the External terminal equipment for processing.

另外，进一步优选地，在系统中还设有脑电采集装置2，脑电采集装置 2与VR显示设备1连接，可以形成一个整体，便于进行携带和检测，用于检测被试人员头部的脑电信号，脑电采集装置2通过通信模块与外接终端设备实现数据传输。如图2所示，这里的脑电采集装置2优选为帽式结构，将其佩戴在被试人员的头部，具体包括多个电极21和与多个电极21连接的信号处理器22，多个电极21分布于被试人员的头部特定位置，优选采用 AgCl电极，采集被试人员的头部特定区域的脑电信息，并通过信号处理器对脑电信息进行硬件滤波、信号增强和模数转换，信号处理器通过通信模块与外接终端设备间进行数据传输，通过外接终端设备对数据进行处理，评定被试人员的认知能力。In addition, further preferably, anEEG acquisition device 2 is also provided in the system, and theEEG acquisition device 2 is connected with the VR display device 1 to form a whole, which is convenient for carrying and detection, and is used to detect the head of the subject. EEG signal, theEEG acquisition device 2 realizes data transmission through a communication module and an external terminal device. As shown in FIG. 2 , theEEG acquisition device 2 here is preferably a cap-type structure, which is worn on the head of the subject, and specifically includes a plurality ofelectrodes 21 and asignal processor 22 connected to the plurality ofelectrodes 21 . Theelectrodes 21 are distributed at a specific position on the subject's head, preferably AgCl electrodes are used to collect the EEG information of the specific area of the subject's head, and perform hardware filtering, signal enhancement and modeling on the EEG information through a signal processor. Digital conversion, the signal processor transmits data between the communication module and the external terminal equipment, processes the data through the external terminal equipment, and evaluates the cognitive ability of the subjects.

本发明通过带有脑电检测、眼动追踪和脑血氧检测功能的一体化VR 显示设备，同步在外接终端设备上得到脑电信号、眼动信号、脑血氧信号几个方面的数据采集，外接设备可以通过所获取到的多种脑功能信号，给脑功能分析提供更全面的参数，使对人的认知作出更为准确的评估。The present invention obtains the data collection of EEG signal, eye movement signal and cerebral blood oxygen signal synchronously on the external terminal equipment through an integrated VR display device with EEG detection, eye movement tracking and cerebral blood oxygen detection functions. , the external device can provide more comprehensive parameters for brain function analysis through the obtained various brain function signals, so that a more accurate evaluation of human cognition can be made.

显然，上述实施例仅仅是为清楚地说明所作的举例，而并非对实施方式的限定。对于所属领域的普通技术人员来说，在上述说明的基础上还可以做出其它不同形式的变化或变动。这里无需也无法对所有的实施方式予以穷举。而由此所引伸出的显而易见的变化或变动仍处于本发明的保护范围之中。Obviously, the above-mentioned embodiments are only examples for clear description, and are not intended to limit the implementation manner. For those of ordinary skill in the art, changes or modifications in other different forms can also be made on the basis of the above description. There is no need and cannot be exhaustive of all implementations here. However, the obvious changes or changes derived therefrom still fall within the protection scope of the present invention.

Claims (10)

Translated fromChinese

1.一种基于眼动追踪-脑功能活动检测的多模态沉浸式同步采集系统，其特征在于，所述系统包括佩戴在头部的VR显示设备(1)，所述VR显示设备(1)内置有显示模块(11)、眼动采集模块(12)、脑血氧采集模块(13)和通信模块，所述眼动采集模块(12)用于采集被试眼部影像，所述脑血氧采集模块(13)用于采集被试头部的脑血氧信号，所述VR显示设备(1)及所述的眼动采集模块(12)、脑血氧采集模块(13)分别通过所述的通信模块与外接终端设备实现数据传输和时钟同步。1. A multimodal immersive synchronous acquisition system based on eye tracking-brain function activity detection, wherein the system comprises a VR display device (1) worn on the head, and the VR display device (1) ) has a built-in display module (11), an eye movement collection module (12), a cerebral blood oxygen collection module (13) and a communication module, the eye movement collection module (12) is used to collect the subject's eye image, and the brain The blood oxygen collection module (13) is used to collect the cerebral blood oxygen signal of the subject's head, and the VR display device (1), the eye movement collection module (12), and the cerebral blood oxygen collection module (13) pass through respectively The communication module realizes data transmission and clock synchronization with the external terminal equipment.2.根据权利要求1所述的基于眼动追踪-脑功能活动检测的多模态沉浸式同步采集系统，其特征在于，所述眼动采集模块(12)包括多个近红外光灯(121)和近红外摄像头(122)，多个所述近红外光灯(121)环绕所述显示模块(11)的边缘区域形成近红外光阵列，所述近红外摄像头(122)位于所述VR显示设备(1)的眼底位置。2. The multimodal immersive synchronous acquisition system based on eye tracking-brain function activity detection according to claim 1, wherein the eye movement acquisition module (12) comprises a plurality of near-infrared lamps (121) ) and a near-infrared camera (122), a plurality of the near-infrared light lamps (121) surround the edge area of the display module (11) to form a near-infrared light array, and the near-infrared camera (122) is located in the VR display Fundus position of device (1).3.根据权利要求1所述的基于眼动追踪-脑功能活动检测的多模态沉浸式同步采集系统，其特征在于，所述脑血氧采集模块(13)包括设置于所述VR显示设备(1)前额位置的探头光源模块(131)和探头感光点(132)；所述探头光源模块(131)用于向被试皮肤照射红外线，所述探头感光点(132)接收所述探头光源模块(131)照射人体组织在不同角度的反射光，并将光信号转为电信号，并通过所述通信模块将电信号传输到外接终端设备进行处理。3. The multimodal immersive synchronous acquisition system based on eye tracking-brain function activity detection according to claim 1, characterized in that, the cerebral blood oxygen acquisition module (13) comprises a device arranged on the VR display device (1) A probe light source module (131) and a probe photosensitive point (132) at the forehead position; the probe light source module (131) is used for irradiating infrared rays to the subject's skin, and the probe photosensitive point (132) receives the probe light source The module (131) illuminates the reflected light of human tissue at different angles, converts the optical signal into an electrical signal, and transmits the electrical signal to an external terminal device for processing through the communication module.4.根据权利要求3所述的基于眼动追踪-脑功能活动检测的多模态沉浸式同步采集系统，其特征在于，所述VR显示设备的显示模块(11)上方还设有用于采集前额区域脑电信号的脑电传感器(3)，所述脑电传感器(3)通过所述通信模块将所采集的脑电信号传输到外接终端设备进行处理。4. The multimodal immersive synchronous acquisition system based on eye tracking-brain function activity detection according to claim 3, characterized in that, above the display module (11) of the VR display device is further provided with a forehead for collecting An EEG sensor (3) for regional EEG signals, the EEG sensor (3) transmits the collected EEG signals to an external terminal device for processing through the communication module.5.根据权利要求3所述的基于眼动追踪-脑功能活动检测的多模态沉浸式同步采集系统，其特征在于，所述探头光源(131)发射红光和红外光，其发光波段为600～900nm。5. The multimodal immersive synchronous acquisition system based on eye tracking-brain function activity detection according to claim 3, wherein the probe light source (131) emits red light and infrared light, and its luminous waveband is 600～900nm.6.根据权利要求1-5任一所述的基于眼动追踪-脑功能活动检测的多模态沉浸式同步采集系统，其特征在于，所述系统还包括脑电采集装置(2)，其与所述VR显示设备(1)连接，用于采集被试头部的脑电信号，所述脑电采集装置(2)通过所述通信模块与外接终端设备实现数据传输。6. The multimodal immersive synchronous acquisition system based on eye tracking-brain function activity detection according to any one of claims 1-5, wherein the system further comprises an EEG acquisition device (2), which It is connected to the VR display device (1), and is used to collect the EEG signal of the subject's head, and the EEG acquisition device (2) realizes data transmission through the communication module and an external terminal device.7.根据权利要求6所述的基于眼动追踪-脑功能活动检测的多模态沉浸式同步采集系统，其特征在于，所述脑电采集装置(2)为帽式结构，其包括多个电极(21)和与多个所述电极(21)连接的信号处理器(22)，所述电极(21)采集被试的脑电信息，并通过所述信号处理器(22)对所述脑电信息进行硬件滤波、信号增强和模数转换，所述信号处理器(22)通过所述通信模块与外接终端设备间数据传输。7. The multi-modal immersive synchronous acquisition system based on eye tracking-brain function activity detection according to claim 6, wherein the EEG acquisition device (2) is a cap-type structure, which comprises a plurality of Electrodes (21) and a signal processor (22) connected to a plurality of the electrodes (21), the electrodes (21) collect the EEG information of the subject, and use the signal processor (22) for the The EEG information is subjected to hardware filtering, signal enhancement and analog-to-digital conversion, and the signal processor (22) transmits data between the communication module and an external terminal device.8.根据权利要求7所述的基于眼动追踪-脑功能活动检测的多模态沉浸式同步采集系统，其特征在于，所述电极(21)为AgCl电极，其分布于人的头部特定区域，用于采集被试脑电信号。8. The multimodal immersive synchronous acquisition system based on eye tracking-brain function activity detection according to claim 7, wherein the electrode (21) is an AgCl electrode, which is distributed on a specific head of a person. The area is used to collect the subject's EEG signals.9.根据权利要求1所述的基于眼动追踪-脑功能活动检测的多模态沉浸式同步采集系统，其特征在于，所述通信模块为与有线USB端口数据进行数据传输和时钟同步的通用数据接口。9. The multimodal immersive synchronous acquisition system based on eye tracking-brain function activity detection according to claim 1, wherein the communication module is a general purpose data transmission and clock synchronization with wired USB port data. Data interface.10.根据权利要求1所述的基于眼动追踪-脑功能活动检测的多模态沉浸式同步采集系统，其特征在于，所述通信模块为蓝牙通信模块，其与外接终端设备间可以实现无线数据传输和时钟同步。10. The multi-modal immersive synchronous acquisition system based on eye tracking-brain function activity detection according to claim 1, wherein the communication module is a Bluetooth communication module, which can realize wireless communication with an external terminal device. Data transfer and clock synchronization.

Images