CN106096544A - Non-contact blink and heart rate joint detection system and method based on second-order blind identification - Google Patents

Abstract

Translated fromChinese

本发明涉及一种基于二阶盲辨识的非接触式眨眼与心率联合检测系统及方法，在受试者自然放松的状态下采集包含眼睛的面部视频数据，采集原始视频数据后选定检测面部区域并对所选视频数据进行预处理，得到预处理后的六通道信号，标记为x＝[x_R1,x_R2,x_G1,x_G2,x_B1,x_B2]^T；使用Meanshift算法得到连续帧中的目标区域，并以同样的方法获取六通道信号；将经过预处理后的原始信号作为输入信号，使用二阶盲辨识算法进行盲源分离，将分离后的源信号记为y＝[y₁,y₂,y₃,y₄,y₅,y₆]^T；对上述步骤S101中得到的分离信号y进行信号辨识与筛选，使用基于谱峭度的分离分量自动识别方法选出所需的眨眼和BVP信号；对得到的眨眼信号进行眨眼频率和时长计算，并对得到的BVP信号进行功率谱谱分析，得到心率估计值，本发明具有准确度高、抗干扰能力强、算法效率高等优点。

The invention relates to a non-contact blinking and heart rate joint detection system and method based on second-order blind recognition, which collects facial video data including eyes in a naturally relaxed state of the subject, and selects the facial area for detection after collecting the original video data And preprocess the selected video data to obtain the preprocessed six-channel signal, marked as x=[x_R1 , x_R2 , x_G1 , x_G2 , x_B1 , x_B2 ]^T ; use the Meanshift algorithm to obtain continuous frames The target area in , and obtain the six-channel signal in the same way; take the preprocessed original signal as the input signal, use the second-order blind identification algorithm to perform blind source separation, and record the separated source signal as y=[y₁ , y₂ , y₃ , y₄ , y₅ , y₆ ]^T ; carry out signal identification and screening on the separated signal y obtained in the above step S101, and use the automatic recognition method of separated components based on spectral kurtosis to select the required Blink and BVP signal; calculate the blink frequency and duration of the obtained blink signal, and perform power spectrum analysis on the obtained BVP signal to obtain an estimated heart rate value. The present invention has high accuracy, strong anti-interference ability, and high algorithm efficiency. advantage.

Description

Translated fromChinese

基于二阶盲辨识的非接触式眨眼与心率联合检测系统及方法System and method for non-contact blink and heart rate joint detection based on second-order blind recognition

技术领域technical field

本发明涉及人体体征研究应用技术领域，具体涉及一种基于二阶盲辨识的非接触式眨眼与心率联合检测系统及方法。The invention relates to the technical field of research and application of human body signs, in particular to a non-contact blinking and heart rate joint detection system and method based on second-order blind identification.

背景技术Background technique

眨眼和心跳作为人体的正常生理现象，两者都与个体生理状态和心理状态密切相关。心率是个体生理活动的重要指标而眨眼则能有效反映个体的心理与精神状态。随着技术的不断发展，眨眼和心率检测早已超越早期监测个体身心状态的基本目的，成为了新一代人体传感和人机交互技术的研究热点。相关的研究和工程应用在健康管理、疾病预防、特定场景下的个体状态检测、人机交互等方面具有广泛的应用前景。当前市面上已有智能手环等产品可以实现对个体的心率等指标进行接触式监测；三星、华为等知名IT厂商也将利用眨眼进行手机功能控制作为特色创新加入新的通信产品之中。由此可见，研究眨眼和心电信号的非接触式采集与分析具有巨大的实际意义。Blinking and heartbeat are normal physiological phenomena of the human body, both of which are closely related to individual physiological and psychological states. Heart rate is an important indicator of individual physiological activities, while blinking can effectively reflect the individual's psychological and spiritual state. With the continuous development of technology, eye blinking and heart rate detection have already surpassed the basic purpose of monitoring the individual's physical and mental state in the early stage, and have become a research hotspot in the new generation of human body sensing and human-computer interaction technology. Related research and engineering applications have broad application prospects in health management, disease prevention, individual state detection in specific scenarios, and human-computer interaction. At present, there are smart bracelets and other products on the market that can realize contact monitoring of individual heart rate and other indicators; well-known IT manufacturers such as Samsung and Huawei will also use blinking to control mobile phone functions as a feature innovation and add them to new communication products. Therefore, it is of great practical significance to study the non-contact acquisition and analysis of blinking and ECG signals.

随着信息技术的发展，现今已有多种心率及眼动监测方法及相关的商业产品，如常用的心电图和眼动仪等。但传统检测方法需要专门的设备和人员，不仅价格昂贵，人力成本高昂，且监测过程中给用户带来明显的不适感。近年来新起的非接触式检测方法因其不适感较少的优点已被普遍应用于医学领域。目前，眨眼多以视频图像的方式进行非接触式检测而心率非接触检测方法大致可分为电磁式检测方法、基于激光的检测方法、基于图像的检测方法、电阻法和超声波法等。由于图像处理方法的不断和高性能图像采集终端的日益普及，基于图像的检测方法已经日益成为主要的非接触式生理参数检测方法。尽管基于图像的非接触式检测方法使得检测过程大为便捷，检测成本也大为降低，但要检测眨眼和心率，目前仍需要两套独立的系统，眨眼和心率两者至今没有能够合理地融入一个统一的检测框架。With the development of information technology, there are many heart rate and eye movement monitoring methods and related commercial products, such as commonly used electrocardiogram and eye tracker. However, the traditional detection method requires specialized equipment and personnel, which is not only expensive and labor costly, but also brings obvious discomfort to users during the monitoring process. In recent years, the new non-contact detection method has been widely used in the medical field because of its advantages of less discomfort. At present, non-contact detection of blinking is mostly performed in the form of video images, and non-contact detection methods of heart rate can be roughly divided into electromagnetic detection methods, laser-based detection methods, image-based detection methods, resistance methods, and ultrasonic methods. Due to the continuous improvement of image processing methods and the increasing popularity of high-performance image acquisition terminals, image-based detection methods have increasingly become the main non-contact physiological parameter detection methods. Although the image-based non-contact detection method makes the detection process much more convenient and the detection cost is greatly reduced, two independent systems are still required to detect blinking and heart rate, and the blinking and heart rate have not been reasonably integrated so far. A unified detection framework.

PPG(PhotoPlethysmoGraphy)技术是一种用来检测微血管中血容量变化的光学测量技术。心脏的(准)周期性搏动会引起血管的周期性收缩与舒张，由此会产生与心脏跳动同步的血液容量变化脉冲(blood volume pulse，BVP)信号。通过对BVP信号的消噪和增强处理，可进一步获取心率、血氧饱和度和呼吸率等重要的生命体征信息。在目前已提出的各种基于PPG技术的BVP信号获取方法中，基于视频图像序列的BVP获取方法因其简便的使用方法和良好的用户体验受到了广泛关注。现有基于视频图像的PPG方法总体上可归结为两类，其一，将视频图像的G通道数据提取后直接进行滤波等简单后续处理后直接得到相应指标。其优点在于算法复杂度小，但G分量数据容易受到面部整体或局部(眼睛和嘴部)运动的影响。其二，利用其他分析处理算法对R、G、B三分量进行处理，以期获得更好的BVP检测效果。但现有分析处理方法仅着眼于检测由心脏搏动产生的BVP相关信号，检测对象的类型较为单一，若能在统一的框架下进行眨眼和心跳的同步检测将大大提高系统应用范围和实际效能。PPG (Photo PlethysmoGraphy) technology is an optical measurement technology used to detect changes in blood volume in microvessels. The (quasi) periodic beating of the heart will cause the periodic contraction and relaxation of blood vessels, thereby generating a blood volume pulse (BVP) signal synchronous with the beating of the heart. By denoising and enhancing the BVP signal, important vital sign information such as heart rate, blood oxygen saturation and respiration rate can be further obtained. Among the various BVP signal acquisition methods based on PPG technology that have been proposed so far, the BVP acquisition method based on video image sequences has attracted extensive attention because of its easy to use method and good user experience. Existing PPG methods based on video images can be generally classified into two categories. One is to extract the G channel data of the video image and directly perform filtering and other simple follow-up processing to directly obtain the corresponding indicators. Its advantage is that the algorithm complexity is small, but the G component data is easily affected by the overall or partial (eyes and mouth) movement of the face. Second, use other analysis and processing algorithms to process the three components of R, G, and B in order to obtain better BVP detection results. However, the existing analysis and processing methods only focus on the detection of BVP-related signals generated by heart beats, and the types of detection objects are relatively single. If the simultaneous detection of eye blinks and heartbeats can be performed under a unified framework, the application range and actual efficiency of the system will be greatly improved.

发明内容Contents of the invention

本发明的目的是：提供一种基于二阶盲辨识的非接触式眨眼与心率联合检测系统及方法，能够同时针对眨眼与心率进行检测，提高检测的效率。The purpose of the present invention is to provide a non-contact blink and heart rate joint detection system and method based on second-order blind identification, which can simultaneously detect blink and heart rate and improve detection efficiency.

为实现上述目的，本方法发明采用的技术方案是：基于二阶盲辨识的非接触式眨眼与心率联合检测方法，该联合检测方法包括如下步骤：In order to achieve the above purpose, the technical solution adopted in the invention of the method is: a non-contact blinking and heart rate joint detection method based on second-order blind identification. The joint detection method includes the following steps:

S100、在受试者自然放松的状态下采集包含眼睛的面部视频数据，采集原始视频数据后选定检测面部区域并对所选视频数据进行预处理，得到预处理后的六通道信号，标记为x＝[x_R1,x_R2,x_G1,x_G2,x_B1,x_B2]^T；S100. Collect facial video data including eyes in a naturally relaxed state of the subject, select the detected facial area after collecting the original video data, and preprocess the selected video data to obtain preprocessed six-channel signals, which are marked as x=[x_R1 , x_R2 , x_G1 , x_G2 , x_B1 , x_B2 ]^T ;

S101、使用Meanshift算法得到连续帧中的目标区域，并以同样的方法获取六通道信号；S101. Use the Meanshift algorithm to obtain the target area in the continuous frames, and obtain six-channel signals in the same way;

S102、将经过预处理后的原始信号作为输入信号，使用二阶盲辨识算法进行盲源分离，将分离后的源信号记为y＝[y₁,y₂,y₃,y₄,y₅,y₆]^T；S102. Using the preprocessed original signal as the input signal, use the second-order blind identification algorithm to perform blind source separation, and record the separated source signal as y=[y₁ ,y₂ ,y₃ ,y₄ ,y₅ ,y₆ ]^T ;

S103、对上述步骤S101中得到的分离信号y进行信号辨识与筛选，使用基于谱峭度的分离分量自动识别方法选出所需的眨眼和BVP信号；S103. Perform signal identification and screening on the separated signal y obtained in the above step S101, and select the required blink and BVP signals by using the automatic identification method of the separated components based on spectral kurtosis;

S104、对得到的眨眼信号进行眨眼频率和时长计算，并对得到的BVP信号进行功率谱谱分析，得到心率估计值。S104. Calculate the blink frequency and duration of the obtained blink signal, and perform power spectrum analysis on the obtained BVP signal to obtain an estimated heart rate value.

本方法发明还存在以下特征：The method invention also has the following features:

所述S100步骤中的视频数据进行预处理的方法包括帧内空间平均的步骤及高通滤波、标准化的步骤。The method for preprocessing the video data in step S100 includes the steps of intra-frame spatial averaging, high-pass filtering, and standardization.

用于进行帧内空间平均步骤的人脸区域是眼睛及眼周的小块面部区域，帧内空间平均的步骤为：The face area used for the intra-frame spatial averaging step is the eyes and small facial areas around the eyes, and the intra-frame spatial averaging steps are:

设选取的人脸区域图像为x_i,j(t)；1≤i≤N,1≤j≤M}，{x_R,x_G,x_B}为图像的R、G、B三基色分量，对人脸视频序列先进行逐帧空间平均得到三通道信号，即Let the selected face area image be x_{i, j} (t); 1≤i≤N, 1≤j≤M}, {x_R , x_G , x_B } are the R, G, B three primary color components of the image , the face video sequence is firstly averaged frame by frame to obtain a three-channel signal, namely

$\begin{array}{c}\mathrm{<span><span>x</span>x</span>}<span><span>=</span>=</span>\left\{\begin{array}{cc}\frac{\mathrm{<span><span>1</span>1</span>}}{\mathrm{<span><span>M</span>m</span>}\mathrm{<span><span>N</span>N</span>}}\underset{\mathrm{<span><span>x</span>x</span>}<span><span>\&Element;</span>\&Element;</span>\mathrm{<span><span>R</span>R</span>}}{<span><span>\&Sigma;</span>\&Sigma;</span>}{\mathrm{<span><span>x</span>x</span>}}_{\mathrm{<span><span>i</span>i</span>}<span><span>,</span>,</span>\mathrm{<span><span>j</span>j</span>}}<span><span>(</span>(</span>\mathrm{<span><span>t</span>t</span>}<span><span>)</span>)</span>& {}_{\mathrm{<span><span>i</span>i</span>}<span><span>=</span>=</span>\mathrm{<span><span>1</span>1</span>}<span><span>,</span>,</span><span><span>...</span>...</span><span><span>,</span>,</span>\mathrm{<span><span>N</span>N</span>}<span><span>;</span>;</span>\mathrm{<span><span>j</span>j</span>}<span><span>=</span>=</span>\mathrm{<span><span>1</span>1</span>}<span><span>,</span>,</span><span><span>...</span>...</span><span><span>,</span>,</span>\mathrm{<span><span>M</span>m</span>}}\\ \frac{\mathrm{<span><span>1</span>1</span>}}{\mathrm{<span><span>M</span>m</span>}\mathrm{<span><span>N</span>N</span>}}\underset{\mathrm{<span><span>x</span>x</span>}<span><span>\&Element;</span>\&Element;</span>\mathrm{<span><span>G</span>G</span>}}{<span><span>\&Sigma;</span>\&Sigma;</span>}{\mathrm{<span><span>x</span>x</span>}}_{\mathrm{<span><span>i</span>i</span>}<span><span>,</span>,</span>\mathrm{<span><span>j</span>j</span>}}<span><span>(</span>(</span>\mathrm{<span><span>t</span>t</span>}<span><span>)</span>)</span>& {}_{\mathrm{<span><span>i</span>i</span>}<span><span>=</span>=</span>\mathrm{<span><span>1</span>1</span>}<span><span>,</span>,</span><span><span>...</span>...</span><span><span>,</span>,</span>\mathrm{<span><span>N</span>N</span>}<span><span>;</span>;</span>\mathrm{<span><span>j</span>j</span>}<span><span>=</span>=</span>\mathrm{<span><span>1</span>1</span>}<span><span>,</span>,</span><span><span>...</span>...</span><span><span>,</span>,</span>\mathrm{<span><span>M</span>m</span>}}\\ \frac{\mathrm{<span><span>1</span>1</span>}}{\mathrm{<span><span>M</span>m</span>}\mathrm{<span><span>N</span>N</span>}}\underset{\mathrm{<span><span>x</span>x</span>}<span><span>\&Element;</span>\&Element;</span>\mathrm{<span><span>B</span>B</span>}}{<span><span>\&Sigma;</span>\&Sigma;</span>}{\mathrm{<span><span>x</span>x</span>}}_{\mathrm{<span><span>i</span>i</span>}<span><span>,</span>,</span>\mathrm{<span><span>j</span>j</span>}}<span><span>(</span>(</span>\mathrm{<span><span>t</span>t</span>}<span><span>)</span>)</span>& {}_{\mathrm{<span><span>i</span>i</span>}<span><span>=</span>=</span>\mathrm{<span><span>1</span>1</span>}<span><span>,</span>,</span><span><span>...</span>...</span><span><span>,</span>,</span>\mathrm{<span><span>N</span>N</span>}<span><span>;</span>;</span>\mathrm{<span><span>j</span>j</span>}<span><span>=</span>=</span>\mathrm{<span><span>1</span>1</span>}<span><span>,</span>,</span><span><span>...</span>...</span><span><span>,</span>,</span>\mathrm{<span><span>M</span>m</span>}}\end{array}\right\}<span><span>=</span>=</span>\left[\begin{array}{c}{\mathrm{<span><span>x</span>x</span>}}_{\mathrm{<span><span>R</span>R</span>}}<span><span>(</span>(</span>\mathrm{<span><span>t</span>t</span>}<span><span>)</span>)</span>\\ {\mathrm{<span><span>x</span>x</span>}}_{\mathrm{<span><span>G</span>G</span>}}<span><span>(</span>(</span>\mathrm{<span><span>t</span>t</span>}<span><span>)</span>)</span>\\ {\mathrm{<span><span>x</span>x</span>}}_{\mathrm{<span><span>b</span>b</span>}}<span><span>(</span>(</span>\mathrm{<span><span>t</span>t</span>}<span><span>)</span>)</span>\end{array}\right]\\ <span><span>=</span>=</span>\left[\begin{array}{cccc}{\mathrm{<span><span>x</span>x</span>}}_{\mathrm{<span><span>R</span>R</span>}}<span><span>(</span>(</span>\mathrm{<span><span>1</span>1</span>}<span><span>)</span>)</span>& {\mathrm{<span><span>x</span>x</span>}}_{\mathrm{<span><span>R</span>R</span>}}<span><span>(</span>(</span>\mathrm{<span><span>2</span>2</span>}<span><span>)</span>)</span>& \mathrm{<span><span>L</span>L</span>}& {\mathrm{<span><span>x</span>x</span>}}_{\mathrm{<span><span>R</span>R</span>}}<span><span>(</span>(</span>\mathrm{<span><span>T</span>T</span>}<span><span>)</span>)</span>\\ {\mathrm{<span><span>x</span>x</span>}}_{\mathrm{<span><span>G</span>G</span>}}<span><span>(</span>(</span>\mathrm{<span><span>1</span>1</span>}<span><span>)</span>)</span>& {\mathrm{<span><span>x</span>x</span>}}_{\mathrm{<span><span>G</span>G</span>}}<span><span>(</span>(</span>\mathrm{<span><span>2</span>2</span>}<span><span>)</span>)</span>& \mathrm{<span><span>L</span>L</span>}& {\mathrm{<span><span>x</span>x</span>}}_{\mathrm{<span><span>G</span>G</span>}}<span><span>(</span>(</span>\mathrm{<span><span>T</span>T</span>}<span><span>)</span>)</span>\\ {\mathrm{<span><span>x</span>x</span>}}_{\mathrm{<span><span>B</span>B</span>}}<span><span>(</span>(</span>\mathrm{<span><span>1</span>1</span>}<span><span>)</span>)</span>& {\mathrm{<span><span>x</span>x</span>}}_{\mathrm{<span><span>B</span>B</span>}}<span><span>(</span>(</span>\mathrm{<span><span>2</span>2</span>}<span><span>)</span>)</span>& \mathrm{<span><span>L</span>L</span>}& {\mathrm{<span><span>x</span>x</span>}}_{\mathrm{<span><span>B</span>B</span>}}<span><span>(</span>(</span>\mathrm{<span><span>T</span>T</span>}<span><span>)</span>)</span>\end{array}\right]\end{array}$

式(1)中N，M为选定面部图像区域的高和宽，t为每帧图像对应的时间；In the formula (1), N, M are the height and width of the selected facial image area, and t is the corresponding time of each frame image;

对每帧图像按照视频图像宽度中线切分为两部分，再分别按照上述方法对R，G，B三分量进行进总体平均，得到六通道数据：Divide each frame of image into two parts according to the midline of the width of the video image, and then carry out the overall average of the three components of R, G, and B according to the above method to obtain six-channel data:

$\left[\begin{array}{c}{\mathrm{<span><span>x</span>x</span>}}_{\mathrm{<span><span>R</span>R</span>}<span><span>,</span>,</span>\mathrm{<span><span>1</span>1</span>}}<span><span>(</span>(</span>\mathrm{<span><span>t</span>t</span>}<span><span>)</span>)</span>\\ {\mathrm{<span><span>x</span>x</span>}}_{\mathrm{<span><span>R</span>R</span>}<span><span>,</span>,</span>\mathrm{<span><span>2</span>2</span>}}<span><span>(</span>(</span>\mathrm{<span><span>t</span>t</span>}<span><span>)</span>)</span>\\ {\mathrm{<span><span>x</span>x</span>}}_{\mathrm{<span><span>G</span>G</span>}<span><span>,</span>,</span>\mathrm{<span><span>1</span>1</span>}}<span><span>(</span>(</span>\mathrm{<span><span>t</span>t</span>}<span><span>)</span>)</span>\\ {\mathrm{<span><span>x</span>x</span>}}_{\mathrm{<span><span>G</span>G</span>}<span><span>,</span>,</span>\mathrm{<span><span>2</span>2</span>}}<span><span>(</span>(</span>\mathrm{<span><span>t</span>t</span>}<span><span>)</span>)</span>\\ {\mathrm{<span><span>x</span>x</span>}}_{\mathrm{<span><span>B</span>B</span>}<span><span>,</span>,</span>\mathrm{<span><span>1</span>1</span>}}<span><span>(</span>(</span>\mathrm{<span><span>t</span>t</span>}<span><span>)</span>)</span>\\ {\mathrm{<span><span>x</span>x</span>}}_{\mathrm{<span><span>B</span>B</span>}<span><span>,</span>,</span>\mathrm{<span><span>2</span>2</span>}}<span><span>(</span>(</span>\mathrm{<span><span>t</span>t</span>}<span><span>)</span>)</span>\end{array}\right]<span><span>=</span>=</span>\left[\begin{array}{cccc}{\mathrm{<span><span>x</span>x</span>}}_{\mathrm{<span><span>1</span>1</span>}}<span><span>(</span>(</span>\mathrm{<span><span>1</span>1</span>}<span><span>)</span>)</span>& {\mathrm{<span><span>x</span>x</span>}}_{\mathrm{<span><span>1</span>1</span>}}<span><span>(</span>(</span>\mathrm{<span><span>2</span>2</span>}<span><span>)</span>)</span>& \mathrm{<span><span>L</span>L</span>}& {\mathrm{<span><span>x</span>x</span>}}_{\mathrm{<span><span>1</span>1</span>}}<span><span>(</span>(</span>\mathrm{<span><span>T</span>T</span>}<span><span>)</span>)</span>\\ {\mathrm{<span><span>x</span>x</span>}}_{\mathrm{<span><span>2</span>2</span>}}<span><span>(</span>(</span>\mathrm{<span><span>1</span>1</span>}<span><span>)</span>)</span>& {\mathrm{<span><span>x</span>x</span>}}_{\mathrm{<span><span>2</span>2</span>}}<span><span>(</span>(</span>\mathrm{<span><span>2</span>2</span>}<span><span>)</span>)</span>& \mathrm{<span><span>L</span>L</span>}& {\mathrm{<span><span>x</span>x</span>}}_{\mathrm{<span><span>2</span>2</span>}}<span><span>(</span>(</span>\mathrm{<span><span>T</span>T</span>}<span><span>)</span>)</span>\\ \mathrm{<span><span>M</span>m</span>}& \mathrm{<span><span>M</span>m</span>}& \mathrm{<span><span>M</span>m</span>}& \mathrm{<span><span>M</span>m</span>}\\ {\mathrm{<span><span>x</span>x</span>}}_{\mathrm{<span><span>6</span>6</span>}}<span><span>(</span>(</span>\mathrm{<span><span>1</span>1</span>}<span><span>)</span>)</span>& {\mathrm{<span><span>x</span>x</span>}}_{\mathrm{<span><span>6</span>6</span>}}<span><span>(</span>(</span>\mathrm{<span><span>2</span>2</span>}<span><span>)</span>)</span>& \mathrm{<span><span>L</span>L</span>}& {\mathrm{<span><span>x</span>x</span>}}_{\mathrm{<span><span>6</span>6</span>}}<span><span>(</span>(</span>\mathrm{<span><span>T</span>T</span>}<span><span>)</span>)</span>\end{array}\right]<span><span>.</span>.</span>$

用于进行高通滤波步骤的高通滤波器的截止频率为0.8Hz。The cut-off frequency of the high-pass filter used for the high-pass filtering step was 0.8 Hz.

用于进行标准化步骤的计算过程为：The calculation procedure used to perform the normalization step is:

$\mathrm{<span><span>x</span>x</span>}<span><span>\&LeftArrow;</span>\&LeftArrow;</span>\frac{\mathrm{<span><span>x</span>x</span>}<span><span>-</span>-</span>\mathrm{<span><span>E</span>E.</span>}<span><span>\&lsqb;</span>\&lsqb;</span>\mathrm{<span><span>x</span>x</span>}<span><span>\&rsqb;</span>\&rsqb;</span>}{\mathrm{<span><span>\&sigma;</span>\&sigma;</span>}}$

式中x为原始信号，E[x]为求均值运算，σ为x中各分量的标准差。In the formula, x is the original signal, E[x] is the mean value operation, and σ is the standard deviation of each component in x.

所述步骤S100中，将x＝[x_R,1,x_R,2,x_G,1,x_G,2,x_B,1,x_B,2]^T重新定义为x＝[x₁,x₂,x₃,x₄,x₅,x₆]^T，则可将混合信号生成方式表达为：In the step S100, x=[x_R,1 ,x_R,2 ,x_G,1 ,x_G,2 ,x_B,1 ,x_B,2 ]^T is redefined as x=[x₁ , x₂ ,x₃ ,x₄ ,x₅ ,x₆ ]^T , then the mixed signal generation method can be expressed as:

$\mathrm{<span><span>X</span>x</span>}<span><span>=</span>=</span>\left[\begin{array}{c}{\mathrm{<span><span>x</span>x</span>}}_{\mathrm{<span><span>1</span>1</span>}}\\ {\mathrm{<span><span>x</span>x</span>}}_{\mathrm{<span><span>2</span>2</span>}}\\ \mathrm{<span><span>M</span>m</span>}\\ {\mathrm{<span><span>x</span>x</span>}}_{\mathrm{<span><span>6</span>6</span>}}\end{array}\right]<span><span>=</span>=</span>\left[\begin{array}{cccc}{\mathrm{<span><span>a</span>a</span>}}_{\mathrm{<span><span>11</span>11</span>}}& {\mathrm{<span><span>a</span>a</span>}}_{\mathrm{<span><span>12</span>12</span>}}& \mathrm{<span><span>L</span>L</span>}& {\mathrm{<span><span>a</span>a</span>}}_{\mathrm{<span><span>16</span>16</span>}}\\ {\mathrm{<span><span>a</span>a</span>}}_{\mathrm{<span><span>21</span>twenty\; one</span>}}& {\mathrm{<span><span>a</span>a</span>}}_{\mathrm{<span><span>22</span>twenty\; two</span>}}& \mathrm{<span><span>L</span>L</span>}& {\mathrm{<span><span>a</span>a</span>}}_{\mathrm{<span><span>26</span>26</span>}}\\ \mathrm{<span><span>M</span>m</span>}& \mathrm{<span><span>M</span>m</span>}& \mathrm{<span><span>M</span>m</span>}& \mathrm{<span><span>M</span>m</span>}\\ {\mathrm{<span><span>a</span>a</span>}}_{\mathrm{<span><span>61</span>61</span>}}& {\mathrm{<span><span>a</span>a</span>}}_{\mathrm{<span><span>62</span>62</span>}}& \mathrm{<span><span>L</span>L</span>}& {\mathrm{<span><span>a</span>a</span>}}_{\mathrm{<span><span>66</span>66</span>}}\end{array}\right]\left[\begin{array}{c}{\mathrm{<span><span>s</span>the\; s</span>}}_{\mathrm{<span><span>1</span>1</span>}}\\ {\mathrm{<span><span>s</span>the\; s</span>}}_{\mathrm{<span><span>2</span>2</span>}}\\ \mathrm{<span><span>M</span>m</span>}\\ {\mathrm{<span><span>s</span>the\; s</span>}}_{\mathrm{<span><span>6</span>6</span>}}\end{array}\right]<span><span>=</span>=</span>\mathrm{<span><span>A</span>A</span>}\mathrm{<span><span>s</span>the\; s</span>}$

式中A为标量混合矩阵，s＝[s₁,s₂,s₃,s₄,s₅,s₆]^T代表隐含的多个源信号，其中包含了眨眼信号分量和心跳BVP分量以及其他成分，式中的混合矩阵A和源s均未知，通过使用基于二阶盲辨识的盲分离算法可以得到对真实源s的估计，即满足如下关系的y：In the formula, A is a scalar mixing matrix, s=[s₁ , s₂ , s₃ , s₄ , s₅ , s₆ ]^T represents multiple hidden source signals, which include the blink signal component and the heartbeat BVP component and Other components, the mixing matrix A and the source s in the formula are unknown, and the estimation of the real source s can be obtained by using the blind separation algorithm based on the second-order blind identification, that is, y that satisfies the following relationship:

$\mathrm{<span><span>y</span>the\; y</span>}<span><span>=</span>=</span>\left[\begin{array}{c}{\mathrm{<span><span>y</span>the\; y</span>}}_{\mathrm{<span><span>1</span>1</span>}}\\ {\mathrm{<span><span>y</span>the\; y</span>}}_{\mathrm{<span><span>2</span>2</span>}}\\ \mathrm{<span><span>M</span>m</span>}\\ {\mathrm{<span><span>y</span>the\; y</span>}}_{\mathrm{<span><span>6</span>6</span>}}\end{array}\right]<span><span>=</span>=</span>\left[\begin{array}{cccc}{\mathrm{<span><span>w</span>w</span>}}_{\mathrm{<span><span>11</span>11</span>}}& {\mathrm{<span><span>w</span>w</span>}}_{\mathrm{<span><span>12</span>12</span>}}& \mathrm{<span><span>L</span>L</span>}& {\mathrm{<span><span>w</span>w</span>}}_{\mathrm{<span><span>16</span>16</span>}}\\ {\mathrm{<span><span>w</span>w</span>}}_{\mathrm{<span><span>21</span>twenty\; one</span>}}& {\mathrm{<span><span>w</span>w</span>}}_{\mathrm{<span><span>22</span>twenty\; two</span>}}& \mathrm{<span><span>L</span>L</span>}& {\mathrm{<span><span>w</span>w</span>}}_{\mathrm{<span><span>26</span>26</span>}}\\ \mathrm{<span><span>M</span>m</span>}& \mathrm{<span><span>M</span>m</span>}& \mathrm{<span><span>M</span>m</span>}& \mathrm{<span><span>M</span>m</span>}\\ {\mathrm{<span><span>w</span>w</span>}}_{\mathrm{<span><span>61</span>61</span>}}& {\mathrm{<span><span>w</span>w</span>}}_{\mathrm{<span><span>62</span>62</span>}}& \mathrm{<span><span>L</span>L</span>}& {\mathrm{<span><span>w</span>w</span>}}_{\mathrm{<span><span>66</span>66</span>}}\end{array}\right]\left[\begin{array}{c}{\mathrm{<span><span>x</span>x</span>}}_{\mathrm{<span><span>1</span>1</span>}}\\ {\mathrm{<span><span>x</span>x</span>}}_{\mathrm{<span><span>2</span>2</span>}}\\ \mathrm{<span><span>M</span>m</span>}\\ {\mathrm{<span><span>x</span>x</span>}}_{\mathrm{<span><span>6</span>6</span>}}\end{array}\right]<span><span>=</span>=</span>\mathrm{<span><span>W</span>W</span>}\mathrm{<span><span>x</span>x</span>}$

式中分离矩阵W为对A^-1的近似估计。In the formula, the separation matrix W is an approximate estimate of A^-1 .

为实现上述目的，本系统发明采用的技术方案是：基于二阶盲辨识的非接触式眨眼与心率联合检测系统，该系统包括视频信号采集与预处理模块、目标跟踪模块、多通道盲分离模块、眨眼及BVP信号筛选模块、眨眼频率计算及时长计算模块与心率估计模块；In order to achieve the above purpose, the technical solution adopted by this system invention is: a non-contact blinking and heart rate joint detection system based on second-order blind identification, the system includes a video signal acquisition and preprocessing module, a target tracking module, and a multi-channel blind separation module , Blink and BVP signal screening module, blink frequency calculation and duration calculation module and heart rate estimation module;

所述视频信号采集与预处理模块，用于获取人脸部图像并使用相干平均法将采集到的包含眼部区域的人脸视频数据按照R,G,B三基色通道进行空间平均，并采用图像宽度中值切分的方法得到六通道原始信号x＝[x_R1,x_G1,x_B1,x_R2,x_G2,x_B2]^T，同时对原始信号x进行高通滤波以去除信号中各分量的低频趋势并对信号进行去均值与方差归一化；The video signal acquisition and preprocessing module is used to acquire human face images and use the coherent averaging method to spatially average the collected human face video data including the eye region according to R, G, and B three primary color channels, and adopt The method of image width median segmentation obtains six-channel original signal x=[x_R1 ,x_G1 ,x_B1 ,x_R2 ,x_G2 ,x_B2 ]^T , and performs high-pass filtering on the original signal x to remove each component in the signal The low-frequency trend of the signal and normalize the mean value and variance of the signal;

所述目标跟踪模块使用Meanshift目标跟踪算法实现对选定面部区域的连续定位；The target tracking module uses the Meanshift target tracking algorithm to realize the continuous positioning of selected facial regions;

所述多通道盲分离模块用于通过基于二阶盲辨识的多通道盲分离算法对原始信号x进行分离以得到对不同源信号的估计；The multi-channel blind separation module is used to separate the original signal x through a multi-channel blind separation algorithm based on second-order blind identification to obtain estimates of different source signals;

所述眨眼及BVP信号筛选模块用于从分离得到的多通道数据中辨识出眨眼和BVP信号。The blink and BVP signal screening module is used to identify blink and BVP signals from the separated multi-channel data.

眨眼频率计算及时长计算模块与心率估计模块用于从分离得到的眨眼和BVP信号中计算得到眨眼时长、眨眼频率与心率。The blink frequency calculation and duration calculation module and the heart rate estimation module are used to calculate the blink duration, blink frequency and heart rate from the separated blink and BVP signals.

该系统还存在以下特征：The system also has the following features:

所述视频信号采集与预处理模块包括帧内空间平均单元、高通滤波单元及标准化单元。The video signal acquisition and preprocessing module includes an intra-frame spatial averaging unit, a high-pass filtering unit and a normalization unit.

高通滤波单元中的高通滤波器的截止频率为0.8Hz。The cut-off frequency of the high-pass filter in the high-pass filter unit is 0.8 Hz.

所述目标跟踪模块使用Meanshift目标跟踪算法进行连续帧间的眼部区域定位，所述眨眼频率计算及时长计算模块与心率估计模块使用计数器计算单位时间内的眨眼次数得到眨眼频率，使用幅度门限得到眨眼脉冲的时长，使用BVP信号的功率谱峰值×60得到心率。The target tracking module uses the Meanshift target tracking algorithm to locate the eye region between consecutive frames, the blink frequency calculation and duration calculation module and the heart rate estimation module use a counter to calculate the number of blinks per unit time to obtain the blink frequency, and use the amplitude threshold to obtain the blink frequency. The duration of the eyeblink pulse was obtained by using the peak value of the power spectrum of the BVP signal × 60 to obtain the heart rate.

与现有技术相比，本发明具备的技术效果为：本发明中借助于多通道盲分离算法实现了对眨眼和BVP信号在内的多源信号的检测。在不增加算法复杂度的情况下实现了对眨眼和BVP信号的同时提取；使用多通道盲分离算法其分离精度更高，因而可以使用仅包含单一眼部的小块面部区域视频即可实现对心率和眨眼的联合检测，使得算法的计算量进一步降低；而且使用基于谱峭度的眨眼与BVP分量自动识别方法，能够从盲分离算法的输出信号中准确的筛选出眨眼与BVP信号，解决盲分离算法排序模糊问题；另外，使用Meanshift目标跟踪算法进行连续帧间的目标区域定位，允许使用者在采集视频图像时存在一定的自然肢体动作，具有更强的抗运动干扰性能；采用二阶盲辨识算法对信号进行盲源分离，能起到较明显的眨眼与BVP信号增强作用，在很大程度上消除光照以及外界环境变化带来的一些干扰。Compared with the prior art, the present invention has the following technical effects: the present invention realizes the detection of multi-source signals including eye blink and BVP signal by means of a multi-channel blind separation algorithm. Simultaneous extraction of blinking and BVP signals is realized without increasing the complexity of the algorithm; the separation accuracy is higher by using the multi-channel blind separation algorithm, so it can be realized by using a small facial area video containing only a single eye The joint detection of heart rate and eye blink further reduces the amount of calculation of the algorithm; and the automatic recognition method of eye blink and BVP components based on spectral kurtosis can accurately screen out eye blink and BVP signals from the output signal of the blind separation algorithm to solve the problem of blindness. Separation algorithm sorting fuzzy problem; in addition, using the Meanshift target tracking algorithm to locate the target area between consecutive frames, allowing users to have certain natural body movements when collecting video images, and has stronger anti-motion interference performance; using second-order blindness The identification algorithm performs blind source separation on the signal, which can play a more obvious role in enhancing the blinking and BVP signals, and largely eliminates some interference caused by illumination and changes in the external environment.

除了上面所描述的目的、特征和优点之外，本发明还有其它的目的、特征和优点。下面将参照图，对本发明作进一步详细的说明。In addition to the objects, features and advantages described above, the present invention has other objects, features and advantages. Hereinafter, the present invention will be described in further detail with reference to the drawings.

附图说明Description of drawings

构成本申请的一部分的说明书附图用来提供对本发明的进一步理解，本发明的示意性实施例及其说明用于解释本发明，并不构成对本发明的不当限定。在附图中：The accompanying drawings constituting a part of the present application are used to provide a further understanding of the present invention, and the schematic embodiments and descriptions of the present invention are used to explain the present invention, and do not constitute an improper limitation of the present invention. In the attached picture:

图1是基于二阶盲辨识的非接触式眨眼与心率联合检测方法的逻辑框图；Figure 1 is a logic block diagram of a non-contact blink and heart rate joint detection method based on second-order blind identification;

图2是基于二阶盲辨识的非接触式眨眼与心率联合检测系统逻辑框图；Figure 2 is a logic block diagram of a non-contact blink and heart rate joint detection system based on second-order blind identification;

图3是基于二阶盲辨识的非接触式眨眼与心率联合检测方法的基本流程图；Fig. 3 is the basic flowchart of the non-contact blinking and heart rate joint detection method based on second-order blind identification;

图4是本发明中选取的人脸眼部区域示意图；Fig. 4 is a schematic diagram of the human face eye region selected in the present invention;

图5是本发明中经过预处理模块得到的观测信号波形图；Fig. 5 is the observed signal waveform figure obtained through the preprocessing module in the present invention;

图6是本发明中经过二阶盲辨识算法盲源分离后的输出信号波形图；Fig. 6 is the waveform diagram of the output signal after the blind source separation of the second-order blind identification algorithm in the present invention;

图7是本发明中自动筛选出的眨眼和BVP信号波形图；Fig. 7 is the blink and the BVP signal waveform figure that screens out automatically among the present invention;

图8是基于检测出的眨眼波形计算眨眼时长和频率的示意图；Fig. 8 is a schematic diagram of calculating blink duration and frequency based on detected blink waveforms;

图9是基于检测出的BVP信号计算的功率谱幅频特性图.Figure 9 is a diagram of the amplitude-frequency characteristics of the power spectrum calculated based on the detected BVP signal.

具体实施方式detailed description

结合图1至图9本发明作进一步地说明：The present invention is further described in conjunction with Fig. 1 to Fig. 9:

基于二阶盲辨识的非接触式眨眼与心率联合检测方法，该联合检测方法包括如下步骤：A non-contact blinking and heart rate joint detection method based on second-order blind identification, the joint detection method includes the following steps:

S100、在受试者自然放松的状态下采集包含眼睛的面部视频数据，采集原始视频数据后选定检测面部区域并对所选视频数据进行预处理，得到预处理后的六通道信号，标记为x＝[x_R1,x_R2,x_G1,x_G2,x_B1,x_B2]^T；S100. Collect facial video data including eyes in a naturally relaxed state of the subject, select the detected facial area after collecting the original video data, and preprocess the selected video data to obtain preprocessed six-channel signals, which are marked as x=[x_R1 , x_R2 , x_G1 , x_G2 , x_B1 , x_B2 ]^T ;

S101、使用Meanshift算法得到连续帧中的目标区域，并以同样的方法获取六通道信号；S101. Use the Meanshift algorithm to obtain the target area in the continuous frames, and obtain six-channel signals in the same way;

S102、将经过预处理后的原始信号作为输入信号，使用二阶盲辨识算法进行盲源分离，将分离后的源信号记为y＝[y₁,y₂,y₃,y₄,y₅,y₆]^T；S102. Using the preprocessed original signal as the input signal, use the second-order blind identification algorithm to perform blind source separation, and record the separated source signal as y=[y₁ ,y₂ ,y₃ ,y₄ ,y₅ ,y₆ ]^T ;

S103、对上述步骤S101中得到的分离信号y进行信号辨识与筛选，使用基于谱峭度的分离分量自动识别方法选出所需的眨眼和BVP信号；S103. Perform signal identification and screening on the separated signal y obtained in the above step S101, and select the required blink and BVP signals by using the automatic identification method of the separated components based on spectral kurtosis;

S104、对得到的眨眼信号进行眨眼频率和时长计算，并对得到的BVP信号进行功率谱谱分析，得到心率估计值。S104. Calculate the blink frequency and duration of the obtained blink signal, and perform power spectrum analysis on the obtained BVP signal to obtain an estimated heart rate value.

本发明实现了从面部有限区域视频中同步提取出眨眼和心率信号；本发明中借助于多通道盲分离算法实现了对眨眼和BVP信号在内的多源信号的检测；眨眼作为和个体心理状态密切相关的生理活动，一直以来缺乏简便高效的检测途径；本发明将传统三通道二阶盲辨识方法扩展到六通道，显著提升了传统PPG方法的多目标检测能力；在不增加算法复杂度的情况下实现了对眨眼和BVP信号的同时提取；通过判读视频统计眨眼信息发现该方法的眨眼检测正确率达到95％以上，与专用脉搏传感器的实际对比表明算法的心率检测正确率达到93％以上；表1给出了本申请书所提眨眼检测方法与传统基于EOG信号的眨眼检测方法的检测结果正确率对比。表2给出了不同使用条件下本申请书所提心率检测方法与标准脉搏血氧仪获取的心率结果对比。The present invention realizes synchronously extracting eye blink and heart rate signal from the video of the limited area of the face; in the present invention, the detection of multi-source signals including eye blink and BVP signal is realized by means of multi-channel blind separation algorithm; Closely related physiological activities have always lacked simple and efficient detection methods; the present invention extends the traditional three-channel second-order blind identification method to six channels, which significantly improves the multi-target detection capability of the traditional PPG method; without increasing the complexity of the algorithm The simultaneous extraction of blink and BVP signals is realized under certain circumstances; through the interpretation of video statistical blink information, it is found that the blink detection accuracy rate of this method reaches more than 95%, and the actual comparison with the special pulse sensor shows that the heart rate detection accuracy rate of the algorithm reaches more than 93%. ; Table 1 shows the comparison of the accuracy of detection results between the blink detection method proposed in this application and the traditional blink detection method based on EOG signals. Table 2 shows the comparison of heart rate results obtained by the heart rate detection method proposed in this application and the standard pulse oximeter under different conditions of use.

表1Table 1

表2Table 2

本发明实现了基于单一眼部区域的生理参数检测，传统方法检测眨眼和BVP信号要基于整个面部区域或者双眼区域，而本发明使用多通道盲分离算法其分离精度更高，因而可以使用仅包含单一眼部的小块面部区域视频即可实现对心率和眨眼的联合检测，使得算法的计算量进一步降低；The present invention realizes the detection of physiological parameters based on a single eye area. The traditional method for detecting blinking and BVP signals is based on the entire face area or binocular area. However, the present invention uses a multi-channel blind separation algorithm with higher separation accuracy, so it can use only A small face area video of a single eye can realize the joint detection of heart rate and blinking, which further reduces the calculation amount of the algorithm;

本发明具有准确选择BVP信号的能力，使用基于谱峭度的眨眼与BVP分量自动识别方法，能够从盲分离算法的输出信号中准确的筛选出眨眼与BVP信号，解决盲分离算法排序模糊问题。The invention has the ability to accurately select BVP signals, uses the automatic identification method of blink and BVP components based on spectral kurtosis, can accurately screen out blink and BVP signals from output signals of the blind separation algorithm, and solves the fuzzy problem of blind separation algorithm sorting.

本发明具有较强的抗干扰能力，本发明使用Meanshift目标跟踪算法进行连续帧间的目标区域定位，允许使用者在采集视频图像时存在一定的自然肢体动作，具有更强的抗运动干扰性能；本发明采用二阶盲辨识算法对信号进行盲源分离，能起到较明显的眨眼与BVP信号增强作用，在很大程度上消除光照以及外界环境变化带来的一些干扰；The present invention has strong anti-interference ability. The present invention uses the Meanshift target tracking algorithm to locate the target area between consecutive frames, allowing users to have certain natural body movements when collecting video images, and has stronger anti-motion interference performance; The present invention adopts the second-order blind identification algorithm to perform blind source separation on the signal, which can play a more obvious role of blinking and BVP signal enhancement, and largely eliminate some interference caused by illumination and changes in the external environment;

眨眼与心率除了与身体的健康状态有关之外，还与个体行为动作，情绪变化等因素直接相关。此外，从非传统生理监护领域看，眨眼与心率可作为全新的人机交互的信息接口，在游戏、虚拟视觉、安全监控、测谎等方面都有广泛的应用前景。Blinking and heart rate are not only related to the health status of the body, but also directly related to individual behaviors, emotional changes and other factors. In addition, from the perspective of non-traditional physiological monitoring, eye blink and heart rate can be used as a new information interface for human-computer interaction, and have broad application prospects in games, virtual vision, security monitoring, lie detection, etc.

总之，本发明具有准确度高、抗干扰能力强、算法效率高等优点，具有较为广阔的应用前景。In a word, the present invention has the advantages of high accuracy, strong anti-interference ability, high algorithm efficiency, etc., and has broad application prospects.

所述S100步骤中的视频数据进行预处理的方法包括帧内空间平均的步骤及高通滤波、标准化的步骤。The method for preprocessing the video data in step S100 includes the steps of intra-frame spatial averaging, high-pass filtering, and standardization.

用于进行帧内空间平均步骤的人脸区域是眼睛及眼周的小块面部区域，帧内空间平均的步骤为：The face area used for the intra-frame spatial averaging step is the eyes and small facial areas around the eyes, and the intra-frame spatial averaging steps are:

设选取的人脸区域图像为x_i,j(t)；1≤i≤N,1≤j≤M}，{x_R,x_G,x_B}为图像的R、G、B三基色分量，对人脸视频序列先进行逐帧空间平均得到三通道信号，即Let the selected face area image be x_{i, j} (t); 1≤i≤N, 1≤j≤M}, {x_R , x_G , x_B } are the R, G, B three primary color components of the image , the face video sequence is firstly averaged frame by frame to obtain a three-channel signal, namely

$\begin{array}{c}\mathrm{<span><span>x</span>x</span>}<span><span>=</span>=</span>\left\{\begin{array}{cc}\frac{\mathrm{<span><span>1</span>1</span>}}{\mathrm{<span><span>M</span>m</span>}\mathrm{<span><span>N</span>N</span>}}\underset{\mathrm{<span><span>x</span>x</span>}<span><span>\&Element;</span>\&Element;</span>\mathrm{<span><span>R</span>R</span>}}{<span><span>\&Sigma;</span>\&Sigma;</span>}{\mathrm{<span><span>x</span>x</span>}}_{\mathrm{<span><span>i</span>i</span>}<span><span>,</span>,</span>\mathrm{<span><span>j</span>j</span>}}<span><span>(</span>(</span>\mathrm{<span><span>t</span>t</span>}<span><span>)</span>)</span>& {}_{\mathrm{<span><span>i</span>i</span>}<span><span>=</span>=</span>\mathrm{<span><span>1</span>1</span>}<span><span>,</span>,</span><span><span>...</span>...</span><span><span>,</span>,</span>\mathrm{<span><span>N</span>N</span>}<span><span>;</span>;</span>\mathrm{<span><span>j</span>j</span>}<span><span>=</span>=</span>\mathrm{<span><span>1</span>1</span>}<span><span>,</span>,</span><span><span>...</span>...</span><span><span>,</span>,</span>\mathrm{<span><span>M</span>m</span>}}\\ \frac{\mathrm{<span><span>1</span>1</span>}}{\mathrm{<span><span>M</span>m</span>}\mathrm{<span><span>N</span>N</span>}}\underset{\mathrm{<span><span>x</span>x</span>}<span><span>\&Element;</span>\&Element;</span>\mathrm{<span><span>G</span>G</span>}}{<span><span>\&Sigma;</span>\&Sigma;</span>}{\mathrm{<span><span>x</span>x</span>}}_{\mathrm{<span><span>i</span>i</span>}<span><span>,</span>,</span>\mathrm{<span><span>j</span>j</span>}}<span><span>(</span>(</span>\mathrm{<span><span>t</span>t</span>}<span><span>)</span>)</span>& {}_{\mathrm{<span><span>i</span>i</span>}<span><span>=</span>=</span>\mathrm{<span><span>1</span>1</span>}<span><span>,</span>,</span><span><span>...</span>...</span><span><span>,</span>,</span>\mathrm{<span><span>N</span>N</span>}<span><span>;</span>;</span>\mathrm{<span><span>j</span>j</span>}<span><span>=</span>=</span>\mathrm{<span><span>1</span>1</span>}<span><span>,</span>,</span><span><span>...</span>...</span><span><span>,</span>,</span>\mathrm{<span><span>M</span>m</span>}}\\ \frac{\mathrm{<span><span>1</span>1</span>}}{\mathrm{<span><span>M</span>m</span>}\mathrm{<span><span>N</span>N</span>}}\underset{\mathrm{<span><span>x</span>x</span>}<span><span>\&Element;</span>\&Element;</span>\mathrm{<span><span>B</span>B</span>}}{<span><span>\&Sigma;</span>\&Sigma;</span>}{\mathrm{<span><span>x</span>x</span>}}_{\mathrm{<span><span>i</span>i</span>}<span><span>,</span>,</span>\mathrm{<span><span>j</span>j</span>}}<span><span>(</span>(</span>\mathrm{<span><span>t</span>t</span>}<span><span>)</span>)</span>& {}_{\mathrm{<span><span>i</span>i</span>}<span><span>=</span>=</span>\mathrm{<span><span>1</span>1</span>}<span><span>,</span>,</span><span><span>...</span>...</span><span><span>,</span>,</span>\mathrm{<span><span>N</span>N</span>}<span><span>;</span>;</span>\mathrm{<span><span>j</span>j</span>}<span><span>=</span>=</span>\mathrm{<span><span>1</span>1</span>}<span><span>,</span>,</span><span><span>...</span>...</span><span><span>,</span>,</span>\mathrm{<span><span>M</span>m</span>}}\end{array}\right\}<span><span>=</span>=</span>\left[\begin{array}{c}{\mathrm{<span><span>x</span>x</span>}}_{\mathrm{<span><span>R</span>R</span>}}<span><span>(</span>(</span>\mathrm{<span><span>t</span>t</span>}<span><span>)</span>)</span>\\ {\mathrm{<span><span>x</span>x</span>}}_{\mathrm{<span><span>G</span>G</span>}}<span><span>(</span>(</span>\mathrm{<span><span>t</span>t</span>}<span><span>)</span>)</span>\\ {\mathrm{<span><span>x</span>x</span>}}_{\mathrm{<span><span>b</span>b</span>}}<span><span>(</span>(</span>\mathrm{<span><span>t</span>t</span>}<span><span>)</span>)</span>\end{array}\right]\\ <span><span>=</span>=</span>\left[\begin{array}{cccc}{\mathrm{<span><span>x</span>x</span>}}_{\mathrm{<span><span>R</span>R</span>}}<span><span>(</span>(</span>\mathrm{<span><span>1</span>1</span>}<span><span>)</span>)</span>& {\mathrm{<span><span>x</span>x</span>}}_{\mathrm{<span><span>R</span>R</span>}}<span><span>(</span>(</span>\mathrm{<span><span>2</span>2</span>}<span><span>)</span>)</span>& \mathrm{<span><span>L</span>L</span>}& {\mathrm{<span><span>x</span>x</span>}}_{\mathrm{<span><span>R</span>R</span>}}<span><span>(</span>(</span>\mathrm{<span><span>T</span>T</span>}<span><span>)</span>)</span>\\ {\mathrm{<span><span>x</span>x</span>}}_{\mathrm{<span><span>G</span>G</span>}}<span><span>(</span>(</span>\mathrm{<span><span>1</span>1</span>}<span><span>)</span>)</span>& {\mathrm{<span><span>x</span>x</span>}}_{\mathrm{<span><span>G</span>G</span>}}<span><span>(</span>(</span>\mathrm{<span><span>2</span>2</span>}<span><span>)</span>)</span>& \mathrm{<span><span>L</span>L</span>}& {\mathrm{<span><span>x</span>x</span>}}_{\mathrm{<span><span>G</span>G</span>}}<span><span>(</span>(</span>\mathrm{<span><span>T</span>T</span>}<span><span>)</span>)</span>\\ {\mathrm{<span><span>x</span>x</span>}}_{\mathrm{<span><span>B</span>B</span>}}<span><span>(</span>(</span>\mathrm{<span><span>1</span>1</span>}<span><span>)</span>)</span>& {\mathrm{<span><span>x</span>x</span>}}_{\mathrm{<span><span>B</span>B</span>}}<span><span>(</span>(</span>\mathrm{<span><span>2</span>2</span>}<span><span>)</span>)</span>& \mathrm{<span><span>L</span>L</span>}& {\mathrm{<span><span>x</span>x</span>}}_{\mathrm{<span><span>B</span>B</span>}}<span><span>(</span>(</span>\mathrm{<span><span>T</span>T</span>}<span><span>)</span>)</span>\end{array}\right]\end{array}$

式(1)中N，M为选定面部图像区域的高和宽，t为每帧图像对应的时间；In the formula (1), N, M are the height and width of the selected facial image area, and t is the corresponding time of each frame image;

对每帧图像按照视频图像宽度中线切分为两部分，再分别按照上述方法对R，G，B三分量进行进总体平均，得到六通道数据：Divide each frame of image into two parts according to the midline of the width of the video image, and then carry out the overall average of the three components of R, G, and B according to the above method to obtain six-channel data:

$\left[\begin{array}{c}{\mathrm{<span><span>x</span>x</span>}}_{\mathrm{<span><span>R</span>R</span>}<span><span>,</span>,</span>\mathrm{<span><span>1</span>1</span>}}<span><span>(</span>(</span>\mathrm{<span><span>t</span>t</span>}<span><span>)</span>)</span>\\ {\mathrm{<span><span>x</span>x</span>}}_{\mathrm{<span><span>R</span>R</span>}<span><span>,</span>,</span>\mathrm{<span><span>2</span>2</span>}}<span><span>(</span>(</span>\mathrm{<span><span>t</span>t</span>}<span><span>)</span>)</span>\\ {\mathrm{<span><span>x</span>x</span>}}_{\mathrm{<span><span>G</span>G</span>}<span><span>,</span>,</span>\mathrm{<span><span>1</span>1</span>}}<span><span>(</span>(</span>\mathrm{<span><span>t</span>t</span>}<span><span>)</span>)</span>\\ {\mathrm{<span><span>x</span>x</span>}}_{\mathrm{<span><span>G</span>G</span>}<span><span>,</span>,</span>\mathrm{<span><span>2</span>2</span>}}<span><span>(</span>(</span>\mathrm{<span><span>t</span>t</span>}<span><span>)</span>)</span>\\ {\mathrm{<span><span>x</span>x</span>}}_{\mathrm{<span><span>B</span>B</span>}<span><span>,</span>,</span>\mathrm{<span><span>1</span>1</span>}}<span><span>(</span>(</span>\mathrm{<span><span>t</span>t</span>}<span><span>)</span>)</span>\\ {\mathrm{<span><span>x</span>x</span>}}_{\mathrm{<span><span>B</span>B</span>}<span><span>,</span>,</span>\mathrm{<span><span>2</span>2</span>}}<span><span>(</span>(</span>\mathrm{<span><span>t</span>t</span>}<span><span>)</span>)</span>\end{array}\right]<span><span>=</span>=</span>\left[\begin{array}{cccc}{\mathrm{<span><span>x</span>x</span>}}_{\mathrm{<span><span>1</span>1</span>}}<span><span>(</span>(</span>\mathrm{<span><span>1</span>1</span>}<span><span>)</span>)</span>& {\mathrm{<span><span>x</span>x</span>}}_{\mathrm{<span><span>1</span>1</span>}}<span><span>(</span>(</span>\mathrm{<span><span>2</span>2</span>}<span><span>)</span>)</span>& \mathrm{<span><span>L</span>L</span>}& {\mathrm{<span><span>x</span>x</span>}}_{\mathrm{<span><span>1</span>1</span>}}<span><span>(</span>(</span>\mathrm{<span><span>T</span>T</span>}<span><span>)</span>)</span>\\ {\mathrm{<span><span>x</span>x</span>}}_{\mathrm{<span><span>2</span>2</span>}}<span><span>(</span>(</span>\mathrm{<span><span>1</span>1</span>}<span><span>)</span>)</span>& {\mathrm{<span><span>x</span>x</span>}}_{\mathrm{<span><span>2</span>2</span>}}<span><span>(</span>(</span>\mathrm{<span><span>2</span>2</span>}<span><span>)</span>)</span>& \mathrm{<span><span>L</span>L</span>}& {\mathrm{<span><span>x</span>x</span>}}_{\mathrm{<span><span>2</span>2</span>}}<span><span>(</span>(</span>\mathrm{<span><span>T</span>T</span>}<span><span>)</span>)</span>\\ \mathrm{<span><span>M</span>m</span>}& \mathrm{<span><span>M</span>m</span>}& \mathrm{<span><span>M</span>m</span>}& \mathrm{<span><span>M</span>m</span>}\\ {\mathrm{<span><span>x</span>x</span>}}_{\mathrm{<span><span>6</span>6</span>}}<span><span>(</span>(</span>\mathrm{<span><span>1</span>1</span>}<span><span>)</span>)</span>& {\mathrm{<span><span>x</span>x</span>}}_{\mathrm{<span><span>6</span>6</span>}}<span><span>(</span>(</span>\mathrm{<span><span>2</span>2</span>}<span><span>)</span>)</span>& \mathrm{<span><span>L</span>L</span>}& {\mathrm{<span><span>x</span>x</span>}}_{\mathrm{<span><span>6</span>6</span>}}<span><span>(</span>(</span>\mathrm{<span><span>T</span>T</span>}<span><span>)</span>)</span>\end{array}\right]<span><span>.</span>.</span>$

用于进行高通滤波步骤的高通滤波器的截止频率为0.8Hz。The cut-off frequency of the high-pass filter used for the high-pass filtering step was 0.8 Hz.

用于进行标准化步骤的计算过程为：The calculation procedure used to perform the normalization step is:

$\mathrm{<span><span>x</span>x</span>}<span><span>\&LeftArrow;</span>\&LeftArrow;</span>\frac{\mathrm{<span><span>x</span>x</span>}<span><span>-</span>-</span>\mathrm{<span><span>E</span>E.</span>}<span><span>\&lsqb;</span>\&lsqb;</span>\mathrm{<span><span>x</span>x</span>}<span><span>\&rsqb;</span>\&rsqb;</span>}{\mathrm{<span><span>\&sigma;</span>\&sigma;</span>}}$

式中x为原始信号，E[x]为求均值运算，σ为x中各分量的标准差。In the formula, x is the original signal, E[x] is the mean value operation, and σ is the standard deviation of each component in x.

所述步骤S100中，将x＝[x_R,1,x_R,2,x_G,1,x_G,2,x_B,1,x_B,2]^T重新定义为x＝[x₁,x₂,x₃,x₄,x₅,x₆]^T，则可将混合信号生成方式表达为：In the step S100, x=[x_R,1 ,x_R,2 ,x_G,1 ,x_G,2 ,x_B,1 ,x_B,2 ]^T is redefined as x=[x₁ , x₂ ,x₃ ,x₄ ,x₅ ,x₆ ]^T , then the mixed signal generation method can be expressed as:

$\mathrm{<span><span>X</span>x</span>}<span><span>=</span>=</span>\left[\begin{array}{c}{\mathrm{<span><span>x</span>x</span>}}_{\mathrm{<span><span>1</span>1</span>}}\\ {\mathrm{<span><span>x</span>x</span>}}_{\mathrm{<span><span>2</span>2</span>}}\\ \mathrm{<span><span>M</span>m</span>}\\ {\mathrm{<span><span>x</span>x</span>}}_{\mathrm{<span><span>6</span>6</span>}}\end{array}\right]<span><span>=</span>=</span>\left[\begin{array}{cccc}{\mathrm{<span><span>a</span>a</span>}}_{\mathrm{<span><span>11</span>11</span>}}& {\mathrm{<span><span>a</span>a</span>}}_{\mathrm{<span><span>12</span>12</span>}}& \mathrm{<span><span>L</span>L</span>}& {\mathrm{<span><span>a</span>a</span>}}_{\mathrm{<span><span>16</span>16</span>}}\\ {\mathrm{<span><span>a</span>a</span>}}_{\mathrm{<span><span>21</span>twenty\; one</span>}}& {\mathrm{<span><span>a</span>a</span>}}_{\mathrm{<span><span>22</span>twenty\; two</span>}}& \mathrm{<span><span>L</span>L</span>}& {\mathrm{<span><span>a</span>a</span>}}_{\mathrm{<span><span>26</span>26</span>}}\\ \mathrm{<span><span>M</span>m</span>}& \mathrm{<span><span>M</span>m</span>}& \mathrm{<span><span>M</span>m</span>}& \mathrm{<span><span>M</span>m</span>}\\ {\mathrm{<span><span>a</span>a</span>}}_{\mathrm{<span><span>61</span>61</span>}}& {\mathrm{<span><span>a</span>a</span>}}_{\mathrm{<span><span>62</span>62</span>}}& \mathrm{<span><span>L</span>L</span>}& {\mathrm{<span><span>a</span>a</span>}}_{\mathrm{<span><span>66</span>66</span>}}\end{array}\right]\left[\begin{array}{c}{\mathrm{<span><span>s</span>the\; s</span>}}_{\mathrm{<span><span>1</span>1</span>}}\\ {\mathrm{<span><span>s</span>the\; s</span>}}_{\mathrm{<span><span>2</span>2</span>}}\\ \mathrm{<span><span>M</span>m</span>}\\ {\mathrm{<span><span>s</span>the\; s</span>}}_{\mathrm{<span><span>6</span>6</span>}}\end{array}\right]<span><span>=</span>=</span>\mathrm{<span><span>A</span>A</span>}\mathrm{<span><span>s</span>the\; s</span>}$

式中A为标量混合矩阵，s＝[s₁,s₂,s₃,s₄,s₅,s₆]^T代表隐含的多个源信号，其中包含了眨眼信号分量和心跳BVP分量以及其他成分，式中的混合矩阵A和源s均未知，通过使用基于二阶盲辨识的盲分离算法可以得到对真实源s的估计，即满足如下关系的y：In the formula, A is a scalar mixing matrix, s=[s₁ , s₂ , s₃ , s₄ , s₅ , s₆ ]^T represents multiple hidden source signals, which include the blink signal component and the heartbeat BVP component and Other components, the mixing matrix A and the source s in the formula are unknown, and the estimation of the real source s can be obtained by using the blind separation algorithm based on the second-order blind identification, that is, y that satisfies the following relationship:

$\mathrm{<span><span>y</span>the\; y</span>}<span><span>=</span>=</span>\left[\begin{array}{c}{\mathrm{<span><span>y</span>the\; y</span>}}_{\mathrm{<span><span>1</span>1</span>}}\\ {\mathrm{<span><span>y</span>the\; y</span>}}_{\mathrm{<span><span>2</span>2</span>}}\\ \mathrm{<span><span>M</span>m</span>}\\ {\mathrm{<span><span>y</span>the\; y</span>}}_{\mathrm{<span><span>6</span>6</span>}}\end{array}\right]<span><span>=</span>=</span>\left[\begin{array}{cccc}{\mathrm{<span><span>w</span>w</span>}}_{\mathrm{<span><span>11</span>11</span>}}& {\mathrm{<span><span>w</span>w</span>}}_{\mathrm{<span><span>12</span>12</span>}}& \mathrm{<span><span>L</span>L</span>}& {\mathrm{<span><span>w</span>w</span>}}_{\mathrm{<span><span>16</span>16</span>}}\\ {\mathrm{<span><span>w</span>w</span>}}_{\mathrm{<span><span>21</span>twenty\; one</span>}}& {\mathrm{<span><span>w</span>w</span>}}_{\mathrm{<span><span>22</span>twenty\; two</span>}}& \mathrm{<span><span>L</span>L</span>}& {\mathrm{<span><span>w</span>w</span>}}_{\mathrm{<span><span>26</span>26</span>}}\\ \mathrm{<span><span>M</span>m</span>}& \mathrm{<span><span>M</span>m</span>}& \mathrm{<span><span>M</span>m</span>}& \mathrm{<span><span>M</span>m</span>}\\ {\mathrm{<span><span>w</span>w</span>}}_{\mathrm{<span><span>61</span>61</span>}}& {\mathrm{<span><span>w</span>w</span>}}_{\mathrm{<span><span>62</span>62</span>}}& \mathrm{<span><span>L</span>L</span>}& {\mathrm{<span><span>w</span>w</span>}}_{\mathrm{<span><span>66</span>66</span>}}\end{array}\right]\left[\begin{array}{c}{\mathrm{<span><span>x</span>x</span>}}_{\mathrm{<span><span>1</span>1</span>}}\\ {\mathrm{<span><span>x</span>x</span>}}_{\mathrm{<span><span>2</span>2</span>}}\\ \mathrm{<span><span>M</span>m</span>}\\ {\mathrm{<span><span>x</span>x</span>}}_{\mathrm{<span><span>6</span>6</span>}}\end{array}\right]<span><span>=</span>=</span>\mathrm{<span><span>W</span>W</span>}\mathrm{<span><span>x</span>x</span>}$

式中分离矩阵W为对A^-1的近似估计。In the formula, the separation matrix W is an approximate estimate of A^-1 .

所述眨眼及BVP信号筛选模块使用一种基于谱峭度的信号自动识别的方法从盲分离算法的多通道输出信号中自动筛选出眨眼和BVP信号。峭度常用于信号高斯性的度量。通过对各通道输出信号计算自相关函数再求傅立叶变换得到各路信号的功率谱，BVP信号具有明显的周期性因而其功率谱谱峰明显，眨眼信号具有一定的准周期性也具有潜在的谱峰，进行峭度计算后其他各路信号与上述两者差异明显，通过简单的门限即可筛选出不同的信号。The eye blink and BVP signal screening module automatically screens eye blink and BVP signals from the multi-channel output signals of the blind separation algorithm by using a method of automatic signal recognition based on spectral kurtosis. Kurtosis is often used as a measure of the Gaussianity of a signal. By calculating the autocorrelation function of the output signals of each channel and then calculating the Fourier transform to obtain the power spectrum of each signal, the BVP signal has obvious periodicity, so its power spectrum peak is obvious, and the blink signal has a certain quasi-periodicity and potential spectrum. Peak, after performing kurtosis calculation, other signals are significantly different from the above two, and different signals can be screened out through a simple threshold.

下面针对基于二阶盲辨识的非接触式眨眼与心率联合检测系统进行介绍：The following is an introduction to the non-contact blinking and heart rate joint detection system based on second-order blind recognition:

基于二阶盲辨识的非接触式眨眼与心率联合检测系统，该系统包括视频信号采集与预处理模块10、目标跟踪模块20、多通道盲分离模块30、眨眼及BVP信号筛选模块40、眨眼频率计算及时长计算模块50与心率估计模块60；A non-contact blink and heart rate joint detection system based on second-order blind identification, the system includes a video signal acquisition and preprocessing module 10, a target tracking module 20, a multi-channel blind separation module 30, a blink and BVP signal screening module 40, and blink frequency Calculation and duration calculation module 50 and heart rate estimation module 60;

所述视频信号采集与预处理模块10，手机摄像头等设备用于获取人脸部图像并使用相干平均法将采集到的包含眼部区域的人脸视频数据按照R,G,B三基色通道进行空间平均，并采用图像宽度中值切分的方法得到六通道原始信号x＝[x_R1,x_G1,x_B1,x_R2,x_G2,x_B2]^T，同时对原始信号x进行高通滤波以去除信号中各分量的低频趋势并对信号进行去均值与方差归一化；The video signal acquisition and preprocessing module 10, mobile phone cameras and other equipment are used to acquire human face images and use the coherent averaging method to process the collected human face video data including the eye area according to R, G, and B three primary color channels. Space average, and use the method of image width median segmentation to obtain six-channel original signal x=[x_R1 ,x_G1 ,x_B1 ,x_R2 ,x_G2 ,x_B2 ]^T , and perform high-pass filtering on the original signal x to Remove the low-frequency trend of each component in the signal and perform de-mean and variance normalization on the signal;

所述目标跟踪模块20使用Meanshift目标跟踪算法实现对选定面部区域的连续定位；Described target tracking module 20 uses the Meanshift target tracking algorithm to realize the continuous positioning of selected facial regions;

所述多通道盲分离模块30用于通过基于二阶盲辨识的多通道盲分离算法对原始信号x进行分离以得到对不同源信号的估计；The multi-channel blind separation module 30 is used to separate the original signal x through a multi-channel blind separation algorithm based on second-order blind identification to obtain estimates of different source signals;

所述眨眼及BVP信号筛选模块40用于从分离得到的多通道数据中辨识出眨眼和BVP信号。The blink and BVP signal screening module 40 is used to identify blink and BVP signals from the separated multi-channel data.

眨眼频率计算及时长计算模块50与心率估计模块60用于从分离得到的眨眼和BVP信号中计算得到眨眼时长、眨眼频率与心率。The blink frequency calculation and duration calculation module 50 and the heart rate estimation module 60 are used to calculate blink duration, blink frequency and heart rate from the separated blink and BVP signals.

采用上述的各模块之间的协调配合，能够从面部有限区域视频中同步提取出眨眼和心率信号，同样地，本发明具有准确度高、抗干扰能力强、算法效率高等优点，具有较为广阔的应用前景。By adopting the coordination and cooperation between the above-mentioned modules, the eye blink and heart rate signals can be extracted synchronously from the video in the limited area of the face. Similarly, the present invention has the advantages of high accuracy, strong anti-interference ability, and high algorithm efficiency, and has a relatively broad application range. Application prospect.

所述视频图像序列采集与预处理模块10包括帧内空间平均单元11、高通滤波单元12及标准化单元13。The video image sequence acquisition and preprocessing module 10 includes an intra-frame spatial averaging unit 11 , a high-pass filtering unit 12 and a normalization unit 13 .

上述的高通滤波单元12中的高通滤波器的截止频率为0.8Hz。The cut-off frequency of the high-pass filter in the above-mentioned high-pass filter unit 12 is 0.8 Hz.

所述目标跟踪模块20使用Meanshift目标跟踪算法进行连续帧间的眼部区域定位，所述眨眼频率计算及时长计算模块50与心率估计模块60使用计数器计算单位时间内的眨眼次数得到眨眼频率，使用幅度门限得到眨眼脉冲的时长，使用BVP信号的功率谱峰值×60得到心率。The target tracking module 20 uses the Meanshift target tracking algorithm to perform eye region positioning between consecutive frames, the blink frequency calculation and duration calculation module 50 and the heart rate estimation module 60 use a counter to calculate the number of blinks per unit time to obtain the blink frequency, using The amplitude threshold was used to obtain the duration of the eyeblink pulse, and the peak value of the power spectrum of the BVP signal × 60 was used to obtain the heart rate.

下面针对附图，对本发明的方法及系统进行简要的说明：The method and system of the present invention are briefly described below with reference to the accompanying drawings:

参见图3，说明了本实施例中采用系统中的视频信号采集与预处理模块10、目标跟踪模块20、盲源分离模块30、眨眼与BVP信号筛选模块40、眨眼时长/频率估计模块50与心率估计模块60。采用二阶盲辨识算法进行盲源分离后，利用基于谱峭度的眨眼与BVP信号自动筛选方法分别得到眨眼与BVP信号，进而求得眨眼时长、频率与心率估计值。方法主要包括以下几个步骤：1)采集人脸眼部视频数据并进行预处理；2)进行目标区域跟踪在连续视频帧中确定目标区域位置；3)将预处理后的六通道数据用二阶盲辨识算法进行盲源分离；4)使用基于谱峭度的眨眼/BVP分量自动识别方法从算法的六通道输出信号中自动筛选出眨眼与BVP信号；5)对眨眼信号进行眨眼时长计算与频率统计；6)对BVP信号进行功率谱分析，最后将最大谱峰频率×60，得到心率估算值。Referring to Fig. 3, it has been illustrated that video signal acquisition and preprocessing module 10, target tracking module 20, blind source separation module 30, blink and BVP signal screening module 40, blink duration/frequency estimation module 50 and Heart rate estimation module 60. After using the second-order blind identification algorithm for blind source separation, the blink and BVP signals were obtained by using the automatic screening method based on spectral kurtosis, and then the blink duration, frequency and heart rate estimates were obtained. The method mainly includes the following steps: 1) collecting face and eye video data and performing preprocessing; 2) performing target area tracking to determine the position of the target area in continuous video frames; 3) using the preprocessed six-channel data with two Blind source separation using the order blind identification algorithm; 4) Use the blink/BVP component automatic identification method based on spectral kurtosis to automatically screen out the blink and BVP signals from the six-channel output signals of the algorithm; 5) Calculate the blink duration of the blink signal and Frequency statistics; 6) Perform power spectrum analysis on the BVP signal, and finally calculate the maximum spectrum peak frequency by 60 to obtain an estimated heart rate value.

参见图4，说明了本实施例中选取的人脸眼部区域示意图，该区域在算法运行时手动从视频第一帧中选取。Referring to FIG. 4 , it illustrates a schematic diagram of the face and eye region selected in this embodiment, which is manually selected from the first frame of the video when the algorithm is running.

参见图5，说明了本实施例中经过预处理模块10后得到的观测信号波形图，此六通道信号即为盲分离算法的处理对象。Referring to FIG. 5 , it illustrates the waveform diagram of the observed signal obtained after the preprocessing module 10 in this embodiment. The six-channel signal is the processing object of the blind separation algorithm.

参见图6，说明了本实施例中使用二阶盲辨识算法对输入信号进行盲源分离后的输出信号波形图，从图中可以看到，六通道输出信号的第一个分量就是眨眼信号，第二个分量是BVP信号，说明了二阶盲辨识算法对眨眼与BVP信号有良好的分离效果。Referring to Fig. 6, it illustrates the waveform diagram of the output signal after blind source separation of the input signal using the second-order blind identification algorithm in this embodiment. It can be seen from the figure that the first component of the six-channel output signal is the blink signal. The second component is the BVP signal, which shows that the second-order blind recognition algorithm has a good separation effect on blinking and BVP signals.

参见图7，说明了本实施例中基于谱峭度的眨眼与BVP分量自动识别得到的眨眼与BVP信号波形图；图中两通道上一通道是自动识别的眨眼信号，下一通道的是自动识别的BVP信号。Referring to Fig. 7, have illustrated the wink and the BVP signal wave diagram that obtain based on the wink of spectral kurtosis and BVP component automatic recognition in the present embodiment; Among the two channels, one channel is the blink signal of automatic identification, and the next channel is the automatic Identify the BVP signal.

参见图8，说明了对得到的眨眼信号进行眨眼频率与时长计算的示意图。使用计数器计算单位时间内的眨眼次数得到眨眼频率，使用幅度门限得到眨眼脉冲的时长。Referring to FIG. 8 , it illustrates a schematic diagram of calculating the blink frequency and duration of the obtained blink signal. Use the counter to calculate the number of blinks per unit time to obtain the blink frequency, and use the amplitude threshold to obtain the duration of the blink pulse.

参见图9，说明了由得到的BVP信号进行功率谱分析的结果，其最大谱峰频率×60，就得到心率估算值。Referring to FIG. 9 , it illustrates the result of power spectrum analysis of the obtained BVP signal. The maximum spectrum peak frequency × 60 is used to obtain an estimated heart rate value.

Claims (10)

1. contactless nictation and the heart rate associated detecting method based on second-order blind identification, it is characterised in that: this joint detectionMethod comprises the steps:

S100, gather in the state of experimenter's natural relaxation and comprise the facial video data of eyes, collection original video dataSelected video data is simultaneously pre-processed by rear selected detection facial zone, obtains pretreated six channel signals, is labeled as x=[x_R1,x_R2,x_G1,x_G2,x_B1,x_B2]^T；

S101, use Meanshift algorithm obtain the target area in successive frame, and obtain six passages letters in the same wayNumber；

S102, second-order blind identification algorithm will be used to carry out blind source separating through pretreated primary signal as input signal,Source signal after separating is designated as y=[y₁,y₂,y₃,y₄,y₅,y₆]^T；

S103, carry out signal recognition and screening to the separation signal y obtaining in above-mentioned steps S101, use based on spectrum kurtosis pointSelect required nictation and BVP signal from component automatic identifying method；

S104, carry out frequency of wink and duration calculation to the signal of blinking obtaining, and power spectrum spectrum is carried out to the BVP signal obtainingAnalyze, obtain heart rate estimate.

2. the contactless nictation based on second-order blind identification according to claim 1 and heart rate associated detecting method, it is specialLevy and be: the method that the video data in described S100 step pre-processes includes step and the high pass filter of space average in frameRipple, standardized step.

3. the contactless nictation based on second-order blind identification according to claim 2 and heart rate associated detecting method, it is specialLevy and be: be eyes and fritter facial zone near the eyes, space in frame for carrying out the human face region of space average step in frameAverage step is:

If the human face region image chosen is x_i,j(t)；1≤i≤N, 1≤j≤M}, { x_R,x_G,x_BIt is R, G, B tri-base of imageColouring component, first carries out space average frame by frame and obtains triple channel signal, i.e. to face video sequence

$\begin{array}{c}x=\left\{\begin{array}{cc}\frac{1}{MN}\underset{x\&Element;R}{\mathrm{\&Sigma;}}{x}_{i,j}\left(t\right)& {}_{i=1,...,N;j=1,...,M}\\ \frac{1}{MN}\underset{x\&Element;G}{\mathrm{\&Sigma;}}{x}_{i,j}\left(t\right)& {}_{i=1,...,N;j=1,...,M}\\ \frac{1}{MN}\underset{x\&Element;B}{\mathrm{\&Sigma;}}{x}_{i,j}\left(t\right)& {}_{i=1,...,N;j=1,...,M}\end{array}\right\}=\left[\begin{array}{c}{x}_R\left(t\right)\\ x_G\left(t\right)\\ x_b\left(t\right)\end{array}\right]\\ =\left[\begin{array}{cccc}{x}_R\left(1\right)& x_R\left(2\right)& L& x_R\left(T\right)\\ x_G\left(1\right)& x_G\left(2\right)& L& x_G\left(T\right)\\ x_B\left(1\right)& x_B\left(2\right)& L& x_B\left(T\right)\end{array}\right]\end{array}$

N in formula, M are the height and width in selected face-image region, and t is every two field picture corresponding time；

It is two parts to every two field picture according to video image width center line cutting, more respectively according to the method described above to R, G, B tri-pointsMeasure into traveling population mean, obtain six channel datas:

$\left[\begin{array}{c}{x}_{R,1}\left(t\right)\\ x_{R,2}\left(t\right)\\ x_{G,1}\left(t\right)\\ x_{G,2}\left(t\right)\\ x_{B,1}\left(t\right)\\ x_{B,2}\left(t\right)\end{array}\right]=\left[\begin{array}{cccc}{x}_1\left(1\right)& x_1\left(2\right)& L& x_1\left(T\right)\\ x_2\left(1\right)& x_2\left(2\right)& L& x_2\left(T\right)\\ M& M& M& M\\ x_6\left(1\right)& x_6\left(2\right)& L& x_6\left(T\right)\end{array}\right].$

4. the contactless nictation based on second-order blind identification according to claim 2 and heart rate associated detecting method, it is specialLevy and be: the cut-off frequency of the high-pass filter for carrying out high-pass filtering step is 0.8Hz.

5. the contactless nictation based on second-order blind identification according to claim 2 and heart rate associated detecting method, it is specialLevy and be: the calculating process for being standardized step is:

$x\&LeftArrow;\frac{x-E\&lsqb;x\&rsqb;}{\mathrm{\&sigma;}}$

In formula, x is primary signal, and E [x] is computing of averaging, and σ is the standard deviation of each component in x.

6. the contactless nictation based on second-order blind identification according to claim 2 and heart rate associated detecting method, it is specialLevy and be: in described step S100, by x=[x_R,1,x_R,2,x_G,1,x_G,2,x_B,1,x_B,2]^TIt is newly defined as x=[x₁,x₂,x₃,x₄,x₅,x₆]^T, then mixed signal generating mode can be expressed as:

$x=\left[\begin{array}{c}{x}_1\\ x_2\\ M\\ x_6\end{array}\right]=\left[\begin{array}{cccc}{a}_{11}& a_{12}& L& a_{16}\\ a_{21}& a_{22}& L& a_{26}\\ M& M& M& M\\ a_{61}& a_{62}& L& a_{66}\end{array}\right]\left[\begin{array}{c}{s}_1\\ s_2\\ M\\ s_6\end{array}\right]=As$

In formula, A is scalar mixing matrix, s=[s₁,s₂,s₃,s₄,s₅,s₆]^TRepresent implicit multiple source signals, wherein contain and blinkEye component of signal and heartbeat BVP component and other compositions, the hybrid matrix A in formula and source s is all unknown, by using based on twoThe blind separation algorithm of rank blind discrimination can obtain the estimation to real source s, i.e. meets the y of following relation:

$y=\left[\begin{array}{c}{y}_1\\ y_2\\ M\\ y_6\end{array}\right]=\left[\begin{array}{cccc}{w}_{11}& w_{12}& L& w_{16}\\ w_{21}& w_{22}& L& w_{26}\\ M& M& M& M\\ w_{61}& w_{62}& L& w_{66}\end{array}\right]\left[\begin{array}{c}{x}_1\\ x_2\\ M\\ x_6\end{array}\right]=Wx$

In formula, separation matrix W is for A^-1Approximate evaluation.

7. contactless nictation and the heart rate combined detection system based on second-order blind identification, it is characterised in that: this system includes regardingFrequently signals collecting and pretreatment module (10), target tracking module (20), multichannel blind separation module (30), nictation and BVP letterNumber screening module (40), frequency of wink calculate and duration computing module (50) and heart rate estimation module (60)；

Described video signal collective and pretreatment module (10), be used for obtaining face image and use coherent averaging technique to gatherTo the face video data comprising ocular carry out space average according to R, G, B three primary colours passage, and use picture traverseThe method of intermediate value cutting obtains six channel original signal x=[x_R1,x_G1,x_B1,x_R2,x_G2,x_B2]^T, primary signal x is carried out simultaneouslyHigh-pass filtering is to remove the low frequency trend of each component in signal and to carry out average and normalized square mean to signal；

Described target tracking module (20) uses Meanshift target tracking algorism to realize determining continuously to selected facial zonePosition；

Described multichannel blind separation module (30) is used for by the multichannel blind separation algorithm based on second-order blind identification to original letterNumber x carries out separating to obtain the estimation to not homologous signal；

Described nictation and BVP signal screening module (40) are for picking out nictation and BVP from the multi-channel data of isolatedSignal.

Frequency of wink calculates and duration computing module (50) is used for the nictation from isolated and BVP with heart rate estimation module (60)Signal is calculated duration nictation, frequency of wink and heart rate.

8. the contactless nictation based on second-order blind identification according to claim 7 and heart rate combined detection system, it is specialLevy and be: described video signal collective and pretreatment module (10) include space average unit (11), high pass filter unit in frameAnd Standardisation Cell (13) (12).

9. the contactless nictation based on second-order blind identification according to claim 8 and heart rate combined detection system, it is specialLevy and be: the cut-off frequency of the high-pass filter in high pass filter unit (12) is 0.8Hz.

10. the contactless nictation based on second-order blind identification according to claim 8 and heart rate combined detection system, it is specialLevy and be: the ocular that described target tracking module (20) uses Meanshift target tracking algorism to carry out continuous interframe is determinedPosition, described frequency of wink calculates and duration computing module (50) uses counter to calculate the unit interval with heart rate estimation module (60)Interior number of winks obtains frequency of wink, uses amplitude threshold to obtain the duration of pulse nictation, uses the power spectral peak of BVP signalValue × 60 obtain heart rate.