CN111539247A - A kind of hyperspectral face recognition method, device, electronic device and storage medium thereof - Google Patents

Abstract

Translated fromChinese

本发明公开了一种超光谱人脸识别方法、装置、电子设备及其存储介质，方法包括获取可见光人脸图像集和红外人脸图像集；对可见光人脸图像集和红外人脸图像集分别进行预处理；构建超光谱图像融合网络模型，对超光谱图像融合网络模型进行训练；通过超光谱图像融合网络模型得到超光谱人脸图像集；构建卷积神经网络人脸识别模型，对卷积神经网络人脸识别模型进行训练；通过卷积神经网络人脸识别模型得到人脸特征集，并使用支持向量机分类器对人脸特征集进行分类处理以实现超光谱人脸识别。本发明提出了一套完整的超光谱人脸识别技术，可以解决传统人脸识别技术的适用范围窄、识别性能不高、且特征提取鲁棒性差等缺陷。

The invention discloses a hyperspectral face recognition method, device, electronic equipment and storage medium thereof. The method includes acquiring a visible light face image set and an infrared face image set; Perform preprocessing; build a hyperspectral image fusion network model, train the hyperspectral image fusion network model; obtain a hyperspectral face image set through the hyperspectral image fusion network model; build a convolutional neural network face recognition model, The neural network face recognition model is trained; the face feature set is obtained through the convolutional neural network face recognition model, and the support vector machine classifier is used to classify the face feature set to realize hyperspectral face recognition. The invention proposes a complete set of hyperspectral face recognition technology, which can solve the defects of traditional face recognition technology such as narrow application range, low recognition performance, and poor robustness of feature extraction.

Description

Translated fromChinese

一种超光谱人脸识别方法、装置、电子设备及其存储介质A kind of hyperspectral face recognition method, device, electronic device and storage medium thereof

技术领域technical field

本发明属于模式识别、数字图像处理技术领域，具体涉及一种超光谱人脸识别方法、装置、电子设备及其存储介质。The invention belongs to the technical field of pattern recognition and digital image processing, and in particular relates to a hyperspectral face recognition method, device, electronic device and storage medium thereof.

背景技术Background technique

人脸识别是使用计算机从图像或视频中自动识别或验证对象身份的技术。由于其采集廉价性和非侵入性的优势，人脸识别已经成为最受关注的生物特征识别模态之一。随着深度学习等理论和技术的应用，人脸识别已经得到了长足的发展。Facial recognition is a technology that uses a computer to automatically identify or verify the identity of an object from an image or video. Due to its advantages of cheap acquisition and non-invasiveness, face recognition has become one of the most concerned biometric identification modalities. With the application of deep learning and other theories and technologies, face recognition has made great progress.

目前人脸识别技术大多采用可见光成像和采集手段，该技术大多局限于白天光线充足的良好背景条件下，在光线不足和恶劣气候等环境下通常表现不佳。随着现实世界中各种复杂环境下应用的出现，基于可见光的人脸识别技术越来越难以满足要求。而红外线成像技术具备背景光线要求低、雨天雾天等气候下成像等等优势，弥补了可见光成像环境的不足。人脸识别技术单纯使用可见光波段将导致在人脸识别特征提取时有用面部信息的利用受到限制，限制了最终人脸识别性能的提升，根据成像原理可知，可见光图像能够以与人类视觉系统相一致的方式提供具有高空间分辨率和清晰度的纹理细节；相比之下，红外线图像可以根据辐射差将目标与背景区分开来，提供与可将光不同类型的热辐射或皮肤反射属性等信息。因此，可以利用图像融合技术将红外图像中的热辐射信息或皮肤反射属性与可见光图像中的详细纹理信息的结合起来，得到能够满足更高人脸识别性能需求的超光谱人脸图像。目前为止，针对人脸图像融合的方法大多为传统方式，如离散小波变化方法(DWT)、主成分分析方法(PCA)、非下采样轮廓变换方法(NSCT)和交叉双线性滤波方法(CBF)等；随着深度学习的越来越成熟，出现了少数基于卷积神经网络的图像融合方法，如DenseFuse方法。At present, most face recognition technologies use visible light imaging and acquisition methods, which are mostly limited to good background conditions with sufficient light during the day, and usually perform poorly in environments such as insufficient light and harsh climates. With the emergence of applications in various complex environments in the real world, face recognition technology based on visible light is more and more difficult to meet the requirements. Infrared imaging technology has the advantages of low background light requirements, imaging in rainy and foggy weather and other climates, etc., which makes up for the shortcomings of visible light imaging environment. The simple use of visible light band in face recognition technology will limit the use of useful facial information in face recognition feature extraction, and limit the improvement of the final face recognition performance. According to the imaging principle, visible light images can be consistent with the human visual system. provides texture detail with high spatial resolution and clarity; in contrast, infrared images can distinguish objects from the background based on radiance differences, providing information such as thermal radiation or skin reflection properties that can distinguish different types of light . Therefore, image fusion technology can be used to combine thermal radiation information or skin reflection properties in infrared images with detailed texture information in visible light images to obtain hyperspectral face images that can meet the requirements of higher face recognition performance. So far, most of the methods for face image fusion are traditional methods, such as discrete wavelet change (DWT), principal component analysis (PCA), non-subsampling contour transform (NSCT) and cross bilinear filtering (CBF). ), etc.; as deep learning becomes more and more mature, there are a few image fusion methods based on convolutional neural networks, such as the DenseFuse method.

但是，上述传统人脸图像融合算法均为人工设计方式，普遍设计繁琐复杂、针对特定情形性能好但是鲁棒性差，对于光照变化下识别效果不佳；而基于深度学习的DenseFuse方法并非针对超光谱人脸识别任务设计的，在人脸识别场合下表现效果仍不够理想。However, the above-mentioned traditional face image fusion algorithms are all artificial design methods, which are generally cumbersome and complex in design, have good performance in specific situations but poor robustness, and have poor recognition effect under illumination changes; and the deep learning-based DenseFuse method is not for hyperspectral. The face recognition task is designed, and the performance effect is still not ideal in the face recognition situation.

发明内容SUMMARY OF THE INVENTION

为了解决现有技术中存在的上述问题，本发明提供了一种超光谱人脸识别方法、装置、电子设备及其存储介质。In order to solve the above problems in the prior art, the present invention provides a hyperspectral face recognition method, device, electronic device and storage medium thereof.

本发明的一个实施例提供了一种超光谱人脸识别方法，该方法包括：An embodiment of the present invention provides a hyperspectral face recognition method, the method comprising:

获取可见光人脸图像集和红外人脸图像集，并将所述可见光人脸图像集分为可见光人脸训练图像集和可见光人脸测试图像集，所述红外人脸图像集分为红外人脸训练图像集和红外人脸测试图像集；Acquire a visible light face image set and an infrared face image set, and divide the visible light face image set into a visible light face training image set and a visible light face test image set, and the infrared face image set is divided into an infrared face Training image set and infrared face test image set;

对所述可见光人脸训练图像集、可见光人脸测试图像集、所述红外人脸训练图像集和红外人脸测试图像集分别进行预处理得到可见光人脸预处理训练图像集、可见光人脸预处理测试图像集、红外人脸预处理训练图像集和红外人脸预处理测试图像集；The visible light face training image set, the visible light face test image set, the infrared face training image set and the infrared face test image set are respectively preprocessed to obtain the visible light face preprocessing training image set, the visible light face preprocessing image set, and the Process the test image set, the infrared face preprocessing training image set and the infrared face preprocessing test image set;

构建超光谱图像融合网络模型，根据所述可见光人脸预处理训练图像集和所述红外人脸预处理训练图像集对所述超光谱图像融合网络模型进行训练得到训练好的超光谱图像融合网络模型；Build a hyperspectral image fusion network model, and train the hyperspectral image fusion network model according to the visible light face preprocessing training image set and the infrared face preprocessing training image set to obtain a trained hyperspectral image fusion network Model;

将所述可见光人脸测试图像集和所述红外人脸测试图像集输入至所述训练好的超光谱图像融合网络模型得到超光谱人脸图像集，并将所述超光谱人脸图像集分为超光谱人脸训练图像集、超光谱人脸测试图像集；The visible light face test image set and the infrared face test image set are input into the trained hyperspectral image fusion network model to obtain a hyperspectral face image set, and the hyperspectral face image set is divided into two parts. For the hyperspectral face training image set, the hyperspectral face test image set;

构建卷积神经网络人脸识别模型，根据所述超光谱人脸训练图像集对所述卷积神经网络人脸识别模型进行训练得到训练好的卷积神经网络人脸识别模型；Constructing a convolutional neural network face recognition model, and training the convolutional neural network face recognition model according to the hyperspectral face training image set to obtain a trained convolutional neural network face recognition model;

将所述超光谱人脸测试图像集输入至所述训练好的卷积神经网络人脸识别模型得到人脸特征集，并使用支持向量机分类器对所述人脸特征集进行分类处理以实现超光谱人脸识别。The hyperspectral face test image set is input into the trained convolutional neural network face recognition model to obtain a face feature set, and the support vector machine classifier is used to classify the face feature set to achieve Hyperspectral face recognition.

在本发明的一个实施例中，对所述可见光人脸训练图像集、可见光人脸测试图像集、所述红外人脸训练图像集和红外人脸测试图像集分别进行预处理得到可见光人脸预处理训练图像集、可见光人脸预处理测试图像集、红外人脸预处理训练图像集和红外人脸预处理测试图像集，包括：In an embodiment of the present invention, the visible light face training image set, the visible light face test image set, the infrared face training image set and the infrared face test image set are preprocessed respectively to obtain the visible light face pre-processing image set. Processing training image set, visible light face preprocessing test image set, infrared face preprocessing training image set and infrared face preprocessing test image set, including:

对所述可见光人脸训练图像集和所述可见光人脸测试图像集分别进行灰度转换、归一化处理得到所述可见光人脸预处理训练图像集和所述可见光人脸预处理测试图像集；Performing grayscale conversion and normalization on the visible light face training image set and the visible light face test image set respectively to obtain the visible light face preprocessing training image set and the visible light face preprocessing test image set ;

对所述红外人脸训练图像集和所述红外人脸测试图像集分别进行图像增强、归一化处理得到所述红外人脸预处理训练图像集和所述红外人脸预处理测试图像集。Perform image enhancement and normalization on the infrared face training image set and the infrared face test image set respectively to obtain the infrared face preprocessing training image set and the infrared face preprocessing test image set.

在本发明的一个实施例中，构建的超光谱图像融合网络模型包括预融合层、编码器模块、融合层和解码器模块，其中，In an embodiment of the present invention, the constructed hyperspectral image fusion network model includes a pre-fusion layer, an encoder module, a fusion layer and a decoder module, wherein,

所述预融合层的输入与输入图像连接，所述预融合层的输出与所述编码器模块的输入连接，所述编码器模块包括依次连接的第一卷积层和密集残差模块，所述密集残差模块的输出与所述预融合层的输出进行全局残差连接输出，所述全局残差连接输出与所述融合层的输入连接；The input of the pre-fusion layer is connected to the input image, the output of the pre-fusion layer is connected to the input of the encoder module, and the encoder module includes a first convolution layer and a dense residual module connected in sequence, so The output of the dense residual module and the output of the pre-fusion layer are connected to the output of a global residual connection, and the output of the global residual connection is connected to the input of the fusion layer;

所述融合层的输出与所述解码器模块的输入连接，所述解码器模块包括依次连接第二卷积层～第五卷积层和反馈层，所述反馈层的输出与所述融合层的输出再次输入至所述第二卷积层构成反馈连接。The output of the fusion layer is connected to the input of the decoder module, and the decoder module includes sequentially connecting the second convolutional layer to the fifth convolutional layer and a feedback layer, and the output of the feedback layer is connected to the fusion layer. The output is input again to the second convolutional layer to form a feedback connection.

在本发明的一个实施例中，所述密集残差模块包括依次连接第一密集残差连接层～第三密集残差连接层、多尺度拼接层、第四密集残差连接层，所述第一密集残差连接层的输入还与所述第一密集残差连接层的输出、所述第二密集残差连接层的输出、所述第三密集残差连接层的输出连接，所述第二密集残差连接层的输入还与所述第二密集残差连接层的输出、所述第三密集残差连接层的输出连接，所述第三密集残差连接层的输入还与所述第三密集残差连接层的输出连接，所述第四密集残差连接层的输出还与所述第一卷积层的输入进行局部残差连接输出，所述局部残差连接输出与所述预融合层的输出进行全局残差连接输出。In an embodiment of the present invention, the dense residual module includes sequentially connecting the first dense residual connection layer to the third dense residual connection layer, a multi-scale splicing layer, and a fourth dense residual connection layer. The input of a dense residual connection layer is also connected to the output of the first dense residual connection layer, the output of the second dense residual connection layer, the output of the third dense residual connection layer, the The input of the second dense residual connection layer is also connected with the output of the second dense residual connection layer, the output of the third dense residual connection layer, and the input of the third dense residual connection layer is also connected with the The output of the third dense residual connection layer is connected, and the output of the fourth dense residual connection layer is further connected with the input of the first convolution layer for local residual connection output, and the output of the local residual connection is connected with the The output of the pre-fusion layer is output by global residual connection.

在本发明的一个实施例中，根据所述可见光人脸预处理训练图像集和所述红外人脸预处理训练图像集对所述超光谱图像融合网络模型进行训练得到训练好的超光谱图像融合网络模型，包括：In an embodiment of the present invention, the hyperspectral image fusion network model is trained according to the visible light face preprocessing training image set and the infrared face preprocessing training image set to obtain a trained hyperspectral image fusion Network model, including:

构建基于结构性损失、像素损失和平均梯度损失的复合损失函数；Build a composite loss function based on structural loss, pixel loss and average gradient loss;

根据所述可见光人脸预处理训练图像集和所述红外人脸预处理训练图像集并利用所述基于结构性损失、像素损失和平均梯度损失的复合损失函数对所述超光谱图像融合网络模型进行训练得到训练好的超光谱图像融合网络模型。According to the visible light face preprocessing training image set and the infrared face preprocessing training image set and using the composite loss function based on structural loss, pixel loss and average gradient loss, the hyperspectral image fusion network model After training, the trained hyperspectral image fusion network model is obtained.

在本发明的一个实施例中，构建的卷积神经网络人脸识别模型包括依次连接的ResNet特征提取模块、特征归一化模块、特征空间映射模块。In an embodiment of the present invention, the constructed convolutional neural network face recognition model includes a ResNet feature extraction module, a feature normalization module, and a feature space mapping module connected in sequence.

在本发明的一个实施例中，根据所述超光谱人脸训练图像集对所述卷积神经网络人脸识别模型进行训练得到训练好的卷积神经网络人脸识别模型，包括：In one embodiment of the present invention, the convolutional neural network face recognition model is trained according to the hyperspectral face training image set to obtain a trained convolutional neural network face recognition model, including:

构建三元组损失函数Triplet loss；Build triplet loss function Triplet loss;

根据超光谱人脸训练图像集并利用所述三元组损失函数Triplet loss对所述卷积神经网络人脸识别模型进行训练得到训练好的卷积神经网络人脸识别模型。According to the hyperspectral face training image set and using the triplet loss function Triplet loss, the convolutional neural network face recognition model is trained to obtain a trained convolutional neural network face recognition model.

本发明的另一个实施例提供了一种超光谱人脸识别装置，该装置包括：Another embodiment of the present invention provides a hyperspectral face recognition device, the device comprising:

数据获取模块，用于获取可见光人脸图像集和红外人脸图像集，并将所述可见光人脸图像集分为可见光人脸训练图像集和可见光人脸测试图像集，所述红外人脸图像集分为红外人脸训练图像集和红外人脸测试图像集；A data acquisition module is used to acquire a visible light face image set and an infrared face image set, and divide the visible light face image set into a visible light face training image set and a visible light face test image set, and the infrared face image set The sets are divided into infrared face training image set and infrared face test image set;

数据预处理模块，用于对所述可见光人脸训练图像集、可见光人脸测试图像集、所述红外人脸训练图像集和红外人脸测试图像集分别进行预处理得到可见光人脸预处理训练图像集、可见光人脸预处理测试图像集、红外人脸预处理训练图像集和红外人脸预处理测试图像集；The data preprocessing module is used to preprocess the visible light face training image set, the visible light face test image set, the infrared face training image set and the infrared face test image set respectively to obtain the visible light face preprocessing training Image set, visible light face preprocessing test image set, infrared face preprocessing training image set and infrared face preprocessing test image set;

第一模型构建训练模块，用于构建超光谱图像融合网络模型，根据所述可见光人脸预处理训练图像集和所述红外人脸预处理训练图像集对所述超光谱图像融合网络模型进行训练得到训练好的超光谱图像融合网络模型；The first model construction training module is used to construct a hyperspectral image fusion network model, and the hyperspectral image fusion network model is trained according to the visible light face preprocessing training image set and the infrared face preprocessing training image set Obtain the trained hyperspectral image fusion network model;

数据生成模块，用于将所述可见光人脸测试图像集和所述红外人脸测试图像集输入至所述训练好的超光谱图像融合网络模型得到超光谱人脸图像集，并将所述超光谱人脸图像集分为超光谱人脸训练图像集、超光谱人脸测试图像集；The data generation module is used to input the visible light face test image set and the infrared face test image set into the trained hyperspectral image fusion network model to obtain a hyperspectral face image set, and use the hyperspectral face image set. The spectral face image set is divided into a hyperspectral face training image set and a hyperspectral face testing image set;

第二模型构建训练模块，用于构建卷积神经网络人脸识别模型，根据所述超光谱人脸训练图像集对所述卷积神经网络人脸识别模型进行训练得到训练好的卷积神经网络人脸识别模型；The second model builds a training module, which is used to build a convolutional neural network face recognition model, and trains the convolutional neural network face recognition model according to the hyperspectral face training image set to obtain a trained convolutional neural network. face recognition model;

数据识别模块，用于将所述超光谱人脸测试图像集输入至所述训练好的卷积神经网络人脸识别模型得到人脸特征集，并使用支持向量机分类器对所述人脸特征集进行分类处理以实现超光谱人脸识别。A data recognition module is used to input the hyperspectral face test image set into the trained convolutional neural network face recognition model to obtain a face feature set, and use a support vector machine classifier to analyze the face features. Sets for classification processing to achieve hyperspectral face recognition.

本发明的再一个实施例提供了一种超光谱人脸识别电子设备，该电子设备包括图像采集器、显示器、处理器、通信接口、存储器和通信总线，其中，所述图像采集器、所述显示器、所述处理器、所述通信接口、所述存储器通过所述通信总线完成相互间的通信；Yet another embodiment of the present invention provides a hyperspectral face recognition electronic device, the electronic device includes an image collector, a display, a processor, a communication interface, a memory and a communication bus, wherein the image collector, the The display, the processor, the communication interface, and the memory communicate with each other through the communication bus;

所述图像采集器用于采集图像数据；The image collector is used to collect image data;

所述显示器用于显示图像识别数据；the display is used for displaying image recognition data;

所述存储器，用于存放计算机程序；the memory for storing computer programs;

所述处理器，用于执行所述存储器上存放的所述计算机程序时，实现上述任一所述的超光谱人脸识别方法。The processor is configured to implement any of the above-mentioned hyperspectral face recognition methods when executing the computer program stored in the memory.

本发明的又一个实施例提供了一种计算机可读存储介质，该计算机可读存储介质内存储有计算机程序，所述计算机程序被处理器执行时实现上述任一所述的超光谱人脸识别方法。Yet another embodiment of the present invention provides a computer-readable storage medium, where a computer program is stored in the computer-readable storage medium, and when the computer program is executed by a processor, any one of the above-mentioned hyperspectral face recognition is implemented method.

与现有技术相比，本发明的有益效果：Compared with the prior art, the beneficial effects of the present invention:

本发明提出了一套完整的超光谱人脸识别技术，可以解决传统人脸识别技术的适用范围窄、识别性能不高、且特征提取鲁棒性差等诸多缺陷；本实施例为人脸识别技术的实用化提供了新理论和新方法支持，使得人脸识别技术变得更加实用、可靠和普及化。The present invention proposes a complete set of hyperspectral face recognition technology, which can solve many defects of traditional face recognition technology, such as narrow application range, low recognition performance, and poor robustness of feature extraction. Practicalization provides new theories and new method support, making face recognition technology more practical, reliable and popular.

以下将结合附图及实施例对本发明做进一步详细说明。The present invention will be further described in detail below with reference to the accompanying drawings and embodiments.

附图说明Description of drawings

图1是本发明实施例提供的一种超光谱人脸识别方法的流程示意图；1 is a schematic flowchart of a hyperspectral face recognition method provided by an embodiment of the present invention;

图2是本发明实施例提供的一种超光谱人脸识别方法中超光谱图像融合网络模型的结构示意图；2 is a schematic structural diagram of a hyperspectral image fusion network model in a hyperspectral face recognition method provided by an embodiment of the present invention;

图3是本发明实施例提供的一种超光谱人脸识别方法中密集残差模块的结构示意图；3 is a schematic structural diagram of a dense residual module in a hyperspectral face recognition method provided by an embodiment of the present invention;

图4是本发明实施例提供的一种超光谱人脸识别方法中卷积神经网络人脸识别模型的结构示意图；4 is a schematic structural diagram of a convolutional neural network face recognition model in a hyperspectral face recognition method provided by an embodiment of the present invention;

图5a～5d是本发明实施例提供的一种超光谱人脸识别方法中可见光人脸图像、红外人脸图像示例示意图；5a-5d are schematic diagrams showing examples of visible light face images and infrared face images in a hyperspectral face recognition method provided by an embodiment of the present invention;

图6a～6b是本发明实施例提供的一种超光谱人脸识别方法中进行超光谱融合后的人脸图像示例示意图；Figures 6a-6b are schematic diagrams of examples of face images after hyperspectral fusion in a hyperspectral face recognition method provided by an embodiment of the present invention;

图7是本发明实施例提供的一种超光谱人脸识别装置的结构示意图；7 is a schematic structural diagram of a hyperspectral face recognition device provided by an embodiment of the present invention;

图8是本发明实施例提供的一种超光谱人脸识别电子设备的结构示意图；8 is a schematic structural diagram of an electronic device for hyperspectral face recognition provided by an embodiment of the present invention;

图9是本发明实施例提供的一种计算机可读存储介质的结构示意图。FIG. 9 is a schematic structural diagram of a computer-readable storage medium provided by an embodiment of the present invention.

具体实施方式Detailed ways

下面结合具体实施例对本发明做进一步详细的描述，但本发明的实施方式不限于此。The present invention will be described in further detail below with reference to specific embodiments, but the embodiments of the present invention are not limited thereto.

实施例一Example 1

目前人脸识别技术大多采用可见光成像和采集手段，因此该技术大多局限于白天光线充足的良好背景条件下，在光线不足和恶劣气候等环境下通常表现不佳。随着现实世界中各种复杂环境下应用的出现，基于可见光的人脸识别技术越来越难以满足要求，而红外线成像技术具备背景光线要求低、雨天雾天等气候下成像等等优势，弥补了可见光成像环境的不足。因此，请参见图1，图1是本发明实施例提供的一种超光谱人脸识别方法的流程示意图，本实施例提供了一种超光谱人脸识别方法，该方法包括以下步骤：At present, most face recognition technologies use visible light imaging and acquisition methods, so the technology is mostly limited to good background conditions with sufficient light during the day, and usually performs poorly in environments such as insufficient light and harsh weather. With the emergence of applications in various complex environments in the real world, face recognition technology based on visible light is more and more difficult to meet the requirements, while infrared imaging technology has the advantages of low background light requirements, imaging in rainy and foggy days and other climates, etc., to make up for It overcomes the shortcomings of the visible light imaging environment. Therefore, please refer to FIG. 1. FIG. 1 is a schematic flowchart of a hyperspectral face recognition method provided by an embodiment of the present invention. The present embodiment provides a hyperspectral face recognition method, and the method includes the following steps:

步骤1、获取可见光人脸图像集和红外人脸图像集，并将可见光人脸图像集分为可见光人脸训练图像集和可见光人脸测试图像集，红外人脸图像集分为红外人脸训练图像集和红外人脸测试图像集。Step 1. Obtain a visible light face image set and an infrared face image set, and divide the visible light face image set into a visible light face training image set and a visible light face test image set, and the infrared face image set is divided into infrared face training. Image set and infrared face test image set.

具体而言，本实施例使用可见光和红外线摄像头同时对些个体进行人脸图像采集得到可见光人脸图像集和红外人脸图像集。可见光人脸图像集和红外人脸图像集用于后续人脸识别。Specifically, in this embodiment, visible light and infrared cameras are used to collect face images of some individuals at the same time to obtain a visible light face image set and an infrared face image set. The visible light face image set and the infrared face image set are used for subsequent face recognition.

步骤2、对可见光人脸训练图像集、可见光人脸测试图像集、红外人脸训练图像集和红外人脸测试图像集分别进行预处理得到可见光人脸预处理训练图像集、可见光人脸预处理测试图像集、红外人脸预处理训练图像集和红外人脸预处理测试图像集。Step 2. Preprocess the visible light face training image set, the visible light face test image set, the infrared face training image set and the infrared face test image set respectively to obtain the visible light face preprocessing training image set and the visible light face preprocessing Test image set, infrared face preprocessing training image set and infrared face preprocessing test image set.

具体而言，为了实现更好的图像融合，本实施例在进行图像融合之前，先对可见光人脸训练图像集、可见光人脸测试图像集、红外人脸训练图像集和红外人脸测试图像集中的人脸图像作归一化和对比度调整，具体地本实施例步骤2包括步骤2.1、步骤2.2：Specifically, in order to achieve better image fusion, in this embodiment, before image fusion is performed, the visible light face training image set, the visible light face test image set, the infrared face training image set and the infrared face test image set are collected. The face image obtained is normalized and contrast adjusted. Specifically,step 2 of this embodiment includes step 2.1 and step 2.2:

步骤2.1、对可见光人脸训练图像集和可见光人脸测试图像集分别进行灰度转换、归一化处理得到可见光人脸预处理训练图像集和可见光人脸预处理测试图像集。Step 2.1. Perform grayscale conversion and normalization on the visible light face training image set and the visible light face test image set respectively to obtain the visible light face preprocessing training image set and the visible light face preprocessing test image set.

具体而言，本实施例将可见光人脸训练图像集和可见光人脸测试图像集中的人脸图像首先进行灰度转换为灰度图像，其灰度转换的公式设计具体如下：Specifically, in this embodiment, the face images in the visible light face training image set and the visible light face test image set are first converted into grayscale images, and the formula design of the grayscale conversion is as follows:

I_gray＝0.2989×R+0.5870×G+0.1140×B (1)I_gray = 0.2989×R+0.5870×G+0.1140×B (1)

其中，I_gray为灰度转换后灰度图像输出，R、G、B为灰度转换前图像对应的RGB值，本实施例具体为可见光人脸训练图像集和可见光人脸测试图像集中人脸图像对应的RGB值。Wherein, I_gray is the grayscale image output after the grayscale conversion, R, G, B are the RGB values corresponding to the image before the grayscale conversion, and the present embodiment is specifically the human face in the visible light face training image set and the visible light face test image set The RGB value corresponding to the image.

然后，再将灰度图像I_gray进行归一化处理，其归一化到[0，255]的归一化公式设计具体如下：Then, the grayscale image I_gray is normalized, and the normalization formula design of its normalization to [0, 255] is as follows:

其中，I_n为灰度图像I_gray的归一化后图像输出，I_max和I_min分别为灰度图像I_gray中的最大及最小灰度值。Wherein, I_n is the normalized image output of the grayscale image I_gray , and_Imax and_Imin are the maximum and minimum grayscale values in the grayscale image I_gray , respectively.

本实施例对可见光人脸训练图像集和可见光人脸测试图像集中的每一幅可见光人脸训练图像、可见光人脸测试图像均分别通过上述公式(1)、公式(2)的处理，进而得到可见光人脸预处理训练图像集和可见光人脸预处理测试图像集。In this embodiment, each visible-light face training image and visible-light face test image in the visible-light face training image set and the visible-light face test image set are processed by the above formula (1) and formula (2) respectively, and then obtain Visible light face preprocessing training image set and visible light face preprocessing test image set.

步骤2.2、对红外人脸训练图像集和红外人脸测试图像集分别进行图像增强、归一化处理得到红外人脸预处理训练图像集和红外人脸预处理测试图像集。Step 2.2: Perform image enhancement and normalization on the infrared face training image set and the infrared face test image set respectively to obtain an infrared face preprocessing training image set and an infrared face preprocessing test image set.

具体而言，本实施例将红外人脸训练图像集和红外人脸测试图像集中的人脸图像首先使用log算子进行图像增强，其图像增强的公式设计具体如下：Specifically, in this embodiment, the face images in the infrared face training image set and the infrared face test image set are first enhanced using the log operator, and the formula design of the image enhancement is as follows:

I＝log(1+X) (3)I=log(1+X) (3)

其中，I为图像增强后的图像，X为图像增强前的图像，本实施例具体为红外人脸训练图像集和红外人脸测试图像集中人脸图像。Wherein, I is an image after image enhancement, X is an image before image enhancement, and this embodiment is specifically a human face image in an infrared face training image set and an infrared face test image set.

然后，再对增强后的图像I进行归一化处理，其归一化到[0，255]的归一化公式如上述公式(2)。Then, normalize the enhanced image I, and the normalization formula for normalizing it to [0, 255] is the above formula (2).

本实施例对红外人脸训练图像集和红外人脸测试图像集中的每一幅红外人脸训练图像、红外人脸测试图像均分别通过上述公式(3)、公式(2)的处理，进而得到红外人脸预处理训练图像集和红外人脸预处理测试图像集。In this embodiment, each infrared face training image and infrared face test image in the infrared face training image set and the infrared face test image set are processed by the above formula (3) and formula (2), respectively, and then obtain Infrared face preprocessing training image set and infrared face preprocessing test image set.

步骤3、构建超光谱图像融合网络模型，根据可见光人脸预处理训练图像集和红外人脸预处理训练图像集对超光谱图像融合网络模型进行训练得到训练好的超光谱图像融合网络模型。Step 3: Build a hyperspectral image fusion network model, and train the hyperspectral image fusion network model according to the visible light face preprocessing training image set and the infrared face preprocessing training image set to obtain a trained hyperspectral image fusion network model.

具体而言，为了获得同时具备丰富纹理信息和热信息的高质量超光谱图像，本实施例提出了一种新型的反馈式深度融合框架将红外图像与可见光图像进行融合，具体地本实施例步骤3包括步骤3.1、步骤3.2：Specifically, in order to obtain high-quality hyperspectral images with rich texture information and thermal information at the same time, this embodiment proposes a novel feedback depth fusion framework to fuse infrared images and visible light images. Specifically, the steps of this embodiment are 3 includes steps 3.1 and 3.2:

步骤3.1、构建超光谱图像融合网络模型。Step 3.1, build a hyperspectral image fusion network model.

具体而言，请参见图2，图2是本发明实施例提供的一种超光谱人脸识别方法中超光谱图像融合网络模型的结构示意图，本实施例提出了一鲁棒性高，且专门针对超光谱人脸图像融合的超光谱图像融合网络模型RFFuseNet，具体地：构建的超光谱图像融合网络模型包括预融合层PF、编码器模块、融合层F和解码器模块，其中，预融合层PF的输入与输入图像连接，预融合层PF的输出与编码器模块的输入连接，编码器模块包括依次连接的第一卷积层C1和密集残差模块RDB，密集残差模块RDB的输出与预融合层PF的输出进行全局残差连接输出，全局残差连接输出与融合层F的输入连接；融合层F的输出与解码器模块的输入连接，解码器模块包括依次连接第二卷积层C2～第五卷积层C5和反馈层FB，反馈层FB的输出与融合层F的输出再次输入至第二卷积层C2构成反馈连接。其中，请参见图3，图3是本发明实施例提供的一种超光谱人脸识别方法中密集残差模块的结构示意图，密集残差模块RDB包括依次连接的第一密集残差连接层C_R_1～第三密集残差连接层C_R_3、多尺度拼接层CC、第四密集残差连接层C_R_4，第一密集残差连接层的输入还与第一密集残差连接层的输出、第二密集残差连接层的输出、第三密集残差连接层的输出连接，第二密集残差连接层的输入还与第二密集残差连接层的输出、第三密集残差连接层的输出连接，第三密集残差连接层的输入还与第三密集残差连接层的输出连接，第四密集残差连接层C_R_4的输出还与第一卷积层C1的输入进行局部残差连接输出，局部残差连接输出与预融合层PF的输出进行全局残差连接输出。Specifically, please refer to FIG. 2. FIG. 2 is a schematic structural diagram of a hyperspectral image fusion network model in a hyperspectral face recognition method provided by an embodiment of the present invention. The hyperspectral image fusion network model RFFuseNet for hyperspectral face image fusion, specifically: the constructed hyperspectral image fusion network model includes a pre-fusion layer PF, an encoder module, a fusion layer F and a decoder module, wherein the pre-fusion layer PF The input of the pre-fusion layer PF is connected with the input image, and the output of the pre-fusion layer PF is connected with the input of the encoder module. The encoder module includes the first convolutional layer C1 and the dense residual module RDB connected in sequence. The output of the dense residual module RDB is connected with the pre- The output of the fusion layer PF is connected to the output of the global residual connection, and the output of the global residual connection is connected to the input of the fusion layer F; the output of the fusion layer F is connected to the input of the decoder module, and the decoder module consists of connecting the second convolution layer C2 in turn ~ The fifth convolution layer C5 and the feedback layer FB, the output of the feedback layer FB and the output of the fusion layer F are input again to the second convolution layer C2 to form a feedback connection. 3 is a schematic structural diagram of a dense residual module in a hyperspectral face recognition method provided by an embodiment of the present invention. The dense residual module RDB includes a first dense residual connection layer C_R_1 connected in sequence ~The third dense residual connection layer C_R_3, the multi-scale splicing layer CC, the fourth dense residual connection layer C_R_4, the input of the first dense residual connection layer is also connected with the output of the first dense residual connection layer, the second dense residual connection layer. The output of the difference connection layer and the output of the third dense residual connection layer are connected, and the input of the second dense residual connection layer is also connected to the output of the second dense residual connection layer and the output of the third dense residual connection layer. The input of the three dense residual connection layer is also connected with the output of the third dense residual connection layer, and the output of the fourth dense residual connection layer C_R_4 is also connected with the input of the first convolutional layer C1 for local residual connection output. The output of the difference connection is connected with the output of the pre-fusion layer PF for global residual connection output.

步骤3.2、根据可见光人脸预处理训练图像集和红外人脸预处理训练图像集对超光谱图像融合网络模型进行训练得到训练好的超光谱图像融合网络模型。Step 3.2, train the hyperspectral image fusion network model according to the visible light face preprocessing training image set and the infrared face preprocessing training image set to obtain a trained hyperspectral image fusion network model.

具体而言，本实施例训练过程中，构建了基于结构性损失、像素损失和平均梯度损失的复合损失函数，具体使用结构性损失、像素损失和平均梯度损失总和作为损失函数，具体该复合损失函数设计如下：Specifically, in the training process of this embodiment, a composite loss function based on structural loss, pixel loss and average gradient loss is constructed, and the sum of structural loss, pixel loss and average gradient loss is used as the loss function. Specifically, the composite loss The function design is as follows:

Loss＝λL_ssim+L_p+0.05L_ag (4)Loss=λL_ssim +L_p +0.05L_ag (4)

其中，λ为参数，可取1、10、100和1000，L_ssim、L_p及L_ag分别表示结构性损失、像素损失和平均梯度损失，具体L_ssim、L_p及L_ag设计如下：Among them, λ is a parameter, which can be 1, 10, 100 and 1000. L_ssim , L_p and La_ag represent structural loss, pixel loss and average gradient loss, respectively. The specific designs of L_ssim , L_p and La_ag are as follows:

L_ssim＝1-SSIM(O,I) (5)L_ssim =1-SSIM(O,I) (5)

L_p＝||O-I||₂ (6)L_p = ||OI||₂ (6)

其中，O和I分别表示超光谱图像融合网络模型对应的输出图像和输入图像，M×N表示输出图像的大小，

和

分别表示输出图像在水平方向和垂直方向的梯度，SSIM(O,I)表示输出图像和输入图像间的结构相似性程度，具体SSIM(O,I)计算方式如下：Among them, O and I represent the output image and input image corresponding to the hyperspectral image fusion network model, respectively, M×N represents the size of the output image,

and

Represent the gradient of the output image in the horizontal and vertical directions, respectively, and SSIM(O,I) represents the degree of structural similarity between the output image and the input image. The specific SSIM(O,I) calculation method is as follows:

SSIM(O,I)＝[l(O,I)^α·c(O,I)^β·s(O,I)^γ] (8)SSIM(O,I)=[l(O,I)^α ·c(O,I)^β ·s(O,I)^γ ] (8)

其中，l(O,I)表示输出图像和输入图像亮度均值，c(O,I)表示输出图像和输入图像对比度方差，s(O,I)表示输出图像和输入图像结构相似度值，α、β、γ为调整超光谱图像融合网络模型中三个成分所占比重的参数。Among them, l(O,I) represents the average brightness of the output image and the input image, c(O,I) represents the contrast variance between the output image and the input image, s(O,I) represents the structural similarity between the output image and the input image, α , β and γ are the parameters to adjust the proportion of the three components in the hyperspectral image fusion network model.

进一步地，根据可见光人脸预处理训练图像集和红外人脸预处理训练图像集并利用基于结构性损失、像素损失和平均梯度损失的复合损失函数对超光谱图像融合网络模型进行训练得到训练好的超光谱图像融合网络模型。Further, the hyperspectral image fusion network model is trained according to the visible light face preprocessing training image set and the infrared face preprocessing training image set and using the composite loss function based on structural loss, pixel loss and average gradient loss. The hyperspectral image fusion network model.

具体而言，本实施例通过上述公式(4)构建的超光谱图像融合网络模型的复合损失函数，基于该复合损失函数，并将可见光人脸预处理训练图像集和红外人脸预处理训练图像集中人脸训练图像输入至超光谱图像融合网络模型中进行训练，训练过程中具体可以采用反向传播算法训练超光谱图像融合网络模型，从而得到本实施例最终训练好的超光谱图像融合网络模型，用于后续超光谱人脸图像集生成。Specifically, in this embodiment, the composite loss function of the hyperspectral image fusion network model constructed by the above formula (4), based on the composite loss function, the visible light face preprocessing training image set and the infrared face preprocessing training image The centralized face training image is input into the hyperspectral image fusion network model for training. In the training process, the back propagation algorithm can be used to train the hyperspectral image fusion network model, so as to obtain the final trained hyperspectral image fusion network model in this embodiment. , for subsequent generation of hyperspectral face image sets.

步骤4、将可见光人脸测试图像集和红外人脸测试图像集输入至训练好的超光谱图像融合网络模型得到超光谱人脸图像集，并将超光谱人脸图像集分为超光谱人脸训练图像集、超光谱人脸测试图像集。Step 4. Input the visible light face test image set and the infrared face test image set into the trained hyperspectral image fusion network model to obtain a hyperspectral face image set, and divide the hyperspectral face image set into hyperspectral faces Training image set, hyperspectral face test image set.

具体而言，本实施例在步骤3中得到了训练好的超光谱图像融合网络模型，利用该超光谱图像融合网络模型对可见光人脸测试图像集和红外人脸测试图像集中的人脸图像进行人脸图像融合处理，从而获得同时具备丰富纹理信息和热信息的高质量超光谱人脸图像集。其中，将超光谱人脸图像集分为超光谱人脸训练图像集、超光谱人脸测试图像集用于后续超光谱人脸识别，优选地，超光谱人脸训练图像集、超光谱人脸测试图像集划分比例为3:1。Specifically, in the present embodiment, a trained hyperspectral image fusion network model is obtained instep 3, and the hyperspectral image fusion network model is used to perform the detection on the face images in the visible light face test image set and the infrared face test image set. Face image fusion processing to obtain a high-quality hyperspectral face image set with rich texture information and thermal information at the same time. Wherein, the hyperspectral face image set is divided into a hyperspectral face training image set and a hyperspectral face test image set for subsequent hyperspectral face recognition, preferably, a hyperspectral face training image set, a hyperspectral face test image set The test image set is divided into a ratio of 3:1.

步骤5、构建卷积神经网络人脸识别模型，根据超光谱人脸训练图像集对卷积神经网络人脸识别模型进行训练得到训练好的卷积神经网络人脸识别模型。Step 5: Construct a convolutional neural network face recognition model, and train the convolutional neural network face recognition model according to the hyperspectral face training image set to obtain a trained convolutional neural network face recognition model.

具体而言，本实施例将深度学习人脸识别技术和深度超光谱融合技术相结合来提高人脸的最终识别率，在步骤4生成超光谱人脸图像集后，又提出了基于深度学习的超光谱人脸识别框架，即卷积神经网络人脸识别模型，具体地本实施例步骤5包括步骤5.1、步骤5.2：Specifically, this embodiment combines the deep learning face recognition technology and the deep hyperspectral fusion technology to improve the final recognition rate of the face. After the hyperspectral face image set is generated instep 4, a deep learning-based The hyperspectral face recognition framework, namely the convolutional neural network face recognition model, specifically, step 5 of this embodiment includes step 5.1 and step 5.2:

步骤5.1、构建卷积神经网络人脸识别模型。Step 5.1, build a convolutional neural network face recognition model.

具体而言，请参见图4，图4是本发明实施例提供的一种超光谱人脸识别方法中卷积神经网络人脸识别模型的结构示意图，本实施例构建的卷积神经网络人脸识别模型包括依次连接的ResNet特征提取模块、特征归一化模块L2、特征空间映射模块ED。其中，卷积神经网络人脸识别模型的输入图像Batch首先被裁剪至固定尺寸；ResNet特征提取模块、特征归一化模块L2、特征空间映射模块ED可以分别通过现有常见方法实现，具体实现方式不限。Specifically, please refer to FIG. 4. FIG. 4 is a schematic structural diagram of a convolutional neural network face recognition model in a hyperspectral face recognition method provided by an embodiment of the present invention. The convolutional neural network face constructed in this embodiment The recognition model includes a ResNet feature extraction module, a feature normalization module L2, and a feature space mapping module ED which are connected in sequence. Among them, the input image Batch of the convolutional neural network face recognition model is first cropped to a fixed size; the ResNet feature extraction module, the feature normalization module L2, and the feature space mapping module ED can be implemented by existing common methods respectively. The specific implementation methods Unlimited.

步骤5.2、根据超光谱人脸训练图像集对卷积神经网络人脸识别模型进行训练得到训练好的卷积神经网络人脸识别模型。Step 5.2, train the convolutional neural network face recognition model according to the hyperspectral face training image set to obtain a trained convolutional neural network face recognition model.

具体而言，本实施例在人脸识别训练过程中构建了三元组损失函数Tripletloss，使用该三元组损失函数Triplet loss来训练卷积神经网络人脸识别模型，所使用的三元组损失函数Triplet loss设计如下：Specifically, in this embodiment, a triplet loss function Tripletloss is constructed in the face recognition training process, and the triplet loss function Tripletloss is used to train the convolutional neural network face recognition model. The triplet loss used is The function Triplet loss is designed as follows:

其中，

以及

表示卷积神经网络人脸识别模型的三张输入图像，

与

是来自同类的两张图像，

与

是来自异类的两张图像，α为间隔参，相应地，

以及

为卷积神经网络人脸识别模型的输出特征，将其组成三元组作为损失函数Triplet loss。in,

as well as

Three input images representing the convolutional neural network face recognition model,

and

are two images from the same class,

and

are two images from heterogeneous, α is the interval parameter, correspondingly,

as well as

For the output features of the convolutional neural network face recognition model, it is composed of triplet as the loss function Triplet loss.

可见，本实施例三元组损失函数Triplet loss需要接收三张图像的特征以及对应的标签作为输入，其目的在于通过大量的三元组训练使得类内距离小于类间距离，训练过程中选用的优化器为Adam，学习率为0.001，批大小定为128，通过Tensor Board插件查看损失函数值与迭代次数的曲线设定一个合适的阈值让模型停止训练，从而得到训练好的卷积神经网络人脸识别模型。其中，三元组损失函数Triplet loss接收的图像来自超光谱人脸训练图像集。It can be seen that the triplet loss function Triplet loss in this embodiment needs to receive the features of three images and the corresponding labels as input, and its purpose is to make the intra-class distance smaller than the inter-class distance through a large number of triplet training. The optimizer is Adam, the learning rate is 0.001, and the batch size is set to 128. Use the Tensor Board plug-in to view the curve of the loss function value and the number of iterations, and set a suitable threshold to stop the model from training, so as to obtain a trained convolutional neural network. face recognition model. Among them, the images received by the triplet loss function Triplet loss are from the hyperspectral face training image set.

步骤6、将超光谱人脸测试图像集输入至训练好的卷积神经网络人脸识别模型得到人脸特征集，并使用支持向量机分类器对人脸特征集进行分类处理以实现超光谱人脸识别。Step 6. Input the hyperspectral face test image set into the trained convolutional neural network face recognition model to obtain the face feature set, and use the support vector machine classifier to classify the face feature set to realize the hyperspectral human face. face recognition.

具体而言，本实施例使用欧氏距离作为度量函数，将超光谱人脸测试图像集输入至训练好的卷积神经网络人脸识别模型进行特征提取得到人脸特征集，将人脸特征集以及其标签输入至支持向量机SVM分类器，通过支持向量机SVM分类器得到人脸识别的正确率以实现超光谱人脸识别。其中，通过支持向量机SVM分类器得到人脸识别的正确率，其步骤具体如下：Specifically, in this embodiment, the Euclidean distance is used as a metric function, and the hyperspectral face test image set is input into the trained convolutional neural network face recognition model for feature extraction to obtain a face feature set. And its label is input to the support vector machine SVM classifier, and the correct rate of face recognition is obtained through the support vector machine SVM classifier to realize hyperspectral face recognition. Among them, the correct rate of face recognition is obtained through the support vector machine SVM classifier, and the steps are as follows:

(1)、以类内最大欧式距离D_max作为设定阈值，将预测标签与当前图像间最小欧式距离D_{bw_min}与D_max比较，若小于D_max则认为预测分类正确，否则认为预测分类错误，统计分类正确与错误的类别数。(1) Using the maximum Euclidean distance_Dmax within the class as the set threshold, compare the minimum Euclidean distance_{Dbw_min} between the predicted label and the current image with_Dmax . If it is less than_Dmax , the predicted classification is considered correct, otherwise, the predicted classification is considered wrong. Count the number of correct and incorrect categories.

(2)、根据(1)中统计的分类正确与错误的类别数，计算得到分类正确率(Accuracy)，具体公式设计如下：(2) According to the number of correct and incorrect categories in (1), the accuracy of classification (Accuracy) is calculated. The specific formula is designed as follows:

其中，TN表示真正例，TP表示真负例，FP表示假正例，FN表示假负例。Among them, TN represents the true example, TP represents the true negative example, FP represents the false positive example, and FN represents the false negative example.

综上所述，本实施例深度图像融合的超光谱人脸识别技术的引入，通过深度学习和图像融合技术来逐步提升最终识别准确率，具体来说：首先，在进行图像融合之前先对可见光图像和红外图像作归一化和对比度调整，然后利用本申请所提出的深度融合框架RFFuseNet(超光谱图像融合网络模型)将红外图像与可见光图像进行融合，以获得同时具备丰富纹理信息和热信息的高质量超光谱图像，最后，利用深度学习人脸识别方法(卷积神经网络人脸识别模型)对超光谱人脸图像进行识别，从而提高了传统基于可见光图像的人脸识别性能。To sum up, the introduction of the hyperspectral face recognition technology of deep image fusion in this embodiment gradually improves the final recognition accuracy through deep learning and image fusion technology. The image and infrared image are normalized and contrast adjusted, and then the infrared image and the visible light image are fused by the deep fusion framework RFFuseNet (hyperspectral image fusion network model) proposed in this application to obtain both rich texture information and thermal information. Finally, a deep learning face recognition method (convolutional neural network face recognition model) is used to recognize the hyperspectral face images, thereby improving the traditional face recognition performance based on visible light images.

为了验证本申请提出的超光谱人脸识别方法的优越性，本申请所使用的人脸数据集为CASIA人脸图像集、QFIRE人脸图像集，请参见图5a～5d，图5a～5d是本发明实施例提供的一种超光谱人脸识别方法中可见光人脸图像、红外人脸图像示例示意图，具体地：图5(a)和图5(c)为可见光在1.5m处拍摄所得人脸图像，图5(b)和图5(d)为在近红外1.5m处拍摄所得人脸图像，图5(a)为CASIA可见光图像集，图5(b)为CASIA近红外图像集，图5(c)为QFIRE可见光图像集，图5(d)为QFIRE近红外图像集。In order to verify the superiority of the hyperspectral face recognition method proposed in this application, the face data sets used in this application are the CASIA face image set and the QFIRE face image set. A schematic diagram of an example of a visible light face image and an infrared face image in a hyperspectral face recognition method provided by an embodiment of the present invention, specifically: Figures 5(a) and 5(c) are people obtained by photographing visible light at a distance of 1.5 m. Face images, Figure 5(b) and Figure 5(d) are the face images captured at near-infrared 1.5m, Figure 5(a) is the CASIA visible light image set, Figure 5(b) is the CASIA near-infrared image set, Figure 5(c) is the QFIRE visible light image set, and Figure 5(d) is the QFIRE near-infrared image set.

本实施例验证过程中超光谱图像融合网络模型中每一层的参数设计具体参见表1，卷积过程中padding方式为0填充。其中，超光谱图像融合网络模型中每一层的参数根据实际情况而设计，本实施例以表1中的具体参数设计进行识别验证。For the parameter design of each layer in the hyperspectral image fusion network model in the verification process of this embodiment, see Table 1 for details, and the padding method is 0 padding in the convolution process. The parameters of each layer in the hyperspectral image fusion network model are designed according to the actual situation, and the specific parameter design in Table 1 is used for identification and verification in this embodiment.

表1超光谱图像融合网络模型中每一层的参数设计Table 1 Parameter design of each layer in the hyperspectral image fusion network model

本实施例设计的实验从以下两个方面进行论证：The experiments designed in this example are demonstrated from the following two aspects:

(1)、为说明本申请的超光谱图像融合网络模型RFFuseNet的融合效果优于传统人脸图像融合和其他主流深度学习方法，将其与基于CBF的图像融合(传统方法)和基于DenseFuse的图像融合(主流深度学习方法)等进行了对比，计算了熵值(EN)、结构相似性(SSIM)、图像边缘保真度(Q_abf)和人工噪声(N_abf)等融合效果的指标。请参见图6a～6b，图6a～6b是本发明实施例提供的一种超光谱人脸识别方法中进行超光谱融合后的人脸图像示例示意图，图6(a)为QFIRE图像融合结果，图6(b)为CASIA图像融合结果。请参见表2，表2为本实施例提供的不同图像融合方法关于熵值(EN)、图像边缘保真度(Q_abf)、结构相似性(SSIM)和人工噪声(N_abf)的对比结果。其中，熵值(EN)、图像边缘保真度(Q_abf)、结构相似性(SSIM)值越大说明融合效果越好，人工噪声(N_abf)值越小说明融合效果越好。(1) To illustrate that the fusion effect of the hyperspectral image fusion network model RFFuseNet of this application is better than traditional face image fusion and other mainstream deep learning methods, it is combined with CBF-based image fusion (traditional method) and DenseFuse-based image fusion Fusion (mainstream deep learning method) is compared, and indicators of fusion effects such as entropy (EN), structural similarity (SSIM), image edge fidelity (Q_abf ) and artificial noise (N_abf ) are calculated. Please refer to FIGS. 6a to 6b. FIGS. 6a to 6b are schematic diagrams of examples of face images after hyperspectral fusion in a hyperspectral face recognition method provided by an embodiment of the present invention, and FIG. 6(a) is the result of QFIRE image fusion. Figure 6(b) is the result of CASIA image fusion. Please refer to Table 2. Table 2 shows the comparison results of entropy (EN), image edge fidelity (Q_abf ), structural similarity (SSIM), and artificial noise (N_abf ) for different image fusion methods provided in this embodiment. . Among them, the larger the entropy value (EN), the image edge fidelity (Q_abf ) and the structural similarity (SSIM) value, the better the fusion effect, and the smaller the artificial noise (N_abf ) value, the better the fusion effect.

表2本实施例提供的不同图像融合方法的对比结果Table 2 Comparison results of different image fusion methods provided in this embodiment

由表2对比结果表明，采用本申请的RFFuseNet网络融合的超光谱人脸图像具有最大熵值、结构相似性、图像边缘保真度以及最小的人工噪声，从而说明本申请所设计的RFFuseNet方法的融合性能优于传统图像融合方法和主流深度学习方法。The comparison results in Table 2 show that the hyperspectral face image fused by the RFFuseNet network of the present application has maximum entropy, structural similarity, image edge fidelity and minimum artificial noise, thereby illustrating the RFFuseNet method designed by the present application. The fusion performance is superior to traditional image fusion methods and mainstream deep learning methods.

(2)、为证明本申请的超光谱图像融合网络模型RFFuseNet的人脸识别性能要高于不使用融合技术的识别性能，分别对不使用融合的单光谱图像(即可见光和红外图像)、使用基于CBF的图像融合、使用基于DenseFuse的图像融合、使用本申请RFFuseNet融合等情形进行人脸识别实验。请参见表3，表3为本实施例提供的不同方法的人脸识别准确率对比结果，包括使用深度融合技术前后的结果。(2) In order to prove that the face recognition performance of the hyperspectral image fusion network model RFFuseNet of this application is higher than the recognition performance without fusion technology, the single spectrum images without fusion (ie visible light and infrared images), Face recognition experiments are carried out in the cases of CBF-based image fusion, using DenseFuse-based image fusion, and using RFFuseNet fusion in this application. Please refer to Table 3. Table 3 shows the comparison results of the accuracy of face recognition of different methods provided in this embodiment, including the results before and after using the deep fusion technology.

表3本实施例提供的不同方法的人脸识别准确率对比结果Table 3 Comparison results of face recognition accuracy rates of different methods provided in this embodiment

由表3对比结果可以看出，使用本申请RFFuseNet融合的超光谱图像识别率明显高于两个单光谱图像，即证明本申请RFFuseNet融合方法可有效提高人脸识别性能，同时本申请RFFuseNet融合的超光谱图像识别率也高于采用CBF和DenseFuse等方法的超光谱图像识别率，即证明本申请RFFuseNet模型的超光谱人脸识别性能优于其他人脸融合方法。From the comparison results in Table 3, it can be seen that the recognition rate of hyperspectral images fused using RFFuseNet of the present application is significantly higher than that of the two single-spectral images, which proves that the RFFuseNet fusion method of the present application can effectively improve the performance of face recognition. The hyperspectral image recognition rate is also higher than that of the hyperspectral image recognition rates using methods such as CBF and DenseFuse, which proves that the hyperspectral face recognition performance of the RFFuseNet model in this application is superior to other face fusion methods.

可见，本实施例针对传统人脸识别方法仅采用可见光的局限性，引入了图像融合思想，将可见光人脸图像和红外线人脸图像进行融合得到超光谱人脸图像，从而使人脸图像同时具备可见光和红外线的互补信息(即丰富纹理和热信息)，达到提升人脸识别的性能的效果；本实施例针对人脸图像融合问题设计了一个新型的残差反馈式超光谱图像融合网络模型RFFuseNet，实验表明，与传统融合方法相比，本申请所提出的超光谱图像融合网络模型RFFuseNet无需手工设计融合规则和选取融合参数，并且能够融合出更高质量、包含信息更丰富的超光谱人脸图像，与其他基于深度学习的融合方法比，也具备更好的融合指标；本实施例将深度学习人脸识别技术和深度超光谱融合技术相结合，提出了基于深度学习的超光谱人脸识别框架，可以成功解决传统人脸识别局限于可见光和识别性能不足等问题，实验表明本申请方法的使用可以明显提高最终人脸识别率。It can be seen that, in view of the limitation that the traditional face recognition method only uses visible light, this embodiment introduces the idea of image fusion, and fuses the visible light face image and the infrared face image to obtain a hyperspectral face image, so that the face image has both The complementary information of visible light and infrared (that is, rich texture and thermal information) can improve the performance of face recognition; this embodiment designs a new residual feedback hyperspectral image fusion network model RFFuseNet for the problem of face image fusion. , Experiments show that, compared with the traditional fusion method, the hyperspectral image fusion network model RFFuseNet proposed in this application does not require manual design of fusion rules and selection of fusion parameters, and can fuse hyperspectral faces with higher quality and richer information. Compared with other fusion methods based on deep learning, images also have better fusion indicators; this embodiment combines deep learning face recognition technology and deep hyperspectral fusion technology, and proposes deep learning based hyperspectral face recognition. The framework can successfully solve the problems of traditional face recognition limited to visible light and insufficient recognition performance. Experiments show that the use of the method in this application can significantly improve the final face recognition rate.

本实施例提出了一套完整的超光谱人脸识别技术，可以解决传统人脸识别技术的适用范围窄、识别性能不高、且特征提取鲁棒性差等诸多缺陷；本实施例为人脸识别技术的实用化提供了新理论和新方法支持，使得人脸识别技术变得更加实用、可靠和普及化；本实施例可以广泛应用于户外、夜间、雨雪和其他复杂环境下的考勤、民用监控、公安执法、进出管控、小区入口等等应用场合。This embodiment proposes a complete set of hyperspectral face recognition technology, which can solve many defects of traditional face recognition technology, such as narrow application range, low recognition performance, and poor robustness of feature extraction; this embodiment is a face recognition technology The practical application of the face recognition technology provides new theories and new methods to support, making face recognition technology more practical, reliable and popular; this embodiment can be widely used in outdoor, night, rain and snow and other complex environments for attendance, civil monitoring , public security law enforcement, access control, community entrance and other applications.

实施例二Embodiment 2

在上述实施例一的基础上，请参见图7，图7是本发明实施例提供的一种超光谱人脸识别装置的结构示意图。本实施例提供了一种超光谱人脸识别装置，该装置包括：On the basis of the above-mentioned first embodiment, please refer to FIG. 7 , which is a schematic structural diagram of a hyperspectral face recognition device provided by an embodiment of the present invention. This embodiment provides a hyperspectral face recognition device, and the device includes:

数据获取模块，用于获取可见光人脸图像集和红外人脸图像集，并将可见光人脸图像集分为可见光人脸训练图像集和可见光人脸测试图像集，红外人脸图像集分为红外人脸训练图像集和红外人脸测试图像集。The data acquisition module is used to acquire a visible light face image set and an infrared face image set, and divide the visible light face image set into a visible light face training image set and a visible light face test image set, and the infrared face image set is divided into infrared face image set Face training image set and infrared face test image set.

数据预处理模块，用于对可见光人脸训练图像集、可见光人脸测试图像集、红外人脸训练图像集和红外人脸测试图像集分别进行预处理得到可见光人脸预处理训练图像集、可见光人脸预处理测试图像集、红外人脸预处理训练图像集和红外人脸预处理测试图像集。The data preprocessing module is used to preprocess the visible light face training image set, the visible light face test image set, the infrared face training image set and the infrared face test image set respectively to obtain the visible light face preprocessing training image set, the visible light face test image set Face preprocessing test image set, infrared face preprocessing training image set and infrared face preprocessing test image set.

具体而言，本实施例数据预处理模块中对可见光人脸训练图像集、可见光人脸测试图像集、红外人脸训练图像集和红外人脸测试图像集分别进行预处理得到可见光人脸预处理训练图像集、可见光人脸预处理测试图像集、红外人脸预处理训练图像集和红外人脸预处理测试图像集，包括：Specifically, in the data preprocessing module of this embodiment, the visible light face training image set, the visible light face test image set, the infrared face training image set and the infrared face test image set are respectively preprocessed to obtain the visible light face preprocessing Training image set, visible light face preprocessing test image set, infrared face preprocessing training image set and infrared face preprocessing test image set, including:

可见光人脸训练图像集和可见光人脸测试图像集分别进行灰度转换、归一化处理得到可见光人脸预处理训练图像集和可见光人脸预处理测试图像集；The visible light face training image set and the visible light face test image set are respectively subjected to grayscale conversion and normalization to obtain the visible light face preprocessing training image set and the visible light face preprocessing test image set;

对红外人脸训练图像集和红外人脸测试图像集分别进行图像增强、归一化处理得到红外人脸预处理训练图像集和红外人脸预处理测试图像集。The infrared face training image set and the infrared face test image set are respectively enhanced and normalized to obtain the infrared face preprocessing training image set and the infrared face preprocessing test image set.

第一模型构建训练模块，用于构建超光谱图像融合网络模型，根据可见光人脸预处理训练图像集和红外人脸预处理训练图像集对超光谱图像融合网络模型进行训练得到训练好的超光谱图像融合网络模型。The first model builds a training module, which is used to build a hyperspectral image fusion network model, and trains the hyperspectral image fusion network model according to the visible light face preprocessing training image set and the infrared face preprocessing training image set to obtain a trained hyperspectral image. Image fusion network model.

具体而言，本实施例第一模型构建训练模块中构建的超光谱图像融合网络模型包括预融合层、编码器模块、融合层和解码器模块，其中，Specifically, the hyperspectral image fusion network model constructed in the first model construction training module in this embodiment includes a pre-fusion layer, an encoder module, a fusion layer and a decoder module, wherein,

预融合层的输入与输入图像连接，预融合层的输出与编码器模块的输入连接，编码器模块包括依次连接的第一卷积层和密集残差模块，密集残差模块的输出与预融合层的输出进行全局残差连接输出，全局残差连接输出与融合层的输入连接；The input of the pre-fusion layer is connected with the input image, and the output of the pre-fusion layer is connected with the input of the encoder module. The encoder module includes the first convolutional layer and the dense residual module connected in sequence. The output of the dense residual module is connected with the pre-fusion. The output of the layer is connected to the output of the global residual connection, and the output of the global residual connection is connected to the input of the fusion layer;

融合层的输出与解码器模块的输入连接，解码器模块包括依次连接第二卷积层～第五卷积层和反馈层，反馈层的输出与融合层的输出再次输入至第二卷积层构成反馈连接。The output of the fusion layer is connected to the input of the decoder module. The decoder module includes sequentially connecting the second convolutional layer to the fifth convolutional layer and the feedback layer. The output of the feedback layer and the output of the fusion layer are input to the second convolutional layer again. form a feedback connection.

进一步地，密集残差模块包括依次连接的第一密集残差连接层～第三密集残差连接层、多尺度拼接层、第四密集残差连接层，第一密集残差连接层的输入还与第一密集残差连接层的输出、第二密集残差连接层的输出、第三密集残差连接层的输出连接，第二密集残差连接层的输入还与第二密集残差连接层的输出、第三密集残差连接层的输出连接，第三密集残差连接层的输入还与第三密集残差连接层的输出连接，第四密集残差连接层的输出还与第一卷积层的输入进行局部残差连接输出，局部残差连接输出与预融合层的输出进行全局残差连接输出。Further, the dense residual module includes a first dense residual connection layer to a third dense residual connection layer, a multi-scale splicing layer, and a fourth dense residual connection layer connected in sequence, and the input of the first dense residual connection layer is also It is connected to the output of the first dense residual connection layer, the output of the second dense residual connection layer, the output of the third dense residual connection layer, and the input of the second dense residual connection layer is also connected to the second dense residual connection layer. The output of the third dense residual connection layer is connected with the output of the third dense residual connection layer, the input of the third dense residual connection layer is also connected with the output of the third dense residual connection layer, and the output of the fourth dense residual connection layer is also connected with the first volume. The input of the accumulation layer is connected to the output of the local residual, and the output of the local residual connection and the output of the pre-fusion layer are connected to the output of the global residual.

进一步地，根据可见光人脸预处理训练图像集和红外人脸预处理训练图像集对超光谱图像融合网络模型进行训练得到训练好的超光谱图像融合网络模型，包括：Further, the hyperspectral image fusion network model is trained according to the visible light face preprocessing training image set and the infrared face preprocessing training image set to obtain a trained hyperspectral image fusion network model, including:

构建基于结构性损失、像素损失和平均梯度损失的复合损失函数；Build a composite loss function based on structural loss, pixel loss and average gradient loss;

根据可见光人脸预处理训练图像集和红外人脸预处理训练图像集并利用基于结构性损失、像素损失和平均梯度损失的复合损失函数对超光谱图像融合网络模型进行训练得到训练好的超光谱图像融合网络模型。According to the visible light face preprocessing training image set and the infrared face preprocessing training image set, and using the composite loss function based on structural loss, pixel loss and average gradient loss to train the hyperspectral image fusion network model, the trained hyperspectral image is obtained. Image fusion network model.

数据生成模块，用于将可见光人脸测试图像集和红外人脸测试图像集输入至训练好的超光谱图像融合网络模型得到超光谱人脸图像集，并将超光谱人脸图像集分为超光谱人脸训练图像集、超光谱人脸测试图像集。The data generation module is used to input the visible light face test image set and the infrared face test image set into the trained hyperspectral image fusion network model to obtain the hyperspectral face image set, and divide the hyperspectral face image set into hyperspectral face image sets. Spectral face training image set, hyperspectral face test image set.

第二模型构建训练模块，用于构建卷积神经网络人脸识别模型，根据超光谱人脸训练图像集对卷积神经网络人脸识别模型进行训练得到训练好的卷积神经网络人脸识别模型。The second model builds a training module, which is used to build a convolutional neural network face recognition model, and trains the convolutional neural network face recognition model according to the hyperspectral face training image set to obtain a trained convolutional neural network face recognition model .

具体而言，本实施例第二模型构建训练模块中构建的卷积神经网络人脸识别模型包括依次连接的ResNet特征提取模块、特征归一化模块、特征空间映射模块。Specifically, the convolutional neural network face recognition model constructed in the second model construction training module in this embodiment includes a ResNet feature extraction module, a feature normalization module, and a feature space mapping module connected in sequence.

进一步地，根据超光谱人脸训练图像集对卷积神经网络人脸识别模型进行训练得到训练好的卷积神经网络人脸识别模型，包括：Further, the convolutional neural network face recognition model is trained according to the hyperspectral face training image set to obtain a trained convolutional neural network face recognition model, including:

构建三元组损失函数Triplet loss；Build triplet loss function Triplet loss;

根据超光谱人脸训练图像集并利用三元组损失函数Triplet loss对卷积神经网络人脸识别模型进行训练得到训练好的卷积神经网络人脸识别模型。According to the hyperspectral face training image set and using the triplet loss function Triplet loss to train the convolutional neural network face recognition model, the trained convolutional neural network face recognition model is obtained.

数据识别模块，用于将超光谱人脸测试图像集输入至训练好的卷积神经网络人脸识别模型得到人脸特征集，并使用支持向量机分类器对人脸特征集进行分类处理以实现超光谱人脸识别。The data recognition module is used to input the hyperspectral face test image set into the trained convolutional neural network face recognition model to obtain the face feature set, and use the support vector machine classifier to classify the face feature set to achieve Hyperspectral face recognition.

本实施例提供的一种超光谱人脸识别装置，可以执行上述超光谱人脸识别方法实施例，其实现原理和技术效果类似，在此不再赘述。The hyperspectral face recognition device provided in this embodiment can execute the above embodiments of the hyperspectral face recognition method, and the implementation principles and technical effects thereof are similar, and are not repeated here.

实施例三Embodiment 3

在上述实施例二的基础上，请参见图8，图8是本发明实施例提供的一种超光谱人脸识别电子设备结构示意图。本实施例提供了一种超光谱人脸识别电子设备，该电子设备包括图像采集器、显示器、处理器、通信接口、存储器和通信总线，其中，图像采集器、显示器、处理器、通信接口、存储器通过通信总线完成相互间的通信；On the basis of the second embodiment above, please refer to FIG. 8 , which is a schematic structural diagram of an electronic device for hyperspectral face recognition provided by an embodiment of the present invention. This embodiment provides a hyperspectral face recognition electronic device, the electronic device includes an image collector, a display, a processor, a communication interface, a memory, and a communication bus, wherein the image collector, the display, the processor, the communication interface, The memories communicate with each other through the communication bus;

图像采集器用于采集图像数据；The image collector is used to collect image data;

显示器用于显示图像识别数据；The display is used to display the image recognition data;

存储器，用于存放计算机程序；memory for storing computer programs;

处理器，用于执行存储器上所存放的计算机程序时，该计算机程序被处理器执行时实现以下步骤：When the processor is used to execute the computer program stored in the memory, the computer program implements the following steps when the computer program is executed by the processor:

步骤1、控制图像采集器进行人脸图像采集，获取可见光人脸图像集和红外人脸图像集，并将可见光人脸图像集分为可见光人脸训练图像集和可见光人脸测试图像集，红外人脸图像集分为红外人脸训练图像集和红外人脸测试图像集。Step 1. Control the image collector to collect face images, obtain a visible-light face image set and an infrared face image set, and divide the visible-light face image set into a visible-light face training image set and a visible-light face test image set. The face image set is divided into an infrared face training image set and an infrared face test image set.

步骤2、对可见光人脸训练图像集、可见光人脸测试图像集、红外人脸训练图像集和红外人脸测试图像集分别进行预处理得到可见光人脸预处理训练图像集、可见光人脸预处理测试图像集、红外人脸预处理训练图像集和红外人脸预处理测试图像集。Step 2. Preprocess the visible light face training image set, the visible light face test image set, the infrared face training image set and the infrared face test image set respectively to obtain the visible light face preprocessing training image set and the visible light face preprocessing Test image set, infrared face preprocessing training image set and infrared face preprocessing test image set.

具体而言，本实施例步骤2中对可见光人脸训练图像集、可见光人脸测试图像集、红外人脸训练图像集和红外人脸测试图像集分别进行预处理得到可见光人脸预处理训练图像集、可见光人脸预处理测试图像集、红外人脸预处理训练图像集和红外人脸预处理测试图像集，包括：Specifically, instep 2 of this embodiment, the visible light face training image set, the visible light face test image set, the infrared face training image set, and the infrared face test image set are respectively preprocessed to obtain the visible light face preprocessing training image set, visible light face preprocessing test image set, infrared face preprocessing training image set and infrared face preprocessing test image set, including:

可见光人脸训练图像集和可见光人脸测试图像集分别进行灰度转换、归一化处理得到可见光人脸预处理训练图像集和可见光人脸预处理测试图像集；The visible light face training image set and the visible light face test image set are respectively subjected to grayscale conversion and normalization to obtain the visible light face preprocessing training image set and the visible light face preprocessing test image set;

对红外人脸训练图像集和红外人脸测试图像集分别进行图像增强、归一化处理得到红外人脸预处理训练图像集和红外人脸预处理测试图像集。The infrared face training image set and the infrared face test image set are respectively enhanced and normalized to obtain the infrared face preprocessing training image set and the infrared face preprocessing test image set.

步骤3、构建超光谱图像融合网络模型，根据可见光人脸预处理训练图像集和红外人脸预处理训练图像集对超光谱图像融合网络模型进行训练得到训练好的超光谱图像融合网络模型。Step 3: Build a hyperspectral image fusion network model, and train the hyperspectral image fusion network model according to the visible light face preprocessing training image set and the infrared face preprocessing training image set to obtain a trained hyperspectral image fusion network model.

具体而言，本实施例步骤3中构建的超光谱图像融合网络模型包括预融合层、编码器模块、融合层和解码器模块，其中，Specifically, the hyperspectral image fusion network model constructed instep 3 of this embodiment includes a pre-fusion layer, an encoder module, a fusion layer and a decoder module, wherein,

预融合层的输入与输入图像连接，预融合层的输出与编码器模块的输入连接，编码器模块包括依次连接的第一卷积层和密集残差模块，密集残差模块的输出与预融合层的输出进行全局残差连接输出，全局残差连接输出与融合层的输入连接；The input of the pre-fusion layer is connected with the input image, and the output of the pre-fusion layer is connected with the input of the encoder module. The encoder module includes the first convolutional layer and the dense residual module connected in sequence. The output of the dense residual module is connected with the pre-fusion. The output of the layer is connected to the output of the global residual connection, and the output of the global residual connection is connected to the input of the fusion layer;

融合层的输出与解码器模块的输入连接，解码器模块包括依次连接第二卷积层～第五卷积层和反馈层，反馈层的输出与融合层的输出再次输入至第二卷积层构成反馈连接。The output of the fusion layer is connected to the input of the decoder module. The decoder module includes sequentially connecting the second convolutional layer to the fifth convolutional layer and the feedback layer. The output of the feedback layer and the output of the fusion layer are input to the second convolutional layer again. form a feedback connection.

进一步地，密集残差模块包括依次连接的第一密集残差连接层～第三密集残差连接层、多尺度拼接层、第四密集残差连接层，第一密集残差连接层的输入还与第一密集残差连接层的输出、第二密集残差连接层的输出、第三密集残差连接层的输出连接，第二密集残差连接层的输入还与第二密集残差连接层的输出、第三密集残差连接层的输出连接，第三密集残差连接层的输入还与第三密集残差连接层的输出连接，第四密集残差连接层的输出还与第一卷积层的输入进行局部残差连接输出，局部残差连接输出与预融合层的输出进行全局残差连接输出。Further, the dense residual module includes a first dense residual connection layer to a third dense residual connection layer, a multi-scale splicing layer, and a fourth dense residual connection layer connected in sequence, and the input of the first dense residual connection layer is also It is connected to the output of the first dense residual connection layer, the output of the second dense residual connection layer, the output of the third dense residual connection layer, and the input of the second dense residual connection layer is also connected to the second dense residual connection layer. The output of the third dense residual connection layer is connected with the output of the third dense residual connection layer, the input of the third dense residual connection layer is also connected with the output of the third dense residual connection layer, and the output of the fourth dense residual connection layer is also connected with the first volume. The input of the accumulation layer is connected to the output of the local residual, and the output of the local residual connection and the output of the pre-fusion layer are connected to the output of the global residual.

进一步地，根据可见光人脸预处理训练图像集和红外人脸预处理训练图像集对超光谱图像融合网络模型进行训练得到训练好的超光谱图像融合网络模型，包括：Further, the hyperspectral image fusion network model is trained according to the visible light face preprocessing training image set and the infrared face preprocessing training image set to obtain a trained hyperspectral image fusion network model, including:

构建基于结构性损失、像素损失和平均梯度损失的复合损失函数；Build a composite loss function based on structural loss, pixel loss and average gradient loss;

根据可见光人脸预处理训练图像集和红外人脸预处理训练图像集并利用基于结构性损失、像素损失和平均梯度损失的复合损失函数对超光谱图像融合网络模型进行训练得到训练好的超光谱图像融合网络模型。According to the visible light face preprocessing training image set and the infrared face preprocessing training image set, and using the composite loss function based on structural loss, pixel loss and average gradient loss to train the hyperspectral image fusion network model, the trained hyperspectral image is obtained. Image fusion network model.

步骤4、将可见光人脸测试图像集和红外人脸测试图像集输入至训练好的超光谱图像融合网络模型得到超光谱人脸图像集，并将超光谱人脸图像集分为超光谱人脸训练图像集、超光谱人脸测试图像集。Step 4. Input the visible light face test image set and the infrared face test image set into the trained hyperspectral image fusion network model to obtain a hyperspectral face image set, and divide the hyperspectral face image set into hyperspectral faces Training image set, hyperspectral face test image set.

步骤5、构建卷积神经网络人脸识别模型，根据超光谱人脸训练图像集对卷积神经网络人脸识别模型进行训练得到训练好的卷积神经网络人脸识别模型。Step 5: Construct a convolutional neural network face recognition model, and train the convolutional neural network face recognition model according to the hyperspectral face training image set to obtain a trained convolutional neural network face recognition model.

具体而言，本实施例步骤5中构建的卷积神经网络人脸识别模型包括依次连接的ResNet特征提取模块、特征归一化模块、特征空间映射模块。Specifically, the convolutional neural network face recognition model constructed in step 5 of this embodiment includes a ResNet feature extraction module, a feature normalization module, and a feature space mapping module that are connected in sequence.

进一步地，根据超光谱人脸训练图像集对卷积神经网络人脸识别模型进行训练得到训练好的卷积神经网络人脸识别模型，包括：Further, the convolutional neural network face recognition model is trained according to the hyperspectral face training image set to obtain a trained convolutional neural network face recognition model, including:

构建三元组损失函数Triplet loss；Build triplet loss function Triplet loss;

根据超光谱人脸训练图像集并利用三元组损失函数Triplet loss对卷积神经网络人脸识别模型进行训练得到训练好的卷积神经网络人脸识别模型。According to the hyperspectral face training image set and using the triplet loss function Triplet loss to train the convolutional neural network face recognition model, the trained convolutional neural network face recognition model is obtained.

步骤6、将超光谱人脸测试图像集输入至训练好的卷积神经网络人脸识别模型得到人脸特征集，并使用支持向量机分类器对人脸特征集进行分类处理以实现超光谱人脸识别。最后将超光谱人脸识别结果输出到显示器中。Step 6. Input the hyperspectral face test image set into the trained convolutional neural network face recognition model to obtain the face feature set, and use the support vector machine classifier to classify the face feature set to realize the hyperspectral human face. face recognition. Finally, the hyperspectral face recognition results are output to the display.

本实施例提供的一种超光谱人脸识别电子设备，可以执行上述超光谱人脸识别方法实施例和上述超光谱人脸识别装置实施例，其实现原理和技术效果类似，在此不再赘述。The hyperspectral face recognition electronic device provided in this embodiment can perform the above-mentioned embodiments of the hyperspectral face recognition method and the above-mentioned embodiments of the hyperspectral face recognition apparatus, and the implementation principles and technical effects thereof are similar, and will not be repeated here. .

实施例四Embodiment 4

在上述实施例三的基础上，请参见图9，图9是本发明实施例提供的一种计算机可读存储介质的结构示意图。本实施例提供的一种计算机可读存储介质，其上存储有计算机程序，上述计算机程序被处理器执行时实现以下步骤：On the basis of the third embodiment above, please refer to FIG. 9 , which is a schematic structural diagram of a computer-readable storage medium provided by an embodiment of the present invention. A computer-readable storage medium provided by this embodiment has a computer program stored thereon, and when the computer program is executed by a processor, the following steps are implemented:

步骤1、获取可见光人脸图像集和红外人脸图像集，并将可见光人脸图像集分为可见光人脸训练图像集和可见光人脸测试图像集，红外人脸图像集分为红外人脸训练图像集和红外人脸测试图像集。Step 1. Obtain a visible light face image set and an infrared face image set, and divide the visible light face image set into a visible light face training image set and a visible light face test image set, and the infrared face image set is divided into infrared face training. Image set and infrared face test image set.

步骤2、对可见光人脸训练图像集、可见光人脸测试图像集、红外人脸训练图像集和红外人脸测试图像集分别进行预处理得到可见光人脸预处理训练图像集、可见光人脸预处理测试图像集、红外人脸预处理训练图像集和红外人脸预处理测试图像集。Step 2. Preprocess the visible light face training image set, the visible light face test image set, the infrared face training image set and the infrared face test image set respectively to obtain the visible light face preprocessing training image set and the visible light face preprocessing Test image set, infrared face preprocessing training image set and infrared face preprocessing test image set.

具体而言，本实施例步骤2中对可见光人脸训练图像集、可见光人脸测试图像集、红外人脸训练图像集和红外人脸测试图像集分别进行预处理得到可见光人脸预处理训练图像集、可见光人脸预处理测试图像集、红外人脸预处理训练图像集和红外人脸预处理测试图像集，包括：Specifically, instep 2 of this embodiment, the visible light face training image set, the visible light face test image set, the infrared face training image set, and the infrared face test image set are respectively preprocessed to obtain the visible light face preprocessing training image set, visible light face preprocessing test image set, infrared face preprocessing training image set and infrared face preprocessing test image set, including:

可见光人脸训练图像集和可见光人脸测试图像集分别进行灰度转换、归一化处理得到可见光人脸预处理训练图像集和可见光人脸预处理测试图像集；The visible light face training image set and the visible light face test image set are respectively subjected to grayscale conversion and normalization to obtain the visible light face preprocessing training image set and the visible light face preprocessing test image set;

对红外人脸训练图像集和红外人脸测试图像集分别进行图像增强、归一化处理得到红外人脸预处理训练图像集和红外人脸预处理测试图像集。The infrared face training image set and the infrared face test image set are respectively enhanced and normalized to obtain the infrared face preprocessing training image set and the infrared face preprocessing test image set.

步骤3、构建超光谱图像融合网络模型，根据可见光人脸预处理训练图像集和红外人脸预处理训练图像集对超光谱图像融合网络模型进行训练得到训练好的超光谱图像融合网络模型。Step 3: Build a hyperspectral image fusion network model, and train the hyperspectral image fusion network model according to the visible light face preprocessing training image set and the infrared face preprocessing training image set to obtain a trained hyperspectral image fusion network model.

具体而言，本实施例步骤3中构建的超光谱图像融合网络模型包括预融合层、编码器模块、融合层和解码器模块，其中，Specifically, the hyperspectral image fusion network model constructed instep 3 of this embodiment includes a pre-fusion layer, an encoder module, a fusion layer and a decoder module, wherein,

预融合层的输入与输入图像连接，预融合层的输出与编码器模块的输入连接，编码器模块包括依次连接的第一卷积层和密集残差模块，密集残差模块的输出与预融合层的输出进行全局残差连接输出，全局残差连接输出与融合层的输入连接；The input of the pre-fusion layer is connected with the input image, and the output of the pre-fusion layer is connected with the input of the encoder module. The encoder module includes the first convolutional layer and the dense residual module connected in sequence. The output of the dense residual module is connected with the pre-fusion. The output of the layer is connected to the output of the global residual connection, and the output of the global residual connection is connected to the input of the fusion layer;

融合层的输出与解码器模块的输入连接，解码器模块包括依次连接第二卷积层～第五卷积层和反馈层，反馈层的输出与融合层的输出再次输入至第二卷积层构成反馈连接。The output of the fusion layer is connected to the input of the decoder module. The decoder module includes sequentially connecting the second convolutional layer to the fifth convolutional layer and the feedback layer. The output of the feedback layer and the output of the fusion layer are input to the second convolutional layer again. form a feedback connection.

进一步地，密集残差模块包括依次连接的第一密集残差连接层～第三密集残差连接层、多尺度拼接层、第四密集残差连接层，第一密集残差连接层的输入还与第一密集残差连接层的输出、第二密集残差连接层的输出、第三密集残差连接层的输出连接，第二密集残差连接层的输入还与第二密集残差连接层的输出、第三密集残差连接层的输出连接，第三密集残差连接层的输入还与第三密集残差连接层的输出连接，第四密集残差连接层的输出还与第一卷积层的输入进行局部残差连接输出，局部残差连接输出与预融合层的输出进行全局残差连接输出。Further, the dense residual module includes a first dense residual connection layer to a third dense residual connection layer, a multi-scale splicing layer, and a fourth dense residual connection layer connected in sequence, and the input of the first dense residual connection layer is also It is connected to the output of the first dense residual connection layer, the output of the second dense residual connection layer, the output of the third dense residual connection layer, and the input of the second dense residual connection layer is also connected to the second dense residual connection layer. The output of the third dense residual connection layer is connected with the output of the third dense residual connection layer, the input of the third dense residual connection layer is also connected with the output of the third dense residual connection layer, and the output of the fourth dense residual connection layer is also connected with the first volume. The input of the accumulation layer is connected to the output of the local residual, and the output of the local residual connection and the output of the pre-fusion layer are connected to the output of the global residual.

进一步地，根据可见光人脸预处理训练图像集和红外人脸预处理训练图像集对超光谱图像融合网络模型进行训练得到训练好的超光谱图像融合网络模型，包括：Further, the hyperspectral image fusion network model is trained according to the visible light face preprocessing training image set and the infrared face preprocessing training image set to obtain a trained hyperspectral image fusion network model, including:

构建基于结构性损失、像素损失和平均梯度损失的复合损失函数；Build a composite loss function based on structural loss, pixel loss and average gradient loss;

根据可见光人脸预处理训练图像集和红外人脸预处理训练图像集并利用基于结构性损失、像素损失和平均梯度损失的复合损失函数对超光谱图像融合网络模型进行训练得到训练好的超光谱图像融合网络模型。According to the visible light face preprocessing training image set and the infrared face preprocessing training image set, and using the composite loss function based on structural loss, pixel loss and average gradient loss to train the hyperspectral image fusion network model, the trained hyperspectral image is obtained. Image fusion network model.

步骤4、将可见光人脸测试图像集和红外人脸测试图像集输入至训练好的超光谱图像融合网络模型得到超光谱人脸图像集，并将超光谱人脸图像集分为超光谱人脸训练图像集、超光谱人脸测试图像集。Step 4. Input the visible light face test image set and the infrared face test image set into the trained hyperspectral image fusion network model to obtain a hyperspectral face image set, and divide the hyperspectral face image set into hyperspectral faces Training image set, hyperspectral face test image set.

步骤5、构建卷积神经网络人脸识别模型，根据超光谱人脸训练图像集对卷积神经网络人脸识别模型进行训练得到训练好的卷积神经网络人脸识别模型。Step 5: Construct a convolutional neural network face recognition model, and train the convolutional neural network face recognition model according to the hyperspectral face training image set to obtain a trained convolutional neural network face recognition model.

具体而言，本实施例步骤5中构建的卷积神经网络人脸识别模型包括依次连接的ResNet特征提取模块、特征归一化模块、特征空间映射模块。Specifically, the convolutional neural network face recognition model constructed in step 5 of this embodiment includes a ResNet feature extraction module, a feature normalization module, and a feature space mapping module that are connected in sequence.

进一步地，根据超光谱人脸训练图像集对卷积神经网络人脸识别模型进行训练得到训练好的卷积神经网络人脸识别模型，包括：Further, the convolutional neural network face recognition model is trained according to the hyperspectral face training image set to obtain a trained convolutional neural network face recognition model, including:

构建三元组损失函数Triplet loss；Build triplet loss function Triplet loss;

根据超光谱人脸训练图像集并利用三元组损失函数Triplet loss对卷积神经网络人脸识别模型进行训练得到训练好的卷积神经网络人脸识别模型。According to the hyperspectral face training image set and using the triplet loss function Triplet loss to train the convolutional neural network face recognition model, the trained convolutional neural network face recognition model is obtained.

步骤6、将超光谱人脸测试图像集输入至训练好的卷积神经网络人脸识别模型得到人脸特征集，并使用支持向量机分类器对人脸特征集进行分类处理以实现超光谱人脸识别。Step 6. Input the hyperspectral face test image set into the trained convolutional neural network face recognition model to obtain the face feature set, and use the support vector machine classifier to classify the face feature set to realize the hyperspectral human face. face recognition.

本实施例提供的一种计算机可读存储介质，可以执行上述超光谱人脸识别方法实施例、上述超光谱人脸识别装置实施例和上述超光谱人脸识别电子设备实施例，其实现原理和技术效果类似，在此不再赘述。A computer-readable storage medium provided in this embodiment can execute the above-mentioned embodiments of the hyperspectral face recognition method, the above-mentioned embodiments of the above-mentioned hyperspectral face recognition apparatus, and the above-mentioned embodiments of the electronic device for hyperspectral face recognition, and its implementation principles and The technical effect is similar and will not be repeated here.

以上内容是结合具体的优选实施方式对本发明所作的进一步详细说明，不能认定本发明的具体实施只局限于这些说明。对于本发明所属技术领域的普通技术人员来说，在不脱离本发明构思的前提下，还可以做出若干简单推演或替换，都应当视为属于本发明的保护范围。The above content is a further detailed description of the present invention in combination with specific preferred embodiments, and it cannot be considered that the specific implementation of the present invention is limited to these descriptions. For those of ordinary skill in the technical field of the present invention, without departing from the concept of the present invention, some simple deductions or substitutions can be made, which should be regarded as belonging to the protection scope of the present invention.

Claims (10)

Translated fromChinese

1.一种超光谱人脸识别方法，其特征在于，包括：1. a hyperspectral face recognition method, is characterized in that, comprises:获取可见光人脸图像集和红外人脸图像集，并将所述可见光人脸图像集分为可见光人脸训练图像集和可见光人脸测试图像集，所述红外人脸图像集分为红外人脸训练图像集和红外人脸测试图像集；Acquire a visible light face image set and an infrared face image set, and divide the visible light face image set into a visible light face training image set and a visible light face test image set, and the infrared face image set is divided into an infrared face Training image set and infrared face test image set;对所述可见光人脸训练图像集、可见光人脸测试图像集、所述红外人脸训练图像集和红外人脸测试图像集分别进行预处理得到可见光人脸预处理训练图像集、可见光人脸预处理测试图像集、红外人脸预处理训练图像集和红外人脸预处理测试图像集；The visible light face training image set, the visible light face test image set, the infrared face training image set and the infrared face test image set are respectively preprocessed to obtain the visible light face preprocessing training image set, the visible light face preprocessing image set, and the Process the test image set, the infrared face preprocessing training image set and the infrared face preprocessing test image set;构建超光谱图像融合网络模型，根据所述可见光人脸预处理训练图像集和所述红外人脸预处理训练图像集对所述超光谱图像融合网络模型进行训练得到训练好的超光谱图像融合网络模型；Build a hyperspectral image fusion network model, and train the hyperspectral image fusion network model according to the visible light face preprocessing training image set and the infrared face preprocessing training image set to obtain a trained hyperspectral image fusion network Model;将所述可见光人脸测试图像集和所述红外人脸测试图像集输入至所述训练好的超光谱图像融合网络模型得到超光谱人脸图像集，并将所述超光谱人脸图像集分为超光谱人脸训练图像集、超光谱人脸测试图像集；The visible light face test image set and the infrared face test image set are input into the trained hyperspectral image fusion network model to obtain a hyperspectral face image set, and the hyperspectral face image set is divided into two parts. For the hyperspectral face training image set, the hyperspectral face test image set;构建卷积神经网络人脸识别模型，根据所述超光谱人脸训练图像集对所述卷积神经网络人脸识别模型进行训练得到训练好的卷积神经网络人脸识别模型；Constructing a convolutional neural network face recognition model, and training the convolutional neural network face recognition model according to the hyperspectral face training image set to obtain a trained convolutional neural network face recognition model;将所述超光谱人脸测试图像集输入至所述训练好的卷积神经网络人脸识别模型得到人脸特征集，并使用支持向量机分类器对所述人脸特征集进行分类处理以实现超光谱人脸识别。The hyperspectral face test image set is input into the trained convolutional neural network face recognition model to obtain a face feature set, and the support vector machine classifier is used to classify the face feature set to achieve Hyperspectral face recognition.2.根据权利要求1所述的超光谱人脸识别方法，其特征在于，对所述可见光人脸训练图像集、可见光人脸测试图像集、所述红外人脸训练图像集和红外人脸测试图像集分别进行预处理得到可见光人脸预处理训练图像集、可见光人脸预处理测试图像集、红外人脸预处理训练图像集和红外人脸预处理测试图像集，包括：2. hyperspectral face recognition method according to claim 1, is characterized in that, to described visible light face training image collection, visible light face test image collection, described infrared face training image collection and infrared face test The image sets are preprocessed respectively to obtain a visible light face preprocessing training image set, a visible light face preprocessing test image set, an infrared face preprocessing training image set and an infrared face preprocessing test image set, including:对所述可见光人脸训练图像集和所述可见光人脸测试图像集分别进行灰度转换、归一化处理得到所述可见光人脸预处理训练图像集和所述可见光人脸预处理测试图像集；Performing grayscale conversion and normalization on the visible light face training image set and the visible light face test image set respectively to obtain the visible light face preprocessing training image set and the visible light face preprocessing test image set ;对所述红外人脸训练图像集和所述红外人脸测试图像集分别进行图像增强、归一化处理得到所述红外人脸预处理训练图像集和所述红外人脸预处理测试图像集。Perform image enhancement and normalization on the infrared face training image set and the infrared face test image set respectively to obtain the infrared face preprocessing training image set and the infrared face preprocessing test image set.3.根据权利要求1所述的超光谱人脸识别方法，其特征在于，构建的超光谱图像融合网络模型包括预融合层、编码器模块、融合层和解码器模块，其中，3. hyperspectral face recognition method according to claim 1, is characterized in that, the hyperspectral image fusion network model of construction comprises pre-fusion layer, encoder module, fusion layer and decoder module, wherein,所述预融合层的输入与输入图像连接，所述预融合层的输出与所述编码器模块的输入连接，所述编码器模块包括依次连接的第一卷积层和密集残差模块，所述密集残差模块的输出与所述预融合层的输出进行全局残差连接输出，所述全局残差连接输出与所述融合层的输入连接；The input of the pre-fusion layer is connected to the input image, the output of the pre-fusion layer is connected to the input of the encoder module, and the encoder module includes a first convolution layer and a dense residual module connected in sequence, so The output of the dense residual module and the output of the pre-fusion layer are connected to the output of a global residual connection, and the output of the global residual connection is connected to the input of the fusion layer;所述融合层的输出与所述解码器模块的输入连接，所述解码器模块包括依次连接第二卷积层～第五卷积层和反馈层，所述反馈层的输出与所述融合层的输出再次输入至所述第二卷积层构成反馈连接。The output of the fusion layer is connected to the input of the decoder module, and the decoder module includes sequentially connecting the second convolutional layer to the fifth convolutional layer and a feedback layer, and the output of the feedback layer is connected to the fusion layer. The output is input again to the second convolutional layer to form a feedback connection.4.根据权利要求3所述的超光谱人脸识别方法，其特征在于，所述密集残差模块包括依次连接的第一密集残差连接层～第三密集残差连接层、多尺度拼接层、第四密集残差连接层，所述第一密集残差连接层的输入还与所述第一密集残差连接层的输出、所述第二密集残差连接层的输出、所述第三密集残差连接层的输出连接，所述第二密集残差连接层的输入还与所述第二密集残差连接层的输出、所述第三密集残差连接层的输出连接，所述第三密集残差连接层的输入还与所述第三密集残差连接层的输出连接，所述第四密集残差连接层的输出还与所述第一卷积层的输入进行局部残差连接输出，所述局部残差连接输出与所述预融合层的输出进行全局残差连接输出。4 . The hyperspectral face recognition method according to claim 3 , wherein the dense residual module comprises a first dense residual connection layer to a third dense residual connection layer and a multi-scale splicing layer connected in sequence. 5 . , the fourth dense residual connection layer, the input of the first dense residual connection layer is also connected with the output of the first dense residual connection layer, the output of the second dense residual connection layer, the third The output of the dense residual connection layer is connected, and the input of the second dense residual connection layer is also connected with the output of the second dense residual connection layer and the output of the third dense residual connection layer. The input of the third dense residual connection layer is also connected with the output of the third dense residual connection layer, and the output of the fourth dense residual connection layer is also connected locally with the input of the first convolutional layer. Output, the local residual connection output and the output of the pre-fusion layer perform global residual connection output.5.根据权利要求1所述的超光谱人脸识别方法，其特征在于，根据所述可见光人脸预处理训练图像集和所述红外人脸预处理训练图像集对所述超光谱图像融合网络模型进行训练得到训练好的超光谱图像融合网络模型，包括：5. hyperspectral face recognition method according to claim 1, is characterized in that, according to described visible light face preprocessing training image set and described infrared face preprocessing training image set to described hyperspectral image fusion network The model is trained to obtain a trained hyperspectral image fusion network model, including:构建基于结构性损失、像素损失和平均梯度损失的复合损失函数；Build a composite loss function based on structural loss, pixel loss and average gradient loss;根据所述可见光人脸预处理训练图像集和所述红外人脸预处理训练图像集并利用所述基于结构性损失、像素损失和平均梯度损失的复合损失函数对所述超光谱图像融合网络模型进行训练得到训练好的超光谱图像融合网络模型。According to the visible light face preprocessing training image set and the infrared face preprocessing training image set and using the composite loss function based on structural loss, pixel loss and average gradient loss, the hyperspectral image fusion network model After training, the trained hyperspectral image fusion network model is obtained.6.根据权利要求1所述的超光谱人脸识别方法，其特征在于，构建的卷积神经网络人脸识别模型包括依次连接的ResNet特征提取模块、特征归一化模块、特征空间映射模块。6. The hyperspectral face recognition method according to claim 1, wherein the constructed convolutional neural network face recognition model comprises a ResNet feature extraction module, a feature normalization module, and a feature space mapping module connected in sequence.7.根据权利要求1所述的超光谱人脸识别方法，其特征在于，根据所述超光谱人脸训练图像集对所述卷积神经网络人脸识别模型进行训练得到训练好的卷积神经网络人脸识别模型，包括：7. hyperspectral face recognition method according to claim 1, is characterized in that, according to described hyperspectral face training image set, described convolutional neural network face recognition model is trained to obtain trained convolutional neural Network face recognition model, including:构建三元组损失函数Tripletloss；Build the triplet loss function Tripletloss;根据超光谱人脸训练图像集并利用所述三元组损失函数Triplet loss对所述卷积神经网络人脸识别模型进行训练得到训练好的卷积神经网络人脸识别模型。According to the hyperspectral face training image set and using the triplet loss function Triplet loss, the convolutional neural network face recognition model is trained to obtain a trained convolutional neural network face recognition model.8.一种超光谱人脸识别装置，其特征在于，所述装置包括：8. A hyperspectral face recognition device, wherein the device comprises:数据获取模块，用于获取可见光人脸图像集和红外人脸图像集，并将所述可见光人脸图像集分为可见光人脸训练图像集和可见光人脸测试图像集，所述红外人脸图像集分为红外人脸训练图像集和红外人脸测试图像集；A data acquisition module is used to acquire a visible light face image set and an infrared face image set, and divide the visible light face image set into a visible light face training image set and a visible light face test image set, and the infrared face image set The sets are divided into infrared face training image set and infrared face test image set;数据预处理模块，用于对所述可见光人脸训练图像集、可见光人脸测试图像集、所述红外人脸训练图像集和红外人脸测试图像集分别进行预处理得到可见光人脸预处理训练图像集、可见光人脸预处理测试图像集、红外人脸预处理训练图像集和红外人脸预处理测试图像集；The data preprocessing module is used to preprocess the visible light face training image set, the visible light face test image set, the infrared face training image set and the infrared face test image set respectively to obtain the visible light face preprocessing training Image set, visible light face preprocessing test image set, infrared face preprocessing training image set and infrared face preprocessing test image set;第一模型构建训练模块，用于构建超光谱图像融合网络模型，根据所述可见光人脸预处理训练图像集和所述红外人脸预处理训练图像集对所述超光谱图像融合网络模型进行训练得到训练好的超光谱图像融合网络模型；The first model construction training module is used to construct a hyperspectral image fusion network model, and the hyperspectral image fusion network model is trained according to the visible light face preprocessing training image set and the infrared face preprocessing training image set Obtain the trained hyperspectral image fusion network model;数据生成模块，用于将所述可见光人脸测试图像集和所述红外人脸测试图像集输入至所述训练好的超光谱图像融合网络模型得到超光谱人脸图像集，并将所述超光谱人脸图像集分为超光谱人脸训练图像集、超光谱人脸测试图像集；The data generation module is used to input the visible light face test image set and the infrared face test image set into the trained hyperspectral image fusion network model to obtain a hyperspectral face image set, and use the hyperspectral face image set. The spectral face image set is divided into a hyperspectral face training image set and a hyperspectral face testing image set;第二模型构建训练模块，用于构建卷积神经网络人脸识别模型，根据所述超光谱人脸训练图像集对所述卷积神经网络人脸识别模型进行训练得到训练好的卷积神经网络人脸识别模型；The second model builds a training module, which is used to build a convolutional neural network face recognition model, and trains the convolutional neural network face recognition model according to the hyperspectral face training image set to obtain a trained convolutional neural network. face recognition model;数据识别模块，用于将所述超光谱人脸测试图像集输入至所述训练好的卷积神经网络人脸识别模型得到人脸特征集，并使用支持向量机分类器对所述人脸特征集进行分类处理以实现超光谱人脸识别。A data recognition module is used to input the hyperspectral face test image set into the trained convolutional neural network face recognition model to obtain a face feature set, and use a support vector machine classifier to analyze the face features. Sets for classification processing to achieve hyperspectral face recognition.9.一种超光谱人脸识别电子设备，其特征在于，所述该电子设备包括图像采集器、显示器、处理器、通信接口、存储器和通信总线，其中，所述图像采集器、所述显示器、所述处理器、所述通信接口、所述存储器通过所述通信总线完成相互间的通信；9. A hyperspectral face recognition electronic device, wherein the electronic device comprises an image collector, a display, a processor, a communication interface, a memory and a communication bus, wherein the image collector, the display , the processor, the communication interface, and the memory communicate with each other through the communication bus;所述图像采集器用于采集图像数据；The image collector is used to collect image data;所述显示器用于显示图像识别数据；the display is used for displaying image recognition data;所述存储器，用于存放计算机程序；the memory for storing computer programs;所述处理器，用于执行所述存储器上存放的所述计算机程序时，实现权利要求1～7任一所述的超光谱人脸识别方法。The processor is configured to implement the hyperspectral face recognition method according to any one of claims 1 to 7 when executing the computer program stored in the memory.10.一种计算机可读存储介质，其特征在于，所述计算机可读存储介质内存储有计算机程序，所述计算机程序被处理器执行时实现权利要求1～7任一所述的超光谱人脸识别方法。10. A computer-readable storage medium, wherein a computer program is stored in the computer-readable storage medium, and when the computer program is executed by a processor, the hyperspectral human according to any one of claims 1 to 7 is implemented. face recognition method.

Images