技术领域technical field
本发明属于图像处理技术领域，具体涉及毫米波图像的隐私保护方法。The invention belongs to the technical field of image processing, and in particular relates to a privacy protection method for millimeter wave images.
背景技术Background technique
毫米波是一种波长为1mm～10mm，频率为30～300GHz的电磁波。由于毫米波可以有效地穿透普通衣物等障碍物，并且具有较高的成像分辨率，同时对人体是无危害的，因此，现如今，毫米波成像系统已经广泛地应用在安检、安防等领域，例如，城市地铁安检点，机场安检点，人群密集区域等。毫米波成像系统按照工作方式可以分为被动式成像(PMMW)和主动式成像(AMMW)。相比被动式成像方法，主动式成像即可以实现二维、三维成像，也具有较高的成像分辨率，可以配合自动目标检测算法，利用人工智能技术准确地识别隐藏在人体中的违禁携带物体。但是，由于较高的成像分辨率，主动式成像设备的成像结果会有很明显的人体轮廓特征以及人体性别特征，很明显地暴露受检人员的隐私，如图1。因此，需要利用图像处理技术来完成毫米波图像的个人隐私保护功能。Millimeter wave is an electromagnetic wave with a wavelength of 1 mm to 10 mm and a frequency of 30 to 300 GHz. Because millimeter waves can effectively penetrate obstacles such as ordinary clothing, have high imaging resolution, and are harmless to the human body, today, millimeter wave imaging systems have been widely used in security inspection, security and other fields. , for example, city subway security checkpoints, airport security checkpoints, crowded areas, etc. Millimeter wave imaging systems can be divided into passive imaging (PMMW) and active imaging (AMMW) according to their working methods. Compared with passive imaging methods, active imaging can achieve two-dimensional and three-dimensional imaging, and also has higher imaging resolution. It can cooperate with automatic target detection algorithms and use artificial intelligence technology to accurately identify prohibited objects hidden in the human body. However, due to the high imaging resolution, the imaging results of the active imaging device will have obvious human contour characteristics and human gender characteristics, which obviously exposes the privacy of the inspected personnel, as shown in Figure 1. Therefore, it is necessary to use image processing technology to complete the personal privacy protection function of millimeter wave images.
在国内，北京无线电计量测试研究所利用去隐私部位的图像处理技术[1]，中科院上海微系统所利用人体隐私部位快速识别定位的技术[2]，来完成对毫米波人体隐私部位的遮挡处理。In China, Beijing Radio Metrology and Testing Institute uses image processing technology to remove private parts [1], and Shanghai Institute of Microsystems, Chinese Academy of Sciences uses the technology of rapid identification and positioning of human privacy parts [2] to complete the occlusion processing of millimeter wave human privacy parts .
在国际上，美国L3公司所研发的Provision产品主要针对机场安检，将人体成像结果映射到一张卡通图中，作为原始毫米波人体图像的替代图，机场辅助安检人员可以看到的仅仅是人体成像结果的替代图，保护了受检人员的隐私。Internationally, the Provision product developed by the American L3 company is mainly aimed at airport security inspection. It maps the results of human body imaging into a cartoon image. As an alternative to the original millimeter-wave human body image, the airport auxiliary security personnel can only see the human body. Alternate plots of imaging results, protecting the privacy of the subject.
不论在国内外，对毫米波成像结果的隐私保护技术一直普遍受到关注。一般所采用的算法流程包括：1)使用图像处理技术确定毫米波图像中人体的位置；2)对图像中人体进行进一步区域划分，从而确定面部、胸部和裆部等隐私部位。然而随着主动毫米波安检仪器的广泛应用，受检人员的体型分布、身高分布扩大，受检人员接受安全检查时的站立姿势并非完全按照受检说明执行，因此导致按照传统的图像处理技术来获取毫米波图像中人体位置、人体轮廓信息的难度增大，最后导致对隐私部位的坐标定位出现较大偏差，对人体违禁物品的坐标映射发生偏差，如图2第二行所示。因此，利用大规模毫米波人体图像，快速、有效地定位人体位置，恢复人体的轮廓信息不仅仅会提升违禁物品检测的精度，而且会提升毫米波图像人体隐私保护算法的精度。Whether at home or abroad, privacy-preserving technologies for millimeter-wave imaging results have always received widespread attention. The general algorithm flow includes: 1) using image processing technology to determine the position of the human body in the millimeter wave image; 2) further dividing the human body in the image to determine the face, chest and crotch and other private parts. However, with the wide application of active millimeter-wave security inspection instruments, the distribution of body size and height of the inspected personnel has expanded, and the standing posture of the inspected personnel during the security inspection is not completely performed in accordance with the inspection instructions, which leads to the traditional image processing technology. It is more difficult to obtain the human body position and contour information in the millimeter wave image, which finally leads to a large deviation in the coordinate positioning of the private parts, and deviation in the coordinate mapping of the prohibited items of the human body, as shown in the second row of Figure 2. Therefore, using large-scale millimeter-wave human body images to quickly and effectively locate the position of the human body and recover the contour information of the human body will not only improve the accuracy of prohibited item detection, but also improve the accuracy of the millimeter-wave image human privacy protection algorithm.
传统对毫米波人体图像的隐私保护算法[1][2][3]通过平滑滤波，阈值选取和形态学处理的方式得到人体轮廓信息。在获得人体轮廓信息后，[1][2]需要设计几何式的手工特征，确定人体隐私部位相对于人体轮廓的几何位置。但若是人体的站姿发生变化，导致手部区域距离面部区域较近时；或者是成像结果由于外界条件的原因发生改变，导致手部区域的成像结果与面部区域相似时，则会导致对面部隐私部位的误判，如图2.B第二行所示。The traditional privacy protection algorithm for millimeter-wave human body images [1][2][3] obtains human body contour information through smooth filtering, threshold selection and morphological processing. After obtaining the human body contour information, [1][2] need to design geometric manual features to determine the geometric position of the human body's private parts relative to the human body contour. However, if the standing posture of the human body changes, causing the hand area to be closer to the face area; or if the imaging result changes due to external conditions, so that the imaging result of the hand area is similar to the face area, it will lead to The misjudgment of private parts is shown in the second row of Figure 2.B.
下面介绍一些有关毫米波人体成像的隐私保护算法:Here are some privacy-preserving algorithms for millimeter-wave body imaging:
1、常规隐私保护算法1. Conventional privacy protection algorithm
基于传统图像处理的隐私保护算法采用横向分割隐私保护算法[1]。具体地，通过平滑滤波、阈值选取、形态学操作得到人体轮廓图；获得人体轮廓图后需要对人体进行身高判断，基于人体轮廓的身高特征，进行面部定位、裆部定位等。The privacy protection algorithm based on traditional image processing adopts the horizontal segmentation privacy protection algorithm [1]. Specifically, the human body contour map is obtained through smoothing filtering, threshold selection, and morphological operations; after obtaining the human body contour map, the height of the human body needs to be judged, and based on the height features of the human body contour, face positioning, crotch positioning, etc. are performed.
1.1获取人体轮廓图1.1 Obtaining the outline of the human body
首先，增加毫米波图像中人体的边缘细节：First, increase the edge detail of the human body in the mmWave image:
g(x,y)＝f(x,y)+A×h(x,y) (1)g(x,y)=f(x,y)+A×h(x,y) (1)
公式(1)中，f(x,y)是原始毫米波图像，h(x,y)是原始毫米波图像经过高通滤波后的结果，常数A是锐化惩罚因此，控制锐化程度，g(x,y)是图像锐化的输出结果。In formula (1), f(x, y) is the original millimeter-wave image, h(x, y) is the result of the high-pass filtering of the original millimeter-wave image, and the constant A is the sharpening penalty. Therefore, to control the degree of sharpening, g (x,y) is the output result of image sharpening.
经过锐化处理后g(x,y)已是边缘较清晰的图像。通过公式(2)，设定阈值。阈值利用最大类间方差来选择[4]，此时人体轮廓与背景的差别最大，二值化的效果最好。After sharpening, g(x, y) is an image with sharper edges. By formula (2), the threshold value is set. The threshold is selected by using the largest inter-class variance [4]. At this time, the difference between the human silhouette and the background is the largest, and the effect of binarization is the best.
T(x,y)代表经过阈值处理后的图像。然后将T(x,y)进行形态学闭运算与腐蚀膨胀算法处理，来填补人体轮廓的空缺部分。T(x,y) represents the thresholded image. Then, T(x,y) is processed by morphological closing operation and erosion expansion algorithm to fill in the vacant part of the human body contour.
1.2人体身高判断1.2 Judgment of human height
为了防止由于不同身高的受检人导致的纵向偏差，常规隐私保护算法需要对人体轮廓图像进行人体身高判断。该方法主要通过从图像纵向方向遍历像素值，来按照人体比例模型求出人体身高。但是此方法不能适用于身材变化较大的受检人，且此方法收到T(x,y)结果的影响较大。In order to prevent longitudinal deviations caused by subjects of different heights, conventional privacy-preserving algorithms need to perform human height judgment on human silhouette images. This method mainly traverses the pixel values from the longitudinal direction of the image to find the height of the human body according to the human scale model. However, this method cannot be applied to subjects with large body changes, and this method is greatly affected by the results of T(x,y).
1.3隐私部位定位1.3 Location of private parts
对T(x,y)进行区域划分。利用受检人员的站姿的先验知识来设计T(x,y)的区域。但是此方法严重收到受检人员的站姿以及成像质量的影响。Divide T(x,y) into regions. The region of T(x,y) is designed using the prior knowledge of the subject's stance. However, this method is seriously affected by the standing posture of the examinee and the imaging quality.
2、基于深度学习的隐私保护算法2. Privacy protection algorithm based on deep learning
本发明提出基于深度学习的隐私保护算法。其依靠采用一阶段目标检测模型来检测到人体的不同区域。基于深度学习的检测模型可以较大程度地提升对人体区域检测的精度，从而可以有效识别出人体的隐私区域，并且有效地完成携带物的坐标投影。具体算法流程参见发明内容部分，本节只介绍基于深度学习的目标检测算法。The present invention proposes a privacy protection algorithm based on deep learning. It relies on employing a one-stage object detection model to detect different regions of the human body. The detection model based on deep learning can greatly improve the detection accuracy of the human body area, so that the privacy area of the human body can be effectively identified, and the coordinate projection of the carried objects can be effectively completed. For the specific algorithm flow, please refer to the content of the invention. This section only introduces the target detection algorithm based on deep learning.
2.1深度学习检测模型2.1 Deep Learning Detection Model
一阶段(one-stage，或者称作one-shot)目标检测模型是指提取候选框和使用候选框来预测地面真实(Ground Truth)在一个阶段完成，通常是端到端的深度学习模型架构。One-stage (or one-shot) object detection model refers to extracting candidate boxes and using candidate boxes to predict ground truth in one stage, usually an end-to-end deep learning model architecture.
本发明利用SSD[9]模型来获取人体区域，其中，关于候选框的概念参考文献[9]。在从候选框中合理地挑选出了一定比例的正负样本之后，训练的代价函数如下：The present invention utilizes the SSD[9] model to obtain the human body region, wherein the concept of the candidate frame is referred to [9]. After a certain proportion of positive and negative samples are reasonably selected from the candidate box, the training cost function is as follows:
其中，N是挑选出的正样本的个数。Lcls(I,C)表示类别预测，Lloc(I,P,G))表示位置回归预测，α表示惩罚因子，C是训练集中的类别个数，I是示性项，当且仅当第i个候选框和第j个Ground Truth匹配时，I＝1。Among them, N is the number of selected positive samples. Lcls (I, C) represents the category prediction, Lloc (I, P, G)) represents the location regression prediction, α represents the penalty factor, C is the number of categories in the training set, I is the indicative term, I=1 if and only if the ith candidate box matches the jth Ground Truth.
回归项如公式(4)，和分别表示第i个候选框和第j个GroundTruth的中心点坐标，和分别表示第i个候选框和第j个Ground Truth的宽和高。是候选框发生的相对偏移。是对第i个候选框发生偏移的回归预测。The regression term is as in formula (4), and Represent the coordinates of the center point of the ith candidate frame and the jth GroundTruth, respectively, and Represent the width and height of the ith candidate box and the jth Ground Truth, respectively. is the relative offset of the candidate box. is the regression prediction for the offset of the i-th candidate box.
类别预测项如公式(6)，是第i个候选框关于第k类的预测概率，是第i类候选框关于背景的预测概率。The category predictor is as formula (6), is the predicted probability of the i-th candidate box about the k-th class, is the predicted probability of the i-th candidate box about the background.
发明内容SUMMARY OF THE INVENTION
本发明的目的在于提供一种速度快、正确率高的毫米波图像的隐私保护方法。The purpose of the present invention is to provide a privacy protection method for millimeter wave images with high speed and high accuracy.
本发明提出的毫米波图像的隐私保护方法，是基于深度卷积神经网络技术的，即采用深度卷积神经网络人体局部区域识别架构(记为DHF)，如图3所示。首先，利用目标检测算法预测人体局部区域，而非全局区域，将人体分割为10个区域；其次，采用最近邻法将人体携带物(图中正方形包围盒标记处)与其中的最近邻人体区域绑定，然后在该人体区域内部进行投影操作，将毫米波原图中的人体携带物标记框投影到卡通图像人体局部区域的对应位置处；最后，利用重构算法还原该卡通图像以及其人体携带物标记框。The privacy protection method of the millimeter wave image proposed by the present invention is based on the deep convolutional neural network technology, that is, the deep convolutional neural network human body local area recognition architecture (denoted as DHF) is used, as shown in FIG. 3 . First, the target detection algorithm is used to predict the local area of the human body instead of the global area, and the human body is divided into 10 areas; secondly, the nearest neighbor method is used to compare the objects carried by the human body (marked by the square bounding box in the figure) with the nearest neighbor human body area. Binding, and then perform a projection operation inside the human body area, and project the human body carrying object marker frame in the original millimeter wave image to the corresponding position of the cartoon image human body part area; finally, use the reconstruction algorithm to restore the cartoon image and its human body. Carry tag box.
本发明利用识别人体的10个区域来完成毫米波人体图像的隐私保护。可通过胸部区域的中心点坐标确定人体脸部坐标，通过裆部区域的中心点确定人体裆部坐标，通过投影和重构算法来完成人体携带物体从原始毫米波图像至卡通图像的坐标变换操作。The present invention completes the privacy protection of the millimeter wave human body image by identifying 10 regions of the human body. The coordinates of the human face can be determined by the coordinates of the center point of the chest area, the coordinates of the crotch of the human body can be determined by the center point of the crotch area, and the coordinate transformation operation of the objects carried by the human body from the original millimeter wave image to the cartoon image can be completed through the projection and reconstruction algorithm. .
具体包括以下几个方面：Specifically, it includes the following aspects:
(1)采用目标检测模型SSD[9]，来检测人体区域，本发明将人体分为10个区域，如图4所示，分别为：1、左小臂，2、右小臂，3、左大臂，4、右大臂，5、胸部，6、裆部，7、左小腿，8、右小腿，9、左大腿，10、右大腿；(1) The target detection model SSD [9] is used to detect the human body area. The present invention divides the human body into 10 areas, as shown in Figure 4, which are: 1. Left forearm, 2. Right forearm, 3. Left upper arm, 4, right upper arm, 5, chest, 6, crotch, 7, left calf, 8, right calf, 9, left thigh, 10, right thigh;
(2)利用SSD检测到的人体10个区域来设置隐私遮挡，如图2.B第一行所示，获取到人体胸部区域后，胸部区域的中心点的横坐标便是面部区域的横坐标，面部区域的纵坐标采用标准人体比例因子计算得到；裆部区域中心点坐标直接即可获得；(2) Use the 10 regions of the human body detected by SSD to set privacy masking, as shown in the first row of Figure 2.B, after obtaining the human chest region, the abscissa of the center point of the chest region is the abscissa of the face region , the ordinate of the face area is calculated by the standard human scale factor; the coordinates of the center point of the crotch area can be obtained directly;
(3)将毫米波原图中的违禁危险物体标记框通过DHF架构投影到卡通图像对应位置处，如图2.C第一行所示。(3) Project the prohibited dangerous object marking frame in the original millimeter wave image to the corresponding position of the cartoon image through the DHF structure, as shown in the first row of Figure 2.C.
本发明采用SSD模型准确获得人体区域的坐标信息，并利用这些信息检测到人体隐私部位，同时利用人体区域信息投影违禁物体标记框。准确率、时效性都超过了传统人体图像隐私保护算法。The invention adopts the SSD model to accurately obtain the coordinate information of the human body area, uses the information to detect the private parts of the human body, and uses the human body area information to project the prohibited object marking frame. The accuracy and timeliness have surpassed traditional human image privacy protection algorithms.
本发明提供的基于卷积神经网络的毫米波图像人体隐私保护方法，包括构建网络结构的方法，预测人体隐私部位的方法，投影人体携带物的方法，以及训练以及测试方案(见具体实施方式)，具体步骤如下：The convolutional neural network-based millimeter-wave image human privacy protection method provided by the present invention includes a method for constructing a network structure, a method for predicting the private parts of a human body, a method for projecting objects carried by the human body, and a training and testing scheme (see specific embodiments) ,Specific steps are as follows:
步骤1、检测人体区域：构建人体结构数据集。Step 1. Detect human body regions: build a human body structure data set.
1.1：划分人体。由于毫米波安检仪要求受检人的站姿是手向上举过头顶，双肩与颈部水平，双脚分开的距离与肩同宽。因此，本发明将人体分割为10个区域，分别是(基于图像从左至右的法则)：1代表左小臂，2代表右小臂，3代表左大臂，4代表右大臂，5代表胸部或背部(背面)，6代表裆部或臀部(背面)，7代表左小腿，8代表右小腿，9代表左大腿，10代表右大腿。如图4所示。1.1: Divide the human body. Because the millimeter-wave security inspection instrument requires the subject to stand with his hands raised above his head, his shoulders and neck are level, and his feet are shoulder-width apart. Therefore, the present invention divides the human body into 10 regions, which are (based on the rule of image from left to right): 1 represents the left forearm, 2 represents the right forearm, 3 represents the left upper arm, 4 represents the right upper arm, 5 Represents the chest or back (back), 6 represents the crotch or butt (back), 7 represents the left calf, 8 represents the right calf, 9 represents the left thigh, and 10 represents the right thigh. As shown in Figure 4.
1.2：对数据集进行标注。本发明选择来自不同地区、不同身高、不同体型的受检人员的毫米波安检仪的扫描结果作为数据集。共计5788张扫描图片，其中2894张正面扫描结果，2894张背面扫描结果。标注方式按照步骤1.1的划分方式进行。1.2: Label the dataset. The present invention selects the scanning results of the millimeter-wave security detectors from the inspected persons of different regions, different heights and different body types as the data set. There are a total of 5788 scanned images, including 2894 front scan results and 2894 back scan results. The labeling method is carried out according to the division method of step 1.1.
步骤2、检测人体区域：检测模型的设计。Step 2. Detect the human body area: the design of the detection model.
2.1：聚类前景目标的面积分布。对步骤1.2的标注结果进行统计，得出前景目标的区域面积的分布范围，如图5所示。采用K-means算法[16]，取K-means算法的聚类种类K＝3，来获得初始化候选框的规模因子smin和smax(参见公式(7))。本发明实施例中将smin设置为0.2，smax设置为0.5。2.1: Area distribution of clustered foreground objects. Statistics are performed on the labeling results of step 1.2 to obtain the distribution range of the area of the foreground target, as shown in Figure 5. The K-means algorithm [16] is used, and the clustering type K=3 of the K-means algorithm is used to obtain the scale factors smin and smax of the initialized candidate frame (see formula (7)). In the embodiment of the present invention, smin is set to 0.2, and smax is set to 0.5.
2.2：对毫米波图像进行下采样操作。本发明采用VGG模型[10]来获得毫米波图像的抽象特征。如图3所示，采用fc7、conv6_2、conv7_2这三个层级的特征图来预测人体的十个部位。其中，fc7、conv6_2、conv7_2分别对原始毫米波图像下采样16倍，32倍，64倍。2.2: Downsampling the millimeter wave image. The present invention adopts the VGG model [10] to obtain the abstract features of the millimeter wave image. As shown in Figure 3, the feature maps of three levels of fc7, conv6_2, and conv7_2 are used to predict ten parts of the human body. Among them, fc7, conv6_2, and conv7_2 downsample the original millimeter wave image by 16 times, 32 times, and 64 times, respectively.
2.3：初始化候选框。基于步骤2.2选出的fc7、conv6_2、conv7_2这三个层级特征图在原图中初始化候选框。这三个层级特征图中的第i个特征点，分别在原始图像中初始化第i个候选框cx是中心点坐标横坐标，cy是中心点纵坐标，w是候选框的宽，h是候选框的高。候选框的初始化方法按照公式(7)-公式(9)。2.3: Initialize the candidate frame. Based on the three-level feature maps of fc7, conv6_2, and conv7_2 selected in step 2.2, the candidate frame is initialized in the original image. The i-th feature point in the three-level feature maps, respectively, initializes the i-th candidate frame in the original image cx is the abscissa of the center point coordinate, cy is the ordinate of the center point, w is the width of the candidate frame, and h is the height of the candidate frame. The initialization method of the candidate frame is according to formula (7)-formula (9).
其中，sk∈{fc7，conv6_2，conv7_2}，表示的含义是参与预测人体区域的候选框的比例因子(针对毫米波图像的宽高比例)；n表示参与预测人体区域的层级特征图的个数，本发明选用fc7，conv6_2，conv7_2这三层参与预测，n＝3；rj代表不同宽高比的集合。W代表毫米波图像的宽度，H代表毫米波图像的高度。Among them, sk ∈ {fc7, conv6_2, conv7_2}, which means the scale factor of the candidate frame involved in predicting the human body region (for the aspect ratio of millimeter wave images); number, the present invention selects three layers of fc7, conv6_2, and conv7_2 to participate in prediction, n=3; rj represents a set of different aspect ratios. W represents the width of the millimeter-wave image, and H represents the height of the millimeter-wave image.
2.4：针对候选框，进一步选择出可供训练的正负样本。步骤2.3对fc7、conv6_2、conv7_2这三层特征图中的每一个特征点，都在原图中产生了候选框。此时按照候选框与地面真实(Ground Truth)的重合度挑选正负样本[9]。若重合度大于阈值θ，则为正样本候选框，反之为负样本候选框。通过OHEM[15]算法来挑选出难以学习的负样本候选框，保持正负样本比例均衡。本发明实施例中，设置阈值θ为0.5。2.4: For the candidate frame, further select positive and negative samples for training. Step 2.3 For each feature point in the three-layer feature map of fc7, conv6_2, and conv7_2, a candidate frame is generated in the original image. At this time, positive and negative samples are selected according to the coincidence of the candidate frame and the ground truth [9]. If the degree of coincidence is greater than the threshold θ, it is a positive sample candidate frame, otherwise it is a negative sample candidate frame. The OHEM [15] algorithm is used to select the difficult-to-learn negative sample candidate boxes to keep the proportion of positive and negative samples balanced. In this embodiment of the present invention, the threshold θ is set to be 0.5.
步骤3、检测人体区域：训练检测器，并且预测人体区域。Step 3. Detect the human body region: train the detector and predict the human body region.
3.1：通过步骤1.2得到有监督的训练样本5788张。为了验证模型训练的性能，随机选择3859张图片(包括正面和背面)作为训练样本，选择1929张图片作为验证样本。在具体实施方案中会介绍模型的实验结果。针对每一张训练样本，步骤2.4都会合理地选择出一定比例的正样本和负样本，后采用公式(3)训练检测器：3.1: Obtain 5788 supervised training samples through step 1.2. To verify the performance of model training, 3859 images (including front and back) were randomly selected as training samples, and 1929 images were selected as validation samples. Experimental results of the model are presented in the specific embodiments. For each training sample, step 2.4 will reasonably select a certain proportion of positive samples and negative samples, and then use formula (3) to train the detector:
其中，N是挑选出的正样本的个数；Lcls(I,C)表示类别预测，Lloc(I,P,G))表示位置回归预测，α表示惩罚因子，C是训练集中的类别个数，I是示性项，当且仅当第i个候选框和第j个Ground Truth匹配时，I＝1。Among them, N is the number of positive samples selected; Lcls (I, C) represents the category prediction, Lloc (I, P, G)) represents the location regression prediction, α represents the penalty factor, and C is the category in the training set number, I is an indicative term, I=1 if and only if the ith candidate box matches the jth Ground Truth.
3.2：在有监督的人体区域数据集中完成训练后，将检测器投入实际应用场景，即DHF架构中。如图3，DHF架构针对每一张测试图片，识别出人体的十个区域的类别以及区域中心点坐标。3.2: After completing the training on the supervised human body region dataset, put the detector into the real application scenario, i.e. the DHF architecture. As shown in Figure 3, the DHF architecture identifies the categories of ten regions of the human body and the coordinates of the regional center points for each test image.
步骤4、遮挡人体隐私部位：对步骤3得到的人体区域进一步处理，遮挡人体隐私部位。Step 4. Blocking the private parts of the human body: The human body area obtained in step 3 is further processed to block the private parts of the human body.
4.1：如图3，在步骤3.2检测到人体的十个区域后，分别对特定的区域添加遮挡。其中，对6号区域裆部或臀部添加遮挡。通过5号区域的中心点横坐标来确定面部区域的中心点横坐标，并且参照标准人体比例因子来确定面部区域的中心点纵坐标。4.1: As shown in Figure 3, after the ten regions of the human body are detected in step 3.2, occlusions are added to specific regions respectively. Among them, add occlusion to the crotch or buttocks of area 6. The abscissa of the center point of the face area is determined by the abscissa of the center point of the No. 5 area, and the ordinate of the center point of the face area is determined with reference to the standard human scale factor.
4.2：步骤4.1获得了隐私部位的中心点坐标，基于提供的中心点坐标，添加遮挡物来保护隐私。4.2: In step 4.1, the coordinates of the center point of the privacy part are obtained, and based on the coordinates of the center point provided, an occluder is added to protect privacy.
步骤5、违禁物标记框投影：Step 5. Prohibited object marking frame projection:
到步骤4.2结束。本发明已经可以利用DHF架构屏蔽人体隐私部位。接下来还需要将自动目标识别算法检测到的危险物体投影到卡通图中，这样可以进一步保护受检人员的隐私安全。Go to the end of step 4.2. The present invention can already use the DHF structure to shield the private parts of the human body. Next, the dangerous objects detected by the automatic target recognition algorithm need to be projected into the cartoon image, which can further protect the privacy of the inspected personnel.
5.1：最近邻算法绑定人体区域。5.1: The nearest neighbor algorithm binds the human body region.
输入：enter:
(1)步骤3.2获得的人体部位的中心点坐标centeri＝(center_xi,center_yi),i∈[1,Z]；Z表示检测器预测的人体区域的个数；(1) The coordinates of the center point of the body part obtained in step 3.2 centeri =(center_xi , center_yi ), i∈[1,Z]; Z represents the number of human body regions predicted by the detector;
(2)自动目标识别算法获得的违禁物体中心点坐标detj＝(det_xj,det_yj),j∈[1,N],N表示一次检出的违禁物的个数；(2) The coordinates of the center point of the prohibited object obtained by the automatic target recognition algorithm detj = (det_xj , det_yj ), j∈[1,N], N represents the number of prohibited objects detected at one time;
要判断违禁物体的中心点坐标与哪个人体部位绑定，本发明通过计算欧式距离，选择欧式距离最短的一个人体部位中心centermin＝(center_xmin,center_ymin)，执行下一步。To determine which body part the center point coordinate of the prohibited object is bound to, the present invention calculates the Euclidean distance, selects the centermin = (center_xmin , center_ymin ) of a body part with the shortest Euclidean distance, and executes the next step.
5.2：投影标记框至卡通图片。5.2: Project the marker frame to the cartoon picture.
若Z＝10，即检测器正常地预测出了10个人体区域，则计算偏移向量b0＝detj-centermin,b1是对b0向量归一化的结果。If Z=10, that is, the detector normally predicts 10 human body regions, then calculate the offset vector b0 =detj -centermin , b1 is the result of normalizing the b0 vector.
若Z<10，即人体区域检测器预测的人体区域小于10个，这种情况是由于人体站姿发生变化导致成像产生较大偏差导致。则计算偏移向量b0＝detj-center5,其中center5表示5号人体部件的中心点坐标。If Z<10, that is, the human body area predicted by the human body area detector is less than 10, which is caused by the large deviation of the imaging caused by the change of the standing posture of the human body. Then calculate the offset vector b0 =detj -center5 , Among them, center5 represents the coordinates of the center point of the No. 5 human body part.
5.3：重构卡通图片中违禁物标记框。5.3: Reconstruct the prohibited object marking box in the cartoon picture.
利用偏移向量b1重构在卡通图中违禁物的中心点坐标。本发明针对图3中的右侧卡通图，提前定义卡通图中的十个人体区域中心点坐标，记center_cark,k∈[1,10]。其中ω＝(ωW,ωH)分别表示卡通图的宽和高。Use the offset vector b1 to reconstruct the coordinates of the center point of the prohibited object in the cartoon image. Aiming at the cartoon picture on the right in FIG. 3 , the present invention defines the coordinates of the center points of ten human body regions in the cartoon picture in advance, and denote center_cark ,k∈[1,10]. where ω=(ωW , ωH ) represent the width and height of the cartoon image, respectively.
按照b1ω+center_carmin，即可简单解出卡通图中违禁物的中心点坐标。According to b1 ω+center_carmin , the coordinates of the center point of the prohibited object in the cartoon picture can be simply solved.
本发明针对毫米波图像，利用卷积神经网络来提取毫米波图像中的人体结构信息，基于人体结构信息完成人体隐私保护算法。本发明将人体分为10个区域，利用SSD[9]模型来检测人体10个区域，将隐私区域进行遮挡处理；针对人体违禁物体的标记框，利用最近邻算法、投影算法将在毫米波图像中的标记框投影至卡通图像中的对应位置。Aiming at the millimeter wave image, the present invention uses the convolutional neural network to extract the human body structure information in the millimeter wave image, and completes the human body privacy protection algorithm based on the human body structure information. The present invention divides the human body into 10 areas, uses the SSD[9] model to detect the 10 areas of the human body, and performs occlusion processing on the privacy area; for the marked frame of the prohibited object of the human body, the nearest neighbor algorithm and the projection algorithm are used to detect the millimeter wave image. The marker box in is projected to the corresponding position in the cartoon image.
附图说明Description of drawings
图1是主动式毫米波成像结果展示。Figure 1 shows the results of active millimeter wave imaging.
图2是本发明提出的算法与传统人体图像隐私保护算法的可视化对比结果，第一行是本发明提出算法的实验结果，第二行是传统方法的实验结果。其中A栏表示毫米波原图，其中右上角的长方形标记框表示违禁物目标检测算法所检测到的人体所携带的违禁物体，B栏是对人体面部加入遮挡的效果图对比，C栏是将检测到的人体携带物投影到卡通图像的效果对比。在C中，长方形标记框是地面真实(Ground Truth)，而带圆点的长方形标记框是采用不同的算法的投影结果。Fig. 2 is the visual comparison result of the algorithm proposed by the present invention and the traditional human body image privacy protection algorithm. The first row is the experimental result of the algorithm proposed by the present invention, and the second row is the experimental result of the traditional method. Column A represents the original millimeter wave image, and the rectangular frame in the upper right corner represents the prohibited objects carried by the human body detected by the prohibited object target detection algorithm. Comparison of the effects of the detected human carried objects projected onto the cartoon image. In C, the rectangular marked box is the ground truth, and the rectangular marked box with dots is the projection result using a different algorithm.
图3是深度卷积神经网络人体局部区域识别架构(DHF)图。其功能是检测人体的10个不同区域，完成隐私部位遮挡物添加，完成违禁物体标记框从原始毫米波图像至卡通图像的投影。Figure 3 is a diagram of a deep convolutional neural network human body local region recognition architecture (DHF). Its function is to detect 10 different areas of the human body, complete the addition of privacy blocking objects, and complete the projection of the prohibited object marking frame from the original millimeter wave image to the cartoon image.
图4是本发明对人体区域的分割方式：将人体分割为10个不同的区域。Fig. 4 is the segmentation method of the human body region according to the present invention: the human body is divided into 10 different regions.
图5是本发明提出的人体部位数据集中的前景目标的统计结果，横坐标GT area表示前景目标的面积大小，纵坐标Number表示前景目标的数目。5 is the statistical result of foreground objects in the human body part data set proposed by the present invention, the abscissa GT area represents the area size of the foreground objects, and the ordinate Number represents the number of foreground objects.
图6是针对人体部位数据集下的验证集结果。实线代表采用SSD架构的全部特征图预测人体部位，圆点实线代表采用fc7，conv6_2，conv7_2预测人体部位。横坐标Trainingiterations表示是训练迭代次数，纵坐标mAP表示的是检测器的性能指标。Figure 6 shows the validation set results for the human body part dataset. The solid line represents the prediction of human body parts using all feature maps of the SSD architecture, and the dotted solid line represents the prediction of human body parts using fc7, conv6_2, and conv7_2. The abscissa Trainingiterations represents the number of training iterations, and the ordinate mAP represents the performance index of the detector.
图7是在本发明设计的人体部位数据集中的预测结果，每张图片都准确预测出了10个人体部位以及其对应的类别概率。FIG. 7 is the prediction result in the human body part data set designed by the present invention, and each picture accurately predicts 10 human body parts and their corresponding class probabilities.
图8是每张毫米波原图中的危险物体标记框到卡通图片的投影结果。其中实线包围框是卡通图上的Ground Truth，而圆点实线包围框是采用DHF架构的投影结果。Figure 8 is the projection result of the dangerous object marker frame in each millimeter-wave original image to the cartoon image. The solid line bounding box is the Ground Truth on the cartoon image, and the dotted solid line bounding box is the projection result using the DHF architecture.
图9是采用DHF算法投影违禁物标记框出现错误的结果。Figure 9 is the result of using the DHF algorithm to project the contraband marker frame with an error.
图中标号：1、左小臂，2、右小臂，3、左大臂，4、右大臂，5、胸部，6、裆部，7、左小腿，8、右小腿，9、左大腿，10、右大腿。Labels in the picture: 1, left forearm, 2, right forearm, 3, left forearm, 4, right forearm, 5, chest, 6, crotch, 7, left calf, 8, right calf, 9, left Thigh, 10, right thigh.
具体实施方式Detailed ways
下面，在毫米波数据集中来说明本发明的具体实施方式。In the following, specific embodiments of the present invention are described in a millimeter-wave data set.
数据集说明：本发明所设计的人体部位数据集来自[5]。在[5]中其中包含15万张带有违禁物体的训练集图像，6454张带有违禁物体的验证集图像，9个标准测试集。Data set description: The human body part data set designed in this invention comes from [5]. In [5] it contains 150,000 images of training set with prohibited objects, 6454 images of validation set with prohibited objects, and 9 standard test sets.
1、预测人体部位实验：1. Prediction of human body parts:
数据集选取与标注：Dataset selection and labeling:
在15万张毫米波扫描结果图中选择5788张，其中2894张正面扫描结果，2894张背面扫描结果。5788张图像中覆盖了不同身材、性别、身高的50个受检人。我们将原始毫米波图像中的人体划分为10个区域，并标注这些区域，如图4所示。Among the 150,000 millimeter wave scan results, 5,788 were selected, including 2,894 front scan results and 2,894 back scan results. The 5788 images cover 50 subjects of different shapes, genders, and heights. We divide the human body in the original mmWave image into 10 regions and label these regions, as shown in Figure 4.
训练实验设置：Training experiment setup:
本节介绍人体部位检测器的训练设置，代码采用caffe[14]编写，选用上述内容介绍的数据集中的3859张图片作为训练样本。并且，本节所有实验都按照如下实验设置进行：This section introduces the training settings of the human body part detector. The code is written in caffe [14], and 3859 images in the data set described above are selected as training samples. Moreover, all experiments in this section are carried out according to the following experimental settings:
初始化学习率:0.001；initial learning rate: 0.001;
训练周期:约40次遍历训练集，又叫做epochs数；Training cycle: about 40 traversal of the training set, also known as the number of epochs;
训练迭代次数：约30000次，每次抓取的batch size的个数：8；Number of training iterations: about 30,000 times, the number of batch sizes captured each time: 8;
优化算法，带冲量SGD，momentum设置为0.9；Optimization algorithm, with impulse SGD, momentum is set to 0.9;
正则项：采用L2，其中惩罚因子(weight decay)设置为0.0005；Regular term: L2 is used, where the penalty factor (weight decay) is set to 0.0005;
预训练模型：加载SSD[9]模型在VOC0712数据集上的训练的最优结果作为初始化参数。Pre-trained model: Load the optimal results of the SSD[9] model training on the VOC0712 dataset as initialization parameters.
测试实验设置：Test the experiment setup:
验证集：1929张毫米波扫描结果。Validation set: 1929 mmWave scan results.
验证过程中，1)实验设置正样本阈值为0.9，即置信度高于0.9的候选框才会被作为网络输出的预测结果；During the verification process, 1) the threshold of the positive sample is set to 0.9 in the experiment, that is, the candidate frame with a confidence level higher than 0.9 will be used as the prediction result of the network output;
2)设置检出重合度设置为0.5，即网络的预测结果与Ground Truth的重合度大于0.5即判断为检出。2) Set the detection coincidence degree to 0.5, that is, the coincidence degree between the prediction result of the network and the Ground Truth is greater than 0.5, and it is judged as detection.
为了识别不同身材、不同身高，不同受检扫描姿态的受检人员的人体部位，我们采用多层级(multi-level)特征图预测的方式。如图6中的6level features代表选择6层特征图预测人体部位，而3level features代表选择fc7，conv6_2，conv7_2来预测人体部位。实验结果表明采用K-means聚类初始化候选框的规模因子，并且采用3层特征图预测人体部位的性能优于采用6层特征图预测人体部位，并且相对于6层级特征图，选择3个层级的特征图预测会降低检测模型的时间复杂度，因此，在DHF架构中，我们选择3个层级特征来识别不同身高、身材、受检姿态的受检人员。其中，在验证集中mAP指标为95.24％；在正样本阈值为0.9的条件下，成功召回19287个人体部位，达到99.99％召回率。In order to identify the human body parts of subjects with different body shapes, different heights, and different scanning postures, we adopt the method of multi-level feature map prediction. As shown in Figure 6, 6level features represent selecting 6-layer feature maps to predict human body parts, while 3level features represent selecting fc7, conv6_2, conv7_2 to predict human body parts. The experimental results show that K-means clustering is used to initialize the scale factor of candidate boxes, and the performance of predicting human body parts with 3-layer feature maps is better than that with 6-layer feature maps. The prediction of the feature map will reduce the time complexity of the detection model. Therefore, in the DHF architecture, we choose 3 levels of features to identify subjects with different heights, shapes and postures. Among them, the mAP index in the validation set is 95.24%; under the condition that the positive sample threshold is 0.9, 19,287 human body parts are successfully recalled, reaching a recall rate of 99.99%.
2、投影违禁物标记框实验：2. Projecting contraband marking box experiment:
数据集选取与标注：Dataset selection and labeling:
在[5]中的9个标准测试集中随机挑选出352张不同身高的模特的不同部位的身体正面扫描结果，224张身体背面扫描结果。352 frontal body scans and 224 back body scans of different parts of models of different heights were randomly selected from the 9 standard test sets in [5].
标注上述所有图片。标注方式如下：Label all images above. The marking method is as follows:
(1)对于在原始毫米波图像中的任意一个违禁物标记框，确认其中心点落在哪个人体部位内，例如，图2中的违禁物标记框的中心点落在了2号部位；(1) For any prohibited object marking frame in the original millimeter wave image, confirm which body part its center point falls on, for example, the center point of the prohibited object marking frame in Figure 2 falls on No. 2 part;
(2)在卡通图中找到上述对应的人体部位，人工标记在卡通图中相对于人体部位的中心点坐标；(2) find the above-mentioned corresponding human body part in the cartoon picture, and manually mark the coordinates of the center point relative to the human body part in the cartoon picture;
(3)在卡通图中违禁物标记框的长和宽由原始毫米波图像与卡通图像的长宽比例系数得到。(3) The length and width of the prohibited object marking frame in the cartoon image are obtained from the length-width ratio coefficient of the original millimeter-wave image and the cartoon image.
我们称在卡通图中的标记结果为投影Ground Truth，利用该标记信息来判断我们的DHF架构对违禁物标记框的投影有效性。We call the labeling result in the cartoon image as Projection Ground Truth, and use this labeling information to judge the effectiveness of our DHF architecture's projection of contraband labeling boxes.
实验设置：Experimental setup:
自动目标识别算法：选择[2]中涉及的毫米波违禁物目标识别算法来获得人体违禁携带物的预测结果。Automatic Target Recognition Algorithm: Select the millimeter-wave contraband target recognition algorithm involved in [2] to obtain the prediction results of the contraband carried by the human body.
数据集：选择本节中所标注的数据集，包含了352张人体正面扫描结果和224张人体背面扫描结果。Datasets: Select the datasets marked in this section, which contain 352 frontal scans and 224 back scans.
评价标准：本节中的评价标准主要负责评价隐私保护算法的性能。其中包括L1_acc指标和L2_acc指标。Evaluation Criteria: The evaluation criteria in this section are mainly responsible for evaluating the performance of privacy-preserving algorithms. These include the L1_acc metric and the L2_acc metric.
(1)L1_acc指标：记δi是算法将原始毫米波图像中的违禁物标记框投影至卡通图中的结果，当δi与投影Ground Truth的重合度大于0.2时，则认为本次算法正确地将原图的标记框投影到了卡通图的对应位置；否则，认为算法没有正确地投影该标记框；(1) L1_acc index: note δi as the result of the algorithm projecting the prohibited object marking frame in the original millimeter wave image to the cartoon image. When the degree of coincidence between δi and the projected Ground Truth is greater than 0.2, the algorithm is considered correct. Project the marker frame of the original image to the corresponding position of the cartoon image; otherwise, it is considered that the algorithm did not correctly project the marker frame;
(2)L2_acc指标：同样地，记δi是算法将原始毫米波图像中的违禁物标记框投影至卡通图中的结果，当δi与投影Ground Truth落在了同一个人体部位时(本例有十个人体部位)，则认为本次算法正确地将原图的标记框投影到了卡通图的对应位置；否则，认为算法没有正确地投影该标记框。(2) L2_acc index: Similarly, note δi is the result of the algorithm projecting the prohibited object marking frame in the original millimeter wave image to the cartoon image, when δi and the projected Ground Truth fall on the same body part (this For example, there are ten human body parts), it is considered that this algorithm correctly projected the marker frame of the original image to the corresponding position of the cartoon image; otherwise, it is considered that the algorithm did not correctly project the marker frame.
对比常规隐私保护算法：Compared with conventional privacy protection algorithms:
表1表明了与常规隐私保护算法对比的实验结果，其中对于DHF的运行时间测试是在NVIDIA TITAN Xp上完成，batch size＝4，取1000次迭代的平均值。Table 1 shows the experimental results compared with conventional privacy-preserving algorithms, in which the running time test for DHF is completed on NVIDIA TITAN Xp, batch size=4, and the average value of 1000 iterations is taken.
对比常规隐私保护算法和DHF算法可知，由于DHF算法获得到有效地人体结构信息，因此对于不同身高、身材、体型、站姿的受检人员的泛化性更强，算法性能稳定，运行速度快。Comparing the conventional privacy protection algorithm and the DHF algorithm, it can be seen that since the DHF algorithm obtains effective human body structure information, it has stronger generalization for subjects with different heights, statures, body shapes, and standing postures, and the algorithm has stable performance and fast running speed. .
表1.与其他的常规隐私保护算法的对比结果(time表示实际的运行时间，单位是毫秒，batch size＝4)Table 1. Comparison results with other conventional privacy protection algorithms (time represents the actual running time, in milliseconds, batch size=4)
3、结果分析：3. Result analysis:
对DHF的实验结果进行分析。在352张正面图片和224张背面图片，每张图片中都带有至少一个违禁物标记框。其中若以L2_acc指标为准，那么DHF算法对正面投影只出错了5个，对背面投影只出错了2个。The experimental results of DHF were analyzed. On 352 front images and 224 rear images, each image was framed with at least one contraband marker. Among them, if the L2_acc index is the criterion, then the DHF algorithm has only 5 errors for the front projection and only 2 errors for the back projection.
本节将对投影出错的图像进行分析，并且分析投影出错的原因，进行下一步讨论。This section will analyze the image with the projection error, and analyze the reasons for the projection error, and discuss the next step.
如图9所示，在卡通图中，实线标记框表示投影Ground Truth，带圆点实线标记框表示DHF的算法投影结果。由于我们在人体分割阶段，将人体划分为10个区域，但是这10个区域中不包含颈部、头部区域，对于面部区域的隐私保护我们是通过胸部区域按照人体比例得到的，因此对于颈部、头部区域出现的违禁物标记框的投影，DHF算法发生了错误。但是这个错误是可以通过进一步细分人体区域而修补。As shown in Figure 9, in the cartoon picture, the solid line marked box represents the projection Ground Truth, and the solid line marked box with dots represents the algorithm projection result of DHF. Since we divide the human body into 10 regions in the human body segmentation stage, but the neck and head regions are not included in these 10 regions, the privacy protection of the facial region is obtained through the chest region according to the proportion of the human body, so for the neck region The projection of the prohibited object marking box appearing in the head and head area, the DHF algorithm has an error. But this error can be fixed by further subdividing the body region.
综上，由于毫米波人体安检仪器在安检领域的广泛应用，对于受检人员的隐私保护是一个重要的研究领域。由于毫米波图像可以穿透衣物，本发明采用DHF架构来给毫米波人体图像添加遮挡，并且利用卡通图替换的方式来提高受检人员的隐私。其中，本发明通过卷积神经网络检测器来获取原始毫米波图像的人体部位信息，召回率达99.99％，通过有效的人体部位信息来添加遮挡物，并且利用最近邻法和投影变换法来完成人体违禁物体标记框向替代图的映射。对比常规的隐私保护算法，基于卷积神经网络来获得人体部位的隐私保护算法更加有效。To sum up, due to the wide application of millimeter-wave human body security inspection instruments in the field of security inspection, the privacy protection of inspected personnel is an important research field. Since the millimeter-wave image can penetrate clothing, the present invention adopts the DHF architecture to add occlusion to the millimeter-wave human body image, and improves the privacy of the inspected person by replacing the cartoon image. Among them, the present invention obtains the human body part information of the original millimeter wave image through the convolutional neural network detector, the recall rate reaches 99.99%, adds the occluder through the effective human body part information, and uses the nearest neighbor method and the projection transformation method to complete Mapping of human prohibited object marker boxes to alternate graphs. Compared with the conventional privacy protection algorithm, the privacy protection algorithm based on convolutional neural network to obtain human body parts is more effective.
为了说明本发明的内容及实施方法，本说明书给出了一个具体实施例。在实施例中引入细节的目的不是限制权利要求书的范围，而是帮助理解本发明所述方法。本领域的技术人员应理解：在不脱离本发明及其所附权利要求的精神和范围内，对最佳实施例步骤的各种修改、变化或替换都是可能的。因此，本发明不应局限于最佳实施例及附图所公开的内容。In order to illustrate the content and implementation method of the present invention, this specification provides a specific embodiment. The purpose of introducing details in the examples is not to limit the scope of the claims, but to aid in understanding the method of the invention. It will be understood by those skilled in the art that various modifications, changes or substitutions of the steps of the preferred embodiment are possible without departing from the spirit and scope of the invention and the appended claims. Therefore, the present invention should not be limited to the contents disclosed in the preferred embodiments and the accompanying drawings.
参考文献references
[1]王凯让，王威，年丰，等.一种基于毫米波成像的隐私保护装置[P].中国：CN102708560B，2012.[1] Wang Kairang, Wang Wei, Nianfeng, et al. A privacy protection device based on millimeter wave imaging [P]. China: CN102708560B, 2012.
[2]叶金晶，周健，等.主动毫米波成像隐私保护算法[J].红外与毫米波学报,2017(4).[2] Ye Jinjing, Zhou Jian, et al. Privacy Protection Algorithm for Active Millimeter Wave Imaging [J]. Journal of Infrared and Millimeter Waves, 2017(4).
[3]Tirosh Y,Birnhack M.Naked in Front of the Machine:Does AirportScanning Violate Privacy？[J].Ohio State Law Journal,2013,74.[3] Tirosh Y, Birnhack M. Naked in Front of the Machine: Does AirportScanning Violate Privacy? [J]. Ohio State Law Journal, 2013, 74.
[4]Otsu N.A Threshold Selection Method from Gray-Level Histograms[J].IEEE Transactions on Systems,Man,and Cybernetics,2007,9(1):62-66.[4]Otsu N.A Threshold Selection Method from Gray-Level Histograms[J].IEEE Transactions on Systems,Man,and Cybernetics,2007,9(1):62-66.
[5]Zhu Y Z Y,Yang M Y M,Wu L W L,et al.Practical millimeter-waveholographic imaging system with good robustness[J].Chinese Optics Letters,2016,14(10):101101-101105.[5]Zhu Y Z Y, Yang M Y M, Wu L W L, et al.Practical millimeter-waveholographic imaging system with good robustness[J].Chinese Optics Letters,2016,14(10):101101-101105.
[6]Liu C,Yang M H,Sun X W.TOWARDS ROBUST HUMAN MILLIMETER WAVEIMAGING INSPECTION SYSTEM IN REAL TIME WITH DEEP LEARNING[J].Progress InElectromagnetics Research,2018,161:87-100.[6] Liu C, Yang M H, Sun X W. TOWARDS ROBUST HUMAN MILLIMETER WAVEIMAGING INSPECTION SYSTEM IN REAL TIME WITH DEEP LEARNING [J]. Progress In Electromagnetics Research, 2018, 161: 87-100.
[7]Ren S,He K,Girshick R,et al.Faster R-CNN:Towards Real-Time ObjectDetection with Region Proposal Networks[J].IEEE Transactions on PatternAnalysis&Machine Intelligence,2015,39(6):1137-1149.[7]Ren S,He K,Girshick R,et al.Faster R-CNN:Towards Real-Time ObjectDetection with Region Proposal Networks[J].IEEE Transactions on PatternAnalysis&Machine Intelligence,2015,39(6):1137-1149.
[8]Lin T Y,Dollár,Piotr,Girshick R,et al.Feature Pyramid Networks forObject Detection[C].In CVPR,2017.[8] Lin T Y, Dollár, Piotr, Girshick R, et al. Feature Pyramid Networks for Object Detection [C]. In CVPR, 2017.
[9]Liu W,Anguelov D,Erhan D,et al.SSD:Single Shot MultiBox Detector[C].In ECCV,2016.[9] Liu W, Anguelov D, Erhan D, et al. SSD: Single Shot MultiBox Detector [C]. In ECCV, 2016.
[10]K.Simonyan and A.Zisserman.Very deep convolutional networks forlarge-scale image recognition.In ICLR,2015.[10]K.Simonyan and A.Zisserman.Very deep convolutional networks for large-scale image recognition.In ICLR,2015.
[11]Hu J,Shen L,Albanie S,et al.Squeeze-and-Excitation Networks[J].InCVPR,2017.[11] Hu J, Shen L, Albanie S, et al. Squeeze-and-Excitation Networks [J]. InCVPR, 2017.
[12]Fu C Y,Liu W,Ranga A,et al.DSSD:Deconvolutional Single ShotDetector[J].In CVPR,2017.[12] Fu C Y, Liu W, Ranga A, et al. DSSD: Deconvolutional Single Shot Detector [J]. In CVPR, 2017.
[13]Shen Z,Liu Z,Li J,et al.DSOD:Learning Deeply Supervised ObjectDetectors from Scratch[J].In ICCV,2017.[13] Shen Z, Liu Z, Li J, et al. DSOD: Learning Deeply Supervised ObjectDetectors from Scratch [J]. In ICCV, 2017.
[14]Jia,Y.,Shelhamer,E.,Donahue,J.,Karayev,S.,Long,J.,Girshick,R.,Guadarrama,S.,Darrell,T.:Caffe:Convolutional architecture for fast featureembedding.In:MM.(2014)[14] Jia, Y., Shelhamer, E., Donahue, J., Karayev, S., Long, J., Girshick, R., Guadarrama, S., Darrell, T.: Caffe: Convolutional architecture for fast featureembedding .In:MM.(2014)
[15]Shrivastava A,Gupta A,Girshick R.Training Region-based ObjectDetectors with Online Hard Example Mining[C].In CVPR,2016.[15]Shrivastava A, Gupta A, Girshick R.Training Region-based ObjectDetectors with Online Hard Example Mining[C].In CVPR,2016.
[16]Hartigan J A,Wong M A.Algorithm AS 136:A K-Means ClusteringAlgorithm[J].Journal of the Royal Statistical Society,1979,28(1):100-108.。[16] Hartigan J A, Wong M A. Algorithm AS 136: A K-Means Clustering Algorithm [J]. Journal of the Royal Statistical Society, 1979, 28(1):100-108.
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910264654.8ACN110334571B (en) | 2019-04-03 | 2019-04-03 | A privacy protection method for human body in millimeter wave images based on convolutional neural network |
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910264654.8ACN110334571B (en) | 2019-04-03 | 2019-04-03 | A privacy protection method for human body in millimeter wave images based on convolutional neural network |
| Publication Number | Publication Date |
|---|---|
| CN110334571Atrue CN110334571A (en) | 2019-10-15 |
| CN110334571B CN110334571B (en) | 2022-12-20 |
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910264654.8AActiveCN110334571B (en) | 2019-04-03 | 2019-04-03 | A privacy protection method for human body in millimeter wave images based on convolutional neural network |
| Country | Link |
|---|---|
| CN (1) | CN110334571B (en) |
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111178447A (en)* | 2019-12-31 | 2020-05-19 | 北京市商汤科技开发有限公司 | Model compression method, image processing method and related device |
| CN111209793A (en)* | 2019-12-05 | 2020-05-29 | 重庆特斯联智慧科技股份有限公司 | Region shielding human body security check method and system based on artificial intelligence |
| CN112767409A (en)* | 2019-11-05 | 2021-05-07 | 珠海格力电器股份有限公司 | Image processing method and device before positioning, storage medium and computer equipment |
| CN113486693A (en)* | 2020-09-09 | 2021-10-08 | 青岛海信电子产业控股股份有限公司 | Video processing method and device |
| CN114332945A (en)* | 2021-12-31 | 2022-04-12 | 杭州电子科技大学 | An Availability Consistent Differential Privacy Human Anonymous Synthesis Method |
| CN114463418A (en)* | 2022-01-26 | 2022-05-10 | 中国电子科技集团公司第十四研究所 | A Semantic Localization Method for Hidden Objects Based on Convolutional Neural Networks |
| CN115311685A (en)* | 2022-08-05 | 2022-11-08 | 杭州电子科技大学 | A method for determining the detection results of millimeter wave images based on the average structural similarity |
| CN116030411A (en)* | 2022-12-28 | 2023-04-28 | 宁波星巡智能科技有限公司 | Human privacy shielding method, device and equipment based on gesture recognition |
| CN117930381A (en)* | 2024-03-25 | 2024-04-26 | 海南中南标质量科学研究院有限公司 | Port non-radiation perspective wave customs clearance inspection system based on IoT big data |
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20090140908A1 (en)* | 2007-06-07 | 2009-06-04 | Robert Patrick Daly | System for deployment of a millimeter wave concealed object detection system using an outdoor passively illuminated structure |
| WO2011100964A2 (en)* | 2010-02-18 | 2011-08-25 | Esw Gmbh | Method for processing multi-channel image recordings in order to detect hidden objects in the optoelectronic inspection of persons |
| US20140086448A1 (en)* | 2012-09-24 | 2014-03-27 | MVT Equity LLC d/b/a Millivision Technologies | Appearance Model Based Automatic Detection in Sensor Images |
| CN106447634A (en)* | 2016-09-27 | 2017-02-22 | 中国科学院上海微系统与信息技术研究所 | Private part positioning and protection method based on active millimeter wave imaging |
| CN107704877A (en)* | 2017-10-09 | 2018-02-16 | 哈尔滨工业大学深圳研究生院 | A kind of image privacy cognitive method based on deep learning |
| CN107730439A (en)* | 2017-09-08 | 2018-02-23 | 深圳市无牙太赫兹科技有限公司 | A kind of human body image mapping method, system and terminal device |
| CN109444967A (en)* | 2018-12-28 | 2019-03-08 | 同方威视技术股份有限公司 | Measuring characteristics of human body method, human body safety check method and fmcw radar-millimeter wave safety check apparatus |
| CN109544563A (en)* | 2018-11-12 | 2019-03-29 | 北京航空航天大学 | A kind of passive millimeter wave image human body target dividing method towards violated object safety check |
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20090140908A1 (en)* | 2007-06-07 | 2009-06-04 | Robert Patrick Daly | System for deployment of a millimeter wave concealed object detection system using an outdoor passively illuminated structure |
| WO2011100964A2 (en)* | 2010-02-18 | 2011-08-25 | Esw Gmbh | Method for processing multi-channel image recordings in order to detect hidden objects in the optoelectronic inspection of persons |
| US20140086448A1 (en)* | 2012-09-24 | 2014-03-27 | MVT Equity LLC d/b/a Millivision Technologies | Appearance Model Based Automatic Detection in Sensor Images |
| CN106447634A (en)* | 2016-09-27 | 2017-02-22 | 中国科学院上海微系统与信息技术研究所 | Private part positioning and protection method based on active millimeter wave imaging |
| CN107730439A (en)* | 2017-09-08 | 2018-02-23 | 深圳市无牙太赫兹科技有限公司 | A kind of human body image mapping method, system and terminal device |
| CN107704877A (en)* | 2017-10-09 | 2018-02-16 | 哈尔滨工业大学深圳研究生院 | A kind of image privacy cognitive method based on deep learning |
| CN109544563A (en)* | 2018-11-12 | 2019-03-29 | 北京航空航天大学 | A kind of passive millimeter wave image human body target dividing method towards violated object safety check |
| CN109444967A (en)* | 2018-12-28 | 2019-03-08 | 同方威视技术股份有限公司 | Measuring characteristics of human body method, human body safety check method and fmcw radar-millimeter wave safety check apparatus |
| Title |
|---|
| CHENGYU LIU，等: "Towards Robust Human Millimeter Wave Imaging Inspection System in Real Time with Deep Learning", 《PROGRESS IN ELECTROMAGNETICS RESEARCH》* |
| HYOUNG LEE,等: "Image registration and fusion of MMW and visual images for concealed object detection", 《PROC. SPIE 7670, PASSIVE MILLIMETER-WAVE IMAGING TECHNOLOGY》* |
| SANTIAGO L´OPEZ TAPIA，等: "Detection and localization of objects in Passive Millimeter Wave Images", 《2016 24TH EUROPEAN SIGNAL PROCESSING CONFERENCE》* |
| SANTIAGO LÓPEZ-TAPIA，等: "Using machine learning to detect and localize concealed objects in passive millimeter-wave images", 《ENGINEERING APPLICATIONS OF ARTIFICIAL INTELLIGENCE》* |
| 叶金晶: "主动毫米波成像隐私保护算法", 《红外与毫米波学报》* |
| 杜琨，等: "主动毫米波图像的人体携带危险物检测研究", 《系统工程与电子技术》* |
| 骆尚，等: "基于卷积神经网络的毫米波图像人体隐匿物检测", 《复旦学报（自然科学版）》* |
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112767409A (en)* | 2019-11-05 | 2021-05-07 | 珠海格力电器股份有限公司 | Image processing method and device before positioning, storage medium and computer equipment |
| CN111209793A (en)* | 2019-12-05 | 2020-05-29 | 重庆特斯联智慧科技股份有限公司 | Region shielding human body security check method and system based on artificial intelligence |
| CN111178447A (en)* | 2019-12-31 | 2020-05-19 | 北京市商汤科技开发有限公司 | Model compression method, image processing method and related device |
| CN111178447B (en)* | 2019-12-31 | 2024-03-08 | 北京市商汤科技开发有限公司 | Model compression method, image processing method and related device |
| CN113486693A (en)* | 2020-09-09 | 2021-10-08 | 青岛海信电子产业控股股份有限公司 | Video processing method and device |
| CN114332945A (en)* | 2021-12-31 | 2022-04-12 | 杭州电子科技大学 | An Availability Consistent Differential Privacy Human Anonymous Synthesis Method |
| CN114463418A (en)* | 2022-01-26 | 2022-05-10 | 中国电子科技集团公司第十四研究所 | A Semantic Localization Method for Hidden Objects Based on Convolutional Neural Networks |
| CN115311685A (en)* | 2022-08-05 | 2022-11-08 | 杭州电子科技大学 | A method for determining the detection results of millimeter wave images based on the average structural similarity |
| CN116030411A (en)* | 2022-12-28 | 2023-04-28 | 宁波星巡智能科技有限公司 | Human privacy shielding method, device and equipment based on gesture recognition |
| CN116030411B (en)* | 2022-12-28 | 2023-08-18 | 宁波星巡智能科技有限公司 | Human privacy shielding method, device and equipment based on gesture recognition |
| CN117930381A (en)* | 2024-03-25 | 2024-04-26 | 海南中南标质量科学研究院有限公司 | Port non-radiation perspective wave customs clearance inspection system based on IoT big data |
| Publication number | Publication date |
|---|---|
| CN110334571B (en) | 2022-12-20 |
| Publication | Publication Date | Title |
|---|---|---|
| CN110334571B (en) | A privacy protection method for human body in millimeter wave images based on convolutional neural network | |
| CN103942577B (en) | Based on the personal identification method for establishing sample database and composite character certainly in video monitoring | |
| US9189855B2 (en) | Three dimensional close interactions | |
| Shufelt et al. | Fusion of monocular cues to detect man-made structures in aerial imagery | |
| Lee et al. | Proposal maps driven mcmc for estimating human body pose in static images | |
| Lee et al. | Human upper body pose estimation in static images | |
| CN112016445B (en) | Monitoring video-based remnant detection method | |
| US10884116B2 (en) | Human-body foreign-matter detection method and system based on millimetre-wave image | |
| CN107239740B (en) | A kind of SAR image automatic target recognition method of multi-source Fusion Features | |
| US20150186716A1 (en) | Learning-based estimation of hand and finger pose | |
| EP2339507B1 (en) | Head detection and localisation method | |
| CN107067413A (en) | A kind of moving target detecting method of time-space domain statistical match local feature | |
| CN114677633B (en) | Multi-component feature fusion-based pedestrian detection multi-target tracking system and method | |
| CN102496016A (en) | Infrared target detection method based on space-time cooperation framework | |
| Gu et al. | Automatic detection of safety helmet wearing based on head region location | |
| CN102609684A (en) | Human body posture detection method and device | |
| CN110245675A (en) | A kind of dangerous objects detection method based on millimeter-wave image human body contextual information | |
| CN106372583A (en) | Millimeter wave image-based human body foreign matter detection method and system | |
| CN103735269A (en) | Height measurement method based on video multi-target tracking | |
| CN112734816B (en) | Heterogeneous image registration method based on CSS-Delaunay | |
| CN103247032A (en) | Weak extended target positioning method based on attitude compensation | |
| CN116884045A (en) | Identity recognition method, identity recognition device, computer equipment and storage medium | |
| Xu et al. | Tracking visual object as an extended target | |
| JP2005351814A (en) | Detector and detecting method | |
| CN110738098A (en) | target identification positioning and locking tracking method |
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |