WO2017084204A1

Movatterモバイル変換

Info

Publication number: WO2017084204A1
Application number: PCT/CN2016/070898
Authority: WO
Inventors: 陈勇杰
Original assignee: GUANGZHOU NEWTEMPO TECHNOLOGIES Co Ltd
Current assignee: GUANGZHOU NEWTEMPO TECHNOLOGIES Co Ltd
Priority date: 2015-11-19
Filing date: 2016-01-14
Publication date: 2017-05-26
Anticipated expiration: 2018-05-19
Also published as: CN105469113B; CN105469113A

Abstract

A method and a system for tracking a human body skeleton point in a two-dimensional video stream, the method comprising: a camera acquiring a two-dimensional video stream, acquiring, by means of a foreground extraction module, a foreground image, and acquiring, by means of a face detection module, the coordinates of a head point and a neck point, the system determining whether the head point is in a screen, if the head point is not in the screen, continuing the face detection, and if the head point is in the screen, dividing the human body into a left part ROI and a right part ROI so as to respectively detect other key points, acquiring, by means of a shoulder point detection module, the coordinates of a left shoulder point and a right shoulder point, acquiring, by means of a hand point detection module, the coordinates of a left hand point and a right hand point, and acquiring, by means of an elbow point detection module, the coordinates of a left elbow point and a right elbow point; and finally summarizing the credibility of each point and displaying credible points.

Description

Translated fromChinese

一种二维视频流中的人体骨骼点追踪方法及系统Human bone point tracking method and system in two-dimensional video stream

技术领域Technical field

本发明涉及图像处理的研究领域，特别涉及一种二维视频流中的人体骨骼点追踪方法及系统。The invention relates to the research field of image processing, and in particular to a human bone point tracking method and system in a two-dimensional video stream.

背景技术Background technique

人机交互技术是指通过计算机的输入输出设备，以便于人们使用的方式实现人与计算机之间有效交流的技术。人体骨骼点追踪技术是人机交互领域的一项重要技术，它可以借助红外线来识别人体的运动，可以对人体的多个部位进行实时追踪而不需要借助任何外部设备进行动作扑捉，在人机交互环境中具有广泛的应用前景。现有技术中的人体骨骼追踪技术一般是构架Kinect和PC主机系统，Kinect主要负责采集图像、深度数据流和骨骼信息，主机负责通过数据库获取图像和深度数据进行骨骼运动轨迹跟踪，并把三维数据的世界坐标系转化为二维数据的图像像素坐标系，再通过降噪滤波每个骨骼数据，以获取人体的骨骼追踪信息，而该技术中最最主要的是识别用户的骨骼信息，现有技术中首先是利用红外线传感器通过黑白光谱的方式以每秒30帧的速度来感知环境并生成景深图像流，然后红外传感器将侦测到的3D深度图像，寻找图像中可能是人体的移动物体，通过逐个像分布来辨别人体的不同部位，再采用分割策略来将人体从背景环境中区分出来，从噪音中提取出有用的信号，最后随机决策树和森林通过身体组件识别推理出像素信息，将所有像素的信息汇集起来形成3D骨架关节位置的可靠预测，给出某个特定像素属于哪个身体部位的可能性。但是该方法对周围的光照环境敏感，光照条件不佳可能会影响追踪的准确性；人体部位的饰物等遮挡物会减少人体的部分局部特征，会对骨骼的追踪造成影响，甚至无法追踪，从而导致识别精度不高，降低了人机交互的效率性和自然性。Human-computer interaction technology refers to the technology of realizing the effective communication between humans and computers through the input and output devices of computers to facilitate the way people use them. Human skeleton point tracking technology is an important technology in the field of human-computer interaction. It can identify the movement of the human body by means of infrared rays, and can track multiple parts of the human body in real time without any external equipment for action capture. It has a wide application prospect in the machine interaction environment. The prior art human bone tracking technology is generally a framework of Kinect and a PC host system. Kinect is mainly responsible for acquiring images, deep data streams and bone information, and the host is responsible for acquiring image and depth data through the database for bone trajectory tracking and three-dimensional data. The world coordinate system is transformed into the image pixel coordinate system of the two-dimensional data, and then each bone data is filtered by the noise reduction to obtain the bone tracking information of the human body, and the most important one in the technology is to identify the user's bone information, and the existing In the technology, the infrared sensor is used to perceive the environment at a speed of 30 frames per second by using a black-and-white spectrum to generate a depth of field image stream, and then the infrared sensor will detect the 3D depth image to find a moving object that may be a human body in the image. By identifying the different parts of the body by image-by-image distribution, the segmentation strategy is used to distinguish the human body from the background environment, and the useful signals are extracted from the noise. Finally, the random decision tree and the forest identify the pixel information through the body component recognition. All pixel information is assembled to form a 3D skeleton joint Reliable prediction opposed, given the possibility of a particular pixel belongs to which body parts. However, this method is sensitive to the surrounding lighting environment, and poor lighting conditions may affect the accuracy of tracking;Obstructions such as ornaments on the human body will reduce some local features of the human body, affecting the tracking of the bones, or even tracking, resulting in low recognition accuracy and reducing the efficiency and naturalness of human-computer interaction.

发明内容Summary of the invention

本发明的主要目的在于克服现有技术的缺点与不足，提出一种二维视频流中的人体骨骼点追踪方法及系统，通过处理深度数据来建立人体各个关节的坐标，利用骨骼追踪确定人体的各个部分。The main object of the present invention is to overcome the shortcomings and shortcomings of the prior art, and to provide a human bone point tracking method and system in a two-dimensional video stream, which is to establish the coordinates of each joint of the human body by processing the depth data, and determine the human body by using bone tracking. Various parts.

为了达到上述目的，本发明采用以下技术方案：In order to achieve the above object, the present invention adopts the following technical solutions:

本发明提供了一种二维视频流中的人体骨骼点追踪方法，该方法包括下述步骤：The invention provides a human skeleton point tracking method in a two-dimensional video stream, the method comprising the following steps:

摄像头获取二维视频流，重构背景并利用减背景的方法提取前景掩膜，去噪处理后输出前景图；The camera acquires a two-dimensional video stream, reconstructs the background, and extracts the foreground mask by using a subtractive background method, and outputs a foreground image after denoising processing;

对输出的前景图检测人脸，获取人脸矩形区域、头部点和颈部点坐标；Detecting a human face on the foreground image of the output, and obtaining coordinates of the rectangular area of the face, the head point, and the neck point;

判断头部点是否在屏幕中，如果否，则继续进行人脸检测模块；如果是，则将人体分为左部分ROI和右部分ROI分别进行其他关键点的检测；Determining whether the head point is on the screen, if not, proceeding to the face detection module; if yes, dividing the human body into a left part ROI and a right part ROI to perform detection of other key points;

使用特定位置扫描并返回有像素值点的方法实现肩部点检测，并获取左肩点和右肩点坐标；Use a specific position to scan and return a pixel value point to achieve shoulder point detection, and obtain left shoulder and right shoulder coordinates;

利用求肤色区域最小外接矩形近端点并返回实现手部点检测，并获取左手点和右手点坐标；Utilizing the near-end point of the minimum circumscribed rectangle of the skin color region and returning to realize the hand point detection, and acquiring the coordinates of the left-hand point and the right-hand point;

利用将手部ROI划分为三个区域，各个区域分别用不同的扫描方式返回点实现肘部点检测，并获取左肘点和右肘点坐标；By dividing the hand ROI into three regions, each region uses different scanning mode return points to realize elbow point detection, and obtain left elbow point and right elbow point coordinates;

最后统计各点可信度并将可信点显示。Finally, the credibility of each point is counted and the credible points are displayed.

作为优选的，在摄像头获取二维视频流，重构背景并利用减背景的方法提取前景掩膜，去噪处理后输出前景图的步骤中，输出前景图的具体方法为：Preferably, in the step of acquiring the two-dimensional video stream by the camera, reconstructing the background and extracting the foreground mask by using the subtractive background method, and outputting the foreground image after the denoising processing, the specific method for outputting the foreground image is:

通过人脸检测算法获取人脸中心位置HEAD(x,y)Obtain the face center position HEAD(x, y) by the face detection algorithm

设定两个参数：左构图阈值left_,右构图阈值right_,左构图指示器left_get＝0,右构图指示器right_get＝0；Set two parameters: left composition threshold left_, right composition threshold right_, left composition indicator left_get=0, right composition indicator right_get=0;

提示用户向左移动，当人脸中心位置横坐标x<left_时，left_get＝1,此时将当前屏幕右半边的图像保存下来，记为image_right；Prompt the user to move to the left. When the horizontal position of the face is x<left_, left_get=1, the image of the right half of the current screen is saved and recorded as image_right;

继续提示用户向右移动，当人脸中心位置横坐标x>right_时，right_get＝1，将当前屏幕左半边的图像保存下来，记为image_left；Continue to prompt the user to move to the right, when the face center position abscissa x>right_, right_get=1, save the image of the left half of the current screen, recorded as image_left;

当left_get＝1且right_get＝1时，将image_left和image_right拼起来，得到背景图BACKGROUND，When left_get=1 and right_get=1, put image_left and image_right together to get the background image BACKGROUND.

BACKGROUND＝image_left+LD(image_right,image_left.clos)BACKGROUND=image_left+LD(image_right,image_left.clos)

其中LD(a,b)表示将图像a整体向右偏移b个像素；Where LD(a,b) represents shifting the image a as a whole to the right by b pixels;

之后每输入一幅图像IMAGE，将IMAGE和BACKGROUND相减并消噪获取前景掩膜foreground_mask，对foreground_mask进行二值化处理得到MASK；Then, after inputting an image IMAGE, the IMAGE and the BACKGROUND are subtracted and denoised to obtain the foreground mask foreground_mask, and the foreground_mask is binarized to obtain the MASK;

将IMAGE与MASK进行与处理输出前景图FOREGROUND。The IMAGE and MASK are processed and processed to output the foreground image FOREGROUND.

作为优选的，在对输出的前景图检测人脸，获取人脸矩形区域、头部点和颈部点坐标的步骤中，使用Haar分类器进行人脸检测，其具体方法为：Preferably, in the step of detecting the face of the output foreground image and acquiring the coordinates of the face rectangular area, the head point and the neck point, the Haar classifier is used for face detection, and the specific method is:

将彩色图转灰度图；Convert the color map to a grayscale image;

对灰度图进行直方图均衡化，增强对比度；Histogram equalization of grayscale images to enhance contrast;

使用Haar分类器检测正脸，有检测到正脸则返回脸部中心点坐标和人脸矩形长宽；The Haar classifier is used to detect the positive face, and when the positive face is detected, the coordinates of the center point of the face and the length and width of the face rectangle are returned;

如果检测不到正脸，则使用Haar分类器检测侧脸，返回脸部中心点坐标和人脸矩形长宽。If the positive face is not detected, use the Haar classifier to detect the side face and return the coordinates of the face center point andThe face is rectangular in length and width.

作为优选的，在使用特定位置扫描并返回有像素值点的方法实现肩部点检测，并获取左肩点和右肩点坐标的步骤中，实现肩部点检测的具体方法为：Preferably, in the step of performing shoulder point detection by using a specific position scanning and returning a pixel value point, and acquiring the coordinates of the left shoulder point and the right shoulder point, the specific method for implementing the shoulder point detection is:

图像预处理获取人体外轮廓；Image preprocessing to obtain the contour of the human body;

取左肩点ROI，其尺寸记为(ROI_HEIGHT,ROI_WIDTH)；Take the left shoulder point ROI, the size is recorded as (ROI_HEIGHT, ROI_WIDTH);

设置SCAN_X，SCAN_X为输入图像宽度的n1倍，其中0＜n1＜1，即SCAN_X＝n 1*ROI_WIDTH；Set SCAN_X, SCAN_X is n1 times the width of the input image, where 0<n1<1, ie SCAN_X=n 1*ROI_WIDTH;

以宽度为SCAN_X从上往下去扫左肩ROI，如果有值大于设定值M，则返回该点坐标；Sweep the left shoulder ROI from the top with the width SCAN_X, and return the coordinates if the value is greater than the set value M;

如果扫不到有值大于M，则以长度为SCAN_Y由右向左去扫左肩ROI，其中SCAN_Y为输入图像长度的n2倍,其中0＜n1＜1，即SCAN_Y＝n2*ROI_HEIGHT如果有值大于M，则返回该点坐标；If the sweep cannot have a value greater than M, sweep the left shoulder ROI from right to left with the length SCAN_Y, where SCAN_Y is n2 times the length of the input image, where 0<n1<1, ie SCAN_Y=n2*ROI_HEIGHT if the value is greater than M, return the coordinates of the point;

利用上述同样的识别方法获取左肩点坐标。The coordinates of the left shoulder point are obtained by the same identification method as described above.

作为优选的，在利用求肤色区域最小外接矩形近端点并返回实现手部点检测，并获取左手点和右手点坐标的步骤中，实现手部点检测的具体方法为：Preferably, in the step of using the minimum circumscribed rectangle near the end point of the skin color region and returning to realize the hand point detection, and acquiring the coordinates of the left hand point and the right hand point, the specific method for implementing the hand point detection is:

将RGB转成YCrCb坐标系，存放在YUU中；Convert RGB into YCrCb coordinate system and store it in YUU;

对YUU三通道分离并分别提取YUU各个通道中得特殊信息组合成新图，存放在BW中；Separate the YUU three channels and extract the special information in each channel of the YUU to form a new map, which is stored in the BW;

对BW进行开运算，除去除噪点、平滑图像并提取外轮廓；Open the BW, except to remove noise, smooth the image and extract the outer contour;

遍历外轮廓并提取最大面积对应的轮廓L，新建L的最小外接矩形K；Traversing the outer contour and extracting the contour L corresponding to the largest area, creating a minimum circumscribed rectangle K of L;

K满足以下条件时直接返回中心点：矩形宽度小于X倍矩形高度且矩形高度小于X倍矩形宽度，其中1＜X＜2；K returns directly to the center point when the following conditions are met: the rectangle width is less than X times the rectangle height and the rectangle height is less than X times the rectangle width, where 1<X<2;

如不满足：If not satisfied:

新建点容器ptt，用来装最小外接矩形K的顶点；Create a new point container ptt for the vertices of the minimum bounding rectangle K;

检测左手，将最左的点找出，定义为ptt[0]，判断次左的点，定义为ptt[1]，定义p1为K的中点，定义p2为ptt[0]和ptt[1]的中点；Detect the left hand, find the leftmost point, define it as ptt[0], determine the next left point, define it as ptt[1], define p1 as the midpoint of K, and define p2 as ptt[0] and ptt[1 Midpoint of

由p1和p2的几何关系确定手的大体位置，并将坐标赋值给p2，当p2在边缘部分时，赋值为(0,0)，值为(0,0)的点不显示；The geometric position of p1 and p2 is used to determine the general position of the hand, and the coordinates are assigned to p2. When p2 is in the edge part, the value is (0, 0), and the value of (0, 0) is not displayed;

返回p2；Return p2;

利用上述同样的方法识别右手的坐标。The coordinates of the right hand are identified using the same method as described above.

作为优选的，在利用将手部ROI划分为三个区域，各个区域分别用不同的扫描方式返回点实现肘部点检测，并获取左肘点和右肘点坐标的步骤中，实现肘部点检测的具体方法为：Preferably, in the step of dividing the hand ROI into three regions, each region uses different scanning mode return points to realize elbow point detection, and acquiring left elbow point and right elbow point coordinates, and implementing elbow point The specific method of detection is:

取左肘部ROI，将ROI分为三个区域，分别对应举手，偏45度，叉腰向下这三种姿势；Take the ROI of the left elbow and divide the ROI into three areas, which respectively correspond to the three postures of raising the hand, 45 degrees, and the hips down;

当肩部点横坐标与手部点横坐标差值大于IMAGE_HEIGHT/50时：When the difference between the abscissa of the shoulder point and the abscissa of the hand point is greater than IMAGE_HEIGHT/50:

举手动作：当手点纵坐标与肩部点纵坐标差值小于阈值IMAGE_HEIGHT/5时，则从下往上扫点，扫到就返回；Raise your hand: When the difference between the ordinate of the hand point and the ordinate of the shoulder point is less than the threshold IMAGE_HEIGHT/5, sweep the point from the bottom up and swipe to return;

偏45度向下：当手点纵坐标与肩部点纵坐标差值大于阈值IMAGE_HEIGHT/5时，则从右往左扫点，扫到点的像素值大于就返回；45 degrees downward: When the difference between the ordinate of the hand point and the ordinate of the shoulder point is greater than the threshold IMAGE_HEIGHT/5, the point is swept from right to left, and the pixel value of the swept point is greater than the return value;

叉腰动作：当肩部点横坐标与手部点横坐标差值小于IMAGE_HEIGHT/50时，则从左向右扫点，返回第一个像素值大于50的点的坐标。Forked waist movement: When the difference between the shoulder coordinate of the shoulder point and the horizontal coordinate of the hand point is less than IMAGE_HEIGHT/50, the point is swept from left to right, and the coordinates of the point with the first pixel value greater than 50 are returned.

作为优选的，该方法还包括下述步骤：Preferably, the method further comprises the steps of:

利用求下半身前景区域最小外接矩形近端点并返回实现脚点检测，所述脚点检测的具体方法为：The foot point detection is implemented by using the near-end point of the minimum circumscribed rectangle of the foreground area of the lower body and returning. The specific method of the foot point detection is:

在全身模式下，以屏幕一半取出前景图的人体下半身ROI；In the whole body mode, the human body lower body ROI of the foreground image is taken out by half of the screen;

提取外轮廓，遍历外轮廓并提取最大面积对应的轮廓L，新建L的最小外接矩形K；Extracting the outer contour, traversing the outer contour and extracting the contour L corresponding to the largest area, creating a minimum circumscribed rectangle K of L;

K满足以下条件时直接返回中心点：矩形宽度小于Y倍矩形高度且矩形高度小于Y倍矩形宽度，其中1＜Y＜2；K returns directly to the center point when the following conditions are met: the rectangle width is less than Y times the rectangle height and the rectangle height is less than Y times the rectangle width, where 1<Y<2;

如不满足：If not satisfied:

新建点容器ptfoot，用来装最小外接矩形K的顶点；Create a new point container ptfoot to hold the vertices of the minimum bounding rectangle K;

检测左脚，将最左的点找出，定义为ptfoot[0]，判断次左的点，定义为ptfoot[1]，定义p1为K的中点，定义p2为ptfoot[0]和ptfoot[1]的中点；Detect the left foot, find the leftmost point, define it as ptfoot[0], determine the next left point, define it as ptfoot[1], define p1 as the midpoint of K, and define p2 as ptfoot[0] and ptfoot[ The midpoint of 1];

由p1和p2的几何关系确定脚的大体位置，并将坐标赋值给p2；Determine the general position of the foot by the geometric relationship of p1 and p2, and assign the coordinates to p2;

当p2在边缘部分时，赋值为(0,0)，值为(0,0)的点不显示；When p2 is in the edge part, the value is (0, 0), and the value of (0, 0) is not displayed;

返回p2；Return p2;

利用上述同样的识别方法获取右脚点坐标。The coordinates of the right foot point are obtained by the same recognition method described above.

利用脚点往上取设定高度的距离进行扫描并返回的方法实现膝部点检测，所述膝部点检测的具体方法为：The knee point detection is realized by scanning and returning the distance from the foot point to the set height. The specific method of the knee point detection is:

背景重构模块获取人体前景，在全身模式下，取下半身人体ROI；The background reconstruction module acquires the foreground of the human body, and in the whole body mode, removes the body ROI of the lower body;

获取人体高度BODY_HEIGHT,BODY_HEIGHT＝FOOT_LEFT_Y–FACE_Y+FACE_HEIGHT/2；Get human height BODY_HEIGHT, BODY_HEIGHT=FOOT_LEFT_Y–FACE_Y+FACE_HEIGHT/2;

取左脚部ROI，其尺寸记为(ROI_HEIGHT,ROI_WIDTH)；Take the left foot ROI, the size is recorded as (ROI_HEIGHT, ROI_WIDTH);

设置SCAN_Y，SCAN_Y为用户高度的0.2倍，即SCAN_Y＝0.2*BODY_HEIGHT；Set SCAN_Y, SCAN_Y is 0.2 times the height of the user, ie SCAN_Y=0.2*BODY_HEIGHT;

以FOOT_LEFT_Y以上SCAN_Y的高度从左往右去扫左脚ROI，如果有值大于50，则返回该点坐标(x+12,y)，其中x+12表示对横坐标做一个12像素的偏移处理，使得膝部点处于膝部的中心位置；Sweep the left foot ROI from left to right with the height of FOOT_LEFT_Y above SCAN_Y, if there is a valueIf it is greater than 50, the coordinate of the point (x+12, y) is returned, where x+12 represents a 12-pixel offset processing on the abscissa, so that the knee point is at the center of the knee;

如果扫不到有值大于50，则返回(0,0)，并置为不可信点；If the value cannot be greater than 50, it returns (0,0) and is set as an untrusted point;

利用上述同样的识别方法获取右膝点坐标。The coordinates of the right knee point are obtained by the same recognition method described above.

本发明还提供一种二维视频流中的人体骨骼点追踪系统，该系统包括：The invention also provides a human skeleton point tracking system in a two-dimensional video stream, the system comprising:

前景提取模块，用于摄像头获取二维视频流，重构背景并利用减背景的方法提取前景掩膜，去噪处理后输出前景图；The foreground extraction module is configured to acquire a two-dimensional video stream, reconstruct a background, and extract a foreground mask by using a subtractive background method, and output a foreground image after denoising processing;

人脸检测模块，用于对输出的前景图检测人脸，获取人脸矩形区域、头部点和颈部点坐标；a face detection module, configured to detect a face of the output foreground image, and obtain a rectangle of the face, a head point, and a neck point coordinate;

判断模块，用于判断头部点是否在屏幕中，如果否，则继续进行人脸检测模块；如果是，则将人体分为左部分ROI和右部分ROI分别进行其他关键点的检测；a judging module, configured to determine whether the head point is in the screen; if not, proceeding to the face detecting module; if yes, dividing the human body into a left part ROI and a right part ROI to perform other key points respectively;

肩部点检测模块，用于使用特定位置扫描并返回有像素值点的方法实现肩部点检测，并获取左肩点和右肩点坐标；a shoulder point detection module for performing shoulder point detection by scanning and returning a pixel value point using a specific position, and acquiring left shoulder point and right shoulder point coordinates;

手部检测模块，用于利用求肤色区域最小外接矩形近端点并返回实现手部点检测，并获取左手点和右手点坐标；The hand detection module is configured to perform the hand point detection by using the minimum circumscribed rectangle near the end point of the skin color region, and obtain the coordinates of the left hand point and the right hand point;

肘部检测模块，用于利用将手部ROI划分为三个区域，各个区域分别用不同的扫描方式返回点实现肘部点检测，并获取左肘点和右肘点坐标；The elbow detection module is configured to divide the hand ROI into three regions, and each region uses different scanning mode return points to realize elbow point detection, and obtain left elbow point and right elbow point coordinates;

统计模块，最后统计各点可信度并将可信点显示。The statistics module finally counts the credibility of each point and displays the trusted points.

作为优选的，该系统还包括脚点检测模块和膝部点检测模块；Preferably, the system further comprises a foot point detection module and a knee point detection module;

所述脚点检测模块，用于利用求下半身前景区域最小外接矩形近端点并返回实现脚点检测；The foot point detecting module is configured to perform a foot point detection by using a minimum circumscribed rectangle near end point of the lower body foreground area and returning;

所述膝部点检测模块，用于利用脚点往上取设定高度的距离进行扫描并返回的方法实现膝部点检测。The knee point detecting module is configured to scan and return by using a foot point to take a set height distanceThe method of returning implements knee point detection.

本发明与现有技术相比，具有如下优点和有益效果：Compared with the prior art, the present invention has the following advantages and beneficial effects:

1.本发明不需要使用到深度信息，可直接利用普通摄像头实现人体骨架点识别，普适性较强。1. The invention does not need to use depth information, and can directly realize the human body skeleton point recognition by using an ordinary camera, and has universal applicability.

2.本发明算法简单，占用计算资源少，对硬件要求低，实时性强；2. The algorithm of the invention is simple, occupies less computing resources, has low hardware requirements, and has strong real-time performance;

3.本发明不受开发平台限制，可应用在移动终端(如手机，平板等)，满足跨平台需求，可移植性强。3. The invention is not limited by the development platform, and can be applied to mobile terminals (such as mobile phones, tablets, etc.) to meet cross-platform requirements and has strong portability.

4.本发明可应对一般场景下的背景复杂，光照不均等问题，鲁棒性较强。4. The invention can cope with the complicated background and uneven illumination in the general scene, and has strong robustness.

附图说明DRAWINGS

图1为本发明的定义的人体的骨架图；Figure 1 is a skeleton diagram of a human body as defined in the present invention;

图2为本发明的二维视频流中的人体骨骼点追踪方法流程图；2 is a flow chart of a method for tracking a human skeleton point in a two-dimensional video stream of the present invention;

图3是本发明输入的原始图像；Figure 3 is an original image input by the present invention;

图4是本发明的背景图；Figure 4 is a background view of the present invention;

图5是本发明的掩膜二值图；Figure 5 is a mask binary diagram of the present invention;

图6是本发明的前景图；Figure 6 is a foreground view of the present invention;

图7是本发明人脸检测区域示意图；7 is a schematic diagram of a face detection area of the present invention;

图8是本发明经过人脸检测获取的头部点和颈部点的示意图；Figure 8 is a schematic view of a head point and a neck point obtained by face detection according to the present invention;

图9是本发明的肩部点的区域示意图；Figure 9 is a schematic view of a region of a shoulder point of the present invention;

图10是本发明手部点的区域示意图；Figure 10 is a schematic view showing the area of the hand point of the present invention;

图11是本发明区域划分示意图；Figure 11 is a schematic view showing the division of the area of the present invention;

图12本发明肘部点的区域示意图；Figure 12 is a schematic view showing the area of the elbow point of the present invention;

图13是本发明整体关键点的识别效果图。Figure 13 is a diagram showing the recognition effect of the overall key points of the present invention.

具体实施方式detailed description

下面结合实施例及附图对本发明作进一步详细的描述，但本发明的实施方式不限于此。The present invention will be further described in detail below with reference to the embodiments and drawings, but the embodiments of the present invention are not limited thereto.

实施例Example

当前，基于深度的骨骼追踪技术通过处理深度数据来建立人体各个关节的坐标，骨骼追踪能够确定人体的各个部分，如那部分是手，头部，以及身体，还能确定他们所在的位置。但是普通摄像头只能获取空间中的二维信息，本算法的目标就是实现二维视频流中的人体骨骼点追踪。Currently, depth-based bone tracking technology creates depth coordinates for each joint of the human body by processing depth data, and bone tracking can determine various parts of the body, such as the hand, head, and body, and determine where they are. However, the ordinary camera can only obtain two-dimensional information in space. The goal of this algorithm is to realize the tracking of human skeleton points in the two-dimensional video stream.

首先如图1所示，定义人体的相关检测点和相关图，如下表1、表2所示；First, as shown in Figure 1, the relevant detection points and related diagrams of the human body are defined, as shown in Table 1 and Table 2 below;

表1Table 1

11头部点HEADHead point HEAD22颈部点SHOULDER_CENTERNeck point SHOULDER_CENTER33左肩点SHOULDER_LEFTLeftshoulder point SHOULDER_LEFT44右肩点SHOULDER_RIGHRight shoulder point SHOULDER_RIGH55左手点HAND_LEFTLefthand point HAND_LEFT66右手点HAND_RIGHTRighthand point HAND_RIGHT77左肘点ELBOW_LEFTLeftelbow point ELBOW_LEFT88右肘点ELBOW_RIGHTRight elbow point ELBOW_RIGHT99臀部点HIP_CENTERHip point HIP_CENTER1010左脚点FOOT_LEFTLeft foot point FOOT_LEFT1111右脚点FOOT_RIGHTRight foot point FOOT_RIGHT1212左膝点KNEE_LEFTLeftknee point KNEE_LEFT1313右膝点KNEE_RIGHTRight knee point KNEE_RIGHT

表2Table 2

原始图original imageIMAGEIMAGE背景图Background imageBACKGROUNDBACKGROUND原图宽度Original widthIMAGE_WIDTHIMAGE_WIDTH前景掩膜Foreground maskMASKMASK原图长度Original lengthIMAGE_HEIGHTIMAGE_HEIGHT前景图Foreground mapFOREGROUNDFOREGROUND

如图2所示，本发明一种二维视频流中的人体骨骼点追踪方法，该方法包括下述步骤：As shown in FIG. 2, the present invention provides a method for tracking a human skeleton point in a two-dimensional video stream, the method comprising the following steps:

步骤S1、摄像头获取二维视频流，重构背景并利用减背景的方法提取前景掩膜，去噪处理后输出前景图；Step S1: The camera acquires a two-dimensional video stream, reconstructs a background, and extracts a foreground mask by using a subtractive background method, and outputs a foreground image after denoising processing;

如图3-图6所示，输出前景图的具体方法为：As shown in Figure 3-6, the specific method of outputting the foreground map is:

S11、通过人脸检测算法获取人脸中心位置HEAD(x,y)S11. Obtain a face center position HEAD(x, y) by using a face detection algorithm.

S12、设定两个参数：左构图阈值left_,右构图阈值right_,左构图指示器left_get＝0,右构图指示器right_get＝0；S12, setting two parameters: left composition threshold left_, right composition threshold right_, left composition indicator left_get=0, right composition indicator right_get=0;

S13、提示用户向左移动，当人脸中心位置横坐标x<left_时，left_get＝1,此时将当前屏幕右半边的图像保存下来，记为image_right；S13, prompting the user to move to the left. When the horizontal position of the face is x<left_, left_get=1, the image of the right half of the current screen is saved, and is recorded as image_right;

S14、继续提示用户向右移动，当人脸中心位置横坐标x>right_时，right_get＝1，将当前屏幕左半边的图像保存下来，记为image_left；S14, continue to prompt the user to move to the right, when the face center position abscissa x>right_, right_get=1, save the image of the left half of the current screen, recorded as image_left;

S15、当left_get＝1且right_get＝1时，将image_left和image_right拼起来，得到背景图BACKGROUND，S15. When left_get=1 and right_get=1, the image_left and the image_right are spelled together to obtain the background image BACKGROUND,

S16、之后每输入一幅图像IMAGE，将IMAGE和BACKGROUND相减并消噪获取前景掩膜foreground_mask，对foreground_mask进行二值化处理得到MASKS16, after each input of an image IMAGE, IMAGE and BACKGROUND are subtracted and denoised to obtain the foreground mask foreground_mask, and the foreground_mask is binarized to obtain MASK

foreground_mask＝abs(IMAGE–BACKGROUND)；Foreground_mask=abs(IMAGE–BACKGROUND);

其中abs(a)表示对a取绝对值；Where abs(a) denotes an absolute value for a;

MASK＝threshold(foreground_mask,55)；MASK=threshold(foreground_mask, 55);

其中threshold(a,T)表示对图像a以阈值b做二值化处理，像素值高于T的点置为255，像素值低于T的点置为0，Where threshold(a, T) indicates that the image a is binarized with the threshold b, the point where the pixel value is higher than T is set to 255, and the point where the pixel value is lower than T is set to 0.

S17、将IMAGE与MASK进行与处理输出前景图FOREGROUND。S17. The IMAGE and the MASK are processed and processed to output a foreground image FOREGROUND.

步骤S2、对输出的前景图检测人脸，获取人脸矩形区域、头部点和颈部点坐标；Step S2: detecting a human face on the foreground image of the output, and acquiring coordinates of the rectangular area of the face, the head point, and the neck point;

本实施例中，如图7、图8所示，采用Haar分类器进行人脸检测，其具体方法为：In this embodiment, as shown in FIG. 7 and FIG. 8, the Haar classifier is used for face detection, and the specific method is:

S21、将彩色图转灰度图；S21, converting the color map to a grayscale image;

S22、对灰度图进行直方图均衡化，增强对比度；S22, performing histogram equalization on the grayscale image to enhance contrast;

S23、使用Haar分类器检测正脸，有检测到正脸则返回脸部中心点坐标和人脸矩形长宽(HEAD_HEIGHT,HEAD_WIDTH)；S23. Using a Haar classifier to detect a positive face, and if a positive face is detected, returning a face center point coordinate and a face rectangle length and width (HEAD_HEIGHT, HEAD_WIDTH);

S24、如果检测不到正脸，则使用Haar分类器检测侧脸，返回脸部中心点坐标和人脸矩形长宽；S24. If the positive face is not detected, the Haar classifier is used to detect the side face, and the coordinates of the center point of the face and the length and width of the face rectangle are returned;

S25、其中脸部中心点坐标作为头部点，以头部点往下取0.75倍的人脸矩形长度，即将S25, wherein the coordinates of the center point of the face are used as the head point, and the length of the face rectangle of 0.75 times is taken downward from the head point, that is,

(HEAD.X,HEAD.Y+0.75*HEAD_HEIGHT)确定为颈部点(HEAD.X, HEAD.Y+0.75*HEAD_HEIGHT) is determined as the neck point

S26、以头部点往下取3倍人脸矩形长度，即将(HEAD.X,HEAD.Y+3*HEAD_HEIGHT)确定为颈部点。S26. Take the length of the face rectangle 3 times from the head point, and determine (HEAD.X, HEAD.Y+3*HEAD_HEIGHT) as the neck point.

Harr特征值的计算公式为(窗口大小N*N)：The Harr eigenvalue is calculated as (window size N*N):

对于给定的一个N*N的窗口I，其积分图计算公式如下：For a given N*N window I, the integral graph is calculated as follows:

对一个窗口图像中的一个方形内的像素求和计算方式如下：The summation of pixels in a square in a window image is calculated as follows:

步骤S3、判断头部点是否在屏幕中，如果否，则继续进行人脸检测模块；如果是，则将人体分为左部分ROI和右部分ROI分别进行其他关键点的检测；Step S3, determining whether the head point is in the screen, if not, proceeding to the face detection module; if yes, dividing the human body into a left part ROI and a right part ROI to perform detection of other key points;

步骤S4、使用特定位置扫描并返回有像素值点的方法实现肩部点检测，并获取左肩点和右肩点坐标；Step S4, using a specific position scanning and returning a pixel value point method to achieve shoulder point detection, and acquiring left shoulder point and right shoulder point coordinates;

如图9所示，实现肩部点检测的具体方法为：As shown in Figure 9, the specific method for implementing shoulder point detection is:

S41、图像预处理获取人体外轮廓；S41. Image preprocessing obtains a human body contour;

S42、取左肩点ROI，其尺寸记为(ROI_HEIGHT,ROI_WIDTH)；S42, taking the left shoulder point ROI, the size of which is recorded as (ROI_HEIGHT, ROI_WIDTH);

S43、设置SCAN_X，SCAN_X为输入图像宽度的0.35倍，即SCAN_X＝0.35*ROI_WIDTH；S43, setting SCAN_X, SCAN_X is 0.35 times of the input image width, that is, SCAN_X=0.35*ROI_WIDTH;

S44、以宽度为SCAN_X从上往下去扫左肩ROI，如果有值大于50，则返回该点坐标；S44, sweeping the left shoulder ROI from the top with a width of SCAN_X, if the value is greater than 50, returning the coordinates of the point;

S45、如果扫不到有值大于50，则以长度为SCAN_Y由右向左去扫左肩ROI，其中SCAN_Y为输入图像长度的0.7倍，即SCAN_Y＝0.7*ROI_HEIGHT如果有值大于50，则返回该点坐标；S45. If the value is greater than 50, the left shoulder ROI is swept from right to left by SCAN_Y, wherein SCAN_Y is 0.7 times the length of the input image, that is, SCAN_Y=0.7*ROI_HEIGHT, if the value is greater than 50, then return Point coordinates

步骤S5、利用求肤色区域最小外接矩形近端点并返回实现手部点检测，并获取左手点和右手点坐标；Step S5, using the minimum circumscribed rectangle near the end point of the skin color region and returning to realize the hand point detection, and acquiring the coordinates of the left hand point and the right hand point;

如图10所示，实现手部点检测的具体方法为：As shown in Figure 10, the specific method for implementing hand point detection is:

S51、将RGB转成YCrCb坐标系，存放在YUU中；S51, converting RGB into YCrCb coordinate system and storing in YUU;

S52、对YUU三通道分离并分别提取YUU各个通道中得特殊信息组合(Y<＝173,Cr<＝127,Cb>＝77)成新图，存放在BW中；S52. Separating the three channels of the YUU and separately extracting special information combinations (Y<=173, Cr<=127, Cb>=77) in each channel of the YUU into a new picture, and storing in the BW;

S53、对BW进行开运算(5*5处理窗)：去除噪点；S53, performing an open operation on the BW (5*5 processing window): removing noise;

S54、膨胀2次(3*3处理窗)：使图像平滑；S54, expansion 2 times (3*3 processing window): smooth the image;

S55、提取外轮廓；S55, extracting an outer contour;

S56、遍历外轮廓并提取最大面积对应的轮廓L；S56, traversing the outer contour and extracting a contour L corresponding to the largest area;

S57、新建L的最小外接矩形K；S57, the minimum external rectangle K of the newly created L;

S58、K满足以下条件时直接返回中心点：矩形宽度小于1.5倍矩形高度且矩形高度小于1.5倍矩形宽度；S58, K directly return to the center point when the following conditions are met: the rectangle width is less than 1.5 times the rectangle height and the rectangle height is less than 1.5 times the rectangle width;

S59、如不满足：S59, if not satisfied:

检测左手，将最左的点找出，定义为ptt[0]；Detect the left hand and find the leftmost point, defined as ptt[0];

判断次左的点，定义为ptt[1]；Determine the next left point, defined as ptt[1];

定义p1为K的中点，定义p2为ptt[0]和ptt[1]的中点；Define p1 as the midpoint of K and define p2 as the midpoint of ptt[0] and ptt[1];

由p1和p2的几何关系确定手的大体位置，并将坐标赋值给p2；Determine the general position of the hand by the geometric relationship of p1 and p2, and assign coordinates to p2;

返回p2；Return p2;

右手的处理同左手；The treatment of the right hand is the same as the left hand;

YCbCr格式可以从RGB格式线性变化得到，转换公式如下：The YCbCr format can be obtained by linearly changing from the RGB format. The conversion formula is as follows:

通过对大量皮肤像素的统计分析可以看到肤色聚类在色度空间中的很小的范围内，下述计算式判断是否属于皮肤区域：Through statistical analysis of a large number of skin pixels, it can be seen that the skin color cluster is in a small range in the chromaticity space, and the following calculation formula determines whether it belongs to the skin area:

(Cb>77And Cb<127)And(Cr>133And Cr<173)。(Cb>77And Cb<127) And(Cr>133And Cr<173).

步骤S6、利用将手部ROI划分为三个区域，各个区域分别用不同的扫描方式返回点实现肘部点检测，并获取左肘点和右肘点坐标；Step S6, the hand ROI is divided into three regions, each region uses different scanning mode return points to realize elbow point detection, and obtain left elbow point and right elbow point coordinates;

如图11-图12所示，实现肘部点检测的方法为：As shown in Figures 11-12, the method for achieving elbow point detection is:

S61、利用将手部ROI划分为三个区域，三个区域如图11所示，各个区域分别用不同的扫描方式返回点实现肘部点识别；S61. The hand ROI is divided into three regions, and the three regions are as shown in FIG. 11, and each region uses different scanning manner return points to realize elbow point recognition;

S62、图像预处理获取人体外轮廓S62, image preprocessing to obtain the contour of the human body

S63、取左肘部ROIS63, take the left elbow ROI

S64、将ROI分为三个区域，分别对应举手，偏45度，叉腰向下这三种姿势S64, the ROI is divided into three regions, respectively corresponding to raising the hand, 45 degrees, and the hips are downward.

S65、当肩部点横坐标与手部点横坐标差值大于IMAGE_HEIGHT/50时：S65. When the difference between the abscissa of the shoulder point and the abscissa of the hand point is greater than IMAGE_HEIGHT/50:

举手动作(区域一)：当手点纵坐标与肩部点纵坐标差值小于阈值IMAGE_HEIGHT/5时Raise hand movement (Zone 1): When the difference between the ordinate of the hand point and the ordinate of the shoulder point is less than the threshold IMAGE_HEIGHT/5

即HAND.y-SHOULDER.y<IMAGE_HEIGHT/5，则从下往上扫点，扫到就返回That is, HAND.y-SHOULDER.y<IMAGE_HEIGHT/5, then sweep from the bottom up, sweep back and return

偏45度向下(区域二)：当手点纵坐标与肩部点纵坐标差值大于阈值IMAGE_HEIGHT/5时45 degrees downward (zone 2): when the difference between the ordinate of the hand point and the ordinate of the shoulder point is greater than the threshold IMAGE_HEIGHT/5

即HAND.y-SHOULDER.y>IMAGE_HEIGHT/5，则从右往左扫点(以ROI往下8个像素取点横向扫)，扫到点的像素值大于就返回That is, HAND.y-SHOULDER.y>IMAGE_HEIGHT/5, sweep the point from right to left (swipe horizontally with the ROI down to 8 pixels), and sweep back to the point where the pixel value is greater than return

叉腰动作(区域三)：当肩部点横坐标与手部点横坐标差值小于IMAGE_HEIGHT/50时：Forklift action (Zone 3): When the difference between the abscissa of the shoulder point and the abscissa of the hand point is less than IMAGE_HEIGHT/50:

即SHOULDER.x–HAND.x<IMAGE_HEIGHT/50时，则从左向右扫点，返回第一个像素值大于50的点的坐标；That is, when SHOULDER.x-HAND.x<IMAGE_HEIGHT/50, the point is swept from left to right, and the coordinates of the point where the first pixel value is greater than 50 are returned;

右肩点的识别与左肩点同。The identification of the right shoulder point is the same as the left shoulder.

步骤S7、最后统计各点可信度并将可信点显示。In step S7, the credibility of each point is finally counted and the trusted point is displayed.

作为上述实施例的一个优化方案，本实施例二维视频流中的人体骨骼点追踪方法，该方法还包括下述步骤：As an optimization scheme of the foregoing embodiment, the human skeleton point chase in the two-dimensional video stream in this embodimentTracking method, the method further comprises the following steps:

S8、利用求下半身前景区域最小外接矩形近端点并返回实现脚点检测，所述脚点检测的具体方法为：S8, using the near-end point of the minimum circumscribed rectangle of the foreground area of the lower body and returning to realize the foot point detection, and the specific method of the foot point detection is:

S81在全身模式下，以屏幕一半取出前景图的人体下半身ROI；S81 in the whole body mode, take the foreground half of the human body lower body ROI with half of the screen;

S82、提取外轮廓，遍历外轮廓并提取最大面积对应的轮廓L，新建L的最小外接矩形K；S82, extracting the outer contour, traversing the outer contour and extracting the contour L corresponding to the largest area, and newly creating a minimum circumscribed rectangle K of L;

S83、K满足以下条件时直接返回中心点：矩形宽度小于1.5倍矩形高度且矩形高度小于1.5倍矩形宽度S83, K directly return to the center point when the following conditions are satisfied: the rectangle width is less than 1.5 times the rectangle height and the rectangle height is less than 1.5 times the rectangle width

S84、如不满足：S84, if not satisfied:

返回p2；Return p2;

S9、利用脚点往上取0.2倍人体高度的距离进行扫描并返回的方法实现膝部点检测，所述膝部点检测的具体方法为：S9: Performing a knee point detection by scanning and returning a distance of 0.2 times the height of the human body from the foot point, and the specific method of the knee point detection is:

S91、背景重构模块获取人体前景，在全身模式下，取下半身人体ROI；S91. The background reconstruction module acquires the foreground of the human body, and in the whole body mode, removes the body ROI of the lower body;

S92、获取人体高度BODY_HEIGHT,BODY_HEIGHT＝FOOT_LEFT_Y–FACE_Y+FACE_HEIGHT/2；S92. Obtain the height of the human body BODY_HEIGHT, BODY_HEIGHT=FOOT_LEFT_Y–FACE_Y+FACE_HEIGHT/2;

S93、取左脚部ROI，其尺寸记为(ROI_HEIGHT,ROI_WIDTH)；S93, taking the left foot ROI, the size of which is recorded as (ROI_HEIGHT, ROI_WIDTH);

S94、设置SCAN_Y，SCAN_Y为用户高度的0.2倍，即SCAN_Y＝0.2*BODY_HEIGHT；S94, set SCAN_Y, SCAN_Y is 0.2 times the height of the user, ie SCAN_Y=0.2*BODY_HEIGHT;

S95、以FOOT_LEFT_Y以上SCAN_Y的高度从左往右去扫左脚ROI，如果有值大于50，则返回该点坐标(x+12,y)，其中x+12表示对横坐标做一个12像素的偏移处理，使得膝部点处于膝部的中心位置；S95. Scan the left foot ROI from left to right with the height of FOOT_LEFT_Y and above SCAN_Y. If the value is greater than 50, return the coordinate of the point (x+12, y), where x+12 indicates that the horizontal coordinate is 12 pixels. Offset processing so that the knee point is at the center of the knee;

S96、如果扫不到有值大于50，则返回(0,0)，并置为不可信点；S96. If the value cannot be swept, the value returns to (0, 0), and is set as an untrusted point;

经过上述的步骤S1-S9，完成对所有整体关键点的识别，如图13所示。After the above steps S1-S9, the identification of all the overall key points is completed, as shown in FIG.

本实施例中，在S1的步骤中，由于现实场景下光照不均和掩膜易受人的影子影响等问题，需要对前景提取模块中得到的掩膜进行优化，使得其能适应光照不均的情况。主要使用GI滤波函数进行掩膜优化，其具体方法为：In this embodiment, in the step of S1, due to the problem of uneven illumination and the shadow of the mask in the real scene, the mask obtained in the foreground extraction module needs to be optimized, so that it can adapt to uneven illumination. Case. The GI filter function is mainly used for mask optimization, and the specific method is as follows:

对输入掩膜进行高斯滤波消除高斯噪声，高斯滤波预设参数为：处理窗口大小为15x15，sigma为20；Gaussian filtering is performed on the input mask to eliminate Gaussian noise. The Gaussian filtering preset parameters are: processing window size is 15x15, sigma is 20;

对消噪后的掩膜图像应用GI滤波，得到0-1过渡图像，GI滤波预设参数为：处理窗大小为8x8，惩罚参数为51；Applying GI filtering to the mask image after denoising, the 0-1 transition image is obtained. The preset parameters of the GI filter are: the processing window size is 8x8, and the penalty parameter is 51;

GI滤波算法，输入为彩色图I和初始掩膜P，输出为结合彩色图边缘信息做补全的优化掩膜，过程如下：The GI filtering algorithm inputs the color map I and the initial mask P, and the output is an optimized mask for complementing the edge information of the color map. The process is as follows:

Algorithm 1.Guided Filter.Algorithm 1.Guided Filter.

Input：filtering input image p，guidance image I，radius r，regularization∈Input:filtering input image p,guidance image I,radius r,regularization∈

Output：filtering output q.Output:filtering output q.

1：mean_I＝f_mean(I)1:mean_I =f_mean (I)

mean_p＝f_mean(p)Mean_p =f_mean (p)

corr_I＝f_mean(I.*I)Corr_I =f_mean (I.*I)

corr_Ip＝f_mean(I.*p)Corr_Ip =f_mean (I.*p)

2：var_I＝corr_I-mean_I.*mean_I2:var_I =corr_I -mean_I .*mean_I

cov_Ip＝corr_Ip-mean_I.*mean_pCov_Ip =corr_Ip -mean_I .*mean_p

3：a＝cov_Ip./(var_I+∈)3: a=cov_Ip ./(var_I +∈)

b＝mean_p-a.*mean_Ib=mean_p -a.*mean_I

4：mean_a＝f_mean(a)4: mean_a = f_mean (a)

mean_b＝f_mean(b)Mean_b =f_mean (b)

5：q＝mean_a.*I+mean_b5:q=mean_a .*I+mean_b

/^*f_mean is a mean filter with a wide variety of O(N)time methods.^*//^* f_mean is a mean filter with a wide variety of O(N)time methods.^* /

其中，mean表示获取图像均值，corr表示求二次矩均值；第2步求图像局部方差；第三步计算线性系数a和b；第4步计算系数均值；第5步实现信息补全。Among them, mean means to obtain the mean value of the image, corr means to find the mean value of the second moment; the second step to find the local variance of the image; the third step to calculate the linear coefficients a and b; the fourth step to calculate the mean value of the coefficient; the fifth step to achieve information completion.

使用3x3的处理窗进行开操作，进一步消除空洞点和离散点；Use the 3x3 processing window to open the operation to further eliminate the void points and discrete points;

寻找掩膜最大连通域，再次进行高斯滤波得到优化的掩膜，高斯滤波预设参数为：处理窗口大小为15x15，sigma为20。Find the maximum connected domain of the mask, and then perform Gaussian filtering to obtain an optimized mask. The Gaussian filter preset parameters are: processing window size is 15x15, sigma is 20.

本发明还公开了一种二维视频流中的人体骨骼点追踪系统，该系统包括：The invention also discloses a human skeleton point tracking system in a two-dimensional video stream, the system comprising:

判断模块，用于判断头部点是否在屏幕中，如果否，则继续进行人脸检测模块；如果是，则将人体分为左部分ROI和右部分ROI分别进行其他关键点的检测；a judging module, configured to determine whether the head point is in the screen, and if not, proceeding to the face detecting module; if yes, dividing the human body into a left part ROI and a right part ROI for performing other key points respectivelyDetection

除上述主要模块外，该系统还包括脚点检测模块和膝部点检测模块；In addition to the above main modules, the system further includes a foot point detection module and a knee point detection module;

所述膝部点检测模块，用于利用脚点往上取0.2倍人体高度的距离进行扫描并返回的方法实现膝部点检测。The knee point detecting module is configured to perform knee point detection by scanning and returning with a distance of 0.2 times the height of the human body.

上述实施例为本发明较佳的实施方式，但本发明的实施方式并不受上述实施例的限制，其他的任何未背离本发明的精神实质与原理下所作的改变、修饰、替代、组合、简化，均应为等效的置换方式，都包含在本发明的保护范围之内。The above embodiments are preferred embodiments of the present invention, but the embodiments of the present invention are not limited to the above embodiments, and any other changes, modifications, substitutions, combinations, and combinations thereof may be made without departing from the spirit and scope of the invention. Simplifications should all be equivalent replacements and are included in the scope of the present invention.