CN107330424B - Interaction area and interaction time period identification method, storage device and mobile terminal - Google Patents
Interaction area and interaction time period identification method, storage device and mobile terminalDownload PDFInfo
- Publication number
- CN107330424B CN107330424BCN201710655807.2ACN201710655807ACN107330424BCN 107330424 BCN107330424 BCN 107330424BCN 201710655807 ACN201710655807 ACN 201710655807ACN 107330424 BCN107330424 BCN 107330424B
- Authority
- CN
- China
- Prior art keywords
- area
- interaction
- image
- interactive
- record
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
- G06V40/103—Static body considered as a whole, e.g. static pedestrian or occupant recognition
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/22—Matching criteria, e.g. proximity measures
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
- G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
- G06F18/2411—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches based on the proximity to a decision surface, e.g. support vector machines
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/70—Denoising; Smoothing
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/90—Dynamic range modification of images or parts thereof
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/20—Analysis of motion
- G06T7/254—Analysis of motion involving subtraction of images
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/40—Extraction of image or video features
- G06V10/50—Extraction of image or video features by performing operations within image blocks; by using histograms, e.g. histogram of oriented gradients [HoG]; by summing image-intensity values; Projection analysis
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10016—Video; Image sequence
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20024—Filtering details
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30196—Human being; Person
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- Multimedia (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Artificial Intelligence (AREA)
- Life Sciences & Earth Sciences (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Bioinformatics & Computational Biology (AREA)
- Evolutionary Biology (AREA)
- Evolutionary Computation (AREA)
- General Engineering & Computer Science (AREA)
- Human Computer Interaction (AREA)
- Image Analysis (AREA)
Abstract
Translated fromChinese
Description
Translated fromChinese技术领域technical field
本发明属于计算机视觉识别领域，特别涉及一种差分像素的互动区域与互动时间段识别方法，还涉及一种可实现上述功能的存储设备及移动终端。The invention belongs to the field of computer vision recognition, in particular to a method for recognizing an interactive area and an interactive time period of differential pixels, and also relates to a storage device and a mobile terminal capable of realizing the above functions.
背景技术Background technique
随着科学技术发展与现代视频技术的广泛运用，基于机器是视觉的图像处理与模式识别方法越来越多的被运用到模式识别、运动分析、视频监控和人工智能等领域。With the development of science and technology and the wide application of modern video technology, more and more image processing and pattern recognition methods based on machine vision are applied to the fields of pattern recognition, motion analysis, video surveillance and artificial intelligence.
现有的与人相关的算法，大多基于模型的检测算法，需要通过特定的算子或模型进行整幅图像的匹配计算，大大提高了运算消耗、损失了运算效率与实时性。当前存在对于人体活动区域产生先验区域的算法，大多无法兼顾实时性和精度要求，在区域判定时对于人体位移型活动(如：行走、奔跑、爬行等)和互动型活动(在图像上显示为动、静状态转换的活动，运动停止与某件空间内物体进行互动，如：起立坐下、出门进门、喝水等)这两种人体活动状态并未有明显区分，没有针对人体互动型活动区域的识别。并且，现有的算法大多只关注空间内的标定，而没有进行活动对应的时间上的记录，没有完整的记录一个人体的活动区域和在该区域活动的时间。Most of the existing human-related algorithms are model-based detection algorithms, which require a specific operator or model to perform matching calculations for the entire image, which greatly increases computational consumption and loses computational efficiency and real-time performance. There are currently algorithms that generate a priori area for human activity areas, but most of them cannot take into account the real-time and accuracy requirements. It is an activity that changes between dynamic and static states, and the movement stops to interact with objects in a certain space, such as standing up and sitting down, going out and entering the door, drinking water, etc.) There is no obvious distinction between these two human activity states, and there is no interaction type for the human body. Identification of active areas. In addition, most of the existing algorithms only focus on the calibration in space, but do not record the time corresponding to the activity, and do not completely record the activity area of a human body and the time spent in this area.
发明内容SUMMARY OF THE INVENTION
针对现有技术中存在的技术问题，本发明提供一种互动区域与互动时间段识别方法、存储设备及移动终端，其可以弥补现有技术的不足，提高数学模型的应用效果。Aiming at the technical problems existing in the prior art, the present invention provides a method for identifying an interactive area and an interactive time period, a storage device and a mobile terminal, which can make up for the deficiencies of the prior art and improve the application effect of the mathematical model.
本发明所提供的一种差分像素的互动区域与互动时间段识别方法，适于在计算设备中执行，该方法包括：A method for identifying an interactive area and an interactive time period of differential pixels provided by the present invention is suitable for execution in a computing device, and the method includes:
接收输入的视频信号，并将所输入的视频信号进行拆帧处理，以生成单帧图像；Receive the input video signal, and deframe the input video signal to generate a single frame image;
对得到的单帧图像进行人形检测；Perform humanoid detection on the obtained single-frame image;
进行全图差分，并记录差分数据；Perform full-image difference and record the difference data;
判断前N帧的图像像素差分数据是否满足人体互动特征模型，并记录触发的互动区域与时间断点以及终止的互动区域与时间断点；以及Determine whether the image pixel difference data of the first N frames satisfies the human interaction feature model, and record the triggered interaction area and time breakpoint as well as the terminated interaction area and time breakpoint; and
记录人体互动区域和时间段。Record the human interaction area and time period.
其中，所述步骤“对得到的单帧图像进行人形检测”包括：Wherein, the step "performing humanoid detection on the obtained single-frame image" includes:
将得到的单帧图像从RGB转换为灰度图像，并对转换后的灰度图像做平滑去噪滤波处理；以及Convert the obtained single-frame image from RGB to grayscale image, and perform smoothing and denoising filtering on the converted grayscale image; and
运用人像HOG算子，进行人像检测。Use the portrait HOG operator to perform portrait detection.
其中，所述步骤“运用人像HOG算子进行人像检测”包括：Wherein, the step "using the portrait HOG operator to perform portrait detection" includes:
进行Gamma校正；Perform Gamma correction;
将图像转灰度；Convert the image to grayscale;
计算图像的梯度与方向，以得到图像的梯度振幅与角度；Calculate the gradient and direction of the image to obtain the gradient amplitude and angle of the image;
8×8网格方向梯度权重直方图统计；以及8×8 grid orientation gradient weight histogram statistics; and
块描述子与特征向量归一化。The block descriptor is normalized to the feature vector.
其中，所述步骤“进行全图差分，并记录差分数据”包括：Wherein, the step of "performing the full image difference and recording the difference data" includes:
分别对全图进行两帧差分、三帧差分及五帧差分，并记录对应的差分结果；以及Perform two-frame difference, three-frame difference and five-frame difference respectively on the whole image, and record the corresponding difference results; and
通过人体运动统计模型将人体一般运动情况下的区域内像素变换运用一维正态分布进行描述，并滤除非人体活动的差分数据。The human body motion statistical model is used to describe the pixel transformation in the region under the general motion of the human body with a one-dimensional normal distribution, and the difference data of non-human activities is filtered.
其中，所述步骤“判断前N帧的图像像素差分数据是否满足人体互动特征模型，并记录触发的互动区域与时间断点以及终止的互动区域与时间断点”包括：Wherein, the step of "judging whether the image pixel difference data of the first N frames meets the human interaction feature model, and recording the triggered interaction area and time breakpoint as well as the terminated interaction area and time breakpoint" includes:
基于获得的人体活动差分数据，运用人体位移活动模型判断差分数据是否满足人体位移活动类型；Based on the obtained human body movement differential data, use the human body displacement activity model to determine whether the differential data meets the human body displacement activity type;
若确定不为位移运动，则将N帧差分数据与人体互动触发模型进行匹配；If it is determined that it is not displacement motion, then match the N frames of differential data with the human interaction trigger model;
若成功匹配，则记录当前触发的互动区域与时间断点；If the match is successful, the currently triggered interactive area and time breakpoint will be recorded;
将N帧差分数据与人体互动终止模型进行匹配；以及matching N frames of differential data to a human interaction termination model; and
若成功匹配，记录当前终止的互动区域与时间断点。If the match is successful, record the currently terminated interactive area and time breakpoint.
其中，所述步骤“记录人体互动区域和时间段”包括：Wherein, the step "recording human interaction area and time period" includes:
计算互动触发区域与终止区域的重合部分，记录为人体活动区域；Calculate the overlapping part of the interactive trigger area and the termination area, and record it as the human activity area;
计算互动触发与终止的间隔时间，并与记录的活动区域做对应；以及Calculate the time interval between the trigger and the termination of the interaction, and map it to the recorded activity area; and
进行同一空间下的多组视频检测，记录所有互动触发区域，并对所有相邻M像素范围内的互动触发区域进行求交集操作，获取交集部分的区域，该交集部分的区域即为最终获得的空间内互动频率最高的区域。Perform multiple sets of video detection in the same space, record all interactive trigger areas, and perform intersection operation on all interactive trigger areas within the range of adjacent M pixels to obtain the area of the intersection part, which is the final obtained area. The area with the highest interaction frequency in the space.
进一步的，还包括：通过不同颜色标记不同互动频率的区域。Further, it also includes: marking regions with different interaction frequencies with different colors.
本发明还提供了一种存储设备，其中存储有多条指令，所述指令适于由处理器加载并执行，所述指令包括：The present invention also provides a storage device in which a plurality of instructions are stored, the instructions are suitable for being loaded and executed by a processor, and the instructions include:
接收输入的视频信号，并将所输入的视频信号进行拆帧处理，以生成单帧图像；Receive the input video signal, and deframe the input video signal to generate a single frame image;
对得到的单帧图像进行人形检测；Perform humanoid detection on the obtained single-frame image;
进行全图差分，并记录差分数据；Perform full-image difference and record the difference data;
判断前N帧的图像像素差分数据是否满足人体互动特征模型，并记录触发的互动区域与时间断点以及终止的互动区域与时间断点；以及Determine whether the image pixel difference data of the first N frames satisfies the human interaction feature model, and record the triggered interaction area and time breakpoint as well as the terminated interaction area and time breakpoint; and
记录人体互动区域和时间段。Record the human interaction area and time period.
本发明还提供了一种移动终端，包括：The present invention also provides a mobile terminal, comprising:
处理器，适于实现各指令；以及a processor adapted to implement the instructions; and
存储设备，适于存储多条指令，所述指令适于由处理器加载并执行，所述指令包括：A storage device adapted to store a plurality of instructions adapted to be loaded and executed by a processor, the instructions comprising:
接收输入的视频信号，并将所输入的视频信号进行拆帧处理，以生成单帧图像；Receive the input video signal, and deframe the input video signal to generate a single frame image;
对得到的单帧图像进行人形检测；Perform humanoid detection on the obtained single-frame image;
进行全图差分，并记录差分数据；Perform full-image difference and record the difference data;
判断前N帧的图像像素差分数据是否满足人体互动特征模型，并记录触发的互动区域与时间断点以及终止的互动区域与时间断点；以及Determine whether the image pixel difference data of the first N frames satisfies the human interaction feature model, and record the triggered interaction area and time breakpoint as well as the terminated interaction area and time breakpoint; and
记录人体互动区域和时间段。Record the human interaction area and time period.
本发明所述的差分像素的互动区域与互动时间段识别方法的算法，主要通过对人体活动在图像上产生的像素变化进行统计，进而结合人体活动模型，进行人体普通位置移动和空间互动两种状态的判定，最终得出人体互动区域与互动时间段。The algorithm of the method for identifying the interactive area and the interactive time period of the differential pixels of the present invention mainly calculates the pixel changes generated by the human body movement on the image, and then combines the human body movement model to carry out two types of human body position movement and spatial interaction. The state is judged, and finally the human interaction area and interaction time period are obtained.
附图说明Description of drawings
图1是本发明所述的一种差分像素的互动区域与互动时间段识别方法的较佳实施方式的流程图。FIG. 1 is a flowchart of a preferred embodiment of a method for identifying an interactive area and an interactive time period of differential pixels according to the present invention.
图2是图1中步骤S2的具体流程图。FIG. 2 is a specific flowchart of step S2 in FIG. 1 .
图3是图1中步骤S3的具体流程图。FIG. 3 is a specific flowchart of step S3 in FIG. 1 .
图4是图1中步骤S4的具体流程图。FIG. 4 is a specific flowchart of step S4 in FIG. 1 .
图5是图1中步骤S5的具体流程图。FIG. 5 is a specific flowchart of step S5 in FIG. 1 .
具体实施方式Detailed ways
为了使本发明实现的技术手段、创作特征、达成目的与功效易于明白了解，下面结合具体图示，进一步阐述本发明。In order to make it easy to understand the technical means, creation features, achieved goals and effects of the present invention, the present invention will be further described below with reference to the specific figures.
在本发明的描述中，需要说明的是，除非另有明确的规定和限定，术语“安装”、“相连”、“连接”应做广义理解，例如，可以是固定连接，也可以是可拆卸连接，或一体地连接；可以是机械连接，也可以是电连接；可以是直接相连，也可以通过中间媒介间接相连，可以是两个元件内部的连通。对于本领域的普通技术人员而言，可以具体情况理解上述术语在本发明中的具体含义。In the description of the present invention, it should be noted that the terms "installed", "connected" and "connected" should be understood in a broad sense, unless otherwise expressly specified and limited, for example, it may be a fixed connection or a detachable connection Connection, or integral connection; can be mechanical connection, can also be electrical connection; can be directly connected, can also be indirectly connected through an intermediate medium, can be internal communication between two elements. For those of ordinary skill in the art, the specific meanings of the above terms in the present invention can be understood in specific situations.
本发明中，人体位移型活动包括诸如行走、跑动、爬行等，为人体产生位移的行为状态；互动性活动包括诸如起立、做下、卧躺、喝水、与物体互动(桌子、椅子等)等小位移行为，区分前述二者可以将人的行为基本分为两类，有助于进一步的姿态识别和行为分析。In the present invention, human body displacement activities include behavioral states such as walking, running, crawling, etc., which generate displacement for the human body; interactive activities include such as standing up, doing down, lying down, drinking water, interacting with objects (tables, chairs, etc. ) and other small displacement behaviors, distinguishing the above two can basically divide human behavior into two categories, which is helpful for further gesture recognition and behavior analysis.
请参考图1所示，其为本发明所述的一种差分像素的互动区域与互动时间段识别方法的较佳实施方式的流程图。所述差分像素的互动区域与互动时间段识别方法的较佳实施方式包括以下步骤：Please refer to FIG. 1 , which is a flowchart of a preferred embodiment of a method for identifying an interactive region and an interactive time period of a differential pixel according to the present invention. A preferred embodiment of the method for identifying the interactive area and the interactive time period of the differential pixel includes the following steps:
步骤S1：接收所输入的视频信号，并将所输入的视频信号进行拆帧处理，以生成单帧图像。Step S1: Receive the input video signal, and perform frame splitting processing on the input video signal to generate a single frame image.
帧率是指视频格式每秒钟播放的静态画面数量，因此可将视频信号拆解为若干静态画面，即拆帧。现有很多软件均可实现拆帧功能，在此不再赘述。Frame rate refers to the number of static images played per second in a video format, so the video signal can be split into several static images, that is, frame splitting. Many existing software can implement the frame splitting function, which will not be repeated here.
步骤S2：对由步骤S1中得到的单帧图像进行人形检测。Step S2: Perform humanoid detection on the single-frame image obtained in Step S1.
人形检测(Human shape recognition，简称HSR)技术是指利用人体成像的一定特征，通过对图形图像的处理，最终在成像空间中发现识别和定位人形目标的技术。人形检测(HSR)是计算机视觉、模式识别、图像处理技术和形态学技术的融合，其可被广泛的应用到智能监控、智能交通、目标循迹、目标跟踪等领域。人形检测的实现过程可以分为目标检测、边界提取、人形目标匹配和人形目标识别等过程。Human shape recognition (HSR) technology refers to the technology of identifying and locating human-shaped targets in the imaging space by using certain features of human imaging and processing graphic images. Humanoid detection (HSR) is a fusion of computer vision, pattern recognition, image processing technology and morphological technology, which can be widely used in intelligent monitoring, intelligent transportation, target tracking, target tracking and other fields. The realization process of humanoid detection can be divided into target detection, boundary extraction, humanoid target matching and humanoid target recognition.
具体的，请参图2所示，人形检测可通过下述方式进行：Specifically, as shown in Figure 2, the humanoid detection can be performed in the following ways:
步骤S21：将由步骤S1中得到的单帧图像从RGB转换为灰度图像，并对转换后的灰度图像做平滑去噪滤波处理。Step S21: Convert the single-frame image obtained in step S1 from RGB to a grayscale image, and perform smoothing and denoising filtering on the converted grayscale image.
步骤S22：运用人像HOG算子，进行人像检测。Step S22: Use the portrait HOG operator to perform portrait detection.
HOG(Histogram of Oriented Gradient，梯度方向直方图)特征是对象识别与模式匹配中的特征提取算法，是基于本地像素块进行特征直方图提取的一种算法。通过HOG特征提取以及SVM(Support Vector Machine，支持向量机)训练，可以得到很好的效果。HOG特征提取的大致流程如下：第一步，Gamma校正，通过相反的非线性转换把该转换反转输出来,主要是对输入图像进行校正，以补偿显示器带来的灰度偏差，常见的系数在2.5左右。第二步，将图像转灰度。第三步，计算图像的梯度与方向，最终得到图像的梯度振幅与角度。第四步，8×8网格方向梯度权重直方图统计。第五步，块内归一化梯度直方图。最终即可实现人形检测的功能。HOG (Histogram of Oriented Gradient, histogram of gradient orientation) feature is a feature extraction algorithm in object recognition and pattern matching, and an algorithm for feature histogram extraction based on local pixel blocks. Through HOG feature extraction and SVM (Support Vector Machine) training, good results can be obtained. The general process of HOG feature extraction is as follows: the first step, Gamma correction, the conversion is inverted and output through the opposite nonlinear transformation, mainly to correct the input image to compensate for the grayscale deviation brought by the display, common coefficients around 2.5. The second step is to convert the image to grayscale. The third step is to calculate the gradient and direction of the image, and finally get the gradient amplitude and angle of the image. The fourth step, 8×8 grid direction gradient weight histogram statistics. The fifth step is to normalize the gradient histogram within the block. Finally, the function of humanoid detection can be realized.
步骤S3：进行全图差分，并记录差分数据。Step S3: Perform a full-image difference, and record the difference data.
具体的，请参图3所示，全图差分可通过下述方式进行：Specifically, as shown in Figure 3, the full-image difference can be performed in the following ways:
步骤S31：分别对全图进行两帧差分、三帧差分及五帧差分，并记录对应的差分结果，分别记为：diff1、diff2及diff3。Step S31: Perform two-frame difference, three-frame difference, and five-frame difference on the whole image, and record the corresponding difference results, which are respectively recorded as diff1, diff2, and diff3.
其中，两帧差分是指将相邻的两帧图像进行相减，得到两帧图像亮度差的绝对值，判断它是否大于阈值来分析视频或图像序列的运动特性，确定图像序列中有无物体运动。三帧差分是指选取任意相邻的三帧图像，分别对前两帧图像和后两帧图像做图像差值运算，之后对上述两个帧差图像阈值化的结构进行“与”运算，从而提取出目标图像，可用于描述图像像素变化剧烈程度。同理，五帧差分是指选取相邻的五帧图像，以第K帧图像作为当前帧，分别与前两帧和后两帧进行帧差运算，得到四个差帧运算结果，对其四个差帧运算结果进行二值化处理，之后对处理之后的四个运算结果先进行“与”运算再进行“或”运算，即可得到图像的目标轮廓。本实施方式中采用两帧差分、三帧差分及五帧差分的目的在于上述三种差分计算方式的计算量较小，可以满足算法实时性要求。通过差分数据获取的差分数据，可用于分析运动类型(位移型和互动型)。Among them, the two-frame difference refers to subtracting two adjacent frames of images to obtain the absolute value of the brightness difference between the two frames of images, judging whether it is greater than the threshold value to analyze the motion characteristics of the video or image sequence, and determine whether there are objects in the image sequence. sports. Three-frame difference refers to selecting any adjacent three frames of images, respectively performing image difference operation on the first two frames of images and the last two frames of images, and then performing an "AND" operation on the thresholded structure of the above two frame difference images, thereby The target image is extracted, which can be used to describe the sharpness of image pixel changes. Similarly, five-frame difference refers to selecting five adjacent frames of images, taking the K-th frame image as the current frame, and performing frame difference operations with the first two frames and the last two frames respectively to obtain four difference frame operation results. Perform binarization processing on the operation results of each difference frame, and then perform an "AND" operation and then an "OR" operation on the four operation results after the processing, and then the target contour of the image can be obtained. The purpose of using the two-frame difference, the three-frame difference, and the five-frame difference in this embodiment is that the calculation amount of the above three difference calculation methods is small, which can meet the real-time requirements of the algorithm. Differential data obtained from differential data can be used to analyze motion types (displacement type and interactive type).
步骤S32：通过人体运动统计模型(即人体运动正态分布)将人体一般运动情况下的区域内像素变换运用一维正态分布进行描述，并滤除非人体活动的差分数据。本实施方式中，此处的统计基于多样本进行统计，以确保统计的准确性。Step S32: Using a human body motion statistical model (ie, human body motion normal distribution) to describe the pixel transformation in the region under the general human motion situation using a one-dimensional normal distribution, and filter out the difference data of non-human activity. In this embodiment, the statistics here are based on multiple samples to ensure the accuracy of the statistics.
步骤S4：判断前20帧的图像像素差分数据是否满足人体互动特征模型。本实施方式中选取20帧图像作为判断标准，其他实施方式中亦可选取其他帧数的图像作为判断标准。Step S4: Determine whether the image pixel difference data of the first 20 frames satisfies the human interaction feature model. In this embodiment, 20 frames of images are selected as the judgment criterion, and in other embodiments, images of other frames may also be selected as the judgment criterion.
具体的，请参图4所示，判断是否满足人体互动特征模型可通过下述方式进行：Specifically, as shown in FIG. 4 , judging whether the human interaction feature model is satisfied can be performed in the following ways:
步骤S41：基于获得的人体活动差分数据，运用人体位移活动模型判断差分数据是否满足人体位移活动类型。本实施方式中，所述人体位移活动模型系根据差分数据的统计数据所得出的正态分布模型，其中数据样本为产生位移型运动的20帧图像差分数据。当差分数据|x-E|<＝σ时，则判定为成功落在分布内，即被判定为人体位移活动，其中x表示图像差分得到的数据，E表示统计模型中的人体位移型活动差分值的平均值，σ表示图像差分数据统计中的计算得出的差分数据方差。Step S41: Based on the obtained human body movement differential data, use a human body displacement movement model to determine whether the differential data satisfies the human body displacement movement type. In this embodiment, the human body displacement activity model is a normal distribution model obtained according to statistical data of differential data, wherein the data samples are differential data of 20 frames of images that generate displacement-type motion. When the difference data |x-E|<=σ, it is determined to fall within the distribution successfully, that is, it is determined to be human body displacement activity, where x represents the data obtained by image difference, and E represents the difference value of the human body displacement type activity in the statistical model. The mean value, σ represents the calculated difference data variance in the image difference data statistics.
本实施方式中选取20帧图像作为数据样本，其他实施方式中亦可选取其他帧数的图像作为数据样本。In this embodiment, 20 frames of images are selected as data samples. In other embodiments, images of other frames may also be selected as data samples.
步骤S42：若确定不为位移运动，则将20帧差分数据diff与人体互动触发模型进行匹配(以两帧差分diff1为例，diff＝∑diff1i，i∈{[1,20],i∈N})。Step S42: If it is determined that it is not displacement motion, match the 20-frame differential data diff with the human interaction trigger model (taking two-frame differential diff1 as an example, diff=∑diff1i , i∈{[1,20],i∈ N}).
步骤S43：若成功匹配，则记录当前触发的互动区域与时间断点。若匹配不成功，则返回至步骤S42持续进行匹配。Step S43: If the match is successful, record the currently triggered interactive area and time breakpoint. If the matching is unsuccessful, return to step S42 to continue matching.
步骤S44：循环检测，将20帧差分数据与人体互动终止模型进行匹配。Step S44: Loop detection, matching the 20 frames of differential data with the human interaction termination model.
步骤S45：若成功匹配，记录当前终止的互动区域与时间断点。若匹配不成功，则返回至步骤S44持续进行匹配。Step S45: If the match is successful, record the currently terminated interactive area and time breakpoint. If the matching is unsuccessful, return to step S44 to continue matching.
本实施方式中，该人体互动终止模型为根据差分数据的统计数据得出的正态分布模型，数据样本为人体停止运动前后10s的图像差分数据。当差分数据|x-E|<＝σ时，则判定为成功落在分布内，即被判定为人体互动触发模型。其中x表示图像差分得到的数据，E表示统计模型中的人体互动型活动的图像差分值的平均值，σ表示图像差分数据统计中计算得出的方差。In this embodiment, the human interaction termination model is a normal distribution model obtained according to statistical data of differential data, and the data sample is image differential data 10s before and after the human body stops moving. When the difference data |x-E|<=σ, it is determined that it falls within the distribution successfully, that is, it is determined to be a human interaction trigger model. Among them, x represents the data obtained by image difference, E represents the average value of image difference values of human interaction activities in the statistical model, and σ represents the variance calculated in the image difference data statistics.
步骤S5：记录人体互动区域和时间段。Step S5: Record the human interaction area and time period.
具体的，请参图5所示，所述步骤S5具体包括以下步骤：Specifically, as shown in FIG. 5 , the step S5 specifically includes the following steps:
步骤S51：计算互动触发区域与终止区域的重合部分，记录为人体活动区域。上述互动触发区域与终止区域由步骤S42及S43所得到。Step S51: Calculate the overlapping part of the interactive trigger area and the termination area, and record it as the human body activity area. The above-mentioned interactive trigger area and termination area are obtained in steps S42 and S43.
步骤S52：计算互动触发与终止的间隔时间，并与记录的活动区域做对应。上述互动触发与终止的见个时间由步骤S42及S43所得到。Step S52: Calculate the interval time between interactive triggering and termination, and make it correspond to the recorded activity area. The above-mentioned time for triggering and terminating the interaction is obtained in steps S42 and S43.
步骤S53：进行同一空间下的多组视频检测，记录所有互动触发区域(area1，area2，area3，……，arean)，并对所有相邻50像素范围内的互动触发区域进行求交集操作，获取交集部分的区域，该交集部分的区域即为最终获得的空间内互动频率最高的区域area。本实施方式中，将所有相邻50像素范围内的互动触发区域进行求交集操作中的“相邻50像素”仅为一示例，其他实施方式中亦可根据用户的需求另行选择。Step S53: Perform multiple sets of video detection in the same space, record all interactive trigger areas (area1 , area2 , area3 , ..., arean ), and calculate all the interactive trigger areas within the adjacent 50-pixel range. In the intersection operation, the area of the intersection part is obtained, and the area of the intersection part is the area with the highest interaction frequency in the final obtained space. In this embodiment, the "adjacent 50 pixels" in the intersection operation of all the interactive trigger regions within the adjacent 50-pixel range is only an example, and other embodiments can also be selected according to user requirements.
步骤S54：通过红黄绿标记不同互动频率的区域。当然，其他实施方式中亦可通过其他不同的颜色甚至其他标识来标记不同互动频率的区域。Step S54: Mark areas with different interaction frequencies by red, yellow and green. Of course, in other embodiments, the regions with different interaction frequencies may also be marked with other different colors or even other identifiers.
本发明所述的差分像素的互动区域与互动时间段识别方法的算法，主要通过对人体活动在图像上产生的像素变化进行统计，进而结合人体活动模型，进行人体普通位置移动和空间互动两种状态的判定，最终得出人体互动区域与互动时间段。上述方法所需的运算消耗较少，可将人体位移型活动和互动型活动进行区分，且能完整的记录一个人体的活动区域和在该区域活动的时间。The algorithm of the method for identifying the interactive area and the interactive time period of the differential pixels of the present invention mainly calculates the pixel changes generated by the human body movement on the image, and then combines the human body movement model to carry out two types of human body position movement and spatial interaction. The state is judged, and finally the human interaction area and interaction time period are obtained. The above method requires less computation consumption, can distinguish human body displacement type activities from interactive type activities, and can completely record the activity area of a human body and the time spent in this area.
本发明还公开了一种存储设备及移动终端。所述存储设备存储有多条指令，所述指令适于由处理器加载并执行，所述指令包括：接收输入的视频信号，并将所输入的视频信号进行拆帧处理，以生成单帧图像；对得到的单帧图像进行人形检测；进行全图差分，并记录差分数据；判断前N帧的图像像素差分数据是否满足人体互动特征模型，并记录触发的互动区域与时间断点以及终止的互动区域与时间断点；以及记录人体互动区域和时间段。The invention also discloses a storage device and a mobile terminal. The storage device stores a plurality of instructions, the instructions are suitable for being loaded and executed by the processor, and the instructions include: receiving an input video signal, and performing frame splitting processing on the input video signal to generate a single-frame image ; Perform humanoid detection on the obtained single-frame image; perform full-image difference, and record the difference data; judge whether the image pixel difference data of the first N frames meet the human interaction feature model, and record the triggered interaction area and time breakpoint and termination. Interaction areas and time breakpoints; and recording human interaction areas and time periods.
所述移动终端包括处理器及存储设备，所述处理器适于实现各指令，所述存储设备适于存储多条指令，所述指令适于由处理器加载并执行，所述指令包括：接收输入的视频信号，并将所输入的视频信号进行拆帧处理，以生成单帧图像；对得到的单帧图像进行人形检测；进行全图差分，并记录差分数据；判断前N帧的图像像素差分数据是否满足人体互动特征模型，并记录触发的互动区域与时间断点以及终止的互动区域与时间断点；以及记录人体互动区域和时间段。The mobile terminal includes a processor and a storage device, the processor is suitable for implementing various instructions, the storage device is suitable for storing a plurality of instructions, the instructions are suitable for being loaded and executed by the processor, and the instructions include: receiving Input video signal, and deframe the input video signal to generate a single frame image; perform humanoid detection on the obtained single frame image; perform full image difference, and record the difference data; judge the image pixels of the first N frames Whether the differential data satisfies the human interaction feature model, and record the triggered interaction area and time breakpoint as well as the terminated interaction area and time breakpoint; and record the human interaction area and time period.
以上仅为本发明的实施方式，并非因此限制本发明的专利范围，凡是利用本发明说明书及附图内容所作的等效结构，直接或间接运用在其他相关的技术领域，均同理在本发明的专利保护范围之内。The above are only the embodiments of the present invention, and are not intended to limit the scope of the present invention. Any equivalent structure made by using the contents of the description and the accompanying drawings of the present invention, directly or indirectly applied to other related technical fields, is equally applicable to the present invention. within the scope of patent protection.
Claims (8)
Translated fromChinesePriority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710655807.2ACN107330424B (en) | 2017-08-03 | 2017-08-03 | Interaction area and interaction time period identification method, storage device and mobile terminal |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710655807.2ACN107330424B (en) | 2017-08-03 | 2017-08-03 | Interaction area and interaction time period identification method, storage device and mobile terminal |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN107330424A CN107330424A (en) | 2017-11-07 |
| CN107330424Btrue CN107330424B (en) | 2020-10-16 |
Family
ID=60225613
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201710655807.2AActiveCN107330424B (en) | 2017-08-03 | 2017-08-03 | Interaction area and interaction time period identification method, storage device and mobile terminal |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN107330424B (en) |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US7457007B2 (en)* | 2003-04-07 | 2008-11-25 | Silverbrook Research Pty Ltd | Laser scanning device for printed product identification codes |
| CN102436301A (en)* | 2011-08-20 | 2012-05-02 | Tcl集团股份有限公司 | Human-machine interaction method and system based on reference region and time domain information |
| CN102509088A (en)* | 2011-11-28 | 2012-06-20 | Tcl集团股份有限公司 | Hand motion detecting method, hand motion detecting device and human-computer interaction system |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP4509917B2 (en)* | 2005-11-21 | 2010-07-21 | 株式会社メガチップス | Image processing apparatus and camera system |
- 2017
- 2017-08-03CNCN201710655807.2Apatent/CN107330424B/enactiveActive
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US7457007B2 (en)* | 2003-04-07 | 2008-11-25 | Silverbrook Research Pty Ltd | Laser scanning device for printed product identification codes |
| CN102436301A (en)* | 2011-08-20 | 2012-05-02 | Tcl集团股份有限公司 | Human-machine interaction method and system based on reference region and time domain information |
| CN102509088A (en)* | 2011-11-28 | 2012-06-20 | Tcl集团股份有限公司 | Hand motion detecting method, hand motion detecting device and human-computer interaction system |
Non-Patent Citations (1)
| Title |
|---|
| "基于HOG特征的人脸识别系统研究";幕春雷;《中国优秀硕士学位论文全文数据库 (电子期刊) 信息科技辑》;20140115;期刊第4.2节* |
Also Published As
| Publication number | Publication date |
|---|---|
| CN107330424A (en) | 2017-11-07 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US11450146B2 (en) | Gesture recognition method, apparatus, and device | |
| CN110427905B (en) | Pedestrian tracking method, device and terminal | |
| US10452893B2 (en) | Method, terminal, and storage medium for tracking facial critical area | |
| CN103971386B (en) | A kind of foreground detection method under dynamic background scene | |
| US20220092882A1 (en) | Living body detection method based on facial recognition, and electronic device and storage medium | |
| US20190304102A1 (en) | Memory efficient blob based object classification in video analytics | |
| Noh et al. | Learning deconvolution network for semantic segmentation | |
| US20190130583A1 (en) | Still and slow object tracking in a hybrid video analytics system | |
| US9536147B2 (en) | Optical flow tracking method and apparatus | |
| US20190130580A1 (en) | Methods and systems for applying complex object detection in a video analytics system | |
| CN104601964B (en) | Pedestrian target tracking and system in non-overlapping across the video camera room of the ken | |
| CN111539273A (en) | A traffic video background modeling method and system | |
| CN109961019A (en) | A spatiotemporal behavior detection method | |
| CN112308095A (en) | Image preprocessing and model training method, device, server and storage medium | |
| CN107798313A (en) | A kind of human posture recognition method, device, terminal and storage medium | |
| CN110349190A (en) | Target tracking method, device and equipment for adaptive learning and readable storage medium | |
| KR20140095333A (en) | Method and apparratus of tracing object on image | |
| CN106778635A (en) | A kind of human region detection method of view-based access control model conspicuousness | |
| Tang et al. | Multiple-kernel adaptive segmentation and tracking (MAST) for robust object tracking | |
| CN118314310B (en) | Obstacle avoidance processing analysis system based on forklift image data acquisition | |
| CN113065379A (en) | Image detection method, device and electronic device for fused image quality | |
| CN118485823B (en) | Image recognition method, device and medium for animals in animal farm under edge scene | |
| CN111582654A (en) | Service quality evaluation method and device based on deep recurrent neural network | |
| CN109902613A (en) | A Human Feature Extraction Method Based on Transfer Learning and Image Enhancement | |
| WO2023000253A1 (en) | Climbing behavior early-warning method and apparatus, electrode device, and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |