技术领域technical field
本发明属于计算机视觉与人机交互领域，具体涉及一种基于深度手势的频道选择方法。The invention belongs to the field of computer vision and human-computer interaction, and in particular relates to a channel selection method based on depth gestures.
背景技术Background technique
随着科学与技术的发展，人机交互的发展尤为迅速，用眼睛控制的“眼标”、用声音来控制的语音系统、计算机的微型化、随身化，人们也不再满足传统的使用外设的控制方式，于是声控，手势控制等新颖的方法也相续发展。With the development of science and technology, the development of human-computer interaction is particularly rapid. With the "eye mark" controlled by eyes, the voice system controlled by voice, the miniaturization and portability of computers, people are no longer satisfied with the traditional use of external devices. With the unique control mode, novel methods such as voice control and gesture control have also been continuously developed.
目前，在日常生活中，使用电视的用户会发觉虽然已经摆脱了有线的束缚但是对于遥控器的束缚依旧没有摆脱，依赖于外设，给使用者在一定程度上带来了不便，例如遥控器的丢失、缺少电池、损坏而导致的不灵敏等。于是希望找到一种新型的控制方法来彻底摆脱遥控器等外设的束缚。At present, in daily life, users who use TV will find that although they have got rid of the shackles of the cable, they still have not got rid of the shackles of the remote control, relying on peripherals, which brings inconvenience to the user to a certain extent, such as the remote control Insensitivity caused by loss, lack of battery, damage, etc. So I hope to find a new control method to completely get rid of the shackles of peripherals such as remote controls.
发明内容Contents of the invention
本发明的目的在于实现不需要控制器的条件下只用手势即可完成对电视机的频道选择，提供了一种基于深度摄像机的频道选择方法。The purpose of the present invention is to realize the channel selection of the TV set only by gestures without the need of a controller, and provides a channel selection method based on a depth camera.
本发明的具体方案如下：Concrete scheme of the present invention is as follows:
一种基于深度手势的频道选择方法，包括如下步骤：A channel selection method based on deep gestures, comprising the steps of:
步骤一，采用深度摄像机提取出手势序列；步骤二，微处理器接收深度摄像机所提取的手势序列并将此手势序列与储存的模板序列进行比对，从而识别出正确的手势信息；步骤三，将不同的手势信息组成频道的索引值；步骤四，微处理器根据索引值发出相应的控制信号；步骤五，控制信号控制电视机进行频道选择。Step 1, using the depth camera to extract the gesture sequence; Step 2, the microprocessor receives the gesture sequence extracted by the depth camera and compares the gesture sequence with the stored template sequence to identify the correct gesture information; Step 3, Composing different gesture information into a channel index value; Step 4, the microprocessor sends a corresponding control signal according to the index value; Step 5, the control signal controls the TV to select a channel.
作为改进，所述步骤一还包括过滤掉其他环境信息保留下带有深度信息的手势信息。As an improvement, the first step further includes filtering out other environmental information and retaining gesture information with depth information.
进一步改进，所述步骤一所采用的深度摄像机为kinect，当深度摄像机kinect采集到彩色与深度数据后，利用kinect的sdk中的手势跟踪模块提取手掌实时空间位置坐标(x,y,z)，然后根据此坐标提取所需的手势速度特征(dx,dy,dz)。Further improvement, the depth camera adopted in the step 1 is kinect, after the depth camera kinect collects color and depth data, utilize the gesture tracking module in the sdk of kinect to extract the real-time spatial position coordinates (x, y, z) of the palm, Then extract the required gesture velocity features (dx, dy, dz) according to this coordinate.
更进一步改进，还包括数据储存步骤，具体为：在微处理器中存储用户预先定义的手势模板；同时储存通过深度摄像机kinect采集到的手势序列。A further improvement also includes a data storage step, specifically: storing user-defined gesture templates in the microprocessor; and simultaneously storing gesture sequences collected by the depth camera kinect.
本发明的有益效果是：实现了不需要控制器的条件下只用手势即可完成对电视机的频道选择。The beneficial effect of the invention is that: the channel selection of the TV set can be completed only by gestures under the condition of not needing a controller.
附图说明Description of drawings
图1为本发明一种基于深度手势的频道选择方法信息处理过程示意图。FIG. 1 is a schematic diagram of an information processing process of a deep gesture-based channel selection method according to the present invention.
具体实施方式Detailed ways
本发明一种基于深度摄像机的频道选择方法，其硬件配置部分为深度摄像机、微处理器和电视机。深度摄像机具有能感受1.2-3.5米物体深度的特性并且其颜色感应的分辨率为640X480，每秒30帧。深度摄像机还拥有57°的水平视角与43°的垂直视角。深度摄像机通过AUX端口转USB接口与微处理器连接进行连接，电视机可以通过HDMI或者USB接口与微处理器进行连接。预先需要在微处理器中安装好深度摄像机的驱动以及手势识别安装软件。手势识别安装软件即图像处理模块，包括手势特征提取模块以及手势信息处理单元和数据存储单元。通过深度摄像机采集外界深度手势图像序列信息并反馈给微处理器，微处理器得到此手势序列后进行匹配与处理，再与处理器中的模板进行比较，当有符合的手势出现时微处理器即产生相应的控制信号给电视机进行频道的切换。基于深度手势的频道选择系统通过深度摄像头采集手势信息，微处理器进行处理发出控制信号，能在不需要控制器的条件下只用手势即可实现电视机的频道选择。The present invention is a channel selection method based on a depth camera, and its hardware configuration includes a depth camera, a microprocessor and a television. The depth camera has the characteristics of being able to feel the depth of objects of 1.2-3.5 meters and its color sensing resolution is 640X480, 30 frames per second. The depth camera also has a horizontal viewing angle of 57° and a vertical viewing angle of 43°. The depth camera is connected to the microprocessor through the AUX port to the USB interface, and the TV can be connected to the microprocessor through the HDMI or USB interface. The driver of the depth camera and the gesture recognition installation software need to be installed in the microprocessor in advance. Gesture recognition installation software is an image processing module, including a gesture feature extraction module, a gesture information processing unit, and a data storage unit. The external depth gesture image sequence information is collected by the depth camera and fed back to the microprocessor. After the microprocessor obtains the gesture sequence, it performs matching and processing, and then compares it with the template in the processor. When a matching gesture appears, the microprocessor That is, a corresponding control signal is generated for the TV to switch channels. The channel selection system based on deepin gestures collects gesture information through the deepin camera, and the microprocessor processes and sends control signals, which can realize the channel selection of the TV only by gestures without the need for a controller.
深度摄像机可以提取景深信息，过滤掉其他环境信息保留下带有深度信息的手势信息，并提取出人体手部进行识别跟踪。且微处理器能接收由深度摄像机采集回来的手势序列并且使用DTW(动态时间规整)算法与自身所存储的模板进行匹配识别，当所采集的手势信息序列与存储的模板序列相符合时，即认为识别出正确的手势信息，然后由不同的手势信息组成了频道的索引值，当识别出确定信号时所述微处理器就会根据索引值发出相应的控制信号，控制信号控制电视机进行频道的选择。The depth camera can extract the depth information, filter out other environmental information to retain the gesture information with depth information, and extract the human hand for recognition and tracking. And the microprocessor can receive the gesture sequence collected by the depth camera and use the DTW (Dynamic Time Warping) algorithm to match and recognize the template stored in itself. When the collected gesture information sequence matches the stored template sequence, it is considered The correct gesture information is recognized, and then the index value of the channel is composed of different gesture information. When a certain signal is recognized, the microprocessor will send a corresponding control signal according to the index value, and the control signal controls the TV to perform channel selection. choose.
在频道选择过程中，使用者需正对着深度摄像机，然后调整摄像机到一个合适的角、位置，便于得出使用者的手势并进行更好的跟踪识别。安置合理后用手写出欲选择的频道，系统即可识别手势并对电视机产生相应的控制信号从而实现频道的选择，这样使用者无需控制器即可方便的选择频道。During the channel selection process, the user needs to face the depth camera, and then adjust the camera to a suitable angle and position, so as to obtain the user's gestures and perform better tracking and recognition. After placing it properly, write out the desired channel by hand, and the system can recognize the gesture and generate a corresponding control signal to the TV to realize the channel selection, so that the user can easily select the channel without a controller.
微处理器需要处理数据，且需要安装深度摄像机的驱动程序等，可以采用小型的工控机。而软件方面，主要有图像处理模块，其包括手势特征提取模块以及手势信息处理单元和数据存储单元，如图1所示。手势特征提取模块用于从深度信息中滤去背景等无用信息，把有用的手势信息提取出来，并且进行跟踪使之成为一个连续的序列。手势信息处理单元用于处理手势向量，使用算法与相存储的模板进行匹配从而处理识别手势。数据存储单元用于存储手势模板以及相应的控制表等信息。The microprocessor needs to process data, and the driver program of the depth camera needs to be installed, so a small industrial computer can be used. In terms of software, there is mainly an image processing module, which includes a gesture feature extraction module, a gesture information processing unit, and a data storage unit, as shown in FIG. 1 . The gesture feature extraction module is used to filter out useless information such as background from depth information, extract useful gesture information, and track it to make it a continuous sequence. The gesture information processing unit is used to process the gesture vector, and use an algorithm to match the stored template to process and recognize the gesture. The data storage unit is used to store information such as gesture templates and corresponding control tables.
当深度摄像头kinect采集到彩色与深度数据后，即可利用kinect的sdk中的手势跟踪模块提取手掌实时空间位置坐标(以kinect为空间坐标原点)，然后即可根据此坐标提取所需的手势速度特征：手掌位置的当前帧空间坐标减去前一帧的空间坐标。得到手势特征后，该处理单元会利用WDTW算法与数据存储单元中的手势模板进行匹配，得到用户欲表达的手势信息。When the depth camera kinect collects the color and depth data, the gesture tracking module in the kinect sdk can be used to extract the real-time spatial position coordinates of the palm (with kinect as the origin of the spatial coordinates), and then the required gesture speed can be extracted based on these coordinates Features: The current frame space coordinate of the palm position minus the space coordinate of the previous frame. After obtaining the gesture features, the processing unit will use the WDTW algorithm to match the gesture template in the data storage unit to obtain the gesture information that the user wants to express.
本发明中，采用储存单元对数据进行储存，数据存储单元用来存储手势信息处理单元所需要的手势模板以及识别的手势结果。在系统运行之前，需要在微处理器中存储用户预先定义的手势模板。用户可以根据自己的需求、爱好，创建各种手势，当用户确定手势模板后，即可利用kinect预先做一遍手势，然后手势特征提取模块会提取手势模板特征，存储在数据存储单元。In the present invention, a storage unit is used to store data, and the data storage unit is used to store gesture templates and recognized gesture results required by the gesture information processing unit. Before the system runs, the user's predefined gesture template needs to be stored in the microprocessor. Users can create various gestures according to their own needs and hobbies. After the user determines the gesture template, he can use kinect to make a gesture in advance, and then the gesture feature extraction module will extract the features of the gesture template and store them in the data storage unit.
深度摄像机开启捕捉用户的手势，使用者在深度摄像机视场范围内书写欲选择的频道数字并且确认后，深度摄像机捕捉到手势序列后传递给微处理器，微处理器对该手势序列进行处理，得到用户欲表达的手势意图并传输给电视机，电视机显示用户的选择结果：频道选择或者切换。The depth camera starts to capture the user's gestures. After the user writes the channel number to be selected within the field of view of the depth camera and confirms it, the depth camera captures the gesture sequence and sends it to the microprocessor. The microprocessor processes the gesture sequence. Get the gesture intention that the user wants to express and transmit it to the TV, and the TV will display the user's selection result: channel selection or switching.
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201410658224.1ACN104333794A (en) | 2014-11-18 | 2014-11-18 | Channel selection method based on depth gestures |
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201410658224.1ACN104333794A (en) | 2014-11-18 | 2014-11-18 | Channel selection method based on depth gestures |
| Publication Number | Publication Date |
|---|---|
| CN104333794Atrue CN104333794A (en) | 2015-02-04 |
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201410658224.1APendingCN104333794A (en) | 2014-11-18 | 2014-11-18 | Channel selection method based on depth gestures |
| Country | Link |
|---|---|
| CN (1) | CN104333794A (en) |
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105930785A (en)* | 2016-04-15 | 2016-09-07 | 丁盛 | Intelligent concealed-type interaction system |
| WO2017028070A1 (en)* | 2015-08-14 | 2017-02-23 | 郭子明 | Method and television system for prompting information when displaying hidden channel on basis of specific hand gesture |
| WO2017035846A1 (en)* | 2015-09-06 | 2017-03-09 | 何兰 | Method and remote control system for prompting information when hand gesture matches channel grouping |
| CN112492211A (en)* | 2020-12-01 | 2021-03-12 | 咪咕文化科技有限公司 | Shooting method, electronic equipment and storage medium |
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101437124A (en)* | 2008-12-17 | 2009-05-20 | 三星电子（中国）研发中心 | Method for processing dynamic gesture identification signal facing (to)television set control |
| CN102801924A (en)* | 2012-07-20 | 2012-11-28 | 合肥工业大学 | Television program host interaction system based on Kinect |
| CN102968178A (en)* | 2012-11-07 | 2013-03-13 | 电子科技大学 | Gesture-based PPT (Power Point) control system |
| CN103294996A (en)* | 2013-05-09 | 2013-09-11 | 电子科技大学 | 3D gesture recognition method |
| CN103353935A (en)* | 2013-07-19 | 2013-10-16 | 电子科技大学 | 3D dynamic gesture identification method for intelligent home system |
| CN103501445A (en)* | 2013-10-12 | 2014-01-08 | 青岛旲天下智能科技有限公司 | Gesture-based interaction two-way interactive digital TV box system and implementation method |
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101437124A (en)* | 2008-12-17 | 2009-05-20 | 三星电子（中国）研发中心 | Method for processing dynamic gesture identification signal facing (to)television set control |
| CN102801924A (en)* | 2012-07-20 | 2012-11-28 | 合肥工业大学 | Television program host interaction system based on Kinect |
| CN102968178A (en)* | 2012-11-07 | 2013-03-13 | 电子科技大学 | Gesture-based PPT (Power Point) control system |
| CN103294996A (en)* | 2013-05-09 | 2013-09-11 | 电子科技大学 | 3D gesture recognition method |
| CN103353935A (en)* | 2013-07-19 | 2013-10-16 | 电子科技大学 | 3D dynamic gesture identification method for intelligent home system |
| CN103501445A (en)* | 2013-10-12 | 2014-01-08 | 青岛旲天下智能科技有限公司 | Gesture-based interaction two-way interactive digital TV box system and implementation method |
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2017028070A1 (en)* | 2015-08-14 | 2017-02-23 | 郭子明 | Method and television system for prompting information when displaying hidden channel on basis of specific hand gesture |
| WO2017035846A1 (en)* | 2015-09-06 | 2017-03-09 | 何兰 | Method and remote control system for prompting information when hand gesture matches channel grouping |
| CN105930785A (en)* | 2016-04-15 | 2016-09-07 | 丁盛 | Intelligent concealed-type interaction system |
| CN112492211A (en)* | 2020-12-01 | 2021-03-12 | 咪咕文化科技有限公司 | Shooting method, electronic equipment and storage medium |
| Publication | Publication Date | Title |
|---|---|---|
| CN103353935B (en) | A kind of 3D dynamic gesture identification method for intelligent domestic system | |
| JP6968154B2 (en) | Control systems and control processing methods and equipment | |
| US9734393B2 (en) | Gesture-based control system | |
| US11087637B2 (en) | Finger reading method and device based on visual gestures | |
| CN102937832B (en) | A kind of gesture method for catching of mobile terminal and device | |
| CN105425964B (en) | A kind of gesture identification method and system | |
| US20160162039A1 (en) | Method and system for touchless activation of a device | |
| EP3847531B1 (en) | Hand-over-face input sensing for interaction with a device having a built-in camera | |
| US20140022159A1 (en) | Display apparatus control system and method and apparatus for controlling a plurality of displays | |
| CN105763917B (en) | A kind of control method and system of terminal booting | |
| CN108712603B (en) | An image processing method and mobile terminal | |
| CN106845335A (en) | Gesture identification method, device and virtual reality device for virtual reality device | |
| CN102200830A (en) | Non-contact control system and control method based on static gesture recognition | |
| CN103105930A (en) | Non-contact type intelligent inputting method based on video images and device using the same | |
| JP2014048937A (en) | Gesture recognition device, control method thereof, display equipment, and control program | |
| KR20120068253A (en) | Method and apparatus for providing response of user interface | |
| CN103716456A (en) | Mobile phone unlocking method based on face feature information | |
| CN103295029A (en) | Interaction method and device of gesture control terminal | |
| US10831273B2 (en) | User action activated voice recognition | |
| CN103000054B (en) | Intelligent teaching machine for kitchen cooking and control method thereof | |
| CN108474950A (en) | HMD device and its control method | |
| CN105741256B (en) | Electronic device and shaving reminder system and method therefor | |
| CN109272473B (en) | Image processing method and mobile terminal | |
| CN104333794A (en) | Channel selection method based on depth gestures | |
| CN104951117A (en) | Image processing system and related method for generating corresponding information by utilizing image identification |
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| WD01 | Invention patent application deemed withdrawn after publication | ||
| WD01 | Invention patent application deemed withdrawn after publication | Application publication date:20150204 |