技术领域technical field
本发明涉及机器视觉领域，具体涉及一种基于双目视觉的嵌入式虚拟键盘及方法。The invention relates to the field of machine vision, in particular to an embedded virtual keyboard and method based on binocular vision.
背景技术Background technique
键盘是用于操作设备运行的一种指令和数据输入装置，是最常用也是最主要的输入设备，通过键盘可以将英文字母、数字、标点符号等输入到计算机中，从而向计算机发出命令、输入数据等。The keyboard is an instruction and data input device used to operate the equipment. It is the most commonly used and the most important input device. Through the keyboard, English letters, numbers, punctuation marks, etc. can be input into the computer, so as to issue commands and input to the computer. data etc.
机械键盘：采用类似金属接触式开关，工作原理是使触点导通或断开，具有工艺简单、易维护、打字时节奏感强，长期使用手感不会改变等特点，但是噪音较大、价格比较昂贵。Mechanical keyboard: similar to metal contact switches, the working principle is to make the contacts on or off. It has the characteristics of simple process, easy maintenance, strong rhythm when typing, and long-term use. relatively expensive.
塑料薄膜式键盘：键盘内部共分四层，实现了无机械磨损。其特点是低价格、低噪音和低成本，但是长期使用后由于材质问题手感会发生变化。Plastic film keyboard: The keyboard is divided into four layers to achieve no mechanical wear. It is characterized by low price, low noise and low cost, but the hand feeling will change after long-term use due to material problems.
无接点静电电容键盘：使用类似电容式开关的原理，通过按键时改变电极间的距离引起电容容量改变从而驱动编码器，特点是无磨损且密封性较好，但是价格昂贵。Non-contact electrostatic capacitive keyboard: Using the principle similar to capacitive switches, changing the distance between the electrodes when pressing the key causes the capacitance to change to drive the encoder. It is characterized by no wear and good sealing, but it is expensive.
发明内容Contents of the invention
为了克服现有技术存在的缺点与不足，本发明提供一种基于双目视觉的嵌入式虚拟键盘及方法。In order to overcome the shortcomings and deficiencies of the prior art, the present invention provides an embedded virtual keyboard and method based on binocular vision.
本发明采用如下技术方案：The present invention adopts following technical scheme:
一种基于双目视觉的嵌入式虚拟键盘，包括双目摄像头、纸质键盘、处理器及移动终端，所述双目摄像头与处理器的输入端连接，所述处理器的输出端与移动终端连接，所述双目摄像头在纸质键盘的上方，纸质键盘在双目摄像头的视线范围内。An embedded virtual keyboard based on binocular vision, comprising a binocular camera, a paper keyboard, a processor and a mobile terminal, the binocular camera is connected to the input end of the processor, and the output end of the processor is connected to the mobile terminal connected, the binocular camera is above the paper keyboard, and the paper keyboard is within the line of sight of the binocular camera.
所述双目摄像头是由两个参数相同的摄像头并排设置而成，两个摄像头的光轴平行，且两个摄像头的光心平面重合。The binocular camera is composed of two cameras with the same parameters arranged side by side, the optical axes of the two cameras are parallel, and the optical center planes of the two cameras coincide.
所述双目摄像头采用彩色摄像头。The binocular camera adopts a color camera.
所述处理器采用三星Exynos4412芯片。The processor adopts Samsung Exynos4412 chip.
嵌入式虚拟键盘的图像处理方法，首先，通过手部前景分割算法识别出手部区域，然后检测出手部轮廓，最后对轮廓进行指尖检测，求出指尖点的坐标。In the image processing method of the embedded virtual keyboard, firstly, the hand area is identified through the hand foreground segmentation algorithm, then the outline of the hand is detected, and finally the outline is detected for the fingertip to obtain the coordinates of the fingertip point.
所述指尖检测包括单指尖检测及多指尖检测；The fingertip detection includes single fingertip detection and multi-fingertip detection;
所述单指尖检测具体为：计算手部轮廓中所有K向量大于0的点，找出上述点中纵坐标Y最小的点作为指尖点；The single fingertip detection is specifically as follows: calculate all points with K vectors greater than 0 in the hand contour, and find out the point with the smallest ordinate Y among the above points as the fingertip point;
所述多指尖检测具体为：检测手部轮廓凸包，求出各凸包顶点的K向量，相近凸包顶点中K向量最大的点记为指尖点。The multi-fingertip detection specifically includes: detecting the convex hull of the hand contour, obtaining the K vector of each convex hull vertex, and marking the point with the largest K vector among the similar convex hull vertices as the fingertip point.
本发明的有益效果：Beneficial effects of the present invention:
本发明使用双目摄像头实现手指的检测，双目摄像头可由普通彩色摄像头组合而成，相比目前市场上的虚拟键盘采用红外摄像头+红外激光辅助定位的检测方案，本发明能够更好的减小环境光线对于系统的干扰，且成本更低。The present invention uses a binocular camera to realize finger detection, and the binocular camera can be composed of ordinary color cameras. Compared with the current virtual keyboard on the market that uses an infrared camera + infrared laser assisted positioning detection scheme, the present invention can better reduce the Ambient light interferes with the system, and the cost is lower.
附图说明Description of drawings
图1是本发明的结构示意图；Fig. 1 is a structural representation of the present invention;
图2是本发明的工作流程图；Fig. 2 is a work flow chart of the present invention;
图3是单指尖检测流程图；Fig. 3 is a single fingertip detection flow chart;
图4是多指尖检测流程图。Fig. 4 is a flow chart of multi-fingertip detection.
具体实施方式detailed description
下面结合实施例及附图，对本发明作进一步地详细说明，但本发明的实施方式不限于此。The present invention will be described in further detail below in conjunction with the embodiments and the accompanying drawings, but the embodiments of the present invention are not limited thereto.
实施例Example
如图1所示，一种基于双目视觉的嵌入式虚拟键盘，包括双目摄像头1，用于采集图像信息，获取手指在纸质键盘2的运动轨迹，然后传输给处理器3，处理器处理后得到手指的按键信号，传输到移动终端4进行按键响应。As shown in Figure 1, an embedded virtual keyboard based on binocular vision includes a binocular camera 1 for collecting image information, obtaining the motion track of the finger on the paper keyboard 2, and then transmitting it to the processor 3, and the processor After processing, the key-press signal of the finger is obtained, and is transmitted to the mobile terminal 4 for key-press response.
所述纸质键盘2，用于模拟键盘按键，供用户输入使用。The paper keyboard 2 is used to simulate keyboard keys for user input.
处理器3，用于运行虚拟键盘程序，对采集到的图像信息进行处理，并将结果传输到移动终端中。The processor 3 is used to run the virtual keyboard program, process the collected image information, and transmit the result to the mobile terminal.
移动终端4，接收到的信息后触发键盘事件，并在移动终端上作出响应。The mobile terminal 4 triggers a keyboard event after receiving the information, and makes a response on the mobile terminal.
本实施中，双目摄像头是指由两个参数相同的摄像头并排设置而成，两个摄像头的光轴平行，左右间隔适当的距离，确保两个摄像头的光心平面重合，本实施例中采用蓝色妖姬S11型USB2.0摄像头，帧速为30帧/秒，默认分辨率为640*480。In this implementation, the binocular camera refers to two cameras with the same parameters arranged side by side, the optical axes of the two cameras are parallel, and there is an appropriate distance between the left and right to ensure that the optical center planes of the two cameras overlap. In this embodiment, Blue Enchantress S11 USB2.0 camera, the frame rate is 30 frames per second, and the default resolution is 640*480.
双目摄像头1位于纸质键盘上方，纸质键盘的尺寸要保证整个键盘区域能够完整的出现在双目摄像头的视野范围内，按键布置没有特殊要求，对于纸质键盘的纸也没有要求，就是普通的纸打印出键盘图案就可以。The binocular camera 1 is located above the paper keyboard. The size of the paper keyboard should ensure that the entire keyboard area can completely appear within the field of view of the binocular camera. Ordinary paper can print out the keyboard pattern.
处理器采用的是三星Exynos4412芯片作为主处理器，运行主频可高达1.5GHz，拥有1GB的RAM，可以支持系统实现图像处理算法所需的复杂运算。The processor uses the Samsung Exynos4412 chip as the main processor, the main frequency can be as high as 1.5GHz, and it has 1GB of RAM, which can support the system to realize the complex operations required by the image processing algorithm.
如图2、图3及图4所示，嵌入式虚拟键盘的图像处理方法包括通过手部前景分割算法识别出手部区域，然后检测出手部轮廓，最后对轮廓进行指尖检测，求出指尖点的坐标。As shown in Figure 2, Figure 3 and Figure 4, the image processing method of the embedded virtual keyboard includes identifying the hand area through the hand foreground segmentation algorithm, then detecting the hand contour, and finally performing fingertip detection on the contour to obtain the fingertip The coordinates of the point.
所述指尖检测包括单指尖检测及多指尖检测；The fingertip detection includes single fingertip detection and multi-fingertip detection;
所述单指尖检测具体为：计算手部轮廓中所有K向量大于0的点，找出上述点中纵坐标Y最小的点作为指尖点；The single fingertip detection is specifically as follows: calculate all points with K vectors greater than 0 in the hand contour, and find out the point with the smallest ordinate Y among the above points as the fingertip point;
所述多指尖检测具体为：检测手部轮廓凸包，求出各凸包顶点的K向量，相近凸包顶点中K向量最大的点记为指尖点。The multi-fingertip detection specifically includes: detecting the convex hull of the hand contour, obtaining the K vector of each convex hull vertex, and marking the point with the largest K vector among the similar convex hull vertices as the fingertip point.
该键盘无需物理键盘实体，而是在一个虚拟的平面上进行操作，通过一套双目摄像头装置对手指动作进行检测，降低了键盘按键损坏的几率；采用双目摄像头进行识别，相比目前市场上虚拟键盘采用的激光投影解决方案，采用双目摄像头的解决方案硬件结构更加简单，在降低成本的同时能够有效减轻外界光照条件对设备的干扰。The keyboard does not need a physical keyboard entity, but operates on a virtual plane, and detects finger movements through a set of binocular camera devices, which reduces the probability of keyboard damage; using binocular cameras for identification, compared with the current market The laser projection solution used on the virtual keyboard and the binocular camera solution have a simpler hardware structure, which can effectively reduce the interference of external lighting conditions on the device while reducing costs.
采用纸质的输入界面，降低了键盘按键损坏的几率，提高了使用寿命；采用双目摄像头识别手部动作，能够有效减轻外界光照条件对设备的干扰。The use of a paper input interface reduces the chance of damage to the keyboard keys and improves the service life; the use of a binocular camera to recognize hand movements can effectively reduce the interference of external light conditions on the device.
上述实施例为本发明较佳的实施方式，但本发明的实施方式并不受所述实施例的限制，其他的任何未背离本发明的精神实质与原理下所作的改变、修饰、替代、组合、简化，均应为等效的置换方式，都包含在本发明的保护范围之内。The above-mentioned embodiment is a preferred embodiment of the present invention, but the embodiment of the present invention is not limited by the embodiment, and any other changes, modifications, substitutions and combinations made without departing from the spirit and principle of the present invention , simplification, all should be equivalent replacement methods, and are all included in the protection scope of the present invention.
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201610309202.3ACN105867638A (en) | 2016-05-10 | 2016-05-10 | Embedded virtual keyboard based on binocular vision and method |
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201610309202.3ACN105867638A (en) | 2016-05-10 | 2016-05-10 | Embedded virtual keyboard based on binocular vision and method |
| Publication Number | Publication Date |
|---|---|
| CN105867638Atrue CN105867638A (en) | 2016-08-17 |
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201610309202.3APendingCN105867638A (en) | 2016-05-10 | 2016-05-10 | Embedded virtual keyboard based on binocular vision and method |
| Country | Link |
|---|---|
| CN (1) | CN105867638A (en) |
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110442242A (en)* | 2019-08-13 | 2019-11-12 | 哈尔滨拓博科技有限公司 | A kind of smart mirror system and control method based on the interaction of binocular space gesture |
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101807114A (en)* | 2010-04-02 | 2010-08-18 | 浙江大学 | Natural interactive method based on three-dimensional gestures |
| CN202018647U (en)* | 2011-04-25 | 2011-10-26 | 郑致远 | Visual keyboard |
| JP2015111478A (en)* | 2015-03-23 | 2015-06-18 | 株式会社ナカヨ | Virtual keyboard input method |
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101807114A (en)* | 2010-04-02 | 2010-08-18 | 浙江大学 | Natural interactive method based on three-dimensional gestures |
| CN202018647U (en)* | 2011-04-25 | 2011-10-26 | 郑致远 | Visual keyboard |
| JP2015111478A (en)* | 2015-03-23 | 2015-06-18 | 株式会社ナカヨ | Virtual keyboard input method |
| Title |
|---|
| 李振伟: "基于图像处理的半虚拟键盘的研究", 《中国优秀硕士学位论文全文数据库（电子期刊）》* |
| 郭康德: "基于视觉的三维指尖检测算法和应用", 《万方学位数据库》* |
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110442242A (en)* | 2019-08-13 | 2019-11-12 | 哈尔滨拓博科技有限公司 | A kind of smart mirror system and control method based on the interaction of binocular space gesture |
| Publication | Publication Date | Title |
|---|---|---|
| CN103809880B (en) | Man-machine interaction system and method | |
| CN104809458B (en) | A kind of pupil center's localization method and device | |
| TWI471815B (en) | Gesture recognition device and method | |
| CN104978012B (en) | One kind points to exchange method, apparatus and system | |
| CN103713738B (en) | A kind of view-based access control model follows the tracks of the man-machine interaction method with gesture identification | |
| CN103927016A (en) | Real-time three-dimensional double-hand gesture recognition method and system based on binocular vision | |
| CN102053702A (en) | Dynamic gesture control system and method | |
| CN106200971A (en) | Man-machine interactive system device based on gesture identification and operational approach | |
| CN101694694A (en) | Finger identification method used in interactive demonstration system | |
| Zhang et al. | A virtual keyboard implementation based on finger recognition | |
| CN111914630A (en) | Method, apparatus, device and storage medium for generating training data for face recognition | |
| Vishwakarma et al. | Hand gesture recognition in low-intensity environment using depth images | |
| Baumgartl et al. | Vision-based hand gesture recognition for human-computer interaction using MobileNetV2 | |
| CN103870071A (en) | Touch source identification method and system | |
| CN111208907A (en) | Sign language recognition system and method based on EMG signal and finger joint deformation signal | |
| CN104714650B (en) | A kind of data inputting method and device | |
| CN108614988A (en) | A kind of motion gesture automatic recognition system under complex background | |
| CN106484108A (en) | Chinese characters recognition method based on double vision point gesture identification | |
| CN105867638A (en) | Embedded virtual keyboard based on binocular vision and method | |
| CN104375630A (en) | Public information terminal based on Kinect | |
| Rokade et al. | Spelled sign word recognition using key frame | |
| CN109961010A (en) | A gesture recognition method based on intelligent robot | |
| KR101567469B1 (en) | Apparatus and method for controlling virtual input device for augmented reality device | |
| CN204155186U (en) | A kind of portable equipment | |
| CN114721508A (en) | Virtual keyboard display method, apparatus, device, medium, and program product |
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| WD01 | Invention patent application deemed withdrawn after publication | ||
| WD01 | Invention patent application deemed withdrawn after publication | Application publication date:20160817 |