A kind of gestural control methodTechnical field
The invention belongs to human-computer interaction technique field, more particularly to a kind of gestural control method.
Background technology
With the development of computer technology, human-computer interaction technology is also more and more intelligent, personalized.From traditional mouse,The interactive meanses such as keyboard, touch-screen, the Non-contact control methods such as gesture control are gradually developed.
In computer realm, including but not limited to virtual reality, augmented reality, mixed reality scene, current gesture is handed overMutual scope includes the gesture operation based on two dimension display object and based on three-dimensional display object.Its principle is typically all to pass through gestureThe image or video of the camera shooting user gesture of identifying system, then identify the quiet of user gesture by computerized algorithmState implication or motion implication, re-map and performed for internal system order, final system according to the order to the various of display objectOperation.
Prior art：Show that the gesture operation of object, such as the two dimensional surface of traditional computer are shown towards two dimensional surfaceObject, the physical equipments such as mouse, keyboard, touch-screen are replaced by gesture operation, more natural man-machine interaction mode can be obtained.As Microsoft Research is located at an entitled Project Prague of the advanced technology laboratory research of Israel Prague itemMesh, it is exactly to capture user gesture by camera device to operate, is mapped as windows internal commands, performs a certain concrete operations.
Show that the gesture operation of object, such as the intelligent glasses hololens hololenses of Microsoft are supported towards 3 D stereoFor the gesture interaction of virtual three-dimensional model.The gesture operation that Microsoft intelligent glasses hololens is supported includes：Bloom bursts forth handGesture, Air tap air tap gesture, pin do not put, roll, pulling, zoom, peg, release or unload application etc., specificallyOperation may be referred to the site information of Microsoft.
In summary object and the gesture operation of three-dimensional display object are shown towards two dimension, prior art is present asks as followsTopic：
1) gesture operation is not hidden enough, it is necessary to which user lifts hand operation, so that main camera " can see " gesture of user；When user operates virtual display object, the people that virtual image is can't see on side can be wondering, especially in public；
2) much operation is all singlehanded gesture identification to existing system, there is certain security risk, when people around is more, systemOther people unconscious gesture is easily judged as system command, causes system mistake；
The content of the invention
It is an object of the invention to provide a kind of gestural control method, to solve the above problems.
To realize above mentioned problem, the present invention uses following technical scheme：
A kind of gestural control method, including bimanual input operation scenario and singlehanded gesture operation scene；
Bimanual input operation scenario includes gesture-capture camera, mainframe computer system and near-end gesture recognition system；HandGesture catches camera and is used to catch user gesture, and gesture-capture camera is connected with near-end gesture recognition system；Near-end gesture is knownOther system is used for after the near-end gesture operation of user is identified, and mainframe computer system interaction, drives mainframe computer systemOriginal control logic, mainframe computer system are the original computer system of equipment for supporting gesture operation；Specifically include following stepSuddenly：
Step 1, user stretches out a hand A, in body near-end, hand A take posture to represent user to start operation virtual aobviousShow object；
Step 2, user stretch out another hand B, in body near-end, concrete operations of the expansion for virtual display object；
Step 3, near-end gesture recognition system judges hand A gestures when being " starting to operate " gesture, and hand B gesture operation isIt can come into force；
Step 4, after near-end gesture recognition system obtains effective hand B operations, and mainframe computer system interaction, driving masterOriginal control logic of computer system；
Including following operation：
1) user stretches out a hand A, and in body near-end, palm stretches, palm or the back of the hand upward, with hand A palm orThe back of the hand represents user and starts to operate two dimensional surface object；
2) user stretches out the concrete operations that another hand B be directed to two dimensional surface object, including situations below：
Hand B stretches out a finger, in hand A palm or the back of the hand, by singly refer to click, the double click operation two-dimensional object, closelyEnd gesture recognition system is mapped as to the clicking of two dimensional surface object, double-clicks order；
Hand B stretches out two fingers, in hand A palm or the back of the hand, refers to opening, closed procedure, near-end gesture identification by doubleSystem is mapped as amplification, the order of diminution to two dimensional surface object；
Hand B stretches out some finger B1, and slide, the mapping of near-end gesture recognition system are carried out in hand A palm or the back of the handFor the scroll command to two dimensional surface object；
Hand B stretches out some finger B2, and in hand A palm or the back of the hand slide, near-end gesture recognition system is mapped as pairThe drag command of two dimensional surface object；
Hand B stretches out some finger, point hitter A some finger, and near-end gesture recognition system is mapped as to two dimensional surface pairAs or 3 D stereo object prompt operation order；
Hand B stretches out some finger, by the prompt operation order of some finger of hitter A, starts handwriting recognition pattern, usesFamily is operated using hand B in the hand A centre of the palm or the back of the hand, the writing for carrying out English alphabet or Chinese character, the mapping of near-end gesture recognition systemFor the input of English alphabet or Chinese character；
Singlehanded gesture operation scene include voice or brain wave control device and its comprising mainframe computer system, gesture catchesCatch camera, near-end gesture recognition system；Voice or brain wave control device gather the voice or brain wave information of user in real time,The voice or eeg signal of user is identified by the equipment；Gesture-capture camera is used to catch user gesture, and gesture-capture is taken the photographAs head is connected with near-end gesture recognition system；Voice or brain wave control device mainframe computer system and near-end gesture recognition systemConnection；Near-end gesture recognition system is used for after the near-end gesture operation of user is identified, and voice or brain wave control deviceMainframe computer system interaction, drive original control logic of mainframe computer system, specifically include following steps：
First, user stretches out a hand A, in body near-end；Hand A takes posture to represent user to start to operate the object；
2nd, the gesture is passed to near-end gesture recognition system by gesture-capture camera, and near-end gesture recognition system judgesWhen hand A gesture is starts operation, near-end gesture recognition system and the mainframe computer system of voice or brain wave control device are handed overMutually, voice or brain wave control device just start to analyze the voice or eeg signal of user, voice or brain wave control deviceAccording to the voice or eeg signal analyzed, send corresponding control signal and be output to miscellaneous equipment；
Further, hand B uses Microsoft Hololens holographic glasses standard operation postures in bimanual input operation scenario, closelyEnd gesture recognition system is mapped as internal command and performed.
Further, the posture that hand A takes is not limited to clench fist, palm stretches.
Further, the computer system of near-end gesture recognition system can be unified with the computer system of main equipment.
Compared with prior art, the present invention has following technique effect：
The present invention completes for most of gesture operation in user's body near-end can, reduces user and lifts hand in the airOperation when interference to other people, improve the disguise of gesture operation.
Technical scheme, when avoiding surrounding environment by way of bimanual input combination and have other people, systemOther people unconscious gesture is judged as system command by mistake, so as to improve the security of system.
Technical scheme, combined by voice or brain wave control device and gesture, avoid user in non-controllingDuring state, interactive voice or brainwave activity are mistaken as system command, improve the compatibility of system；Voice control device andGesture, which coordinates, can also remove the worry that user frequently says wake-up word from；
The present invention completes gesture operation by the both hands of user oneself, can replace the miscellaneous equipments such as mouse, keyboard, soWith cost it is low, be easily achieved the characteristics of.
Embodiment
With reference to embodiment, the present invention is described in detail：
A kind of gestural control method, including bimanual input operation scenario and singlehanded gesture operation scene；
Bimanual input operation scenario includes gesture-capture camera, mainframe computer system and near-end gesture recognition system；HandGesture catches camera and is used to catch user gesture, and gesture-capture camera is connected with near-end gesture recognition system；Near-end gesture is knownOther system is used for after the near-end gesture operation of user is identified, and mainframe computer system interaction, drives mainframe computer systemOriginal control logic, mainframe computer system are the original computer system of equipment for supporting gesture operation；Specifically include following stepSuddenly：
Step 1, user stretches out a hand A, in body near-end, hand A take posture to represent user to start operation virtual aobviousShow object；
Step 2, user stretch out another hand B, in body near-end, concrete operations of the expansion for virtual display object；
Step 3, near-end gesture recognition system judges hand A gestures when being " starting to operate " gesture, and hand B gesture operation isIt can come into force；
Step 4, after near-end gesture recognition system obtains effective hand B operations, and mainframe computer system interaction, driving masterOriginal control logic of computer system；
Including following operation：
1) user stretches out a hand A, and in body near-end, palm stretches, palm or the back of the hand upward, with hand A palm orThe back of the hand represents user and starts to operate two dimensional surface object；
2) user stretches out the concrete operations that another hand B be directed to two dimensional surface object, including situations below：
Hand B stretches out a finger, in hand A palm or the back of the hand, by singly refer to click, the double click operation two-dimensional object, closelyEnd gesture recognition system is mapped as to the clicking of two dimensional surface object, double-clicks order；
Hand B stretches out two fingers, in hand A palm or the back of the hand, refers to opening, closed procedure, near-end gesture identification by doubleSystem is mapped as amplification, the order of diminution to two dimensional surface object；
Hand B stretches out some finger B1, and slide, the mapping of near-end gesture recognition system are carried out in hand A palm or the back of the handFor the scroll command to two dimensional surface object；
Hand B stretches out some finger B2, and in hand A palm or the back of the hand slide, near-end gesture recognition system is mapped as pairThe drag command of two dimensional surface object；
Hand B stretches out some finger, point hitter A some finger, and near-end gesture recognition system is mapped as to two dimensional surface pairAs or 3 D stereo object prompt operation order；
Hand B stretches out some finger, by the prompt operation order of some finger of hitter A, starts handwriting recognition pattern, usesFamily is operated using hand B in the hand A centre of the palm or the back of the hand, the writing for carrying out English alphabet or Chinese character, the mapping of near-end gesture recognition systemFor the input of the English alphabet or Chinese character of browser；
Singlehanded gesture operation scene include voice or brain wave control device and its comprising mainframe computer system, gesture catchesCatch camera, near-end gesture recognition system；Voice or brain wave control device gather the voice or brain wave information of user in real time,The voice or eeg signal of user is identified by the equipment；Gesture-capture camera is used to catch user gesture, and gesture-capture is taken the photographAs head is connected with near-end gesture recognition system；Voice or brain wave control device mainframe computer system and near-end gesture recognition systemConnection；Near-end gesture recognition system is used for after the near-end gesture operation of user is identified, and voice or brain wave control deviceMainframe computer system interaction, drive original control logic of mainframe computer system, specifically include following steps：
Step 1, user stretches out a hand A, in body near-end；Hand A take posture represent user start to operate this it is rightAs；
The gesture is passed to near-end gesture recognition system, near-end gesture recognition system by step 2, gesture-capture cameraWhen judging hand A gesture to start operation, near-end gesture recognition system and the master computer system of voice or brain wave control deviceSystem interaction, voice or brain wave control device just start to analyze the voice or eeg signal of user, voice or brain wave controlEquipment sends corresponding control signal and is output to miscellaneous equipment according to the voice or eeg signal analyzed；
Hand B uses Microsoft Hololens holographic glasses standard operation postures in bimanual input operation scenario, and near-end gesture is knownOther system is mapped as internal command and performed.
The posture that hand A takes is not limited to clench fist, palm stretches.
The computer system of near-end gesture recognition system can be unified with the computer system of main equipment.
Embodiment 1, intelligent glasses user complete the operation that two dimension shows object.
Step 1, intelligent glasses start, user's browsing content；
Step 2, the display units of intelligent glasses are presented some two dimensional image, for example, user open windows edge it is clearLook at device, display unit shows edge browser interfaces；
Step 3, user stretch out a hand A, before body near-end, such as belly or chest, palm or the back of the hand upward,The edge browsers to be operated are represented with hand A palm or the back of the hand；
Step 4, user stretch out the concrete operations that another hand B be directed to edge browsers, including following three kinds of feelingsCondition；
1), hand B stretches out a finger, and in hand A palm or the back of the hand, by clicking, double click operation, system is mapped as pairEdge browser browsing contents are clicked, double-click order；
Gesture recognition system mapping process is：The gesture figure shot according to the downward one or more cameras of intelligent glassesPicture, image sequence or video, by computer system processor and analysis, draw specific gesture motion, may be referred to industry hereinRipe images of gestures and gesture motion recognizer.Then, according to the gesture motion and system control command pre-setGesture is converted to internal control command by mapping relations, gesture recognition system.
2), hand B stretches out two fingers, in hand A palm or the back of the hand, refers to opening, closed procedure by double, system is mapped asAmplification, the order of diminution to edge browser browsing contents；
Further, some finger B1 slide is stretched out by hand B, in system is mapped as browsing edge browsersThe scroll command of appearance；Some finger B2 slide is stretched out by hand B, system is mapped as the dragging to edge browser windowsOrder；
Further, hand B stretches out some finger, point hitter A some finger, and system is mapped as to edge browser interfacesSome prompt operation order.For example hand B point hitters A thumb represents to close window；
Further, hand B stretches out some finger, in hand A palm or the back of the hand, passes through the quick of a hitter A some fingerOperational order, for example, it is nameless, start handwriting recognition pattern, user is using hand B finger in the hand A centre of the palm or the back of the hand, progressThe writing operation of English alphabet or Chinese character, system are mapped as the input of the English alphabet or Chinese character of edge browsers.
Embodiment 2, intelligent glasses user complete the operation of three-dimensional display object.
Step 1, intelligent glasses start, user's browsing content；
Actions menu is presented in step 2, the display unit of intelligent glasses, and menu includes a series of three-dimensional object models；WithLower step is the example that user operates " tellurion " model；
Step 3, user stretches out a hand A, before body near-end, such as belly or chest；The posture that hand A takes is notIt is limited to posture of clenching fist, stretch, hand A given pose represents user and starts to operate；For example user starts to grasp using expression of clenching fistMake；
Step 4, user stretch out another hand B, concrete operations of the expansion for tellurion model；Hand B operations use MicrosoftIntelligent glasses hololens standard operation postures, but different from standard hololens be operating position in body near-end, near-endGesture operation is mapped as internal command and performed by gestural control system；For example coordinated by the sight focus of user, point of useHitter's gesture chooses " tellurion " model, some particular location being then placed into virtual environment.
Embodiment 3, brain wave helmet and gesture coordinate
Brain wave helmet, illustrate by taking the kether of Hangzhou carriage return Electronic Science and Technology Co., Ltd. as an example.
The racing car Kether of a brain wave control of Shenzhen carriage return science and technology typically used as scene is that user passes through brain electricityWave device drives motor sport.After user puts on eeg signal collecting device, focus on, when notice reaches setting valueAfterwards, racing car can run on runway, and the attention force value of experiencer is higher, and racing car gait of march is faster.When the note of experiencerMeaning force value drops to below setting value, and racing car will halt.
In the technical program, the step of brain wave equipment and gesture operation coordinate：
Step 1, the downward one or more of brain wave integration of equipments are used for the camera of gesture identification；
Step 2, user stretch out a hand A, in body near-end, for example are disposed close to the position of belly or chest；Hand ATake certain given pose to represent user to start to operate racing car, the posture that hand A takes is not limited to clench fist, palm stretches；Meanwhile useFamily is focused on, and starts to control racing car speed with brain wave；
Step 3, near-end gestural control system identification hand A gesture, when judging that gesture is to start the gesture of operation racing carWhen, the master computer interaction of near-end gesture recognition system and brain wave equipment, brain wave equipment starts to analyze the brain wave of userControl signal, according to eeg signal intensity, control signal is sent to racing car；If it is determined that gesture does not start to operate racing carGesture, then control signal is not sent to racing car；
Step 4, the control signal that racing car is sent according to brain wave equipment, adjust the gait of march of oneself.
Embodiment 4, voice control device and gesture coordinate
The equipment for supporting Voice command, such as Google glass；By being configured with gesture operation, Google glass is in interactive voiceWhen, it can remove from and frequently say wake-up word；In the technical program, the step of Google glass and gesture operation coordinate：
Step 1, the downward one or more of Google glass integration of equipments are used for the camera of gesture identification；
Step 2, user stretch out a hand A, in body near-end, for example are disposed close to the position of belly or chest；Hand ATake certain given pose to represent user and start interactive voice, the posture that hand A takes is not limited to clench fist, palm stretches；
Step 3, user send voice command；For example user says " photograph " using voice and ordered；
Step 4, near-end gesture recognition system identification hand A gesture, when it is to start the gesture of operation to judge gesture, closelyGesture recognition system and the interaction of the mainframe computer system of Google glass are held, Google glass just proceeds by speech recognition, and controlsMain camera completes photographic actions.