The content of the invention
The purpose of the present invention is that at least solve one of drawbacks described above and deficiency, and the purpose is to be achieved through the following technical solutions's.
The invention provides the device of a kind of integrated gesture identification and ultrasonic wave touch feedback, including：Glove bulk andThe multiple functional units being embedded on the glove bulk, the functional unit are used for the human hand in the glove bulkCarry out gesture data collection and provide ultrasonic wave touch feedback to the human hand according to the human hand gesture data collected.
Further, the multiple functional unit include be used for obtain human hand gesture data camera, be the shootingHead provides the lighting unit and ultrasonic spray nozzle component of light, wherein, the first functional unit in the multiple functional unitIncluding at least one of the camera, the lighting unit and described ultrasonic spray nozzle component, the multiple function listThe second functional unit in member includes at least another in the camera, the lighting unit and the ultrasonic spray nozzle componentOne kind, the 3rd functional unit in the multiple functional unit include the camera, the lighting unit and the ultrasoundAt least remaining one kind in ripple nozzle component.
Further, each functional unit include be used for obtain human hand gesture data camera, be the cameraThe lighting unit and ultrasonic spray nozzle component of light are provided.
Further, in addition to computing module, the computing module are used to carry out at calculating the gesture data collectedReason, gesture identification is carried out, and generate gestural feedback information.
Further, in addition to computing module, the computing module are used for the gesture data collected to the cameraCalculating processing is carried out, carries out gesture identification, and generate gestural feedback information.
Further, in addition to control module and communication module, the control module are used to receive the gestural feedbackInformation simultaneously controls the ultrasonic spray nozzle component to send ultrasonic wave according to the gestural feedback information.
Further, the ultrasonic spray nozzle component includes at least one ultrasonic spray nozzle.
Further, the communication module be USB module, bluetooth module, GPRS/3G/4G/5G modules, WiFi module,One or more in the network communication module of ZigBee module or specific protocol.
Further, the device of the integrated gesture identification and ultrasonic wave touch feedback also include built-in battery and/orThe power interface that can be connected in a wired or wireless manner with external power source.
Further, the glove bulk is fixed at wrist or the glove bulk is not fixed at wrist.
Further, the glove bulk is square body, round or other arbitrary shapes.
Further, it is flexible material or rigid outside the glove bulk.
Further, the glove bulk interior surface is covered with mono coat or patterned coating.
Further, in addition to terminal device, and the terminal device can receive the hand that the camera collectsGesture data, gesture identification is carried out, and generate gestural feedback information.
Present invention also offers a kind of ultrasound is carried out using the device of above-mentioned integrated gesture identification and ultrasonic wave touch feedbackThe method of ripple gesture identification and touch feedback, including：
Virtual environment is selected, and virtual environment information is sent to the dress to integrated gesture identification and ultrasonic wave touch feedbackPut；
The device of the integrated ultrasonic gesture identification and touch feedback human hand that collection is in glove bulk in real timeGesture simultaneously transmits the gesture information of acquisition to computing module；
Gesture identification is carried out using the computing module and generates gestural feedback information, is sent out according to the gestural feedback informationProject ultrasonic wave and touch feedback is provided.
Surpassed present invention also offers another kind using the device of above-mentioned integrated gesture identification and ultrasonic wave touch feedbackThe method of sound wave gesture identification and touch feedback, including：
Virtual environment is selected, and virtual environment information is sent to the dress to integrated gesture identification and ultrasonic wave touch feedbackPut；
The device of the integrated gesture identification and ultrasonic wave touch feedback gathers the human hand in glove bulk in real timeGesture simultaneously transmits the gesture information of acquisition to terminal device；
Gesture identification is carried out using the terminal device and generates gestural feedback information, by the gestural feedback information transferTo the device of the integrated ultrasonic gesture identification and touch feedback；
Gestural feedback information is received using the device of the integrated gesture identification and ultrasonic wave touch feedback, and according to describedGestural feedback information launches ultrasonic wave and provides touch feedback.
Further, in addition to according to the gestural feedback information radiation pattern of ultrasonic wave is adjusted.
Further, adjusting the radiation pattern of ultrasonic wave includes the pulse of regulation ultrasonic wave, the frequency for adjusting ultrasonic wave, adjustsSave the one or more in the direction of the launch of ultrasonic wave, the intensity of regulation ultrasonic wave.
The invention further relates to the device of above-mentioned integrated gesture identification and ultrasonic wave touch feedback existing in virtual reality, enhancingPurposes in reality or mixed reality.
Advantages of the present invention is as follows：
(1) strong sense of reality of touch feedback of the present invention, multiple angles, the sense of touch of dynamics can be simulated, is further increasedThe fidelity and degree of immersing of virtual reality experience.
(2) the integrated gesture identification of the present invention simplifies algorithm, so as to carry to that use environment can be made to become simple inside glovesHigh recognition speed and accuracy of identification.
(3) gesture identification and touch feedback are integrated into gloves by the present invention, and device integration is high, perfect can substitute handHandle controller.
(4) small volume of the present invention, in light weight, can dress and operate for a long time.
Embodiment
The illustrative embodiments of the disclosure are more fully described below with reference to accompanying drawings.Although this public affairs is shown in accompanying drawingThe illustrative embodiments opened, it being understood, however, that may be realized in various forms the disclosure without the reality that should be illustrated hereThe mode of applying is limited.Conversely, there is provided these embodiments are to be able to be best understood from the disclosure, and can be by this public affairsThe scope opened completely is communicated to those skilled in the art.
Fig. 1 is that the integrated gesture identification that the specific embodiment of the invention provides and the assembling of the device of ultrasonic wave touch feedback are shownIt is intended to, as shown in figure 1, the device of a kind of integrated gesture identification based on ultrasonic wave of the present invention and ultrasonic wave touch feedbackIncluding glove bulk 1 and the multiple functional units 2 being embedded on glove bulk 1, each functional unit 2 is used in gloves sheetIt is simultaneously anti-to human hand offer ultrasonic wave tactile according to the human hand gesture data collected that human hand in body 1 carries out gesture data collectionFeedback.
Fig. 2 is the integrated gesture identification and ultrasonic wave touch feedback based on ultrasonic wave that the specific embodiment of the invention providesThe structural representation of the functional unit of device, as shown in Fig. 2 each functional unit 2 includes being used to obtain human hand gesture dataCamera 201, the lighting unit 202 and ultrasonic spray nozzle component 203 of light are provided for the camera.And ultrasonic wave spraysHead assembly 203 is set by program from the position of finger according to the hand that camera 201 provides and carried out with different angle and intensityUltrasonic wave transmitting forms sense of touch with this.
Functional unit 2 it has been covered with inside glove bulk 1, user can utilize functional unit 2 to press touching for dummy objectSense is projected on the both hands of user, allows user in the case of not contact screen in obtaining touch feedback in the air.
In the present embodiment, the functional unit 2 is evenly distributed on each position inside glove bulk 1, in another realityApply in example, appropriate additions and deletions combination can be carried out to the functional unit 2 at some positions.For example, finger, the activity of palm of the hand position compared withMore, activity is less at the back of the hand, can suitably reduce the arrangement of functional unit 2 at the back of the hand, the additions and deletions combination bag of this functional unit 2Include but be not limited to the present embodiment.
In addition, as shown in figure 4, functional unit 2 is not fully identical, some of include camera 201, otherComprising lighting unit 202, there are some only to include ultrasonic spray nozzle 203, their quantity and position distribution is not fully identical,It can be arranged as required to.
The first functional unit in multiple functional units 2 includes camera 201, lighting unit 202 and ultrasonic spray nozzleAt least one of component 203, the second functional unit in multiple functional units 2 include camera 201, lighting unit 202 withAnd it is at least another in ultrasonic spray nozzle component 202, the 3rd functional unit in multiple functional units 2 include camera 201,At least remaining one kind in lighting unit 202 and ultrasonic spray nozzle component 203.
The device energy for the ultrasonic wave touch feedback being made up of the first functional unit, the second functional unit, the 3rd functional unitEnough complete complete gesture identification and touch feedback.
In another embodiment, multiple functional units 2 can also be embedded on a monoblock surface-mounted integrated circuit 6, then will be integratedCircuit board 6 is arranged inside glove bulk 1.As shown in figure 4, multiple functional units 2 are arranged on surface-mounted integrated circuit 6, by functionThe concentrated setting of unit 2 is on a monoblock surface-mounted integrated circuit 6, it is possible to reduce electrical connection.
According to embodiment of the present disclosure, the lighting unit 202 of the functional unit 2 can send visible ray or can notLight is seen so as to track identification gesture motion, for example, lighting unit 202 can send infrared light, even in daytime, camera 201Can also accurate track identification gesture information, do not influenceed by light.In addition, the lighting unit 202 can be according to environment notWith color, frequency or the different light of brightness is sent so as to by the track identification of camera 201, for example, on daytime, because light fillsFoot, lighting unit 202 emits white light, camera 201 can not accurate track identification gesture information, if glowed, camera 201Can accurate track identification gesture information；At night, due to dark, lighting unit 202 can emit white light, to be imagedFirst 201 track identification；For another example, in gaming, according to game progress content, in heavy fighting, lighting unit 202 sends pulseQuick flashing feux rouges, in the gentle stage, lighting unit 202 sends the blue light of pulse；When playing indoors, lighting unit 202 sends brightRelatively low light is spent, when scene of game is transferred to outdoor, lighting unit 202 sends higher light of brightness etc..
Each ultrasonic spray nozzle of ultrasonic spray nozzle component 203 can spray in check power, and such ultrasonic wave is justWhole limbs that it is surrounded can be covered.Each shower nozzle is more compact simultaneously, can perfect cloth to variously-shaped surfaceOn, allow hand to experience the feedback of power, bring more real experience to user, improve the fidelity of virtual reality experience, for example,In gaming, two players are grappled in virtual world, once other side is hit, or oneself is hit, and ultrasonic spray nozzle all canSkin is stimulated to produce sense of touch, when oneself is hit, hand produces sense of touch, feels all right as very by hitting.
The quantity of ultrasonic spray nozzle component 203 on each functional unit 2 can be arranged as required to, as shown in figure 3, figureIn show the present invention functional unit 2 another embodiment structural representation, ultrasonic spray nozzle component 203 is by 3 ultrasoundsRipple shower nozzle forms, and each ultrasonic spray nozzle can spray in check power to stimulate the neuron of hand, is created in handDifferent senses of touch.
User can also be adjusted to the effect of touch feedback, such as the pulse of regulation ultrasonic wave or frequency weaken feedbackDynamics, or increase touch feedback duration come meet personalization Consumer's Experience demand.
Camera 201 and ultrasonic spray nozzle component 203 are integrated on functional unit 2, can be good at tracking human handEach joint, and accurately catch human hand various gestures and action, no matter day and night, can be with round-the-clock recognition and tracking handPortion acts, and realizes gesture identification and control.It can additionally make the virtual article in user and virtual environment is interactive to produce more certainlyFeeling of immersion touch feedback is so had more, substantially increases Consumer's Experience.Gesture identification tracking and touch feedback are incorporated in by the present inventionIn a compact and comfortable equipment, virtual reality experience is set to have more interactivity, more feeling of immersion and more on the spot in person.
Fig. 5 is the structural representation of the integrated ultrasonic sense of touch that the specific embodiment of the invention provides and gesture identifying device,As shown in figure 5, glove bulk 1 also includes control module 3, communication module 4 and computing module 5.
Computing module 5 electrically connects with control module 3, the spatial positional information for each position of the human hand to collectingCalculating processing is carried out by algorithm with gesture information, human hand gesture is carried out to be accurately positioned identification, and generate gestural feedback information.Identification calculating process of the computing module 5 to gesture is not limited to a kind of or a few specific algorithms of class.Can be light using computing module 5It is easy to identify go out user's finger position, improve the degree of accuracy of track identification, it is convenient in virtual environment that user's finger is modeled, carryThe fidelity of high virtual environment, strengthen user experience.
Control module 3 is installed at the back of glove bulk 1, control module 3 is surface-mounted integrated circuit and main control chip, by ultra-fineWire electrically connects each functional unit 2.Control module 3 is used for the human hand gesture identification feedback information for receiving the offer of computing module 5And control ultrasonic spray nozzle component 203 to send ultrasonic wave according to gestural feedback information, generate touch feedback.
In another embodiment, when the present apparatus carries out gesture identification, without that by computing module 5, will directly can imageThe first 201 human hand gesture datas collected pass to terminal device by communication module 4, are entered using the preset algorithm of terminal deviceRow gesture identification.
The human hand gesture data that camera 201 collects is transmitted by communication module 4 will to terminal device, terminal deviceThe human hand gesture data and virtual environment information collected combines, and using the programmed algorithm built in terminal device, generates human hand handGesture feedback information, and human hand gestural feedback information is sent to control module 3 by communication module 4, control module 3 is according to receptionFunctional unit 2 is arrived in the human hand gestural feedback delivering tactile instruction arrived, and the ultrasonic spray nozzle component 203 of functional unit 2 is from eachIndividual orientation hand launches ultrasonic wave, with this to form sense of touch, and gives the tactile feedback generated in virtual environment to operationPerson, operator is set to produce feeling of immersion on the spot in person.
Communication module 4 is fixed on by control module 3, for connecting the device of integrated gesture identification and ultrasonic wave touch feedbackAnd terminal device, realize communication and data exchange, communication module 4 be USB module, bluetooth module, GPRS/3G/4G/5G modules,One or more in the network communication module of WiFi module, ZigBee module or specific protocol.In preferred embodiment, it will communicateModule 4, which is arranged to wireless mode, can allow the present apparatus by space constraint, not improve the free degree used.
The power supply of control module 3 and functional unit 2 supplies, and can both be supplied by the internal battery in gloves, can alsoExternal power supply is connected by wired or wireless power interface to power, the present invention is not limited.
Glove bulk 1 is used to carry and fixed-function unit 2 and other modules, wearable in user's wrist portion, in wristPlace can fix glove bulk 1 with human hand, and human hand is in the internal activity of glove bulk 1.Glove bulk 1 is fixed on to the design of wristGloves during Consumer's Experience can be effectively avoided excessively to loosen and lose feeling of immersion, the flexibility of finger-joint and the dynamics of finger canIt is accurate to pass on, so that user interacts naturally with dummy object.
In other embodiments of the present invention, glove bulk 1 can not be fixed with wrist, be directly disposed as fixingState, i.e., glove bulk 1 is fixedly installed on the table, human hand is movable in glove bulk 1, can now facilitate custom to be sitting inUser's clawing thing body before computer, makes the finger of user more flexible, VR experience of the process is more naturally comfortable.
Glove bulk 1 can be that square body, round or other arbitrary shapes, the present invention are not limited to this, as long as energyEnough accurate recognition and trackings and touch feedback realized to hand motion.For example, it may be flexible glove type orIt is spherical, it might even be possible to not to be totally enclosed (such as Semi surrounding type handguard of the foil in west).
The outside of glove bulk 1 is flexible material or rigid, and glove bulk 1 uses ventilative flexible material, Ke YirangUser's wearing is more comfortable, and it is flexibly movable to be also convenient for hand., can using rigid when glove bulk 1 is arranged into stationary stateIt is convenient to fix.
The interior surface of glove bulk 1 is covered with mono coat or patterned coating, is arranged to patterned coating, these figuresCase and the mark wherein included will be easy to position the relative and absolute position at each position of human hand during gesture identification.
As shown in fig. 6, according to embodiment of the present disclosure, the dress of gesture identification and ultrasonic wave touch feedback is integrated using thisThe method for putting progress touch feedback and gesture identification comprises the following steps：
1) virtual environment is selected, and virtual environment information is sent to the dress to integrated gesture identification and ultrasonic wave touch feedbackPut；
Specifically, after switching on power, the device for integrating gesture identification and ultrasonic wave touch feedback is started working, in the deviceSuitable virtual scene information is selected on corresponding software.
2) the real-time hand for gathering the human hand in glove bulk of device of gesture identification and ultrasonic wave touch feedback is integratedGesture simultaneously transmits the gesture information of acquisition to computing module；
Human hand activity in device of the integrated gesture identification with ultrasonic wave touch feedback, the camera 201 of functional unit 2 are realWhen gather human hand gesture information, and the human hand gesture information of acquisition is transmitted to computing module 5, to provide accurate gesture identificationTracking.
3) carry out gesture identification using computing module and generate gestural feedback information, launched according to the gestural feedback informationGo out ultrasonic wave and touch feedback is provided.
Specifically, computing module 5 is entered after the gesture motion data that the camera 201 gathers are received using algorithmThe processing of row analytic uniform, carries out gesture identification, and generate gestural feedback information.
After integrated gesture identification and the control module 3 of the device of ultrasonic wave touch feedback receive gestural feedback information, controlThe ultrasonic spray nozzle component 203 of the control function unit 2 of molding block 3 launches ultrasonic wave to hand from different angles, is come with thisSense of touch is formed, and operator is produced feeling of immersion on the spot in person to operator the tactile feedback generated in virtual environment.
The radiation pattern of ultrasonic wave is adjusted, the effect of touch feedback can be adjusted, disclosure satisfy that the use of personalizationFamily demand for experience.Adjusting the radiation pattern of ultrasonic wave includes the pulse of regulation ultrasonic wave, the frequency for adjusting ultrasonic wave, regulation ultrasoundOne or more in the direction of the launch of ripple, the intensity of regulation ultrasonic wave.
As shown in fig. 7, in another embodiment, the device of the integrated gesture identification and ultrasonic wave touch feedback is only responsible forThe collection of gesture data, the process of gesture identification occur in terminal device, and the method for the touch feedback and gesture identification is included such asLower step：
1) virtual environment is selected, and virtual environment information is sent to the dress to integrated gesture identification and ultrasonic wave touch feedbackPut.
2) the real-time hand for gathering the human hand in glove bulk of device of gesture identification and ultrasonic wave touch feedback is integratedGesture data simultaneously transmit the gesture information of acquisition to terminal device.
Terminal device carries out analytic uniform processing after the gesture motion data that the camera 201 gathers are received.
3) carry out gesture identification using terminal device and generate gestural feedback information, and gestural feedback information transfer is extremely collectedInto the device of gesture identification and ultrasonic wave touch feedback.
Specifically, the human hand gesture that terminal device will receive from the device of integrated gesture identification and ultrasonic wave touch feedbackInformation and virtual environment information, which combine, carries out data processing with after parsing, carries out gesture identification, and generate gestural feedback information.TogetherWhen, when terminal device detects that hand needs touch feedback (for example, in virtual scene, when hand touches virtual objectWhen, object can give the touch feedback of one power of hand) when, gesture feedback information can be passed through communication module by terminal device4 are sent to the device of integrated gesture identification and ultrasonic wave touch feedback.
4) gestural feedback information is received using the device of integrated gesture identification and ultrasonic wave touch feedback, and according to the handGesture feedback information launches ultrasonic wave and provides touch feedback.
After integrated gesture identification and the control module 3 of the device of ultrasonic wave touch feedback receive gestural feedback information, controlThe ultrasonic spray nozzle component 203 of the control function unit 2 of molding block 3 launches ultrasonic wave from each orientation hand, carrys out shape with thisOperator is set to produce feeling of immersion on the spot in person to operator into sense of touch, and by the tactile feedback generated in virtual environment.
For ease of understanding, the present embodiment is described with reference to specific application scenarios：
First, the device of integrated gesture identification and ultrasonic wave touch feedback is connected to computer by user by WiFi,Suitable scene information is selected corresponding to computer on software, such as occurs earth model in screen.User exists to find somewherePosition on earth model, the action of " the pressing earth " is made, the integrated gesture identification and ultrasonic wave tactile that user is dressed are anti-Functional unit 2 in the device of feedback follows the trail of hand motion, obtains hand motion information, and send in real time to computer, computerThe hand motion information and virtual scene information binding analysis are obtained gestural feedback information, then computer is the gestural feedbackInformation sends the device to integrated gesture identification and ultrasonic wave touch feedback so that integrated gesture identification and ultrasonic wave touch feedbackDevice in ultrasonic spray nozzle component 203 launch ultrasonic wave according to corresponding to the gesture hand feedback information, feel userApplied pressure on hand by strong, produce the similar sense of touch for touching earth model.Meanwhile user can also use the integrated gesture to knowDevice not with ultrasonic wave touch feedback is moved and lifted to the earth model in virtual scene, and produces real moveSensation that is dynamic or lifting.
The present embodiment increases touch feedback in the virtual scene of vision, is realized and virtual scene phase by the activity of handInteraction, a kind of more direct, more natural man-machine interaction mode is provided for operator, improves the experience of user.
In another embodiment of the present invention, integrating the device of gesture identification and ultrasonic wave touch feedback can apply toHelp on patients ' recovery.Traditional rehabilitation project helps postoperative patient to recover locomitivity, example by the way of 2D picturesSuch as, an apple is shown on 2D screens, allows rehabilitation to go to be held by hand, and a reward is shown after catching.Using the present inventionPatient can be allowed really to have the apple of catching of distance perspective, and tactile feedback.
The device of integrated gesture identification provided by the invention and ultrasonic wave touch feedback can also be set with other virtual realitiesStandby to be used cooperatively, virtual scene information carries out Projection Display by glasses, such as when user puts on virtual reality glasses and operates meterCalculation machine, can obtain the study virtual scene information with desk, virtual environment can in virtual reality glasses real-time display, when withWhen family touches desk, it can not only see that 3D scenic pictures are demonstrated by virtual reality glasses, moreover it is possible to pass through the collection based on ultrasonic waveDevice into gesture identification and ultrasonic wave touch feedback feels the action of touch desk, allows user preferably to experience voidThe change in near-ring border, increase and the interaction of virtual environment.
The device of integrated gesture identification provided by the invention and ultrasonic wave touch feedback is high by gesture identification and touch feedbackDegree is integrated on a miniature functional unit, and functional unit is embedded in inside glove bulk, Real-time Feedback operation by human hand, is realizedAccurate identification tracking to user's hand motion, and the tactile data that terminal device is fed back is converted into directly acting on peopleThe power or torque of hand, operator is produced the touch telepresence effect of " on the spot in person ", produced so as to realize in virtual environmentIt is real to touch impression, and it overcomes that handle controller complicated in mechanical structure, the scope of activities of hand be small and traditional data handSet does not have the shortcomings that force feedback function, has extensive prospect of the application, can be applied to virtual reality, augmented reality and mixingIn the various technologies such as reality.
According to embodiment of the present disclosure, terminal device can be the equipment for possessing communication function.For example, terminal device canBe Desktop PC computer, smart mobile phone, tablet personal computer, portable media player, portable medical device, digital camera orWearable device etc..
It is pointed out that the action pointed out in illustrative methods can not be according to suitable shown in accompanying drawing or described in specificationSequence occurs.For example, continuous two actions may be substantially to perform upon execution simultaneously, or partial act is when someTime can perform in reverse order, and this is by depending on involved function.
It is to be appreciated that to realize above-mentioned gesture identification and touch feedback, must there are a set of corresponding driving and application software(not shown), which coordinates, to be completed.
The foregoing is only a preferred embodiment of the present invention, but protection scope of the present invention be not limited thereto,Any one skilled in the art the invention discloses technical scope in, the change or replacement that can readily occur in,It should all be included within the scope of the present invention.Therefore, protection scope of the present invention should be with the protection model of the claimEnclose and be defined.