US7312699B2 - Ear associated machine-human interface - Google Patents

Abstract

A human-machine interface can detect when a user's ear is pulled back to initiate a plurality of procedures. Such procedures include turning on a TV using a laser attached to the user, starting an additional procedure by speaking a command, communicating with other users in environments which have high ambient noise, and interacting with the internet. Head position sensors are used to detect the position of the head of a user and to either initiate a procedure if a characteristic of the head position or positions meets a certain criteria, or to pass the head position information to another device.

Description

This application claims the benefit of U.S. Provisional Application No. 60/459,289 filed Apr. 1, 2003.

FIELD OF THE INVENTION

The present invention generally relates to a human-machine interface structure and method.

BACKGROUND OF THE INVENTION

There are many human activities which can be made possible or made easier using a human-machine interface wherein a human can select certain options, such as turning a TV on or off, without having to use his or her hands, communicate with a computer using only his or her voice. Also, information about the condition of a person such as heart rate for example can be monitored without restricting the movements of the person.

Human-machine interface structures are known in the art. For example U.S. Pat. No. 6,696,973 to Ritter et al., and the references cited therein, teach communications systems which are mobile and carried by a user. U.S. Pat. No. 6,694,180 to Boesen describes biopotential sensing and medical monitoring which uses wireless communication to transmit the information from the sensors.

However, a human-machine interface that is convenient to use and is relatively inexpensive to manufacturer is still highly desirable.

SUMMARY OF THE INVENTION

Shown in a preferred embodiment of the present invention is a transmitting apparatus having a sensor for detecting an ear pull of a user and a laser worn by the user. An electronic module is coupled to both the ear pull sensor and the laser and generates a laser beam upon detection of the ear pull.

Also shown in a preferred embodiment of the present invention is a transmitting apparatus for a user which has a plurality of sensors for detecting a head position of the user, a RF transmitter and an electronic module coupled to the plurality of sensors and to the RF transmitter. The electronic module generates an encoded RF signal containing information about the head position of the user.

Further shown in a preferred embodiment of the invention is a communication apparatus including a portable computer worn by a user together with a microphone and speaker worn by the user and an electronic module. The electronic module is coupled to the microphone, the speaker and the portable computer and receives a voice message from the microphone and sends the voice message to the portable computing device, wherein the portable computing device, in response to the voice message, sends an answering audio communication to the electronic module which, in turn transfers the audio communication to the speaker.

Still further shown in a preferred embodiment of the present invention is a method for transmitting commands including sensing when an ear of a user is pulled back and turning on a laser mounted on the user when the sensing occurs.

OBJECTS OF THE INVENTION

It is, therefore, an object of the present invention to provide human-machine interface that is convenient to use and is relatively inexpensive to manufacture.

Another object is to provide a head worn communications device which communicates when a user pulls back one of his or her ears.

A further object is to provide a human-machine interface that will communicate with a plurality of devices.

A still further object of the present invention is to provide a method for communicating the head position of a user to other device.

An additional object of the present invention is to provide a hands free communication between a user and the internet.

In addition to the above-described objects and advantages of the present invention, various other objects and advantages will become more readily apparent to those persons who are skilled in the same and related arts from the following more detailed description on the invention, particularly, when such description is taken in conjunction with the attached drawing, figures, and appended claims.

DESCRIPTION OF THE DRAWING

FIG. 1A is a block diagram of one embodiment of the human-machine interface of the present invention;

FIG. 1B isFIG. 1A with several elements removed to show one minimal configuration of the present invention;

FIG. 1C shows an alternative embodiment in which a modulated retroflector is worn on each side of the head of auser14.

FIG. 2 isFIG. 1A modified to show other types of devices that can be used with the human-machine interface of the present invention;

FIG. 3 shows two sides of a user's head; and

FIG. 4 is the user ofFIG. 1A wearing a helmet with a laser detector mounted on the helmet.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENT

Prior to proceeding to a much more detailed description of the present invention, it should be noted that identical components which have identical functions have been identified with identical reference numerals throughout the several views illustrated in the drawing figures for the sake of clarity and understanding of the invention.

Turning now to the drawings,FIG. 1A shows several biometric devices inside adashed line box10 proximate to anear12 of anuser14. Theuser14 also has a pair ofglasses16. Mounted on thetemple piece18 of theglasses16 is alaser20 and acamera22. Also shown inFIG. 1A is a portable computing device which, in the preferred embodiment of the invention, is a personal data assistant (PDA)24 with a location sensing device which, in the preferred embodiment of the invention, is a local positioning system (LPS) module or a global positioning system (GPS)module26 attached thereto, acomputer28 connected by acable30 to theinternet32 and aTV set34.

The biometric devices inside thedashed line box10 include muscle actuation detectors which, inFIG. 1A, is astrain gauge36 attached to the skin of theuser14, asecond strain gauge38 attached to or embedded in thetemple piece18, athird strain gauge40 attached to the user's skin and positioned at least partially behind theear12 of theuser14, afourth strain gauge41 placed on the bridge of theglasses16, capacitance plates42 (attached to the back of the ear12) and44 (attached to the head behind the ear12), anear lobe clip46 and a combination microphone and an ambientnoise reducing speaker48 placed inside theear12. Also shown is aRFID chip47 placed underneath the skin of theuser14 behind theear12. The RFID chip could also be attached less intrusively by placing the RFID chip in an ear ring or in theear clip46, or attaching a RFID chip to theear12 by two magnets acting as a clamp. Thecapacitance plates42 and44, thestrain gauges36,38 and40 and theear lobe clip46 are connected by wires to anelectronic module50. Theelectronic module50 contains abattery51 to power theelectronic module50, twotilt meters52, and amagnetic sensor54. The twotilt meters52 measure the tilt from horizontal from a direction from the back to the front of the user's head, and from a direction from one ear to the other ear. Themagnetic sensor54 senses the direction of the earth's magnetic field. The twotilt meters52 and themagnetic sensor54 are used to determine the position of the user's head.

TheTV34 has alaser light sensor56 which responds in a predetermined manner upon detecting a laser light modulated with a predetermined code.

The system shown inFIG. 1A can operate in a number of different ways. In a relatively simple application, theuser14 aims thelaser20 atsensor56 and wiggles or pulls back theear12 by pulling back theear12. Only one of theear movement sensors36,38,40 and the combination of theplates42 and44, is needed, forexample strain gauge38. Other ear movement detectors could also be used such as detectors that detect the change in capacitance betweencapacitor plates44 and45 or betweenplates45 and49, the capacitance between the body of theuser14 andcapacitance plate44 or between the frames of theglasses16 and thecapacitance plate44. Also, theear12 movement can be detected by detecting a change in the magnitude of an RF field or a magnetic field using a detector in theelectronic module50. The RF generator or magnet could be located in theear clip46. Also the resistance of the user's skin proximate to theear12 would change sufficiently to detect anear12 movement. Thestrain gauge38, together with theelectronic module50, detects the change of the strain in thetemple piece18 when theear12 is pulled back. When the ear movement is detected, theelectronic module50, connected to thelaser generator20 by wires hidden behind or inside thetemple piece18 of theglasses16, causes thelaser20 to send the predetermined code which activates thesensor56 to turn on or turn off theTV set34. This simple application uses components that are relatively inexpensive to manufacture.

Thelaser20 could have a beam which is narrow or which diverges to cover a larger area than a narrow beam. Thelaser20 could have a variable divergence that the user could adjust. Thelaser20 could also be replaced with other types of light sources such as an LED, LCD or a flashlight. Still other types of signaling means could be used such as an ultrasonic generator or a high frequency (i.e., 60 Ghz) transmitter which would generate a narrow RF signal could be used.

Other types of strain gauges, such as the flexible strain gauge shown in U.S. Pat. No. 6,360,615 to Smela which could be applied to the back of theear12.

Detecting the movement of theear12 using a capacitance detector can also be accomplished by attaching or embedding two capacitor plates in thetemple piece18 of theglasses16 thereby eliminating the need to attach the capacitor plates to the skin of theuser14. The movement of theear12 can be detected by the change of capacitance between the two plates.

FIG. 1B shows a minimal configuration of the human-machine interface of the present invention which uses only thelaser20,strain gauge40 andelectronic module50 to control theTV set34. Anear bracket63 is used to hold the human-machine interface components behind theear12 of theuser14.

FIG. 1C shows an alternative embodiment where a modulated retroflector is worn on each side of the head of auser14. The modulated retroflector shown inFIG. 1C is worn as astud ear ring65 or adangle ear ring67. The modulatedretroflector65,67 could also be partially covered by placing the modulatedretroflector65,67 in the hair of theuser14. In operation theTV set34 would emit either a light signal or an RF signal from a combination transmitter andreceiver69. The signal from the combination transmitter andreceiver69 would be received by both of the modulatedretroflectors65,67 on each side of the head of theuser14 when theuser14 is looking at theTV set34, and at least one of the modulatedretroflectors65,67 will not receive the signal if theuser14 is looking in another direction.

Each of the modulatedretroflectors65,67 will, upon receipt of a signal from the combination transmitter andreceiver69 emit a light or RF signal which will be received by the combination transmitter andreceiver69. The combination transmitter andreceiver69 will be able to detect if both modulatedretroflectors65,67 on theuser14 are responding by detecting differences in the signals sent by each modulated retroflector. Such differences could be different frequencies or codes sent by each modulatedretroflector65,67. When theuser14 pulls backear12, the modulatedretroflectors65,67 will change signals that the combination transmitter andreceiver69 will detect. If the combination transmitter andreceiver69 detects the change in signal from both modulatedretroflectors65,67 the electronics in theTV set34 will perform a predetermined procedure such as turning on the TC set34.

TheTV set34 could haveadditional sensors58 for controlling other TV functions such as volume control while theear12 is pulled back. The volume increases using one of thesensors58 and decreases using another of thesensors58. Two other of thesensors58 could be used to select the TV channel in the same manner.

Theelectronic module50 can communicate with thePDA24 and thecomputer28 by wireless communication such as the Bluetooth protocol. Thecomputer28 can, in turn, communicate with theinternet32. Using the combination microphone andspeaker48 theuser14 can send audio information to theelectronic module50 which can then digitize the audio signal and send it to thePDA24 for voice recognition. If the audio is too complex for thePDA24, the audio can be sent to thecomputer28 for voice recognition. Thecomputer28 can access theinternet32 for help in the voice recognition if necessary. Finally if none of the equipment inFIG. 1A can recognize the audio, the PDA communicating to theelectronic module50 and the combination microphone andspeaker48 can tell theuser14 to repeat the statement or can ask specific questions of theuser14 which theuser14 can answer by pulling back theear12 either once or twice to answer a yes or no question.

There could also be a set of predetermined voice commands that theuser14 is restricted to. The voice recognition software to recognize the limited list of commands is less complex and more accurate than the software needed to recognize all words. Such voice commands as “channel59” when theear12 is pulled back would be decoded either directly by theelectronic module50 or by thePDA24, encoded and sent back to theelectronic module50 which would, in turn, modulate the laser beam from thelaser20 with the correct code which thesensor56 would decode and theTV set34 would change the channel to channel59. The laser beam would therefore have to aimed at thesensor56 to transmit the encoded laser beam signal to theTV set34. The same sequence could be used to set a thermostat, a VCR, etc.

There are some operations which do not require the use of thelaser20. For example auser14 could say “time” while pulling back theear12 and the time in an audio format would be sent to the speaker in the combination microphone andspeaker48. Also, a telephone number could be spoken and a telephone call would be made, and the call could be terminated when theuser14 says “hang up”.

In this manner more complex commands and communication can be achieved such as using the biometric device and system to simply record an audio message to communicating to any other applications such as viewing and taking a picture of a home appliance that needs repair and having thePDA24, thecomputer28 and the internet recognize the appliance and providing information needed to repair the appliance.

Thelaser20 can be used to send commands to or query many products such as notifying a traffic light that the user wants to cross the street along with the amount of time the user needs to cross the street. The laser could also be used by emergency personnel to cause traffic lights to turn green for them when they are going to an emergency.

Pulling theear12 back can simply be a single pull or can be a more complex action such as pulling back and holding theear12 back until a object, such as a TV, reaches a desired set point, such as reaching the wanted channel. Other actions can be to pull back theear14 twice within 2 seconds, etc. Even more complex movements can be used such as movements which may resemble Morse code signals or be actual Morse code. It is believed that some individuals with training can eventually control the movement of either ear separately and independently, thus generating a user interface capable of even more selectivity, complexity and discrimination.

Also, for a novice user the ear can be pushed back by hand until the user develops the ability to pull back his or her ear without using a hand.

Theear clip46 can be used to monitor the user's physical condition such as pulse rate and pulse oximetry. Other sensors can be attached to the user and wired to theelectronic module50 such as an accelerometer for monitoring other body parameters such as whether theuser14 has a fever on not and whether the person is awake, has fallen, etc.

A simple driving drowsiness detector can be made by having theelectronic module50 issue sporadic random tones to theuser14 using the combination microphone andspeaker48 and requiring theuser14 to respond with an ear wiggle movement at that time. The response delay would indicate the level of a user's reflex time and degree of sleepiness. A prolonged delay would result in a much louder tone to wake up theuser14.

Using a camera, either thecamera22 or another camera, theuser14 could pull back theear12 and say “camera mode” to tell theelectronic module50 to cause the camera to take a picture when theear12 is pulled back. Other camera mode activation means could be used such as a sequence of ear pulls. If the camera is a stand alone camera and the orientation of the camera can be remotely controlled, thetilt sensors52 andmagnetic sensor54 would be used to detect the what area theuser14 is looking at, and the camera would also point at the same area. Thus theuser14 at a sporting event could aim the camera and command the camera to take a picture simply by looking in the desired direction and pulling theear12 back to take a picture.

The combination microphone andspeaker48 could also contain an actuator which would provide tactile signaling for situations such as when the ambient noise is too high for reliable communication using the combination microphone andspeaker48 alone. The tactile signaling could be a signal touch or could be a pattern of touches.

Theelectronic module50 and the combination microphone andspeaker48 could be used as a cell phone with the proper electronics inside themodule50.

FIG. 2 shows the biometric system ofFIG. 1A, but is more generalized as to devices that the laser beam can be used on. Thetarget60 can be a stereo sound system with detectors to enable selecting a particular station, the type of music the user wants to hear, an appliance which needs repair as discussed above, a VCR, a lamp, a thermostat or a burglar alarm system, for example. Thetarget60 could be a refrigerator or a drawer having a laser detection device which, when queried, would provide an audio or digital feedback as to the contents of the refrigerator or drawer. Thetarget60 could be a door lock which would open when a correctly encoded laser signal is beamed to its detector. Of course the predetermined signal could be sent via an RF signal rather than by thelaser20. InFIG. 2 thelaser20 ofFIG. 1A could be modified to detect bar code labels. The reading of bar codes and the connections to the internet could provide information about a product which can not obtained by observing the product alone.

Thetarget60 could have a sensor61 which would receive light or RF signals from theuser14. In this embodiment theuser14 would compose a message and enter the message as an audio signal which would be stored in thePDA24,electronic module50 or a storage device shown aselement38 for this embodiment. When theuser14 approaches thetarget60 and pulls backear12, the stored message is sent as an audio message or a binary message to the sensor61 and thetarget60 will either immediately respond to the message or will store the message for later retrieval.

Thetarget60 could be a luminescent screen which could be written on with thelaser20 when it emits a blue light.

FIG. 3 shows the microphone64 of the combination microphone andspeaker48 ofFIG. 1A placed in one ear and thespeaker66 placed in the other ear. Thespeaker66 is connected to theelectronic module50 by asire68. The use of the microphone64 in one ear and thespeaker68 in the other ear attenuates the feedback from the speaker to the microphone in the combination microphone andspeaker48 ofFIG. 1A.

FIG. 4 shows the biometric devices and system ofFIG. 1A with the addition of ahelmet70 which soldiers or firemen might use. Thehelmet70 has alaser light detector72 on the back of the helmet and awire74 from thehelmet70 to theelectronic module50. Thelaser light detector72 allows another person with essentially the same equipment to communicate with theuser14 by aiming the other person's laser light at thelaser light detector72. The apparatus ofFIG. 4 allows for secure communication from one person to another, and allows communication when there is a high degree of ambient noise since the combination microphone andspeaker48 are in the ear channel which allows the words of the sender to be detected without much ambient noise and the receiver to receive the communication directly into his ear. Theear12 can still receive normal voice communication.

The identity of auser14 can be verified using theRFID chip47. Theelectronic module50 would query theRFID chip47 to verify the identity of the user.

Although the invention has been described in part by making detailed reference to a certain specific embodiment, such detail is intended to be, and will be understood to be, instructional rather than restrictive. It will be appreciated by those skilled in the art that many variations may be made on the structure and mode of operation without departing from the spirit and scope of the invention as disclosed in the teachings contained herein.

Claims (16)

1. A transmitting apparatus comprising:

a) an ear movement sensor disposed in a predetermined position adjacent an ear of a user for detecting an ear movement of said user; and

b) an electronic module coupled to said ear movement sensor for initiating a predetermined procedure for at least one of initiating, stopping and maintaining a predetermined object upon a detection of said ear movement.

2. The transmitting apparatus, as set forth inclaim 1, further including signaling means comprising one of a light source, an ultrasonic generator and a high frequency transmitter wherein said electronic module is coupled to said signaling means and enables said signaling means upon detection of said ear movement.

3. The transmitting apparatus, as set forth inclaim 2, wherein said ear movement is an ear pull.

4. A transmitting apparatus comprising:

a) a sensor for detecting an ear pull of a user;

b) a laser worn by said user;

c) an electronic module coupled to said sensor and said laser for generating an encoded laser beam upon a detection of said ear pull.

5. The transmitting apparatus, as set forth inclaim 4, wherein said laser is mounted on the head of said user.

6. The transmitting apparatus, as set forth inclaim 4, further including a plurality of head position sensors for detecting a head position of said user.

7. The transmitting apparatus, as set forth inclaim 4, further including a laser detector mounted on said user for receiving communication from another laser.

8. A transmitting apparatus comprising:

a) a user;

b) a plurality of sensors for detecting a head position of said user;

c) a RF transmitter; and

c) an electronic module coupled to said plurality of sensors and to said RF transmitter for generating an encoded RF signal containing information about said head position of said user.

9. The transmitting apparatus, as set forth inclaim 8, further including a speaker coupled to said electronic module wherein if said electronic module detects one of a particular head position and a pattern of movement of said head position, a tone is sent to said speaker to alert said user.

10. A transmitting apparatus comprising:

a) a sensor for detecting an ear movement of a user;

b) an electronic module coupled to said ear movement sensor for starting a procedure upon a detection of said ear movement; and

c) signaling means comprising one of a light source, an ultrasonic generator and a high frequency transmitter wherein said electronic module is coupled to said signaling means and enables said signaling means upon detection of said ear movement initiated by pulling on an ear of such user.

11. The transmitting apparatus, as set forth inclaim 10, wherein said signaling means is mounted on the head of said user.

12. The transmitting apparatus, as set forth inclaim 10, further including one or more head position sensors for detecting a head position of said user.

13. The transmitting apparatus, as set forth inclaim 10, wherein said ear pull sensor comprises a strain gauge one of attached to and contained inside a temple piece of a pair of glasses worn by said user.

14. The transmitting apparatus, as set forth inclaim 10, wherein said ear pull sensor comprises two capacitance plates, wherein the capacitance formed between said two capacitance plates changes when said ear is moved.

15. The transmitting apparatus, as set forth inclaim 14, wherein one capacitor plate is the frame of a pair of glasses worn by said user.

16. The transmitting apparatus, as set forth inclaim 11, wherein one capacitor plate is the body of said user.

Images