Description
The present invention relates to methods for data exchange between at least two electronic devices. Furthermore the present invention relates to an electronic device programmed to perform the method of present invention and a computer program product comprising computerexecutable instructions for performing this method. More specifically the invention relates to a computer implemented method that recognizes a gesture such as a handshake, wherein the gesture triggers and initiates a data exchange between the electronic devices, preferably a wearable electronic device.
Wearable technology or wearable devices such as smart watches, glasses, activity trackers, navigation tools, accelerometers, etc. are often also smart electronic devices comprised of microcontrollers that enable the devices to acquire and exchange data, without requiring human intervention. Wearable devices may be worn on the body (on the arms, legs, wrists) of the user or as implants and are rapidly advancing in both personal and business use. In healthcare, wearables have long been used, for example in detecting health disorders and send health vitals readings to physicians. In sports, wearable technology has applications in monitoring and real time feedback during exercise to improve performance or training methods.
Furthermore, smart electronic wearable devices are increasingly being used in business to exchange information with each other. The smart wearable devices may eventually effectively replace business cards or other means of manually exchanging data. These digital wearables are quicker, more sustainable (no paper) and are able to connect and communicate to digital platforms such as the mobile phone, tablet or desktop PC turd integrate with a user's social media platforms such as Facebook, Outlook, Linkedln, etc.
The limitation of present digital wearable electronic devices is that the exchange of data, such as personal information does not occur in an intuitive and natural manner. For example, instead of just shaking hands, users must actively press their wearable against each other or press a button to actively exchange information/data. This detracts from the user experience. Furthermore, smart wearables are generally constrained by limited battery', which makes continuous communication between the devices difficult. In the case of smart wearables that must communicate with many other wearables this problem is even worse. Present wearables cannot be continuously operating their radios due to power drain and limited bandwidth in the radio spectrum and yet must rapidly communicate with each other.
Considering the above, there is a need in the art for digital wearable solutions wherein data is exchanged in an intuitive and natural manner thereby improving the user experience. Furthermore there is a need in the art for wearables solutions having an improved standby and operational time, wherein the solution does not negatively affect the usability., such as weight, size, etc. or the aesthetics of the computer device or smart wearable device.
It is an object of the present invention, amongst other objects, to address the above need in the art. The object of present invention, amongst other objects, is met by the present invention as outlined in the appended claims.
Specifically, the above object, amongst other objects, is met, according to a first aspect, by the present invention by a method for data exchange between at least two electronic devices, wherein the method comprises the steps of
a) detecting movement of an electronic device by at least one sensor on the electronic device, generating movement data;
b) determining the occurrence of a gesture by the electronic device by matching the movement data with data, that predetermines said gesture;
c) transforming the movement data of the gesture determined in step b) to a frequency domain, generating a transformed signal;
d) transmitting the transformed signal by the electronic de vice;
e) receiving the transformed signal from at least one other electronic de vice that also has preformed the steps a) to e);
f) matching the received transformed signal from the at least one other electronic device, with the transformed signal of the electronic device itself;
g) transmitting and receiving user information between the electronic device and the at least one other electronic device that has matching transformed signals.
The present invention relates to the method implemented by an electronic device.
preferably a wearable electronic device that recognizes a gesture such as a handshake, and initiates 25 a data exchange between the wearable electronic devices. A gesture is defined as a. movement of the body or limbs of an individual that expresses or emphasizes an idea, sentiment or attitude; the use of motion s of the l imbs of the body as means of expression .
According to a preferred embodiment, the present invention relates to the method, wherein the electronic device is a wearable electronic device, preferably a smart bracelet. The 30 electronic devices are preferably smart wearable electronic devices, most preferably an electronic smart bracelet. It is a smart wearable device that activates upon a gesture recognized by said device. The method of present invention enables a smart wearable device to exchange and/or broadcast digital information with other smart wearable devices upon recognition of and activation by a gesture. For example, a smart bracelet, that exchanges user information upon a handshake of 35 two users of these smart bracelets. The smart bracelet broadcasts data (personal information, metadata, etc.) upon its user performing gestures such as clapping, raising their hand, or a handshake. The electronic device sends its unique hardware address (like a MAC address in computers) to which the user’s shake-on account (orLinkedln, Facebook, etc.) is linked. Also the device can send its user's name so that the other device can forward that name to the phone of its respective user, who can then see the other person’s name on their phone even without an internet 5 connection.
The compu ter implemented method of present invention makes use of a pattern recognition, matching and decision algorithm that detects its user performing a gesture, e.g. making a handshake, without the additional use of any external de vices. The (wearable) electronic device detects and tracks motion of its user via an accelerometer and/or gyroscope provided in the 10 device, measuring the acceleration and/or rotation rates generated by the specific motion.
According to a preferred embodiment, the present invention relates to the method, wherein step b) is executed by a motion detection algorithm of the electronic device that classifies the movement data into gestures and movements by examining select features in the movement data. The movement data is data generated by the sensor signal and is compared with predetermined movement data of a gesture in order to determine the occurrence of a specific gesture. These features are mathematical characteristics of the data signal whose values and combinations are previously determined to correspond to select gestures. The computer implemented method comprises various algorithms that, enable the computer to exchange information triggered by a gesture. The gesture is preferably a handshake, a handclap or raising 20 one’s hand. A motion detection algorithm detects the occurrence of the specific gesture or motion, based on sensory data, preferably data from an accelerometer and/or gyroscope. The measured acceleration and/or rotation rates are subsequently processed by the decision and matchings criteria specified in the algorithm to match and recognize the specific motion. This enables the computer to save energy by only executing certain functionalities, such as radio communication for data exchange, only when the computer is activated or triggered by the gesture that is determined (e.g. handshake).
According to a preferred embodiment, the present invention relates to the method, wherein in step e) the at least one other electronic device is preferably two or more, more preferably four or more, most preferably five or more other electronic devices. The electronic 30 device performing the method of present invention is capable of receiving signals from many other electronic devices (smart bracelets) that are triggered at the same time, and match with the right one(s).
The motion detection algorithm used in the method of present invention comprises six featu res: the mathematical mean of the signal, the mathemati cal variance of the signal, the 35 energy of the signal, the range of the signal, the zero crossing number in the signal and the power spectrum of the signal. These are all defined with the signal being a discrete-time signal recorded by the sensor(s). Recent signal data is kept in a buffer, which is analysed by the device’s algorithm. The energy of the signal in a given window of time is defined as the square of every value in the signal summed. The range of the signal is defined as the difference between maximum and minimum amplitude of the signal in a given window of time. Zero crossing is defined by how often 5 a signal crosses a reference value (normally a value of zero), or how often the sample signal changes its sign from positive to negative or the other way around. In present invention, the zero reference value is the mean of the signal window. Furthermore, a threshold is used such that the difference between two zero crossing points needs to be large enough to be considered as a zero crossing. The power spectrum is the distribution of power within the frequency components of the 10 signal. The motion detection algorithm follows a decision tree that evaluates the combination of the six features. A certain range and combination of these features determines if the motion can be classified as a gesture, e.g. a handshake, with any other combination being another movement.
Regarding power conservation and quick data exchange, the method of present invention, including its algorithms, plays an important role by only activating the radio upon 15 detection and determination of the gesture. In the present method the data is compressed in the form of a domain transformation. Instead of sending the acceleration and/or rotation rate signals directly from the measurement buffer/window after temporal alignment, a Fast Fourier Transform (FFT) is performed to transform the data into the frequency domain (in the form of a power spectrum). Bv selecting a relatively small frequency range, preferably below 10 Hz, this becomes 20 the ‘gesture signal which is used to match the device with others making the same motion. Each signal is temporally aligned and then converted into the frequency domain, from which the desired frequency range is selected thus creating the “gesture signal”. Sending this signal described in the frequency domain, instead of the temporal domain, takes considerably less data and thus makes the information exchange much quicker and less energy is required as compared to continuous 25 communication and transmission of large datasets between wearables.
According to a preferred embodiment, the present invention relates to the method wherein the frequency domain of step c). is between 0.001 Hz and 100 Hz, preferably between 0.05 Hz and 50 Hz, most preferably between 0.1 Hz and 10 Hz. In step c of present invention, transforming the movement data of the identified gesture into the frequency domain, generates a 30 transformed signal in the form of a. power spectrum.
Furthermore., when triggered, the electronic device collects and processes the gesture signals of any other gesture-triggered electronic device. For instance in case of a handshake, the device receives any other device’s broadcasts (gesture signals) and the matching algorithm determines which of the gestu res is the correct corresponding one, reducing the number 35 of communication steps between electronic devices. This reduction in communication time occurs due to the algorithm deciding locally, on the device, with which other device it is intended to match - no response, confirmation or establ ished communication link is necessary' between devices. In the method of present invention each gesture signal is temporally aligned by the temporal alignment algorithm and then converted into the frequency domain (FFT transform). In more detail· the gesture signals are separated into segments and for each of the segments, FFT is 5 performed generating a power spectrum of each segment. The matching algorithm uses the segmented power spectrums to determine if a handshake is a match. This is done by calculating the coherence of the devices' signals. The coherence, a measure of similarity between two signals, relies on two measurements: auto-spectral power density and cross-spectral power density. Autospectral power density is calculated from a device's own segmented power spectrums. Auto10 spectral power density is calculated by using the received power spectrums from another device in combination with the device's own power spectrums. That is, upon triggering by specific gestures determined and recognized, the electronic device communicates via its radio its own movement's power spectrums to all the other electronic devices. Then each device compares and calculates the coherence between its own power spectrum to that of the all the other spectrums it has received.
The device which sent the power spectrums that led to the highest coherence is determined to be the matching electronic device. Between these matching electronic devices user specific information is already transmitted and received during the sending of the power spectrum (FFT), so no confi rmation or secondary communication is necessary after sending and receiving the first data. However, if a larger amount of information is desired to be transferred, then the bracelets 20 may establish a connection to send any remaining data.
According to a preferred embodiment, the present invention relates to the method wherein the matching in step f) is executed by a matching algorithm that compares the transformed signal of the electronic device itself with the received transformed signals of the at least one other electronic device. The method of present invention compares all simultaneous gestures occurring 25 within a dose proximity of the electronic device (e.g. upon the user of the electronic device shaking hands with another user of another electronic device), and identifies the one correct, corresponding movement (matching handshake). Of course, as it is also possible for multiple handshakes to occur near each other almost simultaneously, the smart electronic device needs to be able to determine which one corresponds to itself. The matching algorithm feature determines the 30 correct person(s) if multiple wearables are in close vicinity and able to communicate with ea ch, by comparing the specific gesture profile (e.g. handshake), as well as the received signal strength indication (RSSI), in order to pair and communicate with the correct user(s). The RSSI value is taken into account when evaluating a matching handshake: a strong signal implies that the electronic device was in close proximity to the other electronic device and thus more likely to be a 35 handshake. Every time a motion occurs (e.g. a handshake) that is registered as such by the wearable, the wearable must rapidly communicate with any other wearable that has also detected this specific motion. A short communication time is mandatory because a user can make quick successive handshakes with different individuals. Also short communications result in a reduced constrains on the power supply.
According to another preferred embodiment, the present invention relates to the method wherein the at least two electronic devices are in close proximity to each other. By measuring the received signal strength of other devices during radio communication, the device can approximate the relative distance between itself and other communicating devices. This received signal strength and approximation of the relative distance is then used as a factor during the matching algorithm, with distant devices being less likely to be accepted as a match. Without 10 taking distance into account, two movements separated by a long distance could still be registered as matches. This is undesirable as the movements should not only be synchronized but also close in order to successfully exchange data.
According to yet another preferred embodiment, the present invention relates to the method wherein after step b, an add itional step is provided, determining a start point in time of the 15 determined gesture within the movement data on the electronic device. Even though two people perform a gesture, such as a handshake, with each other, each wearable device can detect a handshake at a slightly different time. It is mainly caused by the slight difference of electronic device placement, orientation and biomechanics, which make the signals at each electronic device slightly different. This difference results in the potential for two electronic devices that are 20 activated be a specific gesture to trigger (i.e. generate an event signal) via the motion detection algorithm at different times. This difference may result in the potential for incorrect or absence of matching between the two devices. For gesture matching, the signals are aligned in time. Determining the start point in time of the determined gesture within the movement data on each electronic device is executed by a temporal alignment algorithm. Via the algorithm the wearable 25 device, “scans” though its sensor signals and determines the point at. which the specific motion (handshake) starts. Upon determining this point, the device then identifies the part of the signal corresponding to a potential handshake. This part of the signal is then passed on to the matching algorithm for gesture matching; the temporal alignment algorithm selects the part of the signal for the matching algorithm to FFT, broadcast and match. The matching algorithm of the electronic 30 device compares the specific motion information received from any other electronic devices (e.g.
when multiple, simultaneous handshakes occur in the vicinity and all the devices will be broadcasting their “gesture information”) and determines the electronic device(s) that corresponds to the gesture (e.g. handshake) of the other user.
According to another preferred embodiment, the present invention relates to the 35 method wherein the at least one sensor detects changes in acceleration and/or rotation rates induced by the movement of the electronic device. Acceleration and rotation rates are provided as data signals by the sensors in the electronic device. By analysing the acceleration and rotation rate datasets, the motion detection algorithm determines if a given motion is for example a handshake or not. Motions that generate these datasets (signals) are continuously measured during operation and a set of the latest values, the measurement window / buffer, is used for calculation, after applying a low-pass filter. The motion detection algorithm determines if a given motion is a specific gesture or not, for example a handshake, by analysing the accelerations and rotations of the bracelet and matching the movement data with data (features) that predetermines said gesture.
According to yet another preferred embodiment, the present invention relates to the method wherein the at least one sensor is an accelerometer and/or a gyroscope.
According to a preferred embodiment, the present invention relates to the method wherein the gesture is preferably a handshake, a handclap or a hand raising movement.
The present invention, according to a second aspect, relates to an electronic device programmed to perform the method of present invention, wherein the electronic device is a wearable electronic device, preferably a smart bracelet.
According to a preferred embodiment, the present invention relates to the electronic device comprised of a bracelet band that houses the electronics, wherein the electronics are comprised of a printed circuit board (PCB), a microcontroller, at least one sensor, a batten and a radio. The radio of the electronic device of present invention is preferably a 2.4 GHz radio.
The present invention, according to a third aspect, relates to a computer program product comprising computer-executable instructions for performing the method of present invention, when the program is nm on an electronic device.
The present invention will be further detailed in the following example and figures wherein:
Figure 1:. Shows flowchart on the method of present invention. A wearable electronic device, such as a smart bracelet is worn by a user. Personal information has been sent and stored on the device in respect to the user that is wearing that device (10). When the user is performing a gesture, for example performing a handshake, the sensors present in the device are activated (11) and detect movement (sensor) data. This data is analysed by the motion detection algorithm (12) that classifies the movement data into gestures by examining select features in the movement data signal. These features are mathematical characteristics of the data signal whose values and combinations are previously determined to correspond to select gestures. If a. gesture is detected, by the motion detection algorithm, the temporal alignment algorithm (13) will align the gesture signal in time and select the part of the signal corresponding to the gesture. The gesture signal is then separated into segments (14). Subsequently each of the segments is converted into the frequency domain (FFT transform), generating a power spectrum of each segment. The generated power spectrum is broadcasted by the radio (15) in the device to other devices (16). These other devices, that also have performed the previous steps, also 5 broadcast their own power spectrums. The matching algorithm (18) uses the segmented power spectrums received (17) of other devices to determine if a handshake is a match by calculating the coherence of the devices’ signals. Each device compares and calculates the coherence between its own power spectrum to that of the all the other spectrums it has received . The device which sent the power 10 spectrums that led to the highest coherence is determined to be the matching electronic device. Between these matching electronic devices (19) user specific information is transmitted and received.
Matching of a handshake by a wearable device, a smart bracelet
Upon a handshake trigger (when an event signal is generated by the bracelet) the following steps occur;
1. A bracelet determines, via temporal alignment, which part of the recent signal corresponds to the handshake movement. The temporally aligned part of the movement signal is segmented and for each segment an FFT is performed. The bracelet broadcasts these FFTs to any other triggered bracelets and receives theirs.
2. After communicating with other bracelets, the bracelet calculates, per segment, according to equation (1) the auto-spectral power density (Pyx) of its own FFTs and according to equation (2) the cross spectral-power densities (Pn) between its own FFTs and all received FFTs. ..¥(ƒ) is the FFT of the bracelet’s own signal, is the FFT of another bracelet' s signal, ƒ is the sample frequency.
| .... Wl.rcz)*XX f'jS | (1) |
| r) xwtnixy -fs | (2) |
3. Using the auto-power spectral density and cross-power spectral density the coherence. Cyv(f), between the bracelet signals and any received bracelet signals can be computed 3 5 according to equation (3).
r '-'xy
)ï (3)
4. The bracelet with the highest coherence score, Ctolai, according to equation (4) is selected as the matching bracelet, if it exceeds a certain threshold score. The coherence score is the sum of all coherence values squared divided by the sample size (Λ7), where> is the coherence for a given frequency.f„.