BACKGROUND OF THEINVENTION1. Field of the InventionThe present invention relates to the technology field of physiological measurement devices, and more particularly to a health management system using contactless physiological measurement technology.
2. Description of the Prior ArtBecause blood oxygen saturation (SpO2) and heart rate (HR) are two of multiple physiological parameters they are regarded as two important health indices, commercial wearable electronic devices like smart bracelet and smart watch are all designed to have function of measuring these two physiological parameters. In addition, by installing a specially-developed application program in a wearable electronic device, the wearable electronic device therefore becomes a mobile health management system for its wearer (i.e., user). However, it is a pity that user's feedback has demonstrated the fact that the forgoing wearable electronic devices shows some drawbacks in practical use. For example, the wearable electronic devices cause some users who have sensitive skin to be subject to skin allergy.
On the other hand, Taiwan Patent No. 1653601 has disclosed a personalized health management system, which comprises a host device and one or more physiological measurement devices, and is placed in a public region like hospital or convenience store. When using the personalized health management system, a user is firstly required to establish his personal information in the host device, including body height, body weight and age. After that, the user is allowed to start the measurement of personal physiological parameters by using the physiological measurement device, such that a personal health evaluation table is subsequently generated by the host device. However, due to the fact that the personalized health management system is a costly apparatus because of including a huge host device and multiple physiological measurement devices, the personalized health management system is commonly purchased by a government agency, thereby being is placed in a public region for serving the public.
In addition, Taiwan Patent Public No. TW201143712A has disclosed a handheld health management device, which includes at least one contact type measurement unit for use in the measurement of a user's physiological parameters like blood pressure, blood sugar, body temperature, body weight, body fat and so on. Moreover, the handheld health management device further includes a warning unit, which is configured to produce a warning signal in case of at least one of the user's multi physiological parameters is over corresponding reference value. In spite of that, the user is required to touch and/or contact the measurement unit during operating the handheld health management device to accomplish the measurement of his physiological parameters, and that causes a significant inconvenience for the user.
From above descriptions, it is understood that there is still room for improvement in the conventional health management devices. In view of that, inventors of the present invention have made great efforts to make inventive research and eventually provided a health management system using contactless physiological measurement technology.
SUMMARY OF THE INVENTIONThe primary objective of the present invention is to disclose a health management system using contactless physiological measurement technology. The health management system principally comprises a camera and a first processor, of which the camera is faced to a user for capturing a user image. The first processor is particularly configured to have a face detection unit and an activity index calculating unit therein. By such arrangement, after receiving the user image from the camera, the first processor detects a face portion from the user image, thereby subsequently extracting a photoplethysmography (PPG) signal from the face portion.
Consequently, after completing at least one process of the PPG signal, multiple indexes for describing a user's health activity are generated. The health activity indexes include health index, activity index, stability index, relaxation index, metabolism index, and balance index. As a result, the first processor achieves an evaluation of the user's health activity state according to the forgoing health activity indexes.
According to the plurality of health activity indexes, the health management system of the present invention is able to show an infographic for describing variations of the user's health activity state in a period of time.
For achieving the primary objective, the present invention discloses an embodiment for the health management system using contactless physiological measurement technology, comprising:
- a camera, being faced to a user for capturing a user image;
- a first processor, being coupled to the camera, and comprising one or more embedded programs including instructions for:
- detecting a face portion from the user image;
- extracting a photoplethysmography (PPG) signal from the face portion; and
- applying at least one signal process to the PPG signal, thereby generating a plurality of health activity indexes with respect to the user;
- wherein the plurality of health activity indexes comprises health index, activity index, stability index, relaxation index, metabolism index, and balance index.
In one embodiment, the first processor is integrated in an electronic device that is coupled to the camera, such that the first processor is therefore coupled to the camera. The electronic device is selected from a group consisting of cloud server, desktop computer, all-in-one computer, embedded computer, laptop computer, tablet computer, smart phone, smart watch, smart glasses, smart television, video door phone system, and home healthcare computer device, and comprises a second processor, a display, a memory, and a communication interface, such that the second processor receiving the plurality of health activity indexes from the first processor, thereby controlling the display to show the plurality of health activity indexes by a form of numeric values and/or infographics.
In a practicable embodiment, the camera and the first processor are both integrated in an electronic device, and the electronic device being selected from a group consisting of all-in-one computer, embedded computer, laptop computer, tablet computer, smart phone, smart watch, smart glasses, smart television, video door phone system, and home healthcare computer device.
In one embodiment, by applying the at least one signal process to the PPG signal, the PPG signal is firstly converted to a time-domain signal, and at least one time-domain parameter is subsequently extracted from the time-domain signal; the time-domain parameter being selected from a group consisting of standard deviation of all normal to normal intervals (SDNN), root mean square successive differences (RMSSD), number of pairs of adjacent normal to normal intervals differing by more than 50 ms (NN50), proportion of NN50 divided by a total number of all normal to normal intervals (pNN50).
In one embodiment, by applying the at least one signal process to the PPG signal, the PPG signal is firstly converted to a frequency-domain signal, and at least one frequency-domain parameter is subsequently extracted from the frequency-domain signal; the frequency-domain parameter being selected from a group consisting of total power (TP), high frequency power (HF), low frequency power (LF), very low frequency power (VLF), ultra low frequency power (ULF), low frequency proportion (LF %), and LF/HF ratio.
In a practicable embodiment, the first processor further comprises one or more embedded programs including instructions for:
applying at least one signal process to the PPG signal, thereby generating a plurality of physiological parameters with respect to the user;
extracting facial features from the face portion in the user image, and then applying a feature matching between the facial features and a facial feature template that is stored in the memory, thereby accomplishing a face recognition of the user; and
evaluating a health activity state of the user according to the plurality of health activity indexes.
BRIEF DESCRIPTION OF THE DRAWINGSThe invention as well as a preferred mode of use and advantages thereof will be best understood by referring to the following detailed descriptions of an illustrative embodiment in conjunction with the accompanying drawings, wherein:
FIG. 1 shows a first schematic diagram for describing a health management system using contactless physiological measurement technology according to the present invention;
FIG. 2 shows a second schematic diagram for describing the health management system using contactless physiological measurement technology according to the present invention;
FIG. 3 shows a first block diagram of the health management system;
FIG. 4 shows a waveform diagram of a PPG signal, two waveform diagrams of the PPG signal that has been treated with signal process;
FIG. 5A shows a hexagonal radar chart for shows six kinds of health activity indexes;
FIG. 5B shows a hexagonal radar chart for shows six kinds of health activity indexes;
FIG. 6 shows a curve chart for shows six kinds of health activity indexes in a period of time;
FIG. 7 shows a third schematic diagram for describing the health management system using contactless physiological measurement technology according to the present invention; and
FIG. 8 shows a second block diagram of the health management system.
DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTSTo more clearly describe a health management system using contactless physiological measurement technology, embodiments of the health management system using contactless physiological measurement technology according to the present invention will be described in detail with reference to the attached drawings hereinafter.
With reference toFIG. 1, there is shown a first schematic diagram for describing a health management system using contactless physiological measurement technology according to the present invention. Moreover,FIG. 2 shows a second schematic diagram of the health management system. AsFIG. 0.1 andFIG. 2 shows, thehealth management system1 principally consists of a camera and afirst processor12. It is worth noting that,FIG. 1 depicts that thefirst processor12 is integrated in anelectronic device3 by a form of electronic chip. Theelectronic device3 includes asecond processor30, adisplay31, and amemory32, and is coupled to thecamera11, such that thefirst processor12 is able to control thecamera11. In this embodiment, theelectronic device3 can be a cloud server, a desktop computer, an all-in-one computer, an embedded computer, a laptop computer, a tablet computer, a smart phone, a smart watch, a smart glasses, a smart television, a video door phone system, or a home healthcare computer device.
On the other hand,FIG. 2 depicts that thefirst processor12 is integrated in anelectronic device3 that is simultaneously coupled to thecamera11 and adisplay31 and, such that thefirst processor12 is able to control thecamera11. In this embodiment, thedisplay31 can be a television, a computer monitor, a portable monitor, a projector screen, or a touch screen monitor. According to the foregoing descriptions, it is known that thefirst processor12 can be integrated in any one kind of electronic device that includes or is coupled to a display. Similarly, according to the illustrations ofFIG. 1 andFIG. 2, it is also known that thecamera11 can be integrated in or be coupled to the electronic device, thereby being controlled by thefirst processor12.
Please refer toFIG. 1 again, and please simultaneously refer toFIG. 3 that illustrates a first block diagram of thehealth management system1. AsFIG. 1 andFIG. 3 show, thecamera11 is faced to auser2 for capturing a user image, and thefirst processor12 is coupled to thecamera11 so as to receive the user image. Particularly, there are one or more programs embedded in thefirst processor12, such that thefirst processor12 is able to perform multiple functionalities by executing the instructions included in the programs. The programs comprises: facedetection program121, physiologicalparameter calculating program122, health activityindex calculating program123, facerecognition program124, and health activitystate evaluating program125. By such arrangement, after receiving the user image, thefirst processor12 detects a face portion from the user image by executing theface detection program121, and then extracting a photoplethysmography (PPG) signal from the face portion by executing the health activityindex calculating program123. Consequently, after applying at least one signal process to the PPG signal, a plurality of health activity indexes with respect to theuser2 are therefore generated under executing the health activityindex calculating program123. The plurality of health activity indexes comprises health index, activity index, stability index, relaxation index, metabolism index, and balance index. On the other hand, after the PPG signal is extracted from the face portion in the user image, the first processor can also execute the physiologicalparameter calculating program122 to apply at least one signal process to the PPG signal, thereby generating a plurality of physiological parameters with respect to theuser2.
Engineers skilled in development of image processing algorithms certainly know that, facedetection program121 use algorithm to find the user's face within the user image. The algorithm typically starts by searching for user eyes, i.e., one of the easiest features to detect. The algorithm might then attempt to detect eyebrows, mouth, nose, nostrils and iris, so as to determine a facial region (i.e., the foregoing face portion) on the user image. Literature I has relatively complete instructions for the face detection algorithm(s). Herein, literature I is written by Wong et.al, and is entitled with “An efficient algorithm for human face detection and facial feature extraction under different conditions” so as to be published on Pattern Recognition,Volume 34,Issue 10, 2001, Pages 1993-2004.
According to the present invention, the plurality of health activity indexes comprises health index, activity index, stability index, relaxation index, metabolism index, and balance index. On the other hand, engineers skilled in the technology field of non-contact type physiological measurement should know that, thecamera11 and theelectronic device3 having thefirst processor12 therein constitute a contactless physiological measurement system, and this contactless physiological measurement system can be operated for measuring an imaging photoplethysmography (iPPG) signal or a remote photoplethysmography (rPPG) signal from theuser2. Of course, engineers skilled in the technology field of non-contact type physiological measurement certainly knows how to complete the measurement of multiple physiological parameters with respect to the user by collecting the iPPG signal or the rPPG signal. For example, China Patent Publication No. CN106343986A has disclosed the way to measure blood pressure by collecting PPG signal from a man. Moreover, literatures II, III, and IV have disclosed the way to measure SBP, DBP, heart rate (HR), respiratory rate, and blood oxygen saturation (SpO2) by collecting PPG signal from a man.
Herein, literature II is written by Goudarzi et.al, and is entitled with “Using imaging Photoplethysmography (iPPG) Signal for Blood Pressure Estimation” so as to be published on 2020 International Conference on Machine Vision and Image Processing (MVIP), Iran, 2020, pp. 1-6. On the other hand, literature III is written by Kong et.al, and is entitled with “Non-contact detection of oxygen saturation based on visible light imaging device using ambient light” so as to be published on Optics Express Vol. 21, Issue 15, pp. 17464-17471 (2013). Moreover, literature III is written by Sanyal et.al, and is entitled with “Algorithms for Monitoring Heart Rate and Respiratory Rate From the Video of a User's Face” so as to be published on IEEE Journal of Translational Engineering in Health and Medicine, Vol. 6, pp. 1-11, 2018.
According to the present invention, after the PPG signal is extracted from the face portion in the user image, thefirst processor12 executes the health activityindex calculating program123 to applying at least one signal process to the PPG signal, thereby generating a plurality of health activity indexes with respect to theuser2. The plurality of health activity indexes comprises health index, activity index, stability index, relaxation index, metabolism index, and balance index. Doctors and nursing personnel certainly know that, the autonomic nervous system has two principal branches: the sympathetic nervous system and the parasympathetic nervous system. The sympathetic nervous system functions in energy generation and arousal, helping to mobilize the body during times of excitement, stress, and when activity and a quick response might be needed. For example, during threatening situations, the SNS can accelerate the heart, dilate the eyes' pupils, constrict visceral blood vessels, shunt blood to active skeletal muscles, inhibit activity of the stomach and intestine, dilate the bronchioles in the lung, inhibit the emptying of the bladder, and release glucose from the liver. On the contrary, in case of human is in a state of relaxation, sleep, or rest, the parasympathetic nervous system (PSNS) is activated to slow heart and breathing rates, and to lower blood pressure and promotes digestion. Therefore, clinic data have indicated that, human may be subject to anxiety, palpitations, and/or high blood pressure in case of the SNS is over activated. However, when the PSNS is over activated, human may be in a state of poor mental health, inactive, and/or poor physical strength.
Moreover, doctors and nursing personnel also know that, heart rate (HR) and/or heart rate variability (HRV) can be adopted for being indexes to evaluate the state of the autonomic nervous system. For example, when the SNS is activated more than the PSNS, the heart rate has an increase but the heart rate variability is subject to a decrease. On the contrary, the heart rate has a decrease when the PSNS is activated more than the SNS.
Therefore, after detecting a face portion from the user image, thefirst processor12 executes the health activityindex calculating program123 to extract a photoplethysmography (PPG) signal from the face portion, and then to apply at least one signal process to the PPG signal so as to obtain a plurality of health activity indexes with respect to theuser2. As described in more detail below, by applying the at least one signal process to the PPG signal, the PPG signal is firstly converted to a frequency-domain signal. Subsequently, the at least one frequency-domain parameter is subsequently extracted from the frequency-domain signal. The frequency-domain parameters are as follows: total power (TP), high frequency power (HF), low frequency power (LF), very low frequency power (VLF), ultra low frequency power (ULF), low frequency proportion (LF %), and LF/HF ratio.
Moreover, in a practicable embodiment, the PPG signal can be firstly converted to a time-domain signal. Subsequently, at least one time-domain parameter is extracted from the time-domain signal. The time-domain parameters are as follows: standard deviation of all normal to normal intervals (SDNN), root mean square successive differences (RMSSD), number of pairs of adjacent normal to normal intervals differing by more than 50 ms (NN50), proportion of NN50 divided by a total number of all normal to normal intervals (pNN50).
For example,FIG. 4 shows a waveform diagram of a rPPG signal, two waveform diagrams of the PPG signal that has been treated with signal process. As explained in detail below, waveform diagram (a) is a PPG signal extracted from the face portion in the user image, and waveform diagrams (b) and (c) are all converted from the PPG signal. Therefore, thefirst processor12 can obtain a data of SDNN from the waveform diagram (b), and obtain a data of RMSSD from the waveform diagram (c). Engineers skilled in the technology field of physiological measurement certainly know that, SDNN, RMSSD, NN50, and PNN50 are commonly adopted for being indexes to calculate the HRV.
Briefly speaking, as following table (1) shows, after converting the PPG signal to a time-domain signal and/or a frequency-domain signal, thefirst processor12 is able to subsequently obtain the health activity indexes from the time-domain signal and/or the frequency-domain signal.
| TABLE 1 |
|
| health activity | |
| indexes | Signal process |
|
| Health | Applying a signal process of SDNN to the PPG |
| signal, so as to obtain SDNN data for calculating |
| the health index. |
| Activity | Converting the PPG signal to a frequency- |
| domain signal, and then acquiring low frequency |
| (0.04-0.15 Hz) power (LF) data from the |
| frequency-domain signal, thereby calculating the |
| activity index. |
| Stability | Converting the PPG signal to a frequency- |
| domain signal, and then acquiring high frequency |
| (0.15-0.5 Hz) power (HF) data from the frequency- |
| domain signal, thereby calculating the activity |
| index. |
| Relaxation | Applying a signal process of SDNN to the PPG |
| signal, so as to obtain SDNN data for calculating |
| the health index. |
| Metabolism | Converting the PPG signal to a frequency- |
| domain signal, and then calculating low frequency |
| proportion (LF %) after acquiring LF data and HF |
| data from the frequency-domain signal, thereby |
| calculating the metabolism index. |
| Balance | Converting the PPG signal to a frequency- |
| domain signal, and then calculating a ratio of LF to |
| HF (i.e., LF/HF) after acquiring LF data and HF |
| data from the frequency-domain signal, thereby |
| calculating the balance index. |
|
FIG. 5A shows one hexagonal radar chart for shows six kinds of health activity indexes, andFIG. 5B shows another one hexagonal radar chart for shows six kinds of health activity indexes. According toFIG. 5A, it is understood that, the six health activity indexes indicate that theuser2 has a good state in spirit, mental, and physiological. However, according toFIG. 5B, it is known that, the six health activity indexes indicate that theuser2 has a relatively depression state in spirit, mental, and physiological.
In addition,FIG. 3 depicts that thefirst processor12 also has a health activitystate evaluating program125, which is executed by thefirst processor12 to evaluate a health activity state of the user according to the plurality of health activity indexes. For example, in case of the hexagonal radar chart showing that the health index is high but the balance index is relatively low, theuser2 may stay in a situation of intensive concentration, nervous and stress. On the other hand, if the hexagonal radar chart shows that both the metabolism index and the balance index are high but both the activity index and the health index are relatively low, theuser2 may stay in a relax or rest state.
In addition,FIG. 3 also depicts that there is aface recognition program124 installed in thefirst processor12. By executing theface recognition program124, thefirst processor12 is able to extract facial features from the face portion in the user image, and then applying a feature matching between the facial features and a facial feature template that is stored in thememory32, thereby accomplishing a face recognition of the user. Moreover, there are multiple databases established in thememory32 of theelectronic device3, and the multiple databases comprises afirst database321 and asecond database322. According to the present invention, the first databases stores N number of personal information sets, and also stores N number of the facial feature templates that are respectively corresponding to the N number of personal information sets. On the other hand, thesecond database322 stores a plurality of first data set and a plurality of second data set, wherein each the first data set consists of the plurality of physiological parameters, each the second data set consists of the plurality of health activity indexes, and the plurality of first data set and the plurality of second data set are in accordance with the N number of personal information sets.
Because theuser2 may operate thishealth management system1 to measure his physiological parameters and evaluate his health activity state every day, the plurality of physiological parameters enclosed in the first data set are further classified into to a history data and an immediate data, and the plurality of health activity indexes enclosed in the second data set being further classified into a history data and an immediate data. As such, after thesecond processor30 receives the plurality of health activity indexes and the plurality of physiological parameters from thefirst processor12, thesecond processor30 is able to control thedisplay31 to show the plurality of health activity indexes by a form of numeric values and/or infographics, and/or to show the plurality of physiological parameters by a form of numeric values and/or infographics.
FIG. 6 shows a curve chart for shows six kinds of health activity indexes in a period of time. After the user using thishealth management system1 to record his physiological parameters and health activity indexes for a period of time, thesecond processor30 of theelectronic device3 is able to control thedisplay31 to show a curve chart for describing variations of the user's health activity state in this period of time.
Furthermore,FIG. 7 shows a third schematic diagram for describing the health management system using contactless physiological measurement technology according to the present invention. Moreover,FIG. 8 shows a second block diagram of the health management system. AsFIG. 7 andFIG. 8 show, thehealth management system1 of the present invention comprises acamera11, afirst processor12 and anelectronic device3, of which theelectronic device3 comprises asecond processor30, adisplay31, amemory32, and acommunication interface34. By such arrangement, thesecond processor30 is able to communicate with an external electronic device4 (e.g., user's smartphone) through thecommunication interface34, so as to transmit the data comprising the plurality of health activity indexes and the plurality of physiological parameters to the externalelectronic device4. In a practicable embodiment, the externalelectronic device4 can be a smartphone, a smart watch, a tablet computer, or a laptop computer.
Therefore, through above descriptions, the health management system using contactless physiological measurement technology according to the present invention has been introduced completely and clearly. However, the embodiments are not intended to limit the scope of the present invention, and all equivalent implementations or alterations within the spirit of the present invention still fall within the scope of the present invention.