CN101057736A - Cradle device and method for controlling cradle swing state by using image recognition technology - Google Patents

Abstract

A cradle device comprises a cradle, an image capturing module used for capturing image data, an image recognition module used for recognizing the image data captured by the image capturing module, a cradle driving module used for driving the cradle to swing, and a control module coupled with the image recognition module and the cradle driving module and used for controlling the cradle driving module according to the result of the image data recognized by the image recognition module.

Description

Translated fromChinese

利用图像识别技术控制摇篮摇摆状态的摇篮装置及其方法Cradle device and method for controlling cradle swing state by using image recognition technology

技术领域technical field

本发明涉及一种利用图像识别技术来控制摇篮摇摆状态的摇篮装置及其方法，特别涉及一种利用图像识别技术来判断婴儿是否进入睡眠状态以控制摇篮摇摆状态的摇篮装置及其方法。The present invention relates to a cradle device and its method for controlling the swaying state of the cradle by using image recognition technology, in particular to a cradle device and its method for controlling the swaying state of the cradle by using the image recognition technology to determine whether a baby is in a sleeping state.

背景技术Background technique

一般来说，带过婴儿的人都有这样的经验，就是当婴儿要睡觉的时候会将婴儿放入摇篮内，而以手动的方式来摇摇篮，以让婴儿比较容易入睡，而且在摇摇篮的过程中，必须随时注意婴儿的入睡状况，例如当婴儿进入熟睡状态时，便可停止摇动摇篮，以避免婴儿在沉睡当中因为摇篮晃动而受到惊吓，或是婴儿会习惯于熟睡时还需要大人继续摇动摇篮，而造成大人们沉重的负担，尤其是在三更半夜时需亲自在摇篮旁摇动摇篮，更是相当地不便。Generally speaking, people who have taken babies have the experience that when the baby is going to sleep, they will put the baby in the cradle, and shake the cradle manually to make it easier for the baby to fall asleep, and when the baby is shaking the cradle During the process of sleeping, you must always pay attention to the baby's falling asleep status. For example, when the baby falls asleep, you can stop shaking the cradle to avoid the baby being frightened by the shaking of the cradle while sleeping, or when the baby gets used to sleeping and needs an adult to Continue to rock the cradle, and cause the heavy burden of adults, especially in the middle of the night, it is quite inconvenient to shake the cradle by the cradle in person.

发明内容Contents of the invention

本发明提供一种利用图像识别技术来判断婴儿是否进入睡眠状态以控制摇篮摇摆状态的摇篮装置及其方法，以解决上述的问题。The present invention provides a cradle device and method for controlling the swaying state of the cradle by using image recognition technology to determine whether the baby is in a sleep state, so as to solve the above problems.

本发明揭露一种利用图像识别技术来控制摇篮摇摆状态的摇篮装置，其包含有一摇篮，一图像撷取模块，用来撷取一图像数据，一图像识别模块，用来识别该图像撷取模块所撷取的该图像数据，一摇篮驱动模块，用来驱动该摇篮的摇摆运动，以及一控制模块，耦合于该图像识别模块及该摇篮驱动模块，用来依据该图像识别模块识别该图像数据的结果控制该摇篮驱动模块。The present invention discloses a cradle device using image recognition technology to control the swaying state of the cradle, which includes a cradle, an image capture module for capturing image data, and an image recognition module for identifying the image capture module The captured image data, a cradle drive module, used to drive the swing motion of the cradle, and a control module, coupled to the image recognition module and the cradle drive module, used to identify the image data according to the image recognition module The result controls the cradle drive module.

本发明黑揭露一种利用图像识别技术控制一摇篮摇摆状态的方法，其包含下列步骤：(a)撷取一图像数据；(b)识别在步骤(a)所撷取的该图像数据；以及(c)依据步骤(b)的识别结果控制该摇篮的摇摆状态。The present invention discloses a method for controlling the rocking state of a cradle using image recognition technology, which includes the following steps: (a) capturing an image data; (b) identifying the image data captured in step (a); and (c) Controlling the rocking state of the cradle according to the identification result of step (b).

附图说明Description of drawings

图1为本发明摇篮装置的功能方块示意图。FIG. 1 is a functional block diagram of the cradle device of the present invention.

图2为本发明摇篮装置的外观示意图。Fig. 2 is a schematic diagram of the appearance of the cradle device of the present invention.

图3为本发明利用图像识别技术控制摇篮的摇摆状态的流程图。Fig. 3 is a flow chart of the present invention using image recognition technology to control the rocking state of the cradle.

图4为步骤114所得出的眼睛图像轮廓的示意图。FIG. 4 is a schematic diagram of the contour of the eye image obtained instep 114 .

附图符号说明：Explanation of reference symbols:

10  摇篮装置                 12  摇篮10Cradle device 12 Cradle

14  图像撷取模块             16  镜头14Image capture module 16 Lens

18  图像传感器               20  图像识别模块18Image sensor 20 Image recognition module

22  摇篮驱动模块             24  马达22 Cradle DriveModule 24 Motor

26  马达控制单元             28  存储模块26Motor Control Unit 28 Storage Module

30  控制模块                 32  支架30Control Module 32 Bracket

34  杆件                     36  固定杆34Rod 36 Fixed Rod

38  脸部区域检测单元         40  眼睛区域检测单元38 Facearea detection unit 40 Eye area detection unit

具体实施方式Detailed ways

请参阅图1与图2，图1为本发明一种利用图像识别技术来控制摇篮摇摆状态的摇篮装置10的功能方块示意图，图2为本发明摇篮装置10的外观示意图。摇篮装置10包含一摇篮12，用来承载婴儿，一图像撷取模块14，用来撷取图像数据，其可为一摄像机或一照相机，图像撷取模块14包含一镜头16，用来接收欲撷取图像所发射或反射的光线，以及一图像传感器18，用来将镜头16所接收的光线转换成电气信号，图像传感器18可包含至少一感光耦合组件(charge coupled device，CCD)，或至少一互补金属氧化物半导体(complementary metal-oxide semiconductor，CMOS)等。摇篮装置10另包含一图像识别模块20，用来识别图像撷取模块14所撷取的图像数据，其可以是一程序代码或逻辑电路等，以及一摇篮驱动模块22，用来驱动摇篮12的摇摆运动，摇篮驱动模块22是包含一马达24，其可以是一环状转圜的直流马达(DC motor)，藉以摇动摇篮12，以及一马达控制单元26，用来接收控制信号以控制马达24的动作。摇篮装置10另包含一存储模块28，用来储存图像撷取模块14所撷取的图像数据以及做为图像识别模块20处理数据时的数据暂存空间，以及一控制模块30，耦合于图像识别模块20及摇篮驱动模块22，用来依据图像识别模块20识别图像撷取模块14所撷取的图像数据的结果控制摇篮驱动模块22，藉以控制摇篮12的摆动。此外，摇篮装置10另包含一支架32，一杆件34，以可转动的方式连接于支架32，以及一固定杆36，连接于杆件34与摇篮12，用来承载摇篮12，其中图像撷取模块14是安装在杆件34上，且摇篮驱动模块22是连接于杆件34与固定杆36，而摇篮驱动模块22可以X轴作为旋转轴且如图2箭头方向所示进行转动(rolling)，藉以带动固定杆36进行来回摆动而摇摆摇篮12，且可带动杆件34进行转动，以使杆件34上的图像撷取模块14可一直同步对准摇篮12中的婴儿，而连续撷取婴儿的图像特征。此外，图像撷取模块14所撷取的图像数据可以是婴儿脸部的图像数据，而图像识别模块20另包含一脸部区域检测单元38，用来撷取脸部的图像特征信息，以及一眼睛区域检测单元40，用来由脸部区域检测单元38所撷取出的脸部的图像特征信息中撷取出眼睛的图像特征信息，其中控制模块30是用来依据该眼睛的图像特征信息控制摇篮驱动模块22。Please refer to FIG. 1 and FIG. 2 . FIG. 1 is a functional block diagram of acradle device 10 using image recognition technology to control the swing state of the cradle according to the present invention. FIG. 2 is a schematic diagram of the appearance of thecradle device 10 of the present invention.Cradle device 10 comprises acradle 12, is used for carrying baby, and an image capturingmodule 14, is used for capturing image data, and it can be a video camera or a camera, andimage capturing module 14 comprises alens 16, is used for receiving desired Capture the light emitted or reflected by the image, and animage sensor 18, which is used to convert the light received by thelens 16 into an electrical signal. Theimage sensor 18 may include at least one photosensitive coupling device (charge coupled device, CCD), or at least A complementary metal-oxide semiconductor (complementary metal-oxide semiconductor, CMOS), etc. Thecradle device 10 further includes animage recognition module 20, which is used to identify the image data captured by theimage capture module 14, which can be a program code or logic circuit, etc., and acradle driving module 22, which is used to drive thecradle 12. Swing motion, thecradle drive module 22 is to comprise amotor 24, and it can be the DC motor (DC motor) of an annular rotation, so as to shake thecradle 12, and amotor control unit 26, is used for receiving the control signal to control themotor 24 action. Thecradle device 10 further includes astorage module 28, which is used to store the image data captured by theimage capture module 14 and as a data temporary storage space when theimage recognition module 20 processes data, and acontrol module 30, which is coupled to the image recognition Themodule 20 and thecradle driving module 22 are used to control thecradle driving module 22 according to the result of theimage recognition module 20 recognizing the image data captured by theimage capture module 14 , so as to control the swing of thecradle 12 . In addition, thecradle device 10 further includes abracket 32, a rod 34, which is rotatably connected to thebracket 32, and afixed rod 36, which is connected to the rod 34 and thecradle 12, and is used to carry thecradle 12, wherein the image capture Thefetching module 14 is installed on the rod 34, and thecradle driving module 22 is connected to the rod 34 and thefixed rod 36, and thecradle driving module 22 can be rotated on the X axis as shown in the direction of the arrow in Figure 2 (rolling ), so as to drive thefixed rod 36 to swing back and forth to swing thecradle 12, and can drive the rod 34 to rotate, so that theimage capture module 14 on the rod 34 can always be aligned with the baby in thecradle 12 synchronously, and continuously capture Take the image features of the baby. In addition, the image data captured by theimage capture module 14 may be image data of the baby's face, and theimage recognition module 20 further includes a faceregion detection unit 38 for capturing image feature information of the face, and a The eyearea detection unit 40 is used to extract the image feature information of the eyes from the image feature information of the face extracted by the facearea detection unit 38, wherein thecontrol module 30 is used to control the cradle according to the image feature information of theeyes drive module 22.

请参阅图3，图3为本发明利用图像识别技术控制摇篮12的摇摆状态的流程图，该方法包含下列步骤：Please refer to Fig. 3, Fig. 3 is the flow chart that the present invention utilizes image recognition technology to control the rocking state ofcradle 12, and this method comprises the following steps:

步骤100：开始。Step 100: start.

步骤102：利用图像撷取模块14撷取摇篮12中婴儿的脸部图像数据。Step 102: Use theimage capture module 14 to capture the face image data of the baby in thecradle 12 .

步骤104：图像识别模块20的脸部区域检测单元38依据肤色范围撷取出步骤102中图像撷取装置14所撷取的脸部图像数据的图像特征信息。Step 104: The facearea detection unit 38 of theimage recognition module 20 extracts image feature information of the face image data captured by theimage capture device 14 instep 102 according to the skin color range.

步骤106：图像识别模块20的眼睛区域检测单元40依据脸部区域检测单元38所撷取出的脸部图像特征信息产生一眼睛区域图像。Step 106 : The eyearea detection unit 40 of theimage recognition module 20 generates an eye area image according to the feature information of the facial image captured by the facearea detection unit 38 .

步骤108：利用Sobel边缘强化法(Sobel edge enhancement)处理步骤106所得出的该眼睛区域图像。Step 108: Process the eye region image obtained in step 106 by Sobel edge enhancement.

步骤110：利用二值化处理法处理步骤108所产生的图像数据。Step 110: Process the image data generated instep 108 by binarization.

步骤112：撷取步骤110所产生的图像数据的边缘特征点。Step 112: Extract edge feature points of the image data generated instep 110.

步骤114：利用蛇模型(snake model)处理法处理步骤112所产生的图像数据以产生眼睛图像的轮廓。Step 114: Process the image data generated instep 112 using a snake model processing method to generate the outline of the eye image.

步骤116：判断步骤114中所得出的该眼睛图像的轮廓的宽度/长度比是否大于一预定值，若是执行步骤118；若不是则执行步骤120。Step 116: Determine whether the width/length ratio of the outline of the eye image obtained instep 114 is greater than a predetermined value, if so, go tostep 118; if not, go tostep 120.

步骤118：控制模块30控制摇篮驱动模块22，藉以驱动摇篮12进行摇摆动作。Step 118: Thecontrol module 30 controls thecradle driving module 22, so as to drive thecradle 12 to perform a swinging motion.

步骤120：控制模块30控制摇篮驱动模块22，藉以控制摇篮12停止摇摆动作。Step 120: thecontrol module 30 controls thecradle driving module 22, thereby controlling thecradle 12 to stop the swinging action.

步骤122：结束。Step 122: end.

在此对上述步骤做一详细的说明，首先图像撷取模块14可被架设在杆件34上如图2中所示对准摇篮12中婴儿脸部的位置，藉以撷取摇篮12中婴儿的脸部图像数据，而摇篮12在摇摆的过程中由于杆件34与固定杆36会被摇篮驱动模块22同步带动，故杆件34与摇篮12亦会同步摇摆而使得图像撷取模块14可一直对准摇篮12中婴儿脸部的位置，因此可设定每间隔一段时间便利用图像撷取模块14撷取摇篮12中婴儿的脸部图像数据。而图像撷取模块14所撷取的婴儿的脸部图像数据可储存在存储模块28，之后图像识别模块20的脸部区域检测单元38便可依据肤色范围撷取出步骤102中图像撷取装置14所撷取的脸部图像数据的图像特征信息，举例来说，利用现有的肤色理论，依据统计数据中肤色可能的范围，判断步骤102中图像撷取装置14所撷取的脸部图像数据中哪些像素的Cr与Cb值座落在肤色可能的Cr、Cb值范围内，而取得脸部图像数据的图像特征信息(脸部的大小范围)，其中，可设定该脸部图像数据的图像特征信息中脸部肤色部分为白色，其余部分为黑色等。接下来，图像识别模块20的眼睛区域检测单元40可依据脸部区域检测单元38所撷取出的脸部图像特征信息产生该眼睛区域图像，意即可在脸部肤色部分的白色区域中找到两个黑色区域的眼睛区域图像，之后再利用传统的一些图像处理方法对该眼睛区域图像进行图像处理，而得出眼睛图像的外部轮廓，例如在此实施例中可利用Sobel边缘强化法(Sobel edge enhancement)处理步骤106所得出的该眼睛区域图像，再利用二值化处理法处理步骤108所产生的图像数据，再撷取步骤110所产生的图像数据的边缘特征点，最后再利用蛇模型(snake model)处理法处理步骤112所产生的图像数据以产生眼睛图像的轮廓。而由该脸部图像数据产生出该眼睛区域图像，以及由该眼睛区域图像中撷取出眼睛图像的轮廓的图像处理方法，可不局限于上述的图像处理方法，在此不再详述。Here, the above-mentioned steps are described in detail. First, theimage capture module 14 can be erected on the rod 34 to align with the position of the baby's face in thecradle 12 as shown in FIG. Face image data, while thecradle 12 is in the rocking process because the rod 34 and thefixed rod 36 will be driven synchronously by thecradle drive module 22, so the rod 34 and thecradle 12 will also be swayed synchronously so that theimage capture module 14 can always The position of the baby's face in thecradle 12 is aligned, so it can be set to use theimage capture module 14 to capture the face image data of the baby in thecradle 12 at intervals. The baby's face image data captured by theimage capture module 14 can be stored in thestorage module 28, and then the faceregion detection unit 38 of theimage recognition module 20 can capture theimage capture device 14 instep 102 according to the skin color range. The image feature information of the captured facial image data, for example, utilizes the existing skin color theory, according to the possible range of skin color in statistical data, to determine the facial image data captured by theimage capture device 14 instep 102 The Cr and Cb values of which pixels are located in the possible Cr and Cb value ranges of the skin color, and the image feature information (the size range of the face) of the facial image data is obtained, wherein the facial image data can be set In the image feature information, the skin color part of the face is white, and the rest is black. Next, the eyearea detection unit 40 of theimage recognition module 20 can generate the eye area image according to the facial image feature information captured by the facearea detection unit 38, which means that two eyes can be found in the white area of the skin color part of the face. An eye region image of a black region, and then utilize some traditional image processing methods to perform image processing on the eye region image to obtain the outer contour of the eye image, for example, in this embodiment, the Sobel edge enhancement method (Sobel edge method) can be used. enhancement) to process the eye region image obtained in step 106, then use the binarization method to process the image data generated instep 108, and then extract the edge feature points of the image data generated instep 110, and finally use the snake model ( The snake model) processing method processes the image data generated instep 112 to generate the outline of the eye image. The image processing method for generating the eye region image from the face image data and extracting the outline of the eye image from the eye region image is not limited to the above image processing method, and will not be described in detail here.

而所得到的眼睛图像轮廓便为判断婴儿是否进入睡眠状态的主要依据，请参阅图4，图4为步骤114所得出的眼睛图像轮廓的示意图，由该眼睛图像轮廓最外围的边缘特征点可得出一长度L以及一宽度W，当该眼睛图像轮廓的宽度/长度比(W/L)大于该预定值时，则代表婴儿的眼睛尚未阖上，故婴儿此时并未进入睡眠状态；反之，当该眼睛图像轮廓的宽度/长度比(W/L)小于该预定值时，则代表婴儿的眼睛已经阖上，故婴儿此时已进入睡眠状态。之后，控制模块30便可依据该眼睛图像轮廓的宽度/长度比(W/L)与该预定值的比较结果，来控制摇篮驱动模块22的动作，当该眼睛图像轮廓的宽度/长度比(W/L)大于该预定值时，由于此时判定婴儿并未进入睡眠状态，故控制模块30可输出一控制信号至摇篮驱动模块22的马达控制单元26，藉以控制马达24继续转动而驱动摇篮12以进行摇摆动作；而当眼睛图像轮廓的宽度/长度比(W/L)小于该预定值时，由于此时判定婴儿已进入睡眠状态，故控制模块30可输出一控制信号至摇篮驱动模块22的马达控制单元26，藉以控制马达24停止转动而停止驱动摇篮12的摇摆。除此之外，亦可设计为控制模块30可依据眼睛图像轮廓的宽度/长度比(W/L)来控制摇篮驱动模块22的马达24的运转速度，例如当眼睛图像轮廓的宽度/长度比(W/L)越大时，则代表婴儿现正处于清醒的状态，故可控制摇篮驱动模块22的马达24的运转速度较快，而使摇篮12的摆动幅度或摆动频率较大，以帮助婴儿入睡；若当眼睛图像轮廓的宽度/长度比(W/L)越小时，则代表婴儿现正处于慢慢入睡的状态，故可控制摇篮驱动模块22的马达24的运转速度较慢，而使摇篮12的摆动幅度或摆动频率较小，以避免婴儿在沉睡当中因为摇篮12晃动而受到惊吓，或是婴儿会养成熟睡时还需要摇篮12继续摇动的习惯。And the obtained eye image outline is just the main basis for judging whether the baby enters sleep state, please refer to Fig. 4, Fig. 4 is the schematic diagram of the eye image outline thatstep 114 draws, can be obtained by the edge feature point of the outermost edge of this eye image outline A length L and a width W are obtained. When the width/length ratio (W/L) of the eye image outline is greater than the predetermined value, it means that the baby's eyes have not been closed, so the baby has not entered a sleep state at this time; On the contrary, when the width/length ratio (W/L) of the outline of the eye image is smaller than the predetermined value, it means that the baby's eyes have been closed, so the baby has entered a sleep state at this time. Afterwards, thecontrol module 30 can control the action of thecradle driving module 22 according to the comparison result of the width/length ratio (W/L) of the eye image outline and the predetermined value, when the width/length ratio (W/L) of the eye image outline ( When W/L) is greater than the predetermined value, since it is determined that the baby has not entered a sleep state at this time, thecontrol module 30 can output a control signal to themotor control unit 26 of thecradle drive module 22, so as to control themotor 24 to continue to rotate and drive thecradle 12 to perform a rocking action; and when the width/length ratio (W/L) of the eye image outline is less than the predetermined value, since it is determined that the baby has entered a sleep state at this time, thecontrol module 30 can output a control signal to the cradle drive module Themotor control unit 26 of thecradle 22 controls themotor 24 to stop rotating so as to stop driving the swing of thecradle 12 . In addition, it can also be designed that thecontrol module 30 can control the operating speed of themotor 24 of thecradle driving module 22 according to the width/length ratio (W/L) of the eye image outline, for example, when the eye image outline width/length ratio When the (W/L) is larger, it means that the baby is now awake, so themotor 24 of thecradle drive module 22 can be controlled to run at a faster speed, so that the swing amplitude or frequency of thecradle 12 is larger to help The baby falls asleep; if the width/length ratio (W/L) of the eye image outline is smaller, it means that the baby is now slowly falling asleep, so themotor 24 of thecradle driving module 22 can be controlled to operate at a slower speed, and Make the swing amplitude or swing frequency of thecradle 12 smaller, so as to prevent the baby from being frightened by the shaking of thecradle 12 in the middle of a deep sleep, or the baby will develop the habit of still needing thecradle 12 to continue shaking when he is asleep.

相较于现有的摇篮装置，本发明的利用图像识别技术来控制摇篮摇摆状态的摇篮装置，可自动判断婴儿是否进入睡眠状态而实时地控制摇篮摇摆状态，故大人们不必亲自在摇篮旁摇动摇篮，而达到人性化控制摇篮摆动的机制。Compared with the existing cradle device, the cradle device of the present invention uses image recognition technology to control the cradle swing state, which can automatically determine whether the baby is in a sleep state and control the cradle swing state in real time, so adults do not have to rock beside the cradle Cradle, and achieve the mechanism of humanized control of the swing of the cradle.

以上所述仅为本发明的较佳实施例，凡依本发明申请专利范围所做的均等变化与修饰，皆应属本发明专利的涵盖范围。The above descriptions are only preferred embodiments of the present invention, and all equivalent changes and modifications made according to the scope of the patent application of the present invention shall fall within the scope of the patent of the present invention.

Claims (10)

1. cradle device that utilizes image recognition technology to control the cradle swinging condition, it includes:

One cradle;

One capturing images module is used for capturing a view data;

One picture recognition module is used for discerning this view data that this capturing images module is captured;

One cradle driver module is used for driving the oscillating motion of this cradle; And

One control module is coupled in this picture recognition module and this cradle driver module, and the result who is used for discerning according to this picture recognition module this view data controls this cradle driver module.

2. cradle device as claimed in claim 1, wherein, this capturing images module comprises a camera lens, is used for receiving the light that this view data of desire acquisition is launched or reflected, and an imageing sensor, this light that is used for this camera lens is received converts an electric signal to.

3. cradle device as claimed in claim 2, wherein, this imageing sensor comprises a photosensitive coupling component.

4. cradle device as claimed in claim 2, wherein, this imageing sensor comprises a complementary metal oxide semiconductors (CMOS).

5. cradle device as claimed in claim 1, wherein, this cradle driver module comprises a motor, and a motor control unit, is coupled in this control module, is used for controlling according to the control signal that this control module transmits the action of this motor.

6. cradle device as claimed in claim 1, it comprises in addition:

One support;

One rod member is connected in this support in rotating mode; And

One fixed bar is connected in this rod member and this cradle, is used for carrying this cradle;

Wherein, this capturing images module is mounted on this rod member, and this cradle driver module is to be connected in this rod member and this fixed bar.

7. method of utilizing image recognition technology to control a cradle swinging condition, it comprises the following step:

(a) acquisition one view data;

(b) be identified in this view data that step (a) is captured; And

(c) control the swinging condition of this cradle according to the recognition result of step (b).

8. method as claimed in claim 7, wherein, step (b) comprises the following step:

(b1) seek in this view data that step (a) captured between the view data that meets a pre-defined rule;

(b2) view data that is searched out according to step (b1) produces a length-width ratio;

Wherein, step (c) is to control the swinging condition of this cradle according to the length-width ratio that step (b2) produces.

9. method as claimed in claim 8 wherein, when step (c) comprises the length-width ratio that produces when step (b2) less than a predetermined value, is controlled this cradle and is carried out rocking action.

10. method as claimed in claim 8 wherein, when step (c) comprises the length-width ratio that produces when step (b2) greater than a predetermined value, is controlled this cradle and is stopped rocking action.

Images