CN115177938A - Interactive intelligent body-building mirror device - Google Patents

Abstract

The invention discloses an interactive intelligent fitness mirror device, wherein a processor is also used for acquiring a motion track of a user action according to user action data acquired by a sensor and controlling a suspension icon to be displayed on a user interface according to the motion track, and the suspension icon is used for indicating the position indicated by the user action in the user interface. According to the invention, the motion trail of the user action is obtained through the user action data, and the suspension icon is controlled to be displayed on the user interface according to the motion trail, so that the accuracy rate of the user action in the motion process is ensured, and the use efficiency, the smoothness and the comfort are improved.

Description

Translated fromChinese

交互式智能健身镜面装置Interactive smart fitness mirror device

技术领域technical field

本发明涉及智能家居及智能健身领域，具体涉及交互式智能健身镜面装置。The invention relates to the fields of smart home and smart fitness, in particular to an interactive smart fitness mirror device.

背景技术Background technique

随着经济的快速发展，人们生活节奏的加快，高速的生活和高负荷的工作使得城市生活的人们身体逐渐出现了各种健康问题，随之健身成为了人们关注的热点。智能健身镜，是集人工智能、硬件内容服务于一身的新型健身产品。With the rapid development of the economy and the accelerated pace of people's life, high-speed life and high-load work have gradually caused various health problems in people living in cities, and fitness has become a hot spot of people's attention. Smart fitness mirror is a new fitness product that integrates artificial intelligence and hardware content services.

目前，大部分在家健身人员通过智能健身镜观看视频进行学习训练动作，智能健身镜既可以当作镜子又可以看到高画质的教练图像。现有的智能健身镜交互方式麻烦，一般需要通过外部的智能设备(如智能手机)或者触摸屏进行控制，通过外部智能设备进行控制时操作较为复杂，还需要对智能设备和智能健身镜进行配网，而采用触摸控制则容易在镜面上留下指纹和汗水，影响屏幕的显示效果。同时，现在的健身镜体积较大，占用了较多的家庭空间，但是现在的智能健身镜只能显示健身内容及健身相关的数据，在日常生活中使用频率较低。基于此，本发明人通过在智能健身镜上增加个性化的界面及新的交互方式，优化智能健身镜与用户的交互方式，提高智能健身镜在日常生活中的使用频率。At present, most home fitness personnel watch videos through smart fitness mirrors to learn and train. The smart fitness mirror can be used as a mirror and can see high-quality coach images. The existing smart fitness mirrors are cumbersome to interact with, and generally need to be controlled through an external smart device (such as a smartphone) or a touch screen. The operation is complicated when controlled by an external smart device, and the smart device and the smart fitness mirror need to be networked. , and the use of touch control is easy to leave fingerprints and sweat on the mirror surface, affecting the display effect of the screen. At the same time, the current fitness mirrors are larger and occupy more home space, but the current smart fitness mirrors can only display fitness content and fitness-related data, and are used less frequently in daily life. Based on this, the present inventor optimizes the interaction between the smart fitness mirror and the user by adding a personalized interface and a new interaction mode to the smart fitness mirror, and increases the frequency of use of the smart fitness mirror in daily life.

随着研发深入，本发明人发现，用户动作对用户界面进行控制时，例如进入或选择或切入控制指令菜单时，特别是在选择控制指令菜单时，用户不能直观的获取自己动作所到达的位置，进而可能会出现做出动作却未触发相对应的控制指令，降低选择的准确率，影响用户使用舒适度和流畅度。With further research and development, the inventor found that when user actions control the user interface, for example, when entering or selecting or cutting into the control command menu, especially when selecting the control command menu, the user cannot intuitively obtain the position reached by his actions. , and then it may occur that the action is made but the corresponding control command is not triggered, which reduces the accuracy of the selection and affects the user's comfort and fluency.

发明内容SUMMARY OF THE INVENTION

本发明的一个目的在于提供交互式智能健身镜面装置，在用户做出动作时，通过用户动作的运动轨迹显示悬浮图标，提高选择的准确率以及用户使用的舒适度。An object of the present invention is to provide an interactive intelligent fitness mirror device, when a user makes an action, a floating icon is displayed through the motion trajectory of the user's action, so as to improve the accuracy of selection and the comfort of the user.

该目的采用以下技术方案实现：This purpose is achieved by the following technical solutions:

智能健身镜的一个例子包括用于接收健身教练的视频影像的通信接口、可操作地耦合到通信接口以显示健身教练的视频影像的显示器、以及设置在显示器前面以反射与显示器相对的人员的图像的镜子。该镜子用于展示用户在所述镜子上的镜像，所述显示器透过所述镜子向用户显示所述用户界面。An example of a smart fitness mirror includes a communication interface for receiving video imagery of the fitness trainer, a display operably coupled to the communication interface to display the video imagery of the fitness trainer, and disposed in front of the display to reflect an image of a person opposite the display mirror. The mirror is used to show the mirror image of the user on the mirror, and the display shows the user interface to the user through the mirror.

交互式健身方法的一个例子包括以下步骤：(1)将健身内容流式传输到交互式视频系统，该交互式视频系统包括具有部分反射部分的镜子和设置在部分反射部分的一侧的显示器；(2)经由显示器和镜子的部分反射部分向用户显示健身内容；(3)以及利用镜子反射用户的图像使得用户的图像至少部分地叠加在经由显示器和镜子的部分反射部分显示的健身内容上。An example of an interactive exercise method includes the steps of: (1) streaming exercise content to an interactive video system including a mirror having a partially reflective portion and a display disposed on one side of the partially reflective portion; (2) displaying the fitness content to the user via the display and the partially reflective portion of the mirror; (3) and reflecting an image of the user with the mirror such that the user's image is at least partially overlaid on the fitness content displayed via the display and the partially reflective portion of the mirror.

为了在智能健身镜上增加个性化的界面及新的交互方式，优化智能健身镜与用户的交互方式，提高智能健身镜在日常生活中的使用频率，本发明在为智能健身镜设计了一套操作系统Launcher，具体包括：(1)GUI界面；(2)动作控制的系统控制方法；(3)语音控制的系统控制方法；(4)多模态融合的控制方法；(5)可执行的控制指令；(6)数据采集管理的方法；(7)一些具体的功能管理方法。为智能健身镜这种设备定制了一套系统及操作方法，使智能健身镜的功能不止于观看健身锻炼视频，还可以作为智能家居的一个重要组成部分便于用户查看更多的信息，提升使用效率。同时，定制化的操作方法客服了传统智能健身镜需要通过触摸进行控制容易在部分反射镜面上留下指纹的弊病，由于智能健身镜的核心功能是可以让用户观看健身锻炼视频时对比自己镜像的动作，部分反射镜面上留下指纹后对智能健身镜的功能影响较大，导致大多数用户不愿意使用触摸交互，而使用智能终端对智能健身镜控制又不直观，比较麻烦。而采用动作控制的方法则没有这些问题，同时虽然手势控制在一些其他电子产品上也有应用，不过在智能健身镜上应用时也有突破性的进展，由于智能健身镜包括部分反射镜面，我们在进行动作控制时，能清楚的看到自己正在做的动作，降低误控制、误操作的可能，同时由于动作识别是智能健身镜的核心功能，智能健身镜无需额外增加成本就可以实现更高精度的复杂动作识别，使能够完成的控制指令相对于现有的手势控制也丰富很多，最后由于智能健身镜的主要功能是运动健身，将动作控制应用于智能健身镜还需要避免健身动作才来的控制误触，与传统的手势控制也有明显区别。In order to add a personalized interface and a new interaction mode to the smart fitness mirror, optimize the interaction mode between the smart fitness mirror and the user, and increase the frequency of use of the smart fitness mirror in daily life, the present invention designs a set of intelligent fitness mirrors. The operating system Launcher specifically includes: (1) GUI interface; (2) a system control method for motion control; (3) a system control method for voice control; (4) a control method for multimodal fusion; (5) executable Control instructions; (6) data collection and management methods; (7) some specific function management methods. A set of systems and operation methods are customized for a device such as a smart fitness mirror, so that the function of the smart fitness mirror is not only to watch fitness exercise videos, but also as an important part of the smart home to facilitate users to view more information and improve use efficiency. . At the same time, the customized operation method overcomes the disadvantage that traditional smart fitness mirrors need to be controlled by touch and easily leave fingerprints on part of the mirror surface. Because the core function of smart fitness mirrors is to allow users to compare their own mirror images when watching fitness exercise videos Fingerprints left on some reflective mirrors have a great impact on the function of the smart fitness mirror, which makes most users unwilling to use touch interaction, and the use of smart terminals to control the smart fitness mirror is not intuitive and cumbersome. The motion control method does not have these problems. At the same time, although gesture control is also applied to some other electronic products, there is also a breakthrough in the application of smart fitness mirrors. Since smart fitness mirrors include part of the reflective mirror surface, we are conducting During motion control, you can clearly see what you are doing, reducing the possibility of miscontrol and misoperation. At the same time, because motion recognition is the core function of smart fitness mirrors, smart fitness mirrors can achieve higher precision without additional cost. Complex action recognition makes the control commands that can be completed are much richer than the existing gesture control. Finally, because the main function of the smart fitness mirror is exercise and fitness, the application of motion control to the smart fitness mirror also needs to avoid the control that comes from fitness actions. Accidental touch is also significantly different from traditional gesture control.

但是，本发明通过用户动作数据来判断是否生成控制指令，每个人动作幅度且感官有所差异，用户做出动作在健身镜上进行交互时，不能确保做出的动作达到了交互区域，选择所需的交互界面，基于此，本发明在上述交互方式上进行优化，通过用户动作数据获取用户动作的运动轨迹，在用户界面上根据运动轨迹显示悬浮图标，所述悬浮图标用于指示所述用户的动作在所述用户界面中所指示的位置。进而直观的使用户知晓用户动作是否到达所需的位置，确保用户动作的准确率，提高使用的流畅度和舒适度。However, the present invention judges whether to generate a control command through user action data, each person's action range and senses are different, and when the user makes an action to interact on the fitness mirror, it cannot be ensured that the action made reaches the interactive area. Based on this, the present invention optimizes the above-mentioned interaction mode, obtains the motion trajectory of the user's action through user motion data, and displays a floating icon on the user interface according to the motion trajectory, and the floating icon is used to indicate the user. The action is at the indicated location in the user interface. Furthermore, the user can intuitively know whether the user's action has reached the required position, so as to ensure the accuracy of the user's action and improve the fluency and comfort of use.

优选的，当所述处理器判断所述传感器采集到所述用户动作数据所指示的动作匹配预设的悬浮图标唤醒交互所对应的悬浮图标唤醒动作时，启用悬浮图标使得所述悬浮图标随用户的动作移动。用户可根据需求启用或关闭悬浮图标。Preferably, when the processor determines that the action indicated by the user motion data collected by the sensor matches the preset floating icon wake-up interaction corresponding to the floating icon wake-up interaction, the floating icon is enabled so that the floating icon follows the user action moves. Users can enable or disable floating icons according to their needs.

优选的，当所述处理器判断所述用户在当前用户界面完成了选择操作后，所述处理器关闭悬浮图标功能使得悬浮图标不再跟随用户的动作移动。其中，所述选择操作包括手势停留，手势抓取，向前推掌心、向后推掌、摆手、握拳中的任一种。Preferably, after the processor determines that the user has completed the selection operation on the current user interface, the processor disables the floating icon function so that the floating icon no longer moves with the user's action. Wherein, the selection operation includes any one of gesture stop, gesture grab, palm push forward, palm push backward, hand waving, and fist clenching.

优选的，当所述处理器判断所述悬浮图标根据所述用户运动数据移动到所述用户界面中的所述用户界面对象的位置，以及判断所述用户在用户界面对象的位置进行了所述选择操作，所述处理器执行所述用户面对象所对应的操作。Preferably, when the processor determines that the floating icon is moved to the position of the user interface object in the user interface according to the user motion data, and determines that the user performs the above-mentioned operation at the position of the user interface object An operation is selected, and the processor executes the operation corresponding to the user plane object.

以下更详细讨论的前述概念和附加概念的所有组合(假设此类概念并不相互矛盾)可以被预期作为本文公开的发明主题的一部分。具体地，出现在本公开文本结尾的所要求保护的主题的所有组合都可以被预期是本文公开的发明主题的一部分。本文明确采用的并且也可以出现在通过引用并入的任何公开内容中的术语应当被赋予与本文公开的特定概念最一致的含义。All combinations of the foregoing concepts and additional concepts discussed in greater detail below, provided that such concepts are not mutually inconsistent, are contemplated as being part of the inventive subject matter disclosed herein. In particular, all combinations of the claimed subject matter appearing at the end of this disclosure are contemplated as being part of the inventive subject matter disclosed herein. Terms explicitly employed herein and which may also appear in any disclosure incorporated by reference should be given the meanings most consistent with the specific concepts disclosed herein.

本发明提供的一个或多个技术方案，至少具有如下技术效果或优点：本发明通过用户动作数据获取用户动作的运动轨迹，并控制悬浮图标根据运动轨迹在用户界面上显示，悬浮图标用于指示所述用户的动作在所述用户界面中所指示的位置。以保证运动过程中用户动作的准确率，提高使用效率、流畅度以及舒适度。One or more technical solutions provided by the present invention have at least the following technical effects or advantages: the present invention obtains the motion trajectory of user actions through user motion data, and controls the floating icon to be displayed on the user interface according to the motion trajectory, and the floating icon is used to indicate The location of the user's action indicated in the user interface. In order to ensure the accuracy of user actions during exercise, improve the efficiency, fluency and comfort of use.

附图说明Description of drawings

此处所说明的附图用来提供对本发明实施例的进一步理解，构成本申请的一部分，并不构成对本发明实施例的限定。在附图中：The accompanying drawings described herein are used to provide further understanding of the embodiments of the present invention, and constitute a part of the present application, and do not constitute limitations to the embodiments of the present invention. In the attached image:

图1为示例性智能健身镜的框图；1 is a block diagram of an exemplary smart fitness mirror;

图2为智能健身镜上的示例性GUI页面；Figure 2 is an exemplary GUI page on a smart fitness mirror;

图3为智能健身镜上的示例性GUI页面；3 is an exemplary GUI page on a smart fitness mirror;

图4为智能健身镜上的示例性GUI页面；4 is an exemplary GUI page on a smart fitness mirror;

图5-a为智能健身镜上的示例性GUI页面；Figure 5-a is an exemplary GUI page on a smart fitness mirror;

图5-b为智能健身镜上的示例性GUI页面；Figure 5-b is an exemplary GUI page on a smart fitness mirror;

图6为智能健身镜上的示例性GUI页面；6 is an exemplary GUI page on a smart fitness mirror;

图7为智能健身镜上的示例性GUI页面；7 is an exemplary GUI page on a smart fitness mirror;

图8为智能健身镜上的示例性GUI页面；8 is an exemplary GUI page on a smart fitness mirror;

图9为智能健身镜上的示例性GUI页面；9 is an exemplary GUI page on a smart fitness mirror;

图10为智能健身镜的动作识别示意图。FIG. 10 is a schematic diagram of motion recognition of the smart fitness mirror.

具体实施方式Detailed ways

为了能够更清楚地理解本发明的上述目的、特征和优点，下面结合附图和具体实施方式对本发明进行进一步的详细描述。需要说明的是，在相互不冲突的情况下，本发明的实施例及实施例中的特征可以相互组合。In order to understand the above objects, features and advantages of the present invention more clearly, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. It should be noted that the embodiments of the present invention and the features in the embodiments may be combined with each other under the condition that they do not conflict with each other.

在下面的描述中阐述了很多具体细节以便于充分理解本发明，但是，本发明还可以采用其他不同于在此描述范围内的其他方式来实施，因此，本发明的保护范围并不受下面公开的具体实施例的限制。Many specific details are set forth in the following description to facilitate a full understanding of the present invention. However, the present invention can also be implemented in other ways that are different from the scope of this description. Therefore, the protection scope of the present invention is not subject to the following disclosure. The limitations of the specific embodiment.

本领域技术人员应理解的是，在本发明的揭露中，术语“纵向”、“横向”、“上”、“下”、“前”、“后”、“左”、“右”、“竖直”、“水平”、“顶”、“底”“内”、“外”等指示的方位或位置关系是基于附图所示的方位或位置关系，其仅是为了便于描述本发明和简化描述，而不是指示或暗示所指的装置或元件必须具有特定的方位、以特定的方位构造和操作，因此上述术语不能理解为对本发明的限制。It should be understood by those skilled in the art that in the disclosure of the present invention, the terms "portrait", "horizontal", "upper", "lower", "front", "rear", "left", "right", " The orientation or positional relationship indicated by vertical, horizontal, top, bottom, inner, outer, etc. is based on the orientation or positional relationship shown in the accompanying drawings, which are only for the convenience of describing the present invention and The description is simplified rather than indicating or implying that the device or element referred to must have a particular orientation, be constructed and operate in a particular orientation, and thus the above terms should not be construed as limiting the invention.

因此，本专利涉及一种交互式智能健身镜面装置(也被称为“智能健身镜”和“交互式训练系统”)以及使用交互式训练设备的方法。智能健身镜包括被配置为显示锻炼内容(预先录制的视频或直播流)的显示器以及使得用户能够个性化锻炼的界面。此外，智能健身镜可以允许用户和/或教练在锻炼期间以类似于在其中用户和教练在同一个房间的健身房或小型健身工作室进行的常规锻炼的方式彼此交互(例如，向教练提供关于锻炼节奏的反馈，校正用户在特定健身计划期间的形式)。Accordingly, this patent relates to an interactive smart fitness mirror device (also referred to as a "smart fitness mirror" and an "interactive training system") and a method of using an interactive training device. The smart fitness mirror includes a display configured to display workout content (pre-recorded video or live stream) and an interface that enables the user to personalize the workout. Additionally, smart fitness mirrors may allow the user and/or trainer to interact with each other during a workout in a manner similar to a regular workout in a gym or small fitness studio where the user and trainer are in the same room (eg, provide the trainer with information about the workout) rhythmic feedback, correcting the user's form during a particular fitness program).

一个示例性的智能健身镜。An exemplary smart fitness mirror.

智能健身镜中的图像识别传感器(例如：摄像头)可以用于在用户进行活动(例如，锻炼)时获取用户的视频和/或静态图像。然后可以向教练分享用户的视频以允许教练在锻炼期间观察并向用户提供指导。还可以向其他智能健身镜的其他用户分享视频以进行比较或竞争。用户的视频也可以实时显示在显示120上或者存储以供以后回放。例如，通过提供用户与教练的视觉比较，用户的视频可以用于锻炼期间或之后的自我评估。存储的视频还可以允许用户在进行类似健身时评估他们随时间变化的进度或改进。Image recognition sensors (eg, cameras) in smart fitness mirrors can be used to capture video and/or still images of the user while the user is performing activities (eg, exercising). The user's video can then be shared with the coach to allow the coach to observe and provide guidance to the user during the workout. Videos can also be shared with other users of other smart fitness mirrors for comparison or competition. The user's video may also be displayed on display 120 in real-time or stored for later playback. For example, by providing a visual comparison of the user to the coach, the user's video can be used for self-assessment during or after a workout. Stored videos can also allow users to assess their progress or improvement over time while doing similar workouts.

传感器可以被配置为在用户进行活动(例如，锻炼)时获取用户的动态和/或静态图像，并根据动态和/或静态图像判断用户动作是否与标准动作匹配并输出判断结果，若用户动作与标准动作不匹配，则输出动作矫正信息。The sensor can be configured to acquire dynamic and/or static images of the user when the user performs activities (eg, exercise), and determine whether the user's action matches the standard action according to the dynamic and/or static images and output the determination result, if the user's action matches the standard action. If the standard action does not match, output action correction information.

同时，智能健身镜首先通过显示器120显示页面，用户使用智能健身镜时，根据用户右手的位置确定用户选中的页面对象，在交互时，用户右手做出动作指令，根据用户动作数据生成用户的骨骼点坐标，根据骨骼点坐标获取用户界面上悬浮图标对应的坐标，调用坐标对应的运动轨迹，控制悬浮图标根据运动轨迹在用户界面上显示，所述悬浮图标用于指示所述用户的动作在所述用户界面中所指示的位置。因此当用户做出动作指令时，悬浮图标随用户动作的轨迹在用户界面上显示，以便用户直观的知晓动作做出后在对应的用户界面所在的位置，提高选择的准确率。At the same time, the smart fitness mirror first displays the page through the display 120. When the user uses the smart fitness mirror, the page object selected by the user is determined according to the position of the user's right hand. During interaction, the user's right hand makes an action instruction, and the user's skeleton is generated according to the user's action data. Point coordinates, obtain the coordinates corresponding to the floating icon on the user interface according to the coordinates of the skeleton point, call the motion trajectory corresponding to the coordinates, and control the floating icon to be displayed on the user interface according to the motion trajectory. the location indicated in the user interface described above. Therefore, when the user makes an action instruction, the floating icon is displayed on the user interface along with the trajectory of the user's action, so that the user can intuitively know the position of the corresponding user interface after the action is made, and the accuracy of selection is improved.

在一个或多个实施例中，悬浮图标可以为任意形状，包括但不限于球形、三角形、星形。悬浮图标可以为任意颜色，包括但不限于黑色、灰色、白色。In one or more embodiments, the floating icon can be in any shape, including but not limited to a sphere, a triangle, and a star. The hover icon can be any color, including but not limited to black, gray, and white.

具体的，悬浮图标功能的开启或关闭可以由特定的图形用户界面(GUI)或锻炼页面决定，如在图形用户界面(GUI)的主页面默认开启动作命令控制功能而在其他页面默认关闭动作命令控制功能；悬浮图标功能的开启或关闭还可以根据其他传感器检测到的数据进行决定。Specifically, the activation or deactivation of the floating icon function can be determined by a specific graphical user interface (GUI) or exercise page. For example, the action command control function is enabled by default on the main page of the graphical user interface (GUI) and disabled by default on other pages. Control function; the opening or closing of the floating icon function can also be determined according to the data detected by other sensors.

一个具体的智能健身镜可以根据图像识别传感器采集到的用户动作对悬浮图标功能进行控制的示例，所述悬浮图标唤醒页面用于在所述传感器采集到用户动作数据匹配预设的悬浮图标唤醒动作时，启用悬浮图标。预设的悬浮图标唤醒动作可以是预先在智能健身镜中设定好的动作，也可以是用户自己录制的动作。悬浮图标唤醒动作包括但不限于双手举过头顶比心以及各种静态手势，维持指定时间的范围一般为0-5秒或更长。A specific example in which the smart fitness mirror can control the function of the floating icon according to the user action collected by the image recognition sensor, and the floating icon wake-up page is used to match the preset floating icon wake-up action when the user motion data collected by the sensor matches. , enable floating icons. The preset wake-up action of the floating icon can be a pre-set action in the smart fitness mirror, or it can be an action recorded by the user himself. The wake-up action of the floating icon includes, but is not limited to, raising both hands above the head and heart-to-heart and various static gestures, and the range of maintaining the specified time is generally 0-5 seconds or longer.

一个具体的智能健身镜可以根据语音识别传感器采集到的语音对悬浮图标功能进行控制的示例，通过唤醒词开启或关闭悬浮图标功能。A specific example of the smart fitness mirror can control the function of the floating icon according to the voice collected by the voice recognition sensor, and turn on or off the function of the floating icon through a wake-up word.

一个具体的智能健身镜可以根据触摸传感器对悬浮图标功能进行控制的示例，智能健身镜可以根据触摸屏采集到的用户触控指令对悬浮图标功能进行控制。A specific example in which the smart fitness mirror can control the function of the floating icon according to the touch sensor, the smart fitness mirror can control the function of the floating icon according to the user's touch command collected by the touch screen.

在唤醒时，悬浮图标的出现位置包括但不限于从页面上方出现、页面下方出现、页面左边出现、页面右边出现、页面中间出现。When waking up, the positions of the floating icons include, but are not limited to, appearing from the top of the page, appearing at the bottom of the page, appearing on the left side of the page, appearing on the right side of the page, and appearing in the middle of the page.

悬浮图标的出现方式包括但不限于可以为滑出、弹出、淡出。The appearance of the floating icon includes, but is not limited to, slide out, pop up, and fade out.

一个具体的智能健身镜可以根据触摸传感器对悬浮图标功能进行控制的示例，智能健身镜可以在处理器判断所述用户在当前用户界面完成了选择操作后，所述处理器关闭悬浮图标功能使得悬浮图标不再跟随用户的动作移动。当再次需要使用悬浮图标时，通过预设的悬浮图标唤醒交互所对应的悬浮图标唤醒动作唤醒悬浮图标。在一个或多个实施例中，所述选择操作包括手势停留，手势抓取，向前推掌心、向后推掌、摆手、握拳中的任一种。A specific example in which the smart fitness mirror can control the floating icon function according to the touch sensor, the smart fitness mirror can turn off the floating icon function after the processor determines that the user has completed the selection operation on the current user interface, so that the floating icon function is suspended. Icons no longer move with the user's actions. When the suspension icon needs to be used again, the suspension icon is awakened by the suspension icon awakening action corresponding to the preset suspension icon awakening interaction. In one or more embodiments, the selection operation includes any one of gesture staying, gesture grabbing, palm pushing forward, palm pushing backward, hand waving, and fist clenching.

骨骼点坐标Skeleton point coordinates

在一个或多个实施例中，生成用户的骨骼点坐标并在用户界面上获取悬浮图标对应的坐标，包括多种方式，在本实施例中可以为根据用户动作数据获取头顶骨骼点、盆骨骨骼点、左手骨骼点和右手骨骼点；获取头顶骨骼点和盆骨骨骼点之间的直线距离，根据所述直线距离获取坐标系的单位长度；获取头顶骨骼点和盆骨骨骼点之间的直线距离，将直线距离的六分之一作为坐标系的单位长度，即坐标系的单位长度1的大小为直线距离的六分之一；将头顶骨骼点和盆骨骨骼点之间所在直线作为纵坐标、盆骨骨骼点为原点，并以获取的单位长度构建直角坐标系；根据坐标系建立若干交互区域，页面对应的交互区域即为坐标系建立的交互区域。In one or more embodiments, generating the coordinates of the user's skeleton points and obtaining the coordinates corresponding to the floating icon on the user interface includes various methods. Bone point, left-hand bone point, and right-hand bone point; obtain the straight line distance between the top bone point and the pelvic bone point, and obtain the unit length of the coordinate system according to the straight line distance; obtain the distance between the top bone point and the pelvic bone point. For straight-line distance, take one-sixth of the straight-line distance as the unit length of the coordinate system, that is, the size of the unit length 1 of the coordinate system is one-sixth of the straight-line distance; take the straight line between the top bone point and the pelvic bone point as The vertical coordinate and the pelvic bone point are the origin, and a rectangular coordinate system is constructed with the obtained unit length; several interactive areas are established according to the coordinate system, and the interactive area corresponding to the page is the interactive area established by the coordinate system.

所述动作可以设置有个人偏好(左右撇子、残疾等)，同时具备有单手和双手的不同模式，包括左/右手模式、双手同时进行的模式(如左手选择，右手切屏)。The action can be set with personal preferences (left-handed, disabled, etc.), and has different modes of one-handed and two-handed, including left/right-handed mode, two-handed mode (such as left-handed selection, right-handed screen cut).

此外，控制动作和控制指令相对应，匹配一种控制动作，则代表处理器执行和该控制动作想匹配的控制指令。In addition, a control action corresponds to a control instruction, and if a control action is matched, it means that the processor executes a control instruction that wants to match the control action.

页面动效page animation

图形用户界面(GUI)还包括页面动效，所述页面动效包括但不限于页面切换动效和页面交互动效。其中页面切换动效包括翻页动效、返回上一级、进入下一级动效等，页面切换动效主要是针对页面操作和变化的动效，目的是使页面操作和变化在显示时具有更好的用户体验。其中页面交互动效包括选择确认动效、动作引导动效等，页面交互动效主要是针对用户在进行交互时为用户提供引导，如提示用户当前正在进行的交互、交互的进度以及提示用户进行交互时需要进行的操作。进一步具体包括但不限于，在用户进行交互时对涉及的页面信息进行高亮或闪烁、在涉及的页面信息上显示进度条等。还包括在进行动作控制时，对用户正在做的动作进行识别并显示提示动作或建议动作等。还包括在进行语音控制时，根据用户的语音指令进行补全提示等。The graphical user interface (GUI) further includes page animation effects, including but not limited to page switching animation effects and page interaction animation effects. The page switching animation includes page turning, returning to the previous level, entering the next level, etc. The page switching animation is mainly for page operations and changes, and the purpose is to make page operations and changes display. Better user experience. The page interaction animation includes selection confirmation animation, action guidance animation, etc. The page interaction animation mainly provides guidance for the user when interacting, such as prompting the user for the current interaction, the progress of the interaction, and prompting the user to perform Actions that need to be performed when interacting. It further specifically includes, but is not limited to, highlighting or flashing the involved page information when the user interacts, displaying a progress bar on the involved page information, and the like. It also includes recognizing the actions that the user is doing and displaying prompt actions or suggested actions when performing action control. It also includes the completion and prompting according to the user's voice command when performing voice control.

进一步的，一个具体的智能健身镜悬浮动作控制示例，智能健身镜首先通过显示器120显示如图5-a的第三主页面，第三主页面包括四宫格排布并列显示的页面对象，用户双手举过头顶比心维持5秒，悬浮球从第三主页面的上方滑出，用户使用智能健身镜时，根据用户右手的位置确定用户选中的页面对象，即当用户的右手处于人体右侧肩部以上时，选中页面对象“天气”，此时悬浮图标随用户右手出现在“天气”区域中；当用户的右手处于人体右侧肩部以下时，选中页面对象“提醒”，此时悬浮图标随用户右手出现在“提醒”区域中；当用户的右手处于人体左侧肩部以上时，选中页面对象“音乐”，此时悬浮图标随用户右手出现在“音乐”区域中；当用户的右手处于人体右侧肩部以下时，选中页面对象“体脂秤”，此时悬浮图标随用户右手出现在“体脂秤”区域中。Further, for a specific example of the suspension action control of the smart fitness mirror, the smart fitness mirror first displays the third main page as shown in Figure 5-a through the display 120. Raise your hands above your head and hold your heart for 5 seconds. The floating ball slides out from the top of the third main page. When the user uses the smart fitness mirror, the page object selected by the user is determined according to the position of the user's right hand, that is, when the user's right hand is on the right side of the human body When it is above the shoulder, select the page object "Weather", and the floating icon will appear in the "Weather" area along with the user's right hand; when the user's right hand is below the right shoulder of the human body, select the page object "Reminder", and then the floating icon will appear in the "Weather" area. The icon appears in the "Reminder" area with the user's right hand; when the user's right hand is above the left shoulder of the human body, select the page object "Music", and the floating icon appears in the "Music" area with the user's right hand; When the right hand is below the right shoulder of the human body, select the page object "Body Fat Scale", and the floating icon will appear in the "Body Fat Scale" area with the user's right hand.

传感器sensor

在一个或多个实施例中，传感器还可以为图像识别传感器(摄像头)，图像识别传感器(摄像头)获取用户的动态和/或静态图像，获取用户的动态和/或静态图像后处理为骨骼点图像，并根据骨骼点图像判断用户动作是否与标准动作或锻炼界面的视频影像展示的动作匹配并输出判断结果，还可以在用户使用智能健身镜时或在锻炼完成之后实时处理动态和/或静态图像，以基于用户的运动和动作来对智能健身镜进行控制或导出用户的生物识别数据。In one or more embodiments, the sensor may also be an image recognition sensor (camera), and the image recognition sensor (camera) acquires dynamic and/or static images of the user, and then processes the dynamic and/or static images of the user into skeleton points image, and judge whether the user's action matches the standard action or the action shown in the video image of the exercise interface according to the skeleton point image and output the judgment result, and can also process dynamic and/or static in real time when the user uses the smart fitness mirror or after the exercise is completed images to control smart fitness mirrors based on the user's movements and movements or derive the user's biometric data.

传感器还可以为智能健身镜中的红外识别传感器，红外识别传感器可以用于在用户进行活动(例如，锻炼)时获取用户的热成像图，并根据热成像图判断用户运动时锻炼的部位、发力的部位及消耗的热量，并根据用户运动时锻炼的部位、发力的部位判断用户动作是否与标准动作匹配并输出判断结果，若用户动作与标准动作不匹配，则输出动作矫正信息The sensor can also be an infrared recognition sensor in the smart fitness mirror, and the infrared recognition sensor can be used to obtain a thermal image of the user when the user is performing activities (for example, exercise), and judge the part of the user exercising, hair and hair according to the thermal image. The part of the force and the calories consumed, and according to the part where the user is exercising and the part where the force is exerted, determine whether the user's action matches the standard action and output the judgment result. If the user's action does not match the standard action, output the action correction information.

传感器还可以为智能健身镜中的语音识别传感器(麦克风)，语音识别传感器(麦克风)可以用于在用户使用智能健身镜时获取实时处理用户的语音信息，以基于用户的语音信息来对智能健身镜进行控制或与用户进行对话。The sensor can also be a voice recognition sensor (microphone) in the smart fitness mirror, and the voice recognition sensor (microphone) can be used to acquire and process the user's voice information in real time when the user is using the smart fitness mirror, so as to adjust the smart fitness based on the user's voice information. mirrors to control or have a conversation with the user.

一个示例性的智能健身镜。An exemplary smart fitness mirror.

用户可以直接与智能健身镜对接来控制智能健身镜(例如，动作命令、语音命令、触摸命令等)。可以提供在显示器120上显示的图形用户界面(GUI)以促进用户与智能健身镜的交互。The user can directly interface with the smart fitness mirror to control the smart fitness mirror (eg, motion commands, voice commands, touch commands, etc.). A graphical user interface (GUI) displayed on the display 120 may be provided to facilitate user interaction with the smart fitness mirror.

动作命令控制(包括“体式控制”和“手势控制”)Action command control (including "Asana Control" and "Gesture Control")

智能健身镜可以根据图像识别传感器采集到的用户动作对图形用户界面(GUI)进行控制。The smart fitness mirror can control the Graphical User Interface (GUI) according to the user actions collected by the image recognition sensor.

具体的，动作命令控制的动作包括但不限于静态动作和动态动作。静态动作是指用户将特定的身体部位做出特定的姿势并维持指定时间；动态动作是指用户将特定的身体部位做出特定的动作。Specifically, the actions controlled by the action command include but are not limited to static actions and dynamic actions. Static action means that the user makes a specific body part in a specific posture and maintains it for a specified time; dynamic action means that the user makes a specific action for a specific body part.

具体的，静态动作包括但不限于举起左/右手、抬起左/右腿、双手举过头顶比心以及各种静态手势等，维持指定时间的范围一般为0-5秒或更长，只要是通过人体的特定部位相对于空间、人体整体或其他特定部位间的位置关系确定，并有维持的指定时间的动作，均应落入静态动作的保护范围。进一步具体的如：左手打直与水平面呈15°/30°角维持1.5秒、左手与右手呈直角维持1秒、左手向前伸出维持0.1秒等。Specifically, the static movements include but are not limited to raising the left/right hand, raising the left/right leg, raising both hands above the head and heart, and various static gestures, etc. The range of maintaining the specified time is generally 0-5 seconds or longer. As long as it is determined by the positional relationship between a specific part of the human body relative to space, the entire human body, or other specific parts, and is maintained for a specified time, it should fall into the protection scope of static actions. More specifically, the left hand is held at an angle of 15°/30° to the horizontal plane for 1.5 seconds, the left hand is held at a right angle for 1 second, and the left hand is stretched forward for 0.1 seconds.

具体的，动态动作包括但不限于水平滑动左/右手、上下滑动左/右手、双手击掌、跳起、蹲下以及各种动态手势等，只要是通过人体的特定部位相对于空间、人体整体或其他特定部位的移动轨迹确定的动作均应落入动态动作的保护范围。进一步具体的如：水平滑动左手使左手从身体左侧移动到身体右侧、水平滑动左手使左手从身体右侧移动到身体左侧、水平滑动右手使右手从身体右侧移动到身体左侧、上下滑动左手使左手从肩部以上移动到腰部以下、上下滑动左手使左手从人体整体的上20％处以上移动到上40％处以下、双手在头顶击掌、双手在左侧肩部以上击掌等。Specifically, dynamic actions include but are not limited to horizontal sliding left/right hand, up and down sliding left/right hand, high-five with both hands, jumping up, squatting and various dynamic gestures, etc., as long as they are relative to space, the whole human body or the human body through specific parts of the human body. Actions determined by the movement trajectory of other specific parts should fall within the protection scope of dynamic actions. Further specific examples are: sliding the left hand horizontally to move the left hand from the left side of the body to the right side of the body, sliding the left hand horizontally to move the left hand from the right side of the body to the left side of the body, sliding the right hand horizontally to move the right hand from the right side of the body to the left side of the body, Slide the left hand up and down to move the left hand from above the shoulders to below the waist, slide the left hand up and down to move the left hand from the upper 20% of the body to the upper 40% of the body, high five with both hands above the head, high five with both hands above the left shoulder, etc. .

具体的，动作命令控制执行的控制指令包括但不限于对非锻炼页面(唤醒页面、主页面、其他页面)的控制指令和对锻炼页面的控制指令。对非锻炼页面的控制指令主要涉及用户与非锻炼页面的交互指令。对锻炼页面的控制指令主要涉及用户对锻炼课程播放的控制指令。Specifically, the control instructions controlled by the action command include, but are not limited to, control instructions for non-exercise pages (wake-up pages, home pages, other pages) and control instructions for exercise pages. The control instruction on the non-exercise page mainly involves the user's interaction instruction with the non-exercise page. The control instructions on the exercise page mainly involve the user's control instructions for playing the exercise course.

具体的，对非锻炼页面的控制指令包括但不限于非锻炼页面的切换、上一页、下一页、确定、返回、选择课程、课程收藏、加入计划、进入页面切换界面、唤醒语音助手、多媒体音量控制等。Specifically, the control instructions for non-exercise pages include but are not limited to switching non-exercise pages, previous page, next page, confirm, return, select courses, course collections, join plans, enter page switching interface, wake up voice assistant, Multimedia volume control, etc.

具体的，对锻炼页面的控制指令包括但不限于课程暂停、课程播放、上一环节、下一环节、课程背景音乐音量控制、课程教练音量控制、课程评价等。Specifically, the control instructions on the exercise page include but are not limited to course pause, course play, previous link, next link, course background music volume control, course coach volume control, course evaluation, and the like.

具体的，用户还可以通过多个动作的组合实现不同的功能。具体的包括但不限于使用第一动作进入第一级控制指令菜单，然后使用第二动作在第一级控制指令菜单中进行指令选择或进入第二级级控制指令菜单，所述控制指令菜单可以在显示器上展示给用户。进一步的具体的包括但不限于第一级控制指令菜单可以是课程选择控制指令菜单，包括播放课程、收藏课程、显示课程详情、预约课程和进入第二级级控制指令菜单指令，第二级级控制指令菜单可以是课程评价菜单，包括好评、中评、差评指令。Specifically, the user can also implement different functions through a combination of multiple actions. Specifically, it includes but is not limited to using the first action to enter the first-level control instruction menu, and then using the second action to select an instruction in the first-level control instruction menu or enter the second-level control instruction menu. The control instruction menu can be displayed to the user on the display. Further details include, but are not limited to, the first-level control instruction menu may be a course selection control instruction menu, including playing courses, saving courses, displaying course details, reserving courses, and entering the second-level control instruction menu instructions. The control instruction menu can be a course evaluation menu, including praise, moderate evaluation, and negative evaluation instructions.

具体的，还包括在图形用户界面(GUI)对用户动作命令控制进行提示。具体的包括但不限于，在图形用户界面(GUI)显示可操作的动作的图标、在图形用户界面(GUI)对用户静态动作指向的控制指令进行标记或突出显示、在图形用户界面(GUI)对用户静态动作维持时间的进度条进行显示、在图形用户界面(GUI)对用户动态动作进行指引(当用户动态动作完成度超过指定比例时)等。Specifically, it also includes prompting the user action command control in a graphical user interface (GUI). Specifically, it includes, but is not limited to, displaying icons of operable actions on the graphical user interface (GUI), marking or highlighting the control instructions pointed to by the user's static actions on the graphical user interface (GUI), displaying on the graphical user interface (GUI) Display the progress bar of the maintenance time of the user's static action, and guide the user's dynamic action on the graphical user interface (GUI) (when the user's dynamic action completion degree exceeds a specified ratio), etc.

具体的，还包括智能健身镜可以根据图像识别传感器采集到的用户动作对图形用户界面(GUI)进行控制时依赖其他传感器提供辅助判断。具体的包括但不限于通过语音识别传感器开启/中断/确认动作命令控制、通过面部识别传感器判断用户的正面或背面(仅当用户正面面对智能健身镜时开启动作命令控制)、通过面部识别传感器区分不同用户以加载自定义的动作命令控制指令、通过IMU优化动作/轨迹识别精度等。Specifically, it also includes that the smart fitness mirror can rely on other sensors to provide auxiliary judgment when controlling the Graphical User Interface (GUI) according to the user actions collected by the image recognition sensor. Specifically, it includes but is not limited to enabling/interrupting/confirming motion command control through the voice recognition sensor, judging the front or back of the user through the facial recognition sensor (only when the user is facing the smart fitness mirror, turn on the motion command control), through the facial recognition sensor Distinguish different users to load custom motion command control instructions, optimize motion/trajectory recognition accuracy through IMU, etc.

具体的，动作命令控制的动作可以是预先在智能健身镜中设定好的动作，也可以是用户自己录制的动作。动作命令控制执行的控制指令可以是预先在智能健身镜中设定好的指令，也可以是用户自己设置的自动化指令。动作命令控制的动作和执行的控制指令可以是预先设置好的对应关系，也可以是用户自己设置的对应关系，可以是一对一的关系，也可以是多对一的关系。Specifically, the action controlled by the action command may be a pre-set action in the smart fitness mirror, or may be an action recorded by the user himself. The control instructions to be executed by the motion command control can be pre-set instructions in the smart fitness mirror, or can be automated instructions set by the user. The action controlled by the action command and the executed control instruction may be a preset corresponding relationship, or a corresponding relationship set by the user, a one-to-one relationship, or a many-to-one relationship.

具体的，动作命令控制功能的开启或关闭可以由特定的图形用户界面(GUI)或锻炼页面决定，如在图形用户界面(GUI)的主页面默认开启动作命令控制功能而在其他页面默认关闭动作命令控制功能；动作命令控制功能的开启或关闭还可以根据其他传感器检测到的数据进行决定；动作命令控制功能的开启或关闭还可以是全部功能的开启/关闭或部分功能的开启/关闭。Specifically, the activation or deactivation of the action command control function can be determined by a specific graphical user interface (GUI) or exercise page. For example, the action command control function is enabled by default on the main page of the GUI and disabled by default on other pages. The command control function; the opening or closing of the action command control function can also be determined according to the data detected by other sensors; the opening or closing of the action command control function can also be the opening/closing of all functions or the opening/closing of some functions.

一个具体的智能健身镜可以根据图像识别传感器采集到的用户动作对图形用户界面(GUI)进行控制的示例，智能健身镜首先通过显示器120显示图形用户界面(GUI)的第一页面，第一页面上显示有举左手进入页面切换模式的第一提示图标，用户举起左手，第一提示图标进入高亮状态并开启进度条，1.5秒后，进度条完成，智能健身镜通过显示器120显示用于切换页面显示的页面切换页面，切换页面上除突出显示的第一页面，还包括第一页面的上一页“第二页面”和第一页面的下一页“第三页面”，切换页面上还显示有上一页、下一页、退出和确认的提示图标，其中上一页的提示图标代表左手从左往右水平挥动的动作，下一页的提示图标代表右手从右往左水平挥动的动作，退出的提示图标代表举左手，确定的提示图标代表举右手，用户此时做出左手从左往右水平挥动，切换页面变为突出显示第二页面，此时用户举起右手，代表确认的图标进入高亮状态并开启进度条，1.5秒后，进度条完成，智能健身镜通过显示器120显示第二页面。所述第一页面、第二页面、第三页面、第四页面和第五页面可以分别是唤醒页面、主页面、锻炼页面或其他页面。A specific example in which the smart fitness mirror can control the graphical user interface (GUI) according to the user actions collected by the image recognition sensor. The smart fitness mirror first displays the first page of the graphical user interface (GUI) through the display 120. The first page There is a first prompt icon for raising the left hand to enter the page switching mode. When the user raises the left hand, the first prompt icon is highlighted and the progress bar is turned on. After 1.5 seconds, the progress bar is completed, and the smart fitness mirror is displayed on the display 120 for Switch the page displayed on the page. Switch the page. In addition to the highlighted first page, the switch page also includes the previous page "second page" of the first page and the next page "third page" of the first page. There are also prompt icons for the previous page, the next page, exit and confirmation. The prompt icon on the previous page represents the left hand swiping horizontally from left to right, and the prompt icon on the next page represents the right hand waving horizontally from right to left. The prompt icon for exit means raising the left hand, and the prompt icon for confirming means raising the right hand. At this time, the user makes a horizontal wave of the left hand from left to right, and the switching page changes to highlight the second page. At this time, the user raises the right hand, which means The confirmed icon enters the highlighted state and the progress bar is turned on. After 1.5 seconds, the progress bar is completed, and the smart fitness mirror displays the second page through the display 120 . The first page, the second page, the third page, the fourth page and the fifth page may be a wake-up page, a home page, an exercise page or other pages, respectively.

一个具体的智能健身镜可以根据图像识别传感器采集到的用户动作对图形用户界面(GUI)进行控制的示例，智能健身镜首先通过显示器120显示图形用户界面(GUI)的第四页面，第四页面上显示有代表左手从左往右水平挥动进入上一页的第二提示图标和代表右手从右往左水平挥动进入下一页的第三提示图标，用户此时做出右手从右往左水平挥动的动作，智能健身镜通过显示器120显示由第四页面转入第五页面。A specific example in which the smart fitness mirror can control the graphical user interface (GUI) according to the user actions collected by the image recognition sensor, the smart fitness mirror first displays the fourth page of the graphical user interface (GUI) through the display 120, and the fourth page There are a second prompt icon representing the left hand swiping horizontally from left to right to enter the previous page, and a third prompt icon representing the right hand swiping horizontally from right to left to enter the next page. At this time, the user makes the right hand horizontally from right to left. In the waving action, the smart fitness mirror is displayed on the display 120 from the fourth page to the fifth page.

一个具体的智能健身镜可以根据图像识别传感器采集到的用户动作对图形用户界面(GUI)进行控制的示例，智能健身镜首先通过显示器120显示如图5-a的第三主页面，第三主页面包括多个并列显示的页面对象，同时第三主页面还展示了选中的页面对象及进入选中的页面对象的交互对象，以及第三主页面下方还展示了切换页面对象的交互对象，用户可以通过切换页面对象的交互对象的交互切换显示的页面对象，并通过执行进入选中的页面对象的交互对象的交互进入选中的页面对象对应的页面。用户可以通过右手/左手向左斜上方、左斜下方、右斜上方、右斜下方举起来选择选中的页面对象，并在选择完成后维持住右手/左手的姿态，同时使手部做出指定的静态动作如五指张开1秒即可进入选中的页面对象对应的页面。同时，用户可以做出右手从左往右水平挥动的动作切换显示的页面对象，进入第四主页面。A specific example in which the smart fitness mirror can control the graphical user interface (GUI) according to the user actions collected by the image recognition sensor. The smart fitness mirror first displays the third home page as shown in FIG. 5-a through the display 120. The third home page The page includes multiple page objects displayed side by side. At the same time, the third main page also displays the selected page object and the interactive object entering the selected page object, and the bottom of the third main page also displays the interactive object for switching page objects. Users can The displayed page object is switched by the interaction of the interactive object of the switching page object, and the page corresponding to the selected page object is entered by executing the interaction of the interactive object entering the selected page object. The user can select the selected page object by raising the right/left hand diagonally upward, left diagonally downward, right diagonally above, and right diagonally downward, and after the selection is completed, maintain the posture of the right hand/left hand, and make the hand make a designation The static action of , such as opening five fingers for 1 second, can enter the page corresponding to the selected page object. At the same time, the user can switch the displayed page objects by waving the right hand horizontally from left to right, and enter the fourth main page.

语音命令控制voice command control

智能健身镜可以根据语音识别传感器采集到的用户语音对图形用户界面(GUI)进行控制。The smart fitness mirror can control the Graphical User Interface (GUI) according to the user's voice collected by the voice recognition sensor.

具体的，用户可以通过语音命令控制的语音识别结果来代替动作命令控制的动作并实现相同的控制效果。Specifically, the user can use the voice recognition result controlled by the voice command to replace the action controlled by the action command and achieve the same control effect.

具体的，动作命令控制执行的控制指令包括但不限于对非锻炼页面(唤醒页面、主页面、其他页面)的控制指令和对锻炼页面的控制指令。对非锻炼页面的控制指令主要涉及用户与非锻炼页面的交互指令。对锻炼页面的控制指令主要涉及用户对锻炼课程播放的控制指令Specifically, the control instructions controlled by the action command include, but are not limited to, control instructions for non-exercise pages (wake-up pages, home pages, other pages) and control instructions for exercise pages. The control instruction on the non-exercise page mainly involves the user's interaction instruction with the non-exercise page. The control instructions on the exercise page mainly involve the user's control instructions on the exercise course playback

具体的，对非锻炼页面的控制指令包括但不限于非锻炼页面的切换、上一页、下一页、确定、返回、选择课程、课程收藏、加入计划、进入页面切换界面、唤醒语音助手、多媒体音量控制等。Specifically, the control instructions for non-exercise pages include but are not limited to switching non-exercise pages, previous page, next page, confirm, return, select courses, course collections, join plans, enter page switching interface, wake up voice assistant, Multimedia volume control, etc.

具体的，对锻炼页面的控制指令包括但不限于课程暂停、课程播放、上一环节、下一环节、课程背景音乐音量控制、课程教练音量控制、课程评价等。Specifically, the control instructions on the exercise page include but are not limited to course pause, course play, previous link, next link, course background music volume control, course coach volume control, course evaluation, and the like.

具体的，用户还可以通过多个动作的组合实现不同的功能。具体的包括但不限于使用第一语音进入第一级控制指令菜单，然后使用第二语音在第一级控制指令菜单中进行指令选择或进入第二级级控制指令菜单，所述控制指令菜单可以在显示器上展示给用户。进一步的具体的包括但不限于第一级控制指令菜单可以是课程选择控制指令菜单，包括播放课程、收藏课程、显示课程详情、预约课程和进入第二级级控制指令菜单指令，第二级级控制指令菜单可以是课程评价菜单，包括好评、中评、差评指令。Specifically, the user can also implement different functions through a combination of multiple actions. Specifically, it includes but is not limited to using the first voice to enter the first-level control instruction menu, and then using the second voice to select an instruction in the first-level control instruction menu or enter the second-level control instruction menu. The control instruction menu can be displayed to the user on the display. Further details include, but are not limited to, the first-level control instruction menu may be a course selection control instruction menu, including playing courses, saving courses, displaying course details, reserving courses, and entering the second-level control instruction menu instructions. The control instruction menu can be a course evaluation menu, including praise, moderate evaluation, and negative evaluation instructions.

具体的，还包括在图形用户界面(GUI)对用户语音命令控制进行提示。具体的包括但不限于，在图形用户界面(GUI)显示可使用的语音的关键词、在图形用户界面(GUI)对用户语音指令进行补全(当用户动态动作完成度超过指定比例时)等。Specifically, it also includes prompting the user's voice command control on a graphical user interface (GUI). Specifically, it includes, but is not limited to, displaying available voice keywords on the graphical user interface (GUI), and completing the user's voice commands on the graphical user interface (GUI) (when the user's dynamic action completion degree exceeds a specified ratio), etc. .

具体的，还包括智能健身镜可以根据语音识别传感器采集到的用户语音对图形用户界面(GUI)进行控制时依赖其他传感器提供辅助判断。具体的包括但不限于通过动作识别传感器开启/中断/确认动作命令控制、通过面部识别传感器判断用户的正面或背面(仅当用户正面面对智能健身镜时开启动作命令控制)、通过面部识别传感器区分不同用户以加载自定义的语音命令控制指令等。Specifically, it also includes that the smart fitness mirror can rely on other sensors to provide auxiliary judgment when controlling the Graphical User Interface (GUI) according to the user voice collected by the voice recognition sensor. Specifically, it includes but is not limited to enabling/interrupting/confirming the motion command control through the motion recognition sensor, judging the front or back of the user through the facial recognition sensor (only when the user is facing the smart fitness mirror, turn on the motion command control), through the facial recognition sensor Differentiate between different users to load custom voice command control instructions and more.

具体的，语音命令控制的关键词可以是预先在智能健身镜中设定好的关键词，也可以是用户自己录制的关键词。语音命令控制执行的控制指令可以是预先在智能健身镜中设定好的指令，也可以是用户自己设置的自动化指令。语音命令控制的关键词和执行的控制指令可以是预先设置好的对应关系，也可以是用户自己设置的对应关系，可以是一对一的关系，也可以是多对一的关系。Specifically, the keywords controlled by the voice command may be keywords pre-set in the smart fitness mirror, or keywords recorded by the user. The control instructions controlled and executed by the voice command may be pre-set instructions in the smart fitness mirror, or may be automated instructions set by the user. The keywords controlled by the voice command and the executed control instructions may be a preset corresponding relationship, or may be a corresponding relationship set by the user, may be a one-to-one relationship, or may be a many-to-one relationship.

具体的，语音命令控制功能的开启或关闭可以由特定的图形用户界面(GUI)或锻炼页面决定，如在图形用户界面(GUI)的主页面默认开启语音命令控制功能而在其他页面默认关闭动作命令控制功能；语音命令控制功能的开启还可以通过识别唤醒词进行；语音命令控制功能的开启或关闭还可以根据其他传感器检测到的数据进行决定；语音命令控制功能的开启或关闭还可以是全部功能的开启/关闭或部分功能的开启/关闭。Specifically, the activation or deactivation of the voice command control function can be determined by a specific graphical user interface (GUI) or exercise page. For example, the voice command control function is enabled by default on the main page of the GUI and disabled by default on other pages. Command control function; the voice command control function can also be turned on by recognizing wake words; the voice command control function can be turned on or off according to the data detected by other sensors; the voice command control function can be turned on or off by all Function on/off or partial function on/off.

一个具体的智能健身镜可以根据图像识别传感器采集到的用户动作对图形用户界面(GUI)进行控制的示例，智能健身镜首先通过显示器120显示图形用户界面(GUI)的第一页面，用户通过唤醒词开启语音命令控制功能，第四页面上显示有代表语音命令控制功能开启图标，用户此时说出“进入切换页面”或“页面切换”的指令，智能健身镜通过显示器120显示用于切换页面显示的页面切换页面，切换页面上除突出显示的第一页面，还包括第一页面的上一页“第二页面”和第一页面的下一页“第三页面”，切换页面上还显示有代表语音命令控制功能开启图标，用户此时说出“上一页”或“往前翻一页”的指令，切换页面变为突出显示第二页面，此时用户说出“进入”或“确定”的指令，智能健身镜通过显示器120显示第二页面。A specific example in which the smart fitness mirror can control the graphical user interface (GUI) according to the user actions collected by the image recognition sensor. The smart fitness mirror first displays the first page of the graphical user interface (GUI) through the display 120, and the user wakes up by The word turns on the voice command control function, and the fourth page displays an icon representing the voice command control function on. The user speaks the command of "enter the switching page" or "page switching", and the smart fitness mirror is displayed on the display 120 for switching pages. The displayed page switches pages. In addition to the highlighted first page, the switch page also includes the previous page "second page" of the first page and the next page "third page" of the first page, and the switch page also displays There is an icon representing the opening of the voice command control function. At this time, the user speaks the command of "previous page" or "turn forward one page", and the switching page becomes the second page highlighted. At this time, the user speaks "enter" or "" Confirm” instruction, the smart fitness mirror displays the second page through the display 120.

一个具体的智能健身镜可以根据图像识别传感器采集到的用户动作对图形用户界面(GUI)进行控制的示例，智能健身镜首先通过显示器120显示图形用户界面(GUI)的第四页面，用户通过唤醒词开启语音命令控制功能，第四页面上显示有代表语音命令控制功能开启图标，用户此时说出“下一页”或“往后翻一页”的指令，智能健身镜通过显示器120显示由第四页面转入第五页面。A specific example in which the smart fitness mirror can control the graphical user interface (GUI) according to the user actions collected by the image recognition sensor. The smart fitness mirror first displays the fourth page of the graphical user interface (GUI) through the display 120, and the user wakes up by The word turns on the voice command control function, the fourth page displays an icon representing the voice command control function on, and the user speaks the command of "next page" or "turn one page back" at this time, and the smart fitness mirror is displayed on the display 120 by The fourth page goes to the fifth page.

一个具体的智能健身镜可以根据图像识别传感器采集到的用户动作对图形用户界面(GUI)进行控制的示例，智能健身镜首先通过显示器120显示图形用户界面(GUI)的第六页面，用户通过动作命令控制开启语音命令控制功能，第四页面上显示有代表语音命令控制功能开启图标，用户此时说出“开始训练”或“确定”的指令，智能健身镜通过显示器120显示由第六页面转入第七页面。A specific example in which the smart fitness mirror can control the graphical user interface (GUI) according to the user actions collected by the image recognition sensor. The smart fitness mirror first displays the sixth page of the graphical user interface (GUI) through the display 120. Command control to turn on the voice command control function, the fourth page displays an icon representing the voice command control function on, the user speaks the command of "start training" or "OK" at this time, and the smart fitness mirror is displayed on the display 120 from the sixth page to the sixth page. Go to page seven.

所述第一页面、第二页面、第三页面、第四页面和第五页面可以分别是唤醒页面、主页面、锻炼页面或其他页面，第六页面是用来展示锻炼卡片的页面，第七页面是锻炼页面。The first page, second page, third page, fourth page and fifth page can be wake-up page, main page, exercise page or other pages respectively, the sixth page is a page used to display exercise cards, and the seventh page is a page for displaying exercise cards. The page is the workout page.

触摸命令控制touch command control

智能健身镜可以根据触摸屏采集到的用户触控指令对图形用户界面(GUI)进行控制。The smart fitness mirror can control the Graphical User Interface (GUI) according to the user's touch commands collected by the touch screen.

多模态命令控制Multimodal Command Control

多模态命令控制包括组合控制、增强控制和冲突控制。Multimodal command control includes combined control, augmented control and conflict control.

其中组合控制包括但不限于多个动作的动作组合控制及多个关键词的语音组合控制，可以通过连续(间隔时间不超过阈值)的多个动作或关键词实现较为复杂的控制指令(如选中并收藏、选中并推荐给好友、截图并分享等)。The combination control includes but is not limited to the action combination control of multiple actions and the voice combination control of multiple keywords. More complex control instructions (such as selecting and favorite, select and recommend to friends, screenshot and share, etc.).

其中组合控制包括但不限于通过面部识别或场景识别对用户的动作指令控制或语音指令控制进行增强优化，进一步的包括但不限于用户的动作指令控制或语音指令控制为播放音乐时，识别到用户面部表情为不开心，为用户呈现舒缓心情的推荐结果同时对用户进行关心；用户的动作指令控制或语音指令控制为推荐课程时，识别到场景温度较低，减少冥想类课程的推荐。The combined control includes but is not limited to enhancing and optimizing the user's action command control or voice command control through facial recognition or scene recognition, and further includes but is not limited to the user's action command control or voice command control when music is played. When the facial expression is unhappy, it presents the user with a soothing recommendation result and cares for the user; when the user's action command control or voice command control is the recommended course, it recognizes that the scene temperature is low, and reduces the recommendation of meditation courses.

其中冲突控制包括但不限于当用户使用多个不同的控制方法同时对智能健身镜进行控制时，出现了多个冲突的控制指令，通过对不同的控制方法进行优先级判断，确定用户希望执行的控制指令并由智能健身镜进行执行。The conflict control includes, but is not limited to, when the user uses multiple different control methods to control the smart fitness mirror at the same time, there are multiple conflicting control instructions. Control commands and execute them by smart fitness mirrors.

控制指令Control instruction

控制指令包括但不限于系统唤醒指令、页面切换指令、页面交互指令和课程控制指令，用户可以通过这些控制指令实现对智能健身镜的控制。The control instructions include but are not limited to system wake-up instructions, page switching instructions, page interaction instructions, and course control instructions, and the user can control the smart fitness mirror through these control instructions.

其中系统唤醒指令包括将智能健身镜从待机状态唤醒到唤醒页面、主页面、锻炼页面、其他页面的指令和将智能健身镜从唤醒页面切换到主页面、锻炼页面、其他页面的指令。The system wake-up instructions include instructions to wake up the smart fitness mirror from the standby state to the wake-up page, home page, exercise page, and other pages, and instructions to switch the smart fitness mirror from the wake-up page to the main page, exercise page, and other pages.

其中页面切换指令是指用于控制智能健身镜在多个不同的页面中切换的指令，所述不同的页面可以是类型不同的页面，也可以是内容不同的页面。进一步的页面切换指令包括但不限于翻页指令(如下一页面/上一页面)和控制智能健身镜进入页面切换页面的指令。The page switching instruction refers to an instruction used to control the smart fitness mirror to switch among multiple different pages, and the different pages may be pages of different types or pages with different contents. Further page switching instructions include, but are not limited to, page turning instructions (eg, next page/previous page) and instructions for controlling the smart fitness mirror to enter the page switching page.

其中页面交互指令是指用于控制智能健身镜执行交互或输入的指令，所述页面交互指令包括但不限于控制交互指令和输入交互指令。进一步的所述控制交互指令是指控制图形用户界面(GUI)或图形用户界面(GUI)中的页面对象或交互对象实现其交互功能的指令，如确认/返回、加入收藏、加入计划、唤醒语音助手、音量控制、切换账号、切换模式、打开上下文菜单(contextual menu)等。进一步的所述输入交互指令是指在图形用户界面(GUI)或图形用户界面(GUI)中的页面对象或交互对象中输入信息的指令，如在进行课程评价时，每输入一次指令加一颗心；或在通过智能健身镜与其他用户进行聊天时，每输入一次指令切换一个预设表情/回复等The page interaction instruction refers to an instruction used to control the smart fitness mirror to perform interaction or input, and the page interaction instruction includes but is not limited to control interaction instruction and input interaction instruction. Further, the control interaction instruction refers to an instruction to control a graphical user interface (GUI) or a page object or an interactive object in the graphical user interface (GUI) to realize its interactive function, such as confirm/return, add to favorites, add to plan, wake up voice Assistant, volume control, switching accounts, switching modes, opening contextual menus, etc. Further, the input interactive instruction refers to the instruction of inputting information in the graphical user interface (GUI) or the page object or the interactive object in the graphical user interface (GUI). or when chatting with other users through the smart fitness mirror, switch a preset expression/reply every time you enter a command, etc.

其中课程控制指令是指用于控制智能健身镜在锻炼页面播放锻炼视频时的控制指令，主要用于对锻炼视频的播放进行控制。进一步的课程控制指令包括但不限于暂停/播放、上一节/下一节、背景音量控制、教练音量控制、课程评价等。The course control instruction refers to the control instruction used to control the smart fitness mirror to play the exercise video on the exercise page, and is mainly used to control the playback of the exercise video. Further course control commands include but are not limited to pause/play, previous/next section, background volume control, coach volume control, course evaluation, etc.

数据管理data management

智能健身镜的数据源包括但不限于传感器数据和后台数据，所述传感器数据和后台数据用于用户控制指令的输入和页面信息的展示。The data sources of the smart fitness mirror include but are not limited to sensor data and background data, and the sensor data and background data are used for input of user control instructions and presentation of page information.

其中传感器数据包括但不限于镜上传感器数据和外置传感器数据，所述镜上传感器数据包括但不限于图像识别传感器、红外识别传感器和语音识别传感器等安装在智能健身镜上的传感器检测到的数据。进一步具体的，图像识别传感器检测到的数据包括但不限于图像识别传感器检测到的图像和对图像处理识别后得到的数据，如用户动作数据、用户姿态数据、面部识别数据、用户表情数据和环境图像数据等。进一步具体的，红外识别传感器检测到的数据包括但不限于红外识别传感器检测到的热成像图及对热成像图处理识别后得到的数据，如用户热成像数据、用户锻炼部位数据、环境热成像数据等。进一步具体的，语音识别传感器检测到的数据包括但不限于音识别传感器检测到的语音数据及对语音数据处理识别后得到的数据，如语音控制指令、关键词指令、唤醒词指令等。所述镜上传感器数据包括但不限于惯性传感器数据、生理数据传感器数据(如心率带数据、体脂称数据等)等与智能健身镜通过无线连接的传感器检测到的数据。进一步具体的，惯性传感器检测到的数据包括但不限于惯性传感器检测到的速度/加速度数据和对速度/加速度数据处理识别后得到的数据，如用户动作数据等。进一步具体的，生理数据传感器检测到的数据包括但不限于生理数据传感器检测到的生理数据和对生理数据处理识别后得到的数据，如心率数据、血压数据、血氧饱和度数据等。The sensor data includes but is not limited to on-mirror sensor data and external sensor data, and the on-mirror sensor data includes but is not limited to image recognition sensors, infrared recognition sensors, voice recognition sensors and other sensors installed on smart fitness mirrors. data. More specifically, the data detected by the image recognition sensor includes but is not limited to the image detected by the image recognition sensor and the data obtained after image processing and recognition, such as user action data, user gesture data, facial recognition data, user expression data and environment. image data, etc. More specifically, the data detected by the infrared recognition sensor includes but is not limited to the thermal image detected by the infrared recognition sensor and the data obtained after processing and identifying the thermal image, such as user thermal image data, user exercise part data, environmental thermal image data etc. More specifically, the data detected by the voice recognition sensor includes, but is not limited to, the voice data detected by the voice recognition sensor and the data obtained after processing and recognizing the voice data, such as voice control instructions, keyword instructions, wake-up word instructions, and the like. The on-mirror sensor data includes, but is not limited to, inertial sensor data, physiological data sensor data (such as heart rate belt data, body fat scale data, etc.) and other data detected by sensors wirelessly connected to the smart fitness mirror. More specifically, the data detected by the inertial sensor includes, but is not limited to, the speed/acceleration data detected by the inertial sensor and the data obtained after processing and identifying the speed/acceleration data, such as user motion data. More specifically, the data detected by the physiological data sensor includes, but is not limited to, the physiological data detected by the physiological data sensor and the data obtained after processing and identifying the physiological data, such as heart rate data, blood pressure data, blood oxygen saturation data, and the like.

其中后台数据包括但不限于后台用户数据和后台其他数据，所述后台用户数据为智能健身镜和/或与智能健身镜连接的服务器存储的与用户直接关联的数据，所述后台用户数据包括但不限于用户生理数据、用户社交数据、用户日历数据和使用历史数据等。所述使用历史数据包括但不限于用户展示已完成的课程数据、收藏的课程数据、历史最佳成绩数据、课程过程中的生理数据和表现数据、上课的频率数据等。所述用户生理数据包括但不限于用户的身高、体重、性别、年龄、心率、血压、伤病史等。所述用户社交数据包括但不限于用户关注的人/好友相关的数据和用户加入的团体/兴趣组相关的数据，这些信息具体的可以是邀请用户完成指定课程的上课邀请数据、邀请用户进行一对一或多人的挑战邀请数据、用户间的排名及用户自己或其他用户上传的动态数据等。所述动态可以被同步到其他社交平台(如微信、微博、Facebook、Twitter等)。所述用户日历数据包括但不限于用户的日程信息数据、待办事项数据、课程计划数据等。所述后台其他数据为智能健身镜和/或与智能健身镜连接的服务器存储的与用户低关联度的数据，所述后台其他数据包括但不限于IP数据、环境数据、课程数据和第三方数据等。其中IP数据包括但不限于智能健身镜的IP地址数据、Mac地址数据等。其中环境数据包括但不限于智能健身镜的IP所在的区域的时间数据、定位数据、天气数据、温度数据、穿衣指数数据等。所述课程数据包括但不限于锻炼视频数据、教练名称、课程类型、课程时长、课程难度、课程标签和/或预计消耗等。所述课程类型包括但不限于是通过智能健身镜向用户展示HIIT、有氧舞、瑜伽、力量塑形、格斗训练、Barre、普拉提、拉伸、舞蹈、冥想、挑战等。所述课程标签包括但不限于是通过智能健身镜向用户展示是否为AI识别课程、是否为亲子课程、课程主要涉及的身体部位(如全身、手臂、腰腹、腿部等)及课程中需要使用的道具(如无器械、瑜伽垫、弹力圈、弹力带、哑铃等)等。所述第三方数据包括但不限于安装在智能健身镜上的第三方功能的数据。The background data includes but is not limited to background user data and other background data, the background user data is the data directly associated with the user stored by the smart fitness mirror and/or the server connected to the smart fitness mirror, and the background user data includes but It is not limited to user physiological data, user social data, user calendar data and usage history data, etc. The usage history data includes, but is not limited to, data of completed courses displayed by the user, collected course data, historical best score data, physiological data and performance data during the course, frequency data of classes, and the like. The user's physiological data includes, but is not limited to, the user's height, weight, gender, age, heart rate, blood pressure, injury history, and the like. The user social data includes, but is not limited to, data related to people/friends the user follows and data related to groups/interest groups that the user joins. These information may specifically be class invitation data for inviting the user to complete a specified course, and inviting the user to take a class. Challenge invitation data for one or more people, ranking among users, and dynamic data uploaded by users themselves or other users. The dynamics can be synchronized to other social platforms (eg, WeChat, Weibo, Facebook, Twitter, etc.). The user calendar data includes, but is not limited to, the user's schedule information data, to-do item data, lesson plan data, and the like. The other data in the background is the data with low relevance to the user stored by the smart fitness mirror and/or the server connected to the smart fitness mirror, and the other data in the background includes but is not limited to IP data, environmental data, course data and third-party data Wait. The IP data includes but is not limited to the IP address data, Mac address data, etc. of the smart fitness mirror. The environmental data includes but is not limited to time data, positioning data, weather data, temperature data, clothing index data, etc. of the area where the IP of the smart fitness mirror is located. The course data includes, but is not limited to, exercise video data, coach name, course type, course duration, course difficulty, course label and/or estimated consumption, etc. The class types include, but are not limited to, showing users HIIT, aerobic dance, yoga, strength shaping, fighting training, Barre, Pilates, stretching, dance, meditation, challenges, etc. through smart fitness mirrors. The course labels include, but are not limited to, whether it is an AI recognition course, whether it is a parent-child course, and the body parts (such as the whole body, arms, waist and abdomen, legs, etc.) Props used (such as no equipment, yoga mats, elastic rings, elastic bands, dumbbells, etc.), etc. The third-party data includes, but is not limited to, data of third-party functions installed on the smart fitness mirror.

数据存储data storage

智能健身镜还可以取决于所使用的存储空间量在智能健身镜和/或远程存储装置(例如，云服务)上本地存储用户信息。例如，使用很少存储空间的用户信息可以本地存储在智能健身镜上，这些用户信息包括但不限于用户的姓名、年龄、身高、体重和性别。另外，课程数据也可以存储在智能健身镜中以降低可能影响视频流质量的网络等待时间的影响。所存储的视频内容量可能受到智能健身镜的存储容量的限制。在一些配置中，视频内容可以仅每天或每周临时存储，或者取决于所使用的智能健身镜容量的百分比。使用大量存储空间的后台用户数据可以存储在远程存储装置上，这些用户信息包括但不限于生理数据，诸如用户的心率和消耗的卡路里以及在锻炼期间拍摄的用户的视频或骨骼点数据。智能健身镜可以检索该信息以用于后续分析和显示。The smart fitness mirror may also store user information locally on the smart fitness mirror and/or on a remote storage device (eg, a cloud service) depending on the amount of storage space used. For example, user information that uses little storage space can be stored locally on the smart fitness mirror, including but not limited to the user's name, age, height, weight, and gender. In addition, course data can also be stored in the smart fitness mirror to reduce the impact of network latency that can affect the quality of video streaming. The amount of video content stored may be limited by the storage capacity of the smart fitness mirror. In some configurations, the video content may only be temporarily stored on a daily or weekly basis, or as a percentage of the capacity of the smart fitness mirror used. Background user data that uses substantial storage space can be stored on the remote storage device, including but not limited to physiological data such as the user's heart rate and calories burned, and video or skeletal point data of the user captured during exercise. Smart fitness mirrors can retrieve this information for subsequent analysis and display.

可以通过各种方式保护(例如，加密)智能健身镜与远程存储装置之间的数据传输，以防止用户信息的非所需丢失或被盗。例如，低功耗蓝牙协议包括可以由利用该协议的装置使用的内置安全特征。然而，只有在与加密建立连接之前完成蓝牙绑定步骤时，才能使用这些安全特征。在一些情况下，可能未实现各种安全机制或各种安全机制可能发生故障，此时可以结合上述数据的分块规范来实现应用级安全性。例如，可以在前置消息的前导码之前应用消息的高级加密标准(AES)加密。在一些方面，低功耗蓝牙协议经由固件级别的内置安全特征执行类似过程，并且可以提供类似保护以防止人员阅读客户端与服务器之间的通信。Data transmission between the smart fitness mirror and the remote storage device can be secured (eg, encrypted) in various ways to prevent unwanted loss or theft of user information. For example, the Bluetooth Low Energy protocol includes built-in security features that can be used by devices utilizing the protocol. However, these security features can only be used if the Bluetooth bonding steps are completed before establishing a connection with encryption. In some cases, various security mechanisms may not be implemented or various security mechanisms may fail. In this case, application-level security can be implemented in combination with the above-mentioned data block specification. For example, Advanced Encryption Standard (AES) encryption of the message may be applied before the preamble of the preamble message. In some aspects, the Bluetooth Low Energy protocol performs a similar process via built-in security features at the firmware level, and can provide similar protection to prevent personnel from reading communications between the client and server.

当客户端与服务器断开连接时，可以从服务器装置上的服务录制中删除为客户端添加的用于读取/通知消息的GATT服务。这确保了没有任何连接被打开，并且系统不会意外地将信息泄露给邪恶的窥探者。这种连接终止可以由服务器或客户端触发，并依赖于低功耗蓝牙堆栈以向双方提供连接已关闭的通知。如果在初始连接设置中使用蓝牙绑定以提供固件级加密安全性，则可以将绑定信息存储在每个装置上，使得在客户端与服务器之间的后续连接之后不需要重复绑定。When the client is disconnected from the server, the GATT service added for the client to read/notify messages may be removed from the service recording on the server device. This ensures that no connections are opened and that the system does not accidentally leak information to evil snoopers. This connection termination can be triggered by the server or client and relies on the Bluetooth Low Energy stack to provide notification to both parties that the connection has been closed. If Bluetooth bonding is used in the initial connection setup to provide firmware-level cryptographic security, the bonding information can be stored on each device so that subsequent connections between the client and server do not require repeated bonding.

功能管理function management

智能健身镜还可以根据接收到的数据实现以下功能：用户识别、模式切换、注意力判断、身体/情绪评估、穿衣推荐。The smart fitness mirror can also realize the following functions according to the received data: user identification, mode switching, attention judgment, physical/emotional assessment, and clothing recommendation.

其中用户识别功能允许智能健身镜通过输入的数据识别出正在使用智能健身镜的用户，该功能主要通过以下方法实现：用户通过数据输入装置输入用户的特征数据，智能健身镜将用户输入的特征数据与数据库中的特征数据对比，根据对比结果确定正在使用智能健身镜的用户，根据识别结果允许智能健身镜对图形用户界面(GUI)进行调整，如调整字体大小、页面信息偏好等。其中数据输入装置包括但不限于镜上传感器数据和外置传感器，用户的特征数据包括但不限于用户的骨骼点、声纹、体重、静息心率等镜上传感器数据和外置传感器采集到或采集后经过处理得到的数据，所述数据库中记录了用户和特征数据的映射关系。进一步的，用户识别功能可以被用于一个主账号下的多个子账号的识别和切换，也可以用于多个主账号之间的识别和切换。在另一个示例中，用户识别功能允许智能健身镜通过输入的数据识别出正在使用智能健身镜的用户的特征，如男性用户、女性用户、老人、小孩，根据识别结果允许智能健身镜对图形用户界面(GUI)进行调整，如调整字体大小、页面信息等。The user identification function allows the smart fitness mirror to identify the user who is using the smart fitness mirror through the input data. This function is mainly realized by the following methods: the user inputs the user's characteristic data through the data input device, and the smart fitness mirror stores the user's input characteristic data. Compared with the feature data in the database, the user who is using the smart fitness mirror is determined according to the comparison result, and the smart fitness mirror is allowed to adjust the graphical user interface (GUI) according to the recognition result, such as adjusting the font size, page information preference, etc. The data input device includes but is not limited to on-mirror sensor data and external sensors, and the user's characteristic data includes but is not limited to the user's skeletal points, voiceprint, weight, resting heart rate and other on-mirror sensor data and external sensors collected or The data obtained by processing after collection, the mapping relationship between users and feature data is recorded in the database. Further, the user identification function can be used to identify and switch multiple sub-accounts under one main account, and can also be used to identify and switch between multiple main accounts. In another example, the user identification function allows the smart fitness mirror to identify the characteristics of the user who is using the smart fitness mirror through the input data, such as male users, female users, the elderly, and children, and allows the smart fitness mirror to identify the graphic user according to the recognition result. Interface (GUI) to adjust, such as adjusting font size, page information, etc.

其中模式切换功能允许智能健身镜在多个工作模式中切换以适应不同的工作场景。例如，智能健身镜被允许切换为青少年模式，青少年模式中，青少年账号与父母账号或设备主账号绑定，在青少年模式下，智能健身镜展示的页面信息会进行适应性的调整，如课程推荐会调整为优先呈现适合青少年的内容，如跳绳、体感游戏等，同时，在该模式下的实时情况和运动数据可以被同步至父母账号或设备主账号的智能终端。在另一个示例中，智能健身镜被允许切换为访客模式，访客模式中，智能健身镜不需要与账号绑定也可以工作，智能健身镜展示的页面信息也可以进行适应性的调整，如课程推荐会调整为优先呈现适合新手的内容，如难度标签为低的内容等，同时，即使智能健身镜与账号绑定，在该模式下的运动数据不会计入绑定的账号，不会影响绑定账号的训练计划或完课情况。The mode switching function allows the smart fitness mirror to switch between multiple working modes to adapt to different working scenarios. For example, the smart fitness mirror is allowed to switch to the youth mode. In the youth mode, the youth account is bound to the parent account or the main account of the device. In the youth mode, the page information displayed by the smart fitness mirror will be adjusted adaptively, such as course recommendation. It will be adjusted to give priority to the content suitable for teenagers, such as skipping rope, somatosensory games, etc. At the same time, the real-time situation and sports data in this mode can be synchronized to the smart terminal of the parent account or the main account of the device. In another example, the smart fitness mirror is allowed to switch to guest mode. In the guest mode, the smart fitness mirror can work without being bound to an account, and the page information displayed by the smart fitness mirror can also be adjusted adaptively, such as courses Recommendations will be adjusted to give priority to content suitable for beginners, such as content with a low difficulty label. At the same time, even if the smart fitness mirror is bound to an account, the exercise data in this mode will not be included in the bound account and will not affect the binding. The training plan or completion status of the account.

其中注意力判断允许智能健身镜确定设备工作时用户的注意力是否在智能健身镜上，以便于智能健身镜判断用户与镜子的交互意图同时可操作的激活或关闭传感器、其他数输入装置以及功能。注意力判断主要是根据用户的表现来判断用户是否希望使用智能健身镜，可以但不限于通过以下特征进行判断：距离判断(用户与智能健身镜的距离不超过阈值)、体态判断(用户是否正面面对智能健身镜)、动作判断(用户是否做出了指定的动作或锻炼常用的动作)、语义判断(用户语音的识别结果是否与特定关键词相关)等。Among them, the attention judgment allows the smart fitness mirror to determine whether the user's attention is on the smart fitness mirror when the device is working, so that the smart fitness mirror can judge the user's interaction intention with the mirror and can activate or deactivate the sensor, other digital input devices and functions at the same time. . Attention judgment is mainly based on the user's performance to judge whether the user wants to use the smart fitness mirror. It can be judged by but not limited to the following features: distance judgment (the distance between the user and the smart fitness mirror does not exceed the threshold), posture judgment (whether the user is positive or not) Facing the smart fitness mirror), action judgment (whether the user has made a specified action or common exercise), semantic judgment (whether the recognition result of the user's voice is related to a specific keyword), etc.

其中身体/情绪评估允许智能健身镜确定用户使用设备时的身体/情绪状况，以便于智能健身镜根据用户的身体/情绪状况可操作的调整页面信息或其他反馈。用户的身体/情绪状况可以通过生理数据传感器和图像识别传感器(面部识别)进行识别。当识别到用户情绪为不开心，为用户呈现舒缓心情的页面信息推荐结果同时在语音交互时对用户进行关心；进一步的一个实例，用户通过唤醒词唤醒智能健身镜的语音功能时，智能健身镜通过图像识别传感器(面部识别)识别到用户的表情较为沮丧，智能健身镜发起第一轮对话“hey，怎么啦你看起来不太高兴哦，要不要听一首歌来放松一下”。所述身体/情绪状况的识别可以通过对用户的年龄、身体姿态、表情等信息的识别，来判别用户的情绪、压力、疲劳等身体/情绪的状态；还可以通过生理数据传感器检测到的血压、心率数据来进行识别。The physical/emotional assessment allows the smart fitness mirror to determine the user's physical/emotional status when using the device, so that the smart fitness mirror can operably adjust page information or other feedback according to the user's physical/emotional status. The user's physical/emotional condition can be identified through physiological data sensors and image recognition sensors (facial recognition). When it is recognized that the user's mood is unhappy, the user is presented with a page information recommendation result that relieves the mood, and the user is concerned about the voice interaction; a further example, when the user wakes up the voice function of the smart fitness mirror through a wake-up word, the smart fitness mirror Through the image recognition sensor (facial recognition), the user's expression is more depressed, and the smart fitness mirror initiates the first round of dialogue "hey, what's the matter, you don't look very happy, do you want to listen to a song to relax". The identification of the physical/emotional state can be based on the identification of the user's age, body posture, expression and other information to determine the user's physical/emotional state such as emotion, stress, fatigue, etc.; the blood pressure detected by the physiological data sensor can also be used. , heart rate data to identify.

其中穿衣推荐允许智能健身镜基于特定条件向用户发起主动交互。智能健身镜根据采集到的时间数据、定位数据，结合用户信息及用户在智能健身镜前的动作判断用户是否需要智能健身镜发起主动交互。如，在用户早上第一次出现在智能健身镜前并转了一圈时，智能健身镜开启穿衣推荐功能，向用户推荐当前定位适合今天穿着的衣服。进一步的如，在用户周末出现在智能健身镜前并转了一圈时，智能健身镜开启穿衣推荐功能，向用户输出当前衣物的穿搭建议。上述功能还可以用于智能健身镜基于用户和外部信息进行主动互动的其他功能。Among them, the clothing recommendation allows the smart fitness mirror to initiate active interaction with the user based on specific conditions. The smart fitness mirror determines whether the user needs the smart fitness mirror to initiate active interaction based on the collected time data and positioning data, combined with user information and the user's actions in front of the smart fitness mirror. For example, when the user appears in front of the smart fitness mirror for the first time in the morning and makes a circle, the smart fitness mirror turns on the dressing recommendation function, and recommends the clothes that are currently positioned to suit the user today. For example, when the user appears in front of the smart fitness mirror and turns around on the weekend, the smart fitness mirror turns on the dressing recommendation function, and outputs the current clothing recommendations to the user. The above functions can also be used for other functions in which the smart fitness mirror actively interacts based on user and external information.

图1示出了智能健身镜的示例性表示。智能健身镜可以包括处理器110，其用于部分地控制智能健身镜中的各种子部件的操作并且管理流入/流出智能健身镜的数据流(例如，视频内容、来自教练或用户的音频、生物识别反馈分析)。智能健身镜可以包括用于显示视频内容的显示器120、用户可以与其交互并控制智能健身镜的图形用户界面(GUI)、生物识别反馈数据和/或其他视觉内容。传感器130可以耦合到处理器110以采集用户相关数据。天线140可以耦合到处理器110，以在智能健身镜与另一个装置(例如，遥控装置、生物识别传感器、无线路由器)之间提供数据传输。天线140可以包括多个发射器和接收器，每个发射器和接收器针对特定频率和/或无线标准(例如，蓝牙、802.11a、802.11b、802.11g、802.11n、802.11ac、2G、3G、4G、4G LTE、5G)而定制。放大器150可以耦合到处理器110以从处理器110接收音频信号以便通过左扬声器152和/或右扬声器154输出后续声音。Figure 1 shows an exemplary representation of a smart fitness mirror. The smart fitness mirror may include a processor 110 that controls, in part, the operation of various sub-components in the smart fitness mirror and manages the flow of data (eg, video content, audio from a trainer or user, biometric feedback analysis). The smart fitness mirror may include a display 120 for displaying video content, a graphical user interface (GUI) with which a user may interact and control the smart fitness mirror, biometric feedback data, and/or other visual content. Sensors 130 may be coupled to processor 110 to collect user-related data. Antenna 140 may be coupled to processor 110 to provide data transmission between the smart fitness mirror and another device (eg, a remote control, biometric sensor, wireless router). Antenna 140 may include multiple transmitters and receivers, each for a particular frequency and/or wireless standard (eg, Bluetooth, 802.11a, 802.11b, 802.11g, 802.11n, 802.11ac, 2G, 3G , 4G, 4G LTE, 5G). Amplifier 150 may be coupled to processor 110 to receive audio signals from processor 110 for output of subsequent sounds through left speaker 152 and/or right speaker 154 .

智能健身镜还可以包括图1中未示出的附加部件。例如，智能健身镜可以包括开关模式电源(SMPS)、开关以及板载存储器和存储装置(非易失性和/或易失性存储器)，包括但不限于硬盘驱动器(HDD)、固态驱动器(SDD)、快闪存储器、随机存取存储器(RAM)和安全数字(SD)卡。该板载存储器和/或存储装置可以用于存储用于智能健身镜的操作的固件和/或软件。如上所述，板载存储器和/或存储装置还可以用于(临时和/或永久)存储其他数据，包括但不限于视频内容、音频、用户视频、生物识别反馈数据和用户设置。在另一个例子中，智能健身镜可以包括安装和支撑智能健身镜的各种部件。The smart fitness mirror may also include additional components not shown in FIG. 1 . For example, a smart fitness mirror may include a switch mode power supply (SMPS), a switch, and onboard memory and storage (non-volatile and/or volatile memory), including but not limited to hard disk drives (HDDs), solid state drives (SDDs) ), flash memory, random access memory (RAM), and secure digital (SD) cards. The onboard memory and/or storage device may be used to store firmware and/or software for the operation of the smart fitness mirror. As noted above, onboard memory and/or storage may also be used to store other data (temporarily and/or permanently) including, but not limited to, video content, audio, user video, biometric feedback data, and user settings. In another example, the smart fitness mirror may include various components for mounting and supporting the smart fitness mirror.

天线140可以包括多个天线，每个天线用作接收器和/或发射器以与各种外部装置通信，这些外部装置诸如是用户的智能终端(例如，计算机、智能手机、平板计算机)、外置传感器(例如，心率带、惯性传感器、体脂称)和/或用于流式传输或播放视频内容的远程服务器或云服务器。再一次，天线140可以符合各种无线标准，包括但不限于蓝牙、802.11a、802.11b、802.11g、802.11n、802.11ac、2G、3G、4G、4G LTE和5G标准。Antenna 140 may include multiple antennas, each serving as a receiver and/or transmitter to communicate with various external devices, such as a user's smart terminal (eg, computer, smartphone, tablet), external sensors (eg, heart rate belts, inertial sensors, body fat scales) and/or remote or cloud servers for streaming or playback of video content. Again, the antenna 140 may conform to various wireless standards, including but not limited to Bluetooth, 802.11a, 802.11b, 802.11g, 802.11n, 802.11ac, 2G, 3G, 4G, 4G LTE, and 5G standards.

传感器130可以包括多个传感器，多个传感器可以分别是图像识别传感器(摄像头)、红外识别传感器或语音识别传感器(麦克风)中的一个或多个。The sensor 130 may include a plurality of sensors, which may be one or more of an image recognition sensor (camera), an infrared recognition sensor, or a voice recognition sensor (microphone), respectively.

智能健身镜中的图像识别传感器(摄像头)可以用于在用户进行活动(例如，锻炼)时获取用户的视频和/或静态图像。然后可以向教练分享用户的视频以允许教练在锻炼期间观察并向用户提供指导。还可以向其他智能健身镜的其他用户分享视频以进行比较或竞争。用户的视频也可以实时显示在显示120上或者存储以供以后回放。例如，通过提供用户与教练的视觉比较，用户的视频可以用于锻炼期间或之后的自我评估。存储的视频还可以允许用户在进行类似健身时评估他们随时间变化的进度或改进。Image recognition sensors (cameras) in smart fitness mirrors can be used to capture video and/or still images of the user while the user is performing an activity (eg, exercising). The user's video can then be shared with the coach to allow the coach to observe and provide guidance to the user during the workout. Videos can also be shared with other users of other smart fitness mirrors for comparison or competition. The user's video may also be displayed on display 120 in real-time or stored for later playback. For example, by providing a visual comparison of the user to the coach, the user's video can be used for self-assessment during or after a workout. Stored videos can also allow users to assess their progress or improvement over time while doing similar workouts.

图形用户界面(GUI)Graphical User Interface (GUI)

智能健身镜可以通过显示器120来显示图形用户界面(GUI)以促进用户与智能健身镜的交互。The smart fitness mirror may display a graphical user interface (GUI) through the display 120 to facilitate user interaction with the smart fitness mirror.

图形用户界面(GUI)包括但不限于唤醒页面、主页面、锻炼页面和其他页面等。Graphical User Interfaces (GUIs) include, but are not limited to, wakeup pages, home pages, exercise pages, and other pages, among others.

其中唤醒页面是指智能健身镜被唤醒时的初始页面，用于展示一些基础的页面信息，只有检测到用户进行指定操作时，智能健身镜被彻底唤醒进入主页面、锻炼页面或其他页面。The wake-up page refers to the initial page when the smart fitness mirror is awakened, which is used to display some basic page information. Only when the user performs a specified operation is detected, the smart fitness mirror is completely awakened to the main page, exercise page or other pages.

其中主页面展示至少一项页面信息，所述页面信息用于向用户展示数据、可操作的交互对象或交互引导。所述主页面用于向用户展示预设的页面信息或用户的自己设置的页面信息，同时用户可以通过在主页面上进行指定操作对智能健身镜进行控制，所述控制包括但不限于页面切换、功能切换或进入第三方接入的应用程序。The main page displays at least one item of page information, and the page information is used to display data, operable interactive objects or interactive guidance to the user. The main page is used to display the preset page information or the page information set by the user to the user. At the same time, the user can control the smart fitness mirror by performing specified operations on the main page, and the control includes but is not limited to page switching. , function switch, or enter a third-party access application.

其中锻炼页面展示用于播放锻炼视频指导用户进行锻炼的视频播放窗口，还可以展示课程播放信息(如课程总时长、已播放的时长等)、配件连接信息(如心率带连接信息等)和用户生理信息(如心率、血压、消耗的卡路里等)中的至少一种。锻炼页面可以由主页面进入或唤醒页面进入并退回到主页面或唤醒页面，也可以在智能健身镜开启时直接进入。The exercise page displays the video playback window used to play the exercise video to guide the user to exercise, and can also display the course playback information (such as the total course duration, the played duration, etc.), accessory connection information (such as heart rate belt connection information, etc.) and user information. At least one of physiological information (such as heart rate, blood pressure, calories consumed, etc.). The exercise page can be entered from the main page or the wake-up page and returned to the main page or the wake-up page, or it can be directly entered when the smart fitness mirror is turned on.

其中其他页面展示通过主页面进入的预安装的功能页面(如多媒体播放器页面、天气展示页面、饮食推荐页面等)或第三方接入的应用程序页面(如微信、抖音、喜马拉雅等)，其他页面一般由主页面进入或唤醒页面进入并退回到主页面或唤醒页面。Other pages display pre-installed function pages (such as multimedia player pages, weather display pages, diet recommendation pages, etc.) or application pages accessed by third parties (such as WeChat, Douyin, Himalaya, etc.) accessed through the main page, Other pages are generally entered from the main page or the wake-up page and return to the main page or the wake-up page.

页面信息page information

图形用户界面(GUI)可以通过页面对象展示页面信息，每个页面对象用于展示至少一种或一组页面信息，所述展示的页面信息可以是存储在智能健身镜本地的存储装置中的信息，也可以是智能健身镜通过通信接口从其他设备处接收到的信息，包括但不限于可切换展示的课程预览信息、用户信息、环境信息、多媒体信息、外部设备信息等。The graphical user interface (GUI) can display page information through page objects, each page object is used to display at least one or a group of page information, and the displayed page information can be the information stored in the local storage device of the smart fitness mirror , or information received by the smart fitness mirror from other devices through the communication interface, including but not limited to switchable and displayed course preview information, user information, environmental information, multimedia information, external device information, etc.

所述课程预览信息包括但不限于教练名称、课程类型、课程时长、课程难度、课程标签和/或预计消耗等。所述课程类型包括但不限于是通过智能健身镜向用户展示HIIT、有氧舞、瑜伽、力量塑形、格斗训练、Barre、普拉提、拉伸、舞蹈、冥想、挑战等。所述课程标签包括但不限于是通过智能健身镜向用户展示是否为AI识别课程、是否为亲子课程、课程主要涉及的身体部位(如全身、手臂、腰腹、腿部等)及课程中需要使用的道具(如无器械、瑜伽垫、弹力圈、弹力带、哑铃等)等。所述展示课程预览信息的页面对象也称为课程卡片。The course preview information includes, but is not limited to, instructor name, course type, course duration, course difficulty, course label, and/or estimated consumption, etc. The class types include, but are not limited to, showing users HIIT, aerobic dance, yoga, strength shaping, fighting training, Barre, Pilates, stretching, dance, meditation, challenges, etc. through smart fitness mirrors. The course labels include, but are not limited to, whether it is an AI recognition course, whether it is a parent-child course, and the body parts (such as the whole body, arms, waist and abdomen, legs, etc.) Props used (such as no equipment, yoga mats, elastic rings, elastic bands, dumbbells, etc.), etc. The page object displaying course preview information is also called a course card.

所述用户信息包括但不限于使用历史信息、生理信息、社交信息、日历信息等。所述使用历史信息包括但不限于是通过智能健身镜向用户展示已完成的课程、收藏的课程、历史最佳成绩、上课的频率等。所述生理信息包括但不限于是通过智能健身镜向用户展示身高、体重、性别、年龄、心率、血压、伤病史等。所述社交信息包括但不限于是通过智能健身镜向用户展示与用户关注的人/好友相关的信息，也可以是与用户加入的团体/兴趣组相关的信息，这些信息具体的可以是邀请用户完成指定课程的上课邀请、邀请用户进行一对一或多人的挑战邀请、用户间的排名及用户自己或其他用户上传的动态等。所述动态可以被同步到其他社交平台(如微信、微博、Facebook、Twitter等)。所述日历信息包括但不限于是通过智能健身镜向用户展示用户的日程信息、待办事项、课程计划等。所述展示用户信息的页面对象也称为用户卡片。The user information includes, but is not limited to, usage history information, physiological information, social information, calendar information, and the like. The usage history information includes, but is not limited to, displaying completed courses, favorite courses, historical best grades, frequency of classes, etc. to the user through the smart fitness mirror. The physiological information includes, but is not limited to, displaying height, weight, gender, age, heart rate, blood pressure, injury history, etc. to the user through the smart fitness mirror. The social information includes, but is not limited to, the information related to the people/friends that the user follows to show to the user through the smart fitness mirror, or the information related to the group/interest group that the user joins. Complete the class invitation for the specified course, invite users to one-on-one or multi-person challenge invitations, ranking among users, and updates uploaded by users themselves or other users. The dynamics can be synchronized to other social platforms (eg, WeChat, Weibo, Facebook, Twitter, etc.). The calendar information includes, but is not limited to, displaying the user's schedule information, to-do items, and lesson plans to the user through the smart fitness mirror. The page object displaying user information is also called a user card.

所述环境信息包括但不限于天气信息。所述天气信息包括但不限于是通过智能健身镜向用户展示天气预报、温度、湿度、穿衣推荐等。所述展示天气信息的页面对象也称为天气卡片。The environmental information includes but is not limited to weather information. The weather information includes, but is not limited to, displaying the weather forecast, temperature, humidity, clothing recommendations, etc. to the user through the smart fitness mirror. The page object displaying weather information is also called a weather card.

所述多媒体信息包括但不限于通过智能健身镜播放的音乐、视频的信息。所述展示多媒体信息的页面对象也称为多媒体卡片。The multimedia information includes, but is not limited to, music and video information played through the smart fitness mirror. The page object displaying multimedia information is also called a multimedia card.

所述外部设备信息包括但不限于通过智能健身镜展示与智能健身镜建立连接的其他健身装置的信息、其他智能家具的信息(如扫地机器人、智能音箱、智能网关等)。所述展示外部设备信息的页面对象也称为设备卡片。The external device information includes, but is not limited to, displaying information on other fitness devices connected to the smart fitness mirror through the smart fitness mirror, and information on other smart furniture (eg, sweeping robots, smart speakers, smart gateways, etc.). The page object displaying external device information is also called a device card.

图形用户界面(GUI)还可以通过交互对象展示页面信息，所述交互对象可以是图标、文字、高亮选择、其他可以提示用户进行交互的形式或其组合。所述交互对象用于向用户展示用户可以执行的控制或操作(如页面切换、选择、课程的暂停/播放等)。进一步的，所述交互对象还可以与图形用户界面(GUI)的页面或页面对象进行结合以展示不同的交互功能。A graphical user interface (GUI) can also display page information through interactive objects, which can be icons, text, highlighted selections, other forms that can prompt the user to interact, or a combination thereof. The interactive object is used to show the user controls or operations that the user can perform (such as page switching, selection, pause/play of a course, etc.). Further, the interactive object can also be combined with a page or page object of a graphical user interface (GUI) to display different interactive functions.

所述页面对象和交互对象除了可以固定在各个页面上，还可以采用在各个页面进行弹窗或滚动播放的方式进行展示。In addition to being fixed on each page, the page object and the interaction object can also be displayed in a pop-up window or scrolling playback mode on each page.

一些示例性的页面some example pages

方案一如图2、3、4、5-a、5-b所示，其中图2是第一主页面，图3是页面切换页面、图4、5-a、5-b分别是第二主页面、第三主页面和第四主页面。第一主页面通过多个页面对象展示多个页面信息，包括天气卡片、多媒体卡片和推荐的课程卡片等，同时第一主页面上还包括在右上角展示的用于进入页面切换页面的交互对象及与课程卡片组合的交互对象，用户可以通过执行与课程卡片组合的交互对象的交互进入第二主页面，也可以通过执行用于进入页面切换页面的交互对象的交互进入页面切换页面，页面切换页面包括第一主页面、第二主页面、第三主页面的预览图并突出显示其中一个页面，同时页面切换页面上方还展示了返回和确认进入突出显示页面的交互对象及与页面切换页面下方还展示了切换突出显示页面的交互对象。用户可以通过执行切换突出显示页面的交互对象的交互切换突出显示的页面，并通过确认进入突出显示页面的交互对象的交互进入选中的页面。当用户选中并进入第二主页面时，第二主页面包括多个课程卡片并突出显示其中一个课程卡片，同时第二主页面上方还展示了返回和确认进入突出显示课程卡片课程的交互对象及与页面切换页面下方还展示了切换突出显示课程卡片的交互对象，用户可以通过执行切换突出显示课程卡片的交互对象的交互切换突出显示的课程卡片，并通过执行确认进入突出显示课程卡片课程的交互对象的交互进入选中的课程卡片的锻炼页面。当用户选中并进入第三主页面时，第三主页面包括多个并列显示的页面对象，同时第三主页面还展示了选中的页面对象及进入选中的页面对象的交互对象，以及第三主页面下方还展示了切换页面对象的交互对象，用户可以执行通过切换页面对象的交互对象的交互切换显示的页面对象，并通过执行进入选中的页面对象的交互对象的交互进入选中的页面对象对应的页面。Scheme 1 is shown in Figures 2, 3, 4, 5-a, and 5-b, where Figure 2 is the first main page, Figure 3 is the page switching page, and Figures 4, 5-a, and 5-b are the second Main page, third main page and fourth main page. The first home page displays multiple page information through multiple page objects, including weather cards, multimedia cards, and recommended course cards, etc. At the same time, the first home page also includes interactive objects displayed in the upper right corner for entering the page switching page and the interactive object combined with the course card, the user can enter the second main page by executing the interaction with the interactive object combined with the course card, or enter the page switching page by executing the interaction of the interactive object used to enter the page switching page, page switching The page includes preview images of the first main page, the second main page, and the third main page, and highlights one of them. At the same time, the upper part of the page switching page also displays the interactive objects that return and confirm to enter the highlighted page and the lower part of the page switching page. Also shown is an interactive object that toggles the highlighted page. The user can switch the highlighted page by performing the interaction of switching the interactive object of the highlighted page, and enter the selected page by confirming the interaction of the interactive object entering the highlighted page. When the user selects and enters the second home page, the second home page includes multiple course cards and highlights one of the course cards. At the same time, the top of the second home page also displays the interactive objects for returning and confirming entering the highlighted course card course and The interactive object that toggles the highlighted course card is also displayed at the bottom of the page, and the user can switch the highlighted course card by executing the interaction of the interactive object that toggles the highlighted course card, and enter the highlighted course card course interaction by executing the confirmation The object's interaction goes to the exercise page of the selected lesson card. When the user selects and enters the third home page, the third home page includes a plurality of page objects displayed side by side, and the third home page also displays the selected page object and the interactive object entering the selected page object, as well as the third home page The bottom part of the page also shows the interactive object of switching page objects. The user can switch the displayed page object by performing the interaction of switching the interactive object of the page object, and enter the corresponding page object of the selected page object by executing the interaction of the interactive object entering the selected page object. page.

方案二如图6、7、8、9所示，其中图6是第五主页面，图7是第六主页面、图8、9分别是第七主页面、第八主页面。第五主页面通过多个页面对象展示多个页面信息，包括天气卡片、多媒体卡片和推荐的课程卡片等，同时第五主页面上还包括在下方展示的用于切换主页面的交互对象及与课程卡片组合的交互对象，用户可以通过执行与课程卡片组合的交互对象的交互进入第八主页面，也可以通过执行用于切换主页面的交互对象的交互切换显示的主页面。当用户选择切换显示的主页面为第六主页面时，第六主页面包括多个并列显示的页面对象，同时第六主页面还展示了选中的页面对象及进入选中的页面对象的交互对象，以及第六主页面下方还展示了用于切换主页面的交互对象，用户可以通过执行用于切换主页面的交互对象的交互切换显示的主页面，并通过执行进入选中的页面对象的交互对象的交互进入选中的页面对象对应的页面。当用户选择切换显示的主页面为第七主页面时，第七主页面包括多个并列显示的页面对象，同时第七主页面还展示了选中的页面对象及进入选中的页面对象的交互对象，以及第七主页面下方还展示了用于切换主页面的交互对象，用户可以通过执行用于切换主页面的交互对象的交互切换显示的主页面，并通过执行进入选中的页面对象的交互对象的交互进入选中的页面对象对应的页面。当用户选择进入选中的页面对象对应的页面为第八主页面时，第八主页面包括多个并列显示的课程卡片，同时第八主页面还展示了选中的课程卡片及进入选中的课程卡片的交互对象，以及第八主页面下方还展示了用于切换课程卡片的交互对象，用户可以通过执行用于切换课程卡片的交互对象的交互切换显示的课程卡片，并通过执行进入选中的课程卡片的交互对象的交互进入选中的课程卡片对应的锻炼页面。Scheme 2 is shown in Figures 6, 7, 8, and 9, wherein Figure 6 is the fifth main page, Figure 7 is the sixth main page, and Figures 8 and 9 are the seventh and eighth main pages, respectively. The fifth main page displays multiple page information through multiple page objects, including weather cards, multimedia cards, and recommended course cards, etc. At the same time, the fifth main page also includes the interactive objects displayed below for switching the main page and the The interaction object of the course card combination, the user can enter the eighth main page by performing interaction with the interaction object of the course card combination, and can also switch the displayed main page by performing the interaction of the interaction object for switching the main page. When the user chooses to switch the displayed main page to the sixth main page, the sixth main page includes a plurality of page objects displayed side by side, and at the same time, the sixth main page also displays the selected page object and the interactive object that enters the selected page object. And the bottom of the sixth main page also displays the interactive object used to switch the main page, the user can switch the displayed main page by executing the interaction of the interactive object used to switch the main page, and enter the interactive object of the selected page object by executing the Interactively enter the page corresponding to the selected page object. When the user chooses to switch the displayed main page to the seventh main page, the seventh main page includes a plurality of page objects displayed side by side, and the seventh main page also displays the selected page object and the interactive objects that enter the selected page object. And the bottom of the seventh main page also displays the interactive object used to switch the main page, the user can switch the displayed main page by executing the interaction of the interactive object used to switch the main page, and enter the interactive object of the selected page object by executing the Interactively enter the page corresponding to the selected page object. When the user chooses to enter the page corresponding to the selected page object is the eighth main page, the eighth main page includes a plurality of course cards displayed side by side, and the eighth main page also displays the selected course card and the information for entering the selected course card. The interactive object, and the interactive object used to switch course cards is also displayed at the bottom of the eighth main page. The user can switch the displayed course card by executing the interaction of the interactive object used to switch the course card, and enter the selected course card by executing the interactive object. The interaction of the interactive object enters the exercise page corresponding to the selected course card.

页面切换页面包括第一主页面、第二主页面、第三主页面的预览图并突出显示其中一个页面，同时页面切换页面上方还展示了返回和确认进入突出显示页面的交互对象及与页面切换页面下方还展示了切换突出显示页面的交互对象。用户可以通过执行切换突出显示页面的交互对象的交互切换突出显示的页面，并通过确认进入突出显示页面的交互对象的交互进入选中的页面。当用户选中并进入第二主页面时，第二主页面包括多个课程卡片并突出显示其中一个课程卡片，同时第二主页面上方还展示了返回和确认进入突出显示课程卡片课程的交互对象及与页面切换页面下方还展示了切换突出显示课程卡片的交互对象，用户可以通过执行切换突出显示课程卡片的交互对象的交互切换突出显示的课程卡片，并通过执行确认进入突出显示课程卡片课程的交互对象的交互进入选中的课程卡片的锻炼页面。当用户选中并进入第三主页面时，第三主页面包括多个并列显示的页面对象，同时第三主页面还展示了选中的页面对象及进入选中的页面对象的交互对象，以及第三主页面下方还展示了切换页面对象的交互对象，用户可以通过切换页面对象的交互对象的交互切换显示的页面对象，并通过执行进入选中的页面对象的交互对象的交互进入选中的页面对象对应的页面。The page switching page includes the preview images of the first main page, the second main page, and the third main page, and highlights one of them. At the same time, the top of the page switching page also displays the interactive objects that return and confirm to enter the highlighted page and switch with the page. The lower part of the page also shows the interactive object that toggles the highlighted page. The user can switch the highlighted page by performing the interaction of switching the interactive object of the highlighted page, and enter the selected page by confirming the interaction of the interactive object entering the highlighted page. When the user selects and enters the second home page, the second home page includes multiple course cards and highlights one of the course cards. At the same time, the top of the second home page also displays the interactive objects for returning and confirming entering the highlighted course card course and The interactive object that toggles the highlighted course card is also displayed at the bottom of the page, and the user can switch the highlighted course card by executing the interaction of the interactive object that toggles the highlighted course card, and enter the highlighted course card course interaction by executing the confirmation The object's interaction goes to the exercise page of the selected lesson card. When the user selects and enters the third home page, the third home page includes a plurality of page objects displayed side by side, and the third home page also displays the selected page object and the interactive object entering the selected page object, as well as the third home page The bottom part of the page also shows the interactive object for switching page objects. The user can switch the displayed page object through the interaction of the interactive object of the switching page object, and enter the page corresponding to the selected page object by executing the interaction of the interactive object entering the selected page object. .

一种电子装置，包括存储器、处理器以及存储在所述存储器中并可在所述处理器上运行的计算机程序，所述处理器执行所述计算机程序时实现所述多模态交互健身镜的步骤。An electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, when the processor executes the computer program, the multimodal interactive fitness mirror is realized step.

其中，所述处理器可以是中央处理器，还可以是其他通用处理器、数字信号处理器、专用集成电路、现成可编程门阵列或者其他可编程逻辑器件、分立门或者晶体管逻辑器件、分立硬件组件等。通用处理器可以是微处理器或者该处理器也可以是任何常规的处理器等。The processor may be a central processing unit, other general-purpose processors, digital signal processors, application-specific integrated circuits, off-the-shelf programmable gate arrays or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like.

所述存储器可用于存储所述计算机程序和/或模块，所述处理器通过运行或执行存储在所述存储器内的数据，实现发明中多模态交互健身镜的装置的各种功能。所述存储器可主要包括存储程序区和存储数据区，其中，存储程序区可存储操作系统、至少一个功能所需的应用程序(比如声音播放功能、图像播放功能等)等。此外，存储器可以包括高速随机存取存储器、还可以包括非易失性存储器，例如硬盘、内存、插接式硬盘，智能存储卡，安全数字卡，闪存卡、至少一个磁盘存储器件、闪存器件、或其他易失性固态存储器件。The memory can be used to store the computer program and/or module, and the processor implements various functions of the multimodal interactive fitness mirror device in the invention by running or executing the data stored in the memory. The memory may mainly include a stored program area and a stored data area, wherein the stored program area may store an operating system, an application program required for at least one function (such as a sound playback function, an image playback function, etc.) and the like. In addition, the memory may include high-speed random access memory, and may also include non-volatile memory such as hard disks, internal memory, plug-in hard disks, smart memory cards, secure digital cards, flash memory cards, at least one magnetic disk storage device, flash memory devices, or other volatile solid-state storage devices.

视频流传输方法Video streaming method

智能健身镜被配置为在显示面器120上显示来自视频制作工厂的视频内容。视频内容可以作为直播内容或预先录制好的录播内容流式传输。也可以将直播内容录制并存储在云服务器中，使得用户可以稍后请求和播放视频内容，因此成为录播内容。在进行视频流传输时，首先智能健身镜向API服务器发送指定课程的请求，API服务器返回的数据分为两部分，第一部分为指定课程的概览(包括课程图片及课程简介)；第二部分为课程视频的地址(URL)，然后智能健身镜根据收到的URL去OSS服务器上请求对应的课程视频。当课程为录播内容时，智能健身镜使用HLS协议进行视频播放，即课程视频的地址(URL)为对应课程的M3U8文件，然后智能健身镜的播放器根据M3U8文件的记载从OSS服务器上下载对应的课程视频切片。当课程为直播内容时，智能健身镜使用RTMP协议进行视频播放，即课程视频的地址(URL)为对应课程的RTMP直播地址，然后智能健身镜的播放器根据RTMP直播地址从OSS服务器上下载对应的课程视频直播数据。The smart fitness mirror is configured to display video content from a video production facility on the display panel 120 . Video content can be streamed as live content or as pre-recorded recordings. The live content can also be recorded and stored in the cloud server, so that the user can request and play the video content later, so it becomes the recorded content. When performing video streaming, firstly, the smart fitness mirror sends a request for a specified course to the API server. The data returned by the API server is divided into two parts. The first part is an overview of the specified course (including course pictures and course introduction); the second part is The address (URL) of the course video, and then the smart fitness mirror goes to the OSS server to request the corresponding course video according to the received URL. When the course is recorded and broadcasted, the smart fitness mirror uses the HLS protocol to play the video, that is, the address (URL) of the course video is the M3U8 file of the corresponding course, and then the smart fitness mirror player downloads it from the OSS server according to the record of the M3U8 file Corresponding course video slices. When the course is live content, the smart fitness mirror uses the RTMP protocol for video playback, that is, the address (URL) of the course video is the RTMP live broadcast address of the corresponding course, and then the smart fitness mirror player downloads the corresponding course from the OSS server according to the RTMP live broadcast address. The live video data of the course.

一种计算机可读存储介质，所述计算机可读存储介质存储有计算机程序，所述计算机程序被处理器执行时实现所述多模态交互健身镜的步骤。A computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of the multimodal interactive fitness mirror.

本发明实施例的计算机存储介质，可以采用一个或多个计算机可读的介质的任意组合。计算机可读介质可以是计算机可读信号介质或者计算机可读存储介质。计算机可读存储介质可以是但不限于电、磁、光、电磁、红外线、或半导体的系统、装置或器件，或者任意以上的组合。计算机可读存储介质的更具体的例子(非穷举的列表)包括：具有一个或多个导线的电连接、便携式计算机磁盘、硬盘、随机存取存储器(RAM)、只读存储器(ReadOnlyMemory，ROM)、可擦式可编程只读存储器((ErasableProgrammableReadOnlyMemory，EPROM)或闪存)、光纤、便携式紧凑磁盘只读存储器(CD-ROM)、光存储器件、磁存储器件、或者上述的任意合适的组合。在本文件中，计算机可读存储介质可以是任何包含或存储程序的有形介质，该程序可以被指令执行系统、装置或者器件使用或者与其结合使用。The computer storage medium in the embodiments of the present invention may adopt any combination of one or more computer-readable mediums. The computer-readable medium may be a computer-readable signal medium or a computer-readable storage medium. The computer readable storage medium can be, but is not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus or device, or a combination of any of the above. More specific examples (a non-exhaustive list) of computer readable storage media include: electrical connections with one or more wires, portable computer disks, hard disks, random access memory (RAM), read only memory (ROM) ), erasable programmable read only memory (Erasable Programmable Read Only Memory, EPROM) or flash memory), optical fiber, portable compact disk read only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the above. In this document, a computer-readable storage medium can be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, apparatus, or device.

一种动作识别方法，An action recognition method,

如图10所示，在上述实施例的基础上，本装置在交互过程，具体包括以下步骤：As shown in FIG. 10 , on the basis of the above embodiment, the interaction process of the device specifically includes the following steps:

S121健身镜采集区域内用户图像，得到用户的动作图像；S121 The fitness mirror collects the user image in the area, and obtains the user's action image;

S122根据动作图像生成用户的骨骼点信息，根据骨骼点信息建立坐标系；S122 generates skeleton point information of the user according to the action image, and establishes a coordinate system according to the skeleton point information;

S122.1骨骼点信息包括头顶骨骼点、盆骨骨骼点、左手骨骼点和右手骨骼点；S122.1 bone point information includes head bone point, pelvic bone point, left-hand bone point and right-hand bone point;

S122.2获取头顶骨骼点和盆骨骨骼点之间的直线距离，根据所述直线距离获取坐标系的单位长度；S122.2 Obtain the straight line distance between the top bone point and the pelvic bone point, and obtain the unit length of the coordinate system according to the straight line distance;

S122.21获取头顶骨骼点和盆骨骨骼点之间的直线距离，将直线距离的六分之一作为坐标系的单位长度，即坐标系的单位长度1的大小为直线距离的六分之一；S122.21 Obtain the straight-line distance between the top bone point and the pelvic bone point, and take one-sixth of the straight-line distance as the unit length of the coordinate system, that is, the size of the unit length 1 of the coordinate system is one-sixth of the straight-line distance ;

S122.3将头顶骨骼点和盆骨骨骼点之间所在直线作为纵坐标、盆骨骨骼点为原点，并以获取的单位长度构建直角坐标系；S122.3 Take the straight line between the top bone point and the pelvic bone point as the ordinate, the pelvic bone point as the origin, and construct a Cartesian coordinate system with the obtained unit length;

S123根据坐标系建立若干交互区域；S123 establishes several interactive areas according to the coordinate system;

在本实施例中，交互区域共有五个，分别为第一交互区域、第二交互区域、第三交互区域、第四交互区域和第五交互区域，如图4所示，第一交互区域为图3中的A区域，第二交互区域为图4的B区域，第三交互区域为图4的C区域，第四交互区域为图4的D区域，第五交互区域为图4的E区域。In this embodiment, there are five interaction areas, which are a first interaction area, a second interaction area, a third interaction area, a fourth interaction area, and a fifth interaction area. As shown in FIG. 4 , the first interaction area is In area A in FIG. 3 , the second interactive area is area B in FIG. 4 , the third interactive area is area C in FIG. 4 , the fourth interactive area is area D in FIG. 4 , and the fifth interactive area is area E in FIG. 4 .

其中第一交互区域为以头顶骨骼点为中心，纵向5个单位长度，横向6个单位长度作为第一交互区域，即第一交互区域的长度为12个单位长度，高度为10个单位长度。第一交互区域为以头顶骨骼点为中心，纵向向上5个单位长度为上边，纵向向下5个单位长度为下边；以头顶骨骼点为中心，横向向左6个单位长度为左边，横向向右6个单位长度为右边。第一交互区域预设的控制指令为确定A，即当交互姿势位于第一交互区域时，输出第一交互区域对应的控制指令，即输出确定A的控制指令。The first interaction area is centered on the top of the head bone point, with 5 unit lengths in the vertical direction and 6 unit lengths in the horizontal direction as the first interaction area, that is, the length of the first interaction area is 12 unit lengths, and the height is 10 unit lengths. The first interactive area is centered on the top of the head bone point, 5 units of length vertically upwards is the upper side, and 5 units of length vertically downwards is the bottom side; with the top of the head bone point as the center, 6 units of length horizontally to the left are the left side, and the horizontal direction is 6 units. The right 6 unit length is the right side. The preset control instruction of the first interaction area is to determine A, that is, when the interactive gesture is located in the first interaction area, the control instruction corresponding to the first interaction area is output, that is, the control instruction to determine A is output.

第二交互区域的控制指令为向左，第三交互区域的控制指令为向右，第二交互区域为以盆骨骨骼点为起始点，纵向向上2个单位长度为上边，纵向向下10个单位长度为下边，以头顶骨骼点为起始点，横向向左3个单位长度为右边，横向向左16个单位长度为左边构成的长度为13个单位长度，高度为12个单位长度的区域。The control command of the second interactive area is to the left, the control command of the third interactive area is to the right, the second interactive area is based on the pelvic bone point as the starting point, 2 units of length up in the longitudinal direction are the upper side, and 10 in the longitudinal direction. The unit length is the bottom side, the top of the head is the starting point, 3 units of length to the left sideways are the right side, and 16 units of length to the left side are the left side, forming an area with a length of 13 units of length and a height of 12 units of length.

第三交互区域为以盆骨骨骼点为起始点，纵向向上2个单位长度为上边，纵向向下10个单位长度为下边，以头顶骨骼点为起始点，横向向右3个单位长度为左边，横向向右16个单位长度为右边构成的长度为13个单位长度，高度为12个单位长度的区域。The third interactive area is the starting point with the pelvic bone point as the starting point, 2 units of length up in the vertical direction as the upper side, 10 units in the longitudinal direction down as the lower side, with the bone point on the top of the head as the starting point, and 3 units in the horizontal direction to the right as the left side , 16 units horizontally to the right and the right side constitutes an area with a length of 13 units and a height of 12 units.

第四交互区域的控制指令为确定D，第五交互区域的控制指令为确定E，第四交互区域为以头顶骨骼点为起始点，纵向向上4个单位长度为上边，纵向向下5个单位长度为下边，以头顶骨骼点为起始点，横向向左7个单位长度为右边，横向向左16个单位长度为左边构成的长度为11个单位长度，高度为9个单位长度的区域。The control command of the fourth interactive area is OK D, the control command of the fifth interactive area is OK E, the fourth interactive area is the starting point of the top of the head bone point, the length of 4 units in the vertical direction is the upper side, and the length is 5 units in the vertical direction. The length is the bottom, the top of the head is the starting point, 7 units to the left laterally are the right, and 16 units are laterally left to the left. The length of the area is 11 units and the height is 9 units.

第五交互区域为以头顶骨骼点为起始点，纵向向上4个单位长度为上边，纵向向下5个单位长度为下边，以头顶骨骼点为起始点，横向向右7个单位长度为左边，横向向右16个单位长度为右边构成的长度为11个单位长度，高度为9个单位长度的区域。The fifth interaction area is the starting point of the top of the head bone point, 4 units of length vertically upwards is the upper side, 5 unit lengths of the vertical downward direction are the bottom side, starting from the top of the head bone point, 7 units of length horizontally to the right is the left side, The length of 16 units to the right in the horizontal direction is an area of 11 units in length and 9 units in height on the right side.

S124根据骨骼点信息识别交互姿势；S124 recognizes the interactive posture according to the skeleton point information;

S124.1获取左手骨骼点和右手骨骼点之间的距离；S124.1 Get the distance between the left-hand bone point and the right-hand bone point;

S124.2将左手骨骼点和右手骨骼点之间的距离与阈值T对比，若左手骨骼点和右手骨骼点之间的距离小于或等于阈值T，则识别到交互姿势，在本实施例中阈值T为15cm，若左手骨骼点和右手骨骼点之间的距离小于或等于阈值T，则判断用户做出交互姿势，识别到交互姿势，例如在本实施例中交互姿势为击掌，即用户做出击掌动作；若左手骨骼点和右手骨骼点之间的距离大于阈值T，则用户未做出击掌动作；在本实施例中具体的交互姿势只做举例说明并不做限定。S124.2 Compare the distance between the left-hand skeleton point and the right-hand skeleton point with the threshold value T, if the distance between the left-hand skeleton point and the right-hand skeleton point is less than or equal to the threshold value T, the interactive posture is recognized. In this embodiment, the threshold value T is 15cm. If the distance between the left-hand bone point and the right-hand bone point is less than or equal to the threshold T, it is judged that the user has made an interactive gesture, and the interactive gesture is recognized. For example, in this embodiment, the interactive gesture is a high five, that is, the user makes a High-five action; if the distance between the left-hand skeleton point and the right-hand skeleton point is greater than the threshold T, the user does not perform a high-five action; the specific interaction posture in this embodiment is only given as an example and is not limited.

S124.3用户做出交互姿势，则获取左手骨骼点和右手骨骼点之间的中间点坐标；S124.3 When the user makes an interactive gesture, the coordinates of the middle point between the left-hand skeleton point and the right-hand skeleton point are obtained;

S125判断中间点坐标是否位于交互区域中；S125 judges whether the coordinates of the intermediate point are located in the interaction area;

S125.1判断中间点坐标是否位于交互区域中；S125.1 Determine whether the coordinates of the intermediate point are located in the interactive area;

S125.11若中间点坐标位于其中一个交互区域中时，则该交互区域为激活状态，输出该交互区域对应的控制指令；其他交互区域为未激活状态；S125.11 If the coordinates of the intermediate point are located in one of the interactive areas, the interactive area is in an active state, and the control command corresponding to the interactive area is output; other interactive areas are in an inactive state;

S125.12输出该交互区域对应的控制指令的同时，该交互区域为激活状态，其余交互区域为未激活状态；激活状态的交互区域面积增大，未激活状态的交互区域面积缩小。When S125.12 outputs the control instruction corresponding to the interaction area, the interaction area is in an activated state, and the other interaction areas are in an inactive state; the area of the interaction area in the activated state increases, and the area of the interaction area in the inactive state decreases.

在本实施例中，激活状态的交互区域的长度增大1.5个单位长度，高度增大0.5个单位长度，具体的，在本实施例中，激活状态的交互区域的长度向左右两侧依次增大即外扩0.75个单位长度，高度向上下两端依次增大即外扩0.25个单位长度。未激活状态的交互区域长度缩小2个单位长度，高度缩小1个单位长度。具体的，未激活状态的交互区域长度向左右两侧依次缩小即内缩1个单位长度，高度向上下两端缩小即内缩0.5个单位长度。In this embodiment, the length of the interactive area in the active state is increased by 1.5 unit length, and the height is increased by 0.5 unit length. Specifically, in this embodiment, the length of the interactive area in the active state increases sequentially to the left and right sides. If it is large, it will expand by 0.75 unit length, and the height will increase in turn at the upper and lower ends, that is, it will expand by 0.25 unit length. The length of the interactive area in the inactive state is reduced by 2 units, and the height is reduced by 1 unit. Specifically, the length of the interaction area in the inactive state is reduced in turn toward the left and right sides, that is, 1 unit length, and the height is reduced at the upper and lower ends, that is, 0.5 unit length.

在未输出该交互区域对应的控制指令时，交互区域的面积不改变，输出控制指令后，交互区域的面积调整，执行控制指令后，交互区域的面积恢复到原始状态。When the control command corresponding to the interactive area is not output, the area of the interactive area does not change. After the control command is output, the area of the interactive area is adjusted. After the control command is executed, the area of the interactive area returns to the original state.

当用户在第一交互区域中做出交互姿势后，健身镜识别到交互姿势，输出第一交互区域对应的控制指令，第一交互区域的面积扩大，即上边纵向向上增大0.25个单位长度，下边纵向向下增大0.25个单位长度，左边横向向左增大0.75个单位长度，右边横向向右增大0.75个单位长度。When the user makes an interactive gesture in the first interactive area, the fitness mirror recognizes the interactive gesture and outputs the control command corresponding to the first interactive area. The bottom is increased by 0.25 unit length vertically and downward, the left side is increased by 0.75 unit length horizontally to the left, and the right side is increased by 0.75 unit length horizontally and rightward.

本文描述的所有参数、尺寸、材料和配置旨在是示例性的，并且实际参数、尺寸、材料和/或配置将取决于使用本发明教导的一个或多个特定应用。应当理解，前述实施方案主要通过举例方式呈现，并且在所附权利要求及其等同物的范围内，本发明的实施方案可以不同于具体描述和要求保护的方式来实践。本公开文本的发明实施方案涉及本文描述的每个单独的特征、系统、物品、材料、套件和/或方法。All parameters, dimensions, materials and configurations described herein are intended to be exemplary, and actual parameters, dimensions, materials and/or configurations will depend on the particular application or applications in which the teachings of the present invention are used. It is to be understood that the foregoing embodiments have been presented primarily by way of example, and that within the scope of the appended claims and their equivalents, the embodiments of the invention may be practiced otherwise than as specifically described and claimed. Inventive embodiments of the present disclosure are directed to each individual feature, system, article, material, kit, and/or method described herein.

另外，如果此类特征、系统、物品、材料，套件和/或方法不相互矛盾，则两种或更多种这样的特征、系统、物品、材料、套件和/或方法的任何组合包括在本公开文本的发明范围。在不脱离本公开文本的精神的情况下，可以在示例性实施方式的相应元件的设计、操作条件和布置中进行其他替换、修改、改变和省略。数值范围的使用不排除落在范围之外的以相同方式实现相同功能以产生相同结果的等同物。Additionally, any combination of two or more such features, systems, articles, materials, kits and/or methods is included herein to the extent that such features, systems, articles, materials, kits and/or methods are not mutually inconsistent Inventive scope of the disclosure. Other substitutions, modifications, changes and omissions may be made in the design, operating conditions and arrangement of corresponding elements of the exemplary embodiments without departing from the spirit of the present disclosure. The use of numerical ranges does not exclude equivalents falling outside the range that perform the same function and produce the same results.

可以多种方式实现上述实施方案。例如，可以使用硬件、软件或其组合来实现实施方案。当在软件中实现时，软件代码可以在合适的处理器或处理器集合上执行，而无论是在单个计算机中提供还是分布在多个计算机之间。The above-described embodiments can be implemented in various ways. For example, embodiments may be implemented using hardware, software, or a combination thereof. When implemented in software, the software code may be executed on a suitable processor or collection of processors, whether provided in a single computer or distributed among multiple computers.

此外，计算机可以被体现为多种形式中的任何一种，诸如机架式计算机、台式计算机、膝上型计算机或平板计算机。另外，计算机可以嵌入通常不被视为计算机但具有合适处理能力的装置中，该装置包括个人数字助理(PDA)、智能手机或任何其他合适的便携式或固定电子装置。Furthermore, a computer may be embodied in any of a variety of forms, such as a rack computer, desktop computer, laptop computer, or tablet computer. Additionally, a computer may be embedded in a device not generally considered a computer but with suitable processing capabilities, including a personal digital assistant (PDA), a smartphone, or any other suitable portable or fixed electronic device.

此外，计算机可以具有一个或多个输入和输出装置。除其他之外，这些装置还可以用于呈现用户界面。可以用于提供用户界面的输出装置的例子包括用于输出的视觉呈现的打印机或显示屏和用于输出的可听呈现的扬声器或其他发声装置。可以用于用户界面的输入装置的例子包括键盘和指示装置，诸如鼠标、触摸板和数字化平板计算机。作为另一个例子，计算机可以通过语音识别或其他可听格式接收输入信息。Additionally, a computer may have one or more input and output devices. Among other things, these means may also be used to present user interfaces. Examples of output devices that may be used to provide a user interface include a printer or display screen for visual presentation of output and speakers or other sound producing devices for audible presentation of output. Examples of input devices that can be used for a user interface include keyboards and pointing devices, such as mice, touch pads, and digitizing tablets. As another example, a computer may receive input information through speech recognition or other audible format.

此类计算机可以通过一种或多种网络以合适形式互连，这些网络包括局域网或广域网，诸如企业网、智能网(IN)或因特网。此类网络可以基于合适的技术，可以根据合适的协议进行操作，并且可以包括无线网络、有线网络或光纤网络。Such computers may be interconnected in suitable form by one or more networks, including local or wide area networks, such as an enterprise network, an Intelligent Network (IN), or the Internet. Such networks may be based on suitable technologies, may operate according to suitable protocols, and may include wireless, wired, or fiber optic networks.

本文概述的各种方法或过程可以被编码为可在采用各种操作系统或平台中的任何一者的一个或多个处理器上执行的软件。另外，可以使用许多合适的编程语言和/或编程或脚本工具中的任何一种来编写这种软件，并且还可以将这种软件编译为在框架或虚拟机上执行的可执行机器语言代码或中间代码。一些实施方式可以具体地采用特定操作系统或平台以及特定编程语言和/或脚本工具中的一者或多者来促进执行。The various methods or processes outlined herein can be encoded as software executable on one or more processors employing any of a variety of operating systems or platforms. Additionally, such software may be written using any of a number of suitable programming languages and/or programming or scripting tools, and may also be compiled into executable machine language code for execution on a framework or virtual machine or Intermediate code. Some embodiments may specifically employ one or more of a particular operating system or platform and a particular programming language and/or scripting tool to facilitate execution.

此外，各种发明构思可以被体现为一种或多种方法，其中已经提供了这些方法的至少一个例子。作为该方法的一部分执行的动作在一些情况下可以以不同方式排序。因此，在一些发明实施方式中，给定方法的相应动作可以按与具体示出的顺序不同的顺序执行，其可以包括同时执行一些动作(即使此类动作在示例性实施方案中被示为顺序动作)。Furthermore, various inventive concepts may be embodied in one or more methods, at least one example of which has been provided. The actions performed as part of the method may in some cases be ordered differently. Thus, in some inventive embodiments, the corresponding acts of a given method may be performed in an order different from that specifically shown, which may include performing some acts concurrently (even though such acts are shown as sequential in exemplary embodiments) action).

本文提及的所有出版物、专利申请、专利以及其他参考文献都通过引用整体并入。All publications, patent applications, patents, and other references mentioned herein are incorporated by reference in their entirety.

如本文定义和使用的所有定义应被理解为控制字典定义、通过引用并入的文献中的定义和/或所定义术语的普通含义。All definitions, as defined and used herein, should be understood to control over dictionary definitions, definitions in documents incorporated by reference, and/or ordinary meanings of the defined terms.

除非明确指示相互矛盾，否则本说明书和权利要求书中使用的不定冠词“一”和“一个”应被理解为表示“至少一个”。The indefinite articles "a" and "an" as used in this specification and claims should be construed to mean "at least one" unless an express indication of conflict is made.

本说明书和权利要求书中使用的短语“和/或”应被理解为表示如此结合的“一个或两个”元件，即，在一些情况下结合存在并且在其他情况下分离存在的元件。用“和/或”列出的多个元件应以相同方式解释，即，“一个或多个”元件如此结合。除由“和/或”子句具体表示的元件之外，可以可选地存在其他元件，而无论是与具体表示的那些元件相关还是不相关。因此，作为非限制性例子，当与诸如“包括”之类的开放式语言结合使用时，对“A和/或B”的引用可以在一个实施方案中仅指代A(可选地包括除B之外的元件)；在另一个实施方案中，仅指代B(可选地包括除A之外的元件)；在又一个实施方案中，指代A和B两者(可选地包括其他元件)；等等。As used in this specification and in the claims, the phrase "and/or" should be understood to mean "one or both" of the elements so conjoined, that is, in some instances conjoined and in other instances present separately. Multiple elements listed with "and/or" should be construed in the same fashion, ie, "one or more" of the elements are so conjoined. Other elements may optionally be present other than the elements specifically identified by the "and/or" clause, whether related or unrelated to those elements specifically identified. Thus, as a non-limiting example, when used in conjunction with open-ended language such as "includes," a reference to "A and/or B" may in one embodiment refer to only A (optionally including in addition to elements other than B); in another embodiment, refer to B only (optionally including elements other than A); in yet another embodiment, refer to both A and B (optionally including other components); etc.

如本说明书和权利要求书中所用的，“或”应被理解为与如上所定义的“和/或”具有相同含义。例如，当分隔列表中的项目时，“或”或“和/或”应被解释为包含性的，即，包括至少一个，但是也包括许多元件或元件列表中的一个以上元件以及(可选地)附加的未列出项目。只有明确指示相互矛盾，否则诸如“只有一个”或“恰好一个”或者在权利要求中使用时“由...组成”将指代恰好包括许多元件或元件列表中的一个元件。一般而言，如本文中所使用的术语“或”之后有诸如“两者之一”、“中的一个”、“中的仅一个”或“中的恰好一个”之类的排他性术语时仅应被解释为指示排他性备选方案(即，“一个或另一个但不是两个”)。“基本上由...组成”在权利要求中使用时它的普通意义如同在专利法领域中使用的那样。As used in this specification and the claims, "or" should be understood to have the same meaning as "and/or" as defined above. For example, when separating items in a list, "or" or "and/or" should be construed as inclusive, that is, including at least one, but also a number of elements or more than one element of a list of elements and (optionally ground) additional unlisted items. Such terms as "only one" or "exactly one" or "consisting of" as used in the claims will refer to the inclusion of exactly one element of a plurality of elements or a list of elements, unless expressly indicated to contradict each other. In general, the term "or" as used herein is followed by an exclusive term such as "either", "one of", "only one of" or "exactly one of" only when It should be construed to indicate the exclusive alternative (ie, "one or the other but not both"). "Consisting essentially of" is used in the claims in its ordinary meaning as it is used in the field of patent law.

如本说明书和权利要求书中所使用的，关于一个或多个元件的列表，短语“至少一个”应被理解为表示选自元件列表中的任何一个或多个元件的至少一个元件，但不一定包括元件列表中具体列出的每个元件中的至少一个元件，并且不排除元件列表中元件的任何组合。该定义还允许可选地存在除在短语“至少一个”所指代的元件列表内具体表示的元件之外的元件，而无论是与具体表示的那些元件相关还是不相关。因此，作为非限制性例子，“A和B中的至少一者”(或等同地，“A或B中的至少一者”，或等效地“A和/或B中的至少一者”)在一个实施方案中可以指代至少一个A，可选地包括一个以上A，而不存在B(并且可选地包括除B之外的元件)；在另一个实施方案中，指代至少一个B，可选地包括一个以上B，而不存在A(并且可选地包括除A之外的元件)；在又一个实施方案中，指代至少一个A，可选地包括一个以上A和至少一个B，可选地包括一个以上B(和可选地包括其他元件)；等等。As used in this specification and the claims, with respect to a list of one or more elements, the phrase "at least one" should be understood to mean at least one element selected from any one or more of the elements in the list of elements, but not At least one of each of the elements specifically listed in the list of elements must be included, and any combination of elements in the list of elements is not excluded. This definition also allows that elements may optionally be present other than the elements specifically identified within the list of elements to which the phrase "at least one" refers, whether related or unrelated to those elements specifically identified. Thus, by way of non-limiting example, "at least one of A and B" (or equivalently, "at least one of A or B", or equivalently "at least one of A and/or B" ) may in one embodiment refer to at least one A, optionally including more than one A, without the presence of B (and optionally including elements other than B); in another embodiment, to refer to at least one B, optionally including more than one B, without A (and optionally including elements other than A); in yet another embodiment, refers to at least one A, optionally including more than one A and at least one A B, optionally including more than one B (and optionally other elements); and so on.

在权利要求以及上面的说明书中，诸如“包括”、“包含”、“携带”、“具有”、“含有”、“涉及”、“持有”、“由...组成”等之类的所有过渡短语应被理解为开放式的，即，包括但不限于。In the claims and the description above, expressions such as "comprising," "comprising," "carrying," "having," "containing," "involving," "holding," "consisting of," etc. All transitional phrases should be construed as open ended, ie, including but not limited to.

尽管已描述了本发明的优选实施例，但本领域内的技术人员一旦得知了基本创造性概念，则可对这些实施例作出另外的变更和修改。所以，所附权利要求意欲解释为包括优选实施例以及落入本发明范围的所有变更和修改。Although preferred embodiments of the present invention have been described, additional changes and modifications to these embodiments may occur to those skilled in the art once the basic inventive concepts are known. Therefore, the appended claims are intended to be construed to include the preferred embodiment and all changes and modifications that fall within the scope of the present invention.

显然，本领域的技术人员可以对本发明进行各种改动和变型而不脱离本发明的精神和范围。这样，倘若本发明的这些修改和变型属于本发明权利要求及其等同技术的范围之内，则本发明也意图包含这些改动和变型在内。It will be apparent to those skilled in the art that various modifications and variations can be made in the present invention without departing from the spirit and scope of the invention. Thus, provided that these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include these modifications and variations.

Claims (13)

1. An interactive intelligent body-building mirror device is characterized by comprising

A display for displaying a user interface having at least one interactive object;

a mirror to show a mirror image of a user on the mirror, the display to display the user interface to the user through the mirror;

the sensor is used for collecting user action data of a user;

a processor, coupled to the display and the sensor, for generating a control instruction based on user action data collected by the sensor, the control instruction controlling the user interface to complete a specified operation;

the processor is further used for acquiring a motion track of a user action according to the user action data acquired by the sensor, and controlling a floating icon to be displayed on a user interface according to the motion track, wherein the floating icon is used for indicating the position indicated by the user action in the user interface.

2. The interactive intelligent fitness mirror device of claim 1, wherein the interactive objects of the user interface comprise a floating icon wake-up interaction, and when the processor determines that the motion indicated by the user motion data collected by the sensor matches a floating icon wake-up motion corresponding to a preset floating icon wake-up interaction, the processor enables the floating icon to move with the motion of the user.

3. The interactive intelligent fitness mirror device of claim 1 or 2, wherein the processor turns off the floating icon function so that the floating icon no longer follows the user's motion after the processor determines that the user has completed a selection operation at the current user interface.

4. The interactive intelligent fitness mirror device of claim 1 or 2, wherein the processor executes the operation corresponding to the user interface object when the processor determines that the floating icon is moved to the position of the user interface object in the user interface according to the user motion data and determines that the user has performed the selection operation at the position of the user interface object.

5. The interactive intelligent fitness mirror device of claim 4, wherein the selection operation comprises any one of gesturing, pushing palm forward, pushing palm backward, swinging hand, and making a fist.

6. The interactive intelligent fitness mirror device of claim 1 or 2, wherein the user interface further comprises a page movement effect, the page movement effect comprises a page switching movement effect and a page interaction movement effect, wherein the page switching movement effect is used for generating a dynamic effect when a page is operated and changed, and the page interaction movement effect is used for providing guidance for a user when interaction is performed.

7. The interactive intelligent fitness mirror device of claim 1, wherein the processor generates skeletal point coordinates of the user based on the user motion data collected by the sensor, obtains coordinates corresponding to the floating icon on the user interface according to the skeletal point coordinates, invokes a motion trajectory corresponding to the coordinates, and controls the floating icon to be displayed on the user interface according to the motion trajectory.

8. The interactive intelligent fitness mirror device of claim 1, wherein the user interface further comprises a wake-up page configured to switch to another user interface when the sensor collects user motion data matching a preset wake-up motion.

9. The interactive smart workout mirror apparatus of claim 1, wherein the user interface further comprises at least one page object for presenting at least one or at least one set of page information, the page information comprising workout videos, visual data, actionable interaction objects, interaction guides.

10. The interactive intelligent fitness mirror device of claim 1, wherein the user actions collected by the sensor comprise static actions and dynamic actions, the static actions being user gestures performed on predetermined body parts for a specified time, and the dynamic actions being user actions performed on predetermined body parts.

11. The interactive intelligent fitness mirror device of claim 10, wherein static actions include, but are not limited to, lifting the left/right hand for a preset time, lifting the left/right leg for a preset time, and lifting the hands over the top of the head for a preset time.

12. The interactive intelligent fitness mirror device of claim 10, wherein the dynamic actions include, but are not limited to, horizontal sliding left/right hand, up and down sliding left/right hand, double-handed clapping, jumping up, and squatting.

13. The interactive intelligent fitness mirror device of claim 1, wherein the sensor is further configured to collect characteristic data of a user, the processor compares the characteristic data input by the user with characteristic data in a database, determines a user using the mirror device based on the comparison, and allows the mirror device to adjust the graphical user interface based on the recognition, the characteristic data of the user including but not limited to a skeletal point, a voiceprint, a weight, and a resting heart rate of the user, the database having a mapping relationship between the user and the characteristic data recorded therein.

Images