WO2021052139A1 - Gesture input method and electronic device - Google Patents

Abstract

Provided in the present application are a gesture input method and an electronic device. The method comprises: an electronic device acquires a first gesture, the first gesture comprising a first sub-gesture and a second sub-gesture; when the first sub-gesture matches a preset gesture, recognizing a function corresponding to the second sub-gesture; and executing the function corresponding to the second sub-gesture. An operating gesture in the present application may comprise a wakeup gesture, that is, an initial gesture of the operating gesture may be the wakeup gesture, thus reducing gesture operation delay, increasing the response efficiency of the electronic device, and increasing the efficiency of interaction between a user and the electronic device.

Description

Translated fromChinese

手势输入方法及电子设备Gesture input method and electronic equipment

本申请要求于2019年09月18日提交中国专利局、申请号为201910883139.8、申请名称为“手势输入方法及电子设备”的中国专利申请的优先权，其全部内容通过引用结合在本申请中。This application claims the priority of a Chinese patent application filed with the Chinese Patent Office on September 18, 2019, the application number is 201910883139.8, and the application name is "gesture input method and electronic equipment", the entire content of which is incorporated into this application by reference.

技术领域Technical field

本申请涉及人工智能技术领域，尤其涉及一种手势输入方法及电子设备。This application relates to the field of artificial intelligence technology, and in particular to a gesture input method and electronic equipment.

背景技术Background technique

随着人工智能(artificial intelligence，AI)技术在移动终端的发展，通过AI技术时刻感知用户行为，提高用户粘性，为用户提高更加智能化的服务，已经成为了移动终端的火爆卖点。With the development of artificial intelligence (AI) technology in mobile terminals, AI technology is used to perceive user behavior at all times, improve user stickiness, and provide users with more intelligent services, which has become a popular selling point for mobile terminals.

目前，人们与移动终端交互的方式主要包括触摸屏滑动、语音等。动态手势交互作为新一代的人机交互方式，在提升移动终端交互体验有着不可替代的作用。At present, the ways that people interact with mobile terminals mainly include touch screen sliding, voice, and so on. As a new generation of human-computer interaction, dynamic gesture interaction plays an irreplaceable role in enhancing the interactive experience of mobile terminals.

现有技术中动态手势的交互过程为：用户输入指定的单一手势维持一段时间以激活手势交互的功能，然后变换手势以使终端实现相应的功能。该交互过程中用户等待时间长，交互效率低，且在交互过程中需要变换手势，用户体验差。The interaction process of dynamic gestures in the prior art is as follows: a single gesture specified by the user input is maintained for a period of time to activate the gesture interaction function, and then the gesture is changed to enable the terminal to realize the corresponding function. During the interaction process, the user waits for a long time, the interaction efficiency is low, and the gesture needs to be changed during the interaction process, and the user experience is poor.

发明内容Summary of the invention

本申请公开了一种手势输入方法及电子设备，可以使用户通过手势与终端交互，且输入的手势包含唤醒手势，无需额外输入唤醒手势，在交互过程中的交互效率高，用户体验好。The present application discloses a gesture input method and electronic device, which can enable a user to interact with a terminal through gestures, and the input gesture includes a wake-up gesture without additional input of the wake-up gesture, the interaction efficiency in the interaction process is high, and the user experience is good.

第一方面，本申请实施例提供一种手势输入方法，该方法应用于电子设备，该方法包括：电子设备获取第一手势，第一手势包括第一子手势和第二子手势；当第一子手势与预设手势匹配时，识别第二子手势对应的功能；执行第二子手势对应的功能。In the first aspect, an embodiment of the present application provides a gesture input method, which is applied to an electronic device, and the method includes: the electronic device acquires a first gesture, the first gesture includes a first sub-gesture and a second sub-gesture; When the sub-gesture matches the preset gesture, the function corresponding to the second sub-gesture is recognized; the function corresponding to the second sub-gesture is executed.

本申请实施例中，操作手势可以包括唤醒手势，即操作手势的起始手势可以为唤醒手势，减小手势操作时延，提高电子设备的响应效率，提高用户与电子设备的交互效率。In the embodiment of the present application, the operation gesture may include a wake-up gesture, that is, the initial gesture of the operation gesture may be a wake-up gesture, which reduces the gesture operation time delay, improves the response efficiency of the electronic device, and improves the interaction efficiency between the user and the electronic device.

在一种可能实现方式中，电子设备获取第一手势包括：电子设备以第一帧率获取第一子手势；当第一子手势与预设手势匹配时，识别第二子手势对应的功能包括：当第一子手势与预设手势匹配时，以第二帧率继续获取第一子手势；当继续获取的第一子手势与预设手势匹配时，以第三帧率获取第二子手势；其中，第三帧率大于第一帧率；确定第二子手势对应的功能。In a possible implementation manner, the electronic device acquiring the first gesture includes: the electronic device acquires the first sub-gesture at a first frame rate; when the first sub-gesture matches a preset gesture, identifying the function corresponding to the second sub-gesture includes : When the first sub-gesture matches the preset gesture, continue to acquire the first sub-gesture at the second frame rate; when the continuously acquired first sub-gesture matches the preset gesture, the second sub-gesture is acquired at the third frame rate ; Wherein, the third frame rate is greater than the first frame rate; determine the function corresponding to the second sub-gesture.

本申请实施例中，在首次确定第一子手势与预设手势匹配后，可进一步继续采集第一子手势，判断后续采集的第一子手势是否与预设手势匹配。这样可以降低误触发概率，避免由于手势误触发造成的功耗浪费。此外，本申请实施例中，在确定用户输入的手势的起始部分与功能手势的起始部分匹配之后，提高图像采集的帧率，可以提升电子设备识别手势的速度，进而提升电子设备做出响应的速度，提升用户与电子设备的交互效率。In this embodiment of the application, after it is determined for the first time that the first sub-gesture matches the preset gesture, the first sub-gesture may be further collected to determine whether the subsequently collected first sub-gesture matches the preset gesture. This can reduce the probability of false triggers and avoid waste of power consumption due to false triggers of gestures. In addition, in the embodiments of the present application, after determining that the initial part of the gesture input by the user matches the initial part of the functional gesture, increasing the frame rate of image collection can increase the speed of the electronic device in recognizing the gesture, thereby improving the performance of the electronic device. The speed of response improves the interaction efficiency between users and electronic devices.

在一种可能的实现方式中，第二帧率等于第三帧率。In a possible implementation manner, the second frame rate is equal to the third frame rate.

本申请实施例可以在第一次检测到第一手子势与预设手势匹配后立即切换采集图像的帧率，进而以第三帧率采集后续的第一子手势及第二子手势，既可以降低误触发的概率，又可在最大程度上提升电子设备的响应速度，提升用户与电子设备的交互效率。The embodiment of the application can switch the frame rate of the captured image immediately after detecting that the first hand gesture matches the preset gesture for the first time, and then collect the subsequent first and second sub-gestures at the third frame rate. The probability of false triggering can be reduced, the response speed of the electronic device can be improved to the greatest extent, and the interaction efficiency between the user and the electronic device can be improved.

在一种可能的实现方式中，第二帧率等于第一帧率。In a possible implementation manner, the second frame rate is equal to the first frame rate.

具体地，上述第一帧率可以是10帧/秒，即电子设备的摄像头可以每秒采集10帧图像。采集的多帧图像中可能包含第一子手势。Specifically, the above-mentioned first frame rate may be 10 frames per second, that is, the camera of the electronic device may collect 10 frames of images per second. The captured multi-frame images may include the first sub-gesture.

具体地，上述第三帧率可以是30帧/秒，即电子设备的摄像头可以每秒采集30帧图像。采集的图像流中可能包含第二子手势。Specifically, the aforementioned third frame rate may be 30 frames per second, that is, the camera of the electronic device may collect 30 frames of images per second. The captured image stream may include a second sub-gesture.

在一种可能的实现方式中，当继续获取的第一子手势与预设手势匹配时，以第三帧率获取第二子手势包括：当连续X个第一子手势中，有x个第一子手势与预设手势匹配时，以第三帧率获取第二子手势；其中，X及x均为正整数，且X大于或等于x。In a possible implementation manner, when the first sub-gesture that continues to be obtained matches the preset gesture, obtaining the second sub-gesture at the third frame rate includes: when there are x-th consecutive first sub-gestures, When a sub-gesture matches the preset gesture, the second sub-gesture is acquired at the third frame rate; where X and x are both positive integers, and X is greater than or equal to x.

本申请提供了一种具体的确定第一子手势与预设手势匹配的方式，保证识别第一子手势的准确性，降低误触发的概率，避免由于手势误触发造成的功耗浪费。The present application provides a specific method for determining the match between the first sub-gesture and the preset gesture to ensure the accuracy of recognizing the first sub-gesture, reduce the probability of false triggering, and avoid waste of power consumption due to false triggering of the gesture.

在一种可能的实现方式中，上述电子设备获取第一手势之前，该方法还包括：电子设备显示第一用户界面。In a possible implementation manner, before the above-mentioned electronic device acquires the first gesture, the method further includes: the electronic device displays a first user interface.

在一种可能的实现方式中，上述当第一子手势与预设手势匹配时，识别第二子手势对应的功能包括：当第一子手势与预设手势匹配时，在第一用户界面中显示指示符，识别第二子手势对应的功能，其中，指示符用于指示电子设备正在识别手势。In a possible implementation, when the first sub-gesture matches the preset gesture, recognizing the function corresponding to the second sub-gesture includes: when the first sub-gesture matches the preset gesture, in the first user interface The indicator is displayed to identify the function corresponding to the second sub-gesture, where the indicator is used to indicate that the electronic device is recognizing the gesture.

本申请实施例可以在确定第一子手势与预设手势匹配后，在电子设备的显示界面中显示指示符，提示用户当前正在识别手势，可以提升用户与电子设备的交互性。In the embodiment of the present application, after determining that the first sub-gesture matches the preset gesture, an indicator may be displayed on the display interface of the electronic device to prompt the user that the gesture is currently being recognized, which may improve the interaction between the user and the electronic device.

在一种可能的实现方式中，上述当第一子手势与预设手势匹配时，识别第二子手势对应的功能包括：当第一子手势与预设手势匹配时，在第一用户界面中显示提示符，识别第二子手势对应的功能，其中，提示符用于指示电子设备支持的多种手势各自对应的功能。In a possible implementation, when the first sub-gesture matches the preset gesture, recognizing the function corresponding to the second sub-gesture includes: when the first sub-gesture matches the preset gesture, in the first user interface A prompt is displayed to identify the function corresponding to the second sub-gesture, where the prompt is used to indicate the respective functions corresponding to the multiple gestures supported by the electronic device.

本申请实施例可以通过在用户界面中显示提示符，并将显示符的显示样式与支持的功能手势关联起来，可以减轻用户的记忆负担，提高用户与电子设备的交互效率。In the embodiment of the present application, a prompt can be displayed in the user interface and the display style of the indicator can be associated with the supported functional gestures, which can reduce the user's memory burden and improve the interaction efficiency between the user and the electronic device.

在另外一种可能的实现方式中，上述电子设备显示第一用户界面，上述第二子手势对应的功能为第一功能；上述执行第二子手势对应的功能之后，上述方法还包括：电子设备显示第二用户界面，上述第一用户界面与上述第一用户界面不同；电子设备获取上述第一手势；当上述第一手势与上述预设手势匹配时，识别上述第二子手势对应的功能为第二功能；上述第一功能与上述第二功能不同；执行上述第二功能。In another possible implementation manner, the electronic device displays a first user interface, and the function corresponding to the second sub-gesture is the first function; after the function corresponding to the second sub-gesture is executed, the method further includes: the electronic device A second user interface is displayed, the first user interface is different from the first user interface; the electronic device acquires the first gesture; when the first gesture matches the preset gesture, the function corresponding to the second sub-gesture is recognized as The second function; the above-mentioned first function is different from the above-mentioned second function; the above-mentioned second function is performed.

本申请实施例中，电子设备在显示不同的用户界面时，用户输入相同的手势可以实现不同的功能。这样可以根据用户界面当前显示的内容的属性设定与该用户界面对应的手势响应功能，可以充分提高手势输入的利用率，提升用户与电子设备交互的趣味性。In the embodiments of the present application, when the electronic device displays different user interfaces, the user can input the same gesture to achieve different functions. In this way, the gesture response function corresponding to the user interface can be set according to the attributes of the content currently displayed on the user interface, which can fully improve the utilization rate of gesture input and enhance the interest of the interaction between the user and the electronic device.

第二方面，本申请实施例提供了一种电子设备，包括：一个或多个处理器、存储器、显示屏以及摄像头；所述存储器、所述显示屏以及所述第一摄像头与所述一个或多个处理器耦合，所述存储器用于存储计算机程序代码，所述计算机程序代码包括计算机指令，当所述一个或多个处理器执行所述计算机指令时，所述电子设备执行：获取第一手势，所述 第一手势包括第一子手势和第二子手势；当所述第一子手势与预设手势匹配时，识别所述第二子手势对应的功能；执行所述第二子手势对应的功能。In a second aspect, an embodiment of the present application provides an electronic device, including: one or more processors, a memory, a display screen, and a camera; the memory, the display screen, the first camera, and the one or more A plurality of processors are coupled, and the memory is used to store computer program code, and the computer program code includes computer instructions. When the one or more processors execute the computer instructions, the electronic device executes: Gesture, the first gesture includes a first sub-gesture and a second sub-gesture; when the first sub-gesture matches a preset gesture, recognize the function corresponding to the second sub-gesture; execute the second sub-gesture Corresponding function.

在一种可能的实现方式中，上述电子设备执行获取第一手势，具体执行：电子设备以第一帧率获取第一子手势；上述电子设备执行当第一子手势与预设手势匹配时，识别第二子手势对应的功能，具体执行：当第一子手势与预设手势匹配时，以第二帧率继续获取第一子手势；当继续获取的第一子手势与预设手势匹配时，以第三帧率获取第二子手势；其中，第三帧率大于第一帧率；确定第二子手势对应的功能。In a possible implementation manner, the above-mentioned electronic device executes the acquisition of the first gesture, which specifically executes: the electronic device acquires the first sub-gesture at the first frame rate; the above-mentioned electronic device executes when the first sub-gesture matches the preset gesture, Recognize the function corresponding to the second sub-gesture, and specifically execute: when the first sub-gesture matches the preset gesture, continue to acquire the first sub-gesture at the second frame rate; when the continuously acquired first sub-gesture matches the preset gesture , Acquire the second sub-gesture at the third frame rate; wherein the third frame rate is greater than the first frame rate; and determine the function corresponding to the second sub-gesture.

在一种可能的实现方式中，第二帧率等于第三帧率。In a possible implementation manner, the second frame rate is equal to the third frame rate.

在一种可能的实现方式中，第二帧率等于第一帧率。In a possible implementation manner, the second frame rate is equal to the first frame rate.

具体地，上述第一帧率可以是10帧/秒，即电子设备的摄像头可以每秒采集10帧图像。采集的多帧图像中可能包含第一子手势。Specifically, the above-mentioned first frame rate may be 10 frames per second, that is, the camera of the electronic device may collect 10 frames of images per second. The captured multi-frame images may include the first sub-gesture.

具体地，上述第三帧率可以是30帧/秒，即电子设备的摄像头可以每秒采集30帧图像。采集的图像流中可能包含第二子手势。Specifically, the aforementioned third frame rate may be 30 frames per second, that is, the camera of the electronic device may collect 30 frames of images per second. The captured image stream may include a second sub-gesture.

在一种可能的实现方式中，上述电子设备执行当继续获取的第一子手势与预设手势匹配时，以第三帧率获取第二子手势，具体执行：当连续X个第一子手势中，有x个第一子手势与预设手势匹配时，以第三帧率获取第二子手势；其中，X及x均为正整数，且X大于或等于x。In a possible implementation manner, the above electronic device executes the second sub-gesture at the third frame rate when the first sub-gesture that continues to be acquired matches the preset gesture, and specifically executes: when X consecutive first sub-gestures When there are x first sub-gestures matching the preset gesture, the second sub-gesture is acquired at the third frame rate; where X and x are both positive integers, and X is greater than or equal to x.

在一种可能的实现方式中，上述电子设备获取第一手势之前，电子设备还执行：显示第一用户界面。In a possible implementation manner, before the above-mentioned electronic device acquires the first gesture, the electronic device further executes: displaying the first user interface.

在一种可能的实现方式中，上述电子设备执行当第一子手势与预设手势匹配时，识别第二子手势对应的功能，具体执行：当第一子手势与预设手势匹配时，在第一用户界面中显示指示符，识别第二子手势对应的功能，其中，指示符用于指示电子设备正在识别手势。In a possible implementation manner, the above electronic device executes the function of recognizing the second sub-gesture when the first sub-gesture matches the preset gesture, and specifically executes: when the first sub-gesture matches the preset gesture, An indicator is displayed on the first user interface to recognize the function corresponding to the second sub-gesture, where the indicator is used to indicate that the electronic device is recognizing the gesture.

在一种可能的实现方式中，上述电子设备执行上述当第一子手势与预设手势匹配时，识别第二子手势对应的功能，具体执行：当第一子手势与预设手势匹配时，在第一用户界面中显示提示符，识别第二子手势对应的功能，其中，提示符用于指示电子设备支持的多种手势各自对应的功能。In a possible implementation manner, the above-mentioned electronic device executes the above-mentioned function of recognizing the second sub-gesture when the first sub-gesture matches the preset gesture, and specifically executes: when the first sub-gesture matches the preset gesture, A prompt is displayed in the first user interface to identify the function corresponding to the second sub-gesture, where the prompt is used to indicate the respective functions corresponding to the multiple gestures supported by the electronic device.

在一种可能的实现方式中，上述电子设备显示第一用户界面，上述第二子手势对应的功能为第一功能；上述电子设备执行第二子手势对应的功能之后，上述电子设备还执行：显示第二用户界面，上述第一用户界面与上述第一用户界面不同；获取上述第一手势；当上述第一手势与上述预设手势匹配时，识别上述第二子手势对应的功能为第二功能；上述第一功能与上述第二功能不同；执行上述第二功能。In a possible implementation manner, the electronic device displays a first user interface, and the function corresponding to the second sub-gesture is the first function; after the electronic device executes the function corresponding to the second sub-gesture, the electronic device further executes: Display a second user interface, the first user interface is different from the first user interface; the first gesture is acquired; when the first gesture matches the preset gesture, the function corresponding to the second sub-gesture is recognized as the second Function; the above-mentioned first function is different from the above-mentioned second function; the above-mentioned second function is performed.

第三方面，本申请实施例提供了一种计算机可读存储介质，计算机可读存储介质中存储有指令，当其在电子设备上运行时，使得电子设备执行上述第一方面或者第一方面的任意一种实现方式提供的方法。In the third aspect, the embodiments of the present application provide a computer-readable storage medium with instructions stored in the computer-readable storage medium, which when run on an electronic device, cause the electronic device to execute the above-mentioned first aspect or the first aspect. The method provided by any one of the implementations.

第四方面，本申请实施例提供了一种计算机程序产品，当其在电子设备上运行时，使得电子设备执行上述第一方面或者第一方面的任意一种实现方式提供的方法。In a fourth aspect, the embodiments of the present application provide a computer program product that, when it runs on an electronic device, causes the electronic device to execute the method provided in the first aspect or any one of the implementation manners of the first aspect.

可以理解地，上述第二方面提供的电子设备、上述第三方面提供的上述的计算机存储介质或者上述第四方面提供的计算机程序产品均用于执行第一方面所提供的手势输入方法。 因此，其所能达到的有益效果可参考对应方法中的有益效果，此处不再赘述。It is understandable that the electronic device provided in the second aspect, the computer storage medium provided in the third aspect, or the computer program product provided in the fourth aspect are all used to implement the gesture input method provided in the first aspect. Therefore, the beneficial effects that can be achieved can refer to the beneficial effects in the corresponding method, which will not be repeated here.

附图说明Description of the drawings

图1为本申请实施例提供的电子设备的结构示意图；FIG. 1 is a schematic diagram of the structure of an electronic device provided by an embodiment of the application;

图2是本申请实施例提供的电子设备的软件结构框图；2 is a block diagram of the software structure of an electronic device provided by an embodiment of the present application;

图3为本申请实施例提供的一种用户输入手势示意图；FIG. 3 is a schematic diagram of a user input gesture provided by an embodiment of the application;

图4为本申请实施例提供的另外一种电子设备的结果示意图；FIG. 4 is a schematic diagram of the result of another electronic device provided by an embodiment of this application;

图5为本申请实施例提供的一种图像采集过程示意图；FIG. 5 is a schematic diagram of an image acquisition process provided by an embodiment of this application;

图6为本申请实施例提供的另外一种图像采集过程示意图；6 is a schematic diagram of another image acquisition process provided by an embodiment of the application;

图7为本申请实施例提供的应用场景一示意图；FIG. 7 is a schematic diagram ofapplication scenario 1 provided by an embodiment of the application;

图8为本申请实施例提供的应用场景一的一些用户界面示意图；FIG. 8 is a schematic diagram of some user interfaces ofapplication scenario 1 provided by an embodiment of the application;

图9为本申请实施例提供的应用场景一的另一些用户界面示意图；FIG. 9 is a schematic diagram of other user interfaces ofapplication scenario 1 provided by an embodiment of this application;

图10为本申请实施例提供的应用场景二示意图；FIG. 10 is a schematic diagram of application scenario 2 provided by an embodiment of the application;

图11为本申请实施例提供的应用场景二的一些用户界面示意图；11 is a schematic diagram of some user interfaces of application scenario 2 provided by an embodiment of the application;

图12为本申请实施例提供的应用场景三的一些用户界面示意图；FIG. 12 is a schematic diagram of some user interfaces of application scenario 3 provided by an embodiment of the application;

图13为本申请实施例提供的一种手势交互提示符；FIG. 13 is a gesture interaction prompt provided by an embodiment of this application;

图14为本申请实施例提供的另一种手势交互提示符；FIG. 14 is another gesture interaction prompt provided by an embodiment of this application;

图15为本申请实施例提供的一种手势输入方法流程示意图。FIG. 15 is a schematic flowchart of a gesture input method provided by an embodiment of the application.

具体实施方式detailed description

下面将结合附图对本申请实施例中的技术方案进行清楚、详尽地描述。The technical solutions in the embodiments of the present application will be described clearly and in detail below in conjunction with the accompanying drawings.

本申请实施例提供了一种手势输入方法，电子设备可以在识别出用户输入的任意手势的初始动作时先判断该手势是否为电子设备支持的手势，若是则继续识别用户输入的手势，并确定该手势对应的功能。也即是说，电子设备支持的每个操作手势的起始动作可以作为手势交互功能的唤醒手势。这样，与现有技术中先输入唤醒手势再输入操作手势的方式相比，本申请可以省略唤醒手势，减少用户隔空输入手势的时间，减少手势操作时延，提高用户与电子设备的交互效率。The embodiment of the application provides a gesture input method. When the electronic device recognizes the initial action of any gesture input by the user, it can first determine whether the gesture is a gesture supported by the electronic device, and if so, continue to recognize the gesture input by the user, and determine The function corresponding to this gesture. In other words, the initial action of each operation gesture supported by the electronic device can be used as a wake-up gesture of the gesture interaction function. In this way, compared with the prior art method of first inputting the wake-up gesture and then the operation gesture, the present application can omit the wake-up gesture, reduce the time for the user to input the gesture in a space, reduce the gesture operation delay, and improve the interaction efficiency between the user and the electronic device. .

本申请实施例中涉及的电子设备可以是手机、平板电脑、桌面型、膝上型、笔记本电脑、超级移动个人计算机(Ultra-mobile Personal Computer，UMPC)、手持计算机、上网本、个人数字助理(Personal Digital Assistant，PDA)、可穿戴电子设备、虚拟现实设备等。The electronic devices involved in the embodiments of this application may be mobile phones, tablet computers, desktops, laptops, notebook computers, Ultra-mobile Personal Computers (UMPC), handheld computers, netbooks, and personal digital assistants (Personal Digital Assistants). Digital Assistant, PDA), wearable electronic equipment, virtual reality equipment, etc.

接下来，介绍本申请以下实施例中提供的示例性电子设备。Next, an exemplary electronic device provided in the following embodiments of the present application is introduced.

图1示出了电子设备100的结构示意图。FIG. 1 shows a schematic diagram of the structure of an electronic device 100.

电子设备100可以包括处理器110，外部存储器接口120，内部存储器121，通用串行总线(universal serial bus，USB)接口130，充电管理模块140，电源管理模块141，电池142，天线1，天线2，移动通信模块150，无线通信模块160，音频模块170，扬声器170A，受话器170B，麦克风170C，耳机接口170D，传感器模块180，按键190，马达191，指示器192，摄像头193，显示屏194，以及用户标识模块(subscriber identification module，SIM) 卡接口195等。其中传感器模块180可以包括压力传感器180A，陀螺仪传感器180B，气压传感器180C，磁传感器180D，加速度传感器180E，距离传感器180F，接近光传感器180G，指纹传感器180H，温度传感器180J，触摸传感器180K，环境光传感器180L，骨传导传感器180M等。The electronic device 100 may include aprocessor 110, anexternal memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, acharging management module 140, a power management module 141, a battery 142, anantenna 1, and an antenna 2. ,Mobile communication module 150, wireless communication module 160,audio module 170,speaker 170A,receiver 170B,microphone 170C,earphone jack 170D, sensor module 180, buttons 190, motor 191, indicator 192, camera 193, display screen 194, and Subscriber identification module (subscriber identification module, SIM)card interface 195, etc. The sensor module 180 may include a pressure sensor 180A, a gyroscope sensor 180B, an air pressure sensor 180C, a magnetic sensor 180D, an acceleration sensor 180E, a distance sensor 180F, a proximity sensor 180G, a fingerprint sensor 180H, a temperature sensor 180J, a touch sensor 180K, and ambient light Sensor 180L, bone conduction sensor 180M, etc.

可以理解的是，本发明实施例示意的结构并不构成对电子设备100的具体限定。在本申请另一些实施例中，电子设备100可以包括比图示更多或更少的部件，或者组合某些部件，或者拆分某些部件，或者不同的部件布置。图示的部件可以以硬件，软件或软件和硬件的组合实现。It can be understood that the structure illustrated in the embodiment of the present invention does not constitute a specific limitation on the electronic device 100. In other embodiments of the present application, the electronic device 100 may include more or fewer components than shown, or combine certain components, or split certain components, or arrange different components. The illustrated components can be implemented in hardware, software, or a combination of software and hardware.

处理器110可以包括一个或多个处理单元，例如：处理器110可以包括应用处理器(application processor，AP)，调制解调处理器，图形处理器(graphics processing unit，GPU)，图像信号处理器(image signal processor，ISP)，控制器，视频编解码器，数字信号处理器(digital signal processor，DSP)，基带处理器，和/或神经网络处理器(neural-network processing unit，NPU)等。其中，不同的处理单元可以是独立的器件，也可以集成在一个或多个处理器中。Theprocessor 110 may include one or more processing units. For example, theprocessor 110 may include an application processor (AP), a modem processor, a graphics processing unit (GPU), and an image signal processor. (image signal processor, ISP), controller, video codec, digital signal processor (digital signal processor, DSP), baseband processor, and/or neural-network processing unit (NPU), etc. Among them, the different processing units may be independent devices or integrated in one or more processors.

控制器可以根据指令操作码和时序信号，产生操作控制信号，完成取指令和执行指令的控制。The controller can generate operation control signals according to the instruction operation code and timing signals to complete the control of fetching and executing instructions.

处理器110中还可以设置存储器，用于存储指令和数据。在一些实施例中，处理器110中的存储器为高速缓冲存储器。该存储器可以保存处理器110刚用过或循环使用的指令或数据。如果处理器110需要再次使用该指令或数据，可从所述存储器中直接调用。避免了重复存取，减少了处理器110的等待时间，因而提高了系统的效率。A memory may also be provided in theprocessor 110 to store instructions and data. In some embodiments, the memory in theprocessor 110 is a cache memory. The memory can store instructions or data that theprocessor 110 has just used or used cyclically. If theprocessor 110 needs to use the instruction or data again, it can be directly called from the memory. Repeated accesses are avoided, the waiting time of theprocessor 110 is reduced, and the efficiency of the system is improved.

在一些实施例中，处理器110可以包括一个或多个接口。接口可以包括集成电路(inter-integrated circuit，I2C)接口，集成电路内置音频(inter-integrated circuit sound，I2S)接口，脉冲编码调制(pulse code modulation，PCM)接口，通用异步收发传输器(universal asynchronous receiver/transmitter，UART)接口，移动产业处理器接口(mobile industry processor interface，MIPI)，通用输入输出(general-purpose input/output，GPIO)接口，用户标识模块(subscriber identity module，SIM)接口，和/或通用串行总线(universal serial bus，USB)接口等。In some embodiments, theprocessor 110 may include one or more interfaces. The interface can include an integrated circuit (inter-integrated circuit, I2C) interface, an integrated circuit built-in audio (inter-integrated circuit sound, I2S) interface, a pulse code modulation (pulse code modulation, PCM) interface, and a universal asynchronous transmitter (universal asynchronous) interface. receiver/transmitter, UART) interface, mobile industry processor interface (MIPI), general-purpose input/output (GPIO) interface, subscriber identity module (SIM) interface, and / Or Universal Serial Bus (USB) interface, etc.

I2C接口是一种双向同步串行总线，包括一根串行数据线(serial data line，SDA)和一根串行时钟线(derail clock line，SCL)。在一些实施例中，处理器110可以包含多组I2C总线。处理器110可以通过不同的I2C总线接口分别耦合触摸传感器180K，充电器，闪光灯，摄像头193等。例如：处理器110可以通过I2C接口耦合触摸传感器180K，使处理器110与触摸传感器180K通过I2C总线接口通信，实现电子设备100的触摸功能。The I2C interface is a bidirectional synchronous serial bus, which includes a serial data line (SDA) and a serial clock line (SCL). In some embodiments, theprocessor 110 may include multiple sets of I2C buses. Theprocessor 110 may couple the touch sensor 180K, the charger, the flash, the camera 193, etc., respectively through different I2C bus interfaces. For example, theprocessor 110 may couple the touch sensor 180K through an I2C interface, so that theprocessor 110 and the touch sensor 180K communicate through an I2C bus interface to implement the touch function of the electronic device 100.

I2S接口可以用于音频通信。在一些实施例中，处理器110可以包含多组I2S总线。处理器110可以通过I2S总线与音频模块170耦合，实现处理器110与音频模块170之间的通信。在一些实施例中，音频模块170可以通过I2S接口向无线通信模块160传递音频信号，实现通过蓝牙耳机接听电话的功能。The I2S interface can be used for audio communication. In some embodiments, theprocessor 110 may include multiple sets of I2S buses. Theprocessor 110 may be coupled with theaudio module 170 through an I2S bus to implement communication between theprocessor 110 and theaudio module 170. In some embodiments, theaudio module 170 may transmit audio signals to the wireless communication module 160 through an I2S interface, so as to realize the function of answering calls through a Bluetooth headset.

PCM接口也可以用于音频通信，将模拟信号抽样，量化和编码。在一些实施例中，音频模块170与无线通信模块160可以通过PCM总线接口耦合。在一些实施例中，音频模块170也可以通过PCM接口向无线通信模块160传递音频信号，实现通过蓝牙耳机接听电话 的功能。所述I2S接口和所述PCM接口都可以用于音频通信。The PCM interface can also be used for audio communication to sample, quantize and encode analog signals. In some embodiments, theaudio module 170 and the wireless communication module 160 may be coupled through a PCM bus interface. In some embodiments, theaudio module 170 may also transmit audio signals to the wireless communication module 160 through the PCM interface, so as to realize the function of answering calls through the Bluetooth headset. Both the I2S interface and the PCM interface can be used for audio communication.

UART接口是一种通用串行数据总线，用于异步通信。该总线可以为双向通信总线。它将要传输的数据在串行通信与并行通信之间转换。在一些实施例中，UART接口通常被用于连接处理器110与无线通信模块160。例如：处理器110通过UART接口与无线通信模块160中的蓝牙模块通信，实现蓝牙功能。在一些实施例中，音频模块170可以通过UART接口向无线通信模块160传递音频信号，实现通过蓝牙耳机播放音乐的功能。The UART interface is a universal serial data bus used for asynchronous communication. The bus can be a two-way communication bus. It converts the data to be transmitted between serial communication and parallel communication. In some embodiments, the UART interface is generally used to connect theprocessor 110 and the wireless communication module 160. For example, theprocessor 110 communicates with the Bluetooth module in the wireless communication module 160 through the UART interface to realize the Bluetooth function. In some embodiments, theaudio module 170 may transmit audio signals to the wireless communication module 160 through a UART interface, so as to realize the function of playing music through a Bluetooth headset.

MIPI接口可以被用于连接处理器110与显示屏194，摄像头193等外围器件。MIPI接口包括摄像头串行接口(camera serial interface，CSI)，显示屏串行接口(display serial interface，DSI)等。在一些实施例中，处理器110和摄像头193通过CSI接口通信，实现电子设备100的拍摄功能。处理器110和显示屏194通过DSI接口通信，实现电子设备100的显示功能。The MIPI interface can be used to connect theprocessor 110 with the display screen 194, the camera 193 and other peripheral devices. The MIPI interface includes a camera serial interface (camera serial interface, CSI), a display serial interface (display serial interface, DSI), and so on. In some embodiments, theprocessor 110 and the camera 193 communicate through a CSI interface to implement the shooting function of the electronic device 100. Theprocessor 110 and the display screen 194 communicate through a DSI interface to realize the display function of the electronic device 100.

GPIO接口可以通过软件配置。GPIO接口可以被配置为控制信号，也可被配置为数据信号。在一些实施例中，GPIO接口可以用于连接处理器110与摄像头193，显示屏194，无线通信模块160，音频模块170，传感器模块180等。GPIO接口还可以被配置为I2C接口，I2S接口，UART接口，MIPI接口等。The GPIO interface can be configured through software. The GPIO interface can be configured as a control signal or as a data signal. In some embodiments, the GPIO interface can be used to connect theprocessor 110 with the camera 193, the display screen 194, the wireless communication module 160, theaudio module 170, the sensor module 180, and so on. The GPIO interface can also be configured as an I2C interface, I2S interface, UART interface, MIPI interface, etc.

USB接口130是符合USB标准规范的接口，具体可以是Mini USB接口，Micro USB接口，USB Type C接口等。USB接口130可以用于连接充电器为电子设备100充电，也可以用于电子设备100与外围设备之间传输数据。也可以用于连接耳机，通过耳机播放音频。该接口还可以用于连接其他电子设备，例如AR设备等。The USB interface 130 is an interface that complies with the USB standard specification, and specifically may be a Mini USB interface, a Micro USB interface, a USB Type C interface, and so on. The USB interface 130 can be used to connect a charger to charge the electronic device 100, and can also be used to transfer data between the electronic device 100 and peripheral devices. It can also be used to connect earphones and play audio through earphones. This interface can also be used to connect to other electronic devices, such as AR devices.

可以理解的是，本发明实施例示意的各模块间的接口连接关系，只是示意性说明，并不构成对电子设备100的结构限定。在本申请另一些实施例中，电子设备100也可以采用上述实施例中不同的接口连接方式，或多种接口连接方式的组合。It can be understood that the interface connection relationship between the modules illustrated in the embodiment of the present invention is merely a schematic description, and does not constitute a structural limitation of the electronic device 100. In other embodiments of the present application, the electronic device 100 may also adopt different interface connection modes in the foregoing embodiments, or a combination of multiple interface connection modes.

充电管理模块140用于从充电器接收充电输入。其中，充电器可以是无线充电器，也可以是有线充电器。在一些有线充电的实施例中，充电管理模块140可以通过USB接口130接收有线充电器的充电输入。在一些无线充电的实施例中，充电管理模块140可以通过电子设备100的无线充电线圈接收无线充电输入。充电管理模块140为电池142充电的同时，还可以通过电源管理模块141为电子设备供电。Thecharging management module 140 is used to receive charging input from the charger. Among them, the charger can be a wireless charger or a wired charger. In some wired charging embodiments, thecharging management module 140 may receive the charging input of the wired charger through the USB interface 130. In some embodiments of wireless charging, thecharging management module 140 may receive the wireless charging input through the wireless charging coil of the electronic device 100. While thecharging management module 140 charges the battery 142, it can also supply power to the electronic device through the power management module 141.

电源管理模块141用于连接电池142，充电管理模块140与处理器110。电源管理模块141接收电池142和/或充电管理模块140的输入，为处理器110，内部存储器121，显示屏194，摄像头193，和无线通信模块160等供电。电源管理模块141还可以用于监测电池容量，电池循环次数，电池健康状态(漏电，阻抗)等参数。在其他一些实施例中，电源管理模块141也可以设置于处理器110中。在另一些实施例中，电源管理模块141和充电管理模块140也可以设置于同一个器件中。The power management module 141 is used to connect the battery 142, thecharging management module 140 and theprocessor 110. The power management module 141 receives input from the battery 142 and/or thecharging management module 140, and supplies power to theprocessor 110, the internal memory 121, the display screen 194, the camera 193, and the wireless communication module 160. The power management module 141 can also be used to monitor parameters such as battery capacity, battery cycle times, and battery health status (leakage, impedance). In some other embodiments, the power management module 141 may also be provided in theprocessor 110. In other embodiments, the power management module 141 and thecharging management module 140 may also be provided in the same device.

电子设备100的无线通信功能可以通过天线1，天线2，移动通信模块150，无线通信模块160，调制解调处理器以及基带处理器等实现。The wireless communication function of the electronic device 100 can be implemented by theantenna 1, the antenna 2, themobile communication module 150, the wireless communication module 160, the modem processor, and the baseband processor.

天线1和天线2用于发射和接收电磁波信号。电子设备100中的每个天线可用于覆盖单个或多个通信频带。不同的天线还可以复用，以提高天线的利用率。例如：可以将天线1复用为无线局域网的分集天线。在另外一些实施例中，天线可以和调谐开关结合使用。Theantenna 1 and the antenna 2 are used to transmit and receive electromagnetic wave signals. Each antenna in the electronic device 100 can be used to cover a single or multiple communication frequency bands. Different antennas can also be reused to improve antenna utilization. For example:Antenna 1 can be multiplexed as a diversity antenna of a wireless local area network. In other embodiments, the antenna can be used in combination with a tuning switch.

移动通信模块150可以提供应用在电子设备100上的包括2G/3G/4G/5G等无线通信的 解决方案。移动通信模块150可以包括至少一个滤波器，开关，功率放大器，低噪声放大器(low noise amplifier，LNA)等。移动通信模块150可以由天线1接收电磁波，并对接收的电磁波进行滤波，放大等处理，传送至调制解调处理器进行解调。移动通信模块150还可以对经调制解调处理器调制后的信号放大，经天线1转为电磁波辐射出去。在一些实施例中，移动通信模块150的至少部分功能模块可以被设置于处理器110中。在一些实施例中，移动通信模块150的至少部分功能模块可以与处理器110的至少部分模块被设置在同一个器件中。Themobile communication module 150 may provide a wireless communication solution including 2G/3G/4G/5G and the like applied to the electronic device 100. Themobile communication module 150 may include at least one filter, switch, power amplifier, low noise amplifier (LNA), etc. Themobile communication module 150 can receive electromagnetic waves by theantenna 1, and perform processing such as filtering, amplifying and transmitting the received electromagnetic waves to the modem processor for demodulation. Themobile communication module 150 can also amplify the signal modulated by the modem processor, and convert it into electromagnetic wave radiation via theantenna 1. In some embodiments, at least part of the functional modules of themobile communication module 150 may be provided in theprocessor 110. In some embodiments, at least part of the functional modules of themobile communication module 150 and at least part of the modules of theprocessor 110 may be provided in the same device.

调制解调处理器可以包括调制器和解调器。其中，调制器用于将待发送的低频基带信号调制成中高频信号。解调器用于将接收的电磁波信号解调为低频基带信号。随后解调器将解调得到的低频基带信号传送至基带处理器处理。低频基带信号经基带处理器处理后，被传递给应用处理器。应用处理器通过音频设备(不限于扬声器170A，受话器170B等)输出声音信号，或通过显示屏194显示图像或视频。在一些实施例中，调制解调处理器可以是独立的器件。在另一些实施例中，调制解调处理器可以独立于处理器110，与移动通信模块150或其他功能模块设置在同一个器件中。The modem processor may include a modulator and a demodulator. Among them, the modulator is used to modulate the low frequency baseband signal to be sent into a medium and high frequency signal. The demodulator is used to demodulate the received electromagnetic wave signal into a low-frequency baseband signal. Then the demodulator transmits the demodulated low-frequency baseband signal to the baseband processor for processing. After the low-frequency baseband signal is processed by the baseband processor, it is passed to the application processor. The application processor outputs a sound signal through an audio device (not limited to thespeaker 170A, thereceiver 170B, etc.), or displays an image or video through the display screen 194. In some embodiments, the modem processor may be an independent device. In other embodiments, the modem processor may be independent of theprocessor 110 and be provided in the same device as themobile communication module 150 or other functional modules.

无线通信模块160可以提供应用在电子设备100上的包括无线局域网(wireless local area networks，WLAN)(如无线保真(wireless fidelity，Wi-Fi)网络)，蓝牙(bluetooth，BT)，全球导航卫星系统(global navigation satellite system，GNSS)，调频(frequency modulation，FM)，近距离无线通信技术(near field communication，NFC)，红外技术(infrared，IR)等无线通信的解决方案。无线通信模块160可以是集成至少一个通信处理模块的一个或多个器件。无线通信模块160经由天线2接收电磁波，将电磁波信号调频以及滤波处理，将处理后的信号发送到处理器110。无线通信模块160还可以从处理器110接收待发送的信号，对其进行调频，放大，经天线2转为电磁波辐射出去。The wireless communication module 160 can provide applications on the electronic device 100 including wireless local area networks (WLAN) (such as wireless fidelity (Wi-Fi) networks), bluetooth (BT), and global navigation satellites. System (global navigation satellite system, GNSS), frequency modulation (frequency modulation, FM), near field communication technology (near field communication, NFC), infrared technology (infrared, IR) and other wireless communication solutions. The wireless communication module 160 may be one or more devices integrating at least one communication processing module. The wireless communication module 160 receives electromagnetic waves via the antenna 2, frequency modulates and filters the electromagnetic wave signals, and sends the processed signals to theprocessor 110. The wireless communication module 160 may also receive the signal to be sent from theprocessor 110, perform frequency modulation, amplify it, and convert it into electromagnetic waves to radiate through the antenna 2.

在一些实施例中，电子设备100的天线1和移动通信模块150耦合，天线2和无线通信模块160耦合，使得电子设备100可以通过无线通信技术与网络以及其他设备通信。所述无线通信技术可以包括全球移动通讯系统(global system for mobile communications，GSM)，通用分组无线服务(general packet radio service，GPRS)，码分多址接入(code division multiple access，CDMA)，宽带码分多址(wideband code division multiple access，WCDMA)，时分码分多址(time-division code division multiple access，TD-SCDMA)，长期演进(long term evolution，LTE)，BT，GNSS，WLAN，NFC，FM，和/或IR技术等。所述GNSS可以包括全球卫星定位系统(global positioning system，GPS)，全球导航卫星系统(global navigation satellite system，GLONASS)，北斗卫星导航系统(beidou navigation satellite system，BDS)，准天顶卫星系统(quasi-zenith satellite system，QZSS)和/或星基增强系统(satellite based augmentation systems，SBAS)。In some embodiments, theantenna 1 of the electronic device 100 is coupled with themobile communication module 150, and the antenna 2 is coupled with the wireless communication module 160, so that the electronic device 100 can communicate with the network and other devices through wireless communication technology. The wireless communication technology may include global system for mobile communications (GSM), general packet radio service (GPRS), code division multiple access (CDMA), broadband Code division multiple access (wideband code division multiple access, WCDMA), time-division code division multiple access (TD-SCDMA), long term evolution (LTE), BT, GNSS, WLAN, NFC , FM, and/or IR technology, etc. The GNSS may include global positioning system (GPS), global navigation satellite system (GLONASS), Beidou navigation satellite system (BDS), quasi-zenith satellite system (quasi -zenith satellite system, QZSS) and/or satellite-based augmentation systems (SBAS).

电子设备100通过GPU，显示屏194，以及应用处理器等实现显示功能。GPU为图像处理的微处理器，连接显示屏194和应用处理器。GPU用于执行数学和几何计算，用于图形渲染。处理器110可包括一个或多个GPU，其执行程序指令以生成或改变显示信息。The electronic device 100 implements a display function through a GPU, a display screen 194, an application processor, and the like. The GPU is a microprocessor for image processing, connected to the display 194 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. Theprocessor 110 may include one or more GPUs, which execute program instructions to generate or change display information.

显示屏194用于显示图像，视频等。显示屏194包括显示面板。显示面板可以采用液晶显示屏(liquid crystal display，LCD)，有机发光二极管(organic light-emitting diode，OLED)， 有源矩阵有机发光二极体或主动矩阵有机发光二极体(active-matrix organic light emitting diode的，AMOLED)，柔性发光二极管(flex light-emitting diode，FLED)，Miniled，MicroLed，Micro-oLed，量子点发光二极管(quantum dot light emitting diodes，QLED)等。在一些实施例中，电子设备100可以包括1个或N个显示屏194，N为大于1的正整数。The display screen 194 is used to display images, videos, and the like. The display screen 194 includes a display panel. The display panel can adopt liquid crystal display (LCD), organic light-emitting diode (OLED), active-matrix organic light-emitting diode or active-matrix organic light-emitting diode (active-matrix organic light). AMOLED, flexible light-emitting diode (FLED), Miniled, MicroLed, Micro-oLed, quantum dot light-emitting diode (QLED), etc. In some embodiments, the electronic device 100 may include one or N display screens 194, and N is a positive integer greater than one.

电子设备100可以通过ISP，摄像头193，视频编解码器，GPU，显示屏194以及应用处理器等实现拍摄功能。The electronic device 100 can realize a shooting function through an ISP, a camera 193, a video codec, a GPU, a display screen 194, and an application processor.

ISP用于处理摄像头193反馈的数据。例如，拍照时，打开快门，光线通过镜头被传递到摄像头感光元件上，光信号转换为电信号，摄像头感光元件将所述电信号传递给ISP处理，转化为肉眼可见的图像。ISP还可以对图像的噪点，亮度，肤色进行算法优化。ISP还可以对拍摄场景的曝光，色温等参数优化。在一些实施例中，ISP可以设置在摄像头193中。本申请实施例中，ISP可以对摄像头193采集的包含用户手势的图像进行处理。The ISP is used to process the data fed back by the camera 193. For example, when taking a picture, the shutter is opened, the light is transmitted to the photosensitive element of the camera through the lens, the light signal is converted into an electrical signal, and the photosensitive element of the camera transmits the electrical signal to the ISP for processing and is converted into an image visible to the naked eye. ISP can also optimize the image noise, brightness, and skin color. ISP can also optimize the exposure, color temperature and other parameters of the shooting scene. In some embodiments, the ISP may be provided in the camera 193. In this embodiment of the present application, the ISP may process the images collected by the camera 193 that include user gestures.

摄像头193用于捕获静态图像或视频。物体通过镜头生成光学图像投射到感光元件。感光元件可以是电荷耦合器件(charge coupled device，CCD)或互补金属氧化物半导体(complementary metal-oxide-semiconductor，CMOS)光电晶体管。感光元件把光信号转换成电信号，之后将电信号传递给ISP转换成数字图像信号。ISP将数字图像信号输出到DSP加工处理。DSP将数字图像信号转换成标准的RGB，YUV等格式的图像信号。在一些实施例中，电子设备100可以包括1个或N个摄像头193，N为大于1的正整数。本申请实施例中，摄像头193可以用于以第一频率采集图像流，在NPU识别出图像流中包含电子设备支持的手势的起始部分时，以第二频率采集图像流。其中，第一频率低于第二频率。本申请实施例中涉及的摄像头193可以为前置摄像头。The camera 193 is used to capture still images or videos. The object generates an optical image through the lens and is projected to the photosensitive element. The photosensitive element may be a charge coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS) phototransistor. The photosensitive element converts the optical signal into an electrical signal, and then transfers the electrical signal to the ISP to convert it into a digital image signal. ISP outputs digital image signals to DSP for processing. DSP converts digital image signals into standard RGB, YUV and other formats of image signals. In some embodiments, the electronic device 100 may include one or N cameras 193, and N is a positive integer greater than one. In this embodiment of the present application, the camera 193 may be used to capture an image stream at a first frequency, and when the NPU recognizes that the image stream contains the initial part of a gesture supported by the electronic device, the image stream is captured at the second frequency. Among them, the first frequency is lower than the second frequency. The camera 193 involved in the embodiment of the present application may be a front camera.

数字信号处理器用于处理数字信号，除了可以处理数字图像信号，还可以处理其他数字信号。例如，当电子设备100在频点选择时，数字信号处理器用于对频点能量进行傅里叶变换等。Digital signal processors are used to process digital signals. In addition to digital image signals, they can also process other digital signals. For example, when the electronic device 100 selects a frequency point, the digital signal processor is used to perform Fourier transform on the energy of the frequency point.

视频编解码器用于对数字视频压缩或解压缩。电子设备100可以支持一种或多种视频编解码器。这样，电子设备100可以播放或录制多种编码格式的视频，例如：动态图像专家组(moving picture experts group，MPEG)1，MPEG2，MPEG3，MPEG4等。Video codecs are used to compress or decompress digital video. The electronic device 100 may support one or more video codecs. In this way, the electronic device 100 can play or record videos in multiple encoding formats, such as: moving picture experts group (MPEG) 1, MPEG2, MPEG3, MPEG4, and so on.

NPU为神经网络(neural-network，NN)计算处理器，通过借鉴生物神经网络结构，例如借鉴人脑神经元之间传递模式，对输入信息快速处理，还可以不断的自学习。通过NPU可以实现电子设备100的智能认知等应用，例如：图像识别，人脸识别，语音识别，文本理解等。本申请实施例中，NPU可以用于处理摄像头193采集的图像，分析图像中包含的手势是否为电子设备100支持的手势的起始部分，或者分析图像中包含的手势是否为电子设备100支持的手势。NPU is a neural-network (NN) computing processor. By drawing on the structure of biological neural networks, for example, the transfer mode between human brain neurons, it can quickly process input information, and it can also continuously self-learn. Through the NPU, applications such as intelligent cognition of the electronic device 100 can be realized, such as image recognition, face recognition, voice recognition, text understanding, and so on. In the embodiment of the present application, the NPU can be used to process the image collected by the camera 193, and analyze whether the gesture contained in the image is the initial part of the gesture supported by the electronic device 100, or whether the gesture contained in the image is supported by the electronic device 100 gesture.

外部存储器接口120可以用于连接外部存储卡，例如Micro SD卡，实现扩展电子设备100的存储能力。外部存储卡通过外部存储器接口120与处理器110通信，实现数据存储功能。例如将音乐，视频等文件保存在外部存储卡中。Theexternal memory interface 120 may be used to connect an external memory card, such as a Micro SD card, to expand the storage capacity of the electronic device 100. The external memory card communicates with theprocessor 110 through theexternal memory interface 120 to realize the data storage function. For example, save music, video and other files in an external memory card.

内部存储器121可以用于存储计算机可执行程序代码，所述可执行程序代码包括指令。内部存储器121可以包括存储程序区和存储数据区。其中，存储程序区可存储操作系统，至少一个功能所需的应用程序(比如声音播放功能，图像播放功能等)等。存储数据区可存储 电子设备100使用过程中所创建的数据(比如音频数据，电话本等)等。此外，内部存储器121可以包括高速随机存取存储器，还可以包括非易失性存储器，例如至少一个磁盘存储器件，闪存器件，通用闪存存储器(universal flash storage，UFS)等。处理器110通过运行存储在内部存储器121的指令，和/或存储在设置于处理器中的存储器的指令，执行电子设备100的各种功能应用以及数据处理。本申请实施例中，内部存储器121可以用于存储电子设备100支持的手势的模型，以及各个支持的手势对应的功能。可能地，不同的应用程序的应用界面中，支持的手势可能不同，相同的手势实现的功能也可能不同。The internal memory 121 may be used to store computer executable program code, where the executable program code includes instructions. The internal memory 121 may include a storage program area and a storage data area. Among them, the storage program area can store an operating system, at least one application program (such as a sound playback function, an image playback function, etc.) required by at least one function. The data storage area can store data (such as audio data, phone book, etc.) created during the use of the electronic device 100. In addition, the internal memory 121 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one magnetic disk storage device, a flash memory device, a universal flash storage (UFS), and the like. Theprocessor 110 executes various functional applications and data processing of the electronic device 100 by running instructions stored in the internal memory 121 and/or instructions stored in a memory provided in the processor. In the embodiment of the present application, the internal memory 121 may be used to store the model of the gesture supported by the electronic device 100 and the function corresponding to each supported gesture. Possibly, in the application interface of different applications, the gestures supported may be different, and the functions implemented by the same gesture may also be different.

电子设备100可以通过音频模块170，扬声器170A，受话器170B，麦克风170C，耳机接口170D，以及应用处理器等实现音频功能。例如音乐播放，录音等。The electronic device 100 can implement audio functions through theaudio module 170, thespeaker 170A, thereceiver 170B, themicrophone 170C, theearphone interface 170D, and the application processor. For example, music playback, recording, etc.

音频模块170用于将数字音频信息转换成模拟音频信号输出，也用于将模拟音频输入转换为数字音频信号。音频模块170还可以用于对音频信号编码和解码。在一些实施例中，音频模块170可以设置于处理器110中，或将音频模块170的部分功能模块设置于处理器110中。Theaudio module 170 is used to convert digital audio information into an analog audio signal for output, and is also used to convert an analog audio input into a digital audio signal. Theaudio module 170 can also be used to encode and decode audio signals. In some embodiments, theaudio module 170 may be provided in theprocessor 110, or part of the functional modules of theaudio module 170 may be provided in theprocessor 110.

扬声器170A，也称“喇叭”，用于将音频电信号转换为声音信号。电子设备100可以通过扬声器170A收听音乐，或收听免提通话。Thespeaker 170A, also called "speaker", is used to convert audio electrical signals into sound signals. The electronic device 100 can listen to music through thespeaker 170A, or listen to a hands-free call.

受话器170B，也称“听筒”，用于将音频电信号转换成声音信号。当电子设备100接听电话或语音信息时，可以通过将受话器170B靠近人耳接听语音。Thereceiver 170B, also called "earpiece", is used to convert audio electrical signals into sound signals. When the electronic device 100 answers a call or voice message, it can receive the voice by bringing thereceiver 170B close to the human ear.

麦克风170C，也称“话筒”，“传声器”，用于将声音信号转换为电信号。当拨打电话或发送语音信息时，用户可以通过人嘴靠近麦克风170C发声，将声音信号输入到麦克风170C。电子设备100可以设置至少一个麦克风170C。在另一些实施例中，电子设备100可以设置两个麦克风170C，除了采集声音信号，还可以实现降噪功能。在另一些实施例中，电子设备100还可以设置三个，四个或更多麦克风170C，实现采集声音信号，降噪，还可以识别声音来源，实现定向录音功能等。Themicrophone 170C, also called "microphone", "microphone", is used to convert sound signals into electrical signals. When making a call or sending a voice message, the user can make a sound by approaching themicrophone 170C through the human mouth, and input the sound signal into themicrophone 170C. The electronic device 100 may be provided with at least onemicrophone 170C. In other embodiments, the electronic device 100 may be provided with twomicrophones 170C, which can implement noise reduction functions in addition to collecting sound signals. In other embodiments, the electronic device 100 may also be provided with three, four ormore microphones 170C to collect sound signals, reduce noise, identify sound sources, and realize directional recording functions.

耳机接口170D用于连接有线耳机。耳机接口170D可以是USB接口130，也可以是3.5mm的开放移动电子设备平台(open mobile terminal platform，OMTP)标准接口，美国蜂窝电信工业协会(cellular telecommunications industry association of the USA，CTIA)标准接口。Theearphone interface 170D is used to connect wired earphones. Theearphone interface 170D may be a USB interface 130, or a 3.5mm open mobile terminal platform (OMTP) standard interface, or a cellular telecommunications industry association of the USA (CTIA) standard interface.

压力传感器180A用于感受压力信号，可以将压力信号转换成电信号。在一些实施例中，压力传感器180A可以设置于显示屏194。压力传感器180A的种类很多，如电阻式压力传感器，电感式压力传感器，电容式压力传感器等。电容式压力传感器可以是包括至少两个具有导电材料的平行板。当有力作用于压力传感器180A，电极之间的电容改变。电子设备100根据电容的变化确定压力的强度。当有触摸操作作用于显示屏194，电子设备100根据压力传感器180A检测所述触摸操作强度。电子设备100也可以根据压力传感器180A的检测信号计算触摸的位置。在一些实施例中，作用于相同触摸位置，但不同触摸操作强度的触摸操作，可以对应不同的操作指令。例如：当有触摸操作强度小于第一压力阈值的触摸操作作用于短消息应用图标时，执行查看短消息的指令。当有触摸操作强度大于或等于第一压力阈值的触摸操作作用于短消息应用图标时，执行新建短消息的指令。The pressure sensor 180A is used to sense the pressure signal and can convert the pressure signal into an electrical signal. In some embodiments, the pressure sensor 180A may be provided on the display screen 194. There are many types of pressure sensors 180A, such as resistive pressure sensors, inductive pressure sensors, capacitive pressure sensors and so on. The capacitive pressure sensor may include at least two parallel plates with conductive material. When a force is applied to the pressure sensor 180A, the capacitance between the electrodes changes. The electronic device 100 determines the intensity of the pressure according to the change in capacitance. When a touch operation acts on the display screen 194, the electronic device 100 detects the intensity of the touch operation according to the pressure sensor 180A. The electronic device 100 may also calculate the touched position according to the detection signal of the pressure sensor 180A. In some embodiments, touch operations that act on the same touch position but have different touch operation strengths may correspond to different operation instructions. For example, when a touch operation whose intensity of the touch operation is less than the first pressure threshold is applied to the short message application icon, an instruction to view the short message is executed. When a touch operation with a touch operation intensity greater than or equal to the first pressure threshold acts on the short message application icon, an instruction to create a new short message is executed.

陀螺仪传感器180B可以用于确定电子设备100的运动姿态。在一些实施例中，可以通过陀螺仪传感器180B确定电子设备100围绕三个轴(即，x，y和z轴)的角速度。陀螺仪传 感器180B可以用于拍摄防抖。示例性的，当按下快门，陀螺仪传感器180B检测电子设备100抖动的角度，根据角度计算出镜头模组需要补偿的距离，让镜头通过反向运动抵消电子设备100的抖动，实现防抖。陀螺仪传感器180B还可以用于导航，体感游戏场景。The gyro sensor 180B may be used to determine the movement posture of the electronic device 100. In some embodiments, the angular velocity of the electronic device 100 around three axes (ie, x, y, and z axes) can be determined by the gyro sensor 180B. The gyro sensor 180B can be used for shooting anti-shake. Exemplarily, when the shutter is pressed, the gyro sensor 180B detects the shake angle of the electronic device 100, calculates the distance that the lens module needs to compensate according to the angle, and allows the lens to counteract the shake of the electronic device 100 through reverse movement to achieve anti-shake. The gyro sensor 180B can also be used for navigation and somatosensory game scenes.

气压传感器180C用于测量气压。在一些实施例中，电子设备100通过气压传感器180C测得的气压值计算海拔高度，辅助定位和导航。The air pressure sensor 180C is used to measure air pressure. In some embodiments, the electronic device 100 calculates the altitude based on the air pressure value measured by the air pressure sensor 180C to assist positioning and navigation.

磁传感器180D包括霍尔传感器。电子设备100可以利用磁传感器180D检测翻盖皮套的开合。在一些实施例中，当电子设备100是翻盖机时，电子设备100可以根据磁传感器180D检测翻盖的开合。进而根据检测到的皮套的开合状态或翻盖的开合状态，设置翻盖自动解锁等特性。The magnetic sensor 180D includes a Hall sensor. The electronic device 100 may use the magnetic sensor 180D to detect the opening and closing of the flip holster. In some embodiments, when the electronic device 100 is a flip machine, the electronic device 100 can detect the opening and closing of the flip according to the magnetic sensor 180D. Furthermore, according to the detected opening and closing state of the holster or the opening and closing state of the flip cover, features such as automatic unlocking of the flip cover are set.

加速度传感器180E可检测电子设备100在各个方向上(一般为三轴)加速度的大小。当电子设备100静止时可检测出重力的大小及方向。还可以用于识别电子设备姿态，应用于横竖屏切换，计步器等应用。The acceleration sensor 180E can detect the magnitude of the acceleration of the electronic device 100 in various directions (generally three axes). When the electronic device 100 is stationary, the magnitude and direction of gravity can be detected. It can also be used to identify the posture of electronic devices, and apply to applications such as horizontal and vertical screen switching, pedometers and so on.

距离传感器180F，用于测量距离。电子设备100可以通过红外或激光测量距离。在一些实施例中，拍摄场景，电子设备100可以利用距离传感器180F测距以实现快速对焦。Distance sensor 180F, used to measure distance. The electronic device 100 can measure the distance by infrared or laser. In some embodiments, when shooting a scene, the electronic device 100 may use the distance sensor 180F to measure the distance to achieve fast focusing.

接近光传感器180G可以包括例如发光二极管(LED)和光检测器，例如光电二极管。发光二极管可以是红外发光二极管。电子设备100通过发光二极管向外发射红外光。电子设备100使用光电二极管检测来自附近物体的红外反射光。当检测到充分的反射光时，可以确定电子设备100附近有物体。当检测到不充分的反射光时，电子设备100可以确定电子设备100附近没有物体。电子设备100可以利用接近光传感器180G检测用户手持电子设备100贴近耳朵通话，以便自动熄灭屏幕达到省电的目的。接近光传感器180G也可用于皮套模式，口袋模式自动解锁与锁屏。The proximity light sensor 180G may include, for example, a light emitting diode (LED) and a light detector such as a photodiode. The light emitting diode may be an infrared light emitting diode. The electronic device 100 emits infrared light to the outside through the light emitting diode. The electronic device 100 uses a photodiode to detect infrared reflected light from nearby objects. When sufficient reflected light is detected, it can be determined that there is an object near the electronic device 100. When insufficient reflected light is detected, the electronic device 100 can determine that there is no object near the electronic device 100. The electronic device 100 can use the proximity light sensor 180G to detect that the user holds the electronic device 100 close to the ear to talk, so as to automatically turn off the screen to save power. The proximity light sensor 180G can also be used in leather case mode, and the pocket mode will automatically unlock and lock the screen.

环境光传感器180L用于感知环境光亮度。电子设备100可以根据感知的环境光亮度自适应调节显示屏194亮度。环境光传感器180L也可用于拍照时自动调节白平衡。环境光传感器180L还可以与接近光传感器180G配合，检测电子设备100是否在口袋里，以防误触。The ambient light sensor 180L is used to sense the brightness of the ambient light. The electronic device 100 can adaptively adjust the brightness of the display screen 194 according to the perceived brightness of the ambient light. The ambient light sensor 180L can also be used to automatically adjust the white balance when taking pictures. The ambient light sensor 180L can also cooperate with the proximity light sensor 180G to detect whether the electronic device 100 is in the pocket to prevent accidental touch.

指纹传感器180H用于采集指纹。电子设备100可以利用采集的指纹特性实现指纹解锁，访问应用锁，指纹拍照，指纹接听来电等。The fingerprint sensor 180H is used to collect fingerprints. The electronic device 100 can use the collected fingerprint characteristics to realize fingerprint unlocking, access application locks, fingerprint photographs, fingerprint answering calls, and so on.

温度传感器180J用于检测温度。在一些实施例中，电子设备100利用温度传感器180J检测的温度，执行温度处理策略。例如，当温度传感器180J上报的温度超过阈值，电子设备100执行降低位于温度传感器180J附近的处理器的性能，以便降低功耗实施热保护。在另一些实施例中，当温度低于另一阈值时，电子设备100对电池142加热，以避免低温导致电子设备100异常关机。在其他一些实施例中，当温度低于又一阈值时，电子设备100对电池142的输出电压执行升压，以避免低温导致的异常关机。The temperature sensor 180J is used to detect temperature. In some embodiments, the electronic device 100 uses the temperature detected by the temperature sensor 180J to execute a temperature processing strategy. For example, when the temperature reported by the temperature sensor 180J exceeds a threshold value, the electronic device 100 reduces the performance of the processor located near the temperature sensor 180J, so as to reduce power consumption and implement thermal protection. In other embodiments, when the temperature is lower than another threshold, the electronic device 100 heats the battery 142 to avoid abnormal shutdown of the electronic device 100 due to low temperature. In some other embodiments, when the temperature is lower than another threshold, the electronic device 100 boosts the output voltage of the battery 142 to avoid abnormal shutdown caused by low temperature.

触摸传感器180K，也称“触控器件”。触摸传感器180K可以设置于显示屏194，由触摸传感器180K与显示屏194组成触摸屏，也称“触控屏”。触摸传感器180K用于检测作用于其上或附近的触摸操作。触摸传感器可以将检测到的触摸操作传递给应用处理器，以确定触摸事件类型。可以通过显示屏194提供与触摸操作相关的视觉输出。在另一些实施例中，触摸传感器180K也可以设置于电子设备100的表面，与显示屏194所处的位置不同。Touch sensor 180K, also called "touch device". The touch sensor 180K may be disposed on the display screen 194, and the touch screen is composed of the touch sensor 180K and the display screen 194, which is also called a “touch screen”. The touch sensor 180K is used to detect touch operations acting on or near it. The touch sensor can pass the detected touch operation to the application processor to determine the type of touch event. The visual output related to the touch operation can be provided through the display screen 194. In other embodiments, the touch sensor 180K may also be disposed on the surface of the electronic device 100, which is different from the position of the display screen 194.

骨传导传感器180M可以获取振动信号。在一些实施例中，骨传导传感器180M可以获取人体声部振动骨块的振动信号。骨传导传感器180M也可以接触人体脉搏，接收血压跳动信号。在一些实施例中，骨传导传感器180M也可以设置于耳机中，结合成骨传导耳机。音频模块170可以基于所述骨传导传感器180M获取的声部振动骨块的振动信号，解析出语音信号，实现语音功能。应用处理器可以基于所述骨传导传感器180M获取的血压跳动信号解析心率信息，实现心率检测功能。The bone conduction sensor 180M can acquire vibration signals. In some embodiments, the bone conduction sensor 180M can obtain the vibration signal of the vibrating bone mass of the human voice. The bone conduction sensor 180M can also contact the human pulse and receive the blood pressure pulse signal. In some embodiments, the bone conduction sensor 180M may also be provided in the earphone, combined with the bone conduction earphone. Theaudio module 170 can parse the voice signal based on the vibration signal of the vibrating bone block of the voice obtained by the bone conduction sensor 180M, and realize the voice function. The application processor can analyze the heart rate information based on the blood pressure beating signal obtained by the bone conduction sensor 180M, and realize the heart rate detection function.

按键190包括开机键，音量键等。按键190可以是机械按键。也可以是触摸式按键。电子设备100可以接收按键输入，产生与电子设备100的用户设置以及功能控制有关的键信号输入。The button 190 includes a power-on button, a volume button, and so on. The button 190 may be a mechanical button. It can also be a touch button. The electronic device 100 may receive key input, and generate key signal input related to user settings and function control of the electronic device 100.

马达191可以产生振动提示。马达191可以用于来电振动提示，也可以用于触摸振动反馈。例如，作用于不同应用(例如拍照，音频播放等)的触摸操作，可以对应不同的振动反馈效果。作用于显示屏194不同区域的触摸操作，马达191也可对应不同的振动反馈效果。不同的应用场景(例如：时间提醒，接收信息，闹钟，游戏等)也可以对应不同的振动反馈效果。触摸振动反馈效果还可以支持自定义。The motor 191 can generate vibration prompts. The motor 191 can be used for incoming call vibration notification, and can also be used for touch vibration feedback. For example, touch operations that act on different applications (such as photographing, audio playback, etc.) can correspond to different vibration feedback effects. Acting on touch operations in different areas of the display screen 194, the motor 191 can also correspond to different vibration feedback effects. Different application scenarios (for example: time reminding, receiving information, alarm clock, games, etc.) can also correspond to different vibration feedback effects. The touch vibration feedback effect can also support customization.

指示器192可以是指示灯，可以用于指示充电状态，电量变化，也可以用于指示消息，未接来电，通知等。The indicator 192 may be an indicator light, which may be used to indicate the charging status, power change, or to indicate messages, missed calls, notifications, and so on.

SIM卡接口195用于连接SIM卡。SIM卡可以通过插入SIM卡接口195，或从SIM卡接口195拔出，实现和电子设备100的接触和分离。电子设备100可以支持1个或N个SIM卡接口，N为大于1的正整数。SIM卡接口195可以支持Nano SIM卡，Micro SIM卡，SIM卡等。同一个SIM卡接口195可以同时插入多张卡。所述多张卡的类型可以相同，也可以不同。SIM卡接口195也可以兼容不同类型的SIM卡。SIM卡接口195也可以兼容外部存储卡。电子设备100通过SIM卡和网络交互，实现通话以及数据通信等功能。在一些实施例中，电子设备100采用eSIM，即：嵌入式SIM卡。eSIM卡可以嵌在电子设备100中，不能和电子设备100分离。TheSIM card interface 195 is used to connect to the SIM card. The SIM card can be inserted into theSIM card interface 195 or pulled out from theSIM card interface 195 to achieve contact and separation with the electronic device 100. The electronic device 100 may support 1 or N SIM card interfaces, and N is a positive integer greater than 1. TheSIM card interface 195 can support Nano SIM cards, Micro SIM cards, SIM cards, etc. The sameSIM card interface 195 can insert multiple cards at the same time. The types of the multiple cards can be the same or different. TheSIM card interface 195 can also be compatible with different types of SIM cards. TheSIM card interface 195 may also be compatible with external memory cards. The electronic device 100 interacts with the network through the SIM card to implement functions such as call and data communication. In some embodiments, the electronic device 100 adopts an eSIM, that is, an embedded SIM card. The eSIM card can be embedded in the electronic device 100 and cannot be separated from the electronic device 100.

电子设备100的软件系统可以采用分层架构，事件驱动架构，微核架构，微服务架构，或云架构。本发明实施例以分层架构的Android系统为例，示例性说明电子设备100的软件结构。The software system of the electronic device 100 may adopt a layered architecture, an event-driven architecture, a microkernel architecture, a microservice architecture, or a cloud architecture. The embodiment of the present invention takes an Android system with a layered architecture as an example to illustrate the software structure of the electronic device 100 by way of example.

图2是本发明实施例的电子设备100的软件结构框图。FIG. 2 is a block diagram of the software structure of the electronic device 100 according to an embodiment of the present invention.

分层架构将软件分成若干个层，每一层都有清晰的角色和分工。层与层之间通过软件接口通信。在一些实施例中，将Android系统分为四层，从上至下分别为应用程序层，应用程序框架层，安卓运行时(Android runtime)和系统库，以及内核层。The layered architecture divides the software into several layers, and each layer has a clear role and division of labor. Communication between layers through software interface. In some embodiments, the Android system is divided into four layers, from top to bottom, the application layer, the application framework layer, the Android runtime and system library, and the kernel layer.

应用程序层可以包括一系列应用程序包。The application layer can include a series of application packages.

如图2所示，应用程序包可以包括相机，图库，日历，通话，地图，导航，WLAN，蓝牙，音乐，视频，短信息等应用程序。As shown in Figure 2, the application package can include applications such as camera, gallery, calendar, call, map, navigation, WLAN, Bluetooth, music, video, short message, etc.

应用程序框架层为应用程序层的应用程序提供应用编程接口(application programming interface，API)和编程框架。应用程序框架层包括一些预先定义的函数。The application framework layer provides an application programming interface (application programming interface, API) and a programming framework for applications in the application layer. The application framework layer includes some predefined functions.

如图2所示，应用程序框架层可以包括窗口管理器，内容提供器，视图系统，电话管 理器，资源管理器，通知管理器等。As shown in Figure 2, the application framework layer can include a window manager, a content provider, a view system, a phone manager, a resource manager, and a notification manager.

窗口管理器用于管理窗口程序。窗口管理器可以获取显示屏大小，判断是否有状态栏，锁定屏幕，截取屏幕等。The window manager is used to manage window programs. The window manager can obtain the size of the display, determine whether there is a status bar, lock the screen, take a screenshot, etc.

内容提供器用来存放和获取数据，并使这些数据可以被应用程序访问。所述数据可以包括视频，图像，音频，拨打和接听的电话，浏览历史和书签，电话簿等。The content provider is used to store and retrieve data and make these data accessible to applications. The data may include videos, images, audios, phone calls made and received, browsing history and bookmarks, phone book, etc.

视图系统包括可视控件，例如显示文字的控件，显示图片的控件等。视图系统可用于构建应用程序。显示界面可以由一个或多个视图组成的。例如，包括短信通知图标的显示界面，可以包括显示文字的视图以及显示图片的视图。The view system includes visual controls, such as controls that display text, controls that display pictures, and so on. The view system can be used to build applications. The display interface can be composed of one or more views. For example, a display interface that includes a short message notification icon may include a view that displays text and a view that displays pictures.

电话管理器用于提供电子设备100的通信功能。例如通话状态的管理(包括接通，挂断等)。The phone manager is used to provide the communication function of the electronic device 100. For example, the management of the call status (including connecting, hanging up, etc.).

资源管理器为应用程序提供各种资源，比如本地化字符串，图标，图片，布局文件，视频文件等等。The resource manager provides various resources for the application, such as localized strings, icons, pictures, layout files, video files, and so on.

通知管理器使应用程序可以在状态栏中显示通知信息，可以用于传达告知类型的消息，可以短暂停留后自动消失，无需用户交互。比如通知管理器被用于告知下载完成，消息提醒等。通知管理器还可以是以图表或者滚动条文本形式出现在系统顶部状态栏的通知，例如后台运行的应用程序的通知，还可以是以对话窗口形式出现在屏幕上的通知。例如在状态栏提示文本信息，发出提示音，电子设备振动，指示灯闪烁等。The notification manager enables the application to display notification information in the status bar, which can be used to convey notification-type messages, and it can disappear automatically after a short stay without user interaction. For example, the notification manager is used to notify download completion, message reminders, and so on. The notification manager can also be a notification that appears in the status bar at the top of the system in the form of a chart or a scroll bar text, such as a notification of an application running in the background, or a notification that appears on the screen in the form of a dialog window. For example, text messages are prompted in the status bar, prompt sounds, electronic devices vibrate, and indicator lights flash.

Android Runtime包括核心库和虚拟机。Android runtime负责安卓系统的调度和管理。Android Runtime includes core libraries and virtual machines. Android runtime is responsible for the scheduling and management of the Android system.

核心库包含两部分：一部分是java语言需要调用的功能函数，另一部分是安卓的核心库。The core library consists of two parts: one part is the function functions that the java language needs to call, and the other part is the core library of Android.

应用程序层和应用程序框架层运行在虚拟机中。虚拟机将应用程序层和应用程序框架层的java文件执行为二进制文件。虚拟机用于执行对象生命周期的管理，堆栈管理，线程管理，安全和异常的管理，以及垃圾回收等功能。The application layer and the application framework layer run in a virtual machine. The virtual machine executes the java files of the application layer and the application framework layer as binary files. The virtual machine is used to perform functions such as object life cycle management, stack management, thread management, security and exception management, and garbage collection.

系统库可以包括多个功能模块。例如：表面管理器(surface manager)，媒体库(Media Libraries)，三维图形处理库(例如：OpenGL ES)，2D图形引擎(例如：SGL)等。The system library can include multiple functional modules. For example: surface manager (surface manager), media library (Media Libraries), three-dimensional graphics processing library (for example: OpenGL ES), 2D graphics engine (for example: SGL), etc.

表面管理器用于对显示子系统进行管理，并且为多个应用程序提供了2D和3D图层的融合。The surface manager is used to manage the display subsystem and provides a combination of 2D and 3D layers for multiple applications.

媒体库支持多种常用的音频，视频格式回放和录制，以及静态图像文件等。媒体库可以支持多种音视频编码格式，例如:MPEG4，H.264，MP3，AAC，AMR，JPG，PNG等。The media library supports playback and recording of a variety of commonly used audio and video formats, as well as still image files. The media library can support a variety of audio and video encoding formats, such as: MPEG4, H.264, MP3, AAC, AMR, JPG, PNG, etc.

三维图形处理库用于实现三维图形绘图，图像渲染，合成，和图层处理等。The 3D graphics processing library is used to realize 3D graphics drawing, image rendering, synthesis, and layer processing.

2D图形引擎是2D绘图的绘图引擎。The 2D graphics engine is a drawing engine for 2D drawing.

内核层是硬件和软件之间的层。内核层至少包含显示驱动，摄像头驱动，音频驱动，传感器驱动。The kernel layer is the layer between hardware and software. The kernel layer contains at least display driver, camera driver, audio driver, and sensor driver.

下面结合捕获拍照场景，示例性说明电子设备100软件以及硬件的工作流程。In the following, the workflow of the software and hardware of the electronic device 100 will be exemplified in conjunction with capturing a photo scene.

当触摸传感器180K接收到触摸操作，相应的硬件中断被发给内核层。内核层将触摸操作加工成原始输入事件(包括触摸坐标，触摸操作的时间戳等信息)。原始输入事件被存储在内核层。应用程序框架层从内核层获取原始输入事件，识别该输入事件所对应的控件。以该触摸操作是触摸单击操作，该单击操作所对应的控件为相机应用图标的控件为例，相 机应用调用应用框架层的接口，启动相机应用，进而通过调用内核层启动摄像头驱动，通过摄像头193捕获静态图像或视频。When the touch sensor 180K receives a touch operation, the corresponding hardware interrupt is sent to the kernel layer. The kernel layer processes the touch operation into the original input event (including touch coordinates, time stamp of the touch operation, etc.). The original input events are stored in the kernel layer. The application framework layer obtains the original input event from the kernel layer and identifies the control corresponding to the input event. Taking the touch operation as a touch click operation, and the control corresponding to the click operation is the control of the camera application icon as an example, the camera application calls the interface of the application framework layer to start the camera application, and then starts the camera driver by calling the kernel layer. The camera 193 captures still images or videos.

图3示例性示出了一种隔空输入手势的示意图。如图3所示，用户可以在电子设备100前置摄像头193前方的预设范围如5厘米-30厘米内做出功能手势的起始部分，功能手势即为电子设备100支持的手势。Fig. 3 exemplarily shows a schematic diagram of an air input gesture. As shown in FIG. 3, the user can make the initial part of the functional gesture within a preset range in front of the front camera 193 of the electronic device 100 such as 5 cm-30 cm. The functional gesture is a gesture supported by the electronic device 100.

前置摄像头193采集的图像的数据可以是RGB数据或灰度数据等。The image data collected by the front camera 193 may be RGB data or grayscale data.

不限于由前置摄像头193采集图像，在具体实现中还可以通过深度摄像头或红外传感器采集用户的手势的深度数据，此时保存在内部存储器121中的功能手势的数据为深度数据，即为该功能手势的三维模型。深度摄像头例如可以是飞行时间(time of flight，tof)摄像头。本申请实施例对此不作限定。不限于此，在具体实现中，还可以通过雷达传感器采集用户的手势的深度数据。It is not limited to the image collection by the front camera 193. In a specific implementation, the depth data of the user’s gesture can also be collected through a depth camera or infrared sensor. At this time, the data of the functional gesture stored in the internal memory 121 is the depth data, which is the Three-dimensional model of functional gestures. The depth camera may be, for example, a time of flight (tof) camera. The embodiment of the application does not limit this. Not limited to this, in a specific implementation, the depth data of the user's gesture can also be collected through a radar sensor.

图4示例性示出了本申请实施例提供的另外一种电子设备的结构示意图。如图4所示，电子设备可200以包括亮屏检测模块210、图像采集模块220、低功耗动作检测模块230、手势识别模块240及执行模块250。Fig. 4 exemplarily shows a schematic structural diagram of another electronic device provided by an embodiment of the present application. As shown in FIG. 4, the electronic device 200 may include a bright screen detection module 210, an image acquisition module 220, a low-power motion detection module 230, a gesture recognition module 240, and an execution module 250.

具体地，亮屏检测模块210可以用于检测电子设备200是否处于亮屏状态，即电子设备200的显示屏是否被点亮。若电子设备200处于亮屏状态，则图像采集模块220开始以第一帧率采集多帧图像。将采集到的多帧图像以帧为单位作为低功耗动作检测模块230的输入。若低功耗动作检测模块230输出的结果为多帧图像中存在支持的手势的起始部分，则图像采集模块220以第三帧率采集图像流，并作为手势识别模块240的输入。手势识别模块240输出的结果可以是功能手势的识别结果。执行模块250可以根据手势识别模块240输出的识别结果执行该功能手势相应的事件。其中，图像流可以包括多帧连续的图像。第一帧率和第三帧率为图像采集模块220采集图像的频率。Specifically, the bright screen detection module 210 may be used to detect whether the electronic device 200 is in a bright screen state, that is, whether the display screen of the electronic device 200 is lit. If the electronic device 200 is in the bright screen state, the image acquisition module 220 starts to acquire multiple frames of images at the first frame rate. The captured multi-frame images are taken as the input of the low power consumption action detection module 230 in units of frames. If the output result of the low-power motion detection module 230 is that the initial part of the supported gesture exists in the multi-frame image, the image acquisition module 220 acquires the image stream at the third frame rate and serves as the input of the gesture recognition module 240. The result output by the gesture recognition module 240 may be a recognition result of a functional gesture. The execution module 250 may execute an event corresponding to the functional gesture according to the recognition result output by the gesture recognition module 240. Among them, the image stream may include multiple frames of continuous images. The first frame rate and the third frame rate are the frequency at which the image acquisition module 220 acquires images.

其中，第一帧率小于第三帧率。即在没有识别到手势时图像获取单元220可以以第一帧率采集多帧图像，在确定输入的手势为功能手势的起始手势时以第三帧率采集图像流，既可以保证电子设备200识别手势并做出响应的速率，又可以在最大程度上节约电子设备200的功耗，尽可能的提升电子设备200的续航能力。上述第一帧率例如可以是10帧/秒，第三帧率例如可以是30帧/秒。Wherein, the first frame rate is less than the third frame rate. That is, when the gesture is not recognized, the image acquisition unit 220 can acquire multiple frames of images at the first frame rate, and when it is determined that the input gesture is the initial gesture of the functional gesture, the image stream is acquired at the third frame rate, which can ensure that the electronic device 200 The rate of recognizing gestures and responding can also save the power consumption of the electronic device 200 to the greatest extent, and improve the endurance of the electronic device 200 as much as possible. The above-mentioned first frame rate may be, for example, 10 frames/sec, and the third frame rate may be, for example, 30 frames/sec.

其中，图像采集模块220可以是图3中提到的前置摄像头193、深度摄像头、红外传感器或雷达传感器等。The image acquisition module 220 may be the front camera 193, depth camera, infrared sensor or radar sensor mentioned in FIG. 3.

具体地，上述低功耗动作检测模块230、手势识别模块240均可为由NPU执行的算法实现。Specifically, the aforementioned low-power consumption action detection module 230 and gesture recognition module 240 may both be implemented by algorithms executed by the NPU.

具体地，低功耗动作检测模块230可以包括第一神经网络模型和后处理模块。其中，第一神经网络模型可以用于识别输入的图片中包含的手势是否为起始手势。电子设备100可以将图像采集模块220采集的包含手势的图片输入到该模型中，由该模型输出起始手势的识别结果。识别结果可以但不限于用0或1来表征。例如，当识别结果为1时，表征输入的手势为起始手势；当识别结果为0时，表征输入的手势不是起始手势。后处理模块可 以用于处理第一神经网络模型输出的多帧图像中每帧图像各自对应的识别结果，并根据上述识别结果判断是否需要唤醒手势识别模块240。Specifically, the low power consumption action detection module 230 may include a first neural network model and a post-processing module. Among them, the first neural network model can be used to identify whether the gesture contained in the input picture is the initial gesture. The electronic device 100 may input the picture containing the gesture collected by the image acquisition module 220 into the model, and the model will output the recognition result of the initial gesture. The recognition result can be but not limited to be characterized by 0 or 1. For example, when the recognition result is 1, the gesture that characterizes the input is the initial gesture; when the recognition result is 0, the gesture that characterizes the input is not the initial gesture. The post-processing module can be used to process the recognition results corresponding to each frame of the multiple frames of images output by the first neural network model, and determine whether the gesture recognition module 240 needs to be awakened according to the aforementioned recognition results.

具体地，手势识别模块240可以包括第二神经网络模型。其中，第二神经网络模型可以用于识别输入的图像流中可能包含的手势以及各个手势对应的概率。电子设备100可以将图像采集模块220采集的图像流输入到该模型中，由该模型输出手势识别的结果。识别结果可以包括多种可能的功能手势以及各个功能手势的概率。Specifically, the gesture recognition module 240 may include a second neural network model. Among them, the second neural network model can be used to identify gestures that may be included in the input image stream and the probability corresponding to each gesture. The electronic device 100 can input the image stream collected by the image collection module 220 into the model, and the model outputs the result of gesture recognition. The recognition result may include multiple possible functional gestures and the probability of each functional gesture.

具体地，上述执行模块250可以是CPU。Specifically, the above-mentioned execution module 250 may be a CPU.

可能地，功能手势的识别结果可以用于表征用户输入的功能手势具体是什么，例如是手腕上翻还是手腕下翻。CPU可以根据功能手势的识别结果确定该功能手势对应的事件，并执行该事件。Possibly, the recognition result of the functional gesture can be used to characterize what the functional gesture input by the user is, for example, whether the wrist is turned up or down. The CPU may determine the event corresponding to the functional gesture according to the recognition result of the functional gesture, and execute the event.

可能地，功能手势的识别结果可以用于表征用户输入的功能手势对应的事件，CPU可以执行该事件。Possibly, the recognition result of the functional gesture can be used to characterize the event corresponding to the functional gesture input by the user, and the CPU can execute the event.

可能地，功能手势的识别结果可以表征用户输入的手势不属于任意一种功能手势，则手势识别模块240可以不将识别结果发送给CPU。或者手势识别模块240可以将识别结果发送给CPU，但CPU可以不作任何响应。Possibly, the recognition result of the functional gesture may indicate that the gesture input by the user does not belong to any kind of functional gesture, and the gesture recognition module 240 may not send the recognition result to the CPU. Or the gesture recognition module 240 may send the recognition result to the CPU, but the CPU may not make any response.

可能地，若执行模块250执行某功能手势对应的事件后，电子设备100的图像采集模块220在一段时间内(如1秒、2秒、5秒等)没有检测到任何手势，则图像采集模块220可以将帧率从第三帧率降低为第一帧率以节省功耗开销。Possibly, if the execution module 250 executes an event corresponding to a certain functional gesture, the image acquisition module 220 of the electronic device 100 does not detect any gesture within a period of time (such as 1 second, 2 seconds, 5 seconds, etc.), then the image acquisition module 220 can reduce the frame rate from the third frame rate to the first frame rate to save power consumption overhead.

本申请实施例可以使用户隔空输入功能手势操作电子设备时，无需额外输入激活手势，即功能手势的起始手势可作为激活手势，输入的手势一气呵成，减小手势操作时延，且电子设备响应速度快，并避免由手势误触发造成的功耗浪费。The embodiment of the application can enable the user to input the functional gesture to operate the electronic device without additional input of the activation gesture, that is, the initial gesture of the functional gesture can be used as the activation gesture, and the input gesture can be completed in one go, reducing the gesture operation time delay, and the electronic device The response speed is fast, and the waste of power consumption caused by false triggering of gestures is avoided.

在一些可能的实施例中，电子设备200可以不包括上述亮屏检测模块210。也即是说，图像采集模块220可以在电子设备200熄屏的状态下采集图像流。In some possible embodiments, the electronic device 200 may not include the bright screen detection module 210 described above. That is to say, the image acquisition module 220 can acquire an image stream when the electronic device 200 is in a state where the screen is off.

可能地，电子设备200可以在熄屏的状态下确定采集的图像中存在手势，然后进行身份验证。在确定用户身份为机主后，触发手势识别流程。其中，身份验证的方式可以但不限于是掌纹识别、虹膜识别、指纹识别、声纹识别等。Possibly, the electronic device 200 may determine that there is a gesture in the captured image when the screen is turned off, and then perform identity verification. After confirming that the user is the owner, the gesture recognition process is triggered. Among them, the method of identity verification can be, but is not limited to, palmprint recognition, iris recognition, fingerprint recognition, voiceprint recognition, and so on.

接下来结合图4示出的电子设备的结构示意图，分别介绍两种图像采集的过程。图像采集过程的前提为电子设备为亮屏状态，以保证机主的隐私。以下这两种过程均以图像采集模块220为前置摄像头193为例进行说明。Next, in conjunction with the schematic structural diagram of the electronic device shown in FIG. 4, the two image acquisition processes are respectively introduced. The premise of the image acquisition process is that the electronic device is in a bright screen state to ensure the privacy of the owner. The following two processes are described by taking the image acquisition module 220 as the front camera 193 as an example.

首先参见图5。图5示例性示出了一种图像采集过程示意图。See Figure 5 first. Fig. 5 exemplarily shows a schematic diagram of an image acquisition process.

如图5所示，前置摄像头193在电子设备为亮屏状态时持续以第一帧率(如10帧/秒)采集多帧图像。前置摄像头193可以将以第一帧率采集到的图像输入到低功耗动作检测模块230中。假设低功耗动作检测模块230在第5帧图像中识别出有起始手势(即某功能手势的起始部分)，则切换帧率至第三帧率(如30帧/秒)，并进一步的根据接下来的几帧图像中是否依然存在某功能的起始手势。As shown in FIG. 5, the front camera 193 continuously collects multiple frames of images at the first frame rate (for example, 10 frames/sec) when the electronic device is in the on-screen state. The front camera 193 can input the image collected at the first frame rate into the low power consumption action detection module 230. Assuming that the low-power motion detection module 230 recognizes an initial gesture (that is, the initial part of a certain functional gesture) in the fifth frame of image, it switches the frame rate to the third frame rate (such as 30 frames per second), and further According to whether the initial gesture of a certain function still exists in the next few frames of images.

若根据接下来的几帧图像确定存在某功能手势的起始手势，则唤醒手势识别模块240对前置摄像头193采集的图像流进行识别。进一步确定前置摄像头193采集的图像流中是 否存在功能手势。若存在，则电子设备100执行与该功能手势相应的事件。If it is determined that there is an initial gesture of a certain functional gesture according to the next few frames of images, the wake-up gesture recognition module 240 recognizes the image stream collected by the front camera 193. It is further determined whether there are functional gestures in the image stream collected by the front camera 193. If it exists, the electronic device 100 executes an event corresponding to the functional gesture.

接下来依然结合图5，示例性地介绍低功耗动作检测模块230如何确定存在某功能手势的起始手势，以及手势识别模块240如何确定存在功能手势。Next, still referring to FIG. 5, how the low-power motion detection module 230 determines the initial gesture of a certain functional gesture, and how the gesture recognition module 240 determines that the functional gesture exists is exemplarily introduced.

关于低功耗动作检测模块230如何确定存在某功能手势的起始手势的过程示例性地如下：The process of how the low-power motion detection module 230 determines that there is an initial gesture of a certain functional gesture is exemplarily as follows:

具体地，低功耗动作检测模块230中的第一神经网络模型可以分别识别连续X帧图像的每帧图像是否存在某功能手势的起始手势。后处理模块可以根据上述连续X帧图像的识别结果判断是否需要唤醒手势识别模块240。若从当前帧起，之前的连续X帧(包括当前帧)中有超过x帧图像中均包含起始手势，则后处理模块可唤醒手势识别模块240。其中，X和x均为正整数，x小于或等于X。例如，X可以但不限于为7，x可以但不限于为5。若从当前帧起，之前的连续X帧(包括当前帧)中不超过x帧图像包含起始手势，则第一神经网络模型继续识别下一帧图像中是否存在起始手势，直至连续X帧图像中有超过x帧图像中均包括起始手势为止。Specifically, the first neural network model in the low-power motion detection module 230 can respectively identify whether there is an initial gesture of a certain functional gesture for each frame of continuous X frames of images. The post-processing module may determine whether the gesture recognition module 240 needs to be awakened according to the recognition result of the above-mentioned consecutive X frames of images. If, starting from the current frame, more than x frames of images in the previous consecutive X frames (including the current frame) all contain the initial gesture, the post-processing module can wake up the gesture recognition module 240. Among them, X and x are both positive integers, and x is less than or equal to X. For example, X can be but not limited to 7, and x can be but not limited to 5. If from the current frame, no more than x frames of images in the previous consecutive X frames (including the current frame) contain the initial gesture, the first neural network model continues to identify whether there is an initial gesture in the next frame of image, until X consecutive frames There are more than x frames in the image that include the initial gesture.

示例性地，若第一神经网络模型识别出第7帧-第13帧这7帧图像中，有5帧图像(如第7帧、第9帧、第10帧、第12帧、第13帧)中均包含起始手势，则后处理模块可唤醒手势识别模块240。其中，上述当前帧即为第13帧。Exemplarily, if the first neural network model recognizes that the 7th frame-the 13th frame, there are 5 frames of images (such as the 7th frame, the 9th frame, the 10th frame, the 12th frame, and the 13th frame). ) Includes the initial gesture, the post-processing module can wake up the gesture recognition module 240. Among them, the above-mentioned current frame is the 13th frame.

关于手势识别模块240如何确定存在功能手势的过程示例性地如下：The process of how the gesture recognition module 240 determines that there is a functional gesture is illustratively as follows:

若低功耗动作检测模块230在第13帧时刻确定包含的手势为功能手势的起始手势，则在采集第14帧图像时唤醒手势识别模块240。具体可以取第14帧-第23帧作为滑动窗口(以下简称滑窗)。滑窗的长度为10，即滑窗可包含10帧连续的图像，步进为1，即滑窗每次可往前移动一帧。手势识别模块240可以对每个滑窗包含的10帧图像(第14帧-第23帧)的内容进行分析，输出识别结果。即将滑窗包含的10帧图像输入到第二神经网络模型中，输出识别结果。识别结果具体可以是该滑窗包含的图像流中可能存在的各种手势以及各种手势对应概率。各个手势对应的概率和为1。If the low-power motion detection module 230 determines that the included gesture is the initial gesture of the functional gesture at the 13th frame time, the gesture recognition module 240 is awakened when the 14th frame image is collected. Specifically, frames 14 to 23 can be used as sliding windows (hereinafter referred to as sliding windows). The length of the sliding window is 10, that is, the sliding window can contain 10 consecutive images, and the step is 1, that is, the sliding window can move forward one frame at a time. The gesture recognition module 240 can analyze the content of 10 frames of images (frame 14-frame 23) contained in each sliding window, and output the recognition result. That is, the 10 frames of images contained in the sliding window are input into the second neural network model, and the recognition result is output. The recognition result may specifically be various gestures and corresponding probabilities of various gestures that may exist in the image stream contained in the sliding window. The sum of the probabilities corresponding to each gesture is 1.

不限于10帧，在具体实现中上述滑窗的长度还可以是其他值，本申请实施例对此不作限定。It is not limited to 10 frames. In a specific implementation, the length of the above-mentioned sliding window may also be other values, which is not limited in the embodiment of the present application.

如图5所示，可以将包含第14帧-第23帧图像的滑窗称为滑窗1，将包含第15帧-第24帧图像的滑窗称为滑窗2，将包含第16帧-第25帧图像的滑窗称为滑窗3。As shown in Figure 5, the sliding window containing images from frames 14 to 23 can be called slidingwindow 1, and the sliding window containing images from frames 15 to 24 is called sliding window 2, which will contain frame 16 -The sliding window of the 25th frame is called sliding window 3.

示例性地，手势识别模块240对滑窗1包含的图像流进行分析后，可以输出的结果为：手势1，概率50％；手势2，概率30％；手势3，20％。Exemplarily, after the gesture recognition module 240 analyzes the image stream contained in the slidingwindow 1, the results that can be output are:gesture 1, with a probability of 50%; gesture 2, with a probability of 30%; and gesture 3, with a probability of 20%.

手势识别模块240对滑窗2包含的图像流进行分析后，可以输出的结果为：手势1，概率60％；手势2，概率20％；手势3，15％；手势4，5％。After the gesture recognition module 240 analyzes the image stream contained in the sliding window 2, the results that can be output are:gesture 1, probability 60%; gesture 2, probability 20%; gesture 3, 15%; gesture 4, 5%.

将滑窗1和滑窗2输出的结果中各个手势对应的概率加权求和即可得到各个手势分别对应的综合概率，即滑窗2的综合结果。The weighted sum of the probabilities corresponding to each gesture in the results output by the slidingwindow 1 and the sliding window 2 can obtain the comprehensive probabilities corresponding to each gesture, that is, the comprehensive result of the sliding window 2.

假设滑窗1的权重为35％，滑窗2的权重为65％，则：Assuming that the weight of slidingwindow 1 is 35% and the weight of sliding window 2 is 65%, then:

手势1的概率：50％*35％+60％*65％＝56.5％；Probability of gesture 1: 50%*35%+60%*65%=56.5%;

手势2的概率：30％*35％+20％*65％＝23.5％；Probability of gesture 2: 30%*35%+20%*65%=23.5%;

手势3的概率：20％*35％+15％*65％＝16.75％；Probability of gesture 3: 20%*35%+15%*65%=16.75%;

手势4的概率：5％*65％＝3.25％。Probability of gesture 4: 5%*65%=3.25%.

递进地，滑窗3的综合结果可以是将滑窗3输出的结果与滑窗2的综合结果进行加权求和得到的。滑窗4的综合结果可以是将滑窗4输出的结果与滑窗3的综合结果进行加权求和得到的，如此迭代计算，直至计算出最后一个滑窗的综合结果。根据最后一个滑窗的综合结果，可以确定概率值最高的手势即为用户输入的功能手势。Progressively, the integrated result of the sliding window 3 may be obtained by performing a weighted summation of the output result of the sliding window 3 and the integrated result of the sliding window 2. The integrated result of the sliding window 4 may be obtained by weighted summation of the output result of the sliding window 4 and the integrated result of the sliding window 3, and iterative calculation is performed until the integrated result of the last sliding window is calculated. According to the comprehensive result of the last sliding window, it can be determined that the gesture with the highest probability value is the functional gesture input by the user.

图5中示出的种图像采集过程可以在低功耗动作检测模块230识别到有起始手势后就切换第三帧率，这样可以保证手势识别的速度，从而保证电子设备的响应速度。此外，低功耗动作检测模块230在确定用户输入的手势为功能手势的起始手势后再唤醒手势识别模块240，可以避免手势识别模块被误唤醒而导致功耗浪费。The image acquisition process shown in FIG. 5 can switch the third frame rate after the low-power motion detection module 230 recognizes the initial gesture, so that the speed of gesture recognition can be ensured, and the response speed of the electronic device can be ensured. In addition, the low-power motion detection module 230 wakes up the gesture recognition module 240 after determining that the gesture input by the user is the initial gesture of the functional gesture, which can prevent the gesture recognition module from being woken up by mistake and causing waste of power consumption.

图6示例性示出了另外一种图像采集过程示意图。图6与图5的区别在于，切换帧率的时间点不同。具体地，前置摄像头193可以在唤醒手势识别模块240后，从第一帧率(如10帧/秒)切换为第三帧率(30帧/秒)。其他过程与图5示出的图像采集过程一致，此处不再赘述。Fig. 6 exemplarily shows a schematic diagram of another image acquisition process. The difference between FIG. 6 and FIG. 5 is that the time point for switching the frame rate is different. Specifically, the front camera 193 may switch from the first frame rate (for example, 10 frames/sec) to the third frame rate (30 frames/sec) after waking up the gesture recognition module 240. The other processes are the same as the image acquisition process shown in FIG. 5, and will not be repeated here.

不限于上述列举的亮屏状态，在具体实现中图像采集过程也可以在电子设备100熄屏的情况下进行。It is not limited to the above-listed on-screen state, and in specific implementation, the image acquisition process can also be performed when the electronic device 100 is off.

可能地，电子设备可以在熄屏的状态下确定采集的图像中存在手势，然后进行身份验证。在确定用户身份为机主后，触发手势识别流程。其中，身份验证的方式可以但不限于是掌纹识别、虹膜识别、指纹识别、声纹识别等。Possibly, the electronic device can determine that there is a gesture in the captured image when the screen is turned off, and then perform identity verification. After confirming that the user is the owner, the gesture recognition process is triggered. Among them, the method of identity verification can be, but is not limited to, palmprint recognition, iris recognition, fingerprint recognition, voiceprint recognition, and so on.

本申请中，电子设备100可以对用户隔空输入的功能手势做出响应。用户可以直接作出功能手势，无需先输入唤醒手势，再切换成功能手势，输入功能手势的过程一气呵成。即每个功能手势的起始手势均可作为唤醒手势，不同的功能手势的唤醒手势可以不同。与现有技术相比，本申请省略了唤醒手势，减小手势操作时延，提升电子设备响应速度，提升手势输入的效率。In the present application, the electronic device 100 can respond to functional gestures input by the user in an air-space manner. The user can directly make a functional gesture without having to input a wake-up gesture first, and then switch to a functional gesture, and the process of inputting a functional gesture is done in one go. That is, the initial gesture of each functional gesture can be used as a wake-up gesture, and the wake-up gesture of different functional gestures can be different. Compared with the prior art, this application omits the wake-up gesture, reduces the gesture operation time delay, improves the response speed of the electronic device, and improves the efficiency of gesture input.

在一种可能的实施例中，用户可以在电子设备100显示某用户界面时隔空输入手腕上翻(或者手腕下翻)的功能手势，使电子设备100更新显示该用户界面中显示的内容。In a possible embodiment, the user may input a wrist-up (or wrist-down) functional gesture when the electronic device 100 displays a user interface, so that the electronic device 100 can update and display the content displayed in the user interface.

下面介绍本申请实施例涉及的应用场景一以及该应用场景下的用户界面(user interface，UI)实施例。The following describes application scenario one involved in the embodiment of the present application and a user interface (UI) embodiment in the application scenario.

应用场景一：查看微信会话列表的场景。Application scenario 1: View the scenario of WeChat conversation list.

图7示例性示出了用于展示微信会话列表的用户界面。其中，微信是一种即时通讯软件。Fig. 7 exemplarily shows a user interface for displaying a list of WeChat conversations. Among them, WeChat is a kind of instant messaging software.

如图7所示，用于展示微信会话列表的用户界面30可以包括：状态栏301、会话列表302、“微信”菜单控件303、“通讯录”菜单控件304、“发现”菜单控件305及“我”菜单控件306。其中：As shown in FIG. 7, theuser interface 30 for displaying the WeChat conversation list may include: astatus bar 301, aconversation list 302, a "WeChat"menu control 303, a "Contacts"menu control 304, a "Discover"menu control 305, and My"menu control 306. among them:

状态栏301可以包括：运营商指示符(例如运营商的名称“中国移动”)、无线高保真(wireless fidelity，Wi-Fi)信号的一个或多个信号强度指示符、移动通信信号(又可称为 蜂窝信号)的一个或多个信号强度指示符、时间指示符和电池状态指示符。Thestatus bar 301 may include: operator indicators (for example, the operator’s name "China Mobile"), one or more signal strength indicators of wireless fidelity (wireless fidelity, Wi-Fi) signals, mobile communication signals (also Called cellular signal) one or more signal strength indicators, time indicators, and battery status indicators.

会话列表302可以用于按照时间顺序显示与该用户联系的联系人的会话框。电子设备100可以检测到作用于会话列表的用户操作(如上滑操作)，响应于该用户操作，电子设备100可以在会话列表中显示时间更早的会话。Theconversation list 302 may be used to display the conversation boxes of the contacts contacted with the user in chronological order. The electronic device 100 can detect a user operation (such as a slide-up operation) acting on the conversation list, and in response to the user operation, the electronic device 100 can display an earlier conversation in the conversation list.

“微信”菜单控件303可以用于显示会话列表302。电子设备100可以检测到作用于“微信”菜单控件303的点击操作，响应于该点击操作，电子设备100可以显示“微信”菜单的用户界面，该用户界面即为用户界面30。响应于该点击操作，电子设备100还可以使“微信”菜单控件303特殊显示，例如加重显示颜色或者更改显示颜色等。The “WeChat”menu control 303 can be used to display theconversation list 302. The electronic device 100 can detect a click operation acting on the “WeChat”menu control 303, and in response to the click operation, the electronic device 100 can display a user interface of the “WeChat” menu, which is theuser interface 30. In response to the click operation, the electronic device 100 can also make the "WeChat"menu control 303 display special, for example, to emphasize the display color or change the display color.

“通讯录”菜单控件304可以用于显示该用户的联系人列表等。电子设备100可以检测到作用于“通讯录”菜单控件304的点击操作，响应于该点击操作，电子设备100可以显示“通讯录”菜单的用户界面，该用户界面可以包括该用户的联系人列表。响应于该点击操作，电子设备100还可以使“通讯录”菜单控件304特殊显示，例如加重显示颜色或者更改显示颜色等。The "address book"menu control 304 can be used to display the user's contact list and the like. The electronic device 100 can detect a click operation acting on the "address book"menu control 304, and in response to the click operation, the electronic device 100 can display a user interface of the "address book" menu, and the user interface can include the user's contact list . In response to the click operation, the electronic device 100 can also make the “address book”menu control 304 specially displayed, for example, to emphasize the display color or change the display color.

“发现”菜单控件305可以用于显示包括各种功能入口的列表，各种功能例如可以是“朋友圈”、“扫一扫”、“摇一摇”、“看一看”、“搜一搜”等。电子设备100可以检测到作用于“发现”菜单控件305的点击操作，响应于该点击操作，电子设备100可以显示“发现”菜单的用户界面，该用户界面可以包括上述各种功能入口的列表。响应于该点击操作，电子设备100还可以使“发现”菜单控件305特殊显示，例如加重显示颜色或者更改显示颜色等。The "Discover"menu control 305 can be used to display a list of various function entries, such as "Moments of Friends", "Scan", "Shake", "Take a Look", and "Search One". Search" and so on. The electronic device 100 can detect a click operation acting on the "discover"menu control 305, and in response to the click operation, the electronic device 100 can display a user interface of the "discover" menu, and the user interface can include a list of the aforementioned various function entries. In response to the click operation, the electronic device 100 may also make the "discovery"menu control 305 special display, for example, emphasize the display color or change the display color.

“我”菜单控件306可以用于显示与该用户相关的选项，如“支付”、“收藏”、“相册”等。电子设备100可以检测到作用于“我”菜单控件306的点击操作，响应于该点击操作，电子设备100可以显示“我”菜单的用户界面，该用户界面可以包括与该用户相关的选项。响应于该点击操作，电子设备100还可以使“我”菜单控件306特殊显示，例如加重显示颜色或者更改显示颜色等。The "me"menu control 306 can be used to display options related to the user, such as "payment", "favorite", "album" and so on. The electronic device 100 can detect a click operation acting on the "me"menu control 306, and in response to the click operation, the electronic device 100 can display a user interface of the "me" menu, and the user interface can include options related to the user. In response to the click operation, the electronic device 100 can also make the "me" menu control 306 a special display, for example, to emphasize the display color or change the display color.

在电子设备100显示用户界面30时，若用户想要查看会话列表中时间更早的会话，可以通过手势隔空操作电子设备100。隔空输入手势的示意图如图3所示。When the electronic device 100 displays theuser interface 30, if the user wants to view an earlier conversation in the conversation list, he can operate the electronic device 100 through gestures. The schematic diagram of the air input gesture is shown in Figure 3.

图8示例性示出了用户在电子设备100显示用户界面30时隔空输入功能手势(手腕上翻或手腕下翻)后电子设备100做出的响应。FIG. 8 exemplarily shows the response of the electronic device 100 after the user enters a function gesture (wrist up or wrist down) when theuser interface 30 is displayed on the electronic device 100.

电子设备100可以在唤醒手势识别模块240后，在用户界面30中显示指示符307。指示符307可以用于指示电子设备100当前正在检测用户隔空输入的手势。The electronic device 100 may display theindicator 307 in theuser interface 30 after waking up the gesture recognition module 240. Theindicator 307 may be used to indicate that the electronic device 100 is currently detecting a gesture input by the user in the air.

若当前显示的用户界面为图8中的左图，电子设备100在检测到用户手腕上翻的动作后，可更新显示用户界面30中的会话列表302，更新后的用户界面30中的会话列表302如图8中的右图所示。对比虚线框中的部分(即会话列表302)可以看出，更新后的会话列表302中可以显示时间更早的会话。即用户可以隔空输入手腕上翻的功能手势以查看时间更早的会话。If the currently displayed user interface is the left image in FIG. 8, the electronic device 100 may update and display theconversation list 302 in theuser interface 30 after detecting the upward movement of the user's wrist, and the updated conversation list in theuser interface 30 302 is shown on the right in Figure 8. Comparing the part in the dashed box (ie, the conversation list 302), it can be seen that the updatedconversation list 302 can display earlier conversations. That is, the user can input the functional gesture of turning up the wrist to view the earlier conversation.

若当前显示的用户界面为图8中的右图，电子设备100在检测到用户手腕下翻的动作后，可更新显示用户界面30中的会话列表302，更新后的用户界面30中的会话列表302 如图8中的左图所示。对比虚线框中的部分(即会话列表302)可以看出，更新后的会话列表302中可以显示时间更晚的会话。即用户可以隔空输入手腕下翻的功能手势以查看时间更晚的会话。If the currently displayed user interface is the right image in FIG. 8, after detecting the downward movement of the user's wrist, the electronic device 100 may update the displayedsession list 302 in theuser interface 30, and the updated session list in theuser interface 30 302 is shown on the left in Figure 8. Comparing the part in the dashed box (ie, the conversation list 302), it can be seen that the updatedconversation list 302 can display conversations that are later in time. That is, the user can input the functional gesture of turning the wrist down every time to view the conversation that is later.

可知，当用户在用户界面30中输入上滑的触控操作时，电子设备100也可以在用户界面30中显示时间更早的会话。当用户在用户界面30中输入下滑的触控操作时，电子设备100也可以在用户界面30中显示时间更晚的会话。也即是说，手腕上翻的功能手势对应的事件可以与上滑的触控操作一致，手腕下翻的功能手势对应的事件可以与下滑的触控操作一致。It can be seen that when the user inputs a sliding touch operation in theuser interface 30, the electronic device 100 may also display an earlier conversation in theuser interface 30. When the user inputs a sliding touch operation in theuser interface 30, the electronic device 100 may also display a later session in theuser interface 30. In other words, the event corresponding to the functional gesture of turning the wrist up can be consistent with the touch operation of sliding up, and the event corresponding to the functional gesture of turning the wrist down can be consistent with the touch operation of sliding down.

不限于上述列举的应用场景，在具体实现中还可以在其他应用场景中输入手腕上翻或手腕下翻的功能手势使电子设备更新显示用户界面中的显示内容。例如，浏览微博的用户界面、浏览淘宝的用户界面、浏览华为商城的用户界面等，本申请实施例对上述列举的应用场景不作限定。It is not limited to the above-listed application scenarios. In specific implementations, functional gestures of turning the wrist up or down may also be input in other application scenarios to make the electronic device update the display content in the user interface. For example, the user interface of browsing Weibo, the user interface of browsing Taobao, the user interface of browsing Huawei Mall, etc., the embodiment of the present application does not limit the above-listed application scenarios.

不限于通过上述列举的手腕上翻或手腕下翻的功能手势使电子设备更新显示用户界面中的显示内容，在具体实现中还可以有其他的功能手势，如食指隔空上滑或下滑等使电子设备更新显示用户界面中的显示内容，本申请实施例对此不作限定。It is not limited to the above-mentioned functional gestures of turning up the wrist or turning the wrist down to make the electronic device update the display content in the user interface. In the specific implementation, there can also be other functional gestures, such as the index finger sliding up or down in the air. The electronic device updates the display content in the display user interface, which is not limited in the embodiment of the present application.

不限于上述列举的手腕上翻或手腕下翻对应的事件，在具体实现中手腕上翻或手腕下翻还可以对应其他事件，本申请实施例对此不作限定。It is not limited to the events corresponding to the above-mentioned wrist turning up or wrist turning down. In a specific implementation, the wrist turning up or the wrist turning down may also correspond to other events, which are not limited in the embodiment of the present application.

在另外一些实施例中，用户可以在电子设备100显示某用户界面时隔空输入手腕左翻(或者手腕右翻)的功能手势，使电子设备100显示其他的用户界面。In some other embodiments, the user may input a functional gesture of turning the wrist to the left (or turning the wrist to the right) at intervals when the electronic device 100 displays a certain user interface, so that the electronic device 100 displays other user interfaces.

依然以上述场景一为例进行说明。Theabove scenario 1 is still used as an example for description.

图9示例性示出了用户在电子设备100显示用户界面30时隔空输入功能手势(手腕左翻或手腕右翻)后电子设备100做出的响应。FIG. 9 exemplarily shows the response of the electronic device 100 after the user enters a function gesture (turning the wrist left or the wrist right) when theuser interface 30 is displayed on the electronic device 100.

若当前显示的用户界面为图9中的左图，电子设备100在检测到用户手腕左翻的动作后，可显示图9中的右图所示的用户界面40。从图9中可以看出，左图中显示的用户界面30为“微信”菜单的用户界面，右图中显示的用户界面40为“通讯录”菜单的用户界面。对比左图和右图中的“微信”菜单控件303及“通讯录”菜单控件304的显示方式也可以看出当前显示的用户界面的变化。If the currently displayed user interface is the left image in FIG. 9, the electronic device 100 may display theuser interface 40 shown in the right image in FIG. 9 after detecting the left-turning movement of the user's wrist. It can be seen from FIG. 9 that theuser interface 30 shown in the left figure is the user interface of the "WeChat" menu, and theuser interface 40 shown in the right figure is the user interface of the "Contacts" menu. Comparing the display modes of the "WeChat"menu control 303 and the "Contacts"menu control 304 in the left figure and the right figure can also see the change of the currently displayed user interface.

若当前显示的用户界面为图9中的右图，电子设备100在检测到用户手腕右翻的动作后，可显示图9中的左图显示的用户界面30。If the currently displayed user interface is the right image in FIG. 9, the electronic device 100 may display theuser interface 30 shown in the left image in FIG. 9 after detecting the right-turning action of the user's wrist.

可知，当用户在用户界面30中输入左滑的触控操作时，电子设备100也可以将用户界面30切换为用户界面40。当用户在用户界面30中输入右滑的触控操作时，电子设备100也可以将用户界面40切换为用户界面30。也即是说，手腕左翻的功能手势对应的事件可以与左滑的触控操作一致，手腕右翻的功能手势对应的事件可以与右滑的触控操作一致。It can be seen that when the user inputs a left-swiping touch operation in theuser interface 30, the electronic device 100 can also switch theuser interface 30 to theuser interface 40. When the user inputs a right-swiping touch operation in theuser interface 30, the electronic device 100 may also switch theuser interface 40 to theuser interface 30. In other words, the event corresponding to the functional gesture of turning the wrist to the left may be consistent with the touch operation of sliding left, and the event corresponding to the functional gesture of turning the wrist to the right may be consistent with the touch operation of sliding to the right.

若当前显示的用户界面为图9中的右图，电子设备100在检测到用户手腕左翻的动作后，可显示“发现”菜单控件305的用户界面。若当前显示的用户界面为“发现”菜单控件305的用户界面，电子设备100在检测到用户手腕右翻的动作后，可显示“通讯录”菜单控件304的用户界面，即图9中的右图示出的用户界面40。If the currently displayed user interface is the right image in FIG. 9, the electronic device 100 may display the user interface of the “discover”menu control 305 after detecting the left-turning movement of the user's wrist. If the user interface currently displayed is the user interface of the "discover"menu control 305, the electronic device 100 may display the user interface of the "address book"menu control 304 after detecting the right-turning action of the user's wrist, that is, the right in FIG. The figure shows auser interface 40.

若当前显示的用户界面为“发现”菜单控件305的用户界面，电子设备100在检测到用户手腕左翻的动作后，可显示“我”菜单控件306的用户界面。若当前显示的用户界面为“我”菜单控件306的用户界面，电子设备100在检测到用户手腕左翻的动作后，可显示“发现”菜单控件305的用户界面。If the currently displayed user interface is the user interface of the "discover"menu control 305, the electronic device 100 may display the user interface of the "me"menu control 306 after detecting the left-turning movement of the user's wrist. If the currently displayed user interface is the user interface of the "me"menu control 306, the electronic device 100 may display the user interface of the "discover"menu control 305 after detecting the left-turning movement of the user's wrist.

不限于上述列举的应用场景，在具体实现中还可以在其他应用场景中输入手腕左翻或手腕右翻的功能手势使电子设备切换显示用户界面。例如，浏览电子书的用户界面等，本申请实施例对上述列举的应用场景不作限定。It is not limited to the above-listed application scenarios. In specific implementations, a functional gesture of turning the wrist to the left or the wrist to the right may also be input in other application scenarios to make the electronic device switch to display the user interface. For example, to browse the user interface of an e-book, etc., the embodiment of the present application does not limit the above-listed application scenarios.

不限于上述列举的手腕左翻或手腕右翻的功能手势，在具体实现中还可以有其他的功能手势，如食指隔空左滑或右滑等，本申请实施例对此不作限定。It is not limited to the above-mentioned functional gestures for turning the wrist to the left or the wrist to the right. In specific implementation, there may be other functional gestures, such as sliding the index finger to the left or right, which is not limited in the embodiment of the present application.

不限于上述列举的手腕左翻或手腕右翻对应的事件，在具体实现中手腕左翻或右腕下翻还可以对应其他事件，本申请实施例对此不作限定。It is not limited to the above-mentioned events corresponding to the wrist turning left or the wrist turning right. In a specific implementation, the wrist turning left or the right wrist turning down may also correspond to other events, which are not limited in the embodiment of the present application.

本申请中，用户在电子设备100显示不同的用户界面时，隔空输入相同的功能手势，电子设备100可以做出不同的响应。In this application, when the user inputs the same functional gestures in a space when the electronic device 100 displays different user interfaces, the electronic device 100 can make different responses.

在一些的实施例中，用户可以在电子设备100显示某用户界面时隔空输入手腕上翻(或者手腕下翻)的功能手势，使电子设备100更新显示该用户界面中显示的内容，如图8所示。In some embodiments, the user can input a functional gesture of turning up the wrist (or turning down the wrist) when the electronic device 100 displays a user interface, so that the electronic device 100 can update and display the content displayed in the user interface, as shown in FIG. 8 shown.

在另外一些实施例中，用户可以在电子设备100显示某用户界面时隔空输入手腕上翻(或者手腕下翻)的功能手势，使电子设备100增大扬声器170A的播放音量。In other embodiments, the user may input a functional gesture of turning up the wrist (or turning down the wrist) when the electronic device 100 displays a user interface, so that the electronic device 100 increases the playback volume of thespeaker 170A.

下面介绍本申请实施例涉及的应用场景二以及该应用场景下的用户界面(user interface，UI)实施例。The following describes the second application scenario involved in the embodiment of the present application and a user interface (UI) embodiment in this application scenario.

应用场景二：播放音乐的场景。Application Scenario 2: The scene of playing music.

图10示例性示出了用于播放音乐的用户界面50。如图10所示，用户界面50可以包括返回控件501、下载控件502、分享控件503、上一首控件504、暂停控件505、下一首控件506及进度条507。其中：FIG. 10 exemplarily shows auser interface 50 for playing music. As shown in FIG. 10, theuser interface 50 may include areturn control 501, adownload control 502, asharing control 503, aprevious control 504, apause control 505, anext control 506, and a progress bar 507. among them:

返回控件501可以用于返回上一级用户界面。电子设备100可以检测到作用于返回控件501的用户操作(如作用于返回控件501的点击操作)，响应于该操作，电子设备100可以显示该应用程序中用户界面50的上一级用户界面。Thereturn control 501 can be used to return to the upper level user interface. The electronic device 100 can detect a user operation on the return control 501 (for example, a click operation on the return control 501), and in response to the operation, the electronic device 100 can display the upper level user interface of theuser interface 50 in the application.

下载控件502可以用于下载音乐数据至内部存储器121。电子设备100可以检测到作用于下载控件502的用户操作(如作用于下载控件502的点击操作)，响应于该操作，电子设备100可以从服务器下载当前播放的音乐的数据并保存至内部存储器121。Thedownload control 502 can be used to download music data to the internal memory 121. The electronic device 100 can detect a user operation on the download control 502 (such as a click operation on the download control 502), and in response to the operation, the electronic device 100 can download the data of the currently played music from the server and save it to the internal memory 121 .

分享控件503可以用于分享当前播放的音乐给其他用户。电子设备100可以检测到作用于分享控件503的用户操作(如作用于分享控件503的点击操作)，响应于该操作，电子设备100可以分享当前播放的音乐给其他用户。The sharingcontrol 503 can be used to share the currently playing music to other users. The electronic device 100 can detect a user operation on the sharing control 503 (for example, a click operation on the sharing control 503), and in response to the operation, the electronic device 100 can share the currently playing music to other users.

上一首控件504可以用于切换至上一首音乐。电子设备100可以检测到作用于上一首控件504的用户操作(如作用于上一首控件504的点击操作)，响应于该操作，电子设备100可以将当前播放的音乐切换为当前音乐播放列表中的上一首音乐。Theprevious control 504 can be used to switch to the previous music. The electronic device 100 can detect a user operation on the previous control 504 (such as a click operation on the previous control 504), and in response to the operation, the electronic device 100 can switch the currently playing music to the current music playlist The last music in.

暂停控件505可以用于暂停播放音乐。电子设备100可以检测到作用于暂停控件505 的用户操作(如作用于暂停控件505的点击操作)，响应于该操作，电子设备100可以暂停当前播放的音乐。Thepause control 505 can be used to pause playing music. The electronic device 100 can detect a user operation on the pause control 505 (for example, a click operation on the pause control 505), and in response to the operation, the electronic device 100 can pause the currently playing music.

下一首控件506可以用于切换至上一首音乐。电子设备100可以检测到作用于下一首控件506的用户操作(如作用于下一首控件506的点击操作)，响应于该操作，电子设备100可以将当前播放的音乐切换为当前音乐播放列表中的下一首音乐。Thenext control 506 can be used to switch to the previous music. The electronic device 100 can detect a user operation on the next control 506 (such as a click operation on the next control 506), and in response to the operation, the electronic device 100 can switch the currently playing music to the current music playlist The next music in.

进度条507可以用于显示并调整当前音乐的播放进度。电子设备100可以检测到作用于进度条507的用户操作(如作用于进度条507的点击操作)，响应于该操作，电子设备100可以调整当前音乐的播放进度。The progress bar 507 can be used to display and adjust the current music playing progress. The electronic device 100 can detect a user operation (such as a click operation on the progress bar 507) acting on the progress bar 507, and in response to the operation, the electronic device 100 can adjust the current music playback progress.

图11示例性示出了用户在电子设备100显示用户界面50时隔空输入功能手势(手腕上翻或手腕下翻)后电子设备100做出的响应。FIG. 11 exemplarily shows the response of the electronic device 100 after the user enters a function gesture (wrist up or wrist down) when theuser interface 50 is displayed on the electronic device 100.

电子设备100可以在唤醒手势识别模块240后，在用户界面50中显示指示符508。指示符508可以用于指示电子设备100当前正在检测用户隔空输入的手势。The electronic device 100 may display theindicator 508 in theuser interface 50 after waking up the gesture recognition module 240. Theindicator 508 may be used to indicate that the electronic device 100 is currently detecting a gesture input by the user in the air.

如图11所示，若用户隔空输入手腕上翻的功能手势，电子设备100可以在用户界面中显示音量调节符509。此外，电子设备100还可以根据用户手腕上翻的幅度增大扬声器170A的播放音量。若用户隔空输入手腕下翻的功能手势，电子设备100可以在用户界面中显示音量调节符509。此外，电子设备100还可以根据用户手腕下翻的幅度减小扬声器170A的播放音量。As shown in FIG. 11, if the user inputs a functional gesture of turning up the wrist in space, the electronic device 100 may display avolume adjuster 509 in the user interface. In addition, the electronic device 100 can also increase the playback volume of thespeaker 170A according to the amount of upturning the user's wrist. If the user inputs a functional gesture of turning the wrist down in the air, the electronic device 100 may display thevolume adjuster 509 in the user interface. In addition, the electronic device 100 can also reduce the playback volume of thespeaker 170A according to the amount of downward turning of the user's wrist.

限于上述列举的应用场景，在具体实现中还可以在其他应用场景中输入手腕上翻或手腕下翻的功能手势使电子设备调节扬声器170A的播放音量。例如，其他播放音频或视频的用户界面，或者通话的用户界面等，本申请实施例对上述列举的应用场景不作限定。Limited to the above-listed application scenarios, in specific implementations, functional gestures of turning the wrist up or down can also be input in other application scenarios to make the electronic device adjust the playback volume of thespeaker 170A. For example, other user interfaces for playing audio or video, or user interfaces for calling, etc., the embodiment of the present application does not limit the above-listed application scenarios.

不限于通过上述列举的手腕上翻或手腕下翻的功能手势来调节播放音量，在具体实现中还可以有其他的功能手势，如食指隔空上滑或下滑等来调节播放音量，本申请实施例对此不作限定。It is not limited to adjusting the playback volume through the above-mentioned functional gestures of turning up the wrist or turning the wrist down. In specific implementations, there can also be other functional gestures, such as sliding up or down the index finger in the air, to adjust the playback volume. This application implements The example does not limit this.

不限于上述列举的手腕上翻或手腕下翻对应的事件，在具体实现中手腕上翻或手腕下翻还可以对应其他事件，本申请实施例对此不作限定。It is not limited to the events corresponding to the above-mentioned wrist turning up or wrist turning down. In a specific implementation, the wrist turning up or the wrist turning down may also correspond to other events, which are not limited in the embodiment of the present application.

不限于上述列举的手腕上翻或手腕下翻的功能手势，用户界面50中还可以支持其他的功能手势，如手腕左翻或手腕右翻来实现切换音乐至上一首或下一首的功能，或者通过手腕左翻或手腕右翻来实现调节当前音乐的播放进度。本申请实施例对此均不作限定。It is not limited to the functional gestures of wrist up or wrist down listed above. Theuser interface 50 can also support other functional gestures, such as turning left or right wrist to realize the function of switching music to the previous or next song. Or turn left or right wrist to adjust the current music playback progress. The embodiments of the present application do not limit this.

下面介绍本申请实施例涉及的应用场景三以及该应用场景下的用户界面(user interface，UI)实施例。The following introduces the third application scenario involved in the embodiment of the present application and a user interface (UI) embodiment in the application scenario.

应用场景三：播放视频的场景。Application Scenario 3: The scene of playing video.

图12示例性示出了用于播放视频的用户界面60及在该用户界面60中隔空输入功能手势后电子设备100做出的响应。FIG. 12 exemplarily shows a user interface 60 for playing a video and a response of the electronic device 100 after a function gesture is inputted in the user interface 60.

如图12所示，用户界面60可以包括：退出控件601、内容显示区602、暂停控件603、上一集控件604、下一集控件605及进度条606。其中：As shown in FIG. 12, the user interface 60 may include: anexit control 601, acontent display area 602, apause control 603, aprevious set control 604, anext set control 605, and aprogress bar 606. among them:

退出控件601可用于退出当前用户界面。电子设备100可以检测到作用于退出控件601 的用户操作(如在退出控件601上的点击操作)，响应于该操作，电子设备100可以退出当前用户界面。Theexit control 601 can be used to exit the current user interface. The electronic device 100 can detect a user operation (such as a click operation on the exit control 601) acting on theexit control 601, and in response to the operation, the electronic device 100 can exit the current user interface.

内容显示区602可用于显示视频内容。此外，内容显示区602还可调节播放亮度及播放音量。电子设备100可以检测到作用于内容显示区602左侧的触控操作(如上滑或下滑操作)，响应于该操作，电子设备100可以调节当前播放亮度(增大或减小)。电子设备100可以检测到作用于内容显示区602右侧的触控操作(如上滑或下滑操作)，响应于该操作，电子设备100可以调节当前播放音量(增大或减小)。Thecontent display area 602 can be used to display video content. In addition, thecontent display area 602 can also adjust the playback brightness and playback volume. The electronic device 100 can detect a touch operation (such as an up or down operation) acting on the left side of thecontent display area 602, and in response to the operation, the electronic device 100 can adjust the current playback brightness (increase or decrease). The electronic device 100 can detect a touch operation (such as an up or down operation) acting on the right side of thecontent display area 602, and in response to the operation, the electronic device 100 can adjust the current playback volume (increase or decrease).

暂停控件603可用于暂停播放视频。电子设备100可以检测到作用于暂停控件503的用户操作(如在暂停控件603上的点击操作)，响应于该操作，电子设备100可以暂停播放视频。Thepause control 603 can be used to pause playing the video. The electronic device 100 can detect a user operation (such as a click operation on the pause control 603) acting on thepause control 503, and in response to the operation, the electronic device 100 can pause playing the video.

上一集控件604可用于切换播放内容至上一集。电子设备100可以检测到作用于上一集控件604的用户操作(如在上一集控件604上的点击操作)，响应于该操作，电子设备100可以切换播放内容至上一集。Theprevious episode control 604 can be used to switch the playback content to the previous episode. The electronic device 100 can detect a user operation that acts on the previous episode control 604 (such as a click operation on the previous episode control 604), and in response to this operation, the electronic device 100 can switch the playback content to the previous episode.

下一集控件605可用于切换播放内容至下一集。电子设备100可以检测到作用于下一集控件605的用户操作(如在下一集控件605上的点击操作)，响应于该操作，电子设备100可以切换播放内容至下一集。Thenext episode control 605 can be used to switch the playback content to the next episode. The electronic device 100 can detect a user operation on the next episode control 605 (such as a click operation on the next episode control 605), and in response to the operation, the electronic device 100 can switch the playback content to the next episode.

进度条606用于显示当前视频播放进度，且可用于调整播放进度。电子设备100可以检测到作用于进度条606的用户操作(如在进度条606上的滑动操作)，响应于该操作，电子设备100可以调整当前播放进度。Theprogress bar 606 is used to display the current video playback progress, and can be used to adjust the playback progress. The electronic device 100 can detect a user operation (such as a sliding operation on the progress bar 606) acting on theprogress bar 606, and in response to the operation, the electronic device 100 can adjust the current playback progress.

电子设备100可以在唤醒手势识别模块240后，在用户界面60中显示指示符607。指示符607可以用于指示电子设备100当前正在检测用户隔空输入的手势。The electronic device 100 may display the indicator 607 in the user interface 60 after waking up the gesture recognition module 240. The indicator 607 may be used to indicate that the electronic device 100 is currently detecting a gesture input by the user in the air.

如图12所示，若用户隔空输入手腕上翻的功能手势，电子设备100可以根据用户手腕上翻的幅度增加视频播放音量。若用户隔空输入手腕下翻的功能手势，电子设备100可以根据用户手腕下翻的幅度减小视频播放音量。As shown in FIG. 12, if the user inputs a functional gesture of turning up the wrist in space, the electronic device 100 can increase the video playback volume according to the amount of turning up the user's wrist. If the user inputs a functional gesture of turning the wrist down in space, the electronic device 100 may reduce the video playback volume according to the amount of turning the user's wrist down.

若用户隔空输入手腕左翻的功能手势，电子设备100可以根据用户手腕左翻的幅度向后调节视频播放的进度。若用户隔空输入手腕右翻的功能手势，电子设备100可以根据用户手腕右翻的幅度向前调节视频播放的进度。If the user inputs a functional gesture of turning the wrist to the left, the electronic device 100 may adjust the progress of the video playback backward according to the magnitude of the left turning of the user's wrist. If the user inputs a functional gesture of turning the wrist to the right, the electronic device 100 may adjust the progress of the video playback forward according to the magnitude of the right turning of the user's wrist.

以上图6-图12示例性示出了几种本申请实施例适用的应用场景。不限于上述列举的应用场景，本申请实施例还可以适用于其他的应用场景，本申请实施例对此不作限定。Figures 6 to 12 above exemplarily show several application scenarios to which the embodiments of the present application are applicable. It is not limited to the application scenarios listed above, and the embodiments of the present application may also be applicable to other application scenarios, which are not limited in the embodiments of the present application.

此外，在各个应用场景中，用户均可隔空输入由手掌到握拳的功能手势实现截屏的功能。也即是说，当电子设备100检测到用户隔空输入的手势为由手掌到握拳时，电子设备100可以将保存当前显示的用户界面至内部存储器121中。不限于上述列举的手掌到握拳的功能手势，在具体实现中还可以有其他的功能手势使电子设备100实现截屏的功能，本申请实施例对此不作限定。In addition, in various application scenarios, users can input functional gestures from the palm to the fist to realize the screen capture function. That is to say, when the electronic device 100 detects that the user's gesture input from the air is from a palm to a fist, the electronic device 100 can save the currently displayed user interface to the internal memory 121. It is not limited to the above-mentioned functional gestures from the palm to the fist. In specific implementation, there may be other functional gestures to enable the electronic device 100 to realize the screen capture function, which is not limited in the embodiment of the present application.

本申请中，用户在电子设备100显示某用户界面隔空输入功能手势时，电子设备100可以在该用户界面中显示提示符，以提示用户该用户界面中可以支持的多个功能手势。此外，提示符可以包括该用户界面支持的多个功能手势对应的事件的图标，这些事件的图标 可以在显示的方位上与功能手势对应，从而减轻用户的记忆负担，提高用户与电子设备100的交互效率。In this application, when a user displays a gesture for inputting functions in a certain user interface on the electronic device 100, the electronic device 100 may display a prompt on the user interface to remind the user of multiple functional gestures that can be supported in the user interface. In addition, the prompt may include icons of events corresponding to multiple functional gestures supported by the user interface, and the icons of these events may correspond to the functional gestures in the displayed orientation, thereby reducing the user’s memory burden and improving the user’s experience with the electronic device 100. Interactive efficiency.

图13和图14均以应用场景三为例，分别提供了提示符的两种显示方式。Figure 13 and Figure 14 both take Application Scenario 3 as an example, and provide two display modes of prompts respectively.

具体地，电子设备100可以在唤醒手势识别模块240后，在用户界面60中显示提示符。提示符可以用于指示电子设备100当前正在检测用户隔空输入的手势，以及该用户界面中可以支持的多个功能手势对应的事件的图标。Specifically, the electronic device 100 may display a prompt in the user interface 60 after waking up the gesture recognition module 240. The prompt may be used to indicate that the electronic device 100 is currently detecting a gesture inputted by the user in an airspace, and an icon of an event corresponding to multiple functional gestures that can be supported in the user interface.

如图13所示，提示符608可以包括用于指示当前正在检测隔空输入的手势的图标，以及以圆环形式呈现的各个功能手势对应的事件的图标。结合图12中列举的用户界面60支持的各个功能手势及各自对应的事件，从图13中可以看出，视频播放音量增加的图标显示在圆环的上方，用于表征手腕上翻的功能手势可以实现视频播放音量增加；视频播放音量减小的图标显示在圆环的下方，用于表征手腕下翻的功能手势可以实现视频播放音量减小；进度后退的图标显示在圆环的左方，用于表征手腕左翻的功能手势可以实现视频播放进度后退；进度前进的图标显示在圆环的右方，用于表征手腕右翻的功能手势可以实现视频播放进度前进。As shown in FIG. 13, the prompt 608 may include an icon for indicating that a gesture of air input is currently being detected, and an icon of an event corresponding to each functional gesture presented in the form of a circle. Combining the various functional gestures supported by the user interface 60 listed in FIG. 12 and their corresponding events, it can be seen from FIG. 13 that the icon for increasing the video playback volume is displayed above the circle to represent the functional gesture of turning up the wrist. The video playback volume can be increased; the icon for reducing the video playback volume is displayed below the circle, and the functional gesture used to signify the wrist down can realize the reduction of the video playback volume; the icon for the progress backward is displayed on the left of the circle, The functional gesture used to characterize the left turn of the wrist can realize the backward video playback progress; the progress forward icon is displayed on the right of the circle, and the functional gesture used to characterize the right turn of the wrist can realize the progress of the video playback.

如图14所示，提示符608可以包括用于指示当前正在检测隔空输入的手势的图标，以及用箭头图标指向各个功能手势对应的事件的图标。结合图12中列举的用户界面60支持的各个功能手势及各自对应的事件，从图14中可以看出，向上的箭头图标指向视频播放音量增加的图标，用于表征手腕上翻的功能手势可以实现视频播放音量增加；向下的箭头图标指向视频播放音量减小的图标，用于表征手腕下翻的功能手势可以实现视频播放音量减小；向左的箭头图标指向进度后退的图标，用于表征手腕左翻的功能手势可以实现视频播放进度后退；向右的箭头图标指向进度前进的图标，用于表征手腕右翻的功能手势可以实现视频播放进度前进。As shown in FIG. 14, the prompt 608 may include an icon for indicating that a gesture of air input is currently being detected, and an arrow icon to point to an event corresponding to each functional gesture. Combining the various functional gestures supported by the user interface 60 listed in FIG. 12 and their corresponding events, it can be seen from FIG. 14 that the upward arrow icon points to the icon for increasing the video playback volume, which is used to indicate that the functional gesture of turning up the wrist can be Realize the increase of video playback volume; the downward arrow icon points to the icon of reducing the video playback volume, which is used to indicate that the function gesture of turning down the wrist can realize the reduction of video playback volume; the left arrow icon points to the icon of progress backward, which is used to The functional gesture representing the left turning of the wrist can realize the backward video playback progress; the right arrow icon points to the progress forward icon, and the functional gesture representing the right turning of the wrist can realize the progress of the video playback.

通过在用户界面中显示提示符，并将显示符的显示样式与支持的功能手势关联起来，可以减轻用户的记忆负担，提高用户与电子设备100的交互效率。By displaying the prompt in the user interface and associating the display style of the indicator with the supported functional gestures, the user's memory burden can be reduced, and the interaction efficiency between the user and the electronic device 100 can be improved.

接下来介绍本申请实施例提供的手势输入方法，该方法可以由图1或图4示出的电子设备执行。图15示例性示出了本申请实施例提供的手势输入方法流程示意图。Next, the gesture input method provided by the embodiment of the present application is introduced, and the method can be executed by the electronic device shown in FIG. 1 or FIG. 4. FIG. 15 exemplarily shows a schematic flowchart of a gesture input method provided by an embodiment of the present application.

如图15所示，手势输入方法至少可以包括以下几个步骤：As shown in Figure 15, the gesture input method can include at least the following steps:

S1501：电子设备获取第一手势，第一手势包括第一子手势和第二子手势。S1501: The electronic device acquires a first gesture, where the first gesture includes a first sub-gesture and a second sub-gesture.

具体地，第一手势可以是图8-图12中示出的手腕上翻、手腕下翻、手腕左翻或手腕右翻等。第一手势还可以是由手掌到握拳的手势。Specifically, the first gesture may be a wrist turning up, a wrist turning down, a wrist turning left, or a wrist turning right as shown in FIGS. 8-12. The first gesture may also be a gesture from a palm to a fist.

不限于上述列举的手势，在具体实现中第一手势还可以是其他手势，本申请实施例对此不作限定。It is not limited to the gestures listed above. In specific implementation, the first gesture may also be other gestures, which are not limited in the embodiment of the present application.

具体地，第一子手势可以是第一手势的起始部分。第二子手势可以是第一手势中除起始手势外的其他部分。Specifically, the first sub-gesture may be the starting part of the first gesture. The second sub-gesture may be other parts of the first gesture except the initial gesture.

本申请实施例中，第一手势可以包括多个第一子手势。第一子手势可以以帧为单位，具体可以是上述图4实施例中提到的低功耗检测模块230输入的多帧图像中包含的手势。第二子手势可以以图像流为单位，具体是上述图4实施例中提到的手势识别模块240输入 的图像流中包含的手势。In the embodiment of the present application, the first gesture may include multiple first sub-gestures. The first sub-gesture may take a frame as a unit, and specifically may be a gesture included in a multi-frame image input by the low-power detection module 230 mentioned in the embodiment of FIG. 4. The second sub-gesture may take an image stream as a unit, and is specifically a gesture included in the image stream input by the gesture recognition module 240 mentioned in the embodiment of FIG. 4.

S1502：当第一子手势与预设手势匹配时，识别第二子手势对应的功能。S1502: When the first sub-gesture matches the preset gesture, identify the function corresponding to the second sub-gesture.

具体地，预设手势可以是上述电子设备支持的各个手势的起始手势。Specifically, the preset gesture may be the initial gesture of each gesture supported by the above-mentioned electronic device.

具体地，第一神经网络模型可以是由大量的预设手势与非预设手势训练而成。预设手势对应的第一神经网络模型的识别结果为1，非预设手势对应的第一神经网络模型的识别结果为0。Specifically, the first neural network model may be trained by a large number of preset gestures and non-predetermined gestures. The recognition result of the first neural network model corresponding to the preset gesture is 1, and the recognition result of the first neural network model corresponding to the non-predetermined gesture is 0.

S1503：执行第二子手势对应的功能。S1503: Execute the function corresponding to the second sub-gesture.

具体地，第一手势可以是图8实施例中示出的手腕上翻或手腕下翻，第二子手势对应的功能可以是更新显示用户界面30中显示的内容。Specifically, the first gesture may be the up or down of the wrist as shown in the embodiment of FIG. 8, and the function corresponding to the second sub-gesture may be to update the content displayed in theuser interface 30.

具体地，第一手势可以是图9实施例中示出的手腕左翻(或手腕右翻)，第二子手势对应的功能可以是将用户界面30切换为用户界面40(或将用户界面40切换为用户界面30)。Specifically, the first gesture may be the left-turning of the wrist (or the right-turning of the wrist) as shown in the embodiment in FIG. 9, and the function corresponding to the second sub-gesture may be to switch theuser interface 30 to the user interface 40 (or theuser interface 40 Switch to the user interface 30).

具体地，第一手势可以是图11实施例中示出的手腕上翻(或手腕下翻)，第二子手势对应的功能可以是增大扬声器170A的播放音量(或减小扬声器170A的播放音量)。Specifically, the first gesture may be the wrist up (or wrist down) shown in the embodiment of FIG. 11, and the function corresponding to the second sub-gesture may be to increase the playback volume of thespeaker 170A (or decrease the playback volume of thespeaker 170A). volume).

具体地，第一手势可以是图12实施例中示出的手腕上翻(或手腕下翻)，第二子手势对应的功能可以是增大视频播放的音量(或减小视频播放的音量)。Specifically, the first gesture may be the wrist up (or wrist down) shown in the embodiment in FIG. 12, and the function corresponding to the second sub-gesture may be to increase the volume of video playback (or decrease the volume of video playback). .

具体地，第一手势可以是图12实施例中示出的手腕左翻(或手腕右翻)，第二子手势对应的功能可以是向后调节视频播放的进度(或向前调节视频播放的进度)。Specifically, the first gesture may be the left-turning of the wrist (or the right-turning of the wrist) shown in the embodiment of FIG. 12, and the function corresponding to the second sub-gesture may be to adjust the progress of the video playback backward (or to adjust the video playback forward). schedule).

不限于上述列举的第一手势及第二子手势对应的功能，在具体实现中第一手势还可以是其他手势，第二子手势还可以对应其他的功能，本申请实施例对此均不作限定。It is not limited to the functions corresponding to the first gesture and the second sub-gesture listed above. In specific implementation, the first gesture may also be other gestures, and the second sub-gesture may also correspond to other functions, which are not limited in the embodiments of the present application. .

本申请实施例中功能手势可以包括唤醒手势，唤醒电子设备的手势识别功能。即功能手势的起始手势可以作为唤醒手势，不同的功能手势的唤醒手势可以不同。与现有技术中先输入唤醒手势，再切换手势输入功能手势相比，本申请实施例可以省略输入唤醒手势的过程，减小手势操作的时延，提高电子设备的响应速度，提升用户与电子设备的交互效率。The functional gestures in the embodiments of the present application may include a wake-up gesture, which wakes up the gesture recognition function of the electronic device. That is, the initial gesture of the functional gesture can be used as a wake-up gesture, and the wake-up gesture of different functional gestures can be different. Compared with the prior art first inputting the wake-up gesture and then switching the gesture input function, the embodiment of the present application can omit the process of inputting the wake-up gesture, reduce the time delay of the gesture operation, improve the response speed of the electronic device, and improve the user and electronics The interaction efficiency of the device.

在一些可能的实施例中，电子设备可以通过图像采集模块220以第一帧率获取第一子手势。在第一次确定第一子手势与预设手势匹配后，切换为第二帧率，以第二帧率继续获取第一子手势。在确定继续获取的第一子手势与预设手势匹配后，通过低功耗动作检测模块230以第三帧率获取第二子手势，并通过手势识别模块240确定第二子手势对应的功能。其中，第三帧率大于第一帧率。第三帧率例如可以是30帧/秒，第一帧率例如可以是10帧/秒。In some possible embodiments, the electronic device may acquire the first sub-gesture at the first frame rate through the image acquisition module 220. After determining that the first sub-gesture matches the preset gesture for the first time, switch to the second frame rate, and continue to acquire the first sub-gesture at the second frame rate. After determining that the continuously acquired first sub-gesture matches the preset gesture, the low-power motion detection module 230 acquires the second sub-gesture at the third frame rate, and the gesture recognition module 240 determines the function corresponding to the second sub-gesture. Wherein, the third frame rate is greater than the first frame rate. The third frame rate may be 30 frames/sec, for example, and the first frame rate may be 10 frames/sec, for example.

可能地，第二帧率可以为第三帧率。Possibly, the second frame rate may be the third frame rate.

示例性地，如图5所示，电子设备可以在第5帧图像中第一次确定第一张手势与预设手势匹配(即低功耗检测模块230识别到有起始手势)后，从第6帧开始，将帧率从第一帧率切换为第三帧率。以第三帧率继续获取第一子手势，直至第13帧确定第一子手势与预设手势匹配(即确定功能手势为起始手势)后，从第14帧开始以第三帧率获取第二子手势。Exemplarily, as shown in FIG. 5, the electronic device may determine for the first time that the first gesture matches the preset gesture in the fifth frame of image (that is, the low-power detection module 230 recognizes the initial gesture), from Starting from frame 6, switch the frame rate from the first frame rate to the third frame rate. Continue to acquire the first sub-gesture at the third frame rate until it is determined that the first sub-gesture matches the preset gesture in the 13th frame (that is, the functional gesture is determined to be the starting gesture), and the first sub-gesture is acquired at the third frame rate from the 14th frame. Two son gestures.

本申请实施例可以在第一次检测到第一手子势与预设手势匹配后立即切换采集图像的帧率，进而以第三帧率采集后续的第一子手势及第二子手势，既可以降低误触发的概率，又可在最大程度上提升电子设备的响应速度，提升用户与电子设备的交互效率。The embodiment of the application can switch the frame rate of the captured image immediately after detecting that the first hand gesture matches the preset gesture for the first time, and then collect the subsequent first and second sub-gestures at the third frame rate. The probability of false triggering can be reduced, the response speed of the electronic device can be improved to the greatest extent, and the interaction efficiency between the user and the electronic device can be improved.

可能地，第二帧率可以为第一帧率。Possibly, the second frame rate may be the first frame rate.

示例性地，如图6所示，电子设备可以在第5帧图像中第一次确定第一张手势与预设手势匹配(即低功耗检测模块230识别到有起始手势)后，依然以第一帧率继续获取第一子手势，直至第13帧确定第一子手势与预设手势匹配(即确定功能手势为起始手势)后，从第14帧开始以第三帧率获取第二子手势。Exemplarily, as shown in FIG. 6, the electronic device may determine for the first time that the first gesture matches the preset gesture in the 5th frame of the image (that is, the low-power detection module 230 recognizes the initial gesture). Continue to acquire the first sub-gesture at the first frame rate, until it is determined that the first sub-gesture matches the preset gesture in the 13th frame (that is, the functional gesture is determined to be the starting gesture), and the first sub-gesture is acquired at the third frame rate from the 14th frame Two son gestures.

在一些可能的实施例中，当连续X个第一子手势中，有x个第一子手势与预设手势匹配时，以第三帧率获取第二子手势。In some possible embodiments, when there are x first sub-gestures matching the preset gesture among the X consecutive first sub-gestures, the second sub-gesture is acquired at the third frame rate.

示例性地，如图5所示，电子设备可以在第7帧-第13帧这连续7帧图像中识别出第一子手势，当这7帧图像中的7个第一子手势中有5个第一子手势与预设手势匹配，即可确定第一子手势为功能手势的起始手势，唤醒手势识别模块240以第三帧率获取第二子手势。Exemplarily, as shown in FIG. 5, the electronic device can recognize the first sub-gesture in the 7th frame to the 13th frame in the continuous 7 frames of images, when there are 5 of the 7 first sub-gestures in the 7 frames of images. If the first sub-gesture matches the preset gesture, it can be determined that the first sub-gesture is the initial gesture of the functional gesture, and the gesture recognition module 240 is awakened to acquire the second sub-gesture at the third frame rate.

通过识别连续多帧图像中包含的第一子手势是否与预设手势匹配可以降低手势误触发的概率，确保手势识别模块240是唤醒时用户意图明确，而不是被用户不经意间的手势唤醒的，降低手势误触发造成的功耗。By identifying whether the first sub-gesture contained in consecutive frames of images matches the preset gesture, the probability of false triggering of the gesture can be reduced, and the gesture recognition module 240 can ensure that the user's intention is clear when awakening, rather than being awakened by the user's inadvertent gesture. Reduce power consumption caused by false triggering of gestures.

在一些可能的实施例中，上述电子设备获取第一手势之前，上述方法还包括：电子设备显示第一用户界面。In some possible embodiments, before the electronic device acquires the first gesture, the method further includes: the electronic device displays a first user interface.

示例性地，第一用户界面可以是图7中示出的用户界面30。第一用户界面还可以是图10中示出的用户界面50。第一用户界面还可以是图12中示出的用户界面60。Exemplarily, the first user interface may be theuser interface 30 shown in FIG. 7. The first user interface may also be theuser interface 50 shown in FIG. 10. The first user interface may also be the user interface 60 shown in FIG. 12.

不限于上述列举的第一用户界面，在具体实现中，第一用户界面还可以是其他电子设备可以显示的任意的用户界面，本申请实施例对此不作限定。It is not limited to the first user interface listed above. In a specific implementation, the first user interface may also be any user interface that can be displayed by other electronic devices, which is not limited in the embodiment of the present application.

在一些可能的实施例中，当第一子手势与预设手势匹配时，电子设备可以在第一用户界面中显示指示符，并识别第二子手势对应的功能。其中，指示符可以用于指示该电子设备正在识别手势。In some possible embodiments, when the first sub-gesture matches the preset gesture, the electronic device may display an indicator in the first user interface and recognize the function corresponding to the second sub-gesture. Wherein, the indicator can be used to indicate that the electronic device is recognizing a gesture.

具体地，电子设备100可以在唤醒手势识别模块240后，在第一用户界面中显示指示符。Specifically, the electronic device 100 may display the indicator in the first user interface after waking up the gesture recognition module 240.

示例性地，指示符可以是图8示出的指示符307、图11中示出的指示符508、图12中示出的指示符607。Exemplarily, the indicator may be theindicator 307 shown in FIG. 8, theindicator 508 shown in FIG. 11, and the indicator 607 shown in FIG. 12.

本申请实施例可以在确定第一子手势与预设手势匹配后，在电子设备的显示界面中显示指示符，提示用户当前正在识别手势，可以提升用户与电子设备的交互性。In the embodiment of the present application, after determining that the first sub-gesture matches the preset gesture, an indicator may be displayed on the display interface of the electronic device to prompt the user that the gesture is currently being recognized, which may improve the interaction between the user and the electronic device.

在一些可能的实施例中，当第一子手势与预设手势匹配时，电子设备可以在第一用户界面中显示提示符，并识别第二子手势对应的功能。其中，提示符可以用于指示该电子设备支持的多种手势各自对应的功能。In some possible embodiments, when the first sub-gesture matches the preset gesture, the electronic device may display a prompt in the first user interface and recognize the function corresponding to the second sub-gesture. Among them, the prompt may be used to indicate the respective functions corresponding to the various gestures supported by the electronic device.

具体地，电子设备100可以在唤醒手势识别模块240后，在第一用户界面中显示提示符。Specifically, the electronic device 100 may display a prompt in the first user interface after waking up the gesture recognition module 240.

示例性地，指示符可以是图13示出的提示符608、图14中示出的提示符609。Exemplarily, the indicator may be the prompt 608 shown in FIG. 13 and the prompt 609 shown in FIG. 14.

如图13所示，电子设备在显示当前用户界面时，可以支持4种功能手势。各种功能手势对应的功能均不相同。从图13中可以看出，视频播放音量增加的图标显示在圆环的上方，用于表征手腕上翻的功能手势可以实现视频播放音量增加；视频播放音量减小的图标显示在圆环的下方，用于表征手腕下翻的功能手势可以实现视频播放音量减小；进度后退的图 标显示在圆环的左方，用于表征手腕左翻的功能手势可以实现视频播放进度后退；进度前进的图标显示在圆环的右方，用于表征手腕右翻的功能手势可以实现视频播放进度前进。As shown in Figure 13, the electronic device can support four functional gestures when displaying the current user interface. The functions corresponding to various functional gestures are different. It can be seen from Figure 13 that the icon for increasing the volume of video playback is displayed above the circle, and the functional gesture used to indicate that the wrist is turned up can increase the volume of video playback; the icon for decreasing the volume of video playback is displayed below the circle , The functional gesture used to characterize the wrist turning down can reduce the video playback volume; the progress backward icon is displayed on the left of the circle, the functional gesture used to characterize the wrist turning left can realize the video playback progress backward; the progress forward icon Displayed on the right of the circle, the functional gesture used to characterize the right turning of the wrist can realize the progress of the video playback.

图14示例性示出了另外一种提示符的显示方式。如图14所示，提示符608可以包括用于指示当前正在检测隔空输入的手势的图标，以及用箭头图标指向各个功能手势对应的事件的图标。结合图12中列举的用户界面60支持的各个功能手势及各自对应的事件，从图14中可以看出，向上的箭头图标指向视频播放音量增加的图标，用于表征手腕上翻的功能手势可以实现视频播放音量增加；向下的箭头图标指向视频播放音量减小的图标，用于表征手腕下翻的功能手势可以实现视频播放音量减小；向左的箭头图标指向进度后退的图标，用于表征手腕左翻的功能手势可以实现视频播放进度后退；向右的箭头图标指向进度前进的图标，用于表征手腕右翻的功能手势可以实现视频播放进度前进。Figure 14 exemplarily shows another way of displaying prompts. As shown in FIG. 14, the prompt 608 may include an icon for indicating that a gesture of air input is currently being detected, and an arrow icon to point to an event corresponding to each functional gesture. Combining the various functional gestures supported by the user interface 60 listed in FIG. 12 and their corresponding events, it can be seen from FIG. 14 that the upward arrow icon points to the icon for increasing the video playback volume, which is used to indicate that the functional gesture of turning up the wrist can be Realize the increase of video playback volume; the downward arrow icon points to the icon of reducing the video playback volume, which is used to indicate that the function gesture of turning down the wrist can realize the reduction of video playback volume; the left arrow icon points to the icon of progress backward, which is used to The functional gesture representing the left turning of the wrist can realize the backward video playback progress; the right arrow icon points to the progress forward icon, and the functional gesture representing the right turning of the wrist can realize the progress of the video playback.

本申请实施例中，电子设备可以通过各功能手势对应的功能的图标的显示方位将不同的功能手势与其对应的功能关联起来，可以减轻用户的记忆负担，提高用户与电子设备的交互效率。In the embodiments of the present application, the electronic device can associate different functional gestures with their corresponding functions through the display orientation of the icon of the function corresponding to each functional gesture, which can reduce the user's memory burden and improve the interaction efficiency between the user and the electronic device.

在另外一些可能的实施例中，上述电子设备显示第一用户界面，上述第二子手势对应的功能为第一功能；上述执行第二子手势对应的功能之后，上述方法还包括：电子设备显示第二用户界面，上述第一用户界面与上述第一用户界面不同；电子设备获取上述第一手势；当上述第一手势与上述预设手势匹配时，识别上述第二子手势对应的功能为第二功能；上述第一功能与上述第二功能不同；执行上述第二功能。In some other possible embodiments, the electronic device displays the first user interface, and the function corresponding to the second sub-gesture is the first function; after the function corresponding to the second sub-gesture is executed, the method further includes: the electronic device displays For a second user interface, the first user interface is different from the first user interface; the electronic device acquires the first gesture; when the first gesture matches the preset gesture, it recognizes that the function corresponding to the second sub-gesture is the first Two functions; the above-mentioned first function is different from the above-mentioned second function; the above-mentioned second function is performed.

也即是说，电子设备在显示不同的用户界面时，用户输入相同的手势可以实现不同的功能。这样可以根据用户界面当前显示的内容的属性设定与该用户界面对应的手势响应功能，可以充分提高手势输入的利用率，提升用户与电子设备交互的趣味性。That is to say, when the electronic device displays different user interfaces, the user can input the same gesture to achieve different functions. In this way, the gesture response function corresponding to the user interface can be set according to the attributes of the content currently displayed on the user interface, which can fully improve the utilization rate of gesture input and enhance the interest of the interaction between the user and the electronic device.

具体地，第一显示界面例如可以是图8中示出的手腕上翻，第一功能可以是更新显示用户界面30中的显示内容。Specifically, the first display interface may be, for example, turning up the wrist as shown in FIG. 8, and the first function may be to update the display content in thedisplay user interface 30.

第二用户界面例如可以是图11中的手腕上翻，第二功能例如可以是将增大扬声器170A的播放音量(音量加)。The second user interface may be, for example, turning up the wrist in FIG. 11, and the second function may be, for example, increasing the playback volume (volume up) of thespeaker 170A.

不限于上述列举的第一用户界面、第一功能、第二用户界面及第二功能，在具体实现中还可以有其他的第一用户界面、第一功能、第二用户界面及第二功能，本申请实施例对此不作限定。It is not limited to the first user interface, the first function, the second user interface, and the second function listed above, and there may be other first user interface, first function, second user interface, and second function in specific implementation. The embodiment of the application does not limit this.

不限于上述列举的识别功能手势，并使电子设备执行相应的功能，在具体实现中，还可以通过识别用户全身的动作，用于识别体育运动的类型。或者还可以识别用户的面部表情来实现抓拍的功能。或者还可以通过识别用户的手势来实现拍摄特效的添加或更换等。本申请实施例对此不作限定。It is not limited to the above-mentioned recognition function gestures, and makes the electronic device perform corresponding functions. In specific implementation, it can also be used to recognize the type of sports by recognizing the movements of the user's whole body. Or you can recognize the user's facial expressions to achieve the function of capturing photos. Or you can also realize the addition or replacement of shooting special effects by recognizing the user's gestures. The embodiment of the application does not limit this.

本申请实施例还提供了一种计算机可读存储介质，该计算机可读存储介质中存储有指令，当其在计算机或处理器上运行时，使得计算机或处理器执行上述任一个方法中的一个或多个步骤。上述信号处理装置的各组成模块如果以软件功能单元的形式实现并作为独立的产品销售或使用时，可以存储在所述计算机可读取存储介质中。The embodiment of the present application also provides a computer-readable storage medium that stores instructions in the computer-readable storage medium, and when it runs on a computer or a processor, the computer or the processor executes any one of the above methods. Or multiple steps. If each component module of the above-mentioned signal processing device is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in the computer readable storage medium.

在上述实施例中，可以全部或部分地通过软件、硬件、固件或者其任意组合来实现。当使用软件实现时，可以全部或部分地以计算机程序产品的形式实现。所述计算机程序产 品包括一个或多个计算机指令。在计算机上加载和执行所述计算机程序指令时，全部或部分地产生按照本申请实施例所述的流程或功能。所述计算机可以是通用计算机、专用计算机、计算机网络、或者其他可编程装置。所述计算机指令可以存储在计算机可读存储介质中，或者通过所述计算机可读存储介质进行传输。所述计算机指令可以从一个网站站点、计算机、服务器或数据中心通过有线(例如同轴电缆、光纤、数字用户线(DSL))或无线(例如红外、无线、微波等)方式向另一个网站站点、计算机、服务器或数据中心进行传输。所述计算机可读存储介质可以是计算机能够存取的任何可用介质或者是包含一个或多个可用介质集成的服务器、数据中心等数据存储设备。所述可用介质可以是磁性介质，(例如，软盘、硬盘、磁带)、光介质(例如，DVD)、或者半导体介质(例如，固态硬盘(solid state disk，SSD))等。In the above-mentioned embodiments, it may be implemented in whole or in part by software, hardware, firmware, or any combination thereof. When implemented by software, it can be implemented in the form of a computer program product in whole or in part. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on the computer, the processes or functions described in the embodiments of the present application are generated in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions may be stored in a computer-readable storage medium or transmitted through the computer-readable storage medium. The computer instructions can be sent from a website site, computer, server, or data center to another website site via wired (such as coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (such as infrared, wireless, microwave, etc.) , Computer, server or data center for transmission. The computer-readable storage medium may be any available medium that can be accessed by a computer or a data storage device such as a server or a data center integrated with one or more available media. The usable medium may be a magnetic medium (for example, a floppy disk, a hard disk, a magnetic tape), an optical medium (for example, a DVD), or a semiconductor medium (for example, a solid state disk (SSD)).

本领域普通技术人员可以理解实现上述实施例方法中的全部或部分流程，该流程可以由计算机程序来指令相关的硬件完成，该程序可存储于计算机可读取存储介质中，该程序在执行时，可包括如上述各方法实施例的流程。而前述的存储介质包括：ROM或随机存储记忆体RAM、磁碟或者光盘等各种可存储程序代码的介质。A person of ordinary skill in the art can understand that all or part of the process in the above-mentioned embodiment method can be realized. The process can be completed by a computer program instructing relevant hardware. The program can be stored in a computer readable storage medium. , May include the processes of the foregoing method embodiments. The aforementioned storage media include: ROM or random storage RAM, magnetic disks or optical disks and other media that can store program codes.

Claims (12)

Translated fromChinese

一种手势输入方法，其特征在于，所述方法应用于电子设备，所述方法包括：A gesture input method, characterized in that the method is applied to an electronic device, and the method includes:所述电子设备获取第一手势，所述第一手势包括第一子手势和第二子手势；Acquiring a first gesture by the electronic device, where the first gesture includes a first sub-gesture and a second sub-gesture;当所述第一子手势与预设手势匹配时，识别所述第二子手势对应的功能；When the first sub-gesture matches the preset gesture, identifying the function corresponding to the second sub-gesture;执行所述第二子手势对应的功能。Execute the function corresponding to the second sub-gesture.如权利要求1所述的方法，其特征在于，所述电子设备获取第一手势包括：所述电子设备以第一帧率获取第一子手势；The method of claim 1, wherein the electronic device acquiring the first gesture comprises: the electronic device acquiring the first sub-gesture at a first frame rate;所述当所述第一子手势与预设手势匹配时，识别所述第二子手势对应的功能包括：When the first sub-gesture matches the preset gesture, recognizing the function corresponding to the second sub-gesture includes:当所述第一子手势与预设手势匹配时，以第二帧率继续获取所述第一子手势；When the first sub-gesture matches the preset gesture, continue to acquire the first sub-gesture at the second frame rate;当所述继续获取的所述第一子手势与所述预设手势匹配时，以第三帧率获取第二子手势；其中，所述第三帧率大于所述第一帧率；When the continuously acquired first sub-gesture matches the preset gesture, acquire a second sub-gesture at a third frame rate; wherein the third frame rate is greater than the first frame rate;确定所述第二子手势对应的功能。Determine the function corresponding to the second sub-gesture.如权利要求2所述的方法，其特征在于，所述第二帧率等于所述第三帧率。The method of claim 2, wherein the second frame rate is equal to the third frame rate.如权利要求2所述的方法，其特征在于，所述第二帧率等于所述第一帧率。The method of claim 2, wherein the second frame rate is equal to the first frame rate.如权利要求2-4任一项所述的方法，其特征在于，所述当所述继续获取的所述第一子手势与所述预设手势匹配时，以第三帧率获取第二子手势包括：The method according to any one of claims 2-4, wherein when the first sub-gesture that continues to be acquired matches the preset gesture, the second sub-gesture is acquired at a third frame rate. Gestures include:当连续X个第一子手势中，有x个第一子手势与所述预设手势匹配时，以第三帧率获取第二子手势；其中，X及x均为正整数，且X大于或等于x。When there are x first sub-gestures matching the preset gesture in X consecutive first sub-gestures, the second sub-gesture is acquired at the third frame rate; where X and x are both positive integers, and X is greater than Or equal to x.如权利要求1-5任一项所述的方法，其特征在于，所述电子设备获取第一手势之前，所述方法还包括：所述电子设备显示第一用户界面。The method according to any one of claims 1 to 5, wherein before the electronic device acquires the first gesture, the method further comprises: the electronic device displays a first user interface.如权利要求6所述的方法，其特征在于，所述当所述第一子手势与预设手势匹配时，识别所述第二子手势对应的功能，包括：当所述第一子手势与预设手势匹配时，在所述第一用户界面中显示指示符，识别所述第二子手势对应的功能；其中，所述指示符用于指示所述电子设备正在识别手势。7. The method of claim 6, wherein the recognizing a function corresponding to the second sub-gesture when the first sub-gesture matches a preset gesture comprises: when the first sub-gesture matches When the preset gesture is matched, an indicator is displayed in the first user interface to recognize the function corresponding to the second sub-gesture; wherein, the indicator is used to indicate that the electronic device is recognizing the gesture.如权利要求6所述的方法，其特征在于，所述当所述第一子手势与预设手势匹配时，识别所述第二子手势对应的功能，包括：当所述第一子手势与预设手势匹配时，在所述第一用户界面中显示提示符，识别所述第二子手势对应的功能；其中，所述提示符用于指示所述电子设备支持的多种手势各自对应的功能。7. The method of claim 6, wherein the recognizing a function corresponding to the second sub-gesture when the first sub-gesture matches a preset gesture comprises: when the first sub-gesture matches When the preset gesture is matched, a prompt is displayed in the first user interface to identify the function corresponding to the second sub-gesture; wherein, the prompt is used to indicate that the various gestures supported by the electronic device correspond to each Features.如权利要求6-8任一项所述的方法，其特征在于，所述第二子手势对应的功能为第 一功能；The method according to any one of claims 6-8, wherein the function corresponding to the second sub-gesture is the first function;所述执行所述第二子手势对应的功能之后，所述方法还包括：After the function corresponding to the second sub-gesture is executed, the method further includes:所述电子设备显示第二用户界面，所述第一用户界面与所述第二用户界面不同；The electronic device displays a second user interface, and the first user interface is different from the second user interface;所述电子设备获取所述第一手势；Acquiring the first gesture by the electronic device;当所述第一子手势与所述预设手势匹配时，识别所述第二子手势对应的功能为第二功能；所述第一功能与所述第二功能不同；When the first sub-gesture matches the preset gesture, identifying that the function corresponding to the second sub-gesture is a second function; the first function is different from the second function;执行所述第二功能。Perform the second function.一种电子设备，其特征在于，包括：一个或多个处理器、存储器、显示屏以及摄像头；An electronic device, characterized by comprising: one or more processors, memories, display screens, and cameras;所述存储器、所述显示屏以及所述摄像头与所述一个或多个处理器耦合，所述存储器用于存储计算机程序代码，所述计算机程序代码包括计算机指令，当所述一个或多个处理器执行所述计算机指令时，所述电子设备执行如权利要求1-9任一项所述的方法。The memory, the display screen, and the camera are coupled with the one or more processors, and the memory is used to store computer program code, and the computer program code includes computer instructions. When the device executes the computer instruction, the electronic device executes the method according to any one of claims 1-9.一种计算机可读存储介质，所述计算机可读存储介质中存储有指令，当其在计算机或处理器上运行时，使得所述计算机或处理器执行如权利要求1-9任一项所述的方法。A computer-readable storage medium in which instructions are stored, which when run on a computer or processor, cause the computer or processor to execute any one of claims 1-9 Methods.一种包含指令的计算机程序产品，当其在计算机或处理器上运行时，使得所述计算机或处理器执行如权利要求1-9任一项所述的方法。A computer program product containing instructions, when it runs on a computer or processor, causes the computer or processor to execute the method according to any one of claims 1-9.

Images