技术领域Technical Field
本申请涉及计算机技术领域，尤其涉及交互处理方法、装置、设备及存储介质。The present application relates to the field of computer technology, and in particular to interactive processing methods, devices, equipment and storage media.
背景技术Background Art
随着科技的快速发展，各种电子设备不断涌现，例如个人计算机、平板电脑、智能手机等等。拥有自然交互方式的电子设备也受到越来越多人的青睐。因此，智能设备和使用者之间的交互就成为各大智能终端厂商的研发重点，出现了多种多样的在智能终端上实现与使用者操作交互的技术方案。然而，现有技术中大部分都是基于触摸或者语音进行人机交互，用户使用体验没有很大改变，在不方便进行触摸控制或语音控制等特殊情况下，用户无法对电子设备进行操作。With the rapid development of science and technology, various electronic devices continue to emerge, such as personal computers, tablet computers, smart phones, etc. Electronic devices with natural interaction methods are also favored by more and more people. Therefore, the interaction between smart devices and users has become the research and development focus of major smart terminal manufacturers, and a variety of technical solutions have emerged to realize user operation interaction on smart terminals. However, most of the existing technologies are based on touch or voice for human-computer interaction, and the user experience has not changed much. In special circumstances where touch control or voice control is inconvenient, users cannot operate electronic devices.
发明内容Summary of the invention
为克服相关技术中存在的问题，本公开提供了交互处理方法、装置、设备及存储介质。To overcome the problems existing in the related art, the present disclosure provides an interactive processing method, apparatus, device and storage medium.
根据本公开实施例的第一方面，提供一种交互处理方法，所述方法应用于终端设备，所述终端设备设置有包括动态视觉传感器DVS的摄像模组，所述方法包括：According to a first aspect of an embodiment of the present disclosure, an interaction processing method is provided, the method being applied to a terminal device, the terminal device being provided with a camera module including a dynamic vision sensor DVS, the method comprising:
获取所述摄像模组采集的DVS图像数据；Acquire DVS image data collected by the camera module;
从所述DVS图像数据中识别出用于指示终端设备执行操作的目标对象；identifying a target object for instructing a terminal device to perform an operation from the DVS image data;
响应于与所述目标对象对应的操作指令，控制终端设备执行与所述操作指令相匹配的操作。In response to the operation instruction corresponding to the target object, the terminal device is controlled to perform an operation matching the operation instruction.
在一个实施例中，终端设备执行的操作至少包括点亮屏幕，所述获取所述摄像模组采集的DVS图像数据，包括：在终端设备的屏幕处于息屏状态时，获取所述摄像模组采集的DVS图像数据。In one embodiment, the operation performed by the terminal device at least includes lighting up the screen, and the obtaining of the DVS image data collected by the camera module includes: when the screen of the terminal device is in a screen-off state, obtaining the DVS image data collected by the camera module.
在一个实施例中，所述摄像模组配置有低分辨率模式、以及至少一个其他分辨率模式，所述摄像模组在低分辨率模式下处于工作状态的像素单元个数，少于所述摄像模组在其他分辨率模式下处于工作状态的像素单元个数，不同模式在预设模式切换条件满足时切换。In one embodiment, the camera module is configured with a low-resolution mode and at least one other resolution mode. The number of pixel units of the camera module that are in a working state in the low-resolution mode is less than the number of pixel units of the camera module that are in a working state in other resolution modes. Different modes are switched when a preset mode switching condition is met.
在一个实施例中，所述摄像模组以低分辨率模式处于常开状态，或者在终端设备的屏幕处于息屏状态时，所述摄像模组处于低分辨率模式。In one embodiment, the camera module is in a normally-on state in a low-resolution mode, or when the screen of the terminal device is in a screen-off state, the camera module is in a low-resolution mode.
在一个实施例中，所述预设模式切换条件包括以下任一种条件：In one embodiment, the preset mode switching condition includes any one of the following conditions:
依据所述摄像模组在当前模式下采集到的DVS图像数据判定当前环境光的变化满足预设变化条件；Determining whether a change in current ambient light meets a preset change condition based on the DVS image data collected by the camera module in the current mode;
依据所述摄像模组在当前模式下采集到的DVS图像数据，判定所述摄像模组的采集区域内存在待识别对象。According to the DVS image data collected by the camera module in the current mode, it is determined that there is an object to be identified in the collection area of the camera module.
在一个实施例中，所述其他分辨率模式包括高分辨率模式，所述获取所述摄像模组采集的DVS图像数据，包括：获取所述摄像模组在高分辨率模式下采集的DVS图像数据；In one embodiment, the other resolution modes include a high-resolution mode, and the acquiring the DVS image data acquired by the camera module includes: acquiring the DVS image data acquired by the camera module in the high-resolution mode;
所述方法还包括：The method further comprises:
获取所述摄像模组在低分辨率模式下采集的低分辨率DVS图像数据；Acquire low-resolution DVS image data collected by the camera module in a low-resolution mode;
在依据低分辨率DVS图像数据判定当前环境光的变化满足预设变化条件时，控制所述摄像模组从低分辨率模式切换至高分辨率模式。When it is determined based on the low-resolution DVS image data that the change in the current ambient light meets a preset change condition, the camera module is controlled to switch from the low-resolution mode to the high-resolution mode.
在一个实施例中，所述其他分辨率模式包括中分辨率模式和高分辨率模式，所述获取所述摄像模组采集的DVS图像数据，包括：获取所述摄像模组在高分辨率模式下采集的DVS图像数据；In one embodiment, the other resolution modes include a medium resolution mode and a high resolution mode, and the acquiring the DVS image data acquired by the camera module includes: acquiring the DVS image data acquired by the camera module in the high resolution mode;
所述方法还包括：The method further comprises:
获取所述摄像模组在低分辨率模式下采集的低分辨率DVS图像数据；Acquire low-resolution DVS image data collected by the camera module in a low-resolution mode;
在依据低分辨率DVS图像数据判定当前环境光的变化满足预设变化条件时，控制所述摄像模组从低分辨率模式切换至中分辨率模式；When it is determined based on the low-resolution DVS image data that the change in the current ambient light meets a preset change condition, controlling the camera module to switch from the low-resolution mode to the medium-resolution mode;
获取所述摄像模组在中分辨率模式下采集的中分辨率DVS图像数据；Acquire medium-resolution DVS image data collected by the camera module in medium-resolution mode;
在依据所述摄像模组在中分辨率模式下采集到的DVS图像数据，判定所述摄像模组的采集区域内存在待识别对象时，控制所述摄像模组从中分辨率模式切换至高分辨率模式。When it is determined that there is an object to be identified in the acquisition area of the camera module based on the DVS image data acquired by the camera module in the medium resolution mode, the camera module is controlled to switch from the medium resolution mode to the high resolution mode.
在一个实施例中，所述目标对象包括指定手势、指定人脸、和/或指定身体姿势。In one embodiment, the target object includes a specified gesture, a specified face, and/or a specified body posture.
在一个实施例中，预配置有目标对象与操作指令的映射关系，与操作指令相匹配的操作包括以下一种或多种：In one embodiment, a mapping relationship between a target object and an operation instruction is preconfigured, and the operations matching the operation instruction include one or more of the following:
在屏幕息屏状态下触发的解锁屏幕；Unlocking the screen when the screen is off;
在屏幕息屏状态下触发的启动手电筒；The flashlight is triggered when the screen is off;
在屏幕息屏状态下触发的启动指定应用程序；Start a specified application when the screen is off;
在屏幕息屏状态下触发的展示指定应用程序的指定页面；The specified page of the specified application is displayed when the screen is off;
在屏幕息屏状态下触发的展示指定应用程序的新消息；Display new messages of the specified application when the screen is off;
在屏幕息屏状态下接听拨号方的电话。Answer the call from the caller while the screen is off.
根据本公开实施例的第二方面，提供一种交互处理装置，所述装置设于终端设备，所述终端设备设置有包括动态视觉传感器DVS的摄像模组，所述装置包括：According to a second aspect of an embodiment of the present disclosure, an interactive processing device is provided, the device being disposed in a terminal device, the terminal device being provided with a camera module including a dynamic vision sensor DVS, the device comprising:
数据获取模块，用于获取所述摄像模组采集的DVS图像数据；A data acquisition module, used to acquire the DVS image data collected by the camera module;
对象识别模块，用于从所述DVS图像数据中识别出用于指示终端设备执行操作的目标对象；An object recognition module, used to recognize a target object used to instruct a terminal device to perform an operation from the DVS image data;
操作控制模块，用于响应于与所述目标对象对应的操作指令，控制终端设备执行与所述操作指令相匹配的操作。The operation control module is used to control the terminal device to perform an operation matching the operation instruction in response to the operation instruction corresponding to the target object.
在一个实施例中，终端设备执行的操作至少包括点亮屏幕，所述获取所述摄像模组采集的DVS图像数据，包括：在终端设备的屏幕处于息屏状态时，获取所述摄像模组采集的DVS图像数据。In one embodiment, the operation performed by the terminal device at least includes lighting up the screen, and the obtaining of the DVS image data collected by the camera module includes: when the screen of the terminal device is in a screen-off state, obtaining the DVS image data collected by the camera module.
在一个实施例中，所述摄像模组配置有低分辨率模式、以及至少一个其他分辨率模式，所述摄像模组在低分辨率模式下处于工作状态的像素单元个数，少于所述摄像模组在其他分辨率模式下处于工作状态的像素单元个数，不同模式在预设模式切换条件满足时切换。In one embodiment, the camera module is configured with a low-resolution mode and at least one other resolution mode. The number of pixel units of the camera module that are in a working state in the low-resolution mode is less than the number of pixel units of the camera module that are in a working state in other resolution modes. Different modes are switched when a preset mode switching condition is met.
在一个实施例中，所述预设模式切换条件包括以下任一种条件：In one embodiment, the preset mode switching condition includes any one of the following conditions:
依据所述摄像模组在当前模式下采集到的DVS图像数据判定当前环境光的变化满足预设变化条件；Determining whether a change in current ambient light meets a preset change condition based on the DVS image data collected by the camera module in the current mode;
依据所述摄像模组在当前模式下采集到的DVS图像数据，判定所述摄像模组的采集区域内存在待识别对象。According to the DVS image data collected by the camera module in the current mode, it is determined that there is an object to be identified in the collection area of the camera module.
根据本公开实施例的第三方面，提供一种计算机设备，包括基于动态视觉传感器DVS的摄像模组、存储器、处理器及存储在存储器上并可在处理器上运行的计算机程序，其中，所述处理器执行所述程序时实现如上任一项所述方法。According to a third aspect of an embodiment of the present disclosure, a computer device is provided, comprising a camera module based on a dynamic vision sensor DVS, a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements any of the methods described above when executing the program.
根据本公开实施例的第四方面，提供一种计算机可读存储介质，其上存储有计算机程序，该程序被处理器执行时实现上述任一项所述方法的步骤。According to a fourth aspect of an embodiment of the present disclosure, there is provided a computer-readable storage medium on which a computer program is stored, and when the program is executed by a processor, the steps of any of the above methods are implemented.
本公开的实施例提供的技术方案可以包括以下有益效果：The technical solution provided by the embodiments of the present disclosure may have the following beneficial effects:
本实施例通过在终端设备上设置包含动态视觉传感器摄像模组，获取该摄像模组采集的DVS图像数据，并从DVS图像数据中识别出用于指示终端设备执行操作的目标对象，响应于与目标对象对应的操作指令，控制终端设备执行与该操作指令相匹配的操作，从而实现无需触摸和语音输入的情况下，也能与终端设备进行交互，且由于DVS图像数据仅包括亮度发生变化的像素单元的数据，因此，数据量低、且响应速度快。This embodiment provides a camera module including a dynamic vision sensor on the terminal device, obtains DVS image data collected by the camera module, and identifies a target object used to instruct the terminal device to perform an operation from the DVS image data, and controls the terminal device to perform an operation matching the operation instruction in response to an operation instruction corresponding to the target object, thereby realizing interaction with the terminal device without touch and voice input, and because the DVS image data only includes data of pixel units whose brightness changes, the data volume is low and the response speed is fast.
应当理解的是，以上的一般描述和后文的细节描述仅是示例性和解释性的，并不能限制本公开。It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure.
附图说明BRIEF DESCRIPTION OF THE DRAWINGS
此处的附图被并入说明书中并构成本说明书的一部分，示出了符合本公开的实施例，并与说明书一起用于解释本公开的原理。The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and, together with the description, serve to explain the principles of the present disclosure.
图1是本公开根据一示例性实施例示出的一种交互处理方法的流程图。Fig. 1 is a flow chart of an interaction processing method according to an exemplary embodiment of the present disclosure.
图2是本公开根据一示例性实施例示出的一种摄像模组设置位置示意图。FIG. 2 is a schematic diagram of a camera module setting position according to an exemplary embodiment of the present disclosure.
图3是本公开根据一示例性实施例示出的几种手势示意图。Fig. 3 is a schematic diagram showing several gestures according to an exemplary embodiment of the present disclosure.
图4是本公开根据一示例性实施例示出的另一种交互处理方法的流程图。Fig. 4 is a flow chart of another interaction processing method according to an exemplary embodiment of the present disclosure.
图5是本公开根据一示例性实施例示出的另一种交互处理方法的流程图。Fig. 5 is a flow chart of another interaction processing method according to an exemplary embodiment of the present disclosure.
图6是本公开根据一示例性实施例示出的一种交互处理装置的框图。Fig. 6 is a block diagram of an interaction processing device according to an exemplary embodiment of the present disclosure.
图7是本公开根据一示例性实施例示出的另一种交互处理装置的框图。Fig. 7 is a block diagram of another interaction processing device according to an exemplary embodiment of the present disclosure.
图8是本公开根据一示例性实施例示出的另一种交互处理装置的框图。Fig. 8 is a block diagram of another interaction processing device according to an exemplary embodiment of the present disclosure.
图9是本公开根据一示例性实施例示出的一种交互处理装置所在计算机设备的硬件结构图。FIG. 9 is a hardware structure diagram of a computer device where an interactive processing apparatus is located according to an exemplary embodiment of the present disclosure.
具体实施方式DETAILED DESCRIPTION
这里将详细地对示例性实施例进行说明，其示例表示在附图中。下面的描述涉及附图时，除非另有表示，不同附图中的相同数字表示相同或相似的要素。以下示例性实施例中所描述的实施方式并不代表与本公开相一致的所有实施方式。相反，它们仅是与如所附权利要求书中所详述的、本公开的一些方面相一致的装置和方法的例子。Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present disclosure. Instead, they are merely examples of devices and methods consistent with some aspects of the present disclosure as detailed in the appended claims.
在本公开使用的术语是仅仅出于描述特定实施例的目的，而非旨在限制本公开。在本公开和所附权利要求书中所使用的单数形式的“一种”、“所述”和“该”也旨在包括多数形式，除非上下文清楚地表示其他含义。还应当理解，本文中使用的术语“和/或”是指并包含一个或多个相关联的列出项目的任何或所有可能组合。The terms used in this disclosure are for the purpose of describing specific embodiments only and are not intended to limit the disclosure. The singular forms of "a", "said" and "the" used in this disclosure and the appended claims are also intended to include plural forms unless the context clearly indicates otherwise. It should also be understood that the term "and/or" used herein refers to and includes any or all possible combinations of one or more associated listed items.
应当理解，尽管在本公开可能采用术语第一、第二、第三等来描述各种信息，但这些信息不应限于这些术语。这些术语仅用来将同一类型的信息彼此区分开。例如，在不脱离本公开范围的情况下，第一信息也可以被称为第二信息，类似地，第二信息也可以被称为第一信息。取决于语境，如在此所使用的词语“如果”可以被解释成为“在……时”或“当……时”或“响应于确定”。It should be understood that although the terms first, second, third, etc. may be used in the present disclosure to describe various information, such information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other. For example, without departing from the scope of the present disclosure, the first information may also be referred to as the second information, and similarly, the second information may also be referred to as the first information. Depending on the context, the word "if" as used herein may be interpreted as "at the time of" or "when" or "in response to determining".
随着智能终端的广泛使用，越来越多的人离不开手机等终端设备。终端设备和使用者之间的交互就成为各大终端厂商的研发重点，出现了多种多样的在终端设备上实现与使用者操作交互的技术方案。例如，触摸屏幕上的人机交互方案，基于语音的人机交互方案等。在某些不方便进行触摸控制或语音控制等特殊情况下，用户无法对电子设备进行操作，影响用户体验。With the widespread use of smart terminals, more and more people cannot do without mobile phones and other terminal devices. The interaction between terminal devices and users has become the research and development focus of major terminal manufacturers, and a variety of technical solutions have emerged to achieve operational interaction with users on terminal devices. For example, human-computer interaction solutions on touch screens and human-computer interaction solutions based on voice. In some special cases where touch control or voice control is inconvenient, users cannot operate electronic devices, affecting user experience.
鉴于此，本申请实施例提供一种交互方案，通过在终端设备上设置包含动态视觉传感器摄像模组，获取该摄像模组采集的DVS图像数据，并从DVS图像数据中识别出用于指示终端设备执行操作的目标对象，响应于与目标对象对应的操作指令，控制终端设备执行与该操作指令相匹配的操作，从而实现无需触摸和语音输入的情况下，也能与终端设备进行交互，且由于DVS图像数据包括检测到光强发生变化的像素单元的数据，因此，数据量低、且响应速度快。In view of this, an embodiment of the present application provides an interaction solution, which provides a camera module including a dynamic vision sensor on a terminal device, obtains DVS image data collected by the camera module, and identifies a target object used to instruct the terminal device to perform an operation from the DVS image data, and controls the terminal device to perform an operation matching the operation instruction in response to an operation instruction corresponding to the target object, thereby achieving interaction with the terminal device without touch and voice input, and because the DVS image data includes data of pixel units that detect changes in light intensity, the data volume is low and the response speed is fast.
本实施例提供的交互处理方法可以通过软件执行，也可以通过软件和硬件相结合或者硬件执行的方式实现，所涉及的硬件可以由两个或多个物理实体构成，也可以由一个物理实体构成。本实施例方法可以应用于具有所述摄像模组的电子设备。其中，电子设备可以是智能手机、智能学习机、平板电脑、笔记本电脑、PDA(Personal Digital Assistant，个人数字助理)等便携式设备，也可以是台式电脑等固定设备，还可以是智能手环、智能项链等可穿戴设备。The interactive processing method provided in this embodiment can be executed by software, or by a combination of software and hardware or by hardware execution. The hardware involved can be composed of two or more physical entities, or one physical entity. The method of this embodiment can be applied to an electronic device having the camera module. Among them, the electronic device can be a portable device such as a smart phone, a smart learning machine, a tablet computer, a laptop computer, a PDA (Personal Digital Assistant), or a fixed device such as a desktop computer, or a wearable device such as a smart bracelet and a smart necklace.
动态视觉传感器(Dynamic Vision Sensor，DVS)，也可以称为动态事件传感器，是一种模拟基于脉冲触发式神经元的人类视网膜的生物拟态视觉传感器。传感器内部具有由多个像素单元构成的像素单元阵列，其中每个像素单元只有在感应到光强变化时，才会响应并记录光强快速变化的区域。关于动态视觉传感器的具体组成此处不做过多阐述。DVS可以采用事件触发的处理机制，输出异步的事件数据流，事件数据流，例如，可以是光强变化信息(如，光强变化的时间戳和光强值)以及被触发像素单元的坐标位置。DVS的响应速度不再受传统的曝光时间和帧率限制，可以侦测到高达万帧/秒速率运动的高速物体；DVS具有更大的动态范围，在低光照或者高曝光环境下都能准确感应并输出场景变化；DVS功耗更低；由于DVS每个像素单元都是独立响应光强变化，因此DVS不会受运动模糊的影响。Dynamic Vision Sensor (DVS), also known as dynamic event sensor, is a biomimetic vision sensor that simulates the human retina based on pulse-triggered neurons. The sensor has a pixel unit array composed of multiple pixel units, in which each pixel unit will respond and record the area with rapid changes in light intensity only when it senses changes in light intensity. The specific composition of the dynamic vision sensor is not elaborated here. DVS can use an event-triggered processing mechanism to output asynchronous event data streams. The event data stream, for example, can be light intensity change information (such as the timestamp and light intensity value of the light intensity change) and the coordinate position of the triggered pixel unit. The response speed of DVS is no longer limited by traditional exposure time and frame rate, and can detect high-speed objects moving at a rate of up to 10,000 frames per second; DVS has a larger dynamic range and can accurately sense and output scene changes in low light or high exposure environments; DVS has lower power consumption; because each pixel unit of DVS responds to light intensity changes independently, DVS will not be affected by motion blur.
摄像模组除了包括动态视觉传感器，还可以包括镜头、镜座、滤镜、电容、电阻等部件，以构成能采集图像数据的模组即可，在此不做限制。In addition to the dynamic vision sensor, the camera module may also include lenses, lens holders, filters, capacitors, resistors and other components to form a module that can collect image data, and there is no limitation here.
在一个实施例中，以智能手机为例进行示例说明，本公开实施例的执行主体可以是智能手机，也可以是安装在智能手机中的系统服务。需要说明的是，所述智能手机只是本公开实施例所提供的其中一种应用示例，不应理解为本公开实施例所提供的技术方案只能应用于智能手机的场景下。In one embodiment, a smartphone is used as an example for illustration. The execution subject of the embodiment of the present disclosure may be a smartphone or a system service installed in the smartphone. It should be noted that the smartphone is only one of the application examples provided by the embodiment of the present disclosure, and it should not be understood that the technical solution provided by the embodiment of the present disclosure can only be applied to the scenario of a smartphone.
接下来结合附图对本公开实施例进行示例说明。Next, the embodiments of the present disclosure are described with reference to the accompanying drawings.
如图1所示，图1是本公开根据一示例性实施例示出的一种交互处理方法的流程图，包括以下步骤：As shown in FIG. 1 , FIG. 1 is a flowchart of an interaction processing method according to an exemplary embodiment of the present disclosure, comprising the following steps:
在步骤102中，获取摄像模组采集的DVS图像数据；In step 102, DVS image data collected by the camera module is obtained;
在步骤104中，从所述DVS图像数据中识别出用于指示终端设备执行操作的目标对象；In step 104, a target object for instructing a terminal device to perform an operation is identified from the DVS image data;
在步骤106中，响应于与所述目标对象对应的操作指令，控制终端设备执行与所述操作指令相匹配的操作。In step 106, in response to the operation instruction corresponding to the target object, the terminal device is controlled to perform an operation matching the operation instruction.
该方法可以用于终端设备中，终端设备设置有包括动态视觉传感器DVS的摄像模组。所述摄像模组可以设置在终端设备的外表面部，以采集终端设备所处环境的图像数据。如设置在终端设备的正面或背面。在一个实施例中，摄像模组设置在终端设备的摄像头的周围区域。例如，摄像模组可以设置在前置摄像头的周围区域，或，摄像模组也可以设置在后置摄像头的周围区域。如图2所示，图2是本公开根据一示例性实施例示出的一种摄像模组设置位置示意图。该图以终端设备为智能手机为例，在智能手机前置摄像头的右侧设置有包括动态视觉传感器DVS的摄像模组。The method can be used in a terminal device, which is provided with a camera module including a dynamic vision sensor DVS. The camera module can be arranged on the outer surface of the terminal device to collect image data of the environment in which the terminal device is located. For example, it is arranged on the front or back of the terminal device. In one embodiment, the camera module is arranged in the surrounding area of the camera of the terminal device. For example, the camera module can be arranged in the surrounding area of the front camera, or the camera module can also be arranged in the surrounding area of the rear camera. As shown in Figure 2, Figure 2 is a schematic diagram of the setting position of a camera module shown in the present disclosure according to an exemplary embodiment. The figure takes the terminal device as a smart phone as an example, and a camera module including a dynamic vision sensor DVS is arranged on the right side of the front camera of the smart phone.
视觉传感器采集场景中的事件数据，在场景发生变化时可以输出事件。例如，当场景中没有物体相对于终端设备运动时，动态视觉传感器中像素单元检测到的光强不会发生变化，当检测到场景中的某个物体相对于终端设备发生运动，会导致光线发生变化，因而触发像素事件，输出检测到光强发生变化的像素单元的事件数据流，该事件数据流中每个事件数据可以包括检测到亮度发生变化的像素单元的坐标位置和被触发时刻的时间戳信息。DVS图像数据可以由同一时间戳信息对应的事件数据构成。由于在动态视觉传感器中，对于单个像素点，只有接受光强度发生改变时，才会有事件(脉冲)信号输出。比如说亮度增加超过一个阈值，那么将增加一个该像素点亮度增加的事件。因此，DVS图像数据可以是部分图像数据，针对没有检测到光强度发生变化的像素单元没有事件数据。The visual sensor collects event data in the scene and can output events when the scene changes. For example, when there is no object in the scene moving relative to the terminal device, the light intensity detected by the pixel unit in the dynamic visual sensor will not change. When a certain object in the scene is detected to be moving relative to the terminal device, the light will change, thus triggering the pixel event, and outputting the event data stream of the pixel unit that detected the change in light intensity. Each event data in the event data stream can include the coordinate position of the pixel unit that detected the change in brightness and the timestamp information of the time of triggering. DVS image data can be composed of event data corresponding to the same timestamp information. Because in the dynamic visual sensor, for a single pixel point, only when the light intensity changes, will there be an event (pulse) signal output. For example, if the brightness increases by more than a threshold, then an event of the brightness increase of the pixel point will be added. Therefore, the DVS image data can be partial image data, and there is no event data for the pixel unit that does not detect the change in light intensity.
在获得所述摄像模组采集的DVS图像数据后，可以从DVS图像数据中识别出用于指示终端设备执行操作的目标对象。After the DVS image data collected by the camera module is obtained, a target object used to instruct the terminal device to perform an operation can be identified from the DVS image data.
关于目标对象，目标对象是用于指示终端设备执行操作的对象。根据终端设备所需执行的操作，配置不同种类的目标对象。在一个示例中，终端设备执行的操作是身份验证成功后执行的操作，例如，解锁、支付、登录等操作。相应的，目标对象可以是用于进行身份验证的对象。例如，通过人脸识别进行身份验证的场景中，目标对象可以是指定人脸；又如，通过手势识别进行身份验证的场景中，目标对象可以是指定手势等。Regarding the target object, the target object is an object used to instruct the terminal device to perform an operation. Different types of target objects are configured according to the operation that the terminal device needs to perform. In one example, the operation performed by the terminal device is an operation performed after successful identity authentication, such as unlocking, payment, login, etc. Correspondingly, the target object can be an object used for identity authentication. For example, in a scenario where identity authentication is performed through face recognition, the target object can be a specified face; for another example, in a scenario where identity authentication is performed through gesture recognition, the target object can be a specified gesture, etc.
在另一个示例中，终端设备执行的操作是某指定操作，例如，点亮屏幕，打开系统收藏夹，进入指定应用程序的指定页面，展示指定应用程序的新消息，锁屏状态下接听来电电话等操作。相应的，目标对象可以是与指定操作映射的对象。示例的，可以是指定部位的姿势。例如，指定手势或指定身体姿势等。指定手势可以是用手比划所确定的姿势。例如，可以是“六”手势、比心手势、“2”手势(也称为耶手势)、ok手势、点赞手势、手掌打开手势等，还可以是比划其他数字的手势。如图3所示，图3是本公开根据一示例性实施例示出的几种手势示意图。可以理解的是，该示意图仅示例出几种手势，实际上还可以是其他手势，例如，比划1、3、4等手势。每种手势还可以有多种变形，只要能表达相应手势含义即可，在此不做限制。指定身体姿势可以是举手姿势、叉腰姿势等。In another example, the operation performed by the terminal device is a specified operation, for example, lighting up the screen, opening the system favorites, entering the specified page of the specified application, displaying new messages of the specified application, answering incoming calls in the lock screen state, and other operations. Correspondingly, the target object may be an object mapped to the specified operation. For example, it may be a posture of a specified part. For example, a specified gesture or a specified body posture, etc. The specified gesture may be a gesture determined by a hand gesture. For example, it may be a "six" gesture, a heart gesture, a "2" gesture (also known as a yes gesture), an ok gesture, a thumbs-up gesture, an open palm gesture, etc., or a gesture of making other numbers. As shown in FIG. 3 , FIG. 3 is a schematic diagram of several gestures shown in the present disclosure according to an exemplary embodiment. It can be understood that the schematic diagram only illustrates several gestures, and in fact it may also be other gestures, for example, gestures of 1, 3, 4, etc. Each gesture may also have multiple variations, as long as it can express the meaning of the corresponding gesture, and is not limited here. The specified body posture may be a hand-raising gesture, an akimbo gesture, etc.
在一个示例中，可以预先配置目标对象与操作指令的映射关系。操作指令可以是指示终端设备执行操作的指令。在一个例子中，目标对象与操作指令间可以是一一映射关系，以实现每个目标对象触发设备执行一种操作。在另一个例子中，目标对象与操作指令间也可以是多对一的映射关系，以实现多个目标对象触发设备执行一种操作。以手势为例，可以是连续多个手势对应一种操作指令。如，连续比划“3、2、1”三个手势，触发终端设备点亮屏幕，并解锁屏幕。In one example, the mapping relationship between the target object and the operation instruction can be pre-configured. The operation instruction can be an instruction that instructs the terminal device to perform an operation. In one example, there can be a one-to-one mapping relationship between the target object and the operation instruction, so that each target object triggers the device to perform an operation. In another example, there can also be a many-to-one mapping relationship between the target object and the operation instruction, so that multiple target objects trigger the device to perform an operation. Taking gestures as an example, multiple consecutive gestures can correspond to one operation instruction. For example, continuously making the three gestures of "3, 2, 1" triggers the terminal device to light up the screen and unlock the screen.
目标对象与操作指令的映射关系，可以是系统预先配置的，也可以是用户设置的。例如，提供映射关系设置服务，以供用户创建目标对象与操作指令的映射关系。The mapping relationship between the target object and the operation instruction may be pre-configured by the system or set by the user. For example, a mapping relationship setting service is provided for the user to create a mapping relationship between the target object and the operation instruction.
而关于如何从DVS图像数据中识别出目标对象，在一个示例中，可以利用机器学习的方式学习获得能识别目标对象的模型，在模型应用阶段，利用学习获得的模型从DVS图像数据中识别出目标对象。示例的，可以采用有监督学习的方式利用预设的训练样本进行模型训练，获得深度学习网络模型。训练样本可以是带标签的样本图像，标签可以指示出目标对象的位置以及类别。样本图像可以包括DVS图像数据，也可以包括传统的image sensor采集的图像。针对每种目标对象，可以包括不同拍摄角度、和/或不同变形条件下的样本图像，以便提高模型的识别率。示例的，可以利用图像信号处理(Image Signal Processing，ISP)单元进行图像处理。As for how to identify the target object from the DVS image data, in one example, a model that can identify the target object can be learned by machine learning. In the model application stage, the target object can be identified from the DVS image data by using the learned model. For example, a supervised learning method can be used to train the model using preset training samples to obtain a deep learning network model. The training sample can be a labeled sample image, and the label can indicate the location and category of the target object. The sample image can include DVS image data or an image captured by a traditional image sensor. For each target object, sample images under different shooting angles and/or different deformation conditions can be included to improve the recognition rate of the model. For example, an image signal processing (ISP) unit can be used for image processing.
应当理解的是，上述目标对象的识别方法仅为举例，而不应被理解为对本公开的任何限制，其他现有的或将来的识别目标对象的方法均可以适用本公开，均应包括在本公开的保护范围内。It should be understood that the above-mentioned target object identification method is only an example and should not be construed as any limitation to the present disclosure. Other existing or future target object identification methods may be applicable to the present disclosure and should be included in the protection scope of the present disclosure.
关于终端设备执行的操作，可以是身份验证成功后的操作，也可以是预先指定的操作。例如，与操作指令相匹配的操作包括以下一种或多种：The operation performed by the terminal device may be an operation after successful identity authentication or a pre-specified operation. For example, the operations that match the operation instruction include one or more of the following:
点亮屏幕；Light up the screen;
解锁屏幕；Unlock the screen;
启动手电筒；Activate the flashlight;
启动指定应用；Start the specified application;
展示指定应用的指定页面；Display the specified page of the specified application;
展示指定应用的新消息；Display new messages of the specified application;
接听拨号方的电话。Answer the call from the dialing party.
其中，点亮屏幕，可以是控制屏幕从息屏状态切换至亮屏状态，息屏状态，可以是屏幕处于黑屏的状态。亮屏状态，可以是屏幕被点亮后所处的状态。Among them, lighting up the screen may be controlling the screen to switch from a screen-off state to a screen-on state, the screen-off state may be a state where the screen is in a black screen state, and the screen-on state may be a state where the screen is in after being lit up.
为了保证个人信息的安全性，用户往往会对终端设备进行锁屏处理，锁屏后需要通过诸如输入密码等解锁方式才能查看终端设备内容。本实施例可以通过识别出目标对象的方式实现自动解锁。In order to ensure the security of personal information, users often lock the screen of their terminal devices, and after locking the screen, they need to enter a password or other unlocking method to view the content of the terminal device. This embodiment can realize automatic unlocking by identifying the target object.
指定应用程序，可以是终端设备中已安装的应用程序，例如，可以是系统应用，也可以是第三方应用。例如，指定应用程序可以是系统收藏夹/照片应用。启动指定应用程序后，可以展示指定应用程序的首页/默认页。示例的，通过比心手势打开系统收藏夹，以查看系统收藏夹中的内容。The specified application can be an application installed in the terminal device, for example, a system application or a third-party application. For example, the specified application can be a system favorites/photo application. After the specified application is started, the homepage/default page of the specified application can be displayed. For example, the system favorites can be opened by making a heart gesture to view the contents in the system favorites.
指定应用程序的指定页面，可以是用户期望快速查看到的页面，例如，可以是支付程序的支付页面。支付页面可以是包括付款码的页面。该指定应用程序可以是已经启动的应用程序，也可以是未启动的应用程序。针对已经启动的应用程序，可以是前台运行的应用程序，也可以是后台运行的应用程序。示例的，通过耶的手势打开支付程序的支付页面。The designated page of the designated application may be a page that the user expects to quickly view, for example, a payment page of a payment program. The payment page may be a page including a payment code. The designated application may be an application that has been started or an application that has not been started. For an application that has been started, it may be an application that is running in the foreground or in the background. For example, the payment page of the payment program is opened by a "yeah" gesture.
指定应用程序的新消息，可以是所有未读消息，也可以是接收时间最近的未读消息等，可以根据需求设置。该指定应用程序可以是已经启动的应用程序，也可以是未启动的应用程序。针对已经启动的应用程序，可以是前台运行的应用程序，也可以是后台运行的应用程序。示例的，可以通过OK手势打开微信新消息或未读短信。The new messages of the specified application can be all unread messages or the most recently received unread messages, etc., which can be set according to needs. The specified application can be an already started application or an unstarted application. For the already started application, it can be an application running in the foreground or an application running in the background. For example, you can use the OK gesture to open WeChat new messages or unread text messages.
关于接听拨号方的电话，目前，在接收到拨号方的拨号请求时，可以通过触摸接听按钮，以实现接听拨号方的电话，而本实施例通过识别目标对象的方式实现自动接通电话。Regarding answering the call from the dialing party, currently, when receiving the dialing request from the dialing party, the call from the dialing party can be answered by touching the answer button, while this embodiment realizes automatic connection of the call by identifying the target object.
示例的，不管终端设备当前处于何种状态，在识别到目标对象后，可以触发终端设备执行与操作指令相匹配的操作。例如，终端设备当前处于息屏状态，可以从息屏状态开始完成与操作指令相匹配的操作。该实施例从息屏状态直接跳转执行相应操作，可以提高交互效率，给用户带来新的体验。又如，终端设备当前处于亮屏状态，可以从亮屏状态开始完成与操作指令相匹配的操作。For example, no matter what state the terminal device is currently in, after identifying the target object, the terminal device can be triggered to perform an operation matching the operation instruction. For example, if the terminal device is currently in the screen-off state, the operation matching the operation instruction can be completed from the screen-off state. This embodiment directly jumps to perform the corresponding operation from the screen-off state, which can improve the interaction efficiency and bring a new experience to the user. For another example, if the terminal device is currently in the screen-on state, the operation matching the operation instruction can be completed from the screen-on state.
作为其中一种实施例，与操作指令相匹配的操作可以是从息屏状态开始触发的操作。例如，与操作指令相匹配的操作包括以下一种或多种：As one embodiment, the operation matching the operation instruction may be an operation triggered from the screen-off state. For example, the operation matching the operation instruction includes one or more of the following:
在屏幕息屏状态下触发的解锁屏幕；Unlocking the screen when the screen is off;
在屏幕息屏状态下触发的启动手电筒；The flashlight is triggered when the screen is off;
在屏幕息屏状态下触发的启动指定应用程序；Start a specified application when the screen is off;
在屏幕息屏状态下触发的展示指定应用程序的指定页面；The specified page of the specified application is displayed when the screen is off;
在屏幕息屏状态下触发的展示指定应用程序的新消息；Display new messages of the specified application when the screen is off;
在屏幕息屏状态下接听拨号方的电话。Answer the call from the caller while the screen is off.
可见，通过该实施例可以实现在黑屏时检测到目标对象而触发屏幕解锁、启动手电筒、启动指定应用程序、展示指定应用程序的指定页面、展示指定应用程序的新消息或接听拨号方的电话等操作，提高操作效率。It can be seen that through this embodiment, it is possible to detect a target object when the screen is black and trigger screen unlocking, start the flashlight, start a specified application, display a specified page of a specified application, display new messages of a specified application, or answer a call from the dialing party, thereby improving operational efficiency.
另外，上述示例的各操作可以进行组合，示例的，依据终端设备当前所处状态进行组合，例如，终端设备当前处于亮屏状态，则为了展示指定应用程序的指定页面，则可以先点亮屏幕、解锁屏幕，再展示指定应用程序的指定页面。若指定应用程序没有启动，则展示指定页面前，还可以先启动指定应用程序。可见，以上实施方式中的各种操作可以任意进行组合，只要特征之间的组合不存在冲突或矛盾即可。为了达到最终目的，可能还需要执行一些其他操作，在此省略，但应当理解的是，为了达到最终目标，中间不可缺少的操作也囊括在操作指令相匹配的操作中。另外，终端设备执行的操作包括但不限于上述操作，还可以是其他操作，在此不一一列举。In addition, the operations in the above examples can be combined. For example, they can be combined according to the current state of the terminal device. For example, if the terminal device is currently in a screen-on state, in order to display the specified page of the specified application, the screen can be turned on and unlocked first, and then the specified page of the specified application can be displayed. If the specified application is not started, the specified application can be started before displaying the specified page. It can be seen that the various operations in the above implementations can be combined arbitrarily, as long as there is no conflict or contradiction between the combinations of features. In order to achieve the ultimate goal, some other operations may need to be performed, which are omitted here, but it should be understood that in order to achieve the ultimate goal, the indispensable operations in the middle are also included in the operations that match the operation instructions. In addition, the operations performed by the terminal device include but are not limited to the above operations, and can also be other operations, which are not listed here one by one.
由上述实施例可见，本实施例通过在终端设备上设置包含动态视觉传感器的摄像模组，获取摄像模组采集的DVS图像数据，并从DVS图像数据中识别出用于指示终端设备执行操作的目标对象，响应于与目标对象对应的操作指令，控制终端设备执行与该操作指令相匹配的操作，从而实现无需触摸和语音输入的情况下，也能与终端设备进行交互，且存储数据量低、响应速度快。As can be seen from the above embodiments, this embodiment provides a camera module including a dynamic vision sensor on the terminal device, obtains DVS image data collected by the camera module, and identifies a target object used to instruct the terminal device to perform an operation from the DVS image data, and controls the terminal device to perform an operation matching the operation instruction in response to an operation instruction corresponding to the target object, thereby realizing interaction with the terminal device without touch and voice input, and with a low amount of stored data and a fast response speed.
以终端设备执行的操作至少包括点亮屏幕为例进行示例说明，如图4所示，图4是本公开根据一示例性实施例示出的另一种交互处理方法的流程图，该方法可以用于终端设备中，所述终端设备设置有包括动态视觉传感器DVS的摄像模组，包括以下步骤：Taking the operation performed by the terminal device as an example, in which at least the screen is turned on, as shown in FIG4 , FIG4 is a flowchart of another interactive processing method according to an exemplary embodiment of the present disclosure, which can be used in a terminal device, wherein the terminal device is provided with a camera module including a dynamic visual sensor DVS, and comprises the following steps:
在步骤402中，在终端设备的屏幕处于息屏状态时，获取摄像模组采集的DVS图像数据；In step 402, when the screen of the terminal device is in the screen-off state, DVS image data collected by the camera module is obtained;
在步骤404中，从所述DVS图像数据中识别出用于指示终端设备执行操作的目标对象，所述操作至少包括点亮屏幕；In step 404, a target object for instructing a terminal device to perform an operation is identified from the DVS image data, and the operation at least includes lighting up a screen;
在步骤406中，响应于与所述目标对象对应的操作指令，控制终端设备执行与所述操作指令相匹配的操作。In step 406, in response to the operation instruction corresponding to the target object, the terminal device is controlled to perform an operation matching the operation instruction.
其中，图4与图1中相关技术相同，在此不一一赘述。Among them, the related technologies in Figure 4 are the same as those in Figure 1, and are not described in detail here.
本实施例可以在终端设备的屏幕处于息屏状态下，通过目标对象触发终端设备执行相应操作。如，在息屏状态下通过比心手势打开系统收藏夹；在息屏状态下通过耶的手势打开支付程序的支付页面；在息屏状态下通过ok手势打开微信新消息或短信新消息；在息屏状态下通过“六”手势在锁屏来电时接听电话；在息屏状态下通过全手掌打开手势实现屏幕解锁等。该实施例从息屏状态直接跳转执行相应操作，可以提高交互效率，给用户带来新的体验。This embodiment can trigger the terminal device to perform corresponding operations through the target object when the screen of the terminal device is in the screen-off state. For example, in the screen-off state, the system favorites can be opened through the heart gesture; in the screen-off state, the payment page of the payment program can be opened through the "yeah" gesture; in the screen-off state, a new WeChat message or a new SMS message can be opened through the "ok" gesture; in the screen-off state, a call is answered when the lock screen is called through the "six" gesture; in the screen-off state, the screen is unlocked through the full palm opening gesture, etc. This embodiment directly jumps to perform corresponding operations from the screen-off state, which can improve the interaction efficiency and bring a new experience to the user.
在一个实施例中，为了减少摄像模组的功耗，可以为摄像模组配置不同功耗的模式，摄像模组在不同模式下采集到的图像数据的分辨率不同，因此，可以称为分辨率模式。摄像模组在不同模式下所消耗的功耗不同，因此，也可以称为功耗模式。摄像模组的不同分辨率模式可以由动态视觉传感器中处于开启状态(工作状态)的像素单元的个数划分。示例的，为摄像模组配置低分辨率模式(Low Resolution，LR)。摄像模组在低分辨率模式下仅部分像素单元处于工作状态。以视觉传感器能采集100万像素为例，可以将其中二十四分之一的像素单元开启，将剩余的像素单元关闭，以降低功耗。甚至可以控制指定个数的像素单元处于开启状态，其他处于关闭状态。摄像模组还配置有功耗高于低功耗的至少一个其他分辨率模式，摄像模组在低分辨率模式下处于工作状态的像素单元个数，少于摄像模组在其他分辨率模式下处于工作状态的像素单元个数。相应的，摄像模组在低分辨率模式下采集到的图像数据的分辨率，低于摄像模组在其他分辨率模式下采集到的图像数据的分辨率。In one embodiment, in order to reduce the power consumption of the camera module, different power consumption modes can be configured for the camera module. The resolution of the image data collected by the camera module in different modes is different, so it can be called a resolution mode. The power consumption consumed by the camera module in different modes is different, so it can also be called a power consumption mode. The different resolution modes of the camera module can be divided by the number of pixel units in the dynamic vision sensor that are in an on state (working state). For example, a low resolution mode (Low Resolution, LR) is configured for the camera module. Only some pixel units of the camera module are in a working state in the low resolution mode. Taking the visual sensor that can collect 1 million pixels as an example, one-twenty-fourth of the pixel units can be turned on and the remaining pixel units can be turned off to reduce power consumption. It is even possible to control a specified number of pixel units to be in an on state and the others to be in a closed state. The camera module is also configured with at least one other resolution mode with a power consumption higher than low power consumption. The number of pixel units in the camera module that are in a working state in the low resolution mode is less than the number of pixel units in the camera module that are in a working state in other resolution modes. Correspondingly, the resolution of image data collected by the camera module in the low-resolution mode is lower than the resolution of image data collected by the camera module in other resolution modes.
在一个实施例中，为了实现对目标对象的检测，摄像模组可以以低分辨率模式处于常开状态。在预设模式切换条件满足时，进行模式切换。例如，在终端设备息屏状态下，摄像模组处于低分辨率模式。In one embodiment, in order to detect the target object, the camera module can be in a normally open state in a low-resolution mode. When the preset mode switching condition is met, the mode is switched. For example, when the terminal device is in a screen-off state, the camera module is in a low-resolution mode.
可见，在该实施例中，通过控制摄像模组以低分辨率模式处于常开状态，既可以保证实时检测，又可以降低功耗。It can be seen that in this embodiment, by controlling the camera module to be in a normally open state in a low-resolution mode, real-time detection can be ensured and power consumption can be reduced.
在另一个实施例中，在终端设备的屏幕处于息屏状态时，所述摄像模组处于低分辨率模式。而在终端设备的屏幕处于亮屏状态时，可以关闭摄像模组。In another embodiment, when the screen of the terminal device is in the off state, the camera module is in the low-resolution mode. When the screen of the terminal device is in the on state, the camera module can be turned off.
关于预设模式切换条件，可以包括预先设置的从低分辨率模式切换至其他分辨率模式的条件，或者其他分辨率模式间的切换条件，或者从其他分辨率模式切换至低分辨率模式的条件等。The preset mode switching conditions may include pre-set conditions for switching from a low-resolution mode to other resolution modes, or switching conditions between other resolution modes, or conditions for switching from other resolution modes to a low-resolution mode, etc.
在一个实施例中，预设模式切换条件包括依据所述摄像模组在当前模式下采集到的DVS图像数据判定当前环境光的变化满足预设变化条件。In one embodiment, the preset mode switching condition includes determining, based on the DVS image data collected by the camera module in the current mode, whether the change in the current ambient light meets the preset change condition.
关于预设变化条件，可以是预先设置的依据环境光的变化进行模式切换的条件。示例的，预设变化条件可以是当前环境光的光强变化值大于设定阈值。DVS图像数据中事件数据流可以包括光照强度，因此，可以根据至少两帧图像的光照强度判断当前环境光的光强变化值是否大于设定阈值。在另一个示例中，不仅当前环境光的光强变化值大于设定阈值，还可以结合DVS图像数据中检测到光照发生变化的像素单元的个数，判断当前环境光的变化是否满足预设变化条件。The preset change condition may be a condition for switching modes in accordance with the change of ambient light. For example, the preset change condition may be that the light intensity change value of the current ambient light is greater than a set threshold. The event data stream in the DVS image data may include light intensity. Therefore, it is possible to determine whether the light intensity change value of the current ambient light is greater than the set threshold based on the light intensity of at least two frames of images. In another example, not only is the light intensity change value of the current ambient light greater than the set threshold, but it is also possible to determine whether the change in the current ambient light meets the preset change condition based on the number of pixel units in which the light change is detected in the DVS image data.
该条件可以是低分辨率模式切换至其他分辨率模式的条件，在依据摄像模组在低分辨率模式下采集的DVS图像数据，可以判断当前环境光变化是否满足预设变化条件，在预设变化条件满足时，可以触发摄像模组切换至下一级模式，否则可以维持低分辨率模式。示例的，在切换条件满足时，可以向摄像模组发切换通知，以使摄像模组进行模式切换。The condition may be a condition for switching from a low-resolution mode to another resolution mode. Based on the DVS image data collected by the camera module in the low-resolution mode, it may be determined whether the current ambient light change meets the preset change condition. When the preset change condition is met, the camera module may be triggered to switch to the next level mode, otherwise the low-resolution mode may be maintained. For example, when the switching condition is met, a switching notification may be sent to the camera module to enable the camera module to switch modes.
示例的，所述其他分辨率模式可以包括高分辨率模式(High Resolution，HR)。摄像模组在高分辨率模式下所有像素单元可以处于工作状态，以实现摄像模组在高分辨率模式采集到分辨率较高的图像数据。步骤102中获取所述摄像模组采集的DVS图像数据，可以包括：获取所述摄像模组在高分辨率模式下采集的DVS图像数据。相应的，所述方法还包括：For example, the other resolution modes may include a high resolution mode (High Resolution, HR). In the high resolution mode, all pixel units of the camera module may be in a working state, so that the camera module can collect image data with higher resolution in the high resolution mode. Acquiring the DVS image data collected by the camera module in step 102 may include: acquiring the DVS image data collected by the camera module in the high resolution mode. Accordingly, the method also includes:
获取所述摄像模组在低分辨率模式下采集的低分辨率DVS图像数据；Acquire low-resolution DVS image data collected by the camera module in a low-resolution mode;
在依据低分辨率DVS图像数据判定当前环境光的变化满足预设变化条件时，控制所述摄像模组从低分辨率模式切换至高分辨率模式。When it is determined based on the low-resolution DVS image data that the change in the current ambient light meets a preset change condition, the camera module is controlled to switch from the low-resolution mode to the high-resolution mode.
在该实施例中，常开时，DVS处于低分辨率、低分辨率模式，只检测环境光的变化。当检测到环境光变化大于设定阈值时，触发DVS高分辨率模式，识别是否是目标对象，从而保证能实时检测，同时还能降低功耗。In this embodiment, when normally on, the DVS is in low-resolution, low-resolution mode, and only detects changes in ambient light. When the ambient light change is detected to be greater than a set threshold, the DVS high-resolution mode is triggered to identify whether it is a target object, thereby ensuring real-time detection while reducing power consumption.
在另一个实施例中，是否设置多个其他分辨率模式可以依据识别目标对象所需图像分辨率要求决定，例如，一个实施例中，不同类型的目标对象映射不同操作指令，某些目标对象的识别只需要中分辨率的图像即可，有些目标对象的识别需要高分辨率图像，则可以配置多种等级的分辨率模式。In another embodiment, whether to set multiple other resolution modes can be determined based on the image resolution requirements for identifying the target object. For example, in one embodiment, different types of target objects map different operation instructions. The identification of some target objects only requires medium-resolution images, while the identification of some target objects requires high-resolution images. In this case, multiple levels of resolution modes can be configured.
是否设置多个其他分辨率模式还可以依据识别目标对象前是否有必要检测是否存在待识别对象来决定。示例的，某些场景中，可以先判断图像数据中是否存在待识别对象，再判断待识别对象是否为目标对象，从而提高识别准确率。存在待识别对象是进行目标对象识别的基础/前提。以目标对象为指定人脸为例，待识别对象为人脸，可以先判断图像数据中是否存在人脸，可以在确保存在人脸的情况下，再在判断人脸是否为指定人脸。以目标对象为身体姿势为例，待识别对象为人，可以先判断图像数据中是否存在人，在确保存在人的情况下，再判断该人的身体姿态。相应的，预设模式切换条件可以包括：依据所述摄像模组在当前模式下采集到的DVS图像数据，判定所述摄像模组的采集区域内存在待识别对象。Whether to set multiple other resolution modes can also be determined based on whether it is necessary to detect whether there is an object to be identified before identifying the target object. For example, in some scenarios, it is possible to first determine whether there is an object to be identified in the image data, and then determine whether the object to be identified is the target object, thereby improving the recognition accuracy. The existence of an object to be identified is the basis/prerequisite for target object identification. Taking the target object as a specified face as an example, the object to be identified is a face, and it can be determined whether there is a face in the image data first, and then determine whether the face is a specified face after ensuring that there is a face. Taking the target object as a body posture as an example, the object to be identified is a person, and it can be determined whether there is a person in the image data first, and then determine the body posture of the person after ensuring that there is a person. Correspondingly, the preset mode switching condition may include: based on the DVS image data collected by the camera module in the current mode, it is determined that there is an object to be identified in the acquisition area of the camera module.
其中，图像数据中存在待识别对象是进行目标对象识别的基础/前提。判断摄像模组的采集区域内是否存在待识别对象，可以是低分辨率模式切换至其他分辨率模式的切换条件，也可以是其他分辨率模式间的切换条件。The existence of the object to be identified in the image data is the basis/prerequisite for target object identification. Judging whether there is an object to be identified in the acquisition area of the camera module can be a switching condition for switching from a low-resolution mode to other resolution modes, or a switching condition between other resolution modes.
关于如何判断摄像模组的采集区域内是否存在待识别对象，在一个实施例中，可以依据图像数据中是否存在待识别对象的轮廓，来判断摄像模组的采集区域内是否存在待识别对象。可以理解的是，也可以采用其他手段判断摄像模组的采集区域内是否存在待识别对象，如采用相关技术中是否存在人脸/是否存在人的算法来判断摄像模组的采集区域内是否存在待识别对象。Regarding how to determine whether there is an object to be identified in the acquisition area of the camera module, in one embodiment, it can be determined whether there is an object to be identified in the acquisition area of the camera module based on whether there is a contour of the object to be identified in the image data. It is understandable that other means can also be used to determine whether there is an object to be identified in the acquisition area of the camera module, such as using an algorithm in the related art to determine whether there is a face/person to determine whether there is an object to be identified in the acquisition area of the camera module.
示例的，所述其他分辨率模式包括中分辨率模式(Middle Resolution，MR)和高分辨率模式，摄像模组在低分辨率模式、中分辨率模式、高分辨率模式下处于工作状态的像素单元个数依次增多。步骤102中获取所述摄像模组采集的DVS图像数据，可以包括：获取所述摄像模组在高分辨率模式下采集的DVS图像数据。相应的，所述方法还包括：For example, the other resolution modes include a medium resolution mode (Middle Resolution, MR) and a high resolution mode, and the number of pixel units in the camera module in the low resolution mode, the medium resolution mode, and the high resolution mode increases successively. Acquiring the DVS image data collected by the camera module in step 102 may include: acquiring the DVS image data collected by the camera module in the high resolution mode. Accordingly, the method also includes:
获取所述摄像模组在低分辨率模式下采集的低分辨率DVS图像数据；Acquire low-resolution DVS image data collected by the camera module in a low-resolution mode;
在依据低分辨率DVS图像数据判定当前环境光的变化满足预设变化条件时，控制所述摄像模组从低分辨率模式切换至中分辨率模式；When it is determined based on the low-resolution DVS image data that the change in the current ambient light meets a preset change condition, controlling the camera module to switch from the low-resolution mode to the medium-resolution mode;
获取所述摄像模组在中分辨率模式下采集的中分辨率DVS图像数据；Acquire medium-resolution DVS image data collected by the camera module in medium-resolution mode;
在依据所述摄像模组在中分辨率模式下采集到的DVS图像数据，判定所述摄像模组的采集区域内存在待识别对象时，控制所述摄像模组从中分辨率模式切换至高分辨率模式。When it is determined that there is an object to be identified in the acquisition area of the camera module based on the DVS image data acquired by the camera module in the medium resolution mode, the camera module is controlled to switch from the medium resolution mode to the high resolution mode.
该实施例配置三个等级的分辨率模式，逐次切换，可以降低功耗。This embodiment configures three levels of resolution modes, which are switched successively to reduce power consumption.
关于其他分辨率模式切换至低分辨率模式的条件，可以是依据低分辨率DVS图像数据判定当前环境光的变化值小于或等于设定阈值；或者，依据所述摄像模组在中分辨率模式下采集到的DVS图像数据，判定所述摄像模组的采集区域内不存在待识别对象；或者，控制终端设备执行与操作指令相匹配的操作后，且间隔预设延迟时间等。通过设置其他分辨率模式切换至低分辨率模式的条件，以保证摄像模组大部分时间处于低分辨率模式，进一步降低功耗。The conditions for switching from other resolution modes to low resolution modes can be: judging that the change value of the current ambient light is less than or equal to a set threshold value based on the low resolution DVS image data; or judging that there is no object to be identified in the acquisition area of the camera module based on the DVS image data collected by the camera module in the medium resolution mode; or controlling the terminal device to perform an operation matching the operation instruction and after a preset delay time, etc. By setting the conditions for switching from other resolution modes to low resolution modes, it is ensured that the camera module is in low resolution mode most of the time, further reducing power consumption.
应当理解的是，上述预设模式切换条件仅为举例，而不应被理解为对本公开的任何限制，其他现有的或将来的触发模式进行切换的条件可以适用本公开，均应包括在本公开的保护范围内。It should be understood that the above-mentioned preset mode switching conditions are only examples and should not be understood as any limitation to the present disclosure. Other existing or future conditions for triggering mode switching may be applicable to the present disclosure and should be included in the protection scope of the present disclosure.
以上实施方式中的各种技术特征可以任意进行组合，只要特征之间的组合不存在冲突或矛盾，但是限于篇幅，未进行一一描述，因此上述实施方式中的各种技术特征的任意进行组合也属于本说明书公开的范围。The various technical features in the above embodiments can be combined arbitrarily as long as there is no conflict or contradiction between the combinations of features. However, due to space limitations, they are not described one by one. Therefore, any combination of the various technical features in the above embodiments also falls within the scope of this specification.
以下以其中一种组合进行示例说明。The following is an example of one of the combinations.
如图5所示，图5是本公开根据一示例性实施例示出的另一种交互处理方法的流程图，所述方法可以用于终端设备，所述终端设备设置有包括动态视觉传感器DVS的摄像模组，所述摄像模组配置有低分辨率模式和高分辨率模式，所述方法包括：As shown in FIG. 5 , FIG. 5 is a flowchart of another interactive processing method according to an exemplary embodiment of the present disclosure, the method can be used for a terminal device, the terminal device is provided with a camera module including a dynamic vision sensor DVS, the camera module is configured with a low-resolution mode and a high-resolution mode, the method includes:
在步骤502中，获取所述摄像模组在低分辨率模式下采集的低分辨率DVS图像数据；In step 502, low-resolution DVS image data collected by the camera module in a low-resolution mode is obtained;
在步骤504中，依据低分辨率DVS图像数据，判断当前环境光的变化值是否大于设定阈值，若否，返回步骤502，若是，进入步骤506。In step 504 , it is determined whether the change value of the current ambient light is greater than a set threshold value based on the low-resolution DVS image data. If not, the process returns to step 502 . If so, the process proceeds to step 506 .
在步骤506中，控制所述摄像模组从低分辨率模式切换至高分辨率模式。In step 506, the camera module is controlled to switch from a low-resolution mode to a high-resolution mode.
在步骤508中，获取所述摄像模组在高分辨率模式下采集的DVS图像数据；In step 508, DVS image data collected by the camera module in high-resolution mode is obtained;
在步骤510中，从所述DVS图像数据中识别出用于指示终端设备执行操作的目标对象；In step 510, a target object for instructing a terminal device to perform an operation is identified from the DVS image data;
在步骤512中，响应于与所述目标对象对应的操作指令，控制终端设备执行与所述操作指令相匹配的操作。In step 512, in response to the operation instruction corresponding to the target object, the terminal device is controlled to perform an operation matching the operation instruction.
其中，图5与图1或图4中相关技术相同，在此不一一赘述。该实施例为摄像模组配置低分辨率模式LR和高分辨率模式HR，根据不同场景进行模式切换，对功率优化。5 is the same as the related technology in FIG1 or FIG4, and will not be described in detail here. In this embodiment, the camera module is configured with a low-resolution mode LR and a high-resolution mode HR, and the mode is switched according to different scenes to optimize power.
在一个示例中，可以将终端设备的操作限定为息屏状态下触发的操作，步骤502可以限定在终端设备的屏幕处于息屏状态下执行，而在终端设备的屏幕处于亮屏状态时，可以将摄像模组切换为低分辨率模式，或关闭摄像模组。该实施例利用DVS低功耗，可以实现息屏状态下目标对象识别，用户可以黑屏解锁，并且结合目标对象控制终端设备直接执行亮屏下的某指定操作。In one example, the operation of the terminal device can be limited to the operation triggered in the screen-off state, and step 502 can be limited to be executed when the screen of the terminal device is in the screen-off state, and when the screen of the terminal device is in the screen-on state, the camera module can be switched to a low-resolution mode or turned off. This embodiment uses DVS low power consumption to realize target object recognition in the screen-off state, and the user can unlock the black screen, and control the terminal device to directly perform a specified operation in the screen-on state in combination with the target object.
与前述交互处理方法的实施例相对应，本公开还提供了交互处理装置、装置所应用的设备以及存储介质的实施例。Corresponding to the embodiments of the aforementioned interaction processing method, the present disclosure also provides embodiments of an interaction processing apparatus, a device used by the apparatus, and a storage medium.
如图6所示，图6是本公开根据一示例性实施例示出的一种交互处理装置的框图，所述装置设于终端设备，所述终端设备设置有包括动态视觉传感器DVS的摄像模组，所述装置包括：As shown in FIG. 6 , FIG. 6 is a block diagram of an interactive processing device according to an exemplary embodiment of the present disclosure, wherein the device is provided in a terminal device, and the terminal device is provided with a camera module including a dynamic visual sensor DVS, and the device includes:
数据获取模块62，用于获取所述摄像模组采集的DVS图像数据；A data acquisition module 62, used to acquire DVS image data collected by the camera module;
对象识别模块64，用于从所述DVS图像数据中识别出用于指示终端设备执行操作的目标对象；An object recognition module 64, configured to recognize a target object for instructing a terminal device to perform an operation from the DVS image data;
操作控制模块66，用于响应于与所述目标对象对应的操作指令，控制终端设备执行与所述操作指令相匹配的操作。The operation control module 66 is used to control the terminal device to perform an operation matching the operation instruction in response to the operation instruction corresponding to the target object.
在一个实施例中，终端设备执行的操作至少包括点亮屏幕，所述获取所述摄像模组采集的DVS图像数据，包括：在终端设备的屏幕处于息屏状态时，获取所述摄像模组采集的DVS图像数据。In one embodiment, the operation performed by the terminal device at least includes lighting up the screen, and the obtaining of the DVS image data collected by the camera module includes: when the screen of the terminal device is in a screen-off state, obtaining the DVS image data collected by the camera module.
在一个实施例中，所述摄像模组配置有低分辨率模式、以及至少一个其他分辨率模式，所述摄像模组在低分辨率模式下处于工作状态的像素单元个数，少于所述摄像模组在其他分辨率模式下处于工作状态的像素单元个数，不同模式在预设模式切换条件满足时切换。In one embodiment, the camera module is configured with a low-resolution mode and at least one other resolution mode. The number of pixel units of the camera module that are in a working state in the low-resolution mode is less than the number of pixel units of the camera module that are in a working state in other resolution modes. Different modes are switched when a preset mode switching condition is met.
在一个实施例中，所述预设模式切换条件包括以下任一种条件：In one embodiment, the preset mode switching condition includes any one of the following conditions:
依据所述摄像模组在当前模式下采集到的DVS图像数据判定当前环境光的变化满足预设变化条件；Determining whether a change in current ambient light meets a preset change condition based on the DVS image data collected by the camera module in the current mode;
依据所述摄像模组在当前模式下采集到的DVS图像数据，判定所述摄像模组的采集区域内存在待识别对象。According to the DVS image data collected by the camera module in the current mode, it is determined that there is an object to be identified in the collection area of the camera module.
在一个实施例中，所述摄像模组以低分辨率模式处于常开状态，或者在终端设备的屏幕处于息屏状态时，所述摄像模组处于低分辨率模式。In one embodiment, the camera module is in a normally-on state in a low-resolution mode, or when the screen of the terminal device is in a screen-off state, the camera module is in a low-resolution mode.
在一个实施例中，如图7所示，图7是本公开根据一示例性实施例示出的另一种交互处理装置的框图，该实施例在前述图6所示实施例的基础上，所述其他分辨率模式包括高分辨率模式，所述数据获取模块62用于：获取所述摄像模组在高分辨率模式下采集的DVS图像数据；In one embodiment, as shown in FIG. 7 , FIG. 7 is a block diagram of another interactive processing device according to an exemplary embodiment of the present disclosure. In this embodiment, based on the embodiment shown in FIG. 6 , the other resolution modes include a high-resolution mode, and the data acquisition module 62 is used to: acquire DVS image data collected by the camera module in the high-resolution mode;
所述数据获取模块62还用于：获取所述摄像模组在低分辨率模式下采集的低分辨率DVS图像数据；The data acquisition module 62 is also used to: acquire low-resolution DVS image data collected by the camera module in a low-resolution mode;
所述装置还包括模式切换模块70，用于在依据低分辨率DVS图像数据判定当前环境光的变化满足预设变化条件时，控制所述摄像模组从低分辨率模式切换至高分辨率模式。The device further comprises a mode switching module 70 for controlling the camera module to switch from a low-resolution mode to a high-resolution mode when it is determined based on the low-resolution DVS image data that the change in the current ambient light satisfies a preset change condition.
在一个实施例中，如图8所示，图8是本公开根据一示例性实施例示出的另一种交互处理装置的框图，该实施例在前述图6所示实施例的基础上，所述其他分辨率模式包括中分辨率模式和高分辨率模式，所述数据获取模块62用于：获取所述摄像模组在高分辨率模式下采集的DVS图像数据；In one embodiment, as shown in FIG8 , FIG8 is a block diagram of another interactive processing device according to an exemplary embodiment of the present disclosure. In this embodiment, based on the embodiment shown in FIG6 , the other resolution modes include a medium resolution mode and a high resolution mode, and the data acquisition module 62 is used to: acquire DVS image data collected by the camera module in the high resolution mode;
所述数据获取模块62还用于：获取所述摄像模组在低分辨率模式下采集的低分辨率DVS图像数据；The data acquisition module 62 is also used to: acquire low-resolution DVS image data collected by the camera module in a low-resolution mode;
所述装置还包括模式切换模块80，用于在依据低分辨率DVS图像数据判定当前环境光的变化满足预设变化条件时，控制所述摄像模组从低分辨率模式切换至中分辨率模式；The device further includes a mode switching module 80 for controlling the camera module to switch from a low-resolution mode to a medium-resolution mode when it is determined based on the low-resolution DVS image data that the change in the current ambient light meets a preset change condition;
所述数据获取模块62还用于：获取所述摄像模组在中分辨率模式下采集的中分辨率DVS图像数据；The data acquisition module 62 is also used to: acquire medium-resolution DVS image data collected by the camera module in medium-resolution mode;
所述模式切换模块80还用于在依据所述摄像模组在中分辨率模式下采集到的DVS图像数据，判定所述摄像模组的采集区域内存在待识别对象时，控制所述摄像模组从中分辨率模式切换至高分辨率模式。The mode switching module 80 is also used to control the camera module to switch from the medium resolution mode to the high resolution mode when it is determined that there is an object to be identified in the acquisition area of the camera module based on the DVS image data acquired by the camera module in the medium resolution mode.
在一个实施例中，所述目标对象包括指定手势、指定人脸、和/或指定身体姿势。In one embodiment, the target object includes a specified gesture, a specified face, and/or a specified body posture.
在一个实施例中，预配置有目标对象与操作指令的映射关系，与操作指令相匹配的操作包括以下一种或多种：In one embodiment, a mapping relationship between a target object and an operation instruction is preconfigured, and the operations matching the operation instruction include one or more of the following:
在屏幕息屏状态下触发的解锁屏幕；Unlocking the screen when the screen is off;
在屏幕息屏状态下触发的启动手电筒；The flashlight is triggered when the screen is off;
在屏幕息屏状态下触发的启动指定应用程序；Start a specified application when the screen is off;
在屏幕息屏状态下触发的展示指定应用程序的指定页面；The specified page of the specified application is displayed when the screen is off;
在屏幕息屏状态下触发的展示指定应用程序的新消息；Display new messages of the specified application when the screen is off;
在屏幕息屏状态下接听拨号方的电话。Answer the call from the caller while the screen is off.
相应的，本公开还提供一种电子设备，所述设备包括基于动态视觉传感器DVS的摄像模组、存储器、处理器及存储在存储器上并可在处理器上运行的计算机程序，其中，所述处理器执行所述程序时实现如上任一项所述方法。Accordingly, the present disclosure also provides an electronic device, which includes a camera module based on a dynamic vision sensor DVS, a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements any of the methods described above when executing the program.
相应的，本公开还提供一种计算机可读存储介质，其上存储有计算机程序，该程序被处理器执行时实现上述任一项所述方法的步骤。Accordingly, the present disclosure also provides a computer-readable storage medium on which a computer program is stored, and when the program is executed by a processor, the steps of any of the above methods are implemented.
本公开可采用在一个或多个其中包含有程序代码的存储介质(包括但不限于磁盘存储器、CD-ROM、光学存储器等)上实施的计算机程序产品的形式。计算机可用存储介质包括永久性和非永久性、可移动和非可移动媒体，可以由任何方法或技术来实现信息存储。信息可以是计算机可读指令、数据结构、程序的模块或其他数据。计算机的存储介质的例子包括但不限于：相变内存(PRAM)、静态随机存取存储器(SRAM)、动态随机存取存储器(DRAM)、其他类型的随机存取存储器(RAM)、只读存储器(ROM)、电可擦除可编程只读存储器(EEPROM)、快闪记忆体或其他内存技术、只读光盘只读存储器(CD-ROM)、数字多功能光盘(DVD)或其他光学存储、磁盒式磁带，磁带磁磁盘存储或其他磁性存储设备或任何其他非传输介质，可用于存储可以被计算设备访问的信息。The present disclosure may take the form of a computer program product implemented on one or more storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing program code. Computer-usable storage media include permanent and non-permanent, removable and non-removable media, and information storage can be achieved by any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include but are not limited to: phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, read-only compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device.
上述装置中各个模块的功能和作用的实现过程具体详情见上述方法中对应步骤的实现过程，在此不再赘述。The specific details of the implementation process of the functions and effects of each module in the above-mentioned device can be found in the implementation process of the corresponding steps in the above-mentioned method, which will not be repeated here.
对于装置实施例而言，由于其基本对应于方法实施例，所以相关之处参见方法实施例的部分说明即可。以上所描述的装置实施例仅仅是示意性的，其中所述作为分离部件说明的模块可以是或者也可以不是物理上分开的，作为模块显示的部件可以是或者也可以不是物理模块，即可以位于一个地方，或者也可以分布到多个网络模块上。可以根据实际的需要选择其中的部分或者全部模块来实现本公开方案的目的。本领域普通技术人员在不付出创造性劳动的情况下，即可以理解并实施。For the device embodiment, since it basically corresponds to the method embodiment, the relevant parts can refer to the partial description of the method embodiment. The device embodiment described above is only schematic, wherein the modules described as separate components may or may not be physically separated, and the components displayed as modules may or may not be physical modules, that is, they may be located in one place, or they may be distributed on multiple network modules. Some or all of the modules may be selected according to actual needs to achieve the purpose of the disclosed scheme. A person of ordinary skill in the art can understand and implement it without paying creative labor.
如图9所示，图9是本公开根据一示例性实施例示出的一种交互处理装置所在计算机设备的硬件结构图。该装置900可以是移动电话，计算机，数字广播终端，消息收发设备，游戏控制台，平板设备，医疗设备，健身设备，个人数字助理等终端。As shown in Figure 9, Figure 9 is a hardware structure diagram of a computer device where an interactive processing device is located according to an exemplary embodiment of the present disclosure. The device 900 can be a mobile phone, a computer, a digital broadcast terminal, a message transceiver device, a game console, a tablet device, a medical device, a fitness device, a personal digital assistant, and other terminals.
参照图9，装置900可以包括以下一个或多个组件：处理组件902，存储器904，电源组件906，多媒体组件908，音频组件910，输入/输出(I/O)的接口912，传感器组件914，以及通信组件916。装置900还设置有包括动态视觉传感器的摄像模组，图9中未示出。9 , the device 900 may include one or more of the following components: a processing component 902, a memory 904, a power component 906, a multimedia component 908, an audio component 910, an input/output (I/O) interface 912, a sensor component 914, and a communication component 916. The device 900 is also provided with a camera module including a dynamic vision sensor, which is not shown in FIG. 9 .
处理组件902通常控制装置900的整体操作，诸如与显示，电话呼叫，数据通信，相机操作和记录操作相关联的操作。处理组件902可以包括一个或多个处理器920来执行指令，以完成上述的方法的全部或部分步骤。此外，处理组件902可以包括一个或多个模块，便于处理组件902和其他组件之间的交互。例如，处理组件902可以包括多媒体模块，以方便多媒体组件908和处理组件902之间的交互。The processing component 902 generally controls the overall operation of the device 900, such as operations associated with display, phone calls, data communications, camera operations, and recording operations. The processing component 902 may include one or more processors 920 to execute instructions to complete all or part of the steps of the above-mentioned method. In addition, the processing component 902 may include one or more modules to facilitate the interaction between the processing component 902 and other components. For example, the processing component 902 may include a multimedia module to facilitate the interaction between the multimedia component 908 and the processing component 902.
存储器904被配置为存储各种类型的数据以支持在装置900的操作。这些数据的示例包括用于在装置900上操作的任何应用程序或方法的指令，联系人数据，电话簿数据，消息，图片，视频等。存储器904可以由任何类型的易失性或非易失性存储设备或者它们的组合实现，如静态随机存取存储器(SRAM)，电可擦除可编程只读存储器(EEPROM)，可擦除可编程只读存储器(EPROM)，可编程只读存储器(PROM)，只读存储器(ROM)，磁存储器，快闪存储器，磁盘或光盘。The memory 904 is configured to store various types of data to support operations on the device 900. Examples of such data include instructions for any application or method operating on the device 900, contact data, phone book data, messages, pictures, videos, etc. The memory 904 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk.
电源组件906为装置900的各种组件提供电力。电源组件906可以包括电源管理系统，一个或多个电源，及其他与为装置900生成、管理和分配电力相关联的组件。The power supply component 906 provides power to the various components of the device 900. The power supply component 906 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power for the device 900.
多媒体组件908包括在所述装置900和用户之间的提供一个输出接口的屏幕。在一些实施例中，屏幕可以包括液晶显示器(LCD)和触摸面板(TP)。如果屏幕包括触摸面板，屏幕可以被实现为触摸屏，以接收来自用户的输入信号。触摸面板包括一个或多个触摸传感器以感测触摸、滑动和触摸面板上的手势。所述触摸传感器可以不仅感测触摸或滑动动作的边界，而且还检测与所述触摸或滑动操作相关的持续时间和压力。在一些实施例中，多媒体组件908包括一个前置摄像头和/或后置摄像头。当装置900处于操作模式，如拍摄模式或视频模式时，前置摄像头和/或后置摄像头可以接收外部的多媒体数据。每个前置摄像头和后置摄像头可以是一个固定的光学透镜系统或具有焦距和光学变焦能力。The multimedia component 908 includes a screen that provides an output interface between the device 900 and the user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touch screen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touch, slide, and gestures on the touch panel. The touch sensor may not only sense the boundaries of the touch or slide action, but also detect the duration and pressure associated with the touch or slide operation. In some embodiments, the multimedia component 908 includes a front camera and/or a rear camera. When the device 900 is in an operating mode, such as a shooting mode or a video mode, the front camera and/or the rear camera may receive external multimedia data. Each front camera and rear camera may be a fixed optical lens system or have a focal length and optical zoom capability.
音频组件910被配置为输出和/或输入音频信号。例如，音频组件910包括一个麦克风(MIC)，当装置900处于操作模式，如呼叫模式、记录模式和语音识别模式时，麦克风被配置为接收外部音频信号。所接收的音频信号可以被进一步存储在存储器904或经由通信组件916发送。在一些实施例中，音频组件910还包括一个扬声器，用于输出音频信号。The audio component 910 is configured to output and/or input audio signals. For example, the audio component 910 includes a microphone (MIC), and when the device 900 is in an operating mode, such as a call mode, a recording mode, and a speech recognition mode, the microphone is configured to receive an external audio signal. The received audio signal can be further stored in the memory 904 or sent via the communication component 916. In some embodiments, the audio component 910 also includes a speaker for outputting audio signals.
I/O接口912为处理组件902和外围接口模块之间提供接口，上述外围接口模块可以是键盘，点击轮，按钮等。这些按钮可包括但不限于：主页按钮、音量按钮、启动按钮和锁定按钮。I/O interface 912 provides an interface between processing component 902 and peripheral interface modules, such as keyboards, click wheels, buttons, etc. These buttons may include but are not limited to: home button, volume button, start button, and lock button.
传感器组件914包括一个或多个传感器，用于为装置900提供各个方面的状态评估。例如，传感器组件914可以检测到装置900的打开/关闭状态，组件的相对定位，例如所述组件为装置900的显示器和小键盘，传感器组件914还可以检测装置900或装置900中一个组件的位置改变，用户与装置900接触的存在或不存在，装置900方位或加速/减速和装置900的温度变化。传感器组件914可以包括接近传感器，被配置用来在没有任何的物理接触时检测附近物体的存在。传感器组件914还可以包括光传感器，如CMOS或CCD图像传感器，用于在成像应用中使用。在一些实施例中，该传感器组件914还可以包括加速度传感器，陀螺仪传感器，磁传感器，压力传感器或温度传感器。The sensor assembly 914 includes one or more sensors for providing various aspects of status assessment for the device 900. For example, the sensor assembly 914 can detect the open/closed state of the device 900, the relative positioning of components, such as the display and keypad of the device 900, and the sensor assembly 914 can also detect the position change of the device 900 or a component in the device 900, the presence or absence of user contact with the device 900, the orientation or acceleration/deceleration of the device 900, and the temperature change of the device 900. The sensor assembly 914 may include a proximity sensor configured to detect the presence of nearby objects without any physical contact. The sensor assembly 914 may also include an optical sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, the sensor assembly 914 may also include an accelerometer, a gyroscope sensor, a magnetic sensor, a pressure sensor, or a temperature sensor.
通信组件916被配置为便于装置900和其他设备之间有线或无线方式的通信。装置900可以接入基于通信标准的无线网络，如WiFi，2G或3G，或它们的组合。在一个示例性实施例中，通信组件916经由广播信道接收来自外部广播管理系统的广播信号或广播相关信息。在一个示例性实施例中，所述通信组件916还包括近场通信(NFC)模块，以促进短程通信。例如，在NFC模块可基于射频识别(RFID)技术，红外数据协会(IrDA)技术，超宽带(UWB)技术，蓝牙(BT)技术和其他技术来实现。The communication component 916 is configured to facilitate wired or wireless communication between the device 900 and other devices. The device 900 can access a wireless network based on a communication standard, such as WiFi, 2G or 3G, or a combination thereof. In an exemplary embodiment, the communication component 916 receives a broadcast signal or broadcast-related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component 916 also includes a near field communication (NFC) module to facilitate short-range communication. For example, the NFC module can be implemented based on radio frequency identification (RFID) technology, infrared data association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology and other technologies.
在示例性实施例中，装置900可以被一个或多个应用专用集成电路(ASIC)、数字信号处理器(DSP)、数字信号处理设备(DSPD)、可编程逻辑器件(PLD)、现场可编程门阵列(FPGA)、控制器、微控制器、微处理器或其他电子元件实现，用于执行上述方法。In an exemplary embodiment, the apparatus 900 may be implemented by one or more application specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors or other electronic components to perform the above method.
在示例性实施例中，还提供了一种包括指令的非临时性计算机可读存储介质，例如包括指令的存储器904，上述指令可由装置900的处理器920执行以完成上述方法。例如，所述非临时性计算机可读存储介质可以是ROM、随机存取存储器(RAM)、CD-ROM、磁带、软盘和光数据存储设备等。In an exemplary embodiment, a non-transitory computer-readable storage medium including instructions is also provided, such as a memory 904 including instructions, and the instructions can be executed by the processor 920 of the device 900 to perform the above method. For example, the non-transitory computer-readable storage medium can be a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, etc.
其中，当所述存储介质中的指令由所述处理器执行时，使得装置900能够执行上述任一种交互处理方法。When the instructions in the storage medium are executed by the processor, the device 900 is enabled to execute any of the above-mentioned interaction processing methods.
本领域技术人员在考虑说明书及实践这里公开的发明后，将容易想到本公开的其它实施方案。本公开旨在涵盖本公开的任何变型、用途或者适应性变化，这些变型、用途或者适应性变化遵循本公开的一般性原理并包括本公开未公开的本技术领域中的公知常识或惯用技术手段。说明书和实施例仅被视为示例性的，本公开的真正范围和精神由下面的权利要求指出。Those skilled in the art will readily appreciate other embodiments of the present disclosure after considering the specification and practicing the invention disclosed herein. The present disclosure is intended to cover any variations, uses, or adaptations of the present disclosure that follow the general principles of the present disclosure and include common knowledge or customary techniques in the art that are not disclosed in the present disclosure. The description and examples are to be considered exemplary only, and the true scope and spirit of the present disclosure are indicated by the following claims.
应当理解的是，本公开并不局限于上面已经描述并在附图中示出的精确结构，并且可以在不脱离其范围进行各种修改和改变。本公开的范围仅由所附的权利要求来限制。It should be understood that the present disclosure is not limited to the exact structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.
以上所述仅为本公开的较佳实施例而已，并不用以限制本公开，凡在本公开的精神和原则之内，所做的任何修改、等同替换、改进等，均应包含在本公开保护的范围之内。The above description is only a preferred embodiment of the present disclosure and is not intended to limit the present disclosure. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present disclosure should be included in the scope of protection of the present disclosure.
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910425579.9ACN111984347B (en) | 2019-05-21 | 2019-05-21 | Interactive processing method, device, equipment and storage medium |
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910425579.9ACN111984347B (en) | 2019-05-21 | 2019-05-21 | Interactive processing method, device, equipment and storage medium |
| Publication Number | Publication Date |
|---|---|
| CN111984347A CN111984347A (en) | 2020-11-24 |
| CN111984347Btrue CN111984347B (en) | 2024-09-17 |
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910425579.9AActiveCN111984347B (en) | 2019-05-21 | 2019-05-21 | Interactive processing method, device, equipment and storage medium |
| Country | Link |
|---|---|
| CN (1) | CN111984347B (en) |
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112989925B (en)* | 2021-02-02 | 2022-06-10 | 豪威芯仑传感器(上海)有限公司 | A method and system for recognizing hand sliding direction |
| CN112929491B (en)* | 2021-02-07 | 2022-08-26 | 展讯通信（上海）有限公司 | Application program starting method and related device |
| CN112949440A (en)* | 2021-02-22 | 2021-06-11 | 豪威芯仑传感器(上海)有限公司 | Method for extracting gait features of pedestrian, gait recognition method and system |
| CN115550539A (en)* | 2021-06-30 | 2022-12-30 | 北京小米移动软件有限公司 | Function control method, function control device and storage medium |
| CN115278013B (en)* | 2022-07-22 | 2024-10-22 | 杭州研极微电子有限公司 | Video camera and method for controlling the same |
| CN115242952B (en)* | 2022-07-28 | 2024-07-26 | 联想(北京)有限公司 | Image acquisition method and device |
| CN116708655B (en)* | 2022-10-20 | 2024-05-03 | 荣耀终端有限公司 | Screen control method and electronic device based on event camera |
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106446873A (en)* | 2016-11-03 | 2017-02-22 | 北京旷视科技有限公司 | Face detection method and device |
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2014145925A1 (en)* | 2013-03-15 | 2014-09-18 | Moontunes, Inc. | Systems and methods for controlling cameras at live events |
| CN107479771B (en)* | 2017-07-21 | 2020-10-23 | 浙江天益塑业有限公司 | Control method, terminal and storage medium |
| CN107734129A (en)* | 2017-09-27 | 2018-02-23 | 广东欧珀移动通信有限公司 | Unlock control method and related products |
| CN107993194B (en)* | 2017-11-30 | 2021-01-01 | 天津大学 | Super-resolution reconstruction method based on stationary wavelet transform |
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106446873A (en)* | 2016-11-03 | 2017-02-22 | 北京旷视科技有限公司 | Face detection method and device |
| Publication number | Publication date |
|---|---|
| CN111984347A (en) | 2020-11-24 |
| Publication | Publication Date | Title |
|---|---|---|
| CN111984347B (en) | Interactive processing method, device, equipment and storage medium | |
| CN112118380B (en) | Camera manipulation method, device, device and storage medium | |
| CN111988493B (en) | Interaction processing method, device, equipment and storage medium | |
| CN106572299B (en) | Camera opening method and device | |
| EP3136793A1 (en) | Method and apparatus for awakening electronic device | |
| CN105260732A (en) | Image processing method and device | |
| EP3299946B1 (en) | Method and device for switching environment picture | |
| CN106101629A (en) | The method and device of output image | |
| CN108668080A (en) | Method, device, and electronic device for prompting the degree of lens dirt | |
| WO2017084183A1 (en) | Information displaying method and device | |
| CN112148149B (en) | Touch screen control method, touch screen control device and storage medium | |
| CN107704190B (en) | Gesture recognition method and device, terminal and storage medium | |
| CN106326849A (en) | Beauty processing method and device | |
| WO2017080084A1 (en) | Font addition method and apparatus | |
| CN106997281A (en) | The method and smart machine of shared virtual objects | |
| JP2017532618A (en) | Information processing method and apparatus | |
| CN107529699A (en) | Control method of electronic device and device | |
| WO2024067468A1 (en) | Interaction control method and apparatus based on image recognition, and device | |
| WO2019006768A1 (en) | Parking space occupation method and device based on unmanned aerial vehicle | |
| CN107132983B (en) | Split screen window operation method and device | |
| CN112114653B (en) | Terminal device control method, device, equipment and storage medium | |
| CN109922203A (en) | Terminal puts out screen method and apparatus | |
| CN106896917A (en) | Aid in method and device, the electronic equipment of Consumer's Experience virtual reality | |
| CN111538450A (en) | Theme background display method, device and storage medium | |
| CN112423092A (en) | Video recording method and video recording device |
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |