CN111984347A - Interactive processing method, apparatus, device and storage medium - Google Patents
Interactive processing method, apparatus, device and storage mediumDownload PDFInfo
- Publication number
- CN111984347A CN111984347ACN201910425579.9ACN201910425579ACN111984347ACN 111984347 ACN111984347 ACN 111984347ACN 201910425579 ACN201910425579 ACN 201910425579ACN 111984347 ACN111984347 ACN 111984347A
- Authority
- CN
- China
- Prior art keywords
- camera module
- image data
- resolution mode
- screen
- resolution
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/44—Arrangements for executing specific programs
- G06F9/451—Execution arrangements for user interfaces
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/44—Arrangements for executing specific programs
- G06F9/445—Program loading or initiating
- G06F9/44505—Configuring for program initiating, e.g. using registry, configuration files
Landscapes
- Engineering & Computer Science (AREA)
- Software Systems (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Human Computer Interaction (AREA)
- User Interface Of Digital Computer (AREA)
- Telephone Function (AREA)
Abstract
Translated fromChinese
Description
Translated fromChinese技术领域technical field
本申请涉及计算机技术领域，尤其涉及交互处理方法、装置、设备及存储介质。The present application relates to the field of computer technology, and in particular, to an interaction processing method, apparatus, device, and storage medium.
背景技术Background technique
随着科技的快速发展，各种电子设备不断涌现，例如个人计算机、平板电脑、智能手机等等。拥有自然交互方式的电子设备也受到越来越多人的青睐。因此，智能设备和使用者之间的交互就成为各大智能终端厂商的研发重点，出现了多种多样的在智能终端上实现与使用者操作交互的技术方案。然而，现有技术中大部分都是基于触摸或者语音进行人机交互，用户使用体验没有很大改变，在不方便进行触摸控制或语音控制等特殊情况下，用户无法对电子设备进行操作。With the rapid development of technology, various electronic devices are emerging, such as personal computers, tablet computers, smart phones and so on. Electronic devices with natural interaction methods are also favored by more and more people. Therefore, the interaction between the smart device and the user has become the focus of research and development of major smart terminal manufacturers, and a variety of technical solutions have emerged to realize the interaction with the user on the smart terminal. However, most of the existing technologies are based on touch or voice for human-computer interaction, and the user experience is not greatly changed. In special cases such as inconvenient touch control or voice control, the user cannot operate the electronic device.
发明内容SUMMARY OF THE INVENTION
为克服相关技术中存在的问题，本公开提供了交互处理方法、装置、设备及存储介质。To overcome the problems in the related art, the present disclosure provides an interaction processing method, apparatus, device, and storage medium.
根据本公开实施例的第一方面，提供一种交互处理方法，所述方法应用于终端设备，所述终端设备设置有包括动态视觉传感器DVS的摄像模组，所述方法包括：According to a first aspect of the embodiments of the present disclosure, an interaction processing method is provided. The method is applied to a terminal device, and the terminal device is provided with a camera module including a dynamic vision sensor DVS, and the method includes:
获取所述摄像模组采集的DVS图像数据；obtaining the DVS image data collected by the camera module;
从所述DVS图像数据中识别出用于指示终端设备执行操作的目标对象；Identify the target object used to instruct the terminal device to perform an operation from the DVS image data;
响应于与所述目标对象对应的操作指令，控制终端设备执行与所述操作指令相匹配的操作。In response to an operation instruction corresponding to the target object, the terminal device is controlled to perform an operation matching the operation instruction.
在一个实施例中，终端设备执行的操作至少包括点亮屏幕，所述获取所述摄像模组采集的DVS图像数据，包括：在终端设备的屏幕处于息屏状态时，获取所述摄像模组采集的DVS图像数据。In one embodiment, the operation performed by the terminal device at least includes lighting the screen, and the acquiring the DVS image data collected by the camera module includes: when the screen of the terminal device is in a screen-off state, acquiring the camera module Acquired DVS image data.
在一个实施例中，所述摄像模组配置有低分辨率模式、以及至少一个其他分辨率模式，所述摄像模组在低分辨率模式下处于工作状态的像素单元个数，少于所述摄像模组在其他分辨率模式下处于工作状态的像素单元个数，不同模式在预设模式切换条件满足时切换。In one embodiment, the camera module is configured with a low-resolution mode and at least one other resolution mode, and the number of pixel units in a working state of the camera module in the low-resolution mode is less than the number of pixel units in the low-resolution mode. The number of pixel units in the working state of the camera module in other resolution modes. Different modes are switched when the preset mode switching conditions are met.
在一个实施例中，所述摄像模组以低分辨率模式处于常开状态，或者在终端设备的屏幕处于息屏状态时，所述摄像模组处于低分辨率模式。In one embodiment, the camera module is in a normally-on state in a low-resolution mode, or when the screen of the terminal device is in an off-screen state, the camera module is in a low-resolution mode.
在一个实施例中，所述预设模式切换条件包括以下任一种条件：In one embodiment, the preset mode switching condition includes any of the following conditions:
依据所述摄像模组在当前模式下采集到的DVS图像数据判定当前环境光的变化满足预设变化条件；According to the DVS image data collected by the camera module in the current mode, it is determined that the change of the current ambient light satisfies the preset change condition;
依据所述摄像模组在当前模式下采集到的DVS图像数据，判定所述摄像模组的采集区域内存在待识别对象。According to the DVS image data collected by the camera module in the current mode, it is determined that there is an object to be identified in the collection area of the camera module.
在一个实施例中，所述其他分辨率模式包括高分辨率模式，所述获取所述摄像模组采集的DVS图像数据，包括：获取所述摄像模组在高分辨率模式下采集的DVS图像数据；In one embodiment, the other resolution modes include a high-resolution mode, and the acquiring the DVS image data collected by the camera module includes: acquiring the DVS image collected by the camera module in the high-resolution mode data;
所述方法还包括：The method also includes:
获取所述摄像模组在低分辨率模式下采集的低分辨率DVS图像数据；acquiring low-resolution DVS image data collected by the camera module in a low-resolution mode;
在依据低分辨率DVS图像数据判定当前环境光的变化满足预设变化条件时，控制所述摄像模组从低分辨率模式切换至高分辨率模式。When it is determined according to the low-resolution DVS image data that the change of the current ambient light satisfies the preset change condition, the camera module is controlled to switch from the low-resolution mode to the high-resolution mode.
在一个实施例中，所述其他分辨率模式包括中分辨率模式和高分辨率模式，所述获取所述摄像模组采集的DVS图像数据，包括：获取所述摄像模组在高分辨率模式下采集的DVS图像数据；In one embodiment, the other resolution modes include a medium resolution mode and a high resolution mode, and the acquiring the DVS image data collected by the camera module includes: acquiring the camera module in the high-resolution mode DVS image data collected under the
所述方法还包括：The method also includes:
获取所述摄像模组在低分辨率模式下采集的低分辨率DVS图像数据；acquiring low-resolution DVS image data collected by the camera module in a low-resolution mode;
在依据低分辨率DVS图像数据判定当前环境光的变化满足预设变化条件时，控制所述摄像模组从低分辨率模式切换至中分辨率模式；When it is determined according to the low-resolution DVS image data that the change of the current ambient light satisfies the preset change condition, controlling the camera module to switch from the low-resolution mode to the medium-resolution mode;
获取所述摄像模组在中分辨率模式下采集的中分辨率DVS图像数据；Obtain the medium-resolution DVS image data collected by the camera module in the medium-resolution mode;
在依据所述摄像模组在中分辨率模式下采集到的DVS图像数据，判定所述摄像模组的采集区域内存在待识别对象时，控制所述摄像模组从中分辨率模式切换至高分辨率模式。When it is determined that there is an object to be recognized in the collection area of the camera module according to the DVS image data collected by the camera module in the medium resolution mode, the camera module is controlled to switch from the medium resolution mode to the high resolution model.
在一个实施例中，所述目标对象包括指定手势、指定人脸、和/或指定身体姿势。In one embodiment, the target object includes a designated gesture, a designated face, and/or a designated body posture.
在一个实施例中，预配置有目标对象与操作指令的映射关系，与操作指令相匹配的操作包括以下一种或多种：In one embodiment, the mapping relationship between the target object and the operation instruction is preconfigured, and the operation matching the operation instruction includes one or more of the following:
在屏幕息屏状态下触发的解锁屏幕；Unlock the screen triggered when the screen is off;
在屏幕息屏状态下触发的启动手电筒；Start flashlight triggered when the screen is off;
在屏幕息屏状态下触发的启动指定应用程序；Triggered to start the specified application when the screen is off;
在屏幕息屏状态下触发的展示指定应用程序的指定页面；Triggered to display the specified page of the specified application when the screen is off;
在屏幕息屏状态下触发的展示指定应用程序的新消息；Triggered to display a new message of the specified application when the screen is off;
在屏幕息屏状态下接听拨号方的电话。Answer the call of the caller with the screen off.
根据本公开实施例的第二方面，提供一种交互处理装置，所述装置设于终端设备，所述终端设备设置有包括动态视觉传感器DVS的摄像模组，所述装置包括：According to a second aspect of the embodiments of the present disclosure, an interaction processing apparatus is provided, the apparatus is provided in a terminal device, and the terminal device is provided with a camera module including a dynamic vision sensor DVS, and the apparatus includes:
数据获取模块，用于获取所述摄像模组采集的DVS图像数据；a data acquisition module for acquiring the DVS image data collected by the camera module;
对象识别模块，用于从所述DVS图像数据中识别出用于指示终端设备执行操作的目标对象；an object identification module for identifying a target object for instructing the terminal device to perform an operation from the DVS image data;
操作控制模块，用于响应于与所述目标对象对应的操作指令，控制终端设备执行与所述操作指令相匹配的操作。The operation control module is configured to control the terminal device to perform an operation matching the operation instruction in response to the operation instruction corresponding to the target object.
在一个实施例中，终端设备执行的操作至少包括点亮屏幕，所述获取所述摄像模组采集的DVS图像数据，包括：在终端设备的屏幕处于息屏状态时，获取所述摄像模组采集的DVS图像数据。In one embodiment, the operation performed by the terminal device at least includes lighting the screen, and the acquiring the DVS image data collected by the camera module includes: when the screen of the terminal device is in a screen-off state, acquiring the camera module Acquired DVS image data.
在一个实施例中，所述摄像模组配置有低分辨率模式、以及至少一个其他分辨率模式，所述摄像模组在低分辨率模式下处于工作状态的像素单元个数，少于所述摄像模组在其他分辨率模式下处于工作状态的像素单元个数，不同模式在预设模式切换条件满足时切换。In one embodiment, the camera module is configured with a low-resolution mode and at least one other resolution mode, and the number of pixel units in a working state of the camera module in the low-resolution mode is less than the number of pixel units in the low-resolution mode. The number of pixel units in the working state of the camera module in other resolution modes. Different modes are switched when the preset mode switching conditions are met.
在一个实施例中，所述预设模式切换条件包括以下任一种条件：In one embodiment, the preset mode switching condition includes any of the following conditions:
依据所述摄像模组在当前模式下采集到的DVS图像数据判定当前环境光的变化满足预设变化条件；According to the DVS image data collected by the camera module in the current mode, it is determined that the change of the current ambient light satisfies the preset change condition;
依据所述摄像模组在当前模式下采集到的DVS图像数据，判定所述摄像模组的采集区域内存在待识别对象。According to the DVS image data collected by the camera module in the current mode, it is determined that there is an object to be identified in the collection area of the camera module.
根据本公开实施例的第三方面，提供一种计算机设备，包括基于动态视觉传感器DVS的摄像模组、存储器、处理器及存储在存储器上并可在处理器上运行的计算机程序，其中，所述处理器执行所述程序时实现如上任一项所述方法。According to a third aspect of the embodiments of the present disclosure, a computer device is provided, including a camera module based on a dynamic vision sensor DVS, a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the When the processor executes the program, the method described in any of the above is implemented.
根据本公开实施例的第四方面，提供一种计算机可读存储介质，其上存储有计算机程序，该程序被处理器执行时实现上述任一项所述方法的步骤。According to a fourth aspect of the embodiments of the present disclosure, there is provided a computer-readable storage medium on which a computer program is stored, and when the program is executed by a processor, implements the steps of any one of the methods described above.
本公开的实施例提供的技术方案可以包括以下有益效果：The technical solutions provided by the embodiments of the present disclosure may include the following beneficial effects:
本实施例通过在终端设备上设置包含动态视觉传感器摄像模组，获取该摄像模组采集的DVS图像数据，并从DVS图像数据中识别出用于指示终端设备执行操作的目标对象，响应于与目标对象对应的操作指令，控制终端设备执行与该操作指令相匹配的操作，从而实现无需触摸和语音输入的情况下，也能与终端设备进行交互，且由于DVS图像数据仅包括亮度发生变化的像素单元的数据，因此，数据量低、且响应速度快。In this embodiment, a camera module including a dynamic vision sensor is set on the terminal device, the DVS image data collected by the camera module is acquired, and the target object used to instruct the terminal device to perform an operation is identified from the DVS image data, and in response to the The operation instruction corresponding to the target object controls the terminal device to perform the operation matching the operation instruction, so as to realize the interaction with the terminal device without touch and voice input. The data of the pixel unit, therefore, the amount of data is low, and the response speed is fast.
应当理解的是，以上的一般描述和后文的细节描述仅是示例性和解释性的，并不能限制本公开。It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure.
附图说明Description of drawings
此处的附图被并入说明书中并构成本说明书的一部分，示出了符合本公开的实施例，并与说明书一起用于解释本公开的原理。The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the disclosure and together with the description serve to explain the principles of the disclosure.
图1是本公开根据一示例性实施例示出的一种交互处理方法的流程图。FIG. 1 is a flowchart of an interaction processing method according to an exemplary embodiment of the present disclosure.
图2是本公开根据一示例性实施例示出的一种摄像模组设置位置示意图。FIG. 2 is a schematic diagram showing the arrangement position of a camera module according to an exemplary embodiment of the present disclosure.
图3是本公开根据一示例性实施例示出的几种手势示意图。FIG. 3 is a schematic diagram of several gestures according to an exemplary embodiment of the present disclosure.
图4是本公开根据一示例性实施例示出的另一种交互处理方法的流程图。FIG. 4 is a flowchart of another interaction processing method according to an exemplary embodiment of the present disclosure.
图5是本公开根据一示例性实施例示出的另一种交互处理方法的流程图。FIG. 5 is a flowchart of another interaction processing method shown in the present disclosure according to an exemplary embodiment.
图6是本公开根据一示例性实施例示出的一种交互处理装置的框图。Fig. 6 is a block diagram of an interaction processing apparatus according to an exemplary embodiment of the present disclosure.
图7是本公开根据一示例性实施例示出的另一种交互处理装置的框图。FIG. 7 is a block diagram of another interaction processing apparatus according to an exemplary embodiment of the present disclosure.
图8是本公开根据一示例性实施例示出的另一种交互处理装置的框图。FIG. 8 is a block diagram of another interaction processing apparatus according to an exemplary embodiment of the present disclosure.
图9是本公开根据一示例性实施例示出的一种交互处理装置所在计算机设备的硬件结构图。FIG. 9 is a hardware structure diagram of a computer device where an interaction processing apparatus is located according to an exemplary embodiment of the present disclosure.
具体实施方式Detailed ways
这里将详细地对示例性实施例进行说明，其示例表示在附图中。下面的描述涉及附图时，除非另有表示，不同附图中的相同数字表示相同或相似的要素。以下示例性实施例中所描述的实施方式并不代表与本公开相一致的所有实施方式。相反，它们仅是与如所附权利要求书中所详述的、本公开的一些方面相一致的装置和方法的例子。Exemplary embodiments will be described in detail herein, examples of which are illustrated in the accompanying drawings. Where the following description refers to the drawings, the same numerals in different drawings refer to the same or similar elements unless otherwise indicated. The implementations described in the illustrative examples below are not intended to represent all implementations consistent with this disclosure. Rather, they are merely examples of apparatus and methods consistent with some aspects of the present disclosure as recited in the appended claims.
在本公开使用的术语是仅仅出于描述特定实施例的目的，而非旨在限制本公开。在本公开和所附权利要求书中所使用的单数形式的“一种”、“所述”和“该”也旨在包括多数形式，除非上下文清楚地表示其他含义。还应当理解，本文中使用的术语“和/或”是指并包含一个或多个相关联的列出项目的任何或所有可能组合。The terminology used in the present disclosure is for the purpose of describing particular embodiments only and is not intended to limit the present disclosure. As used in this disclosure and the appended claims, the singular forms "a," "the," and "the" are intended to include the plural forms as well, unless the context clearly dictates otherwise. It will also be understood that the term "and/or" as used herein refers to and includes any and all possible combinations of one or more of the associated listed items.
应当理解，尽管在本公开可能采用术语第一、第二、第三等来描述各种信息，但这些信息不应限于这些术语。这些术语仅用来将同一类型的信息彼此区分开。例如，在不脱离本公开范围的情况下，第一信息也可以被称为第二信息，类似地，第二信息也可以被称为第一信息。取决于语境，如在此所使用的词语“如果”可以被解释成为“在……时”或“当……时”或“响应于确定”。It should be understood that although the terms first, second, third, etc. may be used in this disclosure to describe various pieces of information, such information should not be limited by these terms. These terms are only used to distinguish the same type of information from each other. For example, the first information may also be referred to as the second information, and similarly, the second information may also be referred to as the first information, without departing from the scope of the present disclosure. Depending on the context, the word "if" as used herein can be interpreted as "at the time of" or "when" or "in response to determining."
随着智能终端的广泛使用，越来越多的人离不开手机等终端设备。终端设备和使用者之间的交互就成为各大终端厂商的研发重点，出现了多种多样的在终端设备上实现与使用者操作交互的技术方案。例如，触摸屏幕上的人机交互方案，基于语音的人机交互方案等。在某些不方便进行触摸控制或语音控制等特殊情况下，用户无法对电子设备进行操作，影响用户体验。With the widespread use of smart terminals, more and more people are inseparable from terminal devices such as mobile phones. The interaction between terminal equipment and users has become the focus of research and development of major terminal manufacturers, and a variety of technical solutions for realizing interaction with users on terminal equipment have emerged. For example, human-computer interaction scheme on touch screen, voice-based human-computer interaction scheme, etc. In some special cases where it is inconvenient to perform touch control or voice control, the user cannot operate the electronic device, which affects the user experience.
鉴于此，本申请实施例提供一种交互方案，通过在终端设备上设置包含动态视觉传感器摄像模组，获取该摄像模组采集的DVS图像数据，并从DVS图像数据中识别出用于指示终端设备执行操作的目标对象，响应于与目标对象对应的操作指令，控制终端设备执行与该操作指令相匹配的操作，从而实现无需触摸和语音输入的情况下，也能与终端设备进行交互，且由于DVS图像数据包括检测到光强发生变化的像素单元的数据，因此，数据量低、且响应速度快。In view of this, an embodiment of the present application provides an interactive solution, by setting a camera module including a dynamic vision sensor on a terminal device, acquiring DVS image data collected by the camera module, and identifying the DVS image data from the DVS image data for indicating the terminal The target object of the device performing the operation, in response to the operation instruction corresponding to the target object, controls the terminal device to perform the operation matching the operation instruction, so as to realize the interaction with the terminal device without touch and voice input, and Since the DVS image data includes the data of the pixel unit where the change of the light intensity is detected, the data amount is low and the response speed is fast.
本实施例提供的交互处理方法可以通过软件执行，也可以通过软件和硬件相结合或者硬件执行的方式实现，所涉及的硬件可以由两个或多个物理实体构成，也可以由一个物理实体构成。本实施例方法可以应用于具有所述摄像模组的电子设备。其中，电子设备可以是智能手机、智能学习机、平板电脑、笔记本电脑、PDA(Personal Digital Assistant，个人数字助理)等便携式设备，也可以是台式电脑等固定设备，还可以是智能手环、智能项链等可穿戴设备。The interaction processing method provided in this embodiment may be implemented by software, a combination of software and hardware, or implemented by hardware. The involved hardware may be composed of two or more physical entities, or may be composed of one physical entity . The method of this embodiment can be applied to an electronic device having the camera module. Among them, the electronic device may be a portable device such as a smart phone, an intelligent learning machine, a tablet computer, a notebook computer, a PDA (Personal Digital Assistant, personal digital assistant), a fixed device such as a desktop computer, or a smart bracelet, a smart Wearables such as necklaces.
动态视觉传感器(Dynamic Vision Sensor，DVS)，也可以称为动态事件传感器，是一种模拟基于脉冲触发式神经元的人类视网膜的生物拟态视觉传感器。传感器内部具有由多个像素单元构成的像素单元阵列，其中每个像素单元只有在感应到光强变化时，才会响应并记录光强快速变化的区域。关于动态视觉传感器的具体组成此处不做过多阐述。DVS可以采用事件触发的处理机制，输出异步的事件数据流，事件数据流，例如，可以是光强变化信息(如，光强变化的时间戳和光强值)以及被触发像素单元的坐标位置。DVS的响应速度不再受传统的曝光时间和帧率限制，可以侦测到高达万帧/秒速率运动的高速物体；DVS具有更大的动态范围，在低光照或者高曝光环境下都能准确感应并输出场景变化；DVS功耗更低；由于DVS每个像素单元都是独立响应光强变化，因此DVS不会受运动模糊的影响。Dynamic Vision Sensor (DVS), also known as dynamic event sensor, is a biomimetic vision sensor that simulates the human retina based on pulse-triggered neurons. The sensor has a pixel unit array composed of multiple pixel units, and each pixel unit responds to and records the area with rapid changes in light intensity only when it senses a change in light intensity. The specific composition of the dynamic vision sensor will not be elaborated here. DVS can use an event-triggered processing mechanism to output an asynchronous event data stream. The event data stream, for example, can be light intensity change information (such as the time stamp of light intensity change and light intensity value) and the coordinate position of the triggered pixel unit . The response speed of DVS is no longer limited by traditional exposure time and frame rate, and it can detect high-speed objects moving at a rate of up to 10,000 frames per second; DVS has a larger dynamic range and can be accurate in low light or high exposure environments Sensing and outputting scene changes; DVS consumes less power; DVS is not affected by motion blur because each pixel unit of DVS responds to changes in light intensity independently.
摄像模组除了包括动态视觉传感器，还可以包括镜头、镜座、滤镜、电容、电阻等部件，以构成能采集图像数据的模组即可，在此不做限制。In addition to the dynamic vision sensor, the camera module may also include components such as lenses, lens mounts, filters, capacitors, and resistors to form a module capable of collecting image data, which is not limited here.
在一个实施例中，以智能手机为例进行示例说明，本公开实施例的执行主体可以是智能手机，也可以是安装在智能手机中的系统服务。需要说明的是，所述智能手机只是本公开实施例所提供的其中一种应用示例，不应理解为本公开实施例所提供的技术方案只能应用于智能手机的场景下。In one embodiment, a smartphone is used as an example for illustration. The execution body of the embodiment of the present disclosure may be a smartphone or a system service installed in the smartphone. It should be noted that the smart phone is only one of the application examples provided by the embodiments of the present disclosure, and it should not be understood that the technical solutions provided by the embodiments of the present disclosure can only be applied to the scenarios of smart phones.
接下来结合附图对本公开实施例进行示例说明。Next, the embodiments of the present disclosure will be illustrated with reference to the accompanying drawings.
如图1所示，图1是本公开根据一示例性实施例示出的一种交互处理方法的流程图，包括以下步骤：As shown in FIG. 1, FIG. 1 is a flowchart of an interaction processing method according to an exemplary embodiment of the present disclosure, including the following steps:
在步骤102中，获取摄像模组采集的DVS图像数据；In step 102, obtain the DVS image data collected by the camera module;
在步骤104中，从所述DVS图像数据中识别出用于指示终端设备执行操作的目标对象；In
在步骤106中，响应于与所述目标对象对应的操作指令，控制终端设备执行与所述操作指令相匹配的操作。In
该方法可以用于终端设备中，终端设备设置有包括动态视觉传感器DVS的摄像模组。所述摄像模组可以设置在终端设备的外表面部，以采集终端设备所处环境的图像数据。如设置在终端设备的正面或背面。在一个实施例中，摄像模组设置在终端设备的摄像头的周围区域。例如，摄像模组可以设置在前置摄像头的周围区域，或，摄像模组也可以设置在后置摄像头的周围区域。如图2所示，图2是本公开根据一示例性实施例示出的一种摄像模组设置位置示意图。该图以终端设备为智能手机为例，在智能手机前置摄像头的右侧设置有包括动态视觉传感器DVS的摄像模组。The method can be used in a terminal device provided with a camera module including a dynamic vision sensor DVS. The camera module can be arranged on the outer surface of the terminal device to collect image data of the environment where the terminal device is located. Such as setting on the front or back of the terminal equipment. In one embodiment, the camera module is arranged in the surrounding area of the camera of the terminal device. For example, the camera module may be arranged in the surrounding area of the front camera, or the camera module may also be arranged in the surrounding area of the rear camera. As shown in FIG. 2 , FIG. 2 is a schematic diagram showing the arrangement position of a camera module according to an exemplary embodiment of the present disclosure. In the figure, the terminal device is a smartphone as an example, and a camera module including a dynamic vision sensor DVS is arranged on the right side of the front camera of the smartphone.
视觉传感器采集场景中的事件数据，在场景发生变化时可以输出事件。例如，当场景中没有物体相对于终端设备运动时，动态视觉传感器中像素单元检测到的光强不会发生变化，当检测到场景中的某个物体相对于终端设备发生运动，会导致光线发生变化，因而触发像素事件，输出检测到光强发生变化的像素单元的事件数据流，该事件数据流中每个事件数据可以包括检测到亮度发生变化的像素单元的坐标位置和被触发时刻的时间戳信息。DVS图像数据可以由同一时间戳信息对应的事件数据构成。由于在动态视觉传感器中，对于单个像素点，只有接受光强度发生改变时，才会有事件(脉冲)信号输出。比如说亮度增加超过一个阈值，那么将增加一个该像素点亮度增加的事件。因此，DVS图像数据可以是部分图像数据，针对没有检测到光强度发生变化的像素单元没有事件数据。Vision sensors collect event data in the scene, and can output events when the scene changes. For example, when there is no object in the scene moving relative to the terminal device, the light intensity detected by the pixel unit in the dynamic vision sensor will not change. When an object in the scene is detected moving relative to the terminal device, light will occur. change, thus triggering a pixel event, and outputting an event data stream of the pixel unit whose light intensity has changed stamp information. DVS image data may be composed of event data corresponding to the same timestamp information. Because in the dynamic vision sensor, for a single pixel, only when the received light intensity changes, there will be an event (pulse) signal output. For example, if the brightness increases by more than a threshold, then an event that the brightness of the pixel is increased will be added. Thus, the DVS image data may be partial image data, with no event data for pixel cells for which no change in light intensity is detected.
在获得所述摄像模组采集的DVS图像数据后，可以从DVS图像数据中识别出用于指示终端设备执行操作的目标对象。After the DVS image data collected by the camera module is obtained, the target object used to instruct the terminal device to perform an operation can be identified from the DVS image data.
关于目标对象，目标对象是用于指示终端设备执行操作的对象。根据终端设备所需执行的操作，配置不同种类的目标对象。在一个示例中，终端设备执行的操作是身份验证成功后执行的操作，例如，解锁、支付、登录等操作。相应的，目标对象可以是用于进行身份验证的对象。例如，通过人脸识别进行身份验证的场景中，目标对象可以是指定人脸；又如，通过手势识别进行身份验证的场景中，目标对象可以是指定手势等。Regarding the target object, the target object is an object for instructing the terminal device to perform an operation. Configure different kinds of target objects depending on the operations that the end device needs to perform. In one example, the operation performed by the terminal device is an operation performed after the authentication is successful, for example, operations such as unlocking, payment, and login. Correspondingly, the target object may be an object for authentication. For example, in the scene of authentication through face recognition, the target object may be a designated face; in another example, in the scene of authentication through gesture recognition, the target object may be a designated gesture or the like.
在另一个示例中，终端设备执行的操作是某指定操作，例如，点亮屏幕，打开系统收藏夹，进入指定应用程序的指定页面，展示指定应用程序的新消息，锁屏状态下接听来电电话等操作。相应的，目标对象可以是与指定操作映射的对象。示例的，可以是指定部位的姿势。例如，指定手势或指定身体姿势等。指定手势可以是用手比划所确定的姿势。例如，可以是“六”手势、比心手势、“2”手势(也称为耶手势)、ok手势、点赞手势、手掌打开手势等，还可以是比划其他数字的手势。如图3所示，图3是本公开根据一示例性实施例示出的几种手势示意图。可以理解的是，该示意图仅示例出几种手势，实际上还可以是其他手势，例如，比划1、3、4等手势。每种手势还可以有多种变形，只要能表达相应手势含义即可，在此不做限制。指定身体姿势可以是举手姿势、叉腰姿势等。In another example, the operation performed by the terminal device is a specified operation, such as lighting the screen, opening the system favorites, entering the specified page of the specified application, displaying new messages of the specified application, and answering the incoming call in the locked screen state and so on. Correspondingly, the target object may be an object mapped with the specified operation. For example, it may be the gesture of the specified part. For example, specifying gestures or specifying body poses, etc. The designated gesture may be a gesture determined by hand gestures. For example, it can be a "six" gesture, a heart-to-heart gesture, a "2" gesture (also called a jeer gesture), an ok gesture, a thumbs up gesture, a palm opening gesture, etc., and it can also be a gesture of drawing other numbers. As shown in FIG. 3 , FIG. 3 is a schematic diagram of several gestures shown in the present disclosure according to an exemplary embodiment. It can be understood that the schematic diagram only illustrates several gestures, and other gestures, for example, gestures such as gestures 1, 3, and 4, may actually be used. Each gesture can also have multiple deformations, as long as the meaning of the corresponding gesture can be expressed, which is not limited here. The specified body posture may be a raised hand posture, an akimbo posture, and the like.
在一个示例中，可以预先配置目标对象与操作指令的映射关系。操作指令可以是指示终端设备执行操作的指令。在一个例子中，目标对象与操作指令间可以是一一映射关系，以实现每个目标对象触发设备执行一种操作。在另一个例子中，目标对象与操作指令间也可以是多对一的映射关系，以实现多个目标对象触发设备执行一种操作。以手势为例，可以是连续多个手势对应一种操作指令。如，连续比划“3、2、1”三个手势，触发终端设备点亮屏幕，并解锁屏幕。In one example, the mapping relationship between the target object and the operation instruction may be pre-configured. The operation instruction may be an instruction instructing the terminal device to perform an operation. In one example, there may be a one-to-one mapping relationship between the target objects and the operation instructions, so that each target object triggers the device to perform an operation. In another example, there may also be a many-to-one mapping relationship between the target objects and the operation instructions, so that multiple target objects trigger the device to perform an operation. Taking gestures as an example, it may be that a plurality of consecutive gestures correspond to one type of operation instruction. For example, continuous gestures of "3, 2, 1" trigger the terminal device to light up the screen and unlock the screen.
目标对象与操作指令的映射关系，可以是系统预先配置的，也可以是用户设置的。例如，提供映射关系设置服务，以供用户创建目标对象与操作指令的映射关系。The mapping relationship between the target object and the operation instruction can be pre-configured by the system or set by the user. For example, a mapping relationship setting service is provided for users to create a mapping relationship between target objects and operation instructions.
而关于如何从DVS图像数据中识别出目标对象，在一个示例中，可以利用机器学习的方式学习获得能识别目标对象的模型，在模型应用阶段，利用学习获得的模型从DVS图像数据中识别出目标对象。示例的，可以采用有监督学习的方式利用预设的训练样本进行模型训练，获得深度学习网络模型。训练样本可以是带标签的样本图像，标签可以指示出目标对象的位置以及类别。样本图像可以包括DVS图像数据，也可以包括传统的image sensor采集的图像。针对每种目标对象，可以包括不同拍摄角度、和/或不同变形条件下的样本图像，以便提高模型的识别率。示例的，可以利用图像信号处理(Image Signal Processing，ISP)单元进行图像处理。As for how to identify the target object from the DVS image data, in one example, a model that can recognize the target object can be learned by using machine learning. target. For example, a supervised learning method can be used to perform model training using preset training samples to obtain a deep learning network model. The training samples can be labeled sample images, and the labels can indicate the location and category of the target object. The sample image can include DVS image data or images collected by traditional image sensors. For each target object, sample images under different shooting angles and/or different deformation conditions may be included, so as to improve the recognition rate of the model. For example, an image signal processing (Image Signal Processing, ISP) unit may be used to perform image processing.
应当理解的是，上述目标对象的识别方法仅为举例，而不应被理解为对本公开的任何限制，其他现有的或将来的识别目标对象的方法均可以适用本公开，均应包括在本公开的保护范围内。It should be understood that the above-mentioned methods for identifying target objects are only examples, and should not be construed as any limitation to the present disclosure. Other existing or future methods for identifying target objects can be applied to the present disclosure, and should be included in the present disclosure. within the scope of public protection.
关于终端设备执行的操作，可以是身份验证成功后的操作，也可以是预先指定的操作。例如，与操作指令相匹配的操作包括以下一种或多种：The operation performed by the terminal device may be an operation after successful authentication or a pre-specified operation. For example, actions that match an action instruction include one or more of the following:
点亮屏幕；light up the screen;
解锁屏幕；unlock the screen;
启动手电筒；start the flashlight;
启动指定应用；Start the specified application;
展示指定应用的指定页面；Display the specified page of the specified application;
展示指定应用的新消息；Display new messages of the specified application;
接听拨号方的电话。Answer the call from the caller.
其中，点亮屏幕，可以是控制屏幕从息屏状态切换至亮屏状态，息屏状态，可以是屏幕处于黑屏的状态。亮屏状态，可以是屏幕被点亮后所处的状态。Wherein, lighting the screen may be controlling the screen to switch from a screen-off state to a screen-on state, and the screen-off state may be a state where the screen is in a black screen state. The bright screen state can be the state in which the screen is turned on.
为了保证个人信息的安全性，用户往往会对终端设备进行锁屏处理，锁屏后需要通过诸如输入密码等解锁方式才能查看终端设备内容。本实施例可以通过识别出目标对象的方式实现自动解锁。In order to ensure the security of personal information, users often lock the screen of the terminal device. After the screen is locked, it is necessary to use an unlocking method such as entering a password to view the content of the terminal device. This embodiment can realize automatic unlocking by recognizing the target object.
指定应用程序，可以是终端设备中已安装的应用程序，例如，可以是系统应用，也可以是第三方应用。例如，指定应用程序可以是系统收藏夹/照片应用。启动指定应用程序后，可以展示指定应用程序的首页/默认页。示例的，通过比心手势打开系统收藏夹，以查看系统收藏夹中的内容。The specified application can be an installed application in the terminal device, for example, a system application or a third-party application. For example, the designated application may be the system favorites/photos application. After starting the specified application, the home page/default page of the specified application can be displayed. For example, open the system favorites through the heart gesture to view the contents in the system favorites.
指定应用程序的指定页面，可以是用户期望快速查看到的页面，例如，可以是支付程序的支付页面。支付页面可以是包括付款码的页面。该指定应用程序可以是已经启动的应用程序，也可以是未启动的应用程序。针对已经启动的应用程序，可以是前台运行的应用程序，也可以是后台运行的应用程序。示例的，通过耶的手势打开支付程序的支付页面。The designated page of the designated application may be the page that the user expects to view quickly, for example, the payment page of the payment program. The payment page may be a page that includes a payment code. The specified application may be an already launched application or an unlaunched application. For an application that has been started, it can be an application running in the foreground or an application running in the background. For example, open the payment page of the payment program through the gesture of Jesus.
指定应用程序的新消息，可以是所有未读消息，也可以是接收时间最近的未读消息等，可以根据需求设置。该指定应用程序可以是已经启动的应用程序，也可以是未启动的应用程序。针对已经启动的应用程序，可以是前台运行的应用程序，也可以是后台运行的应用程序。示例的，可以通过OK手势打开微信新消息或未读短信。Specify the new message of the application, which can be all unread messages, or the unread messages with the most recent reception time, etc., which can be set according to requirements. The specified application may be an already launched application or an unlaunched application. For an application that has been started, it can be an application running in the foreground or an application running in the background. For example, you can open a new message or unread text message on WeChat through the OK gesture.
关于接听拨号方的电话，目前，在接收到拨号方的拨号请求时，可以通过触摸接听按钮，以实现接听拨号方的电话，而本实施例通过识别目标对象的方式实现自动接通电话。Regarding answering the call of the dialer, at present, when a dialing request from the dialer is received, the answer button can be touched to answer the call of the dialer, and this embodiment realizes the automatic call connection by identifying the target object.
示例的，不管终端设备当前处于何种状态，在识别到目标对象后，可以触发终端设备执行与操作指令相匹配的操作。例如，终端设备当前处于息屏状态，可以从息屏状态开始完成与操作指令相匹配的操作。该实施例从息屏状态直接跳转执行相应操作，可以提高交互效率，给用户带来新的体验。又如，终端设备当前处于亮屏状态，可以从亮屏状态开始完成与操作指令相匹配的操作。For example, no matter what state the terminal device is currently in, after recognizing the target object, the terminal device may be triggered to perform an operation matching the operation instruction. For example, if the terminal device is currently in the screen-off state, the operation matching the operation instruction can be completed from the screen-off state. In this embodiment, the corresponding operation is directly jumped from the screen-off state, which can improve the interaction efficiency and bring a new experience to the user. For another example, the terminal device is currently in a bright screen state, and the operation matching the operation instruction can be completed from the bright screen state.
作为其中一种实施例，与操作指令相匹配的操作可以是从息屏状态开始触发的操作。例如，与操作指令相匹配的操作包括以下一种或多种：As one of the embodiments, the operation matching the operation instruction may be an operation triggered from the screen-off state. For example, actions that match an action instruction include one or more of the following:
在屏幕息屏状态下触发的解锁屏幕；Unlock the screen triggered when the screen is off;
在屏幕息屏状态下触发的启动手电筒；Start flashlight triggered when the screen is off;
在屏幕息屏状态下触发的启动指定应用程序；Triggered to start the specified application when the screen is off;
在屏幕息屏状态下触发的展示指定应用程序的指定页面；Triggered to display the specified page of the specified application when the screen is off;
在屏幕息屏状态下触发的展示指定应用程序的新消息；Triggered to display a new message of the specified application when the screen is off;
在屏幕息屏状态下接听拨号方的电话。Answer the call of the caller with the screen off.
可见，通过该实施例可以实现在黑屏时检测到目标对象而触发屏幕解锁、启动手电筒、启动指定应用程序、展示指定应用程序的指定页面、展示指定应用程序的新消息或接听拨号方的电话等操作，提高操作效率。It can be seen that through this embodiment, when the screen is black, the target object can be detected to trigger the screen to unlock, start the flashlight, start the specified application, display the specified page of the specified application, display the new message of the specified application, or answer the call of the dialer, etc. operation to improve operational efficiency.
另外，上述示例的各操作可以进行组合，示例的，依据终端设备当前所处状态进行组合，例如，终端设备当前处于亮屏状态，则为了展示指定应用程序的指定页面，则可以先点亮屏幕、解锁屏幕，再展示指定应用程序的指定页面。若指定应用程序没有启动，则展示指定页面前，还可以先启动指定应用程序。可见，以上实施方式中的各种操作可以任意进行组合，只要特征之间的组合不存在冲突或矛盾即可。为了达到最终目的，可能还需要执行一些其他操作，在此省略，但应当理解的是，为了达到最终目标，中间不可缺少的操作也囊括在操作指令相匹配的操作中。另外，终端设备执行的操作包括但不限于上述操作，还可以是其他操作，在此不一一列举。In addition, the operations in the above examples can be combined. For example, the combination is based on the current state of the terminal device. For example, if the terminal device is currently in a bright screen state, in order to display the designated page of the designated application, the screen can be first turned on. , unlock the screen, and then display the specified page of the specified application. If the specified application is not started, the specified application can be started before displaying the specified page. It can be seen that various operations in the above embodiments can be combined arbitrarily, as long as there is no conflict or contradiction in the combination of features. In order to achieve the final goal, some other operations may also need to be performed, which are omitted here, but it should be understood that, in order to achieve the final goal, intermediate indispensable operations are also included in the operations that match the operation instructions. In addition, the operations performed by the terminal device include, but are not limited to, the foregoing operations, and may also be other operations, which are not listed one by one here.
由上述实施例可见，本实施例通过在终端设备上设置包含动态视觉传感器的摄像模组，获取摄像模组采集的DVS图像数据，并从DVS图像数据中识别出用于指示终端设备执行操作的目标对象，响应于与目标对象对应的操作指令，控制终端设备执行与该操作指令相匹配的操作，从而实现无需触摸和语音输入的情况下，也能与终端设备进行交互，且存储数据量低、响应速度快。It can be seen from the above embodiment that in this embodiment, a camera module including a dynamic vision sensor is set on the terminal device to obtain the DVS image data collected by the camera module, and from the DVS image data, it is used to instruct the terminal device to perform an operation. The target object, in response to the operation instruction corresponding to the target object, controls the terminal device to perform an operation matching the operation instruction, so as to realize the interaction with the terminal device without touch and voice input, and the amount of stored data is low ,quick response.
以终端设备执行的操作至少包括点亮屏幕为例进行示例说明，如图4所示，图4是本公开根据一示例性实施例示出的另一种交互处理方法的流程图，该方法可以用于终端设备中，所述终端设备设置有包括动态视觉传感器DVS的摄像模组，包括以下步骤：The operation performed by the terminal device at least includes lighting the screen as an example for illustration. As shown in FIG. 4 , FIG. 4 is a flowchart of another interaction processing method shown in the present disclosure according to an exemplary embodiment. In the terminal device, the terminal device is provided with a camera module including a dynamic vision sensor DVS, including the following steps:
在步骤402中，在终端设备的屏幕处于息屏状态时，获取摄像模组采集的DVS图像数据；In
在步骤404中，从所述DVS图像数据中识别出用于指示终端设备执行操作的目标对象，所述操作至少包括点亮屏幕；In
在步骤406中，响应于与所述目标对象对应的操作指令，控制终端设备执行与所述操作指令相匹配的操作。In
其中，图4与图1中相关技术相同，在此不一一赘述。4 is the same as the related technology in FIG. 1 , and details are not repeated here.
本实施例可以在终端设备的屏幕处于息屏状态下，通过目标对象触发终端设备执行相应操作。如，在息屏状态下通过比心手势打开系统收藏夹；在息屏状态下通过耶的手势打开支付程序的支付页面；在息屏状态下通过ok手势打开微信新消息或短信新消息；在息屏状态下通过“六”手势在锁屏来电时接听电话；在息屏状态下通过全手掌打开手势实现屏幕解锁等。该实施例从息屏状态直接跳转执行相应操作，可以提高交互效率，给用户带来新的体验。In this embodiment, when the screen of the terminal device is in an off-screen state, the terminal device can be triggered to perform a corresponding operation through the target object. For example, open the system favorites through the heart gesture in the closed screen state; open the payment page of the payment program through the Ye gesture in the closed screen state; open WeChat new messages or SMS new messages through the ok gesture in the closed screen state; In the off-screen state, the "six" gesture is used to answer the phone when the screen is locked; in the off-screen state, the screen is unlocked by the full palm opening gesture. In this embodiment, the corresponding operation is directly jumped from the screen-off state, which can improve the interaction efficiency and bring a new experience to the user.
在一个实施例中，为了减少摄像模组的功耗，可以为摄像模组配置不同功耗的模式，摄像模组在不同模式下采集到的图像数据的分辨率不同，因此，可以称为分辨率模式。摄像模组在不同模式下所消耗的功耗不同，因此，也可以称为功耗模式。摄像模组的不同分辨率模式可以由动态视觉传感器中处于开启状态(工作状态)的像素单元的个数划分。示例的，为摄像模组配置低分辨率模式(Low Resolution，LR)。摄像模组在低分辨率模式下仅部分像素单元处于工作状态。以视觉传感器能采集100万像素为例，可以将其中二十四分之一的像素单元开启，将剩余的像素单元关闭，以降低功耗。甚至可以控制指定个数的像素单元处于开启状态，其他处于关闭状态。摄像模组还配置有功耗高于低功耗的至少一个其他分辨率模式，摄像模组在低分辨率模式下处于工作状态的像素单元个数，少于摄像模组在其他分辨率模式下处于工作状态的像素单元个数。相应的，摄像模组在低分辨率模式下采集到的图像数据的分辨率，低于摄像模组在其他分辨率模式下采集到的图像数据的分辨率。In one embodiment, in order to reduce the power consumption of the camera module, different power consumption modes can be configured for the camera module, and the resolutions of the image data collected by the camera module in different modes are different. rate mode. The power consumption of the camera module in different modes is different, so it can also be called a power mode. The different resolution modes of the camera module can be divided by the number of pixel units in an on state (working state) in the dynamic vision sensor. For example, configure a low resolution mode (Low Resolution, LR) for the camera module. In the low-resolution mode of the camera module, only some of the pixel units are working. Taking the visual sensor capable of collecting 1 million pixels as an example, twenty-fourth of the pixel units can be turned on, and the remaining pixel units can be turned off to reduce power consumption. It can even control a specified number of pixel units to be on, and others to be off. The camera module is also configured with at least one other resolution mode whose power consumption is higher than the low power consumption. The number of pixel units in the working state of the camera module in the low resolution mode is less than that of the camera module in other resolution modes. The number of pixel units in working state. Correspondingly, the resolution of the image data collected by the camera module in the low resolution mode is lower than the resolution of the image data collected by the camera module in other resolution modes.
在一个实施例中，为了实现对目标对象的检测，摄像模组可以以低分辨率模式处于常开状态。在预设模式切换条件满足时，进行模式切换。例如，在终端设备息屏状态下，摄像模组处于低分辨率模式。In one embodiment, in order to detect the target object, the camera module can be in a normally-on state in a low-resolution mode. When the preset mode switching condition is satisfied, the mode switching is performed. For example, when the screen of the terminal device is off, the camera module is in a low-resolution mode.
可见，在该实施例中，通过控制摄像模组以低分辨率模式处于常开状态，既可以保证实时检测，又可以降低功耗。It can be seen that, in this embodiment, by controlling the camera module to be in a normally-on state in a low-resolution mode, both real-time detection can be ensured, and power consumption can be reduced.
在另一个实施例中，在终端设备的屏幕处于息屏状态时，所述摄像模组处于低分辨率模式。而在终端设备的屏幕处于亮屏状态时，可以关闭摄像模组。In another embodiment, when the screen of the terminal device is in an off-screen state, the camera module is in a low-resolution mode. When the screen of the terminal device is in a bright screen state, the camera module can be turned off.
关于预设模式切换条件，可以包括预先设置的从低分辨率模式切换至其他分辨率模式的条件，或者其他分辨率模式间的切换条件，或者从其他分辨率模式切换至低分辨率模式的条件等。Regarding the preset mode switching conditions, it may include preset conditions for switching from low resolution mode to other resolution modes, or switching conditions between other resolution modes, or switching from other resolution modes to low resolution modes. Wait.
在一个实施例中，预设模式切换条件包括依据所述摄像模组在当前模式下采集到的DVS图像数据判定当前环境光的变化满足预设变化条件。In one embodiment, the preset mode switching condition includes determining that the change of the current ambient light satisfies the preset change condition according to the DVS image data collected by the camera module in the current mode.
关于预设变化条件，可以是预先设置的依据环境光的变化进行模式切换的条件。示例的，预设变化条件可以是当前环境光的光强变化值大于设定阈值。DVS图像数据中事件数据流可以包括光照强度，因此，可以根据至少两帧图像的光照强度判断当前环境光的光强变化值是否大于设定阈值。在另一个示例中，不仅当前环境光的光强变化值大于设定阈值，还可以结合DVS图像数据中检测到光照发生变化的像素单元的个数，判断当前环境光的变化是否满足预设变化条件。Regarding the preset change condition, it may be a preset condition for mode switching according to the change of ambient light. For example, the preset change condition may be that the light intensity change value of the current ambient light is greater than a set threshold. The event data stream in the DVS image data may include the light intensity. Therefore, it may be determined whether the light intensity change value of the current ambient light is greater than the set threshold according to the light intensity of at least two frames of images. In another example, not only the light intensity change value of the current ambient light is greater than the set threshold value, but also can be combined with the number of pixel units in the DVS image data whose illumination changes are detected to determine whether the current ambient light change satisfies the preset change condition.
该条件可以是低分辨率模式切换至其他分辨率模式的条件，在依据摄像模组在低分辨率模式下采集的DVS图像数据，可以判断当前环境光变化是否满足预设变化条件，在预设变化条件满足时，可以触发摄像模组切换至下一级模式，否则可以维持低分辨率模式。示例的，在切换条件满足时，可以向摄像模组发切换通知，以使摄像模组进行模式切换。This condition may be a condition for switching the low-resolution mode to other resolution modes. Based on the DVS image data collected by the camera module in the low-resolution mode, it can be determined whether the current ambient light change meets the preset change condition. When the change conditions are met, the camera module can be triggered to switch to the next-level mode, otherwise the low-resolution mode can be maintained. For example, when the switching condition is satisfied, a switching notification can be sent to the camera module, so that the camera module can perform mode switching.
示例的，所述其他分辨率模式可以包括高分辨率模式(High Resolution，HR)。摄像模组在高分辨率模式下所有像素单元可以处于工作状态，以实现摄像模组在高分辨率模式采集到分辨率较高的图像数据。步骤102中获取所述摄像模组采集的DVS图像数据，可以包括：获取所述摄像模组在高分辨率模式下采集的DVS图像数据。相应的，所述方法还包括：For example, the other resolution modes may include a high resolution mode (High Resolution, HR). In the high-resolution mode of the camera module, all pixel units can be in a working state, so that the camera module can collect image data with higher resolution in the high-resolution mode. Obtaining the DVS image data collected by the camera module in step 102 may include: obtaining the DVS image data collected by the camera module in a high-resolution mode. Correspondingly, the method further includes:
获取所述摄像模组在低分辨率模式下采集的低分辨率DVS图像数据；acquiring low-resolution DVS image data collected by the camera module in a low-resolution mode;
在依据低分辨率DVS图像数据判定当前环境光的变化满足预设变化条件时，控制所述摄像模组从低分辨率模式切换至高分辨率模式。When it is determined according to the low-resolution DVS image data that the change of the current ambient light satisfies the preset change condition, the camera module is controlled to switch from the low-resolution mode to the high-resolution mode.
在该实施例中，常开时，DVS处于低分辨率、低分辨率模式，只检测环境光的变化。当检测到环境光变化大于设定阈值时，触发DVS高分辨率模式，识别是否是目标对象，从而保证能实时检测，同时还能降低功耗。In this embodiment, when normally on, the DVS is in a low-resolution, low-resolution mode, and only detects changes in ambient light. When it is detected that the ambient light change is greater than the set threshold, the DVS high-resolution mode is triggered to identify whether it is a target object, so as to ensure real-time detection and reduce power consumption.
在另一个实施例中，是否设置多个其他分辨率模式可以依据识别目标对象所需图像分辨率要求决定，例如，一个实施例中，不同类型的目标对象映射不同操作指令，某些目标对象的识别只需要中分辨率的图像即可，有些目标对象的识别需要高分辨率图像，则可以配置多种等级的分辨率模式。In another embodiment, whether to set multiple other resolution modes may be determined according to the image resolution requirements required to identify the target object. For example, in one embodiment, different types of target objects map different operation instructions, and certain target objects have different Only medium resolution images are required for recognition. For some target object recognition, high resolution images are required, and various levels of resolution modes can be configured.
是否设置多个其他分辨率模式还可以依据识别目标对象前是否有必要检测是否存在待识别对象来决定。示例的，某些场景中，可以先判断图像数据中是否存在待识别对象，再判断待识别对象是否为目标对象，从而提高识别准确率。存在待识别对象是进行目标对象识别的基础/前提。以目标对象为指定人脸为例，待识别对象为人脸，可以先判断图像数据中是否存在人脸，可以在确保存在人脸的情况下，再在判断人脸是否为指定人脸。以目标对象为身体姿势为例，待识别对象为人，可以先判断图像数据中是否存在人，在确保存在人的情况下，再判断该人的身体姿态。相应的，预设模式切换条件可以包括：依据所述摄像模组在当前模式下采集到的DVS图像数据，判定所述摄像模组的采集区域内存在待识别对象。Whether to set multiple other resolution modes can also be determined according to whether it is necessary to detect whether there is an object to be recognized before recognizing the target object. For example, in some scenarios, it is possible to first determine whether there is an object to be recognized in the image data, and then determine whether the object to be recognized is a target object, thereby improving the recognition accuracy. The existence of the object to be recognized is the basis/premise of target object recognition. Taking the target object as a designated face as an example, if the object to be recognized is a face, you can first determine whether there is a face in the image data, and then determine whether the face is a designated face after ensuring that there is a face. Taking the target object as a body posture as an example, if the object to be recognized is a person, it is possible to first determine whether there is a person in the image data, and then determine the body posture of the person when it is ensured that there is a person. Correspondingly, the preset mode switching condition may include: according to the DVS image data collected by the camera module in the current mode, determining that there is an object to be recognized in the collection area of the camera module.
其中，图像数据中存在待识别对象是进行目标对象识别的基础/前提。判断摄像模组的采集区域内是否存在待识别对象，可以是低分辨率模式切换至其他分辨率模式的切换条件，也可以是其他分辨率模式间的切换条件。Among them, the existence of the object to be recognized in the image data is the basis/premise of the target object recognition. Determining whether there is an object to be identified in the acquisition area of the camera module may be a switching condition for switching from a low resolution mode to other resolution modes, or a switching condition between other resolution modes.
关于如何判断摄像模组的采集区域内是否存在待识别对象，在一个实施例中，可以依据图像数据中是否存在待识别对象的轮廓，来判断摄像模组的采集区域内是否存在待识别对象。可以理解的是，也可以采用其他手段判断摄像模组的采集区域内是否存在待识别对象，如采用相关技术中是否存在人脸/是否存在人的算法来判断摄像模组的采集区域内是否存在待识别对象。Regarding how to determine whether there is an object to be recognized in the collection area of the camera module, in one embodiment, whether there is an object to be recognized in the image data can be judged whether there is an object to be recognized in the collection area of the camera module. It can be understood that other means can also be used to judge whether there is an object to be identified in the collection area of the camera module, such as using the algorithm of whether there is a face or whether there is a person in the related art to determine whether there is an object in the collection area of the camera module. object to be identified.
示例的，所述其他分辨率模式包括中分辨率模式(Middle Resolution，MR)和高分辨率模式，摄像模组在低分辨率模式、中分辨率模式、高分辨率模式下处于工作状态的像素单元个数依次增多。步骤102中获取所述摄像模组采集的DVS图像数据，可以包括：获取所述摄像模组在高分辨率模式下采集的DVS图像数据。相应的，所述方法还包括：Exemplarily, the other resolution modes include a middle resolution mode (Middle Resolution, MR) and a high resolution mode, and the pixels of the camera module that are in a working state in the low resolution mode, the medium resolution mode, and the high resolution mode The number of units increases successively. Obtaining the DVS image data collected by the camera module in step 102 may include: obtaining the DVS image data collected by the camera module in a high-resolution mode. Correspondingly, the method further includes:
获取所述摄像模组在低分辨率模式下采集的低分辨率DVS图像数据；acquiring low-resolution DVS image data collected by the camera module in a low-resolution mode;
在依据低分辨率DVS图像数据判定当前环境光的变化满足预设变化条件时，控制所述摄像模组从低分辨率模式切换至中分辨率模式；When it is determined according to the low-resolution DVS image data that the change of the current ambient light satisfies the preset change condition, controlling the camera module to switch from the low-resolution mode to the medium-resolution mode;
获取所述摄像模组在中分辨率模式下采集的中分辨率DVS图像数据；Obtain the medium-resolution DVS image data collected by the camera module in the medium-resolution mode;
在依据所述摄像模组在中分辨率模式下采集到的DVS图像数据，判定所述摄像模组的采集区域内存在待识别对象时，控制所述摄像模组从中分辨率模式切换至高分辨率模式。When it is determined that there is an object to be recognized in the collection area of the camera module according to the DVS image data collected by the camera module in the medium resolution mode, the camera module is controlled to switch from the medium resolution mode to the high resolution model.
该实施例配置三个等级的分辨率模式，逐次切换，可以降低功耗。In this embodiment, three levels of resolution modes are configured and switched one by one, which can reduce power consumption.
关于其他分辨率模式切换至低分辨率模式的条件，可以是依据低分辨率DVS图像数据判定当前环境光的变化值小于或等于设定阈值；或者，依据所述摄像模组在中分辨率模式下采集到的DVS图像数据，判定所述摄像模组的采集区域内不存在待识别对象；或者，控制终端设备执行与操作指令相匹配的操作后，且间隔预设延迟时间等。通过设置其他分辨率模式切换至低分辨率模式的条件，以保证摄像模组大部分时间处于低分辨率模式，进一步降低功耗。Regarding the conditions for switching from other resolution modes to the low-resolution mode, it may be determined according to the low-resolution DVS image data that the change value of the current ambient light is less than or equal to the set threshold; or, according to the camera module in the medium-resolution mode The DVS image data collected is downloaded, and it is determined that there is no object to be identified in the collection area of the camera module; or, after controlling the terminal device to perform an operation that matches the operation instruction, there is a preset delay time interval, etc. By setting the conditions for switching from other resolution modes to the low-resolution mode, it is ensured that the camera module is in the low-resolution mode most of the time, and the power consumption is further reduced.
应当理解的是，上述预设模式切换条件仅为举例，而不应被理解为对本公开的任何限制，其他现有的或将来的触发模式进行切换的条件可以适用本公开，均应包括在本公开的保护范围内。It should be understood that the above preset mode switching conditions are only examples, and should not be construed as any limitation to the present disclosure. Other existing or future triggering mode switching conditions may be applicable to the present disclosure, and should be included in this disclosure. within the scope of public protection.
以上实施方式中的各种技术特征可以任意进行组合，只要特征之间的组合不存在冲突或矛盾，但是限于篇幅，未进行一一描述，因此上述实施方式中的各种技术特征的任意进行组合也属于本说明书公开的范围。Various technical features in the above embodiments can be combined arbitrarily, as long as there is no conflict or contradiction between the combinations of features, but due to space limitations, they are not described one by one, so the various technical features in the above embodiments can be combined arbitrarily It also belongs to the scope of the disclosure of this specification.
以下以其中一种组合进行示例说明。The following is an example of one of the combinations.
如图5所示，图5是本公开根据一示例性实施例示出的另一种交互处理方法的流程图，所述方法可以用于终端设备，所述终端设备设置有包括动态视觉传感器DVS的摄像模组，所述摄像模组配置有低分辨率模式和高分辨率模式，所述方法包括：As shown in FIG. 5 , FIG. 5 is a flowchart of another interaction processing method shown in the present disclosure according to an exemplary embodiment. The method can be used in a terminal device, and the terminal device is provided with a dynamic visual sensor DVS. A camera module, the camera module is configured with a low-resolution mode and a high-resolution mode, and the method includes:
在步骤502中，获取所述摄像模组在低分辨率模式下采集的低分辨率DVS图像数据；In
在步骤504中，依据低分辨率DVS图像数据，判断当前环境光的变化值是否大于设定阈值，若否，返回步骤502，若是，进入步骤506。In
在步骤506中，控制所述摄像模组从低分辨率模式切换至高分辨率模式。In
在步骤508中，获取所述摄像模组在高分辨率模式下采集的DVS图像数据；In
在步骤510中，从所述DVS图像数据中识别出用于指示终端设备执行操作的目标对象；In
在步骤512中，响应于与所述目标对象对应的操作指令，控制终端设备执行与所述操作指令相匹配的操作。In
其中，图5与图1或图4中相关技术相同，在此不一一赘述。该实施例为摄像模组配置低分辨率模式LR和高分辨率模式HR，根据不同场景进行模式切换，对功率优化。5 is the same as the related technology in FIG. 1 or FIG. 4 , and details are not repeated here. In this embodiment, a low-resolution mode LR and a high-resolution mode HR are configured for the camera module, and mode switching is performed according to different scenarios to optimize power.
在一个示例中，可以将终端设备的操作限定为息屏状态下触发的操作，步骤502可以限定在终端设备的屏幕处于息屏状态下执行，而在终端设备的屏幕处于亮屏状态时，可以将摄像模组切换为低分辨率模式，或关闭摄像模组。该实施例利用DVS低功耗，可以实现息屏状态下目标对象识别，用户可以黑屏解锁，并且结合目标对象控制终端设备直接执行亮屏下的某指定操作。In one example, the operation of the terminal device can be limited to an operation triggered in the off-screen state, and step 502 can be limited to be performed when the screen of the terminal device is in the off-screen state, and when the screen of the terminal device is in the bright-screen state, it can be Switch the camera module to low resolution mode, or turn off the camera module. This embodiment utilizes the low power consumption of DVS, and can realize target object recognition in an off-screen state, the user can unlock the screen with a black screen, and control the terminal device in combination with the target object to directly perform a specified operation under the bright screen.
与前述交互处理方法的实施例相对应，本公开还提供了交互处理装置、装置所应用的设备以及存储介质的实施例。Corresponding to the foregoing embodiments of the interaction processing method, the present disclosure also provides embodiments of an interaction processing apparatus, a device to which the apparatus is applied, and a storage medium.
如图6所示，图6是本公开根据一示例性实施例示出的一种交互处理装置的框图，所述装置设于终端设备，所述终端设备设置有包括动态视觉传感器DVS的摄像模组，所述装置包括：As shown in FIG. 6 , FIG. 6 is a block diagram of an interaction processing apparatus according to an exemplary embodiment of the present disclosure. The apparatus is provided in a terminal device, and the terminal device is provided with a camera module including a dynamic vision sensor DVS , the device includes:
数据获取模块62，用于获取所述摄像模组采集的DVS图像数据；A
对象识别模块64，用于从所述DVS图像数据中识别出用于指示终端设备执行操作的目标对象；The
操作控制模块66，用于响应于与所述目标对象对应的操作指令，控制终端设备执行与所述操作指令相匹配的操作。The
在一个实施例中，终端设备执行的操作至少包括点亮屏幕，所述获取所述摄像模组采集的DVS图像数据，包括：在终端设备的屏幕处于息屏状态时，获取所述摄像模组采集的DVS图像数据。In one embodiment, the operation performed by the terminal device at least includes lighting the screen, and the acquiring the DVS image data collected by the camera module includes: when the screen of the terminal device is in a screen-off state, acquiring the camera module Acquired DVS image data.
在一个实施例中，所述摄像模组配置有低分辨率模式、以及至少一个其他分辨率模式，所述摄像模组在低分辨率模式下处于工作状态的像素单元个数，少于所述摄像模组在其他分辨率模式下处于工作状态的像素单元个数，不同模式在预设模式切换条件满足时切换。In one embodiment, the camera module is configured with a low-resolution mode and at least one other resolution mode, and the number of pixel units in a working state of the camera module in the low-resolution mode is less than the number of pixel units in the low-resolution mode. The number of pixel units in the working state of the camera module in other resolution modes. Different modes are switched when the preset mode switching conditions are met.
在一个实施例中，所述预设模式切换条件包括以下任一种条件：In one embodiment, the preset mode switching condition includes any of the following conditions:
依据所述摄像模组在当前模式下采集到的DVS图像数据判定当前环境光的变化满足预设变化条件；According to the DVS image data collected by the camera module in the current mode, it is determined that the change of the current ambient light satisfies the preset change condition;
依据所述摄像模组在当前模式下采集到的DVS图像数据，判定所述摄像模组的采集区域内存在待识别对象。According to the DVS image data collected by the camera module in the current mode, it is determined that there is an object to be identified in the collection area of the camera module.
在一个实施例中，所述摄像模组以低分辨率模式处于常开状态，或者在终端设备的屏幕处于息屏状态时，所述摄像模组处于低分辨率模式。In one embodiment, the camera module is in a normally-on state in a low-resolution mode, or when the screen of the terminal device is in an off-screen state, the camera module is in a low-resolution mode.
在一个实施例中，如图7所示，图7是本公开根据一示例性实施例示出的另一种交互处理装置的框图，该实施例在前述图6所示实施例的基础上，所述其他分辨率模式包括高分辨率模式，所述数据获取模块62用于：获取所述摄像模组在高分辨率模式下采集的DVS图像数据；In one embodiment, as shown in FIG. 7 , FIG. 7 is a block diagram of another interaction processing apparatus shown in the present disclosure according to an exemplary embodiment. This embodiment is based on the foregoing embodiment shown in FIG. 6 . The other resolution modes include a high-resolution mode, and the
所述数据获取模块62还用于：获取所述摄像模组在低分辨率模式下采集的低分辨率DVS图像数据；The
所述装置还包括模式切换模块70，用于在依据低分辨率DVS图像数据判定当前环境光的变化满足预设变化条件时，控制所述摄像模组从低分辨率模式切换至高分辨率模式。The device further includes a
在一个实施例中，如图8所示，图8是本公开根据一示例性实施例示出的另一种交互处理装置的框图，该实施例在前述图6所示实施例的基础上，所述其他分辨率模式包括中分辨率模式和高分辨率模式，所述数据获取模块62用于：获取所述摄像模组在高分辨率模式下采集的DVS图像数据；In one embodiment, as shown in FIG. 8 , FIG. 8 is a block diagram of another interaction processing apparatus shown in the present disclosure according to an exemplary embodiment. This embodiment is based on the foregoing embodiment shown in FIG. 6 . The other resolution modes include a medium resolution mode and a high resolution mode, and the
所述数据获取模块62还用于：获取所述摄像模组在低分辨率模式下采集的低分辨率DVS图像数据；The
所述装置还包括模式切换模块80，用于在依据低分辨率DVS图像数据判定当前环境光的变化满足预设变化条件时，控制所述摄像模组从低分辨率模式切换至中分辨率模式；The device further includes a
所述数据获取模块62还用于：获取所述摄像模组在中分辨率模式下采集的中分辨率DVS图像数据；The
所述模式切换模块80还用于在依据所述摄像模组在中分辨率模式下采集到的DVS图像数据，判定所述摄像模组的采集区域内存在待识别对象时，控制所述摄像模组从中分辨率模式切换至高分辨率模式。The
在一个实施例中，所述目标对象包括指定手势、指定人脸、和/或指定身体姿势。In one embodiment, the target object includes a designated gesture, a designated face, and/or a designated body posture.
在一个实施例中，预配置有目标对象与操作指令的映射关系，与操作指令相匹配的操作包括以下一种或多种：In one embodiment, the mapping relationship between the target object and the operation instruction is preconfigured, and the operation matching the operation instruction includes one or more of the following:
在屏幕息屏状态下触发的解锁屏幕；Unlock the screen triggered when the screen is off;
在屏幕息屏状态下触发的启动手电筒；Start flashlight triggered when the screen is off;
在屏幕息屏状态下触发的启动指定应用程序；Triggered to start the specified application when the screen is off;
在屏幕息屏状态下触发的展示指定应用程序的指定页面；Triggered to display the specified page of the specified application when the screen is off;
在屏幕息屏状态下触发的展示指定应用程序的新消息；Triggered to display a new message of the specified application when the screen is off;
在屏幕息屏状态下接听拨号方的电话。Answer the call of the caller with the screen off.
相应的，本公开还提供一种电子设备，所述设备包括基于动态视觉传感器DVS的摄像模组、存储器、处理器及存储在存储器上并可在处理器上运行的计算机程序，其中，所述处理器执行所述程序时实现如上任一项所述方法。Correspondingly, the present disclosure also provides an electronic device, the device includes a camera module based on a dynamic vision sensor DVS, a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the When the processor executes the program, the method as described in any of the above is implemented.
相应的，本公开还提供一种计算机可读存储介质，其上存储有计算机程序，该程序被处理器执行时实现上述任一项所述方法的步骤。Correspondingly, the present disclosure also provides a computer-readable storage medium on which a computer program is stored, and when the program is executed by a processor, implements the steps of any one of the methods described above.
本公开可采用在一个或多个其中包含有程序代码的存储介质(包括但不限于磁盘存储器、CD-ROM、光学存储器等)上实施的计算机程序产品的形式。计算机可用存储介质包括永久性和非永久性、可移动和非可移动媒体，可以由任何方法或技术来实现信息存储。信息可以是计算机可读指令、数据结构、程序的模块或其他数据。计算机的存储介质的例子包括但不限于：相变内存(PRAM)、静态随机存取存储器(SRAM)、动态随机存取存储器(DRAM)、其他类型的随机存取存储器(RAM)、只读存储器(ROM)、电可擦除可编程只读存储器(EEPROM)、快闪记忆体或其他内存技术、只读光盘只读存储器(CD-ROM)、数字多功能光盘(DVD)或其他光学存储、磁盒式磁带，磁带磁磁盘存储或其他磁性存储设备或任何其他非传输介质，可用于存储可以被计算设备访问的信息。The present disclosure may take the form of a computer program product embodied on one or more storage media having program code embodied therein, including but not limited to disk storage, CD-ROM, optical storage, and the like. Computer-usable storage media includes permanent and non-permanent, removable and non-removable media, and storage of information can be accomplished by any method or technology. Information may be computer readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read only memory (ROM), Electrically Erasable Programmable Read Only Memory (EEPROM), Flash Memory or other memory technology, Compact Disc Read Only Memory (CD-ROM), Digital Versatile Disc (DVD) or other optical storage, Magnetic tape cassettes, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission medium that can be used to store information that can be accessed by a computing device.
上述装置中各个模块的功能和作用的实现过程具体详情见上述方法中对应步骤的实现过程，在此不再赘述。For the specific details of the implementation process of the functions and functions of each module in the above device, see the implementation process of the corresponding steps in the above method, which will not be repeated here.
对于装置实施例而言，由于其基本对应于方法实施例，所以相关之处参见方法实施例的部分说明即可。以上所描述的装置实施例仅仅是示意性的，其中所述作为分离部件说明的模块可以是或者也可以不是物理上分开的，作为模块显示的部件可以是或者也可以不是物理模块，即可以位于一个地方，或者也可以分布到多个网络模块上。可以根据实际的需要选择其中的部分或者全部模块来实现本公开方案的目的。本领域普通技术人员在不付出创造性劳动的情况下，即可以理解并实施。For the apparatus embodiments, since they basically correspond to the method embodiments, reference may be made to the partial descriptions of the method embodiments for related parts. The device embodiments described above are only illustrative, wherein the modules described as separate components may or may not be physically separated, and the components shown as modules may or may not be physical modules, that is, they may be located in One place, or it can be distributed over multiple network modules. Some or all of the modules can be selected according to actual needs to achieve the purpose of the solution of the present disclosure. Those of ordinary skill in the art can understand and implement it without creative effort.
如图9所示，图9是本公开根据一示例性实施例示出的一种交互处理装置所在计算机设备的硬件结构图。该装置900可以是移动电话，计算机，数字广播终端，消息收发设备，游戏控制台，平板设备，医疗设备，健身设备，个人数字助理等终端。As shown in FIG. 9 , FIG. 9 is a hardware structure diagram of a computer device where an interaction processing apparatus is located according to an exemplary embodiment of the present disclosure. The
参照图9，装置900可以包括以下一个或多个组件：处理组件902，存储器904，电源组件906，多媒体组件908，音频组件910，输入/输出(I/O)的接口912，传感器组件914，以及通信组件916。装置900还设置有包括动态视觉传感器的摄像模组，图9中未示出。9, the
处理组件902通常控制装置900的整体操作，诸如与显示，电话呼叫，数据通信，相机操作和记录操作相关联的操作。处理组件902可以包括一个或多个处理器920来执行指令，以完成上述的方法的全部或部分步骤。此外，处理组件902可以包括一个或多个模块，便于处理组件902和其他组件之间的交互。例如，处理组件902可以包括多媒体模块，以方便多媒体组件908和处理组件902之间的交互。The
存储器904被配置为存储各种类型的数据以支持在装置900的操作。这些数据的示例包括用于在装置900上操作的任何应用程序或方法的指令，联系人数据，电话簿数据，消息，图片，视频等。存储器904可以由任何类型的易失性或非易失性存储设备或者它们的组合实现，如静态随机存取存储器(SRAM)，电可擦除可编程只读存储器(EEPROM)，可擦除可编程只读存储器(EPROM)，可编程只读存储器(PROM)，只读存储器(ROM)，磁存储器，快闪存储器，磁盘或光盘。
电源组件906为装置900的各种组件提供电力。电源组件906可以包括电源管理系统，一个或多个电源，及其他与为装置900生成、管理和分配电力相关联的组件。
多媒体组件908包括在所述装置900和用户之间的提供一个输出接口的屏幕。在一些实施例中，屏幕可以包括液晶显示器(LCD)和触摸面板(TP)。如果屏幕包括触摸面板，屏幕可以被实现为触摸屏，以接收来自用户的输入信号。触摸面板包括一个或多个触摸传感器以感测触摸、滑动和触摸面板上的手势。所述触摸传感器可以不仅感测触摸或滑动动作的边界，而且还检测与所述触摸或滑动操作相关的持续时间和压力。在一些实施例中，多媒体组件908包括一个前置摄像头和/或后置摄像头。当装置900处于操作模式，如拍摄模式或视频模式时，前置摄像头和/或后置摄像头可以接收外部的多媒体数据。每个前置摄像头和后置摄像头可以是一个固定的光学透镜系统或具有焦距和光学变焦能力。
音频组件910被配置为输出和/或输入音频信号。例如，音频组件910包括一个麦克风(MIC)，当装置900处于操作模式，如呼叫模式、记录模式和语音识别模式时，麦克风被配置为接收外部音频信号。所接收的音频信号可以被进一步存储在存储器904或经由通信组件916发送。在一些实施例中，音频组件910还包括一个扬声器，用于输出音频信号。
I/O接口912为处理组件902和外围接口模块之间提供接口，上述外围接口模块可以是键盘，点击轮，按钮等。这些按钮可包括但不限于：主页按钮、音量按钮、启动按钮和锁定按钮。The I/
传感器组件914包括一个或多个传感器，用于为装置900提供各个方面的状态评估。例如，传感器组件914可以检测到装置900的打开/关闭状态，组件的相对定位，例如所述组件为装置900的显示器和小键盘，传感器组件914还可以检测装置900或装置900中一个组件的位置改变，用户与装置900接触的存在或不存在，装置900方位或加速/减速和装置900的温度变化。传感器组件914可以包括接近传感器，被配置用来在没有任何的物理接触时检测附近物体的存在。传感器组件914还可以包括光传感器，如CMOS或CCD图像传感器，用于在成像应用中使用。在一些实施例中，该传感器组件914还可以包括加速度传感器，陀螺仪传感器，磁传感器，压力传感器或温度传感器。
通信组件916被配置为便于装置900和其他设备之间有线或无线方式的通信。装置900可以接入基于通信标准的无线网络，如WiFi，2G或3G，或它们的组合。在一个示例性实施例中，通信组件916经由广播信道接收来自外部广播管理系统的广播信号或广播相关信息。在一个示例性实施例中，所述通信组件916还包括近场通信(NFC)模块，以促进短程通信。例如，在NFC模块可基于射频识别(RFID)技术，红外数据协会(IrDA)技术，超宽带(UWB)技术，蓝牙(BT)技术和其他技术来实现。
在示例性实施例中，装置900可以被一个或多个应用专用集成电路(ASIC)、数字信号处理器(DSP)、数字信号处理设备(DSPD)、可编程逻辑器件(PLD)、现场可编程门阵列(FPGA)、控制器、微控制器、微处理器或其他电子元件实现，用于执行上述方法。In an exemplary embodiment,
在示例性实施例中，还提供了一种包括指令的非临时性计算机可读存储介质，例如包括指令的存储器904，上述指令可由装置900的处理器920执行以完成上述方法。例如，所述非临时性计算机可读存储介质可以是ROM、随机存取存储器(RAM)、CD-ROM、磁带、软盘和光数据存储设备等。In an exemplary embodiment, there is also provided a non-transitory computer-readable storage medium including instructions, such as a
其中，当所述存储介质中的指令由所述处理器执行时，使得装置900能够执行上述任一种交互处理方法。Wherein, when the instructions in the storage medium are executed by the processor, the
本领域技术人员在考虑说明书及实践这里公开的发明后，将容易想到本公开的其它实施方案。本公开旨在涵盖本公开的任何变型、用途或者适应性变化，这些变型、用途或者适应性变化遵循本公开的一般性原理并包括本公开未公开的本技术领域中的公知常识或惯用技术手段。说明书和实施例仅被视为示例性的，本公开的真正范围和精神由下面的权利要求指出。Other embodiments of the present disclosure will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This disclosure is intended to cover any variations, uses, or adaptations of this disclosure that follow the general principles of this disclosure and include common general knowledge or techniques in the technical field not disclosed by this disclosure . The specification and examples are to be regarded as exemplary only, with the true scope and spirit of the disclosure being indicated by the following claims.
应当理解的是，本公开并不局限于上面已经描述并在附图中示出的精确结构，并且可以在不脱离其范围进行各种修改和改变。本公开的范围仅由所附的权利要求来限制。It is to be understood that the present disclosure is not limited to the precise structures described above and illustrated in the accompanying drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.
以上所述仅为本公开的较佳实施例而已，并不用以限制本公开，凡在本公开的精神和原则之内，所做的任何修改、等同替换、改进等，均应包含在本公开保护的范围之内。The above descriptions are only preferred embodiments of the present disclosure, and are not intended to limit the present disclosure. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present disclosure shall be included in the present disclosure. within the scope of protection.
Claims (15)
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910425579.9ACN111984347B (en) | 2019-05-21 | 2019-05-21 | Interactive processing method, device, equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910425579.9ACN111984347B (en) | 2019-05-21 | 2019-05-21 | Interactive processing method, device, equipment and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111984347Atrue CN111984347A (en) | 2020-11-24 |
| CN111984347B CN111984347B (en) | 2024-09-17 |
Family
ID=73436995
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910425579.9AActiveCN111984347B (en) | 2019-05-21 | 2019-05-21 | Interactive processing method, device, equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111984347B (en) |
Cited By (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112929491A (en)* | 2021-02-07 | 2021-06-08 | 展讯通信（上海）有限公司 | Application program starting method and related device |
| CN112949440A (en)* | 2021-02-22 | 2021-06-11 | 豪威芯仑传感器(上海)有限公司 | Method for extracting gait features of pedestrian, gait recognition method and system |
| WO2022165736A1 (en)* | 2021-02-02 | 2022-08-11 | 豪威芯仑传感器(上海)有限公司 | Method and system for identifying hand sliding direction |
| CN115242952A (en)* | 2022-07-28 | 2022-10-25 | 联想(北京)有限公司 | Image acquisition method and device |
| CN115278013A (en)* | 2022-07-22 | 2022-11-01 | 杭州研极微电子有限公司 | Camera and method for controlling camera |
| CN115550539A (en)* | 2021-06-30 | 2022-12-30 | 北京小米移动软件有限公司 | Function control method, function control device and storage medium |
| CN116708655A (en)* | 2022-10-20 | 2023-09-05 | 荣耀终端有限公司 | Screen control method based on event camera and electronic equipment |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20140320662A1 (en)* | 2013-03-15 | 2014-10-30 | Moontunes, Inc. | Systems and Methods for Controlling Cameras at Live Events |
| CN106446873A (en)* | 2016-11-03 | 2017-02-22 | 北京旷视科技有限公司 | Face detection method and device |
| CN107479771A (en)* | 2017-07-21 | 2017-12-15 | 努比亚技术有限公司 | A kind of control method, terminal and storage medium |
| CN107734129A (en)* | 2017-09-27 | 2018-02-23 | 广东欧珀移动通信有限公司 | Unlock control method and related products |
| CN107993194A (en)* | 2017-11-30 | 2018-05-04 | 天津大学 | A kind of super resolution ratio reconstruction method based on Stationary Wavelet Transform |
- 2019
- 2019-05-21CNCN201910425579.9Apatent/CN111984347B/enactiveActive
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20140320662A1 (en)* | 2013-03-15 | 2014-10-30 | Moontunes, Inc. | Systems and Methods for Controlling Cameras at Live Events |
| CN106446873A (en)* | 2016-11-03 | 2017-02-22 | 北京旷视科技有限公司 | Face detection method and device |
| CN107479771A (en)* | 2017-07-21 | 2017-12-15 | 努比亚技术有限公司 | A kind of control method, terminal and storage medium |
| CN107734129A (en)* | 2017-09-27 | 2018-02-23 | 广东欧珀移动通信有限公司 | Unlock control method and related products |
| CN107993194A (en)* | 2017-11-30 | 2018-05-04 | 天津大学 | A kind of super resolution ratio reconstruction method based on Stationary Wavelet Transform |
Cited By (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2022165736A1 (en)* | 2021-02-02 | 2022-08-11 | 豪威芯仑传感器(上海)有限公司 | Method and system for identifying hand sliding direction |
| CN112929491A (en)* | 2021-02-07 | 2021-06-08 | 展讯通信（上海）有限公司 | Application program starting method and related device |
| CN112929491B (en)* | 2021-02-07 | 2022-08-26 | 展讯通信（上海）有限公司 | Application program starting method and related device |
| CN112949440A (en)* | 2021-02-22 | 2021-06-11 | 豪威芯仑传感器(上海)有限公司 | Method for extracting gait features of pedestrian, gait recognition method and system |
| CN115550539A (en)* | 2021-06-30 | 2022-12-30 | 北京小米移动软件有限公司 | Function control method, function control device and storage medium |
| CN115278013A (en)* | 2022-07-22 | 2022-11-01 | 杭州研极微电子有限公司 | Camera and method for controlling camera |
| CN115242952A (en)* | 2022-07-28 | 2022-10-25 | 联想(北京)有限公司 | Image acquisition method and device |
| CN116708655A (en)* | 2022-10-20 | 2023-09-05 | 荣耀终端有限公司 | Screen control method based on event camera and electronic equipment |
| CN116708655B (en)* | 2022-10-20 | 2024-05-03 | 荣耀终端有限公司 | Screen control method and electronic device based on event camera |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111984347B (en) | 2024-09-17 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111988493B (en) | Interaction processing method, device, equipment and storage medium | |
| CN111984347B (en) | Interactive processing method, device, equipment and storage medium | |
| CN106572299B (en) | Camera opening method and device | |
| CN112118380B (en) | Camera manipulation method, device, device and storage medium | |
| EP3136793A1 (en) | Method and apparatus for awakening electronic device | |
| CN110555333A (en) | fingerprint identification method, electronic device and storage medium | |
| CN107786743A (en) | Prevent the method and device of terminal false touch | |
| EP3299946B1 (en) | Method and device for switching environment picture | |
| CN108668080A (en) | Method, device, and electronic device for prompting the degree of lens dirt | |
| CN106101629A (en) | The method and device of output image | |
| CN110262692B (en) | Touch screen scanning method, device and medium | |
| CN107704190B (en) | Gesture recognition method and device, terminal and storage medium | |
| CN106326849A (en) | Beauty processing method and device | |
| CN112148149B (en) | Touch screen control method, touch screen control device and storage medium | |
| CN105791325A (en) | Image sending method and device | |
| CN109039877A (en) | A kind of method, apparatus, electronic equipment and storage medium showing unread message quantity | |
| JP2017532618A (en) | Information processing method and apparatus | |
| CN112331158B (en) | Terminal display adjustment method, device, equipment and storage medium | |
| WO2022095860A1 (en) | Fingernail special effect adding method and device | |
| WO2024067468A1 (en) | Interaction control method and apparatus based on image recognition, and device | |
| CN113361305B (en) | Fingerprint collection method, electronic device and computer-readable storage medium | |
| CN107132983B (en) | Split screen window operation method and device | |
| CN107122697B (en) | Automatic photo acquisition method and device, and electronic device | |
| CN112114653A (en) | Terminal device control method, device, equipment and storage medium | |
| CN106896917A (en) | Aid in method and device, the electronic equipment of Consumer's Experience virtual reality |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |