CN112085942A

Movatterモバイル変換

Info

Publication number: CN112085942A
Application number: CN202011019103.4A
Authority: CN
Inventors: 陈玮
Original assignee: Hansong Nanjing Technology Co ltd
Current assignee: Hansang Nanjing Technology Co ltd
Priority date: 2020-09-25
Filing date: 2020-09-25
Publication date: 2020-12-15
Anticipated expiration: 2040-09-25
Also published as: CN112085942B; CN114399896B; CN114399896A

Abstract

An embodiment of the present specification provides a method and a system for configuring remote control function data according to a remote control device image, including: acquiring a first image of remote control equipment, and determining first characteristics based on the first image, wherein the first characteristics comprise key layout characteristics of the remote control equipment; and determining remote control function data corresponding to the remote control equipment based on the first characteristic. By identifying the image of the remote control equipment and determining the corresponding remote control function data according to the key characteristic layout, the remote control function data can be configured more conveniently and accurately, and the user experience is improved.

Description

Translated fromChinese

一种根据遥控设备图像配置遥控功能数据的方法和系统A method and system for configuring remote control function data according to a remote control device image

技术领域technical field

本申请涉及计算机技术领域，特别涉及一种根据遥控设备图像配置遥控功能数据的方法和系统。The present application relates to the field of computer technology, and in particular, to a method and system for configuring remote control function data according to an image of a remote control device.

背景技术Background technique

随着信息技术的发展，各种不同电子设备如电器、生产设备的遥控已经应用在各个领域。在实现多个电器、生产设备等电子设备的统一遥控控制，或者在电器、生产设备等电子设备的遥控设备不方便使用或遗失时，需要配置与各个电子设备的遥控设备对应的遥控功能数据。每个电子设备对应的品牌和/或型号不同，就对应不同的遥控功能数据，要对电子设备配置对应的遥控功能数据需要找到各个电子设备对应的品牌和/或型号，并根据遥控功能进行配置，效率较低且十分不便。With the development of information technology, remote control of various electronic devices such as electrical appliances and production equipment has been applied in various fields. To achieve unified remote control of multiple electrical appliances, production equipment and other electronic equipment, or when the remote control equipment of electrical appliances, production equipment and other electronic equipment is inconvenient to use or lost, it is necessary to configure the remote control function data corresponding to the remote control equipment of each electronic equipment. The brand and/or model corresponding to each electronic device is different, corresponding to different remote control function data. To configure the corresponding remote control function data of the electronic device, you need to find the brand and/or model corresponding to each electronic device, and configure it according to the remote control function. , which is inefficient and inconvenient.

因此，亟需一种根据遥控设备图像配置遥控功能数据的方法和系统。Therefore, there is an urgent need for a method and system for configuring remote control function data according to an image of a remote control device.

发明内容SUMMARY OF THE INVENTION

本说明书一个方面提供一种根据遥控设备图像配置遥控功能数据的方法。所述方法包括：获取遥控设备的第一图像，并基于所述第一图像确定第一特征，所述第一特征包括所述遥控设备的按键布局特征；基于所述第一特征确定所述遥控设备对应的第一图数据，所述第一图数据包括与多个按键对应的多个第一节点以及至少一个虚拟边；每个所述虚拟边连接所述多个第一节点中的两个，所述第一节点的第一节点属性基于对应的所述按键的属性确定，所述虚拟边的边属性基于所述虚拟边连接的所述第一节点对应的所述按键的位置信息确定；基于所述第一图数据，通过训练好的识别模型确定第一结果，所述第一结果包括得到所述遥控设备对应的遥控功能数据或无所述遥控设备对应的所述遥控功能数据；若第一结果为得到所述遥控设备对应的遥控功能数据，则基于第一结果确定所述遥控设备对应的所述遥控功能数据；若所述第一结果为无所述遥控设备对应的所述遥控功能数据，则：获取与所述遥控设备匹配的被控设备的第二特征，基于所述第二特征和所述第一图数据确定第二图数据，基于所述第二图数据，通过所述训练好的识别模型，得到所述遥控设备对应的所述遥控功能数据；所述识别模型包括图神经网络模型；所述遥控功能数据包括至少一组遥控码，智能设备根据所述至少一组遥控码实现对与所述遥控设备匹配的被控设备的控制。An aspect of the present specification provides a method of configuring remote control function data according to a remote control device image. The method includes: acquiring a first image of a remote control device, and determining a first feature based on the first image, where the first feature includes a key layout feature of the remote control device; determining the remote control based on the first feature First graph data corresponding to the device, the first graph data includes multiple first nodes corresponding to multiple buttons and at least one virtual edge; each virtual edge connects two of the multiple first nodes , the first node attribute of the first node is determined based on the attribute of the corresponding key, and the edge attribute of the virtual edge is determined based on the position information of the key corresponding to the first node connected by the virtual edge; Based on the first image data, the trained recognition model is used to determine a first result, where the first result includes obtaining the remote control function data corresponding to the remote control device or the remote control function data corresponding to no remote control device; if The first result is to obtain the remote control function data corresponding to the remote control device, then determine the remote control function data corresponding to the remote control device based on the first result; if the first result is that there is no remote control corresponding to the remote control device function data, then: acquire the second feature of the controlled device matched with the remote control device, determine the second map data based on the second feature and the first map data, and based on the second map data, The trained identification model is obtained, and the remote control function data corresponding to the remote control device is obtained; the identification model includes a graph neural network model; the remote control function data includes at least one group of remote control codes, and the intelligent device according to the at least one group of The remote control code realizes the control of the controlled device matched with the remote control device.

本说明书另一个方面提供一种根据遥控设备图像配置遥控功能数据的系统。所述系统包括：第一获取模块：用于获取遥控设备的第一图像，并基于所述第一图像确定第一特征，所述第一特征包括所述遥控设备的按键布局特征；基于所述第一特征确定所述遥控设备对应的第一图数据，所述第一图数据包括与多个按键对应的多个第一节点以及至少一个虚拟边；每个所述虚拟边连接所述多个第一节点中的两个，所述第一节点的第一节点属性基于对应的所述按键的属性确定，所述虚拟边的边属性基于所述虚拟边连接的所述第一节点对应的所述按键的位置信息确定；第一识别模块：用于基于所述第一图数据，通过训练好的识别模型确定第一结果，所述第一结果包括得到所述遥控设备对应的遥控功能数据或无所述遥控设备对应的所述遥控功能数据；第一确定模块：用于若第一结果为得到所述遥控设备对应的遥控功能数据，则基于第一结果确定所述遥控设备对应的所述遥控功能数据；第二确定模块：用于若所述第一结果为无所述遥控设备对应的所述遥控功能数据，则：获取与所述遥控设备匹配的被控设备的第二特征，基于所述第二特征和所述第一图数据确定第二图数据，基于所述第二图数据，通过所述训练好的识别模型，得到所述遥控设备对应的所述遥控功能数据；所述识别模型包括图神经网络模型；所述遥控功能数据包括至少一组遥控码，智能设备根据所述至少一组遥控码实现对与所述遥控设备匹配的被控设备的控制。Another aspect of the present specification provides a system for configuring remote control function data according to a remote control device image. The system includes: a first acquisition module: for acquiring a first image of a remote control device, and determining a first feature based on the first image, where the first feature includes a key layout feature of the remote control device; based on the The first feature determines first graph data corresponding to the remote control device, the first graph data includes a plurality of first nodes corresponding to a plurality of buttons and at least one virtual edge; each of the virtual edges connects the plurality of Two of the first nodes, the first node attribute of the first node is determined based on the attribute of the corresponding key, and the edge attribute of the virtual edge is based on the virtual edge connected to the first node corresponding to the first node. The position information of the button is determined; the first identification module is used to determine the first result based on the first map data through the trained identification model, and the first result includes obtaining the remote control function data corresponding to the remote control device or The remote control function data corresponding to the remote control device is not available; the first determination module is configured to determine the remote control function data corresponding to the remote control device based on the first result if the first result is to obtain the remote control function data corresponding to the remote control device remote control function data; a second determination module: for if the first result is that there is no remote control function data corresponding to the remote control device, then: obtain the second feature of the controlled device that matches the remote control device, based on The second feature and the first image data determine second image data, and based on the second image data, the remote control function data corresponding to the remote control device is obtained through the trained recognition model; the The identification model includes a graph neural network model; the remote control function data includes at least one set of remote control codes, and the intelligent device controls the controlled device matched with the remote control device according to the at least one set of remote control codes.

本说明书另一个方面提供一种根据遥控设备图像配置遥控功能数据的装置，包括处理器，所述处理器用于执行根据遥控设备图像配置遥控功能数据的方法。Another aspect of the present specification provides an apparatus for configuring remote control function data according to a remote control device image, including a processor configured to execute a method for configuring remote control function data according to a remote control device image.

本说明书另一个方面提供计算机可读存储介质，所述存储介质存储计算机指令，当计算机读取存储介质中的计算机指令后，计算机执行根据遥控设备图像配置遥控功能数据的方法。Another aspect of the present specification provides a computer-readable storage medium storing computer instructions, and after the computer reads the computer instructions in the storage medium, the computer executes a method of configuring remote control function data according to a remote control device image.

本说明书带来的有益效果包括但不限于：（1）通过识别遥控设备的图像，根据按键特征布局确定对应的遥控功能数据，不需通过查找品牌/或型号后再进行配置，实现更加方便和准确地配置遥控功能数据，提升用户体验；（2）基于遥控设备的图像特征构建图数据，将遥控设备的按键特征和按键间布局信息体现得更加完善，通过图数据识别的方式，实现更加准确地预测；（3）在构建图数据时可以结合被控设备的特征，实现在无法根据遥控设备确定遥控功能数据，可以结合被控设备特征确定，预测更加准确；（4）可以直接通过被控设备的图像、或用户的语音输入、文本输入等得到被控设备的特征信息，实现更加方便地确定对应的遥控功能数据；（5）对确定的遥控功能数据添加标识并存储后，可以直接调用对应被控设备的遥控功能数据来实现控制，控制更加方便高效，提升了用户体验；（6）对确定的遥控功能数据进行测试，可以确保配置的遥控功能数据的准确性，避免错误配置；（7）在确定的遥控功能数据为无效时，基于重新确定的结果更新识别模型的参数，可以提高模型的识别准确率。The beneficial effects brought by this manual include, but are not limited to: (1) By identifying the image of the remote control device and determining the corresponding remote control function data according to the key feature layout, it is not necessary to search for the brand/or model and then configure it, so as to achieve more convenient and Accurately configure the remote control function data to improve the user experience; (2) Construct map data based on the image features of the remote control device, reflect the key features of the remote control device and the layout information between the keys more fully, and achieve more accurate through the method of map data recognition. (3) When constructing map data, it can be combined with the characteristics of the controlled device, so that when the remote control function data cannot be determined according to the remote control device, it can be determined in combination with the characteristics of the controlled device, and the prediction is more accurate; (4) It can be directly predicted by the controlled device. The image of the device, or the user's voice input, text input, etc., can obtain the feature information of the controlled device, so that the corresponding remote control function data can be more easily determined; (5) After adding a logo to the determined remote control function data and storing it, it can be directly called Corresponding to the remote control function data of the controlled device to achieve control, the control is more convenient and efficient, and the user experience is improved; (6) Test the determined remote control function data to ensure the accuracy of the configured remote control function data and avoid misconfiguration; ( 7) When the determined remote control function data is invalid, updating the parameters of the recognition model based on the re-determined result can improve the recognition accuracy of the model.

附图说明Description of drawings

本说明书将以示例性实施例的方式进一步说明，这些示例性实施例将通过附图进行详细描述。这些实施例并非限制性的，在这些实施例中，相同的编号表示相同的结构，其中：The present specification will be further described by way of example embodiments, which will be described in detail with reference to the accompanying drawings. These examples are not limiting, and in these examples, the same numbers refer to the same structures, wherein:

图1是根据本说明书一些实施例所示的根据遥控设备图像配置遥控功能数据的系统的应用场景示意图；1 is a schematic diagram of an application scenario of a system for configuring remote control function data according to a remote control device image according to some embodiments of this specification;

图2是根据本说明书的一些实施例所示的示例性根据遥控设备图像配置遥控功能数据的系统的模块图；2 is a block diagram of an exemplary system for configuring remote control function data according to a remote control device image according to some embodiments of the present specification;

图3是根据本说明书一些实施例所示的根据遥控设备图像配置遥控功能数据的方法的示例性流程图；3 is an exemplary flowchart of a method for configuring remote control function data according to a remote control device image according to some embodiments of the present specification;

图4是根据本说明书一些实施例所示的识别模型训练的方法的示例性流程图；FIG. 4 is an exemplary flowchart of a method for training a recognition model according to some embodiments of the present specification;

图5是根据本说明书一些实施例所示的另一种根据遥控设备图像配置遥控功能数据的方法的示例性流程图；5 is an exemplary flowchart of another method for configuring remote control function data according to a remote control device image according to some embodiments of the present specification;

图6是根据本说明书一些实施例所示的另一种识别模型训练的方法的示例性流程图；FIG. 6 is an exemplary flowchart of another method for training a recognition model according to some embodiments of the present specification;

图7是根据本说明书一些实施例所示的基于第一图数据和第二特征确定第二图数据的示例图。FIG. 7 is an example diagram of determining second map data based on first map data and second features according to some embodiments of the present specification.

具体实施方式Detailed ways

为了更清楚地说明本说明书实施例的技术方案，下面将对实施例描述中所需要使用的附图作简单的介绍。显而易见地，下面描述中的附图仅仅是本说明书的一些示例或实施例，对于本领域的普通技术人员来讲，在不付出创造性劳动的前提下，还可以根据这些附图将本说明书应用于其它类似情景。除非从语言环境中显而易见或另做说明，图中相同标号代表相同结构或操作。In order to illustrate the technical solutions of the embodiments of the present specification more clearly, the following briefly introduces the accompanying drawings used in the description of the embodiments. Obviously, the accompanying drawings in the following description are only some examples or embodiments of the present specification. For those of ordinary skill in the art, without creative efforts, the present specification can also be applied to the present specification according to these drawings. other similar situations. Unless obvious from the locale or otherwise specified, the same reference numbers in the figures represent the same structure or operation.

应当理解，本说明书中所使用的“系统”、“装置”、“单元”和/或“模组”是用于区分不同级别的不同组件、元件、部件、部分或装配的一种方法。然而，如果其他词语可实现相同的目的，则可通过其他表达来替换所述词语。It should be understood that "system", "device", "unit" and/or "module" as used in this specification is a method used to distinguish different components, elements, parts, parts or assemblies at different levels. However, other words may be replaced by other expressions if they serve the same purpose.

如本说明书和权利要求书中所示，除非上下文明确提示例外情形，“一”、“一个”、“一种”和/或“该”等词并非特指单数，也可包括复数。一般说来，术语“包括”与“包含”仅提示包括已明确标识的步骤和元素，而这些步骤和元素不构成一个排它性的罗列，方法或者设备也可能包含其它的步骤或元素。As shown in the specification and claims, unless the context clearly dictates otherwise, the words "a", "an", "an" and/or "the" are not intended to be specific in the singular and may include the plural. Generally speaking, the terms "comprising" and "comprising" only imply that the clearly identified steps and elements are included, and these steps and elements do not constitute an exclusive list, and the method or apparatus may also include other steps or elements.

本说明书中使用了流程图用来说明根据本说明书的实施例的系统所执行的操作。应当理解的是，前面或后面操作不一定按照顺序来精确地执行。相反，可以按照倒序或同时处理各个步骤。同时，也可以将其他操作添加到这些过程中，或从这些过程移除某一步或数步操作。Flowcharts are used in this specification to illustrate operations performed by a system according to an embodiment of this specification. It should be understood that the preceding or following operations are not necessarily performed in the exact order. Instead, the various steps can be processed in reverse order or simultaneously. At the same time, other actions can be added to these procedures, or a step or steps can be removed from these procedures.

图1是根据本说明书的一些实施例所示的根据遥控设备图像配置遥控功能数据的系统的应用场景示意图。FIG. 1 is a schematic diagram of an application scenario of a system for configuring remote control function data according to a remote control device image according to some embodiments of the present specification.

根据遥控设备图像配置遥控功能数据的系统的场景100可以包括终端设备110、处理器120和网络130。Thescenario 100 of a system for configuring remote control function data according to a remote control device image may include aterminal device 110 , aprocessor 120 and anetwork 130 .

根据遥控设备图像配置遥控功能数据的系统的场景100可以用于智能家居中对各个电器的遥控数据配置、工厂中对各个电子设备的遥控数据配置等。在实现多个电器等电子设备的统一控制，或者对电器或电子设备的遥控设备不方便使用或遗失时，需要配置与各个电子设备的遥控设备对应的遥控功能数据。可以通过终端设备110获取遥控设备的图像，然后处理器120可以识别遥控设备的图像，获取遥控设备的相关特征，例如按键布局特征，根据按键布局特征通过识别模型确定对应的遥控功能数据，可以令配置遥控功能数据更加方便。处理器120和终端设备110之间可以通过网络130来进行数据传输。Thescene 100 of the system for configuring remote control function data according to a remote control device image can be used for remote control data configuration for each electrical appliance in a smart home, remote control data configuration for each electronic device in a factory, and the like. When realizing unified control of multiple electrical appliances and other electronic devices, or when the remote control devices of electrical appliances or electronic devices are inconvenient to use or lose, it is necessary to configure remote control function data corresponding to the remote control devices of each electronic device. The image of the remote control device can be obtained through theterminal device 110, and then theprocessor 120 can identify the image of the remote control device, obtain relevant features of the remote control device, such as key layout features, and determine the corresponding remote control function data by identifying the model according to the key layout features. It is more convenient to configure remote control function data. Data transmission may be performed between theprocessor 120 and theterminal device 110 through thenetwork 130 .

终端设备110指可以被用户所使用的一个或多个终端设备或软件。在一些实施例中，使用终端设备110的可以是一个或多个用户，可以包括直接使用服务的用户，也可以包括其他相关用户。在一些实施例中，终端设备110可以是移动设备110-1、平板计算机110-2、膝上型计算机110-3等其他具有获取图像、输入和/或输出功能的设备中的一种或其任意组合。Terminal device 110 refers to one or more terminal devices or software that can be used by a user. In some embodiments, one or more users may use theterminal device 110, which may include users who directly use the service, and may also include other related users. In some embodiments, theterminal device 110 may be one of the mobile device 110-1, the tablet computer 110-2, the laptop computer 110-3, and other devices that have the functions of capturing images, inputting, and/or outputting, or the same. random combination.

处理器120可以处理从其他设备或系统组成部分中获得的数据和/或信息。处理器可以基于这些数据、信息和/或处理结果执行程序指令，以执行一个或多个本申请中描述的功能。在一些实施例中，处理器120可以包含一个或多个子处理设备（例如，单核处理设备或多核多芯处理设备）。仅作为示例，处理器110可以包括中央处理器（CPU）、专用集成电路（ASIC）、专用指令处理器（ASIP）、图形处理器（GPU）、物理处理器（PPU）、数字信号处理器（DSP）、现场可编程门阵列（FPGA）、可编辑逻辑电路（PLD）、控制器、微控制器单元、精简指令集电脑（RISC）、微处理器等或以上任意组合。在一些实施例中，处理器120以及其他可能的系统组成部分中可以包括存储设备。Processor 120 may process data and/or information obtained from other devices or system components. The processor may execute program instructions based on such data, information and/or processing results to perform one or more of the functions described herein. In some embodiments,processor 120 may include one or more sub-processing devices (eg, single-core processing devices or multi-core multi-core processing devices). For example only, theprocessor 110 may include a central processing unit (CPU), an application specific integrated circuit (ASIC), an application specific instruction processor (ASIP), a graphics processor (GPU), a physical processor (PPU), a digital signal processor ( DSP), field programmable gate array (FPGA), programmable logic circuit (PLD), controller, microcontroller unit, reduced instruction set computer (RISC), microprocessor, etc. or any combination of the above. In some embodiments, storage devices may be included in theprocessor 120 and possibly other system components.

网络130可以连接系统的各组成部分和/或连接系统与外部资源部分。网络130使得各组成部分之间，以及与系统之外其他部分之间可以进行通讯，促进数据和/或信息的交换。在一些实施例中，网络130可以是有线网络或无线网络中的任意一种或多种。例如，网络130可以包括电缆网络、光纤网络、电信网络、互联网、局域网络（LAN）、广域网络（WAN）、无线局域网络（WLAN）、城域网（MAN）、公共交换电话网络（PSTN）、蓝牙网络、紫蜂网络（ZigBee）、近场通信（NFC）、设备内总线、设备内线路、线缆连接等或其任意组合。各部分之间的网络连接可以是采用上述一种方式，也可以是采取多种方式。在一些实施例中，网络可以是点对点的、共享的、中心式的等各种拓扑结构或者多种拓扑结构的组合。在一些实施例中，网络130可以包括一个或以上网络接入点。例如，网络130可以包括有线或无线网络接入点，例如基站和/或网络交换点网络130-1、网络130-2、…，通过这些进出点，场景100的一个或多个组件可连接到网络130上以交换数据和/或信息。Thenetwork 130 may connect components of the system and/or connect portions of the system with external resources. Thenetwork 130 enables communication between the various components, as well as with other components outside the system, facilitating the exchange of data and/or information. In some embodiments, thenetwork 130 may be any one or more of a wired network or a wireless network. For example, thenetwork 130 may include a cable network, a fiber optic network, a telecommunications network, the Internet, a local area network (LAN), a wide area network (WAN), a wireless local area network (WLAN), a metropolitan area network (MAN), a public switched telephone network (PSTN) , Bluetooth network, ZigBee network (ZigBee), near field communication (NFC), in-device bus, in-device line, cable connection, etc. or any combination thereof. The network connection between the various parts can be in one of the above-mentioned ways, and can also be in a variety of ways. In some embodiments, the network may be in point-to-point, shared, centralized, etc. various topologies or a combination of multiple topologies. In some embodiments,network 130 may include one or more network access points. For example,network 130 may include wired or wireless network access points, such as base stations and/or network switching points network 130-1, network 130-2, . . . , through which one or more components ofscene 100 may connect to network 130 to exchange data and/or information.

图2是根据本说明书的一些实施例所示的示例性根据遥控设备图像配置遥控功能数据的系统的模块图。2 is a block diagram of an exemplary system for configuring remote control function data from a remote control device image, according to some embodiments of the present specification.

在一些实施例中，根据遥控设备图像配置遥控功能数据的系统200可以包括第一获取模块210、第一识别模块220、第一确定模块230、第二确定模块240、存储模块250、识别模型训练模块260、测试模块270、语音确定模块280。In some embodiments, thesystem 200 for configuring remote control function data according to a remote control device image may include a first acquisition module 210, a first identification module 220, a first determination module 230, asecond determination module 240, astorage module 250, a recognition modeltraining module Module 260 ,testing module 270 ,voice determination module 280 .

第一获取模块210可以用于获取遥控设备的第一图像，并基于所述第一图像确定第一特征，所述第一特征包括所述遥控设备的按键布局特征；基于所述第一特征确定所述遥控设备对应的第一图数据，所述第一图数据包括与多个按键对应的多个第一节点、以及至少一个虚拟边；每个所述虚拟边连接所述多个第一节点中的两个，所述第一节点的第一节点属性基于对应的所述按键的属性确定，所述虚拟边的边属性基于所述虚拟边连接的所述第一节点对应的所述按键的位置信息确定。The first acquisition module 210 may be configured to acquire a first image of a remote control device, and determine a first feature based on the first image, where the first feature includes a key layout feature of the remote control device; determine based on the first feature First graph data corresponding to the remote control device, the first graph data includes a plurality of first nodes corresponding to a plurality of buttons, and at least one virtual edge; each of the virtual edges connects the plurality of first nodes Two of the first node attributes of the first node are determined based on the attributes of the corresponding keys, and the edge attributes of the virtual edges are based on the attributes of the keys corresponding to the first nodes connected by the virtual edges. Location information is determined.

第一识别模块220可以用于基于所述第一图数据，通过训练好的识别模型确定第一结果，所述第一结果包括得到所述遥控设备对应的遥控功能数据或无所述遥控设备对应的所述遥控功能数据。所述识别模型可以包括图神经网络模型。所述遥控功能数据包括至少一组遥控码，智能设备根据所述至少一组遥控码实现对与所述遥控设备匹配的被控设备的控制。The first identification module 220 may be configured to determine a first result based on the first image data and through the trained identification model, where the first result includes obtaining the remote control function data corresponding to the remote control device or without the corresponding remote control device. of the remote control function data. The recognition model may comprise a graph neural network model. The remote control function data includes at least one set of remote control codes, and the smart device controls the controlled device matched with the remote control device according to the at least one set of remote control codes.

第一确定模块230可以用于若第一结果为得到所述遥控设备对应的遥控功能数据，则基于第一结果确定所述遥控设备对应的所述遥控功能数据。The first determining module 230 may be configured to determine the remote control function data corresponding to the remote control device based on the first result if the first result is to obtain the remote control function data corresponding to the remote control device.

第二确定模块240可以用于若所述第一结果为无所述遥控设备对应的所述遥控功能数据，则：获取与所述遥控设备匹配的被控设备的第二特征，基于所述第二特征和所述第一图数据确定第二图数据，基于所述第二图数据，通过所述训练好的识别模型，得到所述遥控设备对应的所述遥控功能数据。在一些实施例中，第二确定模块240还可以用于：确定第二节点，所述第二节点的第二节点属性基于所述第二特征确定；根据所述第二节点和所述第一图数据确定所述第二图数据。在一些实施例中，第二确定模块240还可以用于：获取与所述遥控设备匹配的被控设备的第二图像；将所述第二图像输入特征生成模型，得到所述第二图像对应的所述第二特征。在一些实施例中，第二确定模块240还可以用于：获取用户输入的音频信息和/或文本信息，所述音频信息和所述文本信息包括所述被控设备的设备信息；根据所述音频信息和/或所述文本信息得到所述被控设备的所述第二特征。The second determiningmodule 240 may be configured to: if the first result is that there is no remote control function data corresponding to the remote control device, then: acquire the second feature of the controlled device that matches the remote control device, and based on the first The second feature and the first image data determine second image data, and based on the second image data, the remote control function data corresponding to the remote control device is obtained through the trained recognition model. In some embodiments, the second determiningmodule 240 may be further configured to: determine a second node, the second node attribute of the second node is determined based on the second characteristic; according to the second node and the first Map data determines the second map data. In some embodiments, the second determiningmodule 240 may be further configured to: acquire a second image of the controlled device matched with the remote control device; input the second image into a feature generation model to obtain the corresponding second image of the second feature. In some embodiments, the second determiningmodule 240 may be further configured to: acquire audio information and/or text information input by the user, where the audio information and the text information include device information of the controlled device; according to the The audio information and/or the text information obtain the second characteristic of the controlled device.

存储模块250可以用于对所述遥控设备对应的所述遥控功能数据添加标识并存储，所述标识与所述遥控设备匹配的被控设备相关，得到包括至少一个所述遥控设备对应的所述遥控功能数据的遥控功能数据库；所述智能设备在所述遥控功能数据库中查找与目标被控设备相关的标识，并调用所述目标被控设备相关的标识对应的所述遥控功能数据，实现对所述目标被控设备的控制。Thestorage module 250 can be used to add and store an identifier to the remote control function data corresponding to the remote control device. A remote control function database of remote control function data; the intelligent device searches the remote control function database for an identifier related to the target controlled device, and calls the remote control function data corresponding to the identifier related to the target controlled device, so as to realize the The control of the target controlled device.

识别模型训练模块260可以用于获取至少一个所述遥控设备对应的至少一个第一图数据样本，以及所述至少一个第一图数据样本对应的第一标签，所述第一标签包括所述第一图数据样本对应的所述遥控功能数据或所述第一图数据样本无对应的所述遥控功能数据；获取至少一个与所述遥控设备匹配的被控设备对应的至少一个第二特征样本，基于所述至少一个第二特征样本和所述至少一个第一图数据样本得到至少一个第二图数据样本，以及获取所述至少一个第二图数据样本对应的第二标签，所述第二标签包括所述第二图数据样本对应的所述遥控功能数据；以带有所述第一标签的所述至少一个第一图数据样本和带有所述第二标签的所述至少一个第二图数据样本作为训练样本，对初始识别模型进行训练，得到所述训练好的识别模型。The recognitionmodel training module 260 may be configured to acquire at least one first image data sample corresponding to at least one of the remote control devices, and a first label corresponding to the at least one first image data sample, where the first label includes the first image data sample. The remote control function data corresponding to one image data sample or the first image data sample has no corresponding remote control function data; at least one second feature sample corresponding to at least one controlled device matched with the remote control device is obtained, Obtain at least one second image data sample based on the at least one second feature sample and the at least one first image data sample, and acquire a second label corresponding to the at least one second image data sample, the second label including the remote control function data corresponding to the second image data sample; the at least one first image data sample with the first label and the at least one second image with the second label The data samples are used as training samples to train the initial recognition model to obtain the trained recognition model.

测试模块270可以用于采用所述遥控功能数据对与对应的所述遥控设备匹配的所述被控设备进行测试；若测试成功，则所述遥控功能数据有效；若测试失败，则获取所述被控设备对应的品牌列表和/或型号列表，基于所述品牌列表和/或所述型号列表选择所述遥控设备匹配的被控设备的品牌和/或型号；基于选择的所述品牌和/或所述型号和所述第一特征，重新确定所述遥控设备对应的所述遥控功能数据。在一些实施例中，测试模型270还可以用于：根据测试失败对应的所述遥控功能数据与重新确定的所述遥控设备对应的所述遥控功能数据之间的差异构建损失函数，基于所述损失函数更新所述识别模型的参数。Thetest module 270 can be configured to use the remote control function data to test the controlled device matched with the corresponding remote control device; if the test is successful, the remote control function data is valid; if the test fails, obtain the Brand list and/or model list corresponding to the controlled device, select the brand and/or model of the controlled device matched by the remote control device based on the brand list and/or the model list; based on the selected brand and/or model Or the model and the first feature, the remote control function data corresponding to the remote control device is re-determined. In some embodiments, thetest model 270 may also be used to: construct a loss function according to the difference between the remote control function data corresponding to the test failure and the remote control function data corresponding to the re-determined remote control device, and based on the The loss function updates the parameters of the recognition model.

语音确定模块280可以用于基于所述遥控设备对应的所述遥控功能数据的所述至少一组遥控码，确定所述遥控设备对应的语音控制数据；所述智能设备根据所述语音控制数据，实现对与所述遥控设备匹配的被控设备的语音控制。Thevoice determination module 280 may be configured to determine the voice control data corresponding to the remote control device based on the at least one set of remote control codes of the remote control function data corresponding to the remote control device; the intelligent device, according to the voice control data, The voice control of the controlled device matched with the remote control device is realized.

应当理解，所示的系统及其模块可以利用各种方式来实现。例如，在一些实施例中，系统及其模块可以通过硬件、软件或者软件和硬件的结合来实现。其中，硬件部分可以利用专用逻辑来实现；软件部分则可以存储在存储器中，由适当的指令执行系统，例如微处理器或者专用设计硬件来执行。本领域技术人员可以理解上述的方法和系统可以使用计算机可执行指令和/或包含在处理器控制代码中来实现，例如在诸如磁盘、CD或DVD-ROM的载体介质、诸如只读存储器（固件）的可编程的存储器或者诸如光学或电子信号载体的数据载体上提供了这样的代码。本申请的系统及其模块不仅可以有诸如超大规模集成电路或门阵列、诸如逻辑芯片、晶体管等的半导体、或者诸如现场可编程门阵列、可编程逻辑设备等的可编程硬件设备的硬件电路实现，也可以用例如由各种类型的处理器所执行的软件实现，还可以由上述硬件电路和软件的结合（例如，固件）来实现。It should be understood that the illustrated system and its modules may be implemented in a variety of ways. For example, in some embodiments, the system and its modules may be implemented in hardware, software, or a combination of software and hardware. Wherein, the hardware part can be realized by using dedicated logic; the software part can be stored in a memory and executed by a suitable instruction execution system, such as a microprocessor or specially designed hardware. Those skilled in the art will appreciate that the methods and systems described above may be implemented using computer-executable instructions and/or embodied in processor control code, for example on a carrier medium such as a disk, CD or DVD-ROM, such as a read-only memory (firmware) ) of a programmable memory or a data carrier such as an optical or electronic signal carrier such a code is provided. The system and its modules of the present application can not only be implemented by hardware circuits such as very large scale integrated circuits or gate arrays, semiconductors such as logic chips, transistors, etc., or programmable hardware devices such as field programmable gate arrays, programmable logic devices, etc. , can also be implemented by software executed by various types of processors, for example, or by a combination of the above-mentioned hardware circuits and software (eg, firmware).

需要注意的是，以上对于根据遥控设备图像配置遥控功能数据的系统200及其模块的描述，仅为描述方便，并不能把本说明书限制在所举实施例范围之内。可以理解，对于本领域的技术人员来说，在了解该系统的原理后，可能在不背离这一原理的情况下，对各个模块进行任意组合，或者构成子系统与其他模块连接。例如，第一获取模块210、第一识别模块220、第一确定模块230、第二确定模块240、存储模块250、识别模型训练模块260、测试模块270、语音确定模块280可以共用一个存储模块，各个模块也可以分别具有各自的存储模块。诸如此类的变形，均在本申请的保护范围之内。It should be noted that the above description of thesystem 200 and its modules for configuring remote control function data according to a remote control device image is only for convenience of description, and does not limit the description to the scope of the illustrated embodiments. It can be understood that for those skilled in the art, after understanding the principle of the system, various modules may be combined arbitrarily, or a subsystem may be formed to connect with other modules without departing from the principle. For example, the first acquisition module 210, the first recognition module 220, the first determination module 230, thesecond determination module 240, thestorage module 250, the recognitionmodel training module 260, thetest module 270, and thevoice determination module 280 may share a storage module, Each module may also have its own storage module, respectively. Such deformations are all within the protection scope of the present application.

图3是根据本说明书的一些实施例所示的根据遥控设备图像配置遥控功能数据的方法的示例性流程图。FIG. 3 is an exemplary flowchart of a method for configuring remote control function data according to a remote control device image according to some embodiments of the present specification.

如图3所示，该根据遥控设备图像配置遥控功能数据的方法300可以包括：As shown in FIG. 3 , themethod 300 for configuring remote control function data according to a remote control device image may include:

步骤310，获取遥控设备的第一图像，并基于所述第一图像确定第一特征，所述第一特征包括所述遥控设备的按键布局特征。Step 310: Acquire a first image of the remote control device, and determine a first feature based on the first image, where the first feature includes a key layout feature of the remote control device.

具体的，该步骤310可以由第一获取模块210执行。Specifically, thisstep 310 may be performed by the first obtaining module 210 .

遥控设备是指用于遥控电子设备的设备，例如遥控器、手机、计算机等可以实现遥控的终端设备。Remote control equipment refers to equipment used to remotely control electronic equipment, such as remote control, mobile phone, computer and other terminal equipment that can realize remote control.

包含遥控设备相关图像的图像可以称为第一图像。第一图像至少包括含有遥控设备按键布局的图像，还可以包括遥控设备的标识图像、表现出遥控设备形状、尺寸等外形的整体图像、遥控设备的应用环境图像等其它遥控设备的相关图像。第一图像可以是2D图像、3D图像或其它多维的图像，本实施例不做限制。The image containing the image related to the remote control device may be referred to as the first image. The first image includes at least an image containing the key layout of the remote control device, and may also include a logo image of the remote control device, an overall image showing the shape and size of the remote control device, and images related to other remote control devices such as an application environment image of the remote control device. The first image may be a 2D image, a 3D image, or other multi-dimensional images, which are not limited in this embodiment.

在一些实施例中，获取遥控设备的图像可以通过扫描、拍照、摄影等方式，本实施例不做限制。In some embodiments, the image of the remote control device may be acquired by scanning, photographing, photographing, etc., which is not limited in this embodiment.

遥控功能数据是指用于实现遥控电子设备的控制数据，可以包括遥控码、控制数据包等。遥控功能数据可以用于遥控设备或其它智能设备实现对电子设备的遥控，遥控的方式可以是红外遥控、无线电信号遥控、语音遥控等，本实施例不做限制。The remote control function data refers to the control data used to realize the remote control of the electronic device, which may include remote control codes, control data packets, and the like. The remote control function data can be used for remote control equipment or other intelligent equipment to realize remote control of electronic equipment. The remote control method can be infrared remote control, radio signal remote control, voice remote control, etc., which is not limited in this embodiment.

遥控设备遥控的电子设备可以称为被控设备，遥控设备会有匹配的被控设备，遥控设备对应的遥控功能数据可以用于遥控与其匹配的被控设备。An electronic device that is remotely controlled by a remote control device can be called a controlled device. The remote control device will have a matched controlled device, and the remote control function data corresponding to the remote control device can be used to remotely control the matched controlled device.

在一些实施例中，遥控功能数据至少包括一组遥控码，例如，调高、调低、开启和关闭等功能对应的一组按键所对应的一组遥控码。遥控设备和其它智能设备（手机、计算机等可以实现遥控的智能设备）可以根据至少一组遥控码实现对与所述遥控设备匹配的被控设备的控制。In some embodiments, the remote control function data includes at least a group of remote control codes, for example, a group of remote control codes corresponding to a group of buttons corresponding to functions such as up, down, on, and off. The remote control device and other smart devices (mobile phones, computers, and other smart devices that can realize remote control) can control the controlled device matched with the remote control device according to at least one set of remote control codes.

在一些实施例中，还可以基于遥控设备对应的至少一组遥控码，确定遥控设备对应的语音控制数据。例如可以通过计算机或者数据转换装置将遥控码转换为对应的语音控制数据。遥控设备或其它智能设备可以根据语音控制数据，实现对与遥控设备匹配的被控设备的语音控制。In some embodiments, the voice control data corresponding to the remote control device may also be determined based on at least one set of remote control codes corresponding to the remote control device. For example, the remote control code can be converted into corresponding voice control data by a computer or a data conversion device. The remote control device or other intelligent device can realize the voice control of the controlled device matched with the remote control device according to the voice control data.

在一些实施例中，不同品牌和/或型号的电子设备对应的遥控功能数据可以是不同的，例如，品牌A和品牌B的电子设备对应的遥控功能数据分别是a类型或b类型。不同按键功能对应的遥控功能数据可以是不同的，例如调高按键和调低按键对应的遥控功能数据不同。In some embodiments, the remote control function data corresponding to electronic devices of different brands and/or models may be different. For example, the remote control function data corresponding to electronic devices of brand A and brand B are type a or type b, respectively. The remote control function data corresponding to different button functions may be different, for example, the remote control function data corresponding to the up button and the down button are different.

在一些实施例中，得到遥控设备的第一图像后，可以将第一图像用于确定遥控设备对应的遥控功能数据。例如不同的按键布局对应为不同的品牌、型号和按键功能，即对应为不同的遥控功能数据，根据第一图像中包括的含有遥控设备按键布局的图像，可以去确定与第一图像对应的遥控功能数据，也就是第一图像所对应遥控设备的遥控功能数据。In some embodiments, after the first image of the remote control device is obtained, the first image may be used to determine remote control function data corresponding to the remote control device. For example, different key layouts correspond to different brands, models, and key functions, that is, to different remote control function data. According to the image containing the key layout of the remote control device included in the first image, the remote control corresponding to the first image can be determined. The function data, that is, the remote control function data of the remote control device corresponding to the first image.

在一些实施例中，可以基于第一图像确定第一特征。第一特征是指第一图像包含的相关图像特征，第一特征至少包括遥控设备的按键布局特征，还可以包括遥控设备的标识特征、遥控设备的形状、尺寸等外形特征、应用环境特征等与遥控设备相关的其它图像特征。In some embodiments, the first feature may be determined based on the first image. The first feature refers to the relevant image features contained in the first image. The first feature includes at least the key layout feature of the remote control device, and may also include the identification feature of the remote control device, the shape and size of the remote control device and other external features, application environment features, etc. Other image features related to the remote control device.

在一些实施例中，可以采用对图像进行图像识别得到相应的第一特征，也可以通过将第一图像输入特征生成模型，得到第一图像对应的第一特征。特征生成模型的输入是图像，输出是图像对应的特征向量。具体地，将第一图像输入特征生成模型，输出第一图像对应的特征向量，该特征向量可以作为被控设备对应的第一特征。在一些实施例中，特征生成模型可以采用神经网络模型，例如CNN或VGG等常用的卷积神经网络模型。以VGG为例，将第一图像输入VGG中，通过卷积层、池化层、全连接层，可以输出得到第一图像对应的特征向量。In some embodiments, the corresponding first feature may be obtained by performing image recognition on the image, or the first feature corresponding to the first image may be obtained by inputting the first image into a feature generation model. The input of the feature generation model is an image, and the output is the feature vector corresponding to the image. Specifically, the first image is input into the feature generation model, and a feature vector corresponding to the first image is output, and the feature vector can be used as the first feature corresponding to the controlled device. In some embodiments, the feature generation model may use a neural network model, such as a common convolutional neural network model such as CNN or VGG. Taking VGG as an example, the first image is input into VGG, and the feature vector corresponding to the first image can be output through the convolution layer, pooling layer, and fully connected layer.

第一特征可以用于确定遥控设备对应的遥控功能数据。例如，可以直接基于第一特征确定对应遥控设备的遥控功能数据，还可以基于第一特征去确定对应遥控设备的品牌和/或型号，以及按键功能，并基于品牌和/或型号，以及按键功能确定对应的遥控功能数据。具体地，可以建立包括第一特征、和/或第一特征对应的品牌、型号、按键功能，以及对应的遥控功能数据的数据库，通过查找数据库，实现基于第一特征确定遥控设备对应的遥控功能数据。还可以通过训练神经网络模型，得到可以输入第一特征，输出得到对应遥控设备的遥控功能数据的识别模型来实现。其中，识别模型可以采用逻辑回归模型、分类和回归树、支持向量机、分类和回归树、随机森林或其它用于分类的神经网络模型，本实施例不做限制。The first feature may be used to determine remote control function data corresponding to the remote control device. For example, the remote control function data of the corresponding remote control device can be determined directly based on the first feature, and the brand and/or model of the corresponding remote control device and the key function can also be determined based on the first feature, and based on the brand and/or model, and the key function Determine the corresponding remote control function data. Specifically, a database including the first feature and/or the brand, model, button function corresponding to the first feature, and corresponding remote control function data can be established, and by searching the database, the remote control function corresponding to the remote control device can be determined based on the first feature. data. It can also be realized by training a neural network model to obtain a recognition model that can input the first feature and output the remote control function data corresponding to the remote control device. The identification model may adopt a logistic regression model, a classification and regression tree, a support vector machine, a classification and regression tree, a random forest, or other neural network models for classification, which are not limited in this embodiment.

步骤320，基于所述第一特征确定所述遥控设备对应的第一图数据，所述第一图数据包括与多个按键对应的多个第一节点以及至少一个虚拟边；每个所述虚拟边连接所述多个第一节点中的两个，所述第一节点的第一节点属性基于对应的所述按键的属性确定，所述虚拟边的边属性基于所述虚拟边连接的所述第一节点对应的所述按键的位置信息确定。Step 320: Determine first graph data corresponding to the remote control device based on the first feature, where the first graph data includes a plurality of first nodes corresponding to a plurality of keys and at least one virtual edge; each of the virtual The edge connects two of the plurality of first nodes, the first node attribute of the first node is determined based on the attribute of the corresponding key, and the edge attribute of the virtual edge is based on the virtual edge connected. The position information of the button corresponding to the first node is determined.

具体的，该步骤320可以由第一获取模块210执行。Specifically, thisstep 320 may be performed by the first obtaining module 210 .

在一些实施例中，可以基于第一特征去确定遥控设备对应的第一图数据。图数据是指使用图形结构的数据，包含了节点、边和属性来表示和存储数据。根据第一特征确定的图数据，可以称为第一图数据，如图7中所示。In some embodiments, the first map data corresponding to the remote control device may be determined based on the first feature. Graph data refers to data using a graph structure, including nodes, edges, and attributes to represent and store data. The graph data determined according to the first feature may be referred to as first graph data, as shown in FIG. 7 .

第一节点是指第一图数据中表示按键及按键相关信息的节点，每一个按键对应有一个第一节点，第一节点的属性可以基于对应的按键的属性确定。如图7中700所示，第一图数据中包括有第一节点U1、U2、U3、U4、U5。按键的属性是指按键的相关信息，例如按键的功能、按键的位置、按键的形状、按键的尺寸等。第一特征中，遥控设备的按键可以包括多个，所以对应第一图数据中可以包括与对各按键对应的多个第一节点。The first node refers to a node in the first graph data representing keys and key-related information, each key corresponds to a first node, and an attribute of the first node can be determined based on the attribute of the corresponding key. As shown in 700 in FIG. 7 , the first graph data includes first nodes U1 , U2 , U3 , U4 , and U5 . The attribute of the key refers to the relevant information of the key, such as the function of the key, the position of the key, the shape of the key, the size of the key, and the like. In the first feature, the keys of the remote control device may include a plurality of keys, so the corresponding first graph data may include a plurality of first nodes corresponding to each key.

虚拟边是指图数据中连接节点的边。每一个虚拟边连接多个第一节点中的两个，对应的第一图数据中可以包括至少一个虚拟边。虚拟边的属性可以基于连接的两个第一节点之间的关联关系信息确定。连接的两个第一节点之间的关系信息可以包括连接的两个第一节点对应的按键的位置信息，还可以包括连接的两个第一节点对应的按键之间的功能关联关系、形状关联关系等。在一些实施例中，虚拟边的边属性可以基于按键的连接的两个第一节点对应的按键的位置信息确定，例如虚拟边A连接的两个第一节点对应按键a和按键b，虚拟边A的边属性可以包括按键a的位置（xa，ya）、按键b的位置（xb，yb）、按键a与按键b的直线距离、按键a与按键b连线在坐标系中的方向情况等。Virtual edges are edges that connect nodes in the graph data. Each virtual edge connects two of the multiple first nodes, and the corresponding first graph data may include at least one virtual edge. The attribute of the virtual edge may be determined based on the association relationship information between the connected two first nodes. The relationship information between the two connected first nodes may include the position information of the buttons corresponding to the two connected first nodes, and may also include the function association relationship and shape association between the buttons corresponding to the two connected first nodes. relationship, etc. In some embodiments, the edge attribute of the virtual edge may be determined based on the position information of the key corresponding to the two first nodes connected by the key. For example, the two first nodes connected by the virtual edge A correspond to the key a and the key b, and the virtual edge The edge attributes of A can include the position of button a (xa, ya), the position of button b (xb, yb), the linear distance between button a and button b, the direction of the line connecting button a and button b in the coordinate system, etc. .

通过构建的第一图数据，可以更加优秀和完善地表征遥控设备的按键布局情况，例如按键间的相对位置情况，多个按键之间的分布情况等。有助于在确定遥控设备对应的遥控功能数据时更好地考虑按键布局的情况，得到更加准确的结果。By constructing the first map data, the key layout of the remote control device can be better and better characterized, such as the relative position between keys, the distribution between multiple keys, and the like. This helps to better consider the key layout when determining the remote control function data corresponding to the remote control device, and obtain more accurate results.

步骤330，基于所述第一图数据，通过训练好的识别模型确定第一结果，所述第一结果包括得到所述遥控设备对应的遥控功能数据或无所述遥控设备对应的所述遥控功能数据。Step 330, based on the first map data, determine a first result through the trained recognition model, and the first result includes obtaining the remote control function data corresponding to the remote control device or the remote control function corresponding to the remote control device. data.

具体的，该步骤330可以由第一识别模块220执行。Specifically, thisstep 330 may be performed by the first identification module 220 .

在一些实施例中，可以基于第一图数据去确定遥控设备对应的遥控功能数据。具体地，可以通过训练好的识别模型来确定，识别模型的输入可以为第一图数据，根据识别模型的输出，可以确定遥控设备对应的遥控功能数据。In some embodiments, remote control function data corresponding to the remote control device may be determined based on the first map data. Specifically, it can be determined by a trained recognition model, the input of the recognition model can be the first image data, and the remote control function data corresponding to the remote control device can be determined according to the output of the recognition model.

在一些实施例中，识别模型输出的结果可以称为第一结果，第一结果可以包括得到所述遥控设备对应的遥控功能数据或无所述遥控设备对应的所述遥控功能数据。例如，遥控设备1是一个品牌为A，分布有调高、调低、开启和关闭对应的按键，将遥控设备1对应的第一图数据输入识别模型，可以得到遥控设备对应的遥控功能数据为包括调高、调低、开启和关闭功能的a类型遥控功能数据。再例如，遥控设备2是一个通用遥控器，它没有对应的品牌或者它的品牌没有对应的遥控功能数据，将遥控设备2对应的第一图数据输入识别模型，可以得到无所述遥控设备对应的所述遥控功能数据。In some embodiments, the result output by the identification model may be referred to as the first result, and the first result may include obtaining remote control function data corresponding to the remote control device or without the remote control function data corresponding to the remote control device. For example, the remote control device 1 is a brand of A, and there are buttons corresponding to increase, decrease, open and close. Input the first image data corresponding to the remote control device 1 into the recognition model, and the remote control function data corresponding to the remote control device can be obtained as Type a remote control function data including up, down, on and off functions. For another example, the remote control device 2 is a universal remote control, it does not have a corresponding brand or its brand does not have corresponding remote control function data, input the first image data corresponding to the remote control device 2 into the identification model, and it can be obtained that no corresponding remote control device corresponds. of the remote control function data.

在一些实施例中，识别模型可以包括图神经网络模型，例如GNN、R-GCN等常见的图神经网络模型。以R-GCN为例，将前述遥控设备1对应的第一图数据输入R-GCN，R-GCN模型中，第一图数据通过R-GCN的卷积，可以得到第一图数据的向量表示，最后使用softmax激活函数，得到第一图数据对应的预测类别。In some embodiments, the recognition model may include a graph neural network model, such as GNN, R-GCN and other common graph neural network models. Taking R-GCN as an example, the first image data corresponding to the aforementioned remote control device 1 is input into R-GCN. In the R-GCN model, the first image data is convolved by R-GCN, and the vector representation of the first image data can be obtained. , and finally use the softmax activation function to obtain the predicted category corresponding to the first image data.

在一些实施例中，根据得到所述遥控设备对应的遥控功能数据或无所述遥控设备对应的所述遥控功能数据，可以选择执行步骤332或步骤334。In some embodiments, step 332 or step 334 may be selected to be executed according to obtaining remote control function data corresponding to the remote control device or without the remote control function data corresponding to the remote control device.

步骤332，若第一结果为得到所述遥控设备对应的遥控功能数据，则基于第一结果确定所述遥控设备对应的所述遥控功能数据。Step 332: If the first result is to obtain the remote control function data corresponding to the remote control device, determine the remote control function data corresponding to the remote control device based on the first result.

具体的，该步骤332可以由第一确定模块230执行。Specifically, thisstep 332 may be performed by the first determining module 230 .

第一结果为得到遥控设备对应的遥控功能数据时，第一结果中也就包含了确定的遥控功能数据。基于第一结果确定遥控设备对应的遥控功能数据，就可以将第一结果中确定的遥控功能数据作为遥控设备对应的遥控功能数据。When the first result is to obtain remote control function data corresponding to the remote control device, the first result also includes the determined remote control function data. The remote control function data corresponding to the remote control device is determined based on the first result, and the remote control function data determined in the first result may be used as the remote control function data corresponding to the remote control device.

步骤334，若所述第一结果为无所述遥控设备对应的所述遥控功能数据，则：获取与所述遥控设备匹配的被控设备的第二特征，基于所述第二特征和所述第一图数据确定第二图数据，基于所述第二图数据，通过所述训练好的识别模型，得到所述遥控设备对应的所述遥控功能数据。Step 334, if the first result is that there is no remote control function data corresponding to the remote control device, then: obtain the second feature of the controlled device that matches the remote control device, based on the second feature and the The first image data determines the second image data, and based on the second image data, the remote control function data corresponding to the remote control device is obtained through the trained recognition model.

具体的，该步骤334可以由第二确定模块240执行。Specifically, this step 334 may be performed by thesecond determination module 240 .

第一结果为无遥控设备对应的遥控功能数据时，可以再结合遥控设备匹配的被控设备的相关信息去确定对应的遥控功能数据。例如，遥控设备为通用遥控设备，无对应品牌/型号的相应遥控功能数据，识别模型输出为无遥控设备对应的遥控功能数据，可以基于匹配的被控设备的品牌、型号等相关信息和第一特征，去确定对应的遥控功能数据，也可以基于匹配的被控设备的品牌、型号等相关信息和第一图数据，去确定对应的遥控功能数据。When the first result is that there is no remote control function data corresponding to the remote control device, the corresponding remote control function data can be determined by combining the relevant information of the controlled device matched with the remote control device. For example, if the remote control device is a general-purpose remote control device, and there is no corresponding remote control function data corresponding to the brand/model, the identification model output is the remote control function data corresponding to the non-remote control device, which can be based on the matching information such as the brand and model of the controlled device and the first The corresponding remote control function data can also be determined based on the relevant information such as the brand and model of the matched controlled device and the first image data to determine the corresponding remote control function data.

在一些实施例中，可以获取遥控设备匹配的被控设备的第二特征，第二特征是指被控设备的相关信息特征，第二特征可以包括被控设备的品牌特征、型号特征、标识特征、外形特征等与识别被控设备的品牌和/或型号有关的特征。In some embodiments, the second feature of the controlled device matched by the remote control device may be acquired, the second feature refers to the relevant information feature of the controlled device, and the second feature may include the brand feature, model feature, identification feature of the controlled device , shape features and other features related to identifying the brand and/or model of the controlled device.

在一些实施例中，获取到第二特征后，可以基于第二特征和第一特征去确定对应的遥控功能数据。例如，可以建立包括第二特征、和/或第二特征对应的被控设备的品牌和/或型号，以及对应的遥控功能数据类型的数据库，通过查找数据库，实现基于第二特征确定对应的遥控功能数据类型，例如是a类型遥控功能数据或b类型遥控功能数据。基于确定的遥控功能数据类型和第一特征，可以确定遥控设备对应的遥控功能数据。还可以通过训练神经网络模型，得到可以输入遥控设备对应的第一特征和匹配的被控设备对应的第二特征，输出得到遥控设备的遥控功能数据的识别模型来实现。其中，识别模型可以采用逻辑回归模型、分类和回归树、支持向量机、分类和回归树、随机森林或其它用于分类的神经网络模型，本实施例不做限制。In some embodiments, after the second feature is acquired, corresponding remote control function data may be determined based on the second feature and the first feature. For example, a database including the second feature, and/or the brand and/or model of the controlled device corresponding to the second feature, and the corresponding remote control function data type can be established, and by searching the database, it is possible to determine the corresponding remote control based on the second feature. The function data type is, for example, a-type remote control function data or b-type remote control function data. Based on the determined remote control function data type and the first feature, the remote control function data corresponding to the remote control device may be determined. It can also be realized by training the neural network model to obtain the first feature corresponding to the input remote control device and the second feature corresponding to the matched controlled device, and output the identification model of the remote control function data of the remote control device. The identification model may adopt a logistic regression model, a classification and regression tree, a support vector machine, a classification and regression tree, a random forest, or other neural network models for classification, which are not limited in this embodiment.

在一些实施例中，可以基于第二特征和第一图数据确定第二图数据。第二图数据是指可以表征第一图数据和第二特征信息的图数据。可以基于第二图数据去确定遥控设备对应的遥控功能数据。具体地，可以通过训练好的识别模型来确定，识别模型的输入可以为第二图数据，输出得到遥控设备对应的遥控功能数据。例如，遥控设备2是一个通用遥控器，分布有定时、开启和关闭对应的按键，它没有对应的品牌或者它的品牌没有对应的遥控功能数据，对应的被控设备品牌为A，基于遥控设备2对应的第二特征和对应的第一图数据，确定对应的第二图数据，将第二图数据输入识别模型，输出得到遥控设备对应的所述遥控功能数据为包括定时、开启和关闭功能的a类型遥控功能数据。In some embodiments, the second map data may be determined based on the second feature and the first map data. The second graph data refers to graph data that can represent the first graph data and the second feature information. The remote control function data corresponding to the remote control device may be determined based on the second map data. Specifically, it can be determined by a trained recognition model, the input of the recognition model can be the second image data, and the output can obtain remote control function data corresponding to the remote control device. For example, the remote control device 2 is a universal remote control with buttons corresponding to timing, on and off. It does not have a corresponding brand or its brand has no corresponding remote control function data, and the corresponding controlled device brand is A. Based on the remote control device 2 The corresponding second feature and the corresponding first map data, determine the corresponding second map data, input the second map data into the recognition model, and output the remote control function data corresponding to the remote control device to include timing, opening and closing functions. Type a remote control function data.

在一些实施例中，识别模型可以包括图神经网络模型，例如GNN、R-GCN等常见的图神经网络模型。具体地，关于识别模型可以包括图神经网络模型的更多内容可以参见步骤330的相关描述，此处不再赘述。In some embodiments, the recognition model may include a graph neural network model, such as GNN, R-GCN and other common graph neural network models. Specifically, for more content that the recognition model may include a graph neural network model, reference may be made to the relevant description ofstep 330, which will not be repeated here.

在一些实施例中，基于第二特征和第一图数据确定第二图数据可以通过如下方法：确定第二节点，第二节点的第二节点属性基于第二特征确定，根据第二节点和第一图数据确定第二图数据。In some embodiments, determining the second graph data based on the second feature and the first graph data may be performed by the following method: determining the second node, the second node attribute of the second node is determined based on the second feature, and according to the second node and the first graph data One map data determines the second map data.

第二节点是指图数据中用于表征被控设备的相关信息的节点，一个被控设备可以对应至少一个第二节点。如图7中700所示，第二图数据中包括有第二节点I1。第二节点的属性可以包括被控设备的相关信息，可以基于第二特征确定。例如，第二节点的属性可以包括被控设备的品牌特征、型号特征、标识特征、外形特征等与识别被控设备的品牌和/或型号有关的特征。The second node refers to a node in the graph data used to represent relevant information of the controlled device, and one controlled device may correspond to at least one second node. As shown in 700 in FIG. 7 , the second graph data includes a second node I1. The attribute of the second node may include relevant information of the controlled device, and may be determined based on the second characteristic. For example, the attributes of the second node may include brand features, model features, identification features, and shape features of the controlled device that are related to identifying the brand and/or model of the controlled device.

在一些实施例中，根据第二节点和第一图数据确定第二图数据，可以通过在第一图数据中增加第二节点的方式确定得到第二图数据，也可以通过基于第一图数据的第一节点和虚拟边相关信息，再结合第二节点，重新确定得到第二图数据的方式。In some embodiments, the second graph data is determined according to the second node and the first graph data, the second graph data may be determined by adding a second node to the first graph data, or the second graph data may be determined based on the first graph data The first node and virtual edge related information of , and then combined with the second node to re-determine the way to obtain the second graph data.

在一些实施例中，获取第二特征可以通过从图像信息、文本信息、语音信息、视频信息以及包含被控设备的相关信息的二进制数据等其它可用数据中获取的方式。In some embodiments, the second feature may be obtained by obtaining image information, text information, voice information, video information, and other available data such as binary data containing relevant information of the controlled device.

在一些实施例中，获取第二特征可以通过获取与遥控设备匹配的被控设备的第二图像，将第二图像输入特征生成模型，得到第二图像对应的第二特征。In some embodiments, acquiring the second feature may be obtained by acquiring a second image of the controlled device matched with the remote control device, and inputting the second image into the feature generation model to obtain the second feature corresponding to the second image.

包含被控设备相关图像的图像可以称为第二图像。第二图像可以包括被控设备的标识图像、表现出遥控设备形状、尺寸等外形的整体图像等可以体现被控设备品牌和/或型号的图像，以及还可以包括被控设备的应用环境图像等其它被控设备的相关图像。第二图像可以是2D图像、3D图像或其它多维的图像，本实施例不做限制。The image containing the image related to the controlled device may be referred to as the second image. The second image may include a logo image of the controlled device, an overall image showing the shape, size, etc. of the remote control device, etc., an image that may reflect the brand and/or model of the controlled device, and an image of the application environment of the controlled device, etc. Related images of other controlled devices. The second image may be a 2D image, a 3D image, or other multi-dimensional images, which are not limited in this embodiment.

特征生成模型的输入是图像，输出是图像对应的特征向量。具体地，将第二图像输入特征生成模型，输出第二图像对应的特征向量，该特征向量可以作为被控设备对应的第二特征。在一些实施例中，特征生成模型可以采用神经网络模型，例如CNN或VGG等常用的卷积神经网络模型。以VGG为例，将第二图像输入VGG中，通过卷积层、池化层、全连接层，可以输出得到第二图像对应的特征向量。The input of the feature generation model is an image, and the output is the feature vector corresponding to the image. Specifically, the second image is input into the feature generation model, and a feature vector corresponding to the second image is output, and the feature vector can be used as the second feature corresponding to the controlled device. In some embodiments, the feature generation model may use a neural network model, such as a common convolutional neural network model such as CNN or VGG. Taking VGG as an example, the second image is input into VGG, and the feature vector corresponding to the second image can be output through the convolution layer, pooling layer, and fully connected layer.

在一些实施例中，获取第二特征可以通过获取用户输入的音频信息和/或文本信息，音频信息和所述文本信息包括被控设备的设备信息，根据音频信息和/或文本信息可以得到被控设备的第二特征。例如，用户可以语音输入或文字输入被控设备的设备信息，可以包括品牌特征、型号特征、标识特征、外形特征等被控设备相关信息，对语音进行音频信息读取或对文字内容进行文本信息读取，可以得到被控设备的第二特征。In some embodiments, acquiring the second feature may be obtained by acquiring audio information and/or text information input by the user, the audio information and the text information include device information of the controlled device, and the controlled device may be obtained according to the audio information and/or text information. the second feature of the control device. For example, the user can input the device information of the controlled device by voice input or text, which can include the relevant information of the controlled device such as brand features, model features, identification features, and appearance features, read audio information for voice or text information for text content By reading, the second characteristic of the controlled device can be obtained.

在一些实施例中，在得到遥控设备对应的遥控功能数据后，可以对各个遥控设备对应的遥控功能数据进行存储，以便于遥控设备和其它智能设备在实现对各个被控设备的控制时，调用对应的遥控功能数据。在一些实施例中，可以通过以下步骤340和步骤342的方法实现。在一些实施例中，步骤340和步骤342的方法可以通过存储模块250执行。In some embodiments, after the remote control function data corresponding to the remote control device is obtained, the remote control function data corresponding to each remote control device can be stored, so that the remote control device and other intelligent devices can call the remote control device and other intelligent devices when controlling each controlled device. Corresponding remote control function data. In some embodiments, it can be implemented by the following methods of steps 340 and 342 . In some embodiments, the methods of steps 340 and 342 may be performed by thestorage module 250 .

步骤340、对所述遥控设备对应的所述遥控功能数据添加标识并存储，所述标识与所述遥控设备匹配的被控设备相关，得到包括至少一个所述遥控设备对应的所述遥控功能数据的遥控功能数据库。Step 340: Add and store an identifier to the remote control function data corresponding to the remote control device, the identifier is related to the controlled device matched with the remote control device, and obtain the remote control function data corresponding to at least one of the remote control devices. database of remote control functions.

在一些实施例中，在得到遥控设备对应的遥控功能数据后，可以对各个遥控设备对应的遥控功能数据添加标识并存储。具体地，可以存储在遥控设备和其它智能设备的内存中，或服务器中，或云平台等其它存储空间中，本实施例不做限制。In some embodiments, after the remote control function data corresponding to the remote control device is obtained, the remote control function data corresponding to each remote control device may be identified and stored. Specifically, it may be stored in the memory of the remote control device and other smart devices, or in a server, or in other storage spaces such as a cloud platform, which is not limited in this embodiment.

标识是指用于标注各个遥控功能数据的标签或数据。标识可以基于遥控设备匹配的被控设备的相关设备信息确定。例如，被控设备为空调，则其匹配的遥控设备的遥控功能数据的标识可以为空调，如果空调在卧室，则其匹配的遥控设备的遥控功能数据的标识可以是卧室空调。Identifiers refer to labels or data used to label individual remote control function data. The identification may be determined based on the relevant device information of the controlled device matched by the remote control device. For example, if the controlled device is an air conditioner, the identifier of the remote control function data of the matched remote control device may be the air conditioner. If the air conditioner is in the bedroom, the identifier of the remote control function data of the matched remote control device may be the bedroom air conditioner.

通过对各个遥控设备对应的遥控功能数据添加标识并存储，可以得到一个对应的遥控功能数据库。A corresponding remote control function database can be obtained by adding an identifier to and storing the remote control function data corresponding to each remote control device.

步骤342、所述智能设备在所述遥控功能数据库中查找与目标被控设备相关的标识，并调用所述目标被控设备相关的标识对应的所述遥控功能数据，实现对所述目标被控设备的控制。Step 342: The intelligent device searches the remote control function database for the identifier related to the target controlled device, and calls the remote control function data corresponding to the identifier related to the target controlled device, so as to realize the control of the target controlled device. device control.

在一些实施例中，遥控设备和其它智能设备在实现对各个被控设备的控制时，可以通过遥控功能数据库去调用对应的遥控功能数据，来进行控制被控设备。具体地，可以通过在遥控功能数据库中查找与目标被控设备的标识，调用该标识对应的遥控功能数据来实现。目标被控设备是指需要被控制的被控设备。例如，在遥控功能数据库中存储有标识为卧室空调的遥控功能数据、标识为客厅音响的遥控功能数据、标识为冰箱的遥控功能数据，目标被控设备是卧室空调，则在遥控功能数据库中查找卧室空调的标识，并调用对应的遥控功能数据实现对卧室空调的控制。In some embodiments, when the remote control device and other smart devices control each controlled device, the corresponding remote control function data can be called through the remote control function database to control the controlled device. Specifically, it can be realized by searching for the identifier of the target controlled device in the remote control function database, and calling the remote control function data corresponding to the identifier. The target controlled device refers to the controlled device that needs to be controlled. For example, in the remote control function database, the remote control function data identified as a bedroom air conditioner, the remote control function data identified as a living room audio, and the remote control function data identified as a refrigerator are stored, and the target controlled device is a bedroom air conditioner. The identification of the bedroom air conditioner, and call the corresponding remote control function data to realize the control of the bedroom air conditioner.

在一些实施例中，得到遥控设备对应的遥控功能数据后，还可以采用遥控功能数据对匹配的被控设备进行测试，以验证遥控功能数据的正确与否。在一些实施例中，可以通过步骤344、步骤346和步骤348的方法实现。在一些实施例中，步骤344、步骤346和步骤348的方法可以由测试模块270执行。In some embodiments, after obtaining the remote control function data corresponding to the remote control device, the matched controlled device may be tested by using the remote control function data to verify the correctness of the remote control function data. In some embodiments, the method of step 344 , step 346 and step 348 may be used. In some embodiments, the methods of steps 344 , 346 and 348 may be performed by thetesting module 270 .

步骤344、采用所述遥控功能数据对与对应的所述遥控设备匹配的被控设备进行测试。Step 344: Use the remote control function data to test the controlled device that matches the corresponding remote control device.

测试是指对被控设备进行控制测验。具体地，采用遥控功能数据对遥控设备匹配的被控设备进行测验，是指遥控设备或其它智能设备调用遥控功能数据，可以通过红外、无线、语音等控制方式对匹配的被控设备进行控制，测试遥控功能数据对被控设备是否正确反应，未正确反应则测试失败，正确反应则测试成功。例如，调用遥控功能数据控制空调开启，若空调未开启，则未正确反应，测试失败，若空调开启，则正确反应，测试成功。The test refers to the control test of the controlled equipment. Specifically, using the remote control function data to test the controlled equipment matched by the remote control equipment means that the remote control equipment or other intelligent equipment calls the remote control function data, and the matched controlled equipment can be controlled by infrared, wireless, voice and other control methods. Test whether the remote control function data responds correctly to the controlled device. If it does not respond correctly, the test fails, and if it responds correctly, the test succeeds. For example, call the remote control function data to control the air conditioner to turn on. If the air conditioner is not turned on, it will not respond correctly and the test will fail. If the air conditioner is turned on, it will respond correctly and the test will be successful.

步骤346、若测试成功，则所述遥控功能数据有效。Step 346: If the test is successful, the remote control function data is valid.

在一些实施例中，当测试成功时，可以将对应的遥控功能数据认为是有效的数据。即，使用该遥控功能数据可以实现对匹配的被控设备的有效控制。In some embodiments, when the test is successful, the corresponding remote control function data may be considered valid data. That is, effective control of the matched controlled device can be realized using the remote control function data.

步骤348、若测试失败，则获取所述被控设备对应的品牌列表和/或型号列表，基于所述品牌列表和/或所述型号列表选择所述遥控设备匹配的被控设备的品牌和/或型号；基于选择的所述品牌和/或所述型号和所述第一特征，重新确定所述遥控设备对应的所述遥控功能数据。Step 348, if the test fails, then obtain the brand list and/or model list corresponding to the controlled device, and select the brand and/or brand of the controlled device matched by the remote control device based on the brand list and/or the model list. or model; based on the selected brand and/or model and the first feature, re-determine the remote control function data corresponding to the remote control device.

在一些实施例中，当测试失败时，原确定的遥控功能数据即是错误的，可以重新确定遥控设备对应的遥控功能数据。In some embodiments, when the test fails, the originally determined remote control function data is wrong, and the remote control function data corresponding to the remote control device can be re-determined.

品牌列表和/或型号列表中包含了各个被控设备对应的品牌和/或型号。获取品牌列表和/或型号列表可以通过从保存有品牌列表和/或型号列表的数据库中获取，或输入各个被控设备的相关信息等方式来得到，本实施例对此不做限制。The brand list and/or model list contains the brand and/or model corresponding to each controlled device. The brand list and/or model list can be obtained by obtaining from a database storing the brand list and/or model list, or by inputting relevant information of each controlled device, which is not limited in this embodiment.

从品牌列表和/或型号列表中选择对应的品牌和/或型号可以是用户或者智能机器人等智能设备来实现，本实施例对此不做限制。Selecting a corresponding brand and/or model from the brand list and/or model list may be implemented by a user or an intelligent device such as an intelligent robot, which is not limited in this embodiment.

选择的得到被控设备的品牌和/或型号后，可以基于品牌和/或型号，以及第一特征来重新确定遥控设备对应的遥控功能数据。例如，可以建立包括被控设备的品牌和/或型号，以及对应的遥控功能数据类型的数据库，通过查找数据库，实现基于被控设备的品牌和/或型号确定对应的遥控功能数据类型，例如是a类型遥控功能数据或b类型遥控功能数据。基于确定的遥控功能数据类型和第一特征，可以确定遥控设备对应的遥控功能数据。还可以通过训练神经网络模型，得到可以输入遥控设备对应的第一特征和匹配的被控设备对应的品牌和/或型号特征，输出得到遥控设备的遥控功能数据的识别模型来实现。其中，识别模型可以采用逻辑回归模型、分类和回归树、支持向量机、分类和回归树、随机森林或其它用于分类的神经网络模型，本实施例不做限制。After the brand and/or model of the controlled device is selected, the remote control function data corresponding to the remote control device may be re-determined based on the brand and/or model and the first feature. For example, a database including the brand and/or model of the controlled device and the corresponding remote control function data type can be established, and by searching the database, the corresponding remote control function data type can be determined based on the brand and/or model of the controlled device, such as Type a remote control function data or type b remote control function data. Based on the determined remote control function data type and the first feature, the remote control function data corresponding to the remote control device may be determined. It can also be realized by training the neural network model to obtain the first feature corresponding to the input remote control device and the brand and/or model features corresponding to the matched controlled device, and output the identification model of the remote control function data of the remote control device. The identification model may adopt a logistic regression model, a classification and regression tree, a support vector machine, a classification and regression tree, a random forest, or other neural network models for classification, which are not limited in this embodiment.

在一些实施例中，可以根据重新确定的遥控设备对应的遥控功能数据，对原识别模型进行优化，提高识别模型预测的准确性。在一些实施例中，可以通过步骤352中的方法实现。In some embodiments, the original recognition model may be optimized according to the re-determined remote control function data corresponding to the remote control device, so as to improve the prediction accuracy of the recognition model. In some embodiments, this can be achieved by the method in step 352 .

步骤352、根据测试失败对应的所述遥控功能数据与重新确定的所述遥控设备对应的所述遥控功能数据之间的差异构建损失函数，基于所述损失函数更新所述识别模型的参数。Step 352: Construct a loss function according to the difference between the remote control function data corresponding to the test failure and the re-determined remote control function data corresponding to the remote control device, and update the parameters of the identification model based on the loss function.

在一些实施例中，步骤352可以通过测试模块270执行。In some embodiments, step 352 may be performed bytesting module 270 .

在一些实施例中，基于测试失败对应的遥控功能数据与重新确定的遥控设备对应的遥控功能数据之间的差异构建损失函数，基于该损失函数去更新识别模型的参数。可以是，对识别模型继续进行训练，并以测试失败对应的遥控功能数据与重新确定的遥控设备对应的遥控功能数据之间的差异为损失函数，以最小化损失函数为目标，更新识别模型的参数。In some embodiments, a loss function is constructed based on the difference between the remote control function data corresponding to the test failure and the remote control function data corresponding to the re-determined remote control device, and the parameters of the recognition model are updated based on the loss function. It can be that the recognition model continues to be trained, and the difference between the remote control function data corresponding to the test failure and the remote control function data corresponding to the re-determined remote control device is used as the loss function, and the goal of minimizing the loss function is to update the recognition model. parameter.

步骤340-352在图3中未示出。Steps 340-352 are not shown in FIG. 3 .

图4是根据本说明书的一些实施例所示的识别模型训练的方法的示例性流程图。FIG. 4 is an exemplary flowchart of a method for training a recognition model according to some embodiments of the present specification.

如图4所示，该方法400可以包括：As shown in Figure 4, themethod 400 may include:

步骤410，获取至少一个所述遥控设备对应的至少一个第一图数据样本，以及所述至少一个第一图数据样本对应的第一标签，所述第一标签包括所述第一图数据样本对应的所述遥控功能数据或所述第一图数据样本无对应的所述遥控功能数据。Step 410: Obtain at least one first image data sample corresponding to at least one of the remote control devices, and a first label corresponding to the at least one first image data sample, where the first label includes the corresponding first image data sample. The remote control function data or the first image data sample has no corresponding remote control function data.

具体的，该步骤410可以由识别模型训练模块260执行。Specifically, thisstep 410 may be performed by the recognitionmodel training module 260 .

第一图数据样本是指第一图数据对应的样本数据，可以基于遥控设备样本获取得到，或者从遥控设备对应的图数据库中获取得到。至少一个第一图数据样本可以包括至少一个遥控设备样本的第一图数据样本，每一个遥控设备样本可以对应有至少一个第一图数据样本。The first image data sample refers to sample data corresponding to the first image data, which may be obtained based on a sample of the remote control device, or obtained from a map database corresponding to the remote control device. The at least one first image data sample may include a first image data sample of at least one remote control device sample, and each remote control device sample may correspond to at least one first image data sample.

标签是指对样本数据的标注。在一些实施例中，标签的获取可以通过人为地对样本数据进行标记得到，也可以通过其他方式获取，例如基于已知标识结果的历史样本数据，本实施例对此不做限制。The label refers to the labeling of the sample data. In some embodiments, the label may be obtained by manually marking the sample data, or may be obtained by other methods, such as historical sample data based on known identification results, which is not limited in this embodiment.

第一图数据样本的标注可以称为第一标签。第一标签可以是第一图数据样本对应的遥控功能数据，或者是第一图数据样本无对应的所述遥控功能数据。例如，对于遥控设备样本为遥控设备1，其品牌为A，分布有调高、调低、开启和关闭对应的按键，遥控设备1对应的第一图数据样本的第一标签为包括调高、调低、开启和关闭功能的a类型遥控功能数据。再例如，对于遥控设备样本为遥控设备2，其是一个通用遥控器，它没有对应的品牌或者它的品牌没有对应的遥控功能数据，遥控设备2对应的第一图数据样本的第一标签为无所述遥控设备对应的所述遥控功能数据。The labeling of the first graph data sample may be referred to as a first label. The first label may be remote control function data corresponding to the first image data sample, or the remote control function data not corresponding to the first image data sample. For example, if the remote control device sample is remote control device 1, its brand is A, and buttons corresponding to increase, decrease, open and close are distributed, and the first label of the first image data sample corresponding to remote control device 1 is composed of Type a remote control function data for down, on and off functions. For another example, for the remote control device sample is remote control device 2, which is a universal remote control, it does not have a corresponding brand or its brand does not have corresponding remote control function data, the first label of the first image data sample corresponding to the remote control device 2 is There is no remote control function data corresponding to the remote control device.

步骤420，获取至少一个与所述遥控设备匹配的被控设备对应的至少一个第二特征样本，基于所述至少一个第二特征样本和所述至少一个第一图数据样本得到至少一个第二图数据样本，以及获取所述至少一个第二图数据样本对应的第二标签，所述第二标签包括所述第二图数据样本对应的所述遥控功能数据。Step 420: Obtain at least one second feature sample corresponding to at least one controlled device matched with the remote control device, and obtain at least one second map based on the at least one second feature sample and the at least one first map data sample. data samples, and acquiring a second label corresponding to the at least one second image data sample, where the second label includes the remote control function data corresponding to the second image data sample.

具体的，该步骤420可以由识别模型训练模块260执行。Specifically, this step 420 may be performed by the recognitionmodel training module 260 .

第二特征样本是指第二特征对应的样本数据，可以基于被控设备样本获取或者从被控设备对应的特征数据库中获取得到。至少一个第二特征样本可以包括至少一个被控设备样本的第二特征样本，每一个被控设备样本可以对应有至少一个第二特征样本。The second feature sample refers to sample data corresponding to the second feature, which may be obtained based on a sample of the controlled device or obtained from a feature database corresponding to the controlled device. The at least one second feature sample may include at least one second feature sample of the controlled device sample, and each controlled device sample may correspond to at least one second feature sample.

第二图数据样本是指第二图数据对应的样本数据，可以基于第二特征样本和第一图数据样本得到。至少一个第二特征样本和至少一个第一图数据样本，可以对应得到至少一个第二图数据样本。The second image data sample refers to sample data corresponding to the second image data, and can be obtained based on the second feature sample and the first image data sample. At least one second feature sample and at least one first image data sample can correspondingly obtain at least one second image data sample.

第二图数据样本的标注可以称为第二标签。第二标签可以是第二图数据样本对应的遥控功能数据。例如，遥控设备2是一个通用遥控器，分布有定时、开启和关闭对应的按键，它没有对应的品牌或者它的品牌没有对应的遥控功能数据，对应的被控设备品牌为A，基于遥控设备2对应的第二特征样本和对应的第一图数据样本，确定的第二图数据样本的标签为包括定时、开启和关闭功能的a类型遥控功能数据。The labeling of the second graph data sample may be referred to as a second label. The second label may be remote control function data corresponding to the second image data sample. For example, the remote control device 2 is a universal remote control with buttons corresponding to timing, on and off. It does not have a corresponding brand or its brand has no corresponding remote control function data, and the corresponding controlled device brand is A. Based on the remote control device 2. The corresponding second feature sample and the corresponding first image data sample, and the determined label of the second image data sample is type a remote control function data including timing, on and off functions.

步骤430，以带有所述第一标签的所述至少一个第一图数据样本和带有所述第二标签的所述至少一个第二图数据样本作为训练样本，对初始识别模型进行训练，得到所述训练好的识别模型。Step 430, using the at least one first image data sample with the first label and the at least one second image data sample with the second label as training samples to train the initial recognition model, The trained recognition model is obtained.

具体的，该步骤430可以由识别模型训练模块260执行。Specifically, thisstep 430 may be performed by the recognitionmodel training module 260 .

初始识别模型是指未经训练的原始识别模型，初始识别模型的模型参数还处于初始化的状态。初始识别模型训练好后，即为识别模型。The initial recognition model refers to an untrained original recognition model, and the model parameters of the initial recognition model are still in an initialized state. After the initial recognition model is trained, it is the recognition model.

识别模型可以包括图神经网络模型，关于识别模型的更多内容可以参见图3中步骤330的相关描述，此处不再赘述。The recognition model may include a graph neural network model. For more details about the recognition model, reference may be made to the relevant description ofstep 330 in FIG. 3 , which will not be repeated here.

以带有第一标签的至少一个第一图数据样本和带有第二标签的至少一个第二图数据样本作为训练样本，对初始识别模型进行训练，是指分别将带有第一标签的至少一个第一图数据样本和带有第二标签的至少一个第二图数据样本作为训练样本输入初始识别模型中，对初始识别模型进行训练更新模型参数。Using at least one first image data sample with a first label and at least one second image data sample with a second label as training samples, training the initial recognition model means that at least one image with the first label is respectively used. One first image data sample and at least one second image data sample with a second label are input into the initial recognition model as training samples, and the initial recognition model is trained to update model parameters.

在一些实施例中，识别模型可以基于训练样本，通过常用的方法进行训练，更新模型参数。例如，可以基于梯度下降法进行训练。在一些实施例中，当训练得到的识别模型满足预设条件时，训练结束。其中，预设条件可以是损失函数结果收敛或小于预设阈值等。In some embodiments, the recognition model can be trained based on the training samples by a common method, and the model parameters can be updated. For example, training can be based on gradient descent. In some embodiments, when the recognition model obtained by training satisfies the preset condition, the training ends. The preset condition may be that the result of the loss function converges or is smaller than a preset threshold, or the like.

图5是根据本说明书的一些实施例所示的另一种根据遥控设备图像配置遥控功能数据的方法的示例性流程图。FIG. 5 is an exemplary flowchart of another method for configuring remote control function data according to a remote control device image according to some embodiments of the present specification.

如图5所示，该方法500可以包括：As shown in Figure 5, themethod 500 may include:

步骤510，获取遥控设备的第一图像，并基于所述第一图像确定第一特征，所述第一特征包括所述遥控设备的按键布局特征。Step 510: Acquire a first image of the remote control device, and determine a first feature based on the first image, where the first feature includes a key layout feature of the remote control device.

具体的，该步骤510可以由第二获取模块执行。Specifically, thisstep 510 may be performed by the second obtaining module.

关于遥控功能数据、遥控设备、遥控设备的第一图像、第一特征、遥控设备的按键布局特征以及相应的获取和确定方法可以参见图3步骤310中的相关描述，此处不再赘述。For the remote control function data, the remote control device, the first image of the remote control device, the first feature, the key layout feature of the remote control device, and the corresponding acquisition and determination method, reference may be made to the relevant description instep 310 in FIG.

步骤520，基于所述第一特征，通过训练好的识别模型确定所述遥控设备对应的遥控功能数据。Step 520: Based on the first feature, determine the remote control function data corresponding to the remote control device by using the trained recognition model.

具体的，该步骤520可以由第二识别模块执行。Specifically, thisstep 520 may be performed by the second identification module.

确定得到第一特征后，可以基于第一特征确定遥控设备对应的遥控功能数据。在一些实施例中，可以通过训练好的识别模型来基于第一特征确定遥控设备对应的遥控功能数据。关于基于第一特征，通过识别模型来确定遥控设备对应的遥控功能数据的更多具体内容可以参见图3步骤310中的相关描述，此处不再赘述。After it is determined that the first feature is obtained, remote control function data corresponding to the remote control device may be determined based on the first feature. In some embodiments, the remote control function data corresponding to the remote control device may be determined based on the first feature through a trained recognition model. For more specific content of determining the remote control function data corresponding to the remote control device by identifying the model based on the first feature, reference may be made to the relevant description instep 310 in FIG. 3 , which will not be repeated here.

步骤530，基于所述遥控功能数据确定与所述遥控设备匹配的被控设备。Step 530: Determine a controlled device matching the remote control device based on the remote control function data.

具体的，该步骤530可以由第三确定模块执行。Specifically, thisstep 530 may be performed by a third determining module.

在一些实施例中，当对多个被控设备如冰箱、电视、空调等电器的遥控设备统一进行识别确定对应的遥控功能数据时，并不知道哪个遥控设备对应哪个被控设备。确定得到遥控设备对应的遥控功能数据后，可以基于该遥控功能数据确定与遥控设备匹配的被控设备。In some embodiments, when the remote control devices of multiple controlled devices such as refrigerators, televisions, air conditioners and other electrical appliances are uniformly identified to determine corresponding remote control function data, it is not known which remote control device corresponds to which controlled device. After it is determined that the remote control function data corresponding to the remote control device is obtained, the controlled device that matches the remote control device can be determined based on the remote control function data.

在一些实施例中，基于遥控功能数据确定与遥控设备匹配的被控设备可以通过建立包括遥控功能数据类型、以及与其对应的被控设备信息的数据库来实现。基于确定的遥控功能数据可以知道对应的遥控功能数据类型，通过查找数据库，可以确定于遥控功能数据对应的被控设备信息，即可以确定与遥控设备匹配的被控设备。In some embodiments, determining the controlled device matching the remote control device based on the remote control function data may be implemented by establishing a database including the type of the remote control function data and the information of the controlled device corresponding thereto. Based on the determined remote control function data, the corresponding remote control function data type can be known. By searching the database, the controlled device information corresponding to the remote control function data can be determined, that is, the controlled device matching the remote control device can be determined.

在一些实施例中，基于遥控功能数据确定与遥控设备匹配的被控设备可以通过训练一个被控设备识别模型来实现。被控设备识别模型的输入为遥控功能数据，输出为对应的被控设备。被控设备识别模型可以采用逻辑回归模型、分类和回归树、支持向量机、分类和回归树、随机森林或其它用于分类的神经网络模型，本实施例不做限制。In some embodiments, determining the controlled device matching the remote control device based on the remote control function data may be implemented by training a controlled device identification model. The input of the controlled device identification model is the remote control function data, and the output is the corresponding controlled device. The controlled device identification model may adopt a logistic regression model, a classification and regression tree, a support vector machine, a classification and regression tree, a random forest, or other neural network models for classification, which are not limited in this embodiment.

图6是根据本说明书的一些实施例所示的另一种识别模型训练的方法的示例性流程图。FIG. 6 is an exemplary flowchart of another method for training a recognition model according to some embodiments of the present specification.

如图6所示，该方法600可以包括：As shown in Figure 6, themethod 600 may include:

步骤610，获取至少一个所述遥控设备对应的至少一个第一特征样本，以及所述至少一个第一特征样本对应的样本标签，所述样本标签包括所述第一特征样本对应的所述遥控功能数据。Step 610: Obtain at least one first feature sample corresponding to at least one of the remote control devices, and a sample label corresponding to the at least one first feature sample, where the sample label includes the remote control function corresponding to the first feature sample data.

具体的，该步骤610可以由第二识别模型训练模块执行。Specifically, thisstep 610 may be performed by the second recognition model training module.

第一特征样本是指第一特征对应的样本数据，可以基于遥控设备样本获取得到，或者从遥控设备对应的第一特征库中获取得到。至少一个第一特征样本可以包括至少一个遥控设备样本的第一特征样本，每一个遥控设备样本可以对应有至少一个第一特征样本。The first feature sample refers to sample data corresponding to the first feature, which may be obtained based on a sample of a remote control device, or obtained from a first feature library corresponding to the remote control device. The at least one first feature sample may include a first feature sample of at least one remote control device sample, and each remote control device sample may correspond to at least one first feature sample.

第一特征样本的标注可以称为样本标签。样本标签可以是第一特征样本对应的遥控功能数据。例如，对于遥控设备样本为遥控设备1，其品牌为A，分布有调高、调低、开启和关闭对应的按键，遥控设备1对应的第一特征样本的样本标签为包括调高、调低、开启和关闭功能的a类型遥控功能数据。The labeling of the first feature sample may be referred to as a sample label. The sample label may be remote control function data corresponding to the first feature sample. For example, if the remote control device sample is remote control device 1, its brand is A, and buttons corresponding to increase, decrease, open and close are distributed. , Type a remote control function data for on and off functions.

步骤620，以带有所述样本标签的所述至少一个第一特征样本对初始识别模型进行训练，得到所述训练好的识别模型。Step 620: Train the initial recognition model with the at least one first feature sample with the sample label to obtain the trained recognition model.

具体的，该步骤620可以由第二识别模型训练模块执行。Specifically, thisstep 620 may be performed by the second recognition model training module.

以带有样本标签的至少一个第一特征样本作为训练样本，对初始识别模型进行训练，是指分别将带有样本标签的至少一个第一特征样本输入初始识别模型中，对初始识别模型进行训练更新模型参数。Using at least one first feature sample with a sample label as a training sample to train the initial recognition model refers to respectively inputting at least one first feature sample with a sample label into the initial recognition model, and training the initial recognition model Update model parameters.

在一些实施例中，初始识别模型可以基于训练样本，通过常用的方法进行训练，更新模型参数。例如，可以基于梯度下降法进行训练。在一些实施例中，当训练得到的识别模型满足预设条件时，训练结束。其中，预设条件可以是损失函数结果收敛或小于预设阈值等。In some embodiments, the initial recognition model can be trained based on the training samples by a common method, and the model parameters can be updated. For example, training can be based on gradient descent. In some embodiments, when the recognition model obtained by training satisfies the preset condition, the training ends. The preset condition may be that the result of the loss function converges or is smaller than a preset threshold, or the like.

本说明书实施例还提供一种装置，包括处理器，所述处理器用于执行前述的根据遥控设备图像配置遥控功能数据的方法。所述的方法可以包括：获取遥控设备的第一图像，并基于所述第一图像中确定第一特征，所述第一特征包括所述遥控设备的按键布局特征；基于所述第一特征确定所述遥控设备对应的第一图数据，所述第一图数据包括与多个按键对应的多个第一节点、以及至少一个虚拟边；每个所述虚拟边连接所述多个第一节点中的两个，所述第一节点的第一节点属性基于对应的所述按键的属性确定，所述虚拟边的边属性基于所述虚拟边连接的所述第一节点对应的所述按键的位置信息确定；基于所述第一图数据，通过训练好的识别模型确定第一结果，所述第一结果包括得到所述遥控设备对应的遥控功能数据或无所述遥控设备对应的所述遥控功能数据；若第一结果为得到所述遥控设备对应的遥控功能数据，则基于第一结果确定所述遥控设备对应的所述遥控功能数据；若所述第一结果为无所述遥控设备对应的所述遥控功能数据，则：获取与所述遥控设备匹配的被控设备的第二特征，基于所述第二特征和所述第一图数据确定第二图数据，基于所述第二图数据，通过所述训练好的识别模型，得到所述遥控设备对应的所述遥控功能数据。或者所述的方法可以包括：获取遥控设备的第一图像，并基于所述第一图像中确定第一特征，所述第一特征包括所述遥控设备的按键布局特征；基于所述第一特征，通过训练好的识别模型确定所述遥控设备对应的遥控功能数据；基于所述遥控功能数据确定与所述遥控设备匹配的被控设备。An embodiment of the present specification further provides an apparatus, including a processor, where the processor is configured to execute the aforementioned method for configuring remote control function data according to an image of a remote control device. The method may include: acquiring a first image of a remote control device, and determining a first feature based on the first image, where the first feature includes a key layout feature of the remote control device; determining based on the first feature First graph data corresponding to the remote control device, the first graph data includes a plurality of first nodes corresponding to a plurality of buttons, and at least one virtual edge; each of the virtual edges connects the plurality of first nodes Two of the first node attributes of the first node are determined based on the attributes of the corresponding keys, and the edge attributes of the virtual edges are based on the attributes of the keys corresponding to the first nodes connected by the virtual edges. The location information is determined; based on the first map data, a first result is determined through the trained recognition model, and the first result includes obtaining the remote control function data corresponding to the remote control device or the remote control without the remote control device. function data; if the first result is to obtain the remote control function data corresponding to the remote control device, then determine the remote control function data corresponding to the remote control device based on the first result; if the first result is that there is no corresponding remote control device the remote control function data, then: acquire the second feature of the controlled device matched with the remote control device, determine the second map data based on the second feature and the first map data, and determine the second map data based on the second map data. data, and obtain the remote control function data corresponding to the remote control device through the trained recognition model. Or the method may include: acquiring a first image of a remote control device, and determining a first feature based on the first image, where the first feature includes a key layout feature of the remote control device; based on the first feature , determine the remote control function data corresponding to the remote control device through the trained recognition model; determine the controlled device that matches the remote control device based on the remote control function data.

本说明书实施例还提供一种计算机可读存储介质，所述存储介质存储计算机指令，当计算机读取存储介质中的计算机指令后，计算机执行前述的根据遥控设备图像配置遥控功能数据的方法。所述的方法可以包括：获取遥控设备的第一图像，并基于所述第一图像中确定第一特征，所述第一特征包括所述遥控设备的按键布局特征；基于所述第一特征确定所述遥控设备对应的第一图数据，所述第一图数据包括与多个按键对应的多个第一节点、以及至少一个虚拟边；每个所述虚拟边连接所述多个第一节点中的两个，所述第一节点的第一节点属性基于对应的所述按键的属性确定，所述虚拟边的边属性基于所述虚拟边连接的所述第一节点对应的所述按键的位置信息确定；基于所述第一图数据，通过训练好的识别模型确定第一结果，所述第一结果包括得到所述遥控设备对应的遥控功能数据或无所述遥控设备对应的所述遥控功能数据；若第一结果为得到所述遥控设备对应的遥控功能数据，则基于第一结果确定所述遥控设备对应的所述遥控功能数据；若所述第一结果为无所述遥控设备对应的所述遥控功能数据，则：获取与所述遥控设备匹配的被控设备的第二特征，基于所述第二特征和所述第一图数据确定第二图数据，基于所述第二图数据，通过所述训练好的识别模型，得到所述遥控设备对应的所述遥控功能数据。或者所述的方法可以包括：获取遥控设备的第一图像，并基于所述第一图像中确定第一特征，所述第一特征包括所述遥控设备的按键布局特征；基于所述第一特征，通过训练好的识别模型确定所述遥控设备对应的遥控功能数据；基于所述遥控功能数据确定与所述遥控设备匹配的被控设备。Embodiments of this specification also provide a computer-readable storage medium, which stores computer instructions, and after the computer reads the computer instructions in the storage medium, the computer executes the aforementioned method for configuring remote control function data according to a remote control device image. The method may include: acquiring a first image of a remote control device, and determining a first feature based on the first image, where the first feature includes a key layout feature of the remote control device; determining based on the first feature First graph data corresponding to the remote control device, the first graph data includes a plurality of first nodes corresponding to a plurality of buttons, and at least one virtual edge; each of the virtual edges connects the plurality of first nodes Two of the first node attributes of the first node are determined based on the attributes of the corresponding keys, and the edge attributes of the virtual edges are based on the attributes of the keys corresponding to the first nodes connected by the virtual edges. The location information is determined; based on the first map data, a first result is determined through the trained recognition model, and the first result includes obtaining the remote control function data corresponding to the remote control device or the remote control without the remote control device. function data; if the first result is to obtain the remote control function data corresponding to the remote control device, then determine the remote control function data corresponding to the remote control device based on the first result; if the first result is that there is no corresponding remote control device the remote control function data, then: acquire the second feature of the controlled device matched with the remote control device, determine the second map data based on the second feature and the first map data, and determine the second map data based on the second map data. data, and obtain the remote control function data corresponding to the remote control device through the trained recognition model. Or the method may include: acquiring a first image of a remote control device, and determining a first feature based on the first image, where the first feature includes a key layout feature of the remote control device; based on the first feature , determine the remote control function data corresponding to the remote control device through the trained recognition model; determine the controlled device that matches the remote control device based on the remote control function data.

本说明书实施例可能带来的有益效果包括但不限于：（1）通过识别遥控设备的图像，根据按键特征布局确定对应的遥控功能数据，不需通过查找品牌/或型号后再进行配置，实现更加方便和准确地配置遥控功能数据，提升用户体验；（2）基于遥控设备的图像特征构建图数据，将遥控设备的按键特征和按键间布局信息体现得更加完善，通过图数据识别的方式，实现更加准确地预测；（3）在构建图数据时可以结合被控设备的特征，实现在无法根据遥控设备确定遥控功能数据，可以结合被控设备特征确定，预测更加准确；（4）可以直接通过被控设备的图像、或用户的语音输入、文本输入等得到被控设备的特征信息，实现更加方便地确定对应的遥控功能数据；（5）对确定的遥控功能数据添加标识并存储后，可以直接调用对应被控设备的遥控功能数据来实现控制，控制更加方便高效，提升了用户体验；（6）对确定的遥控功能数据进行测试，可以确保配置的遥控功能数据的准确性，避免错误配置；（7）在确定的遥控功能数据为无效时，基于重新确定的结果更新识别模型的参数，可以提高模型的识别准确率。需要说明的是，不同实施例可能产生的有益效果不同，在不同的实施例里，可能产生的有益效果可以是以上任意一种或几种的组合，也可以是其他任何可能获得的有益效果。The possible beneficial effects of the embodiments of this specification include, but are not limited to: (1) By identifying the image of the remote control device and determining the corresponding remote control function data according to the key feature layout, it is not necessary to search for the brand/or model and then configure it to achieve It is more convenient and accurate to configure the remote control function data to improve the user experience; (2) Construct map data based on the image features of the remote control device, and reflect the key features of the remote control device and the layout information between the keys. Realize more accurate prediction; (3) When constructing map data, it can be combined with the characteristics of the controlled device, so that when the remote control function data cannot be determined according to the remote control device, it can be determined in combination with the characteristics of the controlled device, and the prediction is more accurate; (4) It can be directly The feature information of the controlled device is obtained through the image of the controlled device, or the user's voice input, text input, etc., so as to more conveniently determine the corresponding remote control function data; (5) After adding an identifier to the determined remote control function data and storing it, The remote control function data corresponding to the controlled device can be directly called to realize the control, the control is more convenient and efficient, and the user experience is improved; (6) Test the determined remote control function data to ensure the accuracy of the configured remote control function data and avoid mistakes. (7) When the determined remote control function data is invalid, the parameters of the recognition model are updated based on the re-determined result, which can improve the recognition accuracy of the model. It should be noted that different embodiments may have different beneficial effects, and in different embodiments, the possible beneficial effects may be any one or a combination of the above, or any other possible beneficial effects.

上文已对基本概念做了描述，显然，对于本领域技术人员来说，上述详细披露仅仅作为示例，而并不构成对本说明书的限定。虽然此处并没有明确说明，本领域技术人员可能会对本说明书进行各种修改、改进和修正。该类修改、改进和修正在本说明书中被建议，所以该类修改、改进、修正仍属于本说明书示范实施例的精神和范围。The basic concepts have been described above. Obviously, for those skilled in the art, the above detailed disclosure is merely an example, and does not constitute a limitation of the present specification. Although not explicitly described herein, various modifications, improvements, and corrections to this specification may occur to those skilled in the art. Such modifications, improvements, and corrections are suggested in this specification, so such modifications, improvements, and corrections still belong to the spirit and scope of the exemplary embodiments of this specification.

同时，本说明书使用了特定词语来描述本说明书的实施例。如“一个实施例”、“一实施例”、和/或“一些实施例”意指与本说明书至少一个实施例相关的某一特征、结构或特点。因此，应强调并注意的是，本说明书中在不同位置两次或多次提及的“一实施例”或“一个实施例”或“一个替代性实施例”并不一定是指同一实施例。此外，本说明书的一个或多个实施例中的某些特征、结构或特点可以进行适当的组合。Meanwhile, the present specification uses specific words to describe the embodiments of the present specification. Such as "one embodiment," "an embodiment," and/or "some embodiments" means a certain feature, structure, or characteristic associated with at least one embodiment of this specification. Therefore, it should be emphasized and noted that two or more references to "an embodiment" or "one embodiment" or "an alternative embodiment" in various places in this specification are not necessarily referring to the same embodiment . Furthermore, certain features, structures or characteristics of the one or more embodiments of this specification may be combined as appropriate.

此外，本领域技术人员可以理解，本说明书的各方面可以通过若干具有可专利性的种类或情况进行说明和描述，包括任何新的和有用的工序、机器、产品或物质的组合，或对他们的任何新的和有用的改进。相应地，本说明书的各个方面可以完全由硬件执行、可以完全由软件（包括固件、常驻软件、微码等）执行、也可以由硬件和软件组合执行。以上硬件或软件均可被称为“数据块”、“模块”、“引擎”、“单元”、“组件”或“系统”。此外，本说明书的各方面可能表现为位于一个或多个计算机可读介质中的计算机产品，该产品包括计算机可读程序编码。Furthermore, those skilled in the art will appreciate that aspects of this specification may be illustrated and described in several patentable categories or situations, including any new and useful process, machine, product, or combination of matter, or combinations of them. of any new and useful improvements. Accordingly, various aspects of this specification may be performed entirely by hardware, entirely by software (including firmware, resident software, microcode, etc.), or by a combination of hardware and software. The above hardware or software may be referred to as a "data block", "module", "engine", "unit", "component" or "system". Furthermore, aspects of this specification may be embodied as a computer product comprising computer readable program code embodied in one or more computer readable media.

计算机存储介质可能包含一个内含有计算机程序编码的传播数据信号，例如在基带上或作为载波的一部分。该传播信号可能有多种表现形式，包括电磁形式、光形式等，或合适的组合形式。计算机存储介质可以是除计算机可读存储介质之外的任何计算机可读介质，该介质可以通过连接至一个指令执行系统、装置或设备以实现通讯、传播或传输供使用的程序。位于计算机存储介质上的程序编码可以通过任何合适的介质进行传播，包括无线电、电缆、光纤电缆、RF、或类似介质，或任何上述介质的组合。A computer storage medium may contain a propagated data signal with the computer program code embodied therein, for example, on baseband or as part of a carrier wave. The propagating signal may take a variety of manifestations, including electromagnetic, optical, etc., or a suitable combination. Computer storage media can be any computer-readable media other than computer-readable storage media that can communicate, propagate, or transmit a program for use by coupling to an instruction execution system, apparatus, or device. Program code on a computer storage medium may be transmitted over any suitable medium, including radio, cable, fiber optic cable, RF, or the like, or a combination of any of the foregoing.

本说明书各部分操作所需的计算机程序编码可以用任意一种或多种程序语言编写，包括面向对象编程语言如Java、Scala、Smalltalk、Eiffel、JADE、Emerald、C++、C#、VB.NET、Python等，常规程序化编程语言如C语言、Visual Basic、Fortran2003、Perl、COBOL2002、PHP、ABAP，动态编程语言如Python、Ruby和Groovy，或其他编程语言等。该程序编码可以完全在用户计算机上运行、或作为独立的软件包在用户计算机上运行、或部分在用户计算机上运行部分在远程计算机运行、或完全在远程计算机或处理设备上运行。在后种情况下，远程计算机可以通过任何网络形式与用户计算机连接，比如局域网（LAN）或广域网（WAN），或连接至外部计算机（例如通过因特网），或在云计算环境中，或作为服务使用如软件即服务（SaaS）。The computer program coding required for the operation of the various parts of this manual may be written in any one or more programming languages, including object-oriented programming languages such as Java, Scala, Smalltalk, Eiffel, JADE, Emerald, C++, C#, VB.NET, Python etc., conventional procedural programming languages such as C language, Visual Basic, Fortran2003, Perl, COBOL2002, PHP, ABAP, dynamic programming languages such as Python, Ruby and Groovy, or other programming languages, etc. The program code may run entirely on the user's computer, or as a stand-alone software package on the user's computer, or partly on the user's computer and partly on a remote computer, or entirely on the remote computer or processing device. In the latter case, the remote computer may be connected to the user's computer through any network, such as a local area network (LAN) or wide area network (WAN), or to an external computer (eg, through the Internet), or in a cloud computing environment, or as a service Use eg software as a service (SaaS).

此外，除非权利要求中明确说明，本说明书所述处理元素和序列的顺序、数字字母的使用、或其他名称的使用，并非用于限定本说明书流程和方法的顺序。尽管上述披露中通过各种示例讨论了一些目前认为有用的发明实施例，但应当理解的是，该类细节仅起到说明的目的，附加的权利要求并不仅限于披露的实施例，相反，权利要求旨在覆盖所有符合本说明书实施例实质和范围的修正和等价组合。例如，虽然以上所描述的系统组件可以通过硬件设备实现，但是也可以只通过软件的解决方案得以实现，如在现有的处理设备或移动设备上安装所描述的系统。Furthermore, unless explicitly stated in the claims, the order of processing elements and sequences described in this specification, the use of alphanumerics, or the use of other names is not intended to limit the order of the processes and methods of this specification. While the foregoing disclosure discusses by way of various examples some embodiments of the invention that are presently believed to be useful, it is to be understood that such details are for purposes of illustration only and that the appended claims are not limited to the disclosed embodiments, but rather The requirements are intended to cover all modifications and equivalent combinations falling within the spirit and scope of the embodiments of this specification. For example, although the system components described above may be implemented by hardware devices, they may also be implemented by software-only solutions, such as installing the described systems on existing processing devices or mobile devices.

同理，应当注意的是，为了简化本说明书披露的表述，从而帮助对一个或多个发明实施例的理解，前文对本说明书实施例的描述中，有时会将多种特征归并至一个实施例、附图或对其的描述中。但是，这种披露方法并不意味着本说明书对象所需要的特征比权利要求中提及的特征多。实际上，实施例的特征要少于上述披露的单个实施例的全部特征。Similarly, it should be noted that, in order to simplify the expressions disclosed in this specification and thus help the understanding of one or more embodiments of the invention, in the foregoing description of the embodiments of this specification, various features may sometimes be combined into one embodiment, in the drawings or descriptions thereof. However, this method of disclosure does not imply that the subject matter of the description requires more features than are recited in the claims. Indeed, there are fewer features of an embodiment than all of the features of a single embodiment disclosed above.

一些实施例中使用了描述成分、属性数量的数字，应当理解的是，此类用于实施例描述的数字，在一些示例中使用了修饰词“大约”、“近似”或“大体上”来修饰。除非另外说明，“大约”、“近似”或“大体上”表明所述数字允许有±20%的变化。相应地，在一些实施例中，说明书和权利要求中使用的数值参数均为近似值，该近似值根据个别实施例所需特点可以发生改变。在一些实施例中，数值参数应考虑规定的有效数位并采用一般位数保留的方法。尽管本说明书一些实施例中用于确认其范围广度的数值域和参数为近似值，在具体实施例中，此类数值的设定在可行范围内尽可能精确。Some examples use numbers to describe quantities of ingredients and attributes, it should be understood that such numbers used to describe the examples, in some examples, use the modifiers "about", "approximately" or "substantially" to retouch. Unless stated otherwise, "about", "approximately" or "substantially" means that a variation of ±20% is allowed for the stated number. Accordingly, in some embodiments, the numerical parameters set forth in the specification and claims are approximations that can vary depending upon the desired characteristics of individual embodiments. In some embodiments, the numerical parameters should take into account the specified significant digits and use a general digit reservation method. Notwithstanding that the numerical fields and parameters used in some embodiments of this specification to confirm the breadth of their ranges are approximations, in specific embodiments such numerical values are set as precisely as practicable.

针对本说明书引用的每个专利、专利申请、专利申请公开物和其他材料，如文章、书籍、说明书、出版物、文档等，特此将其全部内容并入本说明书作为参考。与本说明书内容不一致或产生冲突的申请历史文件除外，对本说明书权利要求最广范围有限制的文件（当前或之后附加于本说明书中的）也除外。需要说明的是，如果本说明书附属材料中的描述、定义、和/或术语的使用与本说明书所述内容有不一致或冲突的地方，以本说明书的描述、定义和/或术语的使用为准。For each patent, patent application, patent application publication, and other material, such as article, book, specification, publication, document, etc., cited in this specification, the entire contents of which are hereby incorporated by reference into this specification are hereby incorporated by reference. Application history documents that are inconsistent with or conflict with the contents of this specification are excluded, as are documents (currently or hereafter appended to this specification) limiting the broadest scope of the claims of this specification. It should be noted that, if there is any inconsistency or conflict between the descriptions, definitions and/or use of terms in the accompanying materials of this specification and the contents of this specification, the descriptions, definitions and/or use of terms in this specification shall prevail .

最后，应当理解的是，本说明书中所述实施例仅用以说明本说明书实施例的原则。其他的变形也可能属于本说明书的范围。因此，作为示例而非限制，本说明书实施例的替代配置可视为与本说明书的教导一致。相应地，本说明书的实施例不仅限于本说明书明确介绍和描述的实施例。Finally, it should be understood that the embodiments described in this specification are only used to illustrate the principles of the embodiments of this specification. Other variations are also possible within the scope of this specification. Accordingly, by way of example and not limitation, alternative configurations of the embodiments of this specification may be considered consistent with the teachings of this specification. Accordingly, the embodiments of this specification are not limited to those expressly introduced and described in this specification.