CN113851130A - Universal intelligent device voice control method based on home gateway device - Google Patents

Abstract

Translated fromChinese

本发明涉及一种基于网关设备的通用智能设备语音操控方法和系统。该方法包括对新入网的智能设备进行初始化并完成设备注册和服务注册，并在接收到用户语音命令时将其转化为对应的设备指令，各智能设备解析该设备指令并对本设备可处理的设备指令进行反馈，根据反馈结果进行智能设备的选择和操控。所述系统包括入网管理子系统和语音命令处理子系。本发明的方法和系统能够将内异构厂家的智能设备进行组网管理，对用户指令进行标准化采集并根据应用层接口协议自动分发到相应智能设备，从而实现统一管理。

The invention relates to a voice control method and system for general intelligent equipment based on gateway equipment. The method includes initializing a new networked smart device, completing device registration and service registration, and converting a user voice command into a corresponding device command when receiving a user voice command, and each smart device parses the device command and identifies the devices that can be processed by the device. Instructions are fed back, and smart devices are selected and manipulated according to the feedback results. The system includes a network access management subsystem and a voice command processing subsystem. The method and system of the present invention can perform network management on the intelligent devices of heterogeneous manufacturers, standardize the collection of user instructions and automatically distribute them to the corresponding intelligent devices according to the application layer interface protocol, thereby realizing unified management.

Description

Universal intelligent device voice control method based on home gateway device

Technical Field

The invention relates to an intelligent home network, in particular to a universal intelligent device voice control method and a universal intelligent device voice control system based on home gateway equipment.

Background

The existing household electronic and electric equipment is divided into two types.

The first is a traditional device without any intelligent function, and the use mode is usually that a user operates through keys and knobs on the surface of the device or operates through a traditional infrared remote controller. Such devices do not have a networking function, each device is independently controlled to operate, and a user must be near or at a short distance from the device to operate the device. When many users need to use the remote controller, the remote controller can not be found, or the remote controller is out of power, and the like, which brings great inconvenience to the use.

The second equipment has certain intelligent attribute and has the functions of voice control, network access and the like, but each manufacturer designs voice operation specifications, access modes or respective operation end mobile phone APP according to the own company standard. The non-uniform standard causes different control modes of each device, and the devices lack uniform networking management capability and are difficult to work coordinately. Even if a user can operate by using the mobile phone APP, one APP is still used as one device, the original states of a plurality of intelligent devices are not broken away from on the use level, the advancement and the advantages of the intelligent devices are not really played, and the convenience brought to life by the intelligent devices cannot be fully embodied.

Therefore, a method and a system for networking and managing the smart devices of the home heterogeneous manufacturers are needed, which can perform standardized acquisition on user instructions and automatically distribute the user instructions to the corresponding smart devices according to an application layer interface protocol.

Disclosure of Invention

This summary is provided to introduce a selection of concepts in a simplified form that are further described below in the detailed description. This summary is not intended to identify key features or essential features of the claimed subject matter; nor is it intended to be used as an aid in determining or limiting the scope of the claimed subject matter.

According to the universal intelligent device voice control method and system based on the home gateway device, the intelligent devices of various heterogeneous manufacturers are controlled through universal voice commands in a home wifi access environment by designing an application layer interface protocol in the gateway device. No matter how the inside of each intelligent device is controlled, as long as the interface and the protocol defined in the invention are supported, perfect unified control can be realized, no additional function development and configuration are needed, and free expansion of the device is really realized.

In addition, traditional smart machine carries out control management through the management and control platform of each producer, and data have very big risk in transmission and storage, probably by malicious operation or steal. The invention carries out intensive management and control on the local gateway, and the data and the instruction effectively control the security risk of the data in the storage and control processes in an encryption transmission mode, thereby having great advantages in security.

The invention discloses a gateway-based general intelligent device voice control method, which comprises the steps of networking/networking of intelligent devices, voice acquisition and analysis and instruction control.

The networking/networking of the intelligent equipment comprises the following steps:

the new intelligent device is connected to the gateway through WiFi;

initializing new intelligent equipment, including setting equipment ID and cluster ID of the equipment cluster;

starting triggering network access operation: if the cluster ID consistent with the new intelligent equipment exists in the network environment where the gateway is located, the equipment ID of the new intelligent equipment is stored in an equipment cluster management list corresponding to the cluster ID; and

and if no cluster ID consistent with the new intelligent equipment exists in the network environment where the gateway is located, establishing a new equipment cluster, and storing the equipment ID of the new intelligent equipment into the new equipment cluster management list.

The voice acquisition analysis and instruction control method comprises the following steps:

receiving a voice command from a user;

performing voice recognition, converting the received voice command into a text, and converting the text into an equipment instruction of corresponding equipment;

sending the equipment instruction to each intelligent equipment in the network environment where the gateway is located;

each intelligent device analyzes the device instruction respectively, and if the analyzed device instruction belongs to the processable device, the feedback is sent;

judging whether only one or two or more intelligent devices send feedback;

if two or more devices send feedback, the feedback is sent to the user, the selection from the user is received, and the intelligent device selected by the user is controlled;

if only one intelligent device sends feedback, the intelligent device is directly controlled by the execution, or the feedback is sent back to the user, and the intelligent device is controlled by the execution after the user confirms the feedback.

The general intelligent device voice control system based on the gateway comprises a network access management subsystem and a voice command processing subsystem, wherein:

the network access management subsystem comprises: the intelligent device registration module is used for initializing new intelligent devices when the new intelligent devices are added into the network environment where the gateway is located, and setting the device IDs of the new intelligent devices and the cluster IDs of the device clusters to which the new intelligent devices belong; the intelligent device network access management module is used for registering new intelligent devices, and comprises the steps of judging whether a cluster with the same cluster ID as the cluster ID exists in a network environment or not, if so, storing the device ID of the new intelligent device into an existing device cluster management list corresponding to the cluster ID, and if not, establishing a new device cluster and storing the device ID into the new device cluster management list; the voice analysis service registration module is used for registering the voice analysis service of the new intelligent equipment;

the voice command processing subsystem includes: the voice command acquisition and transmission equipment is used for acquiring an audio signal of a user voice command; the voice command receiving module is used for receiving a user voice command through a standard http protocol; the instruction conversion module is used for analyzing the voice command into a corresponding equipment instruction by calling an application layer interface protocol; the equipment instruction interaction module is used for sending the analyzed equipment instructions to each intelligent equipment and receiving feedback from each equipment only; and a voice command controller for providing feedback to the user of each smart device and receiving a selection and/or confirmation of a smart device by the user.

These and other features and advantages will become apparent upon reading the following detailed description and upon reference to the accompanying drawings. It is to be understood that both the foregoing general description and the following detailed description are explanatory only and are not restrictive of aspects as claimed.

Drawings

The present invention will now be described more fully hereinafter with reference to the accompanying drawings, in which specific embodiments of the invention are shown.

FIG. 1 is a schematic diagram of smart device networking/organizing, according to various embodiments of the present invention;

FIG. 2 is a flow diagram of smart device networking/netting, according to various embodiments of the invention;

FIG. 3 is a flow diagram of voice acquisition parsing and smart device command control according to various embodiments of the invention;

FIG. 4 is a schematic block diagram of a generic smart device voice manipulation system based on a gateway device in accordance with various embodiments of the present invention.

The flowchart and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to embodiments of the present application. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s).

Detailed Description

The present invention will now be described more fully hereinafter with reference to the accompanying drawings, in which specific embodiments of the invention are shown. Various advantages and benefits of the present invention will become apparent to those of ordinary skill in the art upon reading the following detailed description of the specific embodiments. It should be understood, however, that the present invention may be embodied in various forms and should not be limited to the embodiments set forth herein. The following embodiments are provided so that the invention may be more fully understood. Unless otherwise defined, technical or scientific terms used herein shall have the ordinary meaning as understood by those of skill in the art to which this application belongs.

The invention discloses a universal intelligent device voice control method based on home gateway equipment. The first is the network access/networking of the intelligent equipment, and the second is the voice acquisition analysis and instruction control.

FIG. 1 shows (a-1), (a-2), (a-3) and (b-1), (b-2) schematic diagrams of smart device networking/organizing according to an embodiment of the present invention; fig. 2 is a flowchart of network entry/networking of the smart device according to the embodiment shown in fig. 1. The first part of the method of the present invention, i.e. the specific steps of network entry/networking of the smart device, is discussed below with reference to fig. 1 and 2:

as shown in (a-1) of fig. 1, the current network environment includes a gateway and 1 to n other intelligent devices that have already been networked, where the existingintelligent device 1 and the existingintelligent device 2 are of the same type and belong to the same device cluster, and the existing intelligent device n-1 and the existing intelligent device n are of the same type and belong to another device cluster (the cluster of the same type of devices is shown in a dashed box in fig. 1).

The same type of intelligent equipment is classified into the same equipment cluster, and the purpose is to adapt to the same type of semantics. For example, for the environment lamp in a living room and the kitchen lighting lamp, the requirement of brightness adjustment is met, and the voice control of 'light turning on a little' can be uniformly recognized. Similarly, a cluster of televisions, speakers, etc. may recognize a voice control that is "louder". Therefore, the consistency of user experience can be achieved. One voice command for the same cluster may translate to the same device instruction.

When a new network is connected to a smart device (hereinafter referred to as a "smart device")_New") to join the network environment, it is first connected to the gateway through wifi at step S210, and then the smart device registration module registers the smart device at the gateway at step S220_NewConfiguring, i.e. initializing, it, including setting up the smart device_NewAnd the cluster ID of the device cluster to which it belongs. Wherein the device ID is a smart device_NewThe cluster ID is used for marking the intelligent equipment_NewThe cluster of devices to which it belongs.

If, for example, a smart device_NewThe same type as the intelligent devices n-1 and n, corresponding to the case of (a-2) in FIG. 1, the intelligent device will be used_NewIs set to be consistent with the cluster IDs of the existing smart devices n-1 and n.

If smart device_NewDifferent from the types of the existingintelligent devices 1 to n, the intelligent device is the intelligent device corresponding to the situation (a-3) in fig. 1_NewA new device cluster ID is additionally set.

If no other intelligent devices have been deployed in the current network environment, as is the case in (b-1) of FIG. 1, then it is the intelligent device_NewA new device cluster ID is set.

The intelligent equipment_NewAfter the initialization configuration, the intelligent device starts to trigger network access operation_NewSending the device ID and the cluster ID to the device network access management module, and determining whether a cluster of the same type of device as the cluster ID exists in the current network environment in step S230:

if the device cluster exists in the current network environment, the device networking management module finds the received intelligent device after checking_NewIf the cluster ID of (2) is consistent with the existing cluster ID, the process proceeds to S240, and the smart device is connected to the server_NewThe device ID is stored in a device cluster management list corresponding to the cluster ID, and device registration and service registration are completed until the intelligent device_NewCompleting the network access, as shown in FIG. 1 (a-2);

if there is no other intelligent device deployed in the current network as in the case shown in fig. 1(b-2), or there is no cluster of similar devices although there are other intelligent devices deployed as in the case shown in fig. 1(a-3), the step proceeds to S250, that is, the device network entry management module finds out that there is no intelligent device with any existing cluster ID sum after checking_NewIf the cluster IDs are consistent, a new device is newly builtClustering (shown in fig. 1(a-3) with long line segments and dashed boxes), and associating smart devices_NewThe device ID of (2) is stored in the new device cluster management list, and device registration and service registration are completed. If in future with this smart machine_NewNew additions of the same type of smart device may be attributed to the cluster.

To this end, a smart device_NewThe network access is completed. Here, the service registration includes registration of a voice resolution service. Namely, each intelligent device registers the voice analysis service of the intelligent device when accessing the network, and the calling of the voice analysis service is realized in a callback mode when the intelligent device is used later.

The second part of the method of the present invention, the specific steps of speech acquisition and parsing and intelligent device command control, is discussed below in conjunction with fig. 3:

in step S310, a voice command from a user is received. The step can be realized by using a Bluetooth headset with a microphone, and the voice command sent by the user through the Bluetooth headset is received, so that the audio signal acquisition and transmission of the voice command are completed.

In step S320, the received voice command audio signal is processed, including:

the first step is to perform speech recognition to convert the speech input into text, which is standardized;

and in the second step, semantic analysis and understanding are carried out on the text, and then the text is converted into a corresponding device instruction, so that the semantic understanding of different devices is different, and AI training is specially carried out. For example, the speech "lisiang" may be recognized by the smart speaker as a contact name and called out and may also be recognized by the tv set-top box as a hot movie for playing, so that the semantic analysis and understanding requires AI training for different types of devices,

the voice command is analyzed by calling the upper layer voice command to identify and issue an application layer interface protocol, the analyzed semantics are changed into a control instruction of the equipment, the instruction is sent to each intelligent equipment in the network environment,

in step S330, each smart device separately parses the instruction, and if it is recognized that this is an instruction that it can handle, the smart device sends a feedback.

In step S340, it is determined whether only one or two or more smart devices have sent feedback.

If only one smart device sends feedback, the feedback is sent back to the user in step S350, and the user confirms that the smart device is operated in step S370.

If only two or more smart devices are sending feedback, e.g., a voice control such as "light up", there may be multiple light responses for the user to choose which one is desired to adjust. The feedback of the status of each smart device is sent back to the user in step S360, which may be done by audio output. The user may further select by voice, and the user selected smart device is manipulated in step S370 after confirming the user selection through voice recognition.

In another embodiment, if only one smart device sends feedback, the process can directly proceed to step S370 without the user of S350 confirming the process again.

FIG. 4 is a schematic block diagram of a generic smart device voice manipulation system based on a gateway device in accordance with various embodiments of the present invention.

The universal intelligent device voice control system comprises a network access management subsystem 410 and a voice command processing subsystem 420.

The network access management subsystem 410 is configured to perform network access/networking of the smart device, and further includes a dev _ sign _ service smart device registration module 411, a dev _ net _ service smart device network access management module 412, and a voice _ analysis _ service voice parsing service registration module 413. These modules may be integrated in or communicatively connected with a gateway, which is shown in fig. 4 as being integrated in the gateway. Wherein:

smart device registration Module 411 for registering "Smart devices_NewCarry out the intelligent equipment when joining the network_NewInitialization work of, setting up the smart device_NewAnd the cluster ID of the device cluster to which it belongs.

Intelligent device networkingManagement module 412 receives smart devices_NewThe device ID and the cluster ID of the intelligent device are used for judging whether a cluster of the same type of devices with the same cluster ID exists in the current network environment or not, and if the cluster of the same type of devices with the same cluster ID exists, the intelligent device is used for judging whether the cluster of the same type of devices with the same cluster ID exists in the current network environment or not_NewIf not, a new device cluster is established and the device ID is stored in the management list of the new device cluster. Complete the intelligent device_NewThe device registration of (1).

Each intelligent device must register its own voice parsing service through the voice parsing service registration module 413 when accessing the network, so as to realize the calling of the voice parsing service in a callback manner when being used later. Complete the intelligent device_NewThe service registration of (2).

The voice command processing subsystem 420 includes a voice command collecting and transmitting device 421, a voice _ cmd _ collect _ service voice command receiving module 422, a cmd _ transform _ service command converting module 423, a dev _ cmd _ manager _ service device command interacting module 424, and a voice command controller 425. Wherein:

the voice command collecting and transmitting device 421 may be a bluetooth headset with a microphone, or a microphone disposed at various places in the network environment, and may be in wired or wireless communication with the gateway for collecting the audio signal of the user voice command.

The voice command receiving module 422 receives the user voice command collected from the voice command collection transmission device 421 through the standard http protocol.

The command conversion module 423 processes (speech recognition) the user speech command received by the speech command reception module 422, and parses the speech command into a device command by invoking an upper layer speech command recognition issuing application layer interface protocol.

The device command interaction module 424 performs command interaction with the smart devices via the application layer interface protocol, sends the parsed device commands to each smart device, and transmits feedback (enabling execution of the commands) from each smart device to the voice command controller 425.

The voice command controller 425 is used to provide the above-mentioned feedback of each smart device to the user and receive the user's selection and confirmation of a smart device.

The voice command receiving module 422, the instruction translating module 423, and the device instruction interacting module 424 may be integrated in or communicatively coupled with a gateway, which is shown integrated in fig. 4.

The network access/networking mode of the intelligent equipment is completed through a terminal equipment registration module and an equipment networking management module in the local gateway. Remote management through the cloud is not required.

The invention places the management of the intelligent equipment in the application layer of the gateway, has strong universality, can realize the compatible management of various intelligent equipment and the unified management of the intelligent equipment of different manufacturers.

The user uses pronunciation in unison can simply realize the operation to each smart machine, need not to use the remote controller, also need not to use user's cell-phone APP. Better user experience can be obtained.

The data and the instruction effectively control the security risk of the data in the storage and control processes in an encryption transmission mode, and the security is high.

The above embodiments are only used for illustrating the technical solutions of the present application, and not for limiting the same; although the present application has been described in detail with reference to the foregoing embodiments, it should be understood by those of ordinary skill in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some or all of the technical features may be equivalently replaced; such modifications and substitutions do not depart from the spirit and scope of the present disclosure, and the present disclosure should be construed as being covered by the claims and the specification.

Claims (10)

Translated fromChinese

1.一种基于网关的通用智能设备语音操控方法，包括：1. A gateway-based general intelligent device voice control method, comprising:接收来自用户的语音命令；receive voice commands from the user;将所接收的语音命令转化为对应的设备指令；Convert the received voice commands into corresponding device instructions;将所述设备指令发送到所述网关所在网络环境内每一台智能设备；sending the device instruction to each smart device in the network environment where the gateway is located;各智能设备分别对所述设备指令进行解析，如果解析出属于本设备可处理的，则发送反馈；Each smart device parses the device instructions respectively, and sends feedback if it is parsed that the device can process it;判断仅有一台还是有两台及以上智能设备发出反馈；Determine whether there is only one or two or more smart devices to send feedback;如果有两台及以上设备发出反馈，则将所述反馈发送给所述用户；以及If two or more devices send feedback, send the feedback to the user; and接收来自用户的选择并执行对所述用户选择的智能设备的操控。A selection from a user is received and manipulation of the user-selected smart device is performed.2.如权利要求1所述的方法，其特征在于，所述将所接收的语音命令转化为对应的设备指令包括：2. The method of claim 1, wherein the converting the received voice command into a corresponding device instruction comprises:进行语音识别，并将所述语音命令转为文本；以及perform speech recognition and convert said speech commands to text; and将所述文本转化为对应设备的设备指令。The text is converted into device instructions for the corresponding device.3.如权利要求2所述的方法，其特征在于，将所述文本转化为对应设备的设备指令包括针对不同设备进行AI训练。3. The method of claim 2, wherein converting the text into device instructions for the corresponding device comprises performing AI training for different devices.4.如权利要求1所述的方法，其特征在于，还包括：4. The method of claim 1, further comprising:如果仅有一台智能设备发出反馈，则直接执行对该执行对智能设备的操控。If only one smart device sends feedback, the control of the smart device is directly executed.5.如权利要求1所述的方法，其特征在于，还包括：5. The method of claim 1, further comprising:如果仅有一台智能设备发出反馈，则将所述反馈发送回用户，经用户确认后该执行对智能设备的操控。If only one smart device sends feedback, the feedback is sent back to the user, and after the user's confirmation, the manipulation of the smart device should be performed.6.如权利要求1所述的方法，其特征在于，还包括智能设备的入网，所述入网包括：6. The method of claim 1, further comprising network access of the smart device, the network access comprising:新入网的智能设备通过WiFi连接到网关；Newly connected smart devices connect to the gateway via WiFi;对所述新入网的智能设备进行初始化，包括设置设备ID和所属设备集群的集群ID；Initializing the newly connected smart device, including setting the device ID and the cluster ID of the device cluster to which it belongs;启动触发入网操作：Start the trigger network access operation:如果所述网关所在的网络环境中已有与所述新入网的智能设备相一致的集群ID存在，则将所述新入网的智能设备的所述设备ID保存至该集群ID对应的设备集群管理列表中；以及If a cluster ID consistent with the newly connected smart device already exists in the network environment where the gateway is located, save the device ID of the newly connected smart device to the device cluster management corresponding to the cluster ID on the list; and如果所述网关所在的网络环境中无与所述新入网的智能设备相一致的集群ID存在，则新建一个新的设备集群，并将所述新入网的智能设备的所述设备ID保存至该新的设备集群的管理列表中。If there is no cluster ID consistent with the newly connected smart device in the network environment where the gateway is located, create a new device cluster, and save the device ID of the newly connected smart device to the in the management list of the new device cluster.7.一种基于网关的通用智能设备语音操控系统，包括入网管理子系统和语音命令处理子系统，其中所述语音命令处理子系统包括：7. A gateway-based general intelligent device voice control system, comprising a network access management subsystem and a voice command processing subsystem, wherein the voice command processing subsystem comprises:语音命令采集传输设备，用于采集用户语音命令的音频信号；Voice command acquisition and transmission equipment, used to collect the audio signal of the user's voice command;语音命令接收模块，用于通过标准http协议接收用户语音命令；Voice command receiving module, used to receive user voice commands through standard http protocol;指令转化模块，用于通过调用应用层接口协议来将所述语音命令解析为对应的设备指令；an instruction conversion module, configured to parse the voice command into a corresponding device instruction by invoking an application layer interface protocol;设备指令交互模块，用于将解析好的设备指令发送个各智能设备，并接收来自各智能设备的反馈；以及The device command interaction module is used to send the parsed device command to each smart device and receive feedback from each smart device; and语音命令控制器，用于向用户提供各智能设备的所述反馈，并接收用户对某个智能设备的选择和/或确认。The voice command controller is used to provide the user with the feedback of each smart device, and receive the user's selection and/or confirmation of a certain smart device.8.如权利要求7所述的系统，其特征在于，所述入网管理子系统包括：8. The system of claim 7, wherein the network access management subsystem comprises:智能设备注册模块，用于在新入网的智能设备加入所述网关所在网络环境时对所述新入网的智能设备进行初始化；an intelligent device registration module, configured to initialize the newly entered intelligent device when the newly entered intelligent device joins the network environment where the gateway is located;智能设备入网管理模块，用于进行所述新入网的智能设备的设备注册；以及a smart device network access management module, configured to perform device registration of the newly connected smart device; and语音解析服务注册模块，用于进行所述新入网的智能设备的语音解析服务注册。The speech analysis service registration module is used for registering the speech analysis service of the newly connected smart device.9.如权利要求8所述的系统，其特征在于，所述智能设备注册模块设置所述新入网的智能设备的设备ID和所属设备集群的集群ID。9 . The system according to claim 8 , wherein the smart device registration module sets the device ID of the newly connected smart device and the cluster ID of the device cluster to which it belongs. 10 .10.如权利要求8所述的系统，其特征在于，所述智能设备入网管理模块，接收所述新入网的智能设备的设备ID和所属设备集群的集群ID，判断所述网络环境中是否已存在与该集群ID相同的集群，如果有，则将所述新入网的智能设备的设备ID保存到该集群ID对应的已有的设备集群管理列表中，如果没有，则建一个新的设备集群并该设备ID保存至该新的设备集群的管理列表中。10. The system according to claim 8, wherein the smart device network access management module receives the device ID of the newly connected smart device and the cluster ID of the device cluster to which it belongs, and judges whether the network environment has If there is a cluster with the same cluster ID, if there is, save the device ID of the newly connected smart device to the existing device cluster management list corresponding to the cluster ID, if not, create a new device cluster And save the device ID to the management list of the new device cluster.

Images