技术领域technical field
本公开一般涉及物联网领域，具体涉及物联网中设备的操作控制方案，尤其涉及用于物联网设备的可视化操作方法和装置。The present disclosure generally relates to the field of the Internet of Things, specifically relates to an operation control scheme of devices in the Internet of Things, and particularly relates to a visual operation method and device for devices in the Internet of Things.
背景技术Background technique
增强现实技术(Augmented Reality，简称AR)，是一种实时地计算摄影机影像的位置及角度并加上相应图像的技术，这种技术的目标是在屏幕上把虚拟世界套在现实世界并进行互动。这一技术不仅展现了真实世界的信息，而且将虚拟的信息同时显示出来，两种信息相互补充、叠加。Augmented reality technology (Augmented Reality, referred to as AR), is a technology that calculates the position and angle of the camera image in real time and adds the corresponding image. The goal of this technology is to put the virtual world on the screen and interact with the real world. . This technology not only displays real-world information, but also displays virtual information at the same time, and the two kinds of information complement and superimpose each other.
目前，用户在获取物联网中设备的操作界面时，首先需要查找物联网设备的设备列表，在找到设备列表后，需要选中设备列表中的设备图标，之后才可以进入设备的操作界面，需要多个步骤来完成。At present, when the user obtains the operation interface of the device in the Internet of Things, he first needs to search for the device list of the Internet of Things device. After finding the device list, he needs to select the device icon in the device list, and then he can enter the operation interface of the device. steps to complete.
发明内容Contents of the invention
本申请提供了一种用于物联网设备的可视化操作方法和装置。The present application provides a visual operation method and device for IoT devices.
一方面，本申请提供了一种用于物联网设备的可视化操作方法，所述方法包括：响应于接收到采集指令，采集已接入物联网的设备的图像；获取采集的图像在包括所有设备的全景图像中的设备图像；查询关联关系数据库中对应所述设备图像的设备信息；与查询的设备信息对应的设备建立通信连接；呈现所述设备图像和建立通信连接的设备的操作界面。In one aspect, the present application provides a visual operation method for Internet of Things devices, the method comprising: collecting images of devices connected to the Internet of Things in response to receiving a collection instruction; device image in the panoramic image; query the device information corresponding to the device image in the association relationship database; establish a communication connection with the device corresponding to the queried device information; present the operation interface of the device image and the device establishing the communication connection.
在某些实施方式中，所述关联关系数据库基于以下步骤得到：获取各个设备的设备信息；获取所述包括所有设备的全景图像；设定所述全景图像中的各设备图像与各设备信息的关联关系；将所述关联关系添加至关联关系数据库。In some implementations, the association relationship database is obtained based on the following steps: obtaining the device information of each device; obtaining the panoramic image including all devices; setting the relationship between each device image in the panoramic image and each device information Association relationship; adding the association relationship to the association relationship database.
在某些实施方式中，所述设定全景图像中的各设备图像与各设备信息的关联关系包括：接收用户对单个设备图像和设备信息库中的单个设备信息的选中操作；响应于接收用户对单个设备图像和设备信息库中的单个设备信息的选中操作，生成所述单个设备图像与所述单个设备信息的关联关系。In some implementations, the setting the association relationship between each device image in the panoramic image and each device information includes: receiving a user's selection operation on a single device image and a single device information in the device information library; The selection operation of the single device image and the single device information in the device information database generates an association relationship between the single device image and the single device information.
在某些实施方式中，所述设备信息中包括授权信息；所述方法还包括：基于被选中的设备信息中的授权信息，将与被选中的设备信息对应的设备接入物联网。In some embodiments, the device information includes authorization information; the method further includes: based on the authorization information in the selected device information, connecting the device corresponding to the selected device information to the Internet of Things.
在某些实施方式中，所述呈现所述对应所述当前识别结果的设备的操作界面包括：获取所述建立通信连接的设备的操作列表信息；基于所述操作列表信息，呈现操作界面。In some implementations, the presenting the operation interface of the device corresponding to the current recognition result includes: acquiring operation list information of the device establishing the communication connection; presenting an operation interface based on the operation list information.
在某些实施方式中，所述方法还包括：接收用户对所述操作界面的输入操作；将所述输入操作对应的操作指令发送至所述建立通信连接的设备以控制所述建立通信连接的设备。In some embodiments, the method further includes: receiving a user's input operation on the operation interface; sending an operation instruction corresponding to the input operation to the device for establishing the communication connection to control the device for establishing the communication connection equipment.
第二方面，本申请提供了一种用于物联网设备的可视化操作装置，所述装置包括：图像采集单元，用于响应于接收到采集指令，采集已接入物联网的设备的图像；图像映射单元，用于获取采集的图像在包括所有设备的全景图像中的设备图像；信息查询单元，用于查询关联关系数据库中对应所述设备图像的设备信息；通信连接单元，用于与查询的设备信息对应的设备建立通信连接；界面呈现单元，用于呈现所述设备图像和建立通信连接的设备的操作界面。In a second aspect, the present application provides a visual operation device for Internet of Things devices, the device includes: an image acquisition unit, configured to collect images of devices connected to the Internet of Things in response to receiving an acquisition instruction; The mapping unit is used to acquire the device image of the captured image in the panoramic image including all devices; the information query unit is used to query the device information corresponding to the device image in the relational database; the communication connection unit is used to communicate with the query The device corresponding to the device information establishes a communication connection; the interface presentation unit is configured to present the device image and the operation interface of the device establishing the communication connection.
在某些实施方式中，所述信息查询单元所调用的关联关系数据库基于以下单元得到：信息获取单元，用于获取各个设备的设备信息；图像获取单元，用于获取所述包括所有设备的全景图像；关系设定单元，用于设定所述全景图像中的各设备图像与各设备信息的关联关系；数据添加单元，用于将所述关联关系添加至关联关系数据库。In some embodiments, the association relationship database invoked by the information query unit is obtained based on the following units: an information acquisition unit, configured to acquire device information of each device; an image acquisition unit, configured to acquire the panorama including all devices image; a relationship setting unit, configured to set an association relationship between each device image and each device information in the panoramic image; a data adding unit, configured to add the association relationship to an association relationship database.
在某些实施方式中，所述关系设定单元包括：操作接收子单元，用于接收用户对单个设备图像和设备信息库中的单个设备信息的选中操作；关系生成子单元，用于响应于接收用户对单个设备图像和设备信息库中的单个设备信息的选中操作，生成所述单个设备图像与所述单个设备信息的关联关系。In some implementations, the relationship setting unit includes: an operation receiving subunit, configured to receive a user's selection operation on a single device image and a single device information in the device information library; a relationship generating subunit, configured to respond to An operation of selecting a single device image and a single device information in the device information library by the user is received, and an association relationship between the single device image and the single device information is generated.
在某些实施方式中，所述信息获取单元获取的设备信息中包括授权信息；以及所述装置还包括：接入物联网单元，用于基于被选中的设备信息中的授权信息，将与被选中的设备信息对应的设备接入物联网。In some embodiments, the device information obtained by the information obtaining unit includes authorization information; and the device further includes: an Internet of Things access unit, configured to communicate with the selected device information based on the authorization information in the selected device information The device corresponding to the selected device information is connected to the Internet of Things.
在某些实施方式中，所述界面呈现单元包括：信息获取子单元，用于获取所述建立通信连接的设备的操作列表信息；界面呈现子单元，用于基于所述操作列表信息，呈现操作界面。In some embodiments, the interface presentation unit includes: an information acquisition subunit, configured to acquire the operation list information of the device establishing the communication connection; an interface presentation subunit, configured to present the operation list information based on the operation list information interface.
在某些实施方式中，所述装置还包括：操作接收单元，用于接收用户对所述操作界面的输入操作；设备控制单元，用于将所述输入操作对应的操作指令发送至所述建立通信连接的设备以控制所述建立通信连接的设备。In some embodiments, the apparatus further includes: an operation receiving unit, configured to receive user input operations on the operation interface; a device control unit, configured to send an operation instruction corresponding to the input operation to the establishment The device for establishing the communication connection is used to control the device for establishing the communication connection.
第三方面，本申请提供了一种终端设备，所述终端设备包括如上所述的用于物联网设备的可视化操作装置。In a third aspect, the present application provides a terminal device, where the terminal device includes the above-mentioned visualized operation device for an Internet of Things device.
本申请提供的用于物联网设备的可视化操作方法和装置，可应用于终端设备，可以通过响应于接收到采集指令，采集已接入物联网的设备的图像，之后获取采集的图像在包括所有设备的全景图像中的设备图像，之后查询关联关系数据库中对应设备图像的设备信息，然后与查询的设备信息对应的设备建立通信连接，最后呈现设备图像和建立通信连接的设备的操作界面，从而通过响应于接收到采集指令而采集设备的图像，可向用户提供设备的可视化图像操作界面，简化了用户获取设备的操作界面的步骤，且向用户呈现的为可视化图像操作界面，呈现的方式更为直观。The visualized operation method and device for Internet of Things devices provided by this application can be applied to terminal devices, which can collect images of devices connected to the Internet of Things in response to receiving collection instructions, and then obtain the collected images including all The device image in the panoramic image of the device is then queried for the device information corresponding to the device image in the association relationship database, and then a communication connection is established with the device corresponding to the queried device information, and finally the device image and the operation interface of the device establishing the communication connection are presented, thereby By collecting the image of the device in response to receiving the collection instruction, the visual image operation interface of the device can be provided to the user, which simplifies the steps for the user to obtain the operation interface of the device, and what is presented to the user is a visual image operation interface in a more convenient way. for intuitive.
附图说明Description of drawings
通过阅读参照以下附图所作的对非限制性实施例所作的详细描述，本申请的其它特征、目的和优点将会变得更明显：Other characteristics, objects and advantages of the present application will become more apparent by reading the detailed description of non-limiting embodiments made with reference to the following drawings:
图1示出了可以应用本申请的用于物联网设备的可视化操作方法或装置的实施例的示例性系统架构；FIG. 1 shows an exemplary system architecture that can be applied to an embodiment of the visualized operation method or device for an Internet of Things device;
图2示出了根据本申请实施例的用于物联网设备的可视化操作方法的一个示例性流程图；FIG. 2 shows an exemplary flow chart of a visual operation method for an Internet of Things device according to an embodiment of the present application;
图3示出了根据本申请实施例的获取关联关系数据库的方法的一个示例性流程图；FIG. 3 shows an exemplary flow chart of a method for obtaining an association relationship database according to an embodiment of the present application;
图4a示出了用户扫描二维码获取设备信息的一个示例性场景图；Fig. 4a shows an exemplary scene diagram in which a user scans a two-dimensional code to obtain device information;
图4b示出了物联网设备的可视化场景图的一个示例性截屏效果图；Fig. 4b shows an exemplary screenshot effect diagram of a visualized scene graph of an Internet of Things device;
图4c示出了在可视化场景图中显示设备的操作界面的一个示例性截屏效果图；以及Fig. 4c shows an exemplary screenshot rendering of the operation interface of the display device in the visualized scene graph; and
图5示出了按照本申请的实施例的用于物联网设备的可视化操作装置的一个结构示意图。Fig. 5 shows a schematic structural diagram of a visual operation device for an Internet of Things device according to an embodiment of the present application.
图6示出了按照本申请的实施例的获取关联关系数据库的装置的一个结构示意图。Fig. 6 shows a schematic structural diagram of an apparatus for acquiring an association relationship database according to an embodiment of the present application.
具体实施方式detailed description
下面结合附图和实施例对本申请作进一步的详细说明。可以理解的是，此处所描述的具体实施例仅仅用于解释相关发明，而非对该发明的限定。另外还需要说明的是，为了便于描述，附图中仅示出了与有关发明相关的部分。The application will be further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described here are only used to explain related inventions, rather than to limit the invention. It should also be noted that, for the convenience of description, only the parts related to the related invention are shown in the drawings.
需要说明的是，在不冲突的情况下，本申请中的实施例及实施例中的特征可以相互组合。下面将参考附图并结合实施例来详细说明本申请。It should be noted that, in the case of no conflict, the embodiments in the present application and the features in the embodiments can be combined with each other. The present application will be described in detail below with reference to the accompanying drawings and embodiments.
图1示出了可以应用本申请的用于物联网设备的可视化操作方法或用于物联网设备的可视化操作装置的实施例的示例性系统架构100。Fig. 1 shows an exemplary system architecture 100 to which embodiments of the visualized operation method for IoT devices or the visualized operation device for IoT devices of the present application can be applied.
如图1所示，系统架构100可以包括终端设备101、102，网络103和物联网设备104、105、106、107。网络103用以在终端设备101、102和物联网设备104、105、106、107之间提供通信链路的介质。网络103可以采用无线通信链路。As shown in FIG. 1 , the system architecture 100 may include terminal devices 101 , 102 , a network 103 and IoT devices 104 , 105 , 106 , 107 . The network 103 is used as a medium for providing communication links between the terminal devices 101 , 102 and the IoT devices 104 , 105 , 106 , 107 . Network 103 may employ wireless communication links.
用户可以使用终端设备101、102通过网络103与物联网设备104、105、106、107交互，以接收或发送消息等。终端设备101、102上可以安装有各种通讯客户端应用，例如物联网设备管理类应用、网页浏览器应用、购物类应用、搜索类应用、即时通信工具、邮箱客户端、社交平台软件等。Users can use terminal devices 101 , 102 to interact with IoT devices 104 , 105 , 106 , 107 through network 103 to receive or send messages and the like. Various communication client applications can be installed on the terminal devices 101 and 102, such as IoT device management applications, web browser applications, shopping applications, search applications, instant messaging tools, email clients, social platform software, and the like.
终端设备101、102可以是具有显示屏并且支持无线通信的各种电子设备，包括但不限于智能手机、平板电脑、电子书阅读器、MP3播放器(Moving Picture Experts GroupAudio Layer III，动态影像专家压缩标准音频层面3)、MP4(Moving Picture ExpertsGroup Audio Layer IV，动态影像专家压缩标准音频层面4)播放器、膝上型便携计算机和台式计算机等等。Terminal devices 101, 102 can be various electronic devices with display screens and support wireless communication, including but not limited to smart phones, tablet computers, e-book readers, MP3 players (Moving Picture Experts GroupAudio Layer III, moving picture expert compression Standard audio layer 3), MP4 (Moving Picture ExpertsGroup Audio Layer IV, moving picture experts compressed standard audio layer 4) player, laptop portable computer and desktop computer, etc.
物联网设备104、105、106、107可以是提供各种服务的物联网设备，例如对终端设备101、102上物联网设备管理类应用提供支持的后物联网设备。物联网设备可以对接收到的终端的操作指令等数据进行处理，并按照处理结果进行动作。The IoT devices 104 , 105 , 106 , and 107 may be IoT devices that provide various services, such as post-IoT devices that support IoT device management applications on the terminal devices 101 and 102 . IoT devices can process data such as received terminal operation instructions, and act according to the processing results.
需要说明的是，本申请实施例所提供的用于物联网设备的可视化操作方法一般由终端设备101、102执行，相应地，用于物联网设备的可视化操作装置一般设置于终端设备101、102中。It should be noted that the visualized operation method for IoT devices provided in the embodiment of the present application is generally executed by terminal devices 101 and 102, and correspondingly, the visualized operation device for IoT devices is generally set on terminal devices 101 and 102 middle.
应该理解，图1中的终端设备、网络和服务器的数目仅仅是示意性的。根据实现需要，可以具有任意数目的终端设备、网络和服务器。It should be understood that the numbers of terminal devices, networks and servers in Fig. 1 are only illustrative. According to the implementation needs, there can be any number of terminal devices, networks and servers.
图2示出了根据本申请实施例的用于物联网设备的可视化操作方法的示例性流程图。Fig. 2 shows an exemplary flow chart of a visualized operation method for an Internet of Things device according to an embodiment of the present application.
如图2所示，用于物联网设备的可视化操作方法的示例性流程图200可以包括：As shown in FIG. 2 , an exemplary flow chart 200 of a visualized operation method for an Internet of Things device may include:
在步骤201中，响应于接收到采集指令，采集已接入物联网的设备的图像。In step 201, in response to receiving an acquisition instruction, an image of a device connected to the Internet of Things is acquired.
在本实施例中，采集指令可以来源于用户输入，也可以来源于由程序响应于满足预设条件而输入的采集指令，可以通过终端设备的摄像头来采集已接入物联网的设备的图像，也可以通过外接于终端设备的摄像装置来进行采集。In this embodiment, the collection instruction may come from the user input, or from the collection instruction input by the program in response to meeting the preset conditions, and the image of the device connected to the Internet of Things may be collected through the camera of the terminal device. It can also be collected through a camera device externally connected to the terminal device.
在步骤202中，获取采集的图像在包括所有设备的全景图像中的设备图像。In step 202, the device images of the collected images in the panoramic image including all devices are acquired.
在本实施例中，可以获取所有设备的全景图像，并进一步识别全景图像中的设备图像，将步骤201中采集的图像与识别得到的设备图像逐一进行比对，得到与采集的图像相对应的设备图像作为获取的结果。In this embodiment, the panoramic images of all devices can be obtained, and the device images in the panoramic images can be further identified, and the images collected in step 201 can be compared with the recognized device images one by one to obtain the images corresponding to the collected images. Device image as a result of fetching.
在这里，所有设备的全景图像，为预先通过全景摄像头拍摄的包括所有设备的图像，可以存储于终端设备本地或云端服务器，以供终端设备调用。这里的全景摄像头，可以为终端设备上配置的全景摄像头，也可以为外接于终端设备的全景摄像头，还可以为独立的可与终端设备进行通讯的全景摄像头。这里的所有设备，是指物联网中所有可以接受终端设备控制的设备。Here, the panoramic images of all devices are images including all devices captured by the panoramic camera in advance, which can be stored in the local or cloud server of the terminal device for calling by the terminal device. The panoramic camera here may be a panoramic camera configured on the terminal device, may also be a panoramic camera externally connected to the terminal device, or may be an independent panoramic camera capable of communicating with the terminal device. All devices here refer to all devices in the Internet of Things that can be controlled by terminal devices.
在步骤203中，查询关联关系数据库中对应设备图像的设备信息。In step 203, the device information corresponding to the device image in the relational database is queried.
在本实施例中，基于步骤202中得到的设备图像，也即得到了目前终端设备所采集的图像在全景图像中的映射图像，可以通过预设的关联关系数据库查询与该映射图像具有关联关系的设备信息，以便与该具有关联关系的设备信息进行交互。In this embodiment, based on the device image obtained in step 202, that is, the mapping image of the image captured by the current terminal device in the panoramic image is obtained, the association relationship with the mapping image can be queried through the preset association relationship database device information in order to interact with the associated device information.
这里的设备信息，可以包括但不限于设备的名称、图标、型号、连接方式以及用于验证身份的授权信息等。The device information here may include, but is not limited to, the device's name, icon, model, connection method, and authorization information for identity verification.
在本实施例的一些可选实现方式中，如图3所示，可以基于获取关联关系数据库的方法300得到关联关系数据库。In some optional implementation manners of this embodiment, as shown in FIG. 3 , the association relationship database may be obtained based on the method 300 for obtaining the association relationship database.
在获取关联关系数据库的方法的示意性流程图300中，包括：In the schematic flow chart 300 of the method for obtaining an association relationship database, it includes:
在步骤301中，获取各个设备的设备信息。In step 301, device information of each device is acquired.
在本实现方式中，可以通过现有技术中或未来发展的技术中的获取设备信息的方法来获取各个设备的设备信息。例如，可以通过扫描设备表面的设备序列号、二维码以及设备图标等信息载体获取设备信息，还可以通过无线通信连接从存有设备信息的装置读取设备信息，示例性的，可以通过近场通信技术这一无线通信连接技术读取设备信息。In this implementation manner, the device information of each device may be obtained through a method for obtaining device information in the existing technology or in a technology developed in the future. For example, the device information can be obtained by scanning information carriers such as the device serial number, two-dimensional code, and device icon on the surface of the device, and the device information can also be read from a device storing the device information through a wireless communication connection. Field communication technology, a wireless communication connection technology, reads device information.
应当理解，在获取各个设备的设备信息之后，可以将各个设备的设备信息存入设备信息的集合，例如设备信息列表或设备信息数据库中，以便在后续需要使用设备信息时进行调用。It should be understood that after the device information of each device is acquired, the device information of each device can be stored in a device information set, such as a device information list or a device information database, so as to be called when the device information needs to be used later.
在步骤302中，获取包括所有设备的全景图像。In step 302, a panoramic image including all devices is acquired.
在本实现方式中，包括所有设备的全景图像可以为预先通过全景摄像机拍摄的包括所有设备的图像，可以存储于终端设备本地或云端服务器，以供终端设备调用。这里的所有设备，是指物联网中所有可以接受终端设备控制的设备。In this implementation manner, the panoramic image including all devices may be an image including all devices captured by a panoramic camera in advance, and may be stored locally on the terminal device or on a cloud server for calling by the terminal device. All devices here refer to all devices in the Internet of Things that can be controlled by terminal devices.
在步骤303中，设定全景图像中的各设备图像与各设备信息的关联关系。In step 303, an association relationship between each device image and each device information in the panoramic image is set.
在本实现方式中，可以通过用户输入或读取预设的关联关系来设定全景图像中的各设备图像与各设备信息的关联关系。In this implementation manner, the association relationship between each device image and each device information in the panoramic image may be set through user input or reading of a preset association relationship.
可选地，设定全景图像中的各设备图像与各设备信息的关联关系可以包括：响应于接收用户对单个设备图像和设备信息库中的单个设备信息的选中操作，生成单个设备图像与单个设备信息的关联关系。Optionally, setting the association relationship between each device image and each device information in the panoramic image may include: generating a single device image and a single Association relationship of device information.
这里的响应于接收用户对单个设备图像和设备信息库中的单个设备信息的选中操作，生成单个设备图像与单个设备信息的关联关系，可以为用户先后选择单个设备图像和单个设备信息进行配对，也可以以为用户同时选择单个设备图像和单个设备信息进行配对，还可以为用户先后选择单个设备信息和单个设备图像进行配对，本申请对此不做限定。Here, in response to receiving the user's selection operation on a single device image and a single device information in the device information library, an association relationship between a single device image and a single device information is generated, and a single device image and a single device information can be selected successively for the user to pair, It is also possible to select a single device image and a single device information for the user to be paired at the same time, and it is also possible for the user to select a single device information and a single device image successively for pairing, which is not limited in this application.
示例性的，运行用于物联网设备的可视化操作方法的终端设备，可以响应于接收用户对设备图像的选中操作，呈现设备信息库中的设备信息，响应于接收用户对呈现的设备信息的选择操作，生成被选择的设备信息与被选中的设备图像的关联关系，将所述关联关系添加至关联关系数据库。Exemplarily, the terminal device running the visualized operation method for the Internet of Things device may respond to receiving the user's selection operation on the device image, present the device information in the device information library, and respond to receiving the user's selection of the presented device information Operation, generate an association relationship between the selected equipment information and the selected equipment image, and add the association relationship to the association relationship database.
在步骤304中，将关联关系添加至关联关系数据库。In step 304, the association relationship is added to the association relationship database.
在本实施例的一些可选实现方式中，在上述响应于接收用户对单个设备图像和设备信息库中的单个设备信息的选中操作，生成单个设备图像与单个设备信息的关联关系时，为了后续提高建立通信连接的安全性，设备信息中还可以包括授权信息；上述获取关联关系数据库的方法还可以包括：基于被选中的设备信息中的授权信息，将与被选中的设备信息对应的设备接入物联网，以便后续建立通信连接。In some optional implementations of this embodiment, when generating the association relationship between a single device image and a single device information in response to receiving the user's selection operation on a single device image and a single device information in the device information library, for subsequent To improve the security of establishing a communication connection, the device information may also include authorization information; the method for obtaining the association relationship database may also include: based on the authorization information in the selected device information, connecting the device corresponding to the selected device information into the Internet of Things for the subsequent establishment of communication connections.
返回图2，在步骤204中，与查询的设备信息对应的设备建立通信连接。Returning to FIG. 2, in step 204, the device corresponding to the queried device information establishes a communication connection.
在本实施例中，基于步骤203中查询得到的设备信息，可以找到与该设备信息对应的设备，并进一步与该设备建立通信连接。In this embodiment, based on the device information queried in step 203, the device corresponding to the device information can be found, and a communication connection can be further established with the device.
在步骤205中，呈现设备图像和建立通信连接的设备的操作界面。In step 205, the image of the device and the operation interface of the device for which the communication connection is established are presented.
在本实施例中，基于在步骤204中与查询的设备信息对应的设备建立了通信连接，可以直接从建立了通信连接的设备处获取操作界面，之后呈现操作界面，也可以从建立了通信连接的设备处获取设备中存储的用于生成操作界面的操作列表信息，之后基于操作列表信息呈现操作界面。In this embodiment, based on the establishment of a communication connection with the device corresponding to the queried device information in step 204, the operation interface can be obtained directly from the device with the establishment of the communication connection, and then the operation interface can be presented. The device acquires the operation list information stored in the device for generating the operation interface, and then presents the operation interface based on the operation list information.
在这里，呈现操作界面的方式可以为现有技术或未来发展的技术中的呈现方式，本申请对此不作限定。示例性的，呈现操作界面的方式可以通过终端设备的显示屏幕呈现，也可以通过全息影像呈现。Here, the manner of presenting the operation interface may be a presenting manner in the existing technology or a technology developed in the future, which is not limited in the present application. Exemplarily, the manner of presenting the operation interface may be presented through a display screen of the terminal device, or may be presented through a holographic image.
在本实施例的一些可选实现方式中，上述用于物联网设备的可视化操作方法还包括：接收用户对操作界面的输入操作；将输入操作对应的操作指令发送至建立通信连接的设备以控制建立通信连接的设备。In some optional implementations of this embodiment, the above-mentioned visualized operation method for IoT devices further includes: receiving user input operations on the operation interface; sending operation instructions corresponding to the input operations to the device establishing a communication connection to control A device that establishes a communication link.
以下参考图4a、图4b和图4c说明根据本申请实施例的运行用于物联网设备的可视化操作方法的一个应用场景。An application scenario of running a visualized operation method for an Internet of Things device according to an embodiment of the present application will be described below with reference to FIG. 4a, FIG. 4b and FIG. 4c.
在图4a中，用户通过手持式终端设备410扫描物联网设备420表面的二维码421，获取设备信息，并将设备信息添加至设备信息库。In FIG. 4 a , the user scans the QR code 421 on the surface of the Internet of Things device 420 through the handheld terminal device 410 to obtain device information and add the device information to the device information database.
之后在图4b中，用户使用全景摄像机拍摄包括所有物联网设备的全景图像，并将全景图像中的各个物联网设备的图像与设备信息库中的设备信息建立关联关系，得到物联网设备的可视化场景图。Then in Figure 4b, the user uses a panoramic camera to take a panoramic image including all IoT devices, and establishes an association relationship between the images of each IoT device in the panoramic image and the device information in the device information database to obtain the visualization of the IoT device scene graph.
之后在图4c中，用户通过手持终端设备410查看实际场景中的设备430，终端设备410在自身的显示屏上呈现可视化场景图中的设备430，并在该场景图中显示设备430的操作界面440，以便用户选择操作界面中的操作选项来完成对物联网设备的操控。Then in FIG. 4c, the user views the device 430 in the actual scene through the handheld terminal device 410, and the terminal device 410 presents the device 430 in the visualized scene graph on its own display screen, and displays the operation interface of the device 430 in the scene graph 440, so that the user selects an operation option in the operation interface to complete the manipulation of the IoT device.
本申请上述实施例提供的用于物联网设备的可视化操作方法，通过响应于接收到采集指令而采集设备的图像，可向用户提供设备的可视化图像操作界面，简化了用户获取设备的操作界面的步骤，且向用户呈现的为可视化图像操作界面，呈现的方式更为直观。The visualized operation method for the Internet of Things device provided by the above-mentioned embodiments of the present application can provide the user with a visualized image operation interface of the device by collecting the image of the device in response to receiving a collection instruction, which simplifies the process for the user to obtain the operation interface of the device. Steps, and what is presented to the user is a visual image operation interface, and the presentation method is more intuitive.
进一步参考图5，其示出了按照本申请的实施例的用于物联网设备的可视化操作装置的一个结构示意图。Further referring to FIG. 5 , it shows a schematic structural diagram of a visual operation device for an Internet of Things device according to an embodiment of the present application.
如图5所示，用于图1所示的终端设备101、102的用于物联网设备的可视化操作装置500可以包括：图像采集单元510、图像映射单元520、信息查询单元530、通信连接单元540和界面呈现单元550。As shown in FIG. 5 , the visualized operation device 500 for Internet of Things devices used for the terminal devices 101 and 102 shown in FIG. 1 may include: an image acquisition unit 510 , an image mapping unit 520 , an information query unit 530 , and a communication connection unit 540 and an interface presentation unit 550.
其中，图像采集单元510，配置用于响应于接收到采集指令，采集已接入物联网的设备的图像；图像映射单元520，配置用于获取图像在可以包括所有设备的全景图像中的设备图像；信息查询单元530，配置用于查询关联关系数据库中对应设备图像的设备信息；通信连接单元540，配置用于与查询的设备信息对应的设备建立通信连接；界面呈现单元550，配置用于呈现设备图像和建立通信连接的设备的操作界面。Wherein, the image collection unit 510 is configured to collect images of devices that have been connected to the Internet of Things in response to receiving a collection instruction; the image mapping unit 520 is configured to obtain images of devices in a panoramic image that may include all devices The information query unit 530 is configured to query the device information corresponding to the device image in the association relationship database; the communication connection unit 540 is configured to establish a communication connection with the device corresponding to the queried device information; the interface presentation unit 550 is configured to present Image of the device and the operator interface of the device with which the communication connection is established.
在本实施例的一些可选实现方式中，界面呈现单元550可以包括：信息获取子单元551，配置用于获取建立通信连接的设备的操作列表信息；界面呈现子单元552，配置用于基于操作列表信息，呈现操作界面。In some optional implementations of this embodiment, the interface presentation unit 550 may include: an information acquisition subunit 551 configured to acquire operation list information of devices establishing a communication connection; an interface presentation subunit 552 configured to List information and present the operation interface.
在本实施例的一些可选实现方式中，装置还可以包括：操作接收单元560，配置用于接收用户对操作界面的输入操作；设备控制单元570，配置用于将输入操作对应的操作指令发送至建立通信连接的设备以控制建立通信连接的设备。In some optional implementations of this embodiment, the device may further include: an operation receiving unit 560 configured to receive user input operations on the operation interface; a device control unit 570 configured to send an operation instruction corresponding to the input operation To the device establishing the communication connection to control the device establishing the communication connection.
进一步参考图6，其示出了按照本申请的实施例的获取关联关系数据库的装置的一个结构示意图。Further refer to FIG. 6 , which shows a schematic structural diagram of an apparatus for acquiring an association relationship database according to an embodiment of the present application.
应当理解，用于物联网设备的可视化操作装置500中记载的诸单元或模块与参考图2描述的方法中的各个步骤相对应。由此，上文针对方法描述的操作和特征同样适用于系统500及其中包含的单元，在此不再赘述。It should be understood that the units or modules recorded in the visualized operation apparatus 500 for Internet of Things devices correspond to each step in the method described with reference to FIG. 2 . Therefore, the operations and features described above for the method are also applicable to the system 500 and the units contained therein, and will not be repeated here.
如图6所示，上述信息查询单元所调用的关联关系数据库基于获取关联关系数据库的装置得到，获取关联关系数据库的装置600可以包括：信息获取单元610、图像获取单元620、关系设定单元630和数据添加单元640。As shown in Figure 6, the relationship database called by the above-mentioned information query unit is obtained based on the device for obtaining the relationship database, and the device 600 for obtaining the relationship database may include: an information acquisition unit 610, an image acquisition unit 620, and a relationship setting unit 630 and data adding unit 640.
其中，信息获取单元610，配置用于获取各个设备的设备信息；图像获取单元620，配置用于获取可以包括所有设备的全景图像；关系设定单元630，配置用于设定全景图像中的各设备图像与各设备信息的关联关系；数据添加单元640，配置用于将关联关系添加至关联关系数据库。Among them, the information obtaining unit 610 is configured to obtain device information of each device; the image obtaining unit 620 is configured to obtain a panoramic image that may include all devices; the relationship setting unit 630 is configured to set each The association relationship between the device image and each device information; the data adding unit 640 is configured to add the association relationship to the association relationship database.
在本实施例的一些可选实现方式中，关系设定单元630可以包括：操作接收子单元631，配置用于接收用户对单个设备图像和设备信息库中的单个设备信息的选中操作；关系生成子单元632，配置用于响应于接收用户对单个设备图像和设备信息库中的单个设备信息的选中操作，生成单个设备图像与单个设备信息的关联关系。In some optional implementations of this embodiment, the relationship setting unit 630 may include: an operation receiving subunit 631 configured to receive a user's selection operation on a single device image and a single device information in the device information library; The subunit 632 is configured to generate an association relationship between a single device image and a single device information in response to receiving a user's selection operation on a single device image and a single device information in the device information library.
在本实施例的一些可选实现方式中，信息获取单元获取的设备信息中可以包括授权信息；以及装置还可以包括：接入物联网单元(图中未示出)，配置用于基于被选中的设备信息中的授权信息，将与被选中的设备信息对应的设备接入物联网。In some optional implementations of this embodiment, the device information acquired by the information acquisition unit may include authorization information; and the device may further include: an Internet of Things access unit (not shown in the figure), configured to The authorization information in the device information will connect the device corresponding to the selected device information to the Internet of Things.
在本申请的实施例中，还可以提供一种终端设备，终端设备包括如上所述的用于物联网设备的可视化操作装置。In an embodiment of the present application, a terminal device may also be provided, and the terminal device includes the above-mentioned visualized operation device for an Internet of Things device.
应当理解，获取关联关系数据库的装置600中记载的诸单元或模块与参考图3描述的方法中的各个步骤相对应。由此，上文针对方法描述的操作和特征同样适用于装置600及其中包含的单元，在此不再赘述。It should be understood that the various units or modules recorded in the apparatus 600 for obtaining an association relationship database correspond to each step in the method described with reference to FIG. 3 . Therefore, the operations and features described above for the method are also applicable to the device 600 and the units contained therein, and will not be repeated here.
上述用于物联网设备的可视化操作装置500和上述获取关联关系数据库的装置600可以预先设置在诸如图1所示的终端设备101、102中，也可以通过下载等方式而加载到终端设备中。用于物联网设备的可视化操作装置500和上述获取关联关系数据库的装置600中的相应单元可以与终端设备中的单元相互配合以实现用于物联网设备的可视化操作方法。例如，图像映射单元520可以与终端设备的处理器协同工作，以获取采集的图像在包括所有设备的全景图像中的设备图像。界面呈现单元550可以与终端设备的屏幕部分协同工作，以将屏幕部分作为呈现操作界面的载体。获取关联关系数据库的装置600可以与终端设备的存储器协同工作，以将关联关系数据库存储在存储器中。The above-mentioned visualized operation device 500 for Internet of Things devices and the above-mentioned device 600 for obtaining an association relationship database can be pre-set in terminal devices 101 and 102 such as shown in FIG. The corresponding units in the visualized operation apparatus 500 for Internet of Things devices and the above-mentioned apparatus 600 for obtaining an association relationship database can cooperate with units in the terminal device to implement a visualized operation method for Internet of Things devices. For example, the image mapping unit 520 may work in cooperation with the processor of the terminal device to acquire the device image of the captured image in the panoramic image including all devices. The interface presentation unit 550 may work in cooperation with the screen part of the terminal device, so as to use the screen part as a carrier for presenting the operation interface. The apparatus 600 for acquiring the association relationship database may work in cooperation with the memory of the terminal device, so as to store the association relationship database in the memory.
附图中的流程图和框图，图示了按照本申请各种实施例的系统、方法和计算机程序产品的可能实现的体系架构、功能和操作。在这点上，流程图或框图中的每个方框可以代表一个模块、程序段、或代码的一部分，所述模块、程序段、或代码的一部分包含一个或多个用于实现规定的逻辑功能的可执行指令。也应当注意，在有些作为替换的实现中，方框中所标注的功能也可以以不同于附图中所标注的顺序发生。例如，两个接连地表示的方框实际上可以基本并行地执行，它们有时也可以按相反的顺序执行，这依所涉及的功能而定。也要注意的是，框图和/或流程图中的每个方框、以及框图和/或流程图中的方框的组合，可以用执行规定的功能或操作的专用的基于硬件的系统来实现，或者可以用专用硬件与计算机指令的组合来实现。The flowchart and block diagrams in the Figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present application. In this regard, each block in a flowchart or block diagram may represent a module, program segment, or portion of code that contains one or more logic devices for implementing the specified Executable instructions for a function. It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or they may sometimes be executed in the reverse order, depending upon the functionality involved. It should also be noted that each block of the block diagrams and/or flowchart illustrations, and combinations of blocks in the block diagrams and/or flowchart illustrations, can be implemented by a dedicated hardware-based system that performs the specified functions or operations , or may be implemented by a combination of dedicated hardware and computer instructions.
描述于本申请实施例中所涉及到的单元可以通过软件的方式实现，也可以通过硬件的方式来实现。所描述的单元也可以设置在处理器中，例如，可以描述为：一种处理器包括图像采集单元、图像映射单元、信息查询单元、通信连接单元和界面呈现单元。其中，这些单元的名称在某种情况下并不构成对该单元本身的限定，例如，图像采集单元还可以被描述为“响应于接收到采集指令，采集已接入物联网的设备的图像的单元”。The units involved in the embodiments described in the present application may be implemented by means of software or by means of hardware. The described units can also be set in a processor, for example, it can be described as: a processor includes an image acquisition unit, an image mapping unit, an information query unit, a communication connection unit and an interface presentation unit. Wherein, the names of these units do not constitute a limitation to the unit itself under certain circumstances. For example, the image acquisition unit can also be described as "acquisition of an image of a device that has been connected to the Internet of Things in response to receiving an acquisition instruction." unit".
作为另一方面，本申请还提供了一种非易失性计算机存储介质，该非易失性计算机存储介质可以是上述实施例中所述装置中所包含的非易失性计算机存储介质；也可以是单独存在，未装配入终端中的非易失性计算机存储介质。上述非易失性计算机存储介质存储有一个或者多个程序，当所述一个或者多个程序被一个设备执行时，使得所述设备：响应于接收到采集指令，采集已接入物联网的设备的图像；获取采集的图像在包括所有设备的全景图像中的设备图像；查询关联关系数据库中对应设备图像的设备信息；与查询的设备信息对应的设备建立通信连接；呈现设备图像和建立通信连接的设备的操作界面。As another aspect, the present application also provides a non-volatile computer storage medium, which may be the non-volatile computer storage medium contained in the device described in the above embodiments; It may be a non-volatile computer storage medium that exists independently and is not assembled into the terminal. The above-mentioned non-volatile computer storage medium stores one or more programs, and when the one or more programs are executed by a device, the device: responds to receiving a collection instruction, collects the device connected to the Internet of Things image; obtain the device image of the collected image in the panoramic image including all devices; query the device information corresponding to the device image in the relational database; establish a communication connection with the device corresponding to the queried device information; present the device image and establish a communication connection The user interface of the device.
以上描述仅为本申请的较佳实施例以及对所运用技术原理的说明。本领域技术人员应当理解，本申请中所涉及的发明范围，并不限于上述技术特征的特定组合而成的技术方案，同时也应涵盖在不脱离所述发明构思的情况下，由上述技术特征或其等同特征进行任意组合而形成的其它技术方案。例如上述特征与本申请中公开的(但不限于)具有类似功能的技术特征进行互相替换而形成的技术方案。The above description is only a preferred embodiment of the present application and an illustration of the applied technical principle. Those skilled in the art should understand that the scope of the invention involved in this application is not limited to the technical solution formed by the specific combination of the above-mentioned technical features, but should also cover the technical solution formed by the above-mentioned technical features without departing from the inventive concept. Other technical solutions formed by any combination of or equivalent features thereof. For example, a technical solution formed by replacing the above-mentioned features with technical features with similar functions disclosed in (but not limited to) this application.
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201610859055.7ACN106445298A (en) | 2016-09-27 | 2016-09-27 | Visual operation method and device for internet-of-things device |
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201610859055.7ACN106445298A (en) | 2016-09-27 | 2016-09-27 | Visual operation method and device for internet-of-things device |
| Publication Number | Publication Date |
|---|---|
| CN106445298Atrue CN106445298A (en) | 2017-02-22 |
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201610859055.7APendingCN106445298A (en) | 2016-09-27 | 2016-09-27 | Visual operation method and device for internet-of-things device |
| Country | Link |
|---|---|
| CN (1) | CN106445298A (en) |
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109068161A (en)* | 2018-07-25 | 2018-12-21 | 三星电子（中国）研发中心 | A kind of equipment connection method for building up and device |
| CN110945863A (en)* | 2018-05-23 | 2020-03-31 | 华为技术有限公司 | A photographing method and terminal device |
| CN111314398A (en)* | 2018-12-11 | 2020-06-19 | 阿里巴巴集团控股有限公司 | Equipment control method, network distribution method, system and equipment |
| CN111722770A (en)* | 2020-07-20 | 2020-09-29 | 中国人民武装警察部队工程大学 | An intelligent human-computer interaction infinite network control module |
| CN113326849A (en)* | 2021-07-20 | 2021-08-31 | 广东魅视科技股份有限公司 | Visual data acquisition method and system |
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102073308A (en)* | 2010-10-26 | 2011-05-25 | 华南理工大学 | Family security remote monitoring system and method based on Internet of things |
| CN103873959A (en)* | 2012-12-13 | 2014-06-18 | 联想(北京)有限公司 | Control method and electronic device |
| US20140244001A1 (en)* | 2013-02-25 | 2014-08-28 | Qualcomm Incorporated | Controlling many different devices from a smart controller |
| US20140257532A1 (en)* | 2013-03-05 | 2014-09-11 | Electronics And Telecommunications Research Institute | Apparatus for constructing device information for control of smart appliances and method thereof |
| CN105204742A (en)* | 2015-09-28 | 2015-12-30 | 小米科技有限责任公司 | Control method and device of electronic equipment and terminal |
| CN105760106A (en)* | 2016-03-08 | 2016-07-13 | 网易（杭州）网络有限公司 | Interaction method and interaction device of intelligent household equipment |
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102073308A (en)* | 2010-10-26 | 2011-05-25 | 华南理工大学 | Family security remote monitoring system and method based on Internet of things |
| CN103873959A (en)* | 2012-12-13 | 2014-06-18 | 联想(北京)有限公司 | Control method and electronic device |
| US20140244001A1 (en)* | 2013-02-25 | 2014-08-28 | Qualcomm Incorporated | Controlling many different devices from a smart controller |
| US20140257532A1 (en)* | 2013-03-05 | 2014-09-11 | Electronics And Telecommunications Research Institute | Apparatus for constructing device information for control of smart appliances and method thereof |
| CN105204742A (en)* | 2015-09-28 | 2015-12-30 | 小米科技有限责任公司 | Control method and device of electronic equipment and terminal |
| CN105760106A (en)* | 2016-03-08 | 2016-07-13 | 网易（杭州）网络有限公司 | Interaction method and interaction device of intelligent household equipment |
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110945863A (en)* | 2018-05-23 | 2020-03-31 | 华为技术有限公司 | A photographing method and terminal device |
| CN110945863B (en)* | 2018-05-23 | 2021-05-11 | 华为技术有限公司 | Photographing method and terminal equipment |
| US11409493B2 (en) | 2018-05-23 | 2022-08-09 | Huawei Technologies Co., Ltd. | Photographing method and terminal device |
| US11435975B2 (en) | 2018-05-23 | 2022-09-06 | Huawei Technologies Co., Ltd. | Preview display method based on multi-angle and communication system |
| US11797255B2 (en) | 2018-05-23 | 2023-10-24 | Huawei Technologies Co., Ltd. | Preview display method based on multi-angle and communication system |
| CN109068161A (en)* | 2018-07-25 | 2018-12-21 | 三星电子（中国）研发中心 | A kind of equipment connection method for building up and device |
| CN109068161B (en)* | 2018-07-25 | 2021-04-16 | 三星电子（中国）研发中心 | Equipment connection establishing method and device |
| CN111314398A (en)* | 2018-12-11 | 2020-06-19 | 阿里巴巴集团控股有限公司 | Equipment control method, network distribution method, system and equipment |
| CN111722770A (en)* | 2020-07-20 | 2020-09-29 | 中国人民武装警察部队工程大学 | An intelligent human-computer interaction infinite network control module |
| CN113326849A (en)* | 2021-07-20 | 2021-08-31 | 广东魅视科技股份有限公司 | Visual data acquisition method and system |
| Publication | Publication Date | Title |
|---|---|---|
| US10937249B2 (en) | Systems and methods for anchoring virtual objects to physical locations | |
| EP3200417B1 (en) | Method and device for information interaction and association between human biological feature data and account | |
| US20160306505A1 (en) | Computer-implemented methods and systems for automatically creating and displaying instant presentations from selected visual content items | |
| US10152625B2 (en) | Method for providing control function using fingerprint sensor and electronic device thereof | |
| CN110868763B (en) | Method, device and system for establishing a Bluetooth connection | |
| US20160337290A1 (en) | Message Push Method and Apparatus | |
| US8965844B2 (en) | Apparatuses and methods for sharing contents | |
| WO2020061033A1 (en) | Cross-platform digital content storage and sharing system | |
| CN106445298A (en) | Visual operation method and device for internet-of-things device | |
| US12112435B1 (en) | Collaboration spaces in extended reality conference sessions | |
| WO2015077259A1 (en) | Image sharing for online collaborations | |
| CN110390641B (en) | Image desensitizing method, electronic device and storage medium | |
| US20150160824A1 (en) | Systems and method for mobile social network interactions | |
| TWI713327B (en) | Message sending method and device and electronic equipment | |
| CN110472558B (en) | Image processing method and device | |
| US20230168786A1 (en) | Methods and Systems for Location-Based Accessing of Predesignated Data Payloads Using Extended Reality | |
| CN103218132A (en) | Method for processing pictures and device | |
| CN105005599A (en) | A photo sharing method and mobile terminal | |
| CN110536075A (en) | Video generation method and device | |
| WO2016161890A1 (en) | Method of uploading data and device thereof | |
| CN112764857A (en) | Information processing method and device and electronic equipment | |
| EP2990947A1 (en) | Method and apparatus for backing up data and electronic device | |
| CN108470377A (en) | AR sees room device | |
| JP2014222374A (en) | Planogram information generation device, planogram information generation method, program, planogram reproduction system and planogram reproduction device | |
| CN114445500B (en) | Augmented reality scene construction method, device, terminal equipment and storage medium |
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication | Application publication date:20170222 |