技术领域technical field
本发明涉及增强现实领域，尤其涉及一种增强现实(Augmented Reality，AR)反馈方法及设备。The present invention relates to the field of augmented reality, in particular to an augmented reality (Augmented Reality, AR) feedback method and device.
背景技术Background technique
AR技术是一种将真实世界信息和虚拟世界信息无缝集成的技术，是通过传感、计算和图形等科学技术将原本在现实世界的一定时间空间范围内很难体验到的视觉、声音、味道和触觉等实体信息，模拟仿真后再叠加，将虚拟的信息应用到真实世界，被人类感官所感知，从而达到超越现实的感官体验。AR technology is a technology that seamlessly integrates real-world information and virtual world information. It uses technologies such as sensing, computing, and graphics to integrate vision, sound, Physical information such as taste and touch are simulated and then superimposed, and the virtual information is applied to the real world, which is perceived by human senses, so as to achieve a sensory experience beyond reality.
目前，可利用增强现实技术反馈设备所处位置以及故障等状态信息，目前反馈的设备状态信息主要通过诸如在显示界面上进行文字表达等机器视觉展示方式进行反馈。例如应用增强现实技术反馈中央处理器(Central Processing Unit，CPU)温度过高故障时，一般是将温度值信息直接叠加在图形化界面上进行显示，如图1所示。At present, augmented reality technology can be used to feed back status information such as the location of the device and faults. Currently, the feedback of device status information is mainly fed back through machine vision display methods such as text expressions on the display interface. For example, when the augmented reality technology is used to feed back the overheating fault of the central processing unit (Central Processing Unit, CPU), generally the temperature value information is directly superimposed on the graphical interface for display, as shown in FIG. 1 .
上述进行设备状态反馈的方式中，通过机器视觉反馈设备状态，单纯的通过文字信息进行反馈，并不能使用户快速直观的获知设备状态。In the above-mentioned method of device state feedback, the device state is fed back through machine vision, and the user cannot know the device state quickly and intuitively by simply giving feedback through text information.
发明内容Contents of the invention
本发明实施例提供一种增强现实反馈方法及设备，以方便用户快速直观的获知设备状态。Embodiments of the present invention provide an augmented reality feedback method and device, so that users can quickly and intuitively know device status.
第一方面，提供一种增强现实反馈方法，在该方法中预先设定针对设备状态数据进行增强现实感官反馈的规则，增强现实设备获取到物理设备的设备状态数据后，根据预设规则以及所述获取的设备状态数据，进行增强现实感官反馈。所述增强现实感官反馈可以是视觉增强反馈、触觉增强反馈、听觉增强反馈和嗅觉增强反馈中一种，也可以是多种，故本发明实施例中增强现实设备进行增强现实感官反馈，反馈方式多样丰富，并能够基于获取的设备状态数据对设备状态进行更形象的反馈，能够方便用户快速直观的获知设备状态。In the first aspect, an augmented reality feedback method is provided, in which the rules for augmented reality sensory feedback for device state data are preset, and after the augmented reality device obtains the device state data of the physical device, it The acquired device status data is used to provide augmented reality sensory feedback. The augmented reality sensory feedback can be one of visual augmented feedback, tactile augmented feedback, auditory augmented feedback, and olfactory augmented feedback, and can also be multiple. Therefore, in the embodiment of the present invention, the augmented reality device performs augmented reality sensory feedback. The feedback method It is diverse and rich, and can provide more vivid feedback on the device status based on the obtained device status data, which can facilitate users to quickly and intuitively know the device status.
本发明实施例中，所述预设规则中可包括设备状态数据类型、设备状态数据级别和增强现实感官反馈之间的对应关系，所述设备状态数据类型包括但不限于温度异常(温度过高或过低)、漏水、漏电、压力异常(压力过高或过低)、吞吐量过大、设备断网、丢包率过大等。所述设备状态数据级别一般包括至少两种级别。设备状态数据的每一级别对应固定的增强现实感官反馈，不同级别具有不同的增强现实感官反馈。每一类型以及每一级别的设备状态数据对应的固定的增强现实感官反馈，可以是视觉增强反馈、触觉增强反馈、听觉增强反馈和嗅觉增强反馈中一种，也可以是多种。In the embodiment of the present invention, the preset rules may include the correspondence between device status data types, device status data levels, and augmented reality sensory feedback. The device status data types include but are not limited to temperature abnormalities (too high or too low), water leakage, electric leakage, abnormal pressure (too high or too low pressure), excessive throughput, equipment disconnection, excessive packet loss rate, etc. The device status data levels generally include at least two levels. Each level of device state data corresponds to a fixed augmented reality sensory feedback, and different levels have different augmented reality sensory feedback. The fixed augmented reality sensory feedback corresponding to each type and level of device state data may be one of visual enhancement feedback, tactile enhancement feedback, auditory enhancement feedback, and olfactory enhancement feedback, or multiple types.
本发明实施例中所述设备状态数据包括但不限于吞吐量状态数据、网络连接状态数据、温度状态数据、湿度状态数据、压力状态数据和电力状态数据。增强现实设备获取的设备状态数据可以是以上所列设备状态数据中的一种或者多种。The device state data in the embodiment of the present invention includes but not limited to throughput state data, network connection state data, temperature state data, humidity state data, pressure state data and power state data. The device status data acquired by the augmented reality device may be one or more of the device status data listed above.
本发明实施例中进行增强现实感官反馈时，可根据所述获取的设备状态数据，确定获取的所述设备状态数据所属类型以及所属级别，并根据确定的设备状态数据所属类型以及所属级别查找预设规则，确定对应获取的所述设备状态数据所属类型以及所属级别的增强现实感官反馈，对获取的所述设备状态数据进行对应确定的设备状态数据所属类型以及所属级别的增强现实感官反馈，以实现对不同的设备状态数据进行不同的增强现实感官反馈。In the embodiment of the present invention, when performing augmented reality sensory feedback, the type and level of the acquired device state data can be determined according to the acquired device state data, and the predetermined type and level of the device state data can be searched according to the determined type and level of the device state data. Set rules to determine the type and level of augmented reality sensory feedback corresponding to the acquired device status data, and perform augmented reality sensory feedback on the acquired device status data corresponding to the type and level of the device status data, to Realize different augmented reality sensory feedback for different device status data.
本发明实施例中确定的对应所述所述获取的设备状态数据所属类型以及所属级别的增强现实感官反馈可以是视觉增强反馈、触觉增强反馈、听觉增强反馈和嗅觉增强反馈中的一种，也可以是多种。The augmented reality sensory feedback corresponding to the type and level of the acquired device state data determined in the embodiment of the present invention may be one of visual enhancement feedback, tactile enhancement feedback, auditory enhancement feedback, and olfactory enhancement feedback, or Can be many.
若确定的对应所述获取的设备状态数据所属类型以及所属级别的增强现实感官反馈中包括视觉增强反馈，则可从预设的图形库中调用匹配所述确定的设备状态数据所属类型以及所属级别的视觉增强反馈图形，通过图像显示装置呈现所述调用的视觉增强反馈图形，对所述获取的设备状态数据进行视觉增强反馈，达到从颜色、形状、透明度以及物理现象等对设备状态数据进行可视化增强的效果。If the determined augmented reality sensory feedback corresponding to the type and level of the acquired device status data includes visual enhancement feedback, it can be called from a preset graphics library to match the determined type and level of the device status data. The visually enhanced feedback graphics, the visually enhanced feedback graphics of the call are presented through the image display device, and the visually enhanced feedback is performed on the acquired equipment state data, so as to visualize the equipment state data from the aspects of color, shape, transparency and physical phenomena, etc. enhanced effect.
若确定的对应所述获取的设备状态数据所属类型以及所属级别的增强现实感官反馈中包括听觉增强反馈，则可从预设的声音库文件中调用匹配所述确定的设备状态数据所属类型以及所属级别的声觉增强反馈音频文件，通过音频设备播放所述调用的音频文件，对所述获取的设备状态数据进行声觉增强反馈，达到用风声、水声、火声等声音对设备状态数据进行模拟增强的效果。If the determined augmented reality sensory feedback corresponding to the type and level of the acquired device state data includes auditory enhancement feedback, then the audio system that matches the determined type and level of the device state data can be called from the preset sound library file. Level acoustic enhancement feedback audio files, play the called audio files through the audio equipment, and perform acoustic enhancement feedback on the acquired equipment status data, so as to use sounds such as wind, water, and fire to perform audio feedback on the equipment status data. Simulates the effect of augmentation.
若确定的对应所述获取的设备状态数据所属类型以及所属级别的增强现实感官反馈中包括触觉增强反馈，则可控制加热设备的温度为匹配所述确定的设备状态数据所属类型以及所属级别的触觉增强反馈的温度，对所述获取的设备状态数据进行触觉增强反馈，达到通过温度触感对设备状态数据进行模拟增强的效果。If the determined augmented reality sensory feedback corresponding to the type and level of the acquired device state data includes tactile enhancement feedback, the temperature of the heating device may be controlled to match the tactile sense of the determined type and level of the device state data. The temperature of the feedback is enhanced, and the tactile enhancement feedback is performed on the acquired device state data, so as to achieve the effect of simulated enhancement of the device state data through temperature and tactile sensation.
若确定的对应所述获取的设备状态数据所属类型以及所属级别的增强现实感官反馈中包括嗅觉增强反馈，则可控制气味散发设备散发匹配所述确定的设备状态数据所属类型以及所属级别的嗅觉增强反馈的气味，对所述获取的设备状态数据进行嗅觉增强反馈，达到用气味对设备状态数据进行模拟增强的效果。If the determined augmented reality sensory feedback corresponding to the type and level of the acquired device state data includes olfactory enhancement feedback, the scent emitting device can be controlled to emit olfactory enhancement that matches the determined type and level of the device state data. Feedback smell, the sense of smell enhancement feedback is performed on the acquired device state data, so as to achieve the effect of using smell to simulate and enhance the device state data.
第二方面，提供一种增强现实反馈设备，该增强现实反馈设备具备实现上述增强现实反馈方法的全部功能，所述功能可以通过硬件实现，也可以通过硬件执行相应的软件实现。所述硬件或软件包括一个或多个与上述功能相对应的模块。所述模块可以是软件和/或硬件。In a second aspect, an augmented reality feedback device is provided. The augmented reality feedback device has all the functions for implementing the above augmented reality feedback method, and the functions may be realized by hardware, or may be realized by executing corresponding software on the hardware. The hardware or software includes one or more modules corresponding to the above functions. The modules may be software and/or hardware.
一种可能的设计中，所述增强现实反馈设备可包括获取单元和处理单元，所述获取单元，用于获取物理设备的设备状态数据；所述处理单元，用于根据预设规则以及获取的所述设备状态数据，进行增强现实感官反馈。In a possible design, the augmented reality feedback device may include an acquisition unit and a processing unit, the acquisition unit is used to acquire the device status data of the physical device; the processing unit is used to The device status data is used for augmented reality sensory feedback.
另一种可能的设计中，所述增强现实反馈设备可包括处理器和通信接口，所述处理器被配置为支持增强现实反馈设备执行上述方法中相应的功能，所述通信接口用于支持增强现实反馈设备与物理设备之间的通信。进一步的，所述增强现实反馈设备还可以包括存储器，所述存储器用于与处理器耦合，其保存增强现实反馈设备必要的程序指令和数据。In another possible design, the augmented reality feedback device may include a processor and a communication interface, the processor is configured to support the augmented reality feedback device to perform corresponding functions in the above method, and the communication interface is used to support the augmented reality Communication between reality feedback devices and physical devices. Further, the augmented reality feedback device may further include a memory, which is used to be coupled with a processor, and stores necessary program instructions and data of the augmented reality feedback device.
本发明实施例中，所述增强现实反馈设备还可包括图像显示装置、音频设备、加热设备和气味散发设备中的一种或多种。所述图像显示装置用于呈现调用的匹配确定的设备状态数据所属类型以及所属级别的视觉增强反馈图形，对获取的设备状态数据进行视觉增强反馈。所述音频设备用于播放调用的匹配确定的设备状态数据所属类型以及所属级别的声觉增强反馈音频文件，对获取的设备状态数据进行声觉增强反馈。所述加热设备用于控制温度为匹配确定的设备状态数据所属类型以及所属级别的触觉增强反馈的温度，对获取的设备状态数据进行触觉增强反馈。所述气味散发设备用于散发匹配确定的设备状态数据所属类型以及所属级别的嗅觉增强反馈的气味，对获取的所述设备状态数据进行嗅觉增强反馈。In the embodiment of the present invention, the augmented reality feedback device may further include one or more of an image display device, an audio device, a heating device, and a scent emitting device. The image display device is used for presenting a visually enhanced feedback graph of the type and level of the called equipment state data determined by matching, and performing visually enhanced feedback on the acquired equipment state data. The audio device is used to play the audio file for acoustic enhancement feedback of the type and level of the equipment state data determined by matching and calling, and to perform acoustic enhancement feedback on the acquired equipment state data. The heating device is used to control the temperature to match the determined type and level of tactile enhancement feedback of the device state data, and to perform tactile enhancement feedback on the acquired device state data. The odor emitting device is used to emit an odor that matches the type and level of olfactory enhancement feedback of the determined equipment state data, and perform olfactory enhancement feedback on the acquired equipment state data.
第三方面，本发明实施例还提供一种计算机存储介质，用于储存为上述增强现实反馈设备所用的计算机软件指令，其包含用于执行上述方面媒体文件分享设备所涉及的程序。In a third aspect, an embodiment of the present invention further provides a computer storage medium for storing computer software instructions for the above-mentioned augmented reality feedback device, which includes a program for executing the above-mentioned media file sharing device.
本发明实施例提供的增强现实反馈方法和增强现实反馈设备，对设备状态进行增强现实感官反馈，反馈方式多样丰富，并能够基于获取的设备状态数据对设备状态进行更形象的反馈，方便用户快速直观的获知设备状态。The augmented reality feedback method and the augmented reality feedback device provided by the embodiments of the present invention perform augmented reality sensory feedback on the device state, and the feedback methods are diverse and rich, and can provide more vivid feedback on the device state based on the obtained device state data, which is convenient for users to quickly Intuitively know the status of the equipment.
附图说明Description of drawings
图1为现有技术中增强现实反馈的效果示意图；FIG. 1 is a schematic diagram of the effect of augmented reality feedback in the prior art;
图2为本发明实施例提供的增强现实反馈方法应用的系统结构图；Fig. 2 is a system structure diagram of the application of the augmented reality feedback method provided by the embodiment of the present invention;
图3为本发明实施例涉及的物理设备结构示意图；FIG. 3 is a schematic structural diagram of a physical device involved in an embodiment of the present invention;
图4为本发明实施例涉及的增强现实设备的结构示意图；FIG. 4 is a schematic structural diagram of an augmented reality device according to an embodiment of the present invention;
图5为本发明实施例提供的一种增强现实反馈的实施方法流程图；FIG. 5 is a flowchart of an implementation method of augmented reality feedback provided by an embodiment of the present invention;
图6为本发明实施例提供的获取设备状态数据的一种实施过程示意图；FIG. 6 is a schematic diagram of an implementation process for obtaining device status data provided by an embodiment of the present invention;
图7为本发明实施例提供的获取设备状态数据的另一种实施过程示意图；FIG. 7 is a schematic diagram of another implementation process for acquiring device status data provided by an embodiment of the present invention;
图8为本发明实施例提供的根据预设规则进行增强现实反馈的实施方法流程图；FIG. 8 is a flow chart of an implementation method for augmented reality feedback according to preset rules provided by an embodiment of the present invention;
图9为本发明实施例提供的进行视觉增强反馈的实施方法流程图；FIG. 9 is a flow chart of an implementation method for visual enhancement feedback provided by an embodiment of the present invention;
图10为本发明实施例提供的增强现实感官反馈的一种效果图；Fig. 10 is an effect diagram of the augmented reality sensory feedback provided by the embodiment of the present invention;
图11为本发明实施例提供的增强现实感官反馈的另一种效果图；Fig. 11 is another rendering of the augmented reality sensory feedback provided by the embodiment of the present invention;
图12为本发明实施例提供的增强现实感官反馈的又一种效果图；Fig. 12 is another rendering of the augmented reality sensory feedback provided by the embodiment of the present invention;
图13A至图13D为本发明实施例提供的增强现实感官反馈设备的结构示意图。13A to 13D are schematic structural diagrams of an augmented reality sensory feedback device provided by an embodiment of the present invention.
具体实施方式detailed description
为使本发明实施例的目的、技术方案和优点更加清楚，下面将结合本发明实施例中的附图，对本发明实施例中的技术方案进行描述。In order to make the purpose, technical solutions and advantages of the embodiments of the present invention more clear, the technical solutions in the embodiments of the present invention will be described below in conjunction with the drawings in the embodiments of the present invention.
本发明实施例描述的网络架构以及业务场景是为了更加清楚的说明本发明实施例的技术方案，并不构成对于本发明实施例提供的技术方案的限定，本领域普通技术人员可知，随着网络架构的演变和新业务场景的出现，本发明实施例提供的技术方案对于类似的技术问题，同样适用。The network architecture and business scenarios described in the embodiments of the present invention are to more clearly illustrate the technical solutions of the embodiments of the present invention, and do not constitute limitations on the technical solutions provided by the embodiments of the present invention. The evolution of the architecture and the emergence of new business scenarios, the technical solutions provided by the embodiments of the present invention are also applicable to similar technical problems.
图2所示为本发明实施例提供的增强现实反馈方法应用的系统结构图。图2中，物理设备是用户需要检测运行状态的设备，用户佩戴的增强现实设备主要用于实现增强现实功能以及检测功能。物理设备与增强现实设备之间通过网络进行数据联通。增强现实反馈方法应用的系统结构中还包括有系统服务器(图2未示出)，该系统服务器用于收集和处理物理设备所产生的数据，并将处理后的数据发送给增强现实设备。FIG. 2 is a system structure diagram of the application of the augmented reality feedback method provided by the embodiment of the present invention. In FIG. 2 , the physical device is the device that the user needs to detect the running status, and the augmented reality device worn by the user is mainly used to implement the augmented reality function and the detection function. Data communication between the physical device and the augmented reality device is carried out through the network. The system structure of the application of the augmented reality feedback method also includes a system server (not shown in FIG. 2 ), which is used to collect and process data generated by the physical device, and send the processed data to the augmented reality device.
本发明实施例以下，将对本发明技术实现中应用到的物理设备和增强现实设备进行分别说明：Embodiments of the present invention In the following, the physical devices and augmented reality devices used in the technical implementation of the present invention will be described separately:
物理设备：Physical device:
本发明实施例中的物理设备主要是指待检测运行状态的设备，如实验室或数据中心里面的交换机、路由器、机柜等。图3所示为本发明实施例涉及的物理设备结构示意图，如图3所示，所述物理设备在功能上可划分为传感器数据采集模块A100、处理模块A201、通信模块A301和对象识别模块A401。本发明实施例中传感器数据采集模块A100能检测到被测量的信息，并能将检测到的信息，按一定规律变换成为电信号或其他所需形式的信息输出，以满足信息的传输、处理、存储、显示、记录和控制等要求。传感器数据采集模块A100主要由硬件数据采集模块A110和软件数据采集模块A120组成。硬件数据采集模块A110由多种传感器组成，包括但不限于温度传感器A111、压力传感器A112、湿度传感器A113、音频传感器A115所组成，主要采集物理设备的物理特征包括但不限于温度、压力、湿度以及声音，如，温度传感器A111可用于探测设备温度是否高于或低于预设阈值，压力传感器A112用于探测设备的压力值是否高于或低于预设阈值，设备漏水。软件数据采集模块A120包括但不限于数据包采集模块A121和变频功率传感器A122，主要采集物理设备的软件模块的数据传输特征，例如，主要通过软件实现探测包括但不限于数据吞吐量、网络连接状况、丢包率、功率等。本发明实施例中处理模块A201用于接收从硬件数据采集模块A110和软件数据采集模块A120发送过来的设备数据，并通过多传感器数据融合技术对数据进行分析以及处理，按照预先定义的规则将用户所需的设备状态数据发送给增强现实设备。处理模块A201也可依据增强现实设备指令调取，下发，更新或新建数据并保存。通信模块A301可使用蜂窝，以太网，无线保真(Wireless Fidelity，wifi)，蓝牙，红外等通信方式接收其它设备发来的指令信息，也可以将物理设备的数据发送给云端或者其它设备。对象识别模块A401，可以包含物理标签、可见光学签名或非可见签名，用于增强现实设备近距离识别物理设备，。一些实施方式中，可以将对象识别模块A401布置在物理设备的表面上，如打印二维码或条形码的物理贴纸标签，使用无线射频识别(Radio Frequency Identification，RFID)射频装置对每台物理设备进行数字标识，以区分不同的物理设备。The physical device in the embodiment of the present invention mainly refers to the device whose running state is to be detected, such as a switch, a router, a cabinet, etc. in a laboratory or a data center. Fig. 3 is a schematic structural diagram of a physical device involved in an embodiment of the present invention. As shown in Fig. 3, the physical device can be functionally divided into a sensor data collection module A100, a processing module A201, a communication module A301 and an object recognition module A401 . In the embodiment of the present invention, the sensor data acquisition module A100 can detect the measured information, and can convert the detected information into electrical signals or other required forms of information output according to certain rules, so as to meet the requirements of information transmission, processing, Storage, display, recording and control requirements. The sensor data acquisition module A100 is mainly composed of a hardware data acquisition module A110 and a software data acquisition module A120. The hardware data acquisition module A110 is composed of various sensors, including but not limited to temperature sensor A111, pressure sensor A112, humidity sensor A113, and audio sensor A115. It mainly collects physical characteristics of physical equipment including but not limited to temperature, pressure, humidity and Sound, for example, the temperature sensor A111 can be used to detect whether the temperature of the device is higher or lower than a preset threshold, the pressure sensor A112 is used to detect whether the pressure value of the device is higher or lower than a preset threshold, and the device is leaking. The software data acquisition module A120 includes but not limited to the data packet acquisition module A121 and the variable frequency power sensor A122, which mainly collects the data transmission characteristics of the software module of the physical device, for example, mainly realizes detection through software, including but not limited to data throughput, network connection status , packet loss rate, power, etc. In the embodiment of the present invention, the processing module A201 is used to receive the device data sent from the hardware data acquisition module A110 and the software data acquisition module A120, analyze and process the data through multi-sensor data fusion technology, and send the user The required device state data is sent to the augmented reality device. The processing module A201 can also call, issue, update or create new data and save them according to the instructions of the augmented reality device. The communication module A301 can use cellular, Ethernet, Wireless Fidelity (wifi), Bluetooth, infrared and other communication methods to receive command information from other devices, and can also send data from physical devices to the cloud or other devices. The object recognition module A401 may include a physical tag, a visible optical signature or an invisible signature, and is used for the augmented reality device to recognize the physical device at a short distance. In some embodiments, the object recognition module A401 can be arranged on the surface of the physical equipment, such as a physical sticker label printed with a two-dimensional code or a barcode, and each physical equipment is checked using a radio frequency identification (Radio Frequency Identification, RFID) radio frequency device. Numerical identification to distinguish different physical devices.
增强现实设备：Augmented reality device:
本发明实施例中涉及的增强现实设备可以设置于穿戴式智能设备上，穿戴式智能设备是应用穿戴式技术对日常穿戴进行智能化设计、并开发出的可以穿戴的设备的总称，如眼镜、手套、手表、服饰及鞋等。所述增强现实设备可以是头戴式增强现实设备(Head-mounted displays，HMD)，将头戴式显示器，机器视觉系统，移动计算机这三个组件集成到一个单元中，放置在用带子绑定的设备中，通过佩戴的方式戴在使用者的头部，该设备能将增强现实类的信息传输至类似于普通眼镜的显示器上或投射至用户的眼球中进行显示，用以增强用户的视觉沉浸感。The augmented reality device involved in the embodiment of the present invention can be set on a wearable smart device. A wearable smart device is a general term for wearable devices that are intelligently designed and developed for daily wear using wearable technology, such as glasses, Gloves, watches, clothing and shoes, etc. The augmented reality device may be a head-mounted augmented reality device (Head-mounted displays, HMD), which integrates the three components of the head-mounted display, the machine vision system, and the mobile computer into one unit, and places them on a strap bound In the device, by wearing it on the user's head, the device can transmit augmented reality information to a display similar to ordinary glasses or project it into the user's eyeball for display, so as to enhance the user's vision immersion.
图4所示为本发明实施例涉及的增强现实设备的结构示意图，如图4所示，本发明实施例涉及的增强现实设备在现有增强现实模块C200的基础上包含有感官增强模块C100。感官增强模块C100用于处理传感器数据采集模块A100和处理模块A201发送的设备状态数据，并根据预设规则对所述获取的设备状态数据进行多感官维度的增强现实感官反馈，其中，所述增强现实感官反馈包括视觉增强反馈、触觉增强反馈、听觉增强反馈和嗅觉增强反馈中的至少一种。如图4所示，感官增强模块C100可包括视觉增强模块C101、听觉增强模块C102、触觉增强模块C103和嗅觉增强模块C104。其中，视觉增强模块C101由视觉增强应用程序软件及图形库所组成，用于根据物理设备发送的设备状态数据以及预设规则调用相应的图形，达到从颜色、形状、透明度以及物理现象等对设备状态数据进行可视化增强的效果。听觉增强模块C102由听觉增强应用程序软件以及如听筒、扬声器等现有音频设备所组成，用于根据物理设备发送的设备状态数据以及预设规则调用相应的声音库文件，达到用风声、水声、火声等声音对设备状态数据进行模拟增强的效果。触觉增强模块C103由触觉增强应用程序软件以及加热设备组成，加热设备一般是由加热元件和附件组成的器件，用于根据物理设备发送的设备状态数据以及预设规则，通过软件模块控制加热设备进行温度控制，达到通过温度触感对设备状态数据进行模拟增强的效果。嗅觉增强模块C104由可以散发气味的气味散发设备组成，用于根据物理设备发送的设备状态数据以及预设规则，控制气味散发设备释放相应的气味，达到用气味对设备状态数据进行模拟增强的效果。FIG. 4 is a schematic structural diagram of an augmented reality device according to an embodiment of the present invention. As shown in FIG. 4 , the augmented reality device according to this embodiment of the present invention includes a sensory enhancement module C100 on the basis of the existing augmented reality module C200. The sensory enhancement module C100 is used to process the device state data sent by the sensor data acquisition module A100 and the processing module A201, and perform augmented reality sensory feedback of multi-sensory dimensions on the acquired device state data according to preset rules, wherein the enhanced The realistic sensory feedback includes at least one of visual enhancement feedback, tactile enhancement feedback, auditory enhancement feedback and olfactory enhancement feedback. As shown in FIG. 4 , the sensory enhancement module C100 may include a visual enhancement module C101 , an auditory enhancement module C102 , a tactile enhancement module C103 and an olfactory enhancement module C104 . Among them, the visual enhancement module C101 is composed of visual enhancement application software and a graphics library, which is used to call corresponding graphics according to the device status data sent by the physical device and preset rules, so as to realize the accuracy of the device in terms of color, shape, transparency and physical phenomena. The effect of visual enhancement of state data. The auditory enhancement module C102 is composed of auditory enhancement application software and existing audio equipment such as earpieces and speakers. , fire and other sounds to simulate and enhance the device status data. The tactile enhancement module C103 is composed of tactile enhancement application software and heating equipment. The heating equipment is generally composed of heating elements and accessories, and is used to control the heating equipment through the software module according to the equipment status data and preset rules sent by the physical equipment. Temperature control achieves the effect of simulating and enhancing device status data through temperature touch. Smell enhancement module C104 is composed of odor emitting devices that can emit odors. It is used to control the odor emitting devices to release corresponding odors according to the device status data sent by physical devices and preset rules, so as to achieve the effect of using odors to simulate and enhance device status data. .
如图4所示，现有增强现实模块C200包括机器视觉模块C201、图像显示模块C202、通信模块C203、通信端口C204、处理模块C205和采集模块C206。其中，机器视觉模块C201包括不限于通过机器视觉图像摄取装置，深度信息摄取装置，将被摄取目标转换成图像信号，传送给图形计算机系统，得到被摄目标的形态信息，根据像素分布和亮度、颜色等信息，转变成数字化信号；图像系统对这些信号进行各种运算来抽取目标的特征，进而根据判别的结果来生成各类指令，在本发明实施例中机器视觉模块C201主要用于识别发生故障的物理设备。图像显示模块C202包括各类可实现增强现实功能的显示装置，如液晶显示屏，红外光影投射装置。更具体的，如头戴式显示设备所使用的基于光学原理的穿透式投影显示屏以及基于视频合成技术的透射式显示技术的显示装置。在本发明实施例中图像显示模块C202，主要用以呈现视觉增强模块C101所下发的图像信息，并实时反馈用户操作的操作结果。通信模块C203，可使用蜂窝，以太网，wifi，蓝牙，红外等通讯方式接收其它设备发来的指令信息，同时也可以将增强现实设备的数据发送给云端或者其它设备。通信端口C204使用传输控制协议(Transmission Control Protocol，TCP)/因特网协议(InternetProtocol，IP)作为网络通信的标准协议，使增强现实设备能够有单独的IP地址，用于获取从网络发送过来的设备状态数据。处理模块C205用于控制增强现实设备的各部分硬件设备和应用程序软件等，本发明实施例中主要用于接收从感官增强模块C100发送过来的数据，结合机器视觉模块C201反馈回来的数据进行处理后下发至图像显示模块C202。采集模块C206，主要用于使用摄像头，麦克风等传感器设备采集音视频等数据，通过摄像头、红外传感器、触摸屏等采集用户的手势。As shown in FIG. 4 , the existing augmented reality module C200 includes a machine vision module C201 , an image display module C202 , a communication module C203 , a communication port C204 , a processing module C205 and an acquisition module C206 . Among them, the machine vision module C201 includes, but is not limited to, a machine vision image capture device and a depth information capture device to convert the captured object into an image signal and transmit it to the graphics computer system to obtain the shape information of the captured object. According to pixel distribution and brightness, Information such as color is converted into digital signals; the image system performs various operations on these signals to extract the characteristics of the target, and then generates various instructions according to the results of the discrimination. In the embodiment of the present invention, the machine vision module C201 is mainly used to identify Faulty physical device. The image display module C202 includes various display devices capable of realizing augmented reality functions, such as liquid crystal display screens and infrared light and shadow projection devices. More specifically, such as the optical principle-based transmissive projection display screen used by the head-mounted display device and the display device based on the transmissive display technology of the video synthesis technology. In the embodiment of the present invention, the image display module C202 is mainly used to present the image information issued by the visual enhancement module C101, and to feed back the operation result of the user's operation in real time. The communication module C203 can use cellular, Ethernet, wifi, bluetooth, infrared and other communication methods to receive instruction information from other devices, and can also send data from augmented reality devices to the cloud or other devices. The communication port C204 uses Transmission Control Protocol (Transmission Control Protocol, TCP)/Internet Protocol (Internet Protocol, IP) as a standard protocol for network communication, so that the augmented reality device can have a separate IP address for obtaining the status of the device sent from the network data. The processing module C205 is used to control various hardware devices and application software of the augmented reality device. In the embodiment of the present invention, it is mainly used to receive the data sent from the sensory enhancement module C100, and process the data fed back by the machine vision module C201. Then send it to the image display module C202. The acquisition module C206 is mainly used to collect data such as audio and video using sensors such as cameras and microphones, and collect user gestures through cameras, infrared sensors, and touch screens.
本发明实施例以下将对本发明技术实现中涉及的技术进行分别说明。Embodiments of the present invention The technologies involved in the technical implementation of the present invention will be described separately below.
机器视觉技术：Machine Vision Technology:
机器视觉就是用机器代替人眼来做测量和判断。机器视觉系统是指通过机器视觉产品(图像摄取装置)将被摄取目标转换成图像信号，传送给专用的图像处理系统，得到被摄目标的形态信息，根据像素分布和亮度、颜色等信息，转变成数字化信号；图像系统对这些信号进行各种运算来抽取目标的特征，进而根据判别的结果来控制现场的设备动作。机器视觉系统的特点是提高生产的柔性和自动化程度。在一些不适合于人工作业的危险工作环境或人工视觉难以满足要求的场合中，经常使用机器视觉来替代人工视觉。Machine vision is the use of machines instead of human eyes for measurement and judgment. The machine vision system refers to converting the ingested target into an image signal through a machine vision product (image capture device), and sending it to a dedicated image processing system to obtain the shape information of the captured target. According to the pixel distribution, brightness, color and other information, transform The image system performs various calculations on these signals to extract the characteristics of the target, and then controls the on-site equipment actions according to the results of the discrimination. The machine vision system is characterized by improving the flexibility and automation of production. In some dangerous working environments that are not suitable for manual work or where artificial vision is difficult to meet the requirements, machine vision is often used to replace artificial vision.
图像叠加显示技术：Image overlay display technology:
图像叠加显示即是通过计算机的实时运算能力，使预先生成并存储的图形、属性，特征等被调用并叠合在一个基本图形或真实物体上的过程或方法，在本发明实施例中，主要应用其将预制图形叠加在真实物体上的能力。Image overlay display is a process or method in which pre-generated and stored graphics, attributes, features, etc. are called and superimposed on a basic graphic or real object through the real-time computing power of the computer. In the embodiment of the present invention, the main Apply its ability to superimpose pre-made graphics on real objects.
多传感器数据融合技术：Multi-sensor data fusion technology:
多传感器数据融合可以概括为把分布在不同位置的多个同类或不同类传感器所提供的局部数据资源加以综合，采用计算机技术对其进行分析，消除多传感器信息之间可能存在的冗余和矛盾，加以互补，降低其不确实性，获得被测对象的一致性解释与描述，从而提高系统决策、规划、反应的快速性和正确性，使系统获得更充分的信息。Multi-sensor data fusion can be summarized as synthesizing local data resources provided by multiple sensors of the same type or different types distributed in different locations, using computer technology to analyze them, and eliminating possible redundancy and contradictions between multi-sensor information. , complement each other, reduce its uncertainty, and obtain a consistent explanation and description of the measured object, thereby improving the speed and correctness of system decision-making, planning, and response, and enabling the system to obtain more sufficient information.
以下，将对本发明实施例涉及的增强现实反馈方法进行详细说明。Hereinafter, the augmented reality feedback method involved in the embodiment of the present invention will be described in detail.
图5所示为本发明实施例提供的增强现实反馈装置进行增强现实反馈的方法流程图，其中，所述增强现实反馈装置可以是增强现实设备。图5中以增强现实反馈装置为增强现实设备为例进行说明，如图5所示，本发明实施例提供的增强现实反馈方法包括：FIG. 5 is a flow chart of a method for performing augmented reality feedback by an augmented reality feedback device according to an embodiment of the present invention, wherein the augmented reality feedback device may be an augmented reality device. In FIG. 5, the augmented reality feedback device is used as an example of an augmented reality device for illustration. As shown in FIG. 5, the augmented reality feedback method provided by the embodiment of the present invention includes:
S101：增强现实设备获取物理设备的设备状态数据。S101: The augmented reality device acquires device state data of a physical device.
本发明实施例中所述设备状态数据包括但不限于吞吐量状态数据、网络连接状态数据、温度状态数据、湿度状态数据、压力状态数据和电力状态数据。增强现实设备获取的设备状态数据可以是以上所列设备状态数据中的一种或者多种。The device state data in the embodiment of the present invention includes but not limited to throughput state data, network connection state data, temperature state data, humidity state data, pressure state data and power state data. The device status data acquired by the augmented reality device may be one or more of the device status data listed above.
S102：增强现实设备根据预设规则以及所述获取的设备状态数据，进行增强现实感官反馈。S102: The augmented reality device performs augmented reality sensory feedback according to preset rules and the acquired device status data.
本发明实施例中，可预先设定针对设备状态数据进行增强现实感官反馈的规则，所述预设规则中包括设备状态数据类型、设备状态数据级别和增强现实感官反馈之间的对应关系。In the embodiment of the present invention, rules for performing augmented reality sensory feedback on device status data may be preset, and the preset rules include correspondence between device status data types, device status data levels, and augmented reality sensory feedback.
本发明实施例中，设备状态数据类型包括但不限于温度异常(温度过高或过低)、漏水、漏电、压力异常(压力过高或过低)、吞吐量过大、设备断网、丢包率过大等。In the embodiment of the present invention, the device status data types include but are not limited to abnormal temperature (too high or too low temperature), water leakage, electric leakage, abnormal pressure (too high or The packet rate is too large, etc.
本发明实施例中，设备状态数据级别一般包括至少两种级别，当然可根据实际情况划分设备状态所属级别，例如所述设备状态数据所属级别包括低级别和高级别两种，也可以包括低级别、中级别和高级别三种，还可以包括低级别、中级别、高级别、紧急级别和严重级别五种。In the embodiment of the present invention, the level of equipment status data generally includes at least two levels. Of course, the level to which the equipment status belongs can be divided according to the actual situation. For example, the level to which the equipment status data belongs includes low level and high level, and can also include low level There are three types of levels, medium level and high level, and five levels of low level, medium level, high level, urgent level and severe level.
本发明实施例中，增强现实感官反馈可以是视觉增强反馈、触觉增强反馈、听觉增强反馈和嗅觉增强反馈中一种，也可以是多种。本发明实施例中每一种增强现实感官反馈所呈现的图形、声音、触感和气味是符合设备状态在真实物理世界上所呈现的形式的，例如对温度状态数据进行的视觉增强反馈，可包括发出红光、冒出火花、流出熔岩，对温度状态数据进行的听觉增强反馈包括发出蜂鸣声、噼啪火花声，对漏水状态数据进行的听觉增强反馈包括发出溪水声、瀑布声、海浪声等等。In the embodiment of the present invention, the augmented reality sensory feedback may be one of enhanced visual feedback, enhanced tactile feedback, enhanced auditory feedback, and enhanced olfactory feedback, or may be multiple types. The graphics, sound, touch and smell presented by each augmented reality sensory feedback in the embodiment of the present invention are in line with the form presented by the device state in the real physical world, such as visually enhanced feedback on temperature state data, which may include Red light, sparks, and lava flow, the auditory enhancement feedback of temperature status data includes beeping and crackling sparks, and the auditory enhancement feedback of water leakage status data includes the sound of streams, waterfalls, ocean waves, etc. Wait.
本发明实施例中包括设备状态数据类型、设备状态数据级别和增强现实感官反馈之间的对应关系的预设规则可如表1所示：In the embodiment of the present invention, the preset rules including the corresponding relationship between the device state data type, the device state data level and the augmented reality sensory feedback can be shown in Table 1:
表1Table 1
由表1可知，本发明实施例中，设备状态数据的每一级别对应固定的增强现实感官反馈，每一类型以及每一级别的设备状态数据对应的固定的增强现实感官反馈，可以是视觉增强反馈、触觉增强反馈、听觉增强反馈和嗅觉增强反馈中一种，也可以是多种。It can be seen from Table 1 that in the embodiment of the present invention, each level of device state data corresponds to fixed augmented reality sensory feedback, and the fixed augmented reality sensory feedback corresponding to each type and level of device state data can be visually enhanced One or more of feedback, enhanced tactile feedback, enhanced auditory feedback and enhanced olfactory feedback.
由表1可知，本发明实施例中进行增强现实感官反馈时，增强现实感官反馈的图形、声音、触感和气味不同级别具有不同的反馈形式，例如，温度状态过高情况下，从低级别到高级别对应的不同视觉增强反馈是发出红光、冒出火花、流出熔岩，听觉反馈。漏水状态吓，从低级别到高级别对应的不同听觉感官反馈是发出溪水声、瀑布声、海浪声等等。As can be seen from Table 1, when performing augmented reality sensory feedback in the embodiment of the present invention, different levels of graphics, sound, touch and smell of augmented reality sensory feedback have different feedback forms, for example, when the temperature state is too high, from low level to The different visually enhanced feedback corresponding to the high level is red light, sparks, lava, and auditory feedback. Water leakage state, from low level to high level corresponding to different auditory sensory feedback is the sound of streams, waterfalls, waves and so on.
需要说明的是，本发明实施例中进行增强现实感官反馈时，增强现实感官反馈所反馈的图形、声音、触感和气味可以是比物理设备状态的真实物理现象夸张的，以更容易引起用户的注意，提高用户的定位故障的效率，例如，以温度过高为例，物理设备实际情况是并没有起火，但增强现实感官反馈的图形、声音、触感和气味可以是起火对应的物理现象。It should be noted that when performing augmented reality sensory feedback in the embodiment of the present invention, the graphics, sound, touch and smell fed back by the augmented reality sensory feedback may be exaggerated than the real physical phenomenon of the physical device state, so as to more easily arouse the user's Attention should be paid to improve the user’s efficiency in locating faults. For example, if the temperature is too high as an example, the actual physical equipment does not catch fire, but the graphics, sound, touch and smell of augmented reality sensory feedback can be the physical phenomenon corresponding to the fire.
本发明实施例中，每一设备状态数据类型对应设备状态数据的至少两个级别，设备状态数据的每一级别对应固定的增强现实感官反馈，以实现对不同的设备状态数据进行不同的增强现实感官反馈。In the embodiment of the present invention, each device state data type corresponds to at least two levels of device state data, and each level of device state data corresponds to fixed augmented reality sensory feedback, so as to implement different augmented reality for different device state data. sensory feedback.
本发明实施例中增强现实设备通过视觉增强反馈、触觉增强反馈、听觉增强反馈和嗅觉增强反馈等进行增强现实感官反馈，反馈方式多样丰富，并能够基于获取的设备状态数据对设备状态进行更形象的反馈，能够方便用户快速直观的获知设备状态。In the embodiment of the present invention, the augmented reality device performs augmented reality sensory feedback through visual enhanced feedback, tactile enhanced feedback, auditory enhanced feedback, and olfactory enhanced feedback. The feedback can facilitate the user to quickly and intuitively know the status of the device.
通常，增强现实反馈设备用于反馈物理设备的异常运行状态，故本发明实施例中所述获取的设备状态数据可以是反映设备运行状态异常的设备状态数据。其中，设备运行状态异常包括但不限于温度异常、漏水、漏电和断网等。Usually, the augmented reality feedback device is used to feed back the abnormal operation state of the physical device, so the device state data obtained in the embodiment of the present invention may be the device state data reflecting the abnormal operation state of the device. Among them, the abnormal operation status of the equipment includes but is not limited to abnormal temperature, water leakage, electricity leakage, and network disconnection.
本发明实施例中，增强现实设备获取物理设备的设备状态数据可采用如下方式：In the embodiment of the present invention, the augmented reality device can obtain the device status data of the physical device in the following manner:
一种可能的实施方式中，由物理设备识别自身的运行状态，并将识别到的物理设备运行状态异常的设备状态数据通过网络发送给增强现实设备，增强现实设备可通过网络获取物理设备发送的设备状态数据，具体的实施过程可参阅图6所示，图6中，物理设备的硬件数据采集模块A110和软件数据采集模块A120采集物理设备的设备状态数据，并发送给物理设备的处理模块A201。其中，硬件数据采集模块A110采集的设备状态数据可以包括温度状态数据、气味状态数据、环境湿度状态数据等。软件数据采集模块A120采集的设备状态数据可以包括吞吐量状态数据、网络连接状态数据、丢包率状态数据等。处理模块A201接收硬件数据采集模块A110和软件数据采集模块A120发送的设备状态数据后，通过多传感器数据融合技术判断所述设备状态数据是否为反映设备运行状态异常的设备状态数据，若是，则物理设备的处理模块A201将所述设备状态数据发送给增强现实设备的处理模块C205。若否，则物理设备的处理模块不做处理，继续接收硬件数据采集模块A110和软件数据采集模块A120发送的设备状态数据。增强现实设备的处理模块C205接收到所述设备状态数据后，将所述设备状态数据发送给增强现实设备的感官增强模块C100，由增强现实设备的感官增强模块C100根据预设规则进行增强现实感官反馈，并由增强现实设备的图像显示模块C202呈现增强现实感官反馈的效果。In a possible implementation manner, the physical device identifies its own running state, and sends the identified device state data indicating that the physical device is in an abnormal running state to the augmented reality device through the network, and the augmented reality device can obtain the information sent by the physical device through the network. For the device state data, the specific implementation process can be referred to as shown in Figure 6. In Figure 6, the hardware data acquisition module A110 and the software data acquisition module A120 of the physical device collect the device state data of the physical device and send it to the processing module A201 of the physical device . The device state data collected by the hardware data collection module A110 may include temperature state data, odor state data, environmental humidity state data, and the like. The device status data collected by the software data collection module A120 may include throughput status data, network connection status data, packet loss rate status data, and the like. After the processing module A201 receives the device status data sent by the hardware data acquisition module A110 and the software data acquisition module A120, it judges whether the device status data is the device status data reflecting the abnormal operation status of the device through multi-sensor data fusion technology, and if so, the physical The processing module A201 of the device sends the device state data to the processing module C205 of the augmented reality device. If not, the processing module of the physical device does not process, and continues to receive the device status data sent by the hardware data collection module A110 and the software data collection module A120. After the processing module C205 of the augmented reality device receives the device status data, it sends the device status data to the sensory enhancement module C100 of the augmented reality device, and the sensory enhancement module C100 of the augmented reality device performs augmented reality sensory processing according to preset rules. Feedback, and the image display module C202 of the augmented reality device presents the effect of augmented reality sensory feedback.
另一种可能的实施方式中，也可不通过网络环境，而是由增强现实设备直接识别物理设备的运行状态并获取设备状态数据，实施过程示意图可参阅图7所示。图7所示的实现过程中，增强现实设备实现增强现实感官反馈的处理过程与图6所示的处理过程类似，在此不再赘述。In another possible implementation manner, instead of going through the network environment, the augmented reality device may directly identify the operating status of the physical device and acquire device status data. The schematic diagram of the implementation process may refer to FIG. 7 . In the implementation process shown in FIG. 7 , the processing process for the augmented reality device to implement the augmented reality sensory feedback is similar to the processing process shown in FIG. 6 , and will not be repeated here.
本发明实施例中，增强现实设备根据预设规则以及所述获取的设备状态数据进行增强现实感官反馈可采用如图8所示的实现方式，如图8所示，包括：In the embodiment of the present invention, the augmented reality sensory feedback performed by the augmented reality device according to the preset rules and the obtained device state data may adopt the implementation method shown in FIG. 8, as shown in FIG. 8, including:
S1021：根据所述获取的设备状态数据，确定所述获取的设备状态数据所属类型以及所属级别。S1021: According to the obtained device state data, determine the type and level to which the obtained device state data belongs.
S1022：根据所述确定的设备状态数据所属类型以及所属级别查找预设规则，确定对应所述获取的设备状态数据所属类型以及所属级别的增强现实感官反馈。S1022: Search for preset rules according to the determined type and level of the device status data, and determine augmented reality sensory feedback corresponding to the type and level of the acquired device status data.
S1023：对所述获取的设备状态数据进行对应所述确定的设备状态数据所属类型以及所属级别的增强现实感官反馈。S1023: Perform augmented reality sensory feedback corresponding to the type and level of the determined device status data on the acquired device status data.
本发明实施例，S1022中确定的对应所述所述获取的设备状态数据所属类型以及所属级别的增强现实感官反馈可以是视觉增强反馈、触觉增强反馈、听觉增强反馈和嗅觉增强反馈中的一种，也可以是多种。In the embodiment of the present invention, the augmented reality sensory feedback corresponding to the type and level of the acquired device state data determined in S1022 may be one of visual augmented feedback, haptic augmented feedback, auditory augmented feedback, and olfactory augmented feedback , can also be multiple.
若确定的对应所述获取的设备状态数据所属类型以及所属级别的增强现实感官反馈中包括视觉增强反馈，则可从预设的图形库中调用匹配所述确定的设备状态数据所属类型以及所属级别的视觉增强反馈图形，通过图像显示装置呈现所述调用的视觉增强反馈图形，对所述获取的设备状态数据进行视觉增强反馈。If the determined augmented reality sensory feedback corresponding to the type and level of the acquired device status data includes visual enhancement feedback, it can be called from a preset graphics library to match the determined type and level of the device status data. The visually enhanced feedback graphic is presented through the image display device to present the called visually enhanced feedback graphic, and the visually enhanced feedback is performed on the acquired device status data.
若确定的对应所述获取的设备状态数据所属类型以及所属级别的增强现实感官反馈中包括听觉增强反馈，则可从预设的声音库文件中调用匹配所述确定的设备状态数据所属类型以及所属级别的声觉增强反馈音频文件，通过音频设备播放所述调用的音频文件，对所述获取的设备状态数据进行声觉增强反馈。If the determined augmented reality sensory feedback corresponding to the type and level of the acquired device state data includes auditory enhancement feedback, then the audio system that matches the determined type and level of the device state data can be called from the preset sound library file. The level of acoustic enhancement feedback audio file, the audio equipment is used to play the called audio file, and the acoustic enhancement feedback is performed on the acquired equipment state data.
若确定的对应所述获取的设备状态数据所属类型以及所属级别的增强现实感官反馈中包括触觉增强反馈，则可控制加热设备的温度为匹配所述确定的设备状态数据所属类型以及所属级别的触觉增强反馈的温度，对所述获取的设备状态数据进行触觉增强反馈。If the determined augmented reality sensory feedback corresponding to the type and level of the acquired device state data includes tactile enhancement feedback, the temperature of the heating device may be controlled to match the tactile sense of the determined type and level of the device state data. Enhance the temperature of the feedback, and perform haptic enhanced feedback on the acquired device status data.
若确定的对应所述获取的设备状态数据所属类型以及所属级别的增强现实感官反馈中包括嗅觉增强反馈，则可控制气味散发设备散发匹配所述确定的设备状态数据所属类型以及所属级别的嗅觉增强反馈的气味，对所述获取的设备状态数据进行嗅觉增强反馈。If the determined augmented reality sensory feedback corresponding to the type and level of the acquired device state data includes olfactory enhancement feedback, the scent emitting device can be controlled to emit olfactory enhancement that matches the determined type and level of the device state data. Feedback smell, performing olfactory enhanced feedback on the acquired device status data.
本发明实施例以下结合实际应用对上述实施例涉及的增强现实感官反馈的具体实现过程进行说明。Embodiments of the present invention The specific implementation process of the augmented reality sensory feedback involved in the above embodiments will be described below in combination with practical applications.
本发明实施例以进行视觉增强反馈为例进行举例说明。The embodiment of the present invention is described by taking visual enhancement feedback as an example.
图9所示为本发明实施例实现视觉增强反馈的一种实施流程图，如图9所示，包括：Fig. 9 shows an implementation flow chart of implementing visual enhancement feedback according to an embodiment of the present invention, as shown in Fig. 9, including:
S201：增强现实设备的处理模块C205获取物理设备的设备状态数据，并识别设备状态数据所属类型以及所属级别。S201: The processing module C205 of the augmented reality device acquires device state data of the physical device, and identifies the type and level of the device state data.
本发明实施例中可由物理设备的处理模块A201将识别出的反映设备运行状态异常的设备状态数据发送给系统服务器，并由系统服务器将设备状态数据发送给增强现实设备的处理模块C205。In the embodiment of the present invention, the processing module A201 of the physical device can send the identified device status data reflecting the abnormal operation status of the device to the system server, and the system server can send the device status data to the processing module C205 of the augmented reality device.
本发明实施例中增强现实设备的处理模块C205识别出获取的设备状态数据所属类型以及所属级别中对应的增强现实感官反馈中包括视觉增强反馈，则向增强现实设备的视觉增强模块C101发送进行视觉增强反馈的指令信息。例如，所述设备状态数据为温度状态数据，增强现实设备的处理模块C205识别出的设备状态数据类型为温度过高，设备状态数据级别为中等级别，则可通过查询预设规则(例如表1)，确定需要进行视觉增强反馈。In the embodiment of the present invention, the processing module C205 of the augmented reality device recognizes that the acquired device status data belongs to the type and the corresponding level of augmented reality sensory feedback includes visual enhancement feedback, and then sends a visual enhancement feedback to the visual enhancement module C101 of the augmented reality device. Enhanced feedback instruction information. For example, the device status data is temperature status data, the device status data type identified by the processing module C205 of the augmented reality device is too high temperature, and the device status data level is a medium level, then you can query the preset rules (such as Table 1 ), determining the need for visual augmentation feedback.
S202：增强现实设备的处理模块C205根据识别出的设备状态数据所属类型以及所属级别，发送进行视觉增强反馈的指令信息给相应的视觉增强模块C101。S202: The processing module C205 of the augmented reality device sends instruction information for visual enhancement feedback to the corresponding visual enhancement module C101 according to the identified type and level of the device status data.
S203：视觉增强模块C101根据预设规则中对应的视觉增强反馈，从图形库中调用相应的视觉增强反馈图形数据，并将调用的视觉增强反馈图形数据发送给处理模块C205。S203: The visual enhancement module C101 calls the corresponding visual enhancement feedback graphic data from the graphics library according to the corresponding visual enhancement feedback in the preset rules, and sends the called visual enhancement feedback graphic data to the processing module C205.
S204：处理模块C205接收视觉增强模块C101发送的视觉增强反馈图形后，向机器视觉模块C201发送识别物理设备的对象识别模块A401的指示信息。S204: After receiving the visual enhancement feedback graphic sent by the visual enhancement module C101, the processing module C205 sends instruction information to the machine vision module C201 to identify the object recognition module A401 of the physical device.
S205：若机器视觉模块C201识别到物理设备的对象识别模块A401，则处理模块C205将所述视觉增强反馈图形以及对象识别模块A401识别的物理设备标识(例如表征位置信息的)发送给图像显示模块C202。若机器视觉模块C201未识别到物理设备的对象识别模块A401，则处理模块C205将所述视觉增强反馈图形发送给图像显示模块C202。S205: If the machine vision module C201 recognizes the object recognition module A401 of the physical device, the processing module C205 sends the visual enhancement feedback graphic and the physical device identification (such as representing position information) recognized by the object recognition module A401 to the image display module C202. If the machine vision module C201 does not recognize the object recognition module A401 of the physical device, the processing module C205 sends the visual enhancement feedback graphic to the image display module C202.
S206：图像显示模块C202呈现所述视觉增强反馈图形。S206: The image display module C202 presents the visual enhancement feedback graphic.
本发明实施例中，图像显示模块C202对显示图形进行颜色、形状和透明度的处理，以形成对应所述视觉增强反馈图形的显示图形。In the embodiment of the present invention, the image display module C202 performs color, shape and transparency processing on the display graphics to form a display graphics corresponding to the visual enhancement feedback graphics.
进一步的，本发明实施例中若图像显示模块C202接收到对象识别模块A401识别的物理设备标识，则在所述物理设备标识的位置处呈现所述视觉增强反馈图形。Further, in the embodiment of the present invention, if the image display module C202 receives the physical device identifier recognized by the object recognition module A401, the visual enhancement feedback graphic is presented at the position of the physical device identifier.
本发明实施例中实现听觉增强反馈、触觉增强反馈以及嗅觉增强反馈的实施过程，与实现视觉增强反馈的过程类似，不同之处就在于具体进行增强反馈时所应用的设备不同，例如实现听觉增强反馈时，需要从声音库文件中调用声觉增强反馈音频文件，并通过现有的诸如听筒和扬声器等音频设备播放所述调用的音频文件，再例如，实现触觉增强反馈时，无需调用文件，而是通过控制加热设备的温度匹配触觉增强反馈，进行触觉增强反馈。实现嗅觉增强反馈时，也无需调用文件，而是通过控制气味散发设备散发气味，进行嗅觉增强反馈。The implementation process of implementing auditory enhancement feedback, tactile enhancement feedback, and olfactory enhancement feedback in the embodiment of the present invention is similar to the process of realizing visual enhancement feedback. When giving feedback, it is necessary to call the acoustic enhancement feedback audio file from the sound library file, and play the called audio file through existing audio devices such as earpieces and speakers. For example, when implementing tactile enhancement feedback, there is no need to call the file. Instead, the haptic augmentation feedback is performed by controlling the temperature of the heating device to match the haptic augmentation feedback. When implementing olfactory enhancement feedback, there is no need to call the file, but to control the odor emission device to emit odors to perform olfactory enhancement feedback.
本发明实施例中进行增强现实反馈时所用设备，若增强现实设备中配备了能够实现相应功能的设备，则利用现有已配备的设备，例如进行声觉增强反馈时，现有已配备有听筒，则可利用现有听筒进行声音播放。若增强现实设备中未配备相应功能的设备，则可新增具有相应功能的设备，例如进行触觉增强反馈时，可新增诸如电阻丝等能够发热的设备，进行触觉反馈。The device used for augmented reality feedback in the embodiment of the present invention, if the augmented reality device is equipped with a device that can realize the corresponding function, then use the existing equipped device, for example, when performing acoustic enhancement feedback, the existing earpiece is already equipped , you can use the existing earpiece for sound playback. If the augmented reality device is not equipped with a device with the corresponding function, a device with the corresponding function can be added. For example, when performing tactile enhancement feedback, a device capable of heating such as a resistance wire can be added for tactile feedback.
本发明实施例中增强现实设备进行对获取到的设备状态数据进行增强现实感官反馈可以包括多维度的感官反馈，例如对于温度状态数据，进行增强现实感官反馈时，可以包括如图10所示的进行火焰显示的视觉增强反馈，还可以包括发出蜂鸣声或者噼啪火花声的声觉增强反馈。又例如对于漏水状态数据，进行增强现实感官反馈时，可以包括如图11所示的流水效果显示的视觉增强反馈，还可以包括有流水声效的声觉增强反馈，所述声觉增强反馈还可以根据漏水状态数据所属级别具有小溪、瀑布、大海等不同的声效，或者还可以根据用户与物理设备之间的距离体现不同音量的声效(距离远则声音小，距离近则声音大)。再例如对于吞吐量状态数据，进行增强现实感官反馈时，可以包括如图12所示的显示设备形状臃肿以及颜色变得苍白的显示效果，还可以包括发出嗡鸣声的声觉增强反馈。In the embodiment of the present invention, the augmented reality sensory feedback performed by the augmented reality device on the acquired device state data may include multi-dimensional sensory feedback. For example, when performing augmented reality sensory feedback on temperature state data, it may include the Visually enhanced feedback for flame display may also include audible enhanced feedback for beeping or crackling sparks. For another example, when performing augmented reality sensory feedback for water leakage status data, it may include visually enhanced feedback as shown in FIG. According to the level of the water leakage status data, there are different sound effects such as streams, waterfalls, seas, etc., or sound effects of different volumes can be reflected according to the distance between the user and the physical device (the sound is low when the distance is long, and the sound is loud if the distance is short). For another example, for the throughput status data, when performing augmented reality sensory feedback, it may include the display effect of the display device being bloated in shape and pale in color as shown in FIG.
需要说明的是，本发明实施例图10、图11和图12所示的效果图仅是进行示意性说明，并不引以为限，例如对于图10而言，本发明实施例中可根据温度状态数据的级别呈现不同的视觉增强反馈图形，例如较低级别时呈现火花效果，较高级别时呈现熔岩喷出等图形。It should be noted that the effect diagrams shown in FIG. 10 , FIG. 11 and FIG. 12 of the embodiment of the present invention are only for schematic illustration and are not limiting. For example, for FIG. The level of temperature status data presents different visually enhanced feedback graphics, such as spark effects at lower levels and lava eruption at higher levels.
本发明实施例中增强现实设备通过视觉增强反馈、触觉增强反馈、听觉增强反馈和嗅觉增强反馈等进行增强现实感官反馈，反馈方式多样丰富，并能够基于获取的设备状态数据对设备状态进行更形象的反馈，方便用户快速直观的获知设备状态。In the embodiment of the present invention, the augmented reality device performs augmented reality sensory feedback through visual enhanced feedback, tactile enhanced feedback, auditory enhanced feedback, and olfactory enhanced feedback. Feedback, which is convenient for users to quickly and intuitively know the status of the device.
上述主要对本发明实施例提供的增强现实反馈方法进行了介绍。可以理解的是，用于实现上述增强现实反馈方法的增强现实反馈设备包含了执行各个功能相应的硬件结构和/或软件模块。结合本发明中所公开的实施例描述的各示例的单元及算法步骤，本发明实施例能够以硬件或硬件和计算机软件的结合形式来实现。某个功能究竟以硬件还是计算机软件驱动硬件的方式来执行，取决于技术方案的特定应用和设计约束条件。本领域技术人员可以对每个特定的应用来使用不同的方法来实现所描述的功能，但是这种实现不应认为超出本发明实施例的技术方案的范围。The foregoing mainly introduces the augmented reality feedback method provided by the embodiment of the present invention. It can be understood that the augmented reality feedback device used to implement the above augmented reality feedback method includes corresponding hardware structures and/or software modules for performing various functions. Combining the units and algorithm steps of each example described in the embodiments disclosed in the present invention, the embodiments of the present invention can be implemented in the form of hardware or a combination of hardware and computer software. Whether a certain function is executed by hardware or computer software drives hardware depends on the specific application and design constraints of the technical solution. Those skilled in the art may use different methods to implement the described functions for each specific application, but such implementation should not be regarded as exceeding the scope of the technical solutions of the embodiments of the present invention.
本发明实施例可以根据上述方法示例对实现上述增强现实反馈方法的增强现实反馈设备进行功能单元的划分，例如，可以对应各个功能划分各个功能单元，也可以将两个或两个以上的功能集成在一个处理单元中。上述集成的单元既可以采用硬件的形式实现，也可以采用软件功能单元的形式实现。需要说明的是，本发明实施例中对单元的划分是示意性的，仅仅为一种逻辑功能划分，实际实现时可以有另外的划分方式。The embodiments of the present invention can divide the functional units of the augmented reality feedback device implementing the augmented reality feedback method according to the above method example, for example, each functional unit can be divided corresponding to each function, or two or more functions can be integrated in one processing unit. The above-mentioned integrated units can be implemented in the form of hardware or in the form of software functional units. It should be noted that the division of the units in the embodiment of the present invention is schematic, and is only a logical function division, and there may be another division manner in actual implementation.
在采用集成的单元的情况下，图13A示出了本发明实施例提供的一种可能的增强现实反馈设备的结构示意图。本发明实施例提供的增强现实反馈设备100包括获取单元101和处理单元102，其中，所述获取单元101用于获取物理设备的设备状态数据。所述处理单元102用于根据预设规则以及获取的所述设备状态数据，进行增强现实感官反馈，例如，所述处理单元102用于支持所述增强现实反馈设备执行图2中的过程S102，图8中的过程S1021至过程S1023，和/或用于本文所描述的技术的其它过程。In the case of using an integrated unit, FIG. 13A shows a schematic structural diagram of a possible augmented reality feedback device provided by an embodiment of the present invention. The augmented reality feedback device 100 provided by the embodiment of the present invention includes an acquisition unit 101 and a processing unit 102, wherein the acquisition unit 101 is configured to acquire device state data of a physical device. The processing unit 102 is configured to perform augmented reality sensory feedback according to preset rules and the acquired device state data, for example, the processing unit 102 is configured to support the augmented reality feedback device to execute the process S102 in FIG. 2 , Process S1021 to process S1023 in FIG. 8, and/or other processes for the techniques described herein.
其中，处理单元102可以是处理器或控制器，例如可以是中央处理器(CentralProcessing Unit，CPU)，通用处理器，数字信号处理器(Digital Signal Processor，DSP)，专用集成电路(Application-Specific Integrated Circuit，ASIC)，现场可编程门阵列(Field Programmable Gate Array，FPGA)或者其他可编程逻辑器件、晶体管逻辑器件、硬件部件或者其任意组合。其可以实现或执行结合本发明公开内容所描述的各种示例性的逻辑方框，模块和电路。所述处理器也可以是实现计算功能的组合，例如包含一个或多个微处理器组合，DSP和微处理器的组合等等。获取单元101可以是通信接口、收发器、收发电路等，其中，通信接口是统称，可以包括一个或多个接口。Wherein, the processing unit 102 may be a processor or a controller, such as a central processing unit (Central Processing Unit, CPU), a general processor, a digital signal processor (Digital Signal Processor, DSP), an application-specific integrated circuit (Application-Specific Integrated Circuit, ASIC), Field Programmable Gate Array (Field Programmable Gate Array, FPGA) or other programmable logic devices, transistor logic devices, hardware components or any combination thereof. It can implement or execute the various illustrative logical blocks, modules and circuits described in connection with the present disclosure. The processor may also be a combination of computing functions, for example, a combination of one or more microprocessors, a combination of DSP and a microprocessor, and so on. The acquisition unit 101 may be a communication interface, a transceiver, a transceiver circuit, etc., where the communication interface is a general term and may include one or more interfaces.
所述增强现实反馈设备还可包括存储单元103，用于存储所述增强现实反馈设备100的程序代码和数据，如图13B所示。The augmented reality feedback device may further include a storage unit 103 for storing program codes and data of the augmented reality feedback device 100 , as shown in FIG. 13B .
具体实施过程中，所述增强现实反馈设备100还可包括图像显示装置104、音频设备105、加热设备106和气味散发设备107中的一种或多种。图13C为本发明实施例提供的增强现实反馈设备100包括上述图像显示装置104、音频设备105、加热设备106和气味散发设备107的结构示意图。In a specific implementation process, the augmented reality feedback device 100 may further include one or more of an image display device 104 , an audio device 105 , a heating device 106 and a scent emitting device 107 . FIG. 13C is a schematic structural diagram of the augmented reality feedback device 100 provided by the embodiment of the present invention, including the above-mentioned image display device 104 , audio device 105 , heating device 106 and scent emitting device 107 .
其中，所述处理单元102确定的对应获取的设备状态数据所属类型以及所属级别的增强现实感官反馈中包括视觉增强反馈情况下，所述处理单元102，可从预设的图形库中调用匹配确定的设备状态数据所属类型以及所属级别的视觉增强反馈图形，通过所述图像显示装置104呈现调用的所述视觉增强反馈图形，对获取的所述设备状态数据进行视觉增强反馈。Wherein, when the type and level of the augmented reality sensory feedback determined by the processing unit 102 corresponding to the acquired device state data includes visual enhancement feedback, the processing unit 102 may call the matching determination from the preset graphics library. The visually enhanced feedback graphics of the type and level to which the device status data belongs, the image display device 104 presents the called visually enhanced feedback graphics, and performs visually enhanced feedback on the acquired device status data.
所述处理单元102确定的对应获取的设备状态数据所属类型以及所属级别的增强现实感官反馈中包括听觉增强反馈情况下，所述处理单元102，可从预设的声音库文件中调用匹配确定的设备状态数据所属类型以及所属级别的声觉增强反馈音频文件，通过所述音频设备105播放调用的所述音频文件，对获取的所述设备状态数据进行声觉增强反馈。In the case where the type and level of the augmented reality sensory feedback determined by the processing unit 102 corresponds to the acquired device status data includes auditory enhancement feedback, the processing unit 102 may call the matching determined audio from the preset sound library file. The type and level of the device status data belong to the acoustic enhancement feedback audio file, and the audio device 105 plays the invoked audio file to perform acoustic enhancement feedback on the acquired device status data.
所述处理单元102确定的对应获取的设备状态数据所属类型以及所属级别的增强现实感官反馈中包括触觉增强反馈的情况下，所述处理单元102，可控制所述加热设备106的温度为匹配确定的设备状态数据所属类型以及所属级别的触觉增强反馈的温度，对获取的所述设备状态数据进行触觉增强反馈。When the type and level of the augmented reality sensory feedback determined by the processing unit 102 corresponding to the acquired device status data includes tactile enhancement feedback, the processing unit 102 may control the temperature of the heating device 106 to match the determined The device state data belongs to the type and the temperature of the tactile enhancement feedback of the level, and the tactile enhancement feedback is performed on the acquired device state data.
所述处理单元102确定的对应获取的所述设备状态数据所属类型以及所属级别的增强现实感官反馈中包括嗅觉增强反馈的情况下，所述处理单元102，可控制所述气味散发设备107散发匹配确定的设备状态数据所属类型以及所属级别的嗅觉增强反馈的气味，对获取的所述设备状态数据进行嗅觉增强反馈。In the case that the type and level of the augmented reality sensory feedback determined by the processing unit 102 corresponding to the acquired device status data includes olfactory enhancement feedback, the processing unit 102 may control the scent emitting device 107 to emit a matching The determined device state data belongs to the type and level of smell enhancement feedback, and performs smell enhancement feedback on the acquired device state data.
本发明实施例上述集成的单元采用硬件的形式实现时，处理单元102为处理器，获取单元101为通信接口，所述存储单元103为存储器，本发明实施例所涉及的增强现实反馈设备可以为图13D所示的增强现实反馈设备。In the embodiment of the present invention, when the above-mentioned integrated units are implemented in the form of hardware, the processing unit 102 is a processor, the acquisition unit 101 is a communication interface, and the storage unit 103 is a memory. The augmented reality feedback device involved in the embodiment of the present invention may be The augmented reality feedback device shown in Figure 13D.
参阅图13D所示，该增强现实反馈设备1000包括：处理器1001、通信接口1002、存储器1003。可选的，增强现实反馈设备1000还可以包括总线1004。其中，通信接口1002、处理器1001以及存储器1003可以通过总线1004相互连接；总线1004可以是外设部件互连标准(Peripheral Component Interconnect，简称PCI)总线或扩展工业标准结构(ExtendedIndustry Standard Architecture，简称EISA)总线等。所述总线1004可以分为地址总线、数据总线、控制总线等。为便于表示，图13C中仅用一条粗线表示，但并不表示仅有一根总线或一种类型的总线。Referring to FIG. 13D , the augmented reality feedback device 1000 includes: a processor 1001 , a communication interface 1002 , and a memory 1003 . Optionally, the augmented reality feedback device 1000 may also include a bus 1004 . Wherein, the communication interface 1002, the processor 1001 and the memory 1003 can be connected to each other through a bus 1004; the bus 1004 can be a Peripheral Component Interconnect (PCI for short) bus or an Extended Industry Standard Architecture (EISA for short). ) bus, etc. The bus 1004 can be divided into address bus, data bus, control bus and so on. For ease of representation, only one thick line is used in FIG. 13C , but it does not mean that there is only one bus or one type of bus.
本发明实施例中所述增强现实反馈设备可以是上述实施例中涉及的增强现实设备，用于实现上述方法实施例中涉及的增强现实反馈方法的所有功能，其具体实现过程可以参照上述方法实施例的相关描述，此处不再赘述。The augmented reality feedback device described in the embodiment of the present invention may be the augmented reality device involved in the above-mentioned embodiments, and is used to realize all the functions of the augmented reality feedback method involved in the above-mentioned method embodiments, and its specific implementation process can be implemented with reference to the above-mentioned method The relevant description of the example is omitted here.
本发明实施例还提供一种计算机存储介质，用于储存为上述增强现实反馈设备所用的计算机软件指令，其包含用于执行上述方面增强现实反馈设备所涉及的程序。An embodiment of the present invention also provides a computer storage medium for storing computer software instructions used by the above-mentioned augmented reality feedback device, which includes a program for executing the above aspect of the augmented reality feedback device.
显然，本领域的技术人员可以对本发明进行各种改动和变型而不脱离本发明的精神和范围。这样，倘若本发明的这些修改和变型属于本发明权利要求及其等同技术的范围之内，则本发明也意图包含这些改动和变型在内。Obviously, those skilled in the art can make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and equivalent technologies thereof, the present invention also intends to include these modifications and variations.
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201610458015.1ACN106095108B (en) | 2016-06-22 | 2016-06-22 | Augmented reality feedback method and device |
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201610458015.1ACN106095108B (en) | 2016-06-22 | 2016-06-22 | Augmented reality feedback method and device |
| Publication Number | Publication Date |
|---|---|
| CN106095108Atrue CN106095108A (en) | 2016-11-09 |
| CN106095108B CN106095108B (en) | 2019-02-05 |
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201610458015.1AActiveCN106095108B (en) | 2016-06-22 | 2016-06-22 | Augmented reality feedback method and device |
| Country | Link |
|---|---|
| CN (1) | CN106095108B (en) |
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107564107A (en)* | 2017-07-19 | 2018-01-09 | 中国农业大学 | A design method and device for implementing an augmented reality tool |
| CN107733755A (en)* | 2017-09-30 | 2018-02-23 | 联想(北京)有限公司 | Processing method and its system and computing system |
| CN108243247A (en)* | 2018-01-03 | 2018-07-03 | 广州供电局有限公司 | A kind of data center's intelligent robot inspection tour system |
| CN108845745A (en)* | 2018-07-04 | 2018-11-20 | 百度在线网络技术（北京）有限公司 | Triggering method, device, electronic equipment and the storage medium of augmented reality |
| CN109448155A (en)* | 2018-10-15 | 2019-03-08 | 国网河南省电力公司济源供电公司 | Equipment-patrolling method based on AR technology |
| CN110100199A (en)* | 2017-01-13 | 2019-08-06 | 努诺·安图内斯 | System and method for acquisition, registration and multimedia management |
| CN110244582A (en)* | 2019-05-05 | 2019-09-17 | 浙江乌镇街科技有限公司 | A kind of digitlization smell data correlation platform |
| CN110427103A (en)* | 2019-07-10 | 2019-11-08 | 佛山科学技术学院 | A kind of virtual reality fusion emulation experiment multi-modal interaction method and system |
| CN111914789A (en)* | 2020-08-13 | 2020-11-10 | 杭州云笔智能科技有限公司 | Method and system for determining running state of target equipment |
| CN112634462A (en)* | 2020-12-21 | 2021-04-09 | 上海影创信息科技有限公司 | Temperature matching augmented reality method and system for AR glasses |
| CN112783316A (en)* | 2019-11-08 | 2021-05-11 | 浙江商汤科技开发有限公司 | Augmented reality-based control method and apparatus, electronic device, and storage medium |
| CN115707340A (en)* | 2020-05-29 | 2023-02-17 | 嘉吉公司 | Time intensity sensory management (TiDS) |
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101923669A (en)* | 2008-07-18 | 2010-12-22 | 史迪芬·凯斯 | Smart Adaptive Design |
| CN102592484A (en)* | 2010-11-16 | 2012-07-18 | 韩国电子通信研究院 | Reconfigurable platform management apparatus for virtual reality-based training simulator |
| CN102789313A (en)* | 2012-03-19 | 2012-11-21 | 乾行讯科（北京）科技有限公司 | User interaction system and method |
| CN103793063A (en)* | 2014-03-11 | 2014-05-14 | 哈尔滨工业大学 | Multi-channel augmented reality system |
| US20140184496A1 (en)* | 2013-01-03 | 2014-07-03 | Meta Company | Extramissive spatial imaging digital eye glass apparatuses, methods and systems for virtual or augmediated vision, manipulation, creation, or interaction with objects, materials, or other entities |
| US20150120617A1 (en)* | 2013-10-25 | 2015-04-30 | Weast John C. | Apparatus and methods for capturing and generating user experiences |
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101923669A (en)* | 2008-07-18 | 2010-12-22 | 史迪芬·凯斯 | Smart Adaptive Design |
| CN102592484A (en)* | 2010-11-16 | 2012-07-18 | 韩国电子通信研究院 | Reconfigurable platform management apparatus for virtual reality-based training simulator |
| CN102789313A (en)* | 2012-03-19 | 2012-11-21 | 乾行讯科（北京）科技有限公司 | User interaction system and method |
| US20140184496A1 (en)* | 2013-01-03 | 2014-07-03 | Meta Company | Extramissive spatial imaging digital eye glass apparatuses, methods and systems for virtual or augmediated vision, manipulation, creation, or interaction with objects, materials, or other entities |
| US20150120617A1 (en)* | 2013-10-25 | 2015-04-30 | Weast John C. | Apparatus and methods for capturing and generating user experiences |
| CN103793063A (en)* | 2014-03-11 | 2014-05-14 | 哈尔滨工业大学 | Multi-channel augmented reality system |
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110100199B (en)* | 2017-01-13 | 2022-06-14 | 努诺·安图内斯 | System and method for acquisition, registration and multimedia management |
| CN110100199A (en)* | 2017-01-13 | 2019-08-06 | 努诺·安图内斯 | System and method for acquisition, registration and multimedia management |
| CN107564107A (en)* | 2017-07-19 | 2018-01-09 | 中国农业大学 | A design method and device for implementing an augmented reality tool |
| CN107733755A (en)* | 2017-09-30 | 2018-02-23 | 联想(北京)有限公司 | Processing method and its system and computing system |
| CN108243247A (en)* | 2018-01-03 | 2018-07-03 | 广州供电局有限公司 | A kind of data center's intelligent robot inspection tour system |
| CN108845745A (en)* | 2018-07-04 | 2018-11-20 | 百度在线网络技术（北京）有限公司 | Triggering method, device, electronic equipment and the storage medium of augmented reality |
| CN109448155A (en)* | 2018-10-15 | 2019-03-08 | 国网河南省电力公司济源供电公司 | Equipment-patrolling method based on AR technology |
| CN110244582A (en)* | 2019-05-05 | 2019-09-17 | 浙江乌镇街科技有限公司 | A kind of digitlization smell data correlation platform |
| CN110427103B (en)* | 2019-07-10 | 2022-04-26 | 佛山科学技术学院 | A multi-channel interaction method and system for virtual-real fusion simulation experiment |
| CN110427103A (en)* | 2019-07-10 | 2019-11-08 | 佛山科学技术学院 | A kind of virtual reality fusion emulation experiment multi-modal interaction method and system |
| CN112783316A (en)* | 2019-11-08 | 2021-05-11 | 浙江商汤科技开发有限公司 | Augmented reality-based control method and apparatus, electronic device, and storage medium |
| CN115707340A (en)* | 2020-05-29 | 2023-02-17 | 嘉吉公司 | Time intensity sensory management (TiDS) |
| CN111914789A (en)* | 2020-08-13 | 2020-11-10 | 杭州云笔智能科技有限公司 | Method and system for determining running state of target equipment |
| CN112634462A (en)* | 2020-12-21 | 2021-04-09 | 上海影创信息科技有限公司 | Temperature matching augmented reality method and system for AR glasses |
| Publication number | Publication date |
|---|---|
| CN106095108B (en) | 2019-02-05 |
| Publication | Publication Date | Title |
|---|---|---|
| CN106095108A (en) | A kind of augmented reality feedback method and equipment | |
| US10725297B2 (en) | Method and system for implementing a virtual representation of a physical environment using a virtual reality environment | |
| KR101930657B1 (en) | System and method for immersive and interactive multimedia generation | |
| JP6684883B2 (en) | Method and system for providing camera effects | |
| US20180357978A1 (en) | Method and devices used for implementing augmented reality interaction and displaying | |
| CN107862270B (en) | Face classifier training method, face detection method and device and electronic equipment | |
| EP3921806A1 (en) | Body pose estimation | |
| TW201814445A (en) | Performing operations based on gestures | |
| CN104699380B (en) | A kind of information processing method and electronic equipment | |
| KR20130084543A (en) | Apparatus and method for providing user interface | |
| JP7654756B2 (en) | Managing devices with additional displays | |
| KR20150079387A (en) | Illuminating a Virtual Environment With Camera Light Data | |
| CN109426343B (en) | Collaborative training method and system based on virtual reality | |
| CN108563327B (en) | Augmented reality method, device, storage medium and electronic device | |
| JP7672271B2 (en) | Determining Latency for Human Interface Devices | |
| WO2022143322A1 (en) | Augmented reality interaction method and electronic device | |
| JP2022529869A (en) | Human-computer interaction control methods, devices, systems, and electronic devices | |
| CN108234659B (en) | Data processing method, device and system | |
| WO2017052880A1 (en) | Augmented reality with off-screen motion sensing | |
| CN104851134A (en) | Augmented Reality System and Method Combining Virtual Trigger and Real Object Trigger | |
| CN116821990A (en) | Situation unified modeling method based on human engineering, server and storage medium | |
| WO2018017856A1 (en) | Displaying and interacting with scanned environment geometry in virtual reality | |
| CN103279265B (en) | The generation method at the terminal at display networks interface, system and interface | |
| CN112579144B (en) | Data processing method and device | |
| CN108121491B (en) | Display method and device |
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |