CN111766936A - Control method, device, terminal device and storage medium for virtual content - Google Patents

Abstract

Translated fromChinese

本申请实施例公开了一种虚拟内容的控制方法、装置、终端设备及存储介质，涉及显示技术领域。该虚拟内容的控制方法应用于终端设备，所述终端设备与穿戴设备通信连接，该虚拟内容的控制方法包括：获取所述穿戴设备与所述终端设备之间的相对空间位置关系；获取待显示的虚拟内容的内容数据；根据所述内容数据和所述相对空间位置关系，生成所述虚拟内容；对所述虚拟内容进行显示；根据所述穿戴设备检测到的操控动作参数以及所述相对空间位置关系的变化信息中的至少一种，对显示的所述虚拟内容进行控制。本方法可以利用穿戴设备控制显示的虚拟内容，提高用户与虚拟内容的交互性。

The embodiments of the present application disclose a virtual content control method, device, terminal device and storage medium, which relate to the technical field of display. The method for controlling virtual content is applied to a terminal device, and the terminal device is communicatively connected to a wearable device, and the method for controlling virtual content includes: acquiring a relative spatial positional relationship between the wearable device and the terminal device; acquiring a to-be-displayed device content data of the virtual content; generate the virtual content according to the content data and the relative spatial positional relationship; display the virtual content; according to the manipulation action parameters detected by the wearable device and the relative space At least one of the change information of the positional relationship controls the displayed virtual content. The method can use the wearable device to control the displayed virtual content, and improve the interaction between the user and the virtual content.

Description

Translated fromChinese

虚拟内容的控制方法、装置、终端设备及存储介质Control method, device, terminal device and storage medium for virtual content

技术领域technical field

本申请涉及显示技术领域，更具体地，涉及一种虚拟内容的控制方法、装置、终端设备及存储介质。The present application relates to the field of display technology, and more particularly, to a method, apparatus, terminal device and storage medium for controlling virtual content.

背景技术Background technique

随着科技的发展，机器智能化及信息智能化日益普及，通过机器视觉或者虚拟视觉等图像采集装置来识别用户影像以实现人机交互的技术越来越重要。增强现实技术(Augmented Reality，AR)借助计算机图形技术和可视化技术构建现实环境中不存在的虚拟内容，并通过图像识别定位技术将虚拟内容准确地融合到真实环境中，借助显示设备将虚拟内容与真实环境融为一体，并显示给使用者真实的感观体验。增强现实技术要解决的首要技术难题是如何将虚拟内容准确地融合到真实世界中，也就是要使虚拟内容以正确的角度姿态出现在真实场景的正确位置上，从而产生强烈的视觉真实感。在传统的技术中，通过在真实场景图像中叠加虚拟内容进行增强现实或混合现实等的显示，而与虚拟内容的交互控制是增强现实或混合现实的重要研究方向。With the development of science and technology, machine intelligence and information intelligence are becoming more and more popular, and the technology of identifying user images through image acquisition devices such as machine vision or virtual vision to realize human-computer interaction is becoming more and more important. Augmented Reality (AR) uses computer graphics technology and visualization technology to construct virtual content that does not exist in the real environment, and accurately integrates the virtual content into the real environment through image recognition and positioning technology. The real environment is integrated and displayed to the user with a real sensory experience. The primary technical problem to be solved by augmented reality technology is how to accurately integrate the virtual content into the real world, that is, to make the virtual content appear in the correct position of the real scene with the correct angle and posture, thus producing a strong sense of visual reality. In the traditional technology, the display of augmented reality or mixed reality is performed by superimposing virtual content on the real scene image, and the interactive control with virtual content is an important research direction of augmented reality or mixed reality.

发明内容SUMMARY OF THE INVENTION

鉴于上述问题，本申请实施例提出了一种虚拟内容的控制方法、装置、终端设备及存储介质，能够利用穿戴设备控制显示的虚拟内容，提高用户与虚拟内容的交互性。In view of the above problems, the embodiments of the present application propose a method, apparatus, terminal device and storage medium for controlling virtual content, which can utilize wearable devices to control displayed virtual content and improve the interactivity between users and virtual content.

第一方面，本申请实施例提供了一种虚拟内容的控制方法，应用于终端设备，所述终端设备与穿戴设备通信连接，所述方法包括：获取所述穿戴设备与所述终端设备之间的相对空间位置关系；获取待显示的虚拟内容的内容数据；根据所述内容数据和所述相对空间位置关系，生成所述虚拟内容；对所述虚拟内容进行显示；根据所述穿戴设备检测到的操控动作参数以及所述相对空间位置关系的变化信息中的至少一种，对显示的所述虚拟内容进行控制。In a first aspect, an embodiment of the present application provides a method for controlling virtual content, which is applied to a terminal device, where the terminal device is communicatively connected to a wearable device, and the method includes: acquiring a relationship between the wearable device and the terminal device obtain the content data of the virtual content to be displayed; generate the virtual content according to the content data and the relative spatial positional relationship; display the virtual content; At least one of the manipulation action parameters and the change information of the relative spatial position relationship is used to control the displayed virtual content.

第二方面，本申请实施例提供了一种虚拟内容的控制装置，应用于终端设备，所述终端设备与穿戴设备通信连接，所述装置包括：位置获取模块、数据获取模块、内容生成模块、显示控制模块以及内容控制模块，其中，所述位置获取模块用于获取所述穿戴设备与所述终端设备之间的相对空间位置关系；所述数据获取模块用于获取待显示的虚拟内容的内容数据；所述内容生成模块用于根据所述内容数据和所述相对空间位置关系，生成所述虚拟内容；所述显示控制模块用于对所述虚拟内容进行显示；所述内容控制模块用于根据所述穿戴设备检测到的操控动作参数以及所述相对空间位置关系的变化信息中的至少一种，对显示的所述虚拟内容进行控制。In a second aspect, an embodiment of the present application provides a virtual content control device, which is applied to a terminal device, where the terminal device is communicatively connected to a wearable device, and the device includes: a location acquisition module, a data acquisition module, a content generation module, A display control module and a content control module, wherein the position acquisition module is used to acquire the relative spatial position relationship between the wearable device and the terminal device; the data acquisition module is used to acquire the content of the virtual content to be displayed data; the content generation module is used to generate the virtual content according to the content data and the relative spatial positional relationship; the display control module is used to display the virtual content; the content control module is used to The displayed virtual content is controlled according to at least one of the manipulation action parameter detected by the wearable device and the change information of the relative spatial positional relationship.

第三方面，本申请实施例提供了一种终端设备，包括：一个或多个处理器；存储器；一个或多个应用程序，其中一个或多个应用程序被存储在存储器中并被配置为由一个或多个处理器执行，一个或多个程序配置用于执行上述第一方面提供的虚拟内容的控制方法。In a third aspect, embodiments of the present application provide a terminal device, including: one or more processors; a memory; and one or more application programs, wherein the one or more application programs are stored in the memory and configured to be One or more processors execute, and one or more programs are configured to execute the method for controlling virtual content provided in the first aspect.

第四方面，本申请实施例提供了一种计算机可读取存储介质，计算机可读取存储介质中存储有程序代码，程序代码可被处理器调用执行上述第一方面提供的虚拟内容的控制方法。In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, where a program code is stored in the computer-readable storage medium, and the program code can be invoked by a processor to execute the control method for virtual content provided in the first aspect above .

本申请实施例提供的方案，应用于终端设备，通过获取穿戴设备与终端设备之间的相对空间位置关系，以及获取待显示的虚拟内容的内容数据，以根据该内容数据和相对空间位置关系，生成虚拟内容，然后对虚拟内容进行显示，并根据穿戴设备检测到的操控动作参数以及相对空间位置关系的变化信息中的至少一种，对显示的虚拟内容进行控制，从而实现根据穿戴设备的空间位置，在虚拟空间中显示虚拟内容，并通过穿戴设备来控制虚拟内容的显示，实现了穿戴设备与终端设备之间的交互，提高了用户与虚拟内容的交互性和交互便利性。The solution provided by the embodiment of the present application is applied to a terminal device. By acquiring the relative spatial position relationship between the wearable device and the terminal device, and acquiring the content data of the virtual content to be displayed, according to the content data and the relative spatial position relationship, Generate virtual content, then display the virtual content, and control the displayed virtual content according to at least one of the manipulation action parameters detected by the wearable device and the change information of the relative spatial position relationship, so as to realize the space according to the wearable device. position, display the virtual content in the virtual space, and control the display of the virtual content through the wearable device, realize the interaction between the wearable device and the terminal device, and improve the interactivity and interaction convenience between the user and the virtual content.

附图说明Description of drawings

为了更清楚地说明本申请实施例中的技术方案，下面将对实施例描述中所需要使用的附图作简单地介绍，显而易见地，下面描述中的附图仅仅是本申请的一些实施例，对于本领域技术人员来讲，在不付出创造性劳动的前提下，还可以根据这些附图获得其他的附图。In order to illustrate the technical solutions in the embodiments of the present application more clearly, the following briefly introduces the drawings that are used in the description of the embodiments. Obviously, the drawings in the following description are only some embodiments of the present application. For those skilled in the art, other drawings can also be obtained from these drawings without creative effort.

图1示出了一种适用于本申请实施例的应用环境示意图。FIG. 1 shows a schematic diagram of an application environment suitable for this embodiment of the present application.

图2示出了一种适用于本申请实施例的穿戴设备的结构示意图。FIG. 2 shows a schematic structural diagram of a wearable device suitable for an embodiment of the present application.

图3示出了根据本申请一个实施例的虚拟内容的控制方法流程图。FIG. 3 shows a flowchart of a method for controlling virtual content according to an embodiment of the present application.

图4示出了根据本申请实施例的一种显示效果示意图。FIG. 4 shows a schematic diagram of a display effect according to an embodiment of the present application.

图5示出了根据本申请实施例的另一种显示效果示意图。FIG. 5 shows another schematic diagram of a display effect according to an embodiment of the present application.

图6示出了根据本申请实施例的又一种显示效果示意图。FIG. 6 shows another schematic diagram of a display effect according to an embodiment of the present application.

图7示出了根据本申请实施例的再一种显示效果示意图。FIG. 7 shows a schematic diagram of still another display effect according to an embodiment of the present application.

图8示出了根据本申请另一个实施例的虚拟内容的控制方法流程图。FIG. 8 shows a flowchart of a method for controlling virtual content according to another embodiment of the present application.

图9示出了根据本申请实施例的虚拟内容的显示方法中步骤S210的流程图。FIG. 9 shows a flowchart of step S210 in the method for displaying virtual content according to an embodiment of the present application.

图10示出了根据本申请实施例的虚拟内容的控制方法中步骤S220的流程图。FIG. 10 shows a flowchart of step S220 in the method for controlling virtual content according to an embodiment of the present application.

图11示出了根据本申请实施例的一种显示效果示意图。FIG. 11 shows a schematic diagram of a display effect according to an embodiment of the present application.

图12示出了根据本申请实施例的另一种显示效果示意图。FIG. 12 shows another schematic diagram of a display effect according to an embodiment of the present application.

图13示出了根据本申请实施例的又一种显示效果示意图。FIG. 13 shows another schematic diagram of a display effect according to an embodiment of the present application.

图14示出了根据本申请实施例的再一种显示效果示意图。FIG. 14 shows a schematic diagram of still another display effect according to an embodiment of the present application.

图15示出了根据本申请实施例的还一种显示效果示意图。FIG. 15 shows a schematic diagram of still another display effect according to an embodiment of the present application.

图16示出了根据本申请实施例的又另一种显示效果示意图。FIG. 16 shows yet another schematic diagram of a display effect according to an embodiment of the present application.

图17示出了根据本申请实施例的又再一种显示效果示意图。FIG. 17 shows yet another schematic diagram of a display effect according to an embodiment of the present application.

图18示出了根据本申请实施例的又还一种显示效果示意图。FIG. 18 shows yet another schematic diagram of a display effect according to an embodiment of the present application.

图19示出了根据本申请实施例的再另一种显示效果示意图。FIG. 19 shows a schematic diagram of still another display effect according to an embodiment of the present application.

图20示出了根据本申请实施例的再又一种显示效果示意图。FIG. 20 shows yet another schematic diagram of a display effect according to an embodiment of the present application.

图21A-21B示出了根据本申请实施例的再还一种显示效果示意图。21A-21B show still another schematic diagram of a display effect according to an embodiment of the present application.

图22示出了根据本申请一个实施例的虚拟内容的控制装置的框图。FIG. 22 shows a block diagram of an apparatus for controlling virtual content according to an embodiment of the present application.

图23是本申请实施例的用于执行根据本申请实施例的虚拟内容的控制方法的终端设备的框图。FIG. 23 is a block diagram of a terminal device for executing a method for controlling virtual content according to an embodiment of the present application.

图24是本申请实施例的用于保存或者携带实现根据本申请实施例的虚拟内容的控制方法的程序代码的存储单元。FIG. 24 is a storage unit used to store or carry a program code for implementing a method for controlling virtual content according to an embodiment of the present application, according to an embodiment of the present application.

具体实施方式Detailed ways

为了使本技术领域的人员更好地理解本申请方案，下面将结合本申请实施例中的附图，对本申请实施例中的技术方案进行清楚、完整地描述。In order to make those skilled in the art better understand the solutions of the present application, the following will clearly and completely describe the technical solutions in the embodiments of the present application with reference to the accompanying drawings in the embodiments of the present application.

近年来，随着增强现实(AR，Augmented Reality)技术的发展，AR相关的电子设备逐渐走入了人们的日常生活中。其中，AR是通过计算机系统提供的信息增加用户对现实世界感知的技术，其将计算机生成的虚拟对象、场景或系统提示信息等内容对象叠加到真实场景中，来增强或修改对现实世界环境或表示现实世界环境的数据的感知。在目前，可实现在移动终端的显示屏或头戴显示器的显示组件上对应的位置显示虚拟图像，使虚拟图像与真实场景叠加显示，让用户享受科幻式的观感体验。In recent years, with the development of Augmented Reality (AR, Augmented Reality) technology, AR-related electronic devices have gradually entered people's daily life. Among them, AR is a technology that increases the user's perception of the real world through the information provided by the computer system. It superimposes computer-generated virtual objects, scenes or system prompt information and other content objects into the real scene to enhance or modify the real-world environment or Perception of data representing a real-world environment. At present, it is possible to display a virtual image at a corresponding position on the display screen of the mobile terminal or the display component of the head-mounted display, so that the virtual image and the real scene are superimposed and displayed, so that the user can enjoy a sci-fi look and feel experience.

发明人经过长期研究发现，传统的AR显示技术中，在实现与虚拟内容的交互时，通常需要使用手持式控制器，才能与AR头显进行交互，从而实现与虚拟内容的交互，交互性差，且不利于随身携带，限制了使用场合。基于上述问题，发明人提出了本申请实施例中的虚拟内容的控制方法、装置、终端设备及存储介质，以较好地实现与显示的虚拟内容之间的交互。After long-term research, the inventor found that in traditional AR display technology, when interacting with virtual content, it is usually necessary to use a handheld controller to interact with the AR head-mounted display, so as to realize the interaction with virtual content, and the interactivity is poor. And it is not conducive to carry it with you, which limits the occasion of use. Based on the above problems, the inventor proposes a virtual content control method, device, terminal device and storage medium in the embodiments of the present application, so as to better realize the interaction with the displayed virtual content.

下面对本申请实施例提供的虚拟内容的控制方法的应用场景进行介绍。The following describes application scenarios of the virtual content control method provided by the embodiments of the present application.

请参见图1，示出了本申请实施例提供的虚拟内容的控制方法的应用场景的示意图，该应用场景包括交互系统10。该交互系统10包括：终端设备100以及穿戴设备200。Referring to FIG. 1 , it is a schematic diagram showing an application scenario of the virtual content control method provided by the embodiment of the present application, where the application scenario includes aninteraction system 10 . Theinteractive system 10 includes: aterminal device 100 and awearable device 200 .

在本申请实施例中，终端设备100可以是头戴显示装置，也可以是手机、平板等移动设备。终端设备100为头戴显示装置时，头戴显示装置可以为一体式头戴显示装置。终端设备100也可以是与外接式/接入式头戴显示装置连接的手机等智能终端，即终端设备100可作为头戴显示装置的处理和存储设备，插入或者接入外接式头戴显示装置，在头戴显示装置中对虚拟内容进行显示。In this embodiment of the present application, theterminal device 100 may be a head-mounted display device, or may be a mobile device such as a mobile phone or a tablet. When theterminal device 100 is a head-mounted display device, the head-mounted display device may be an integrated head-mounted display device. Theterminal device 100 can also be a smart terminal such as a mobile phone connected to an external/accessible head-mounted display device, that is, theterminal device 100 can be used as a processing and storage device of the head-mounted display device, and can be inserted into or connected to the external head-mounted display device. , the virtual content is displayed on the head-mounted display device.

在本申请实施例中，穿戴设备200可以是智能手表、智能手环等智能穿戴设备，也可以是传统手表等仅具有数据显示功能的传统穿戴设备。In the embodiment of the present application, thewearable device 200 may be a smart wearable device such as a smart watch and a smart bracelet, or may be a traditional wearable device such as a traditional watch that only has a data display function.

在一些实施方式中，穿戴设备200可包括无线通信模块，穿戴设备200可通过无线通信模块与终端设备100建立通信连接。其中，无线通信模块可以是蓝牙、WiFi(Wireless-Fidelity，无线保真)、ZigBee(紫峰技术)等模块。与终端设备100通信连接的穿戴设备200，可以与终端设备100进行信息以及指令的交互。In some embodiments, thewearable device 200 may include a wireless communication module, and thewearable device 200 may establish a communication connection with theterminal device 100 through the wireless communication module. The wireless communication module may be a module such as Bluetooth, WiFi (Wireless-Fidelity, wireless fidelity), ZigBee (Zifeng technology). Thewearable device 200 communicatively connected to theterminal device 100 can interact with theterminal device 100 for information and instructions.

在一些实施方式中，穿戴设备200上设置有标记物201。其中，该标记物201可以包括至少一个具有一个或者多个特征点的子标记物。在一些实施方式中，标记物201可以集成于穿戴设备200中，也可以粘贴附着于穿戴设备200上，还可以是显示于穿戴设备200的显示屏上。当标记物201处于终端设备100的视野范围内时，终端设备100可将上述处于视野范围内的标记物201作为目标标记物，并可以识别该目标标记物的图像，得到该目标标记物相对终端设备100的位置、朝向等空间位置信息，从而得到终端设备100与穿戴设备200之间的相对空间位置信息。终端设备100可基于目标标记物相对终端设备100的空间位置信息显示相应的虚拟对象，同时可根据该目标标记物实现对穿戴设备200的定位及追踪。可以理解地，具体的标记物201在本申请实施例中并不作为限定，仅需可以被终端设备100识别追踪即可。In some embodiments, amarker 201 is provided on thewearable device 200 . Wherein, themarker 201 may include at least one sub-marker having one or more feature points. In some embodiments, themarker 201 can be integrated into thewearable device 200 , can also be pasted and attached to thewearable device 200 , or can be displayed on a display screen of thewearable device 200 . When themarker 201 is within the field of view of theterminal device 100, theterminal device 100 can use themarker 201 within the field of view as a target marker, and can recognize the image of the target marker, and obtain the relative terminal of the target marker The relative spatial position information between theterminal device 100 and thewearable device 200 is obtained by obtaining spatial position information such as the position and orientation of thedevice 100 . Theterminal device 100 can display a corresponding virtual object based on the spatial position information of the target marker relative to theterminal device 100 , and can simultaneously locate and track thewearable device 200 according to the target marker. It can be understood that thespecific marker 201 is not limited in this embodiment of the present application, and only needs to be identified and tracked by theterminal device 100 .

在一些实施方式中，终端设备100还可以追踪穿戴设备200的形状，确定终端设备100与穿戴设备200之间的相对空间位置关系。In some embodiments, theterminal device 100 may also track the shape of thewearable device 200 to determine the relative spatial positional relationship between theterminal device 100 and thewearable device 200 .

在一些实施方式中，终端设备100还可以根据设置在穿戴设备200上的光点，确定终端设备100与穿戴设备200之间的相对空间位置关系。In some embodiments, theterminal device 100 may also determine the relative spatial positional relationship between theterminal device 100 and thewearable device 200 according to the light spots set on thewearable device 200 .

在一些实施方式中，穿戴设备200包括惯性测量单元(Inertial measurementunit，IMU)。终端设备100可以根据IMU的测量数据以及追踪的穿戴设备200的形状信息，精确得到终端设备100与穿戴设备200之间的相对空间位置关系。终端设备100可以根据IMU的测量数据以及设置在穿戴设备200上的光点，精确得到终端设备100与穿戴设备200之间的相对空间位置关系。In some embodiments, thewearable device 200 includes an inertial measurement unit (IMU). Theterminal device 100 can accurately obtain the relative spatial positional relationship between theterminal device 100 and thewearable device 200 according to the measurement data of the IMU and the tracked shape information of thewearable device 200 . Theterminal device 100 can accurately obtain the relative spatial positional relationship between theterminal device 100 and thewearable device 200 according to the measurement data of the IMU and the light spot set on thewearable device 200 .

在一些实施方式中，穿戴设备200上可以包括按键、触摸屏和转盘中的至少一种。例如，在智能手表的表盘中设置转盘，在智能手表的表盘的侧面设置按键，在智能手表的表盘的表面设置触控屏。穿戴设备200可以根据用户在按键、触摸屏和转盘上的操控操作生成操控动作参数，并将该操控动作参数发送至终端设备100。终端设备100可根据操控动作参数控制虚拟内容的显示(例如控制虚拟内容旋转、位移、缩放等)。In some embodiments, thewearable device 200 may include at least one of buttons, a touch screen, and a dial. For example, a dial is arranged on the watch face of the smart watch, buttons are arranged on the side of the watch face of the smart watch, and a touch screen is arranged on the surface of the watch face of the smart watch. Thewearable device 200 can generate the manipulation action parameters according to the manipulation operations of the user on the buttons, the touch screen and the turntable, and send the manipulation action parameters to theterminal device 100 . Theterminal device 100 can control the display of the virtual content according to the manipulation action parameters (eg, control the rotation, displacement, zoom, etc. of the virtual content).

在一些实施方式中，穿戴设备200上设置有摄像头，可用于采集用户或周围环境的图像。其中，该摄像头可以设置成可旋转模式，可以以圆的方式进行旋转，以采集不同视野范围内的图像。例如，智能手表上包含摄像头，摄像头可设置在表盘的侧面，通过转动表盘，可以转动摄像头的位置，使摄像头可以采集到不同视野范围内的图像。In some embodiments, thewearable device 200 is provided with a camera, which can be used to capture images of the user or the surrounding environment. Among them, the camera can be set to a rotatable mode, and can be rotated in a circular manner to collect images in different fields of view. For example, a smart watch contains a camera, and the camera can be set on the side of the dial. By turning the dial, the position of the camera can be rotated, so that the camera can collect images in different fields of view.

例如，请再次参阅图1，终端设备100为头戴显示装置，穿戴设备200为智能手表，用户可以通过佩戴的头戴显示装置，实时扫描穿戴设备200上的标记物201，可以看到虚拟汽车模型401叠加显示于现实空间的穿戴设备200上，体现了虚拟内容的增强现实的显示效果，体现了终端设备与穿戴设备之间的交互。For example, please refer to FIG. 1 again, theterminal device 100 is a head-mounted display device, and thewearable device 200 is a smart watch. The user can scan themarker 201 on thewearable device 200 in real time through the wearing head-mounted display device, and can see the virtual car Themodel 401 is superimposed and displayed on thewearable device 200 in the real space, which reflects the display effect of the augmented reality of the virtual content and the interaction between the terminal device and the wearable device.

请参见图2，图2示出了本申请实施例提供的一种穿戴设备的示意图，穿戴设备200为智能手表，包括智能手表的主体220以及表带240，智能手表的主体220包括表盘221。其中，表盘221的侧面设有按键222，表盘221的表面设有触摸屏223，表盘221的外围设有转盘224。标记物201设置在表盘221的外圈，与表盘固定。可以理解地，穿戴设备并不仅限于图2所示的形态，可以是其他形态，在此不作限制。Referring to FIG. 2 , FIG. 2 shows a schematic diagram of a wearable device provided by an embodiment of the present application. Thewearable device 200 is a smart watch, including amain body 220 of the smart watch and astrap 240 , and themain body 220 of the smart watch includes adial 221 . The side of thedial 221 is provided withbuttons 222 , the surface of thedial 221 is provided with atouch screen 223 , and the periphery of thedial 221 is provided with aturntable 224 . Themarker 201 is provided on the outer circumference of thedial 221 and is fixed to the dial. It can be understood that the wearable device is not limited to the form shown in FIG. 2 , and may be in other forms, which are not limited here.

基于上述交互系统，本申请实施例提供了一种虚拟内容的控制方法，应用于上述交互系统的终端设备以及穿戴设备。下面对具体的虚拟内容的控制方法进行介绍。Based on the above interactive system, an embodiment of the present application provides a method for controlling virtual content, which is applied to a terminal device and a wearable device of the above interactive system. The specific control method of virtual content is introduced below.

请参阅图3，本申请实施例提供了一种虚拟内容的控制方法，可应用于上述终端设备，终端设备与穿戴设备通信连接，该虚拟内容的控制方法可以包括：Referring to FIG. 3 , an embodiment of the present application provides a method for controlling virtual content, which can be applied to the above-mentioned terminal device, where the terminal device is communicatively connected to a wearable device, and the method for controlling virtual content may include:

步骤S110：获取穿戴设备与终端设备之间的相对空间位置关系。Step S110: Obtain the relative spatial positional relationship between the wearable device and the terminal device.

传统的增强现实的显示技术中，在实现与虚拟内容的交互时，通常需要使用手持式控制器，才能与AR头显进行交互，交互性差，且不利于随身携带，限制了使用场合。因此，可以通过随身佩戴的穿戴设备，与显示的虚拟内容进行交互，以提高用户与显示的虚拟内容之间的交互性和携带便利性。In the traditional augmented reality display technology, when interacting with virtual content, it is usually necessary to use a handheld controller to interact with the AR head-mounted display. Therefore, it is possible to interact with the displayed virtual content through a wearable device that is worn around the body, so as to improve the interactivity and portability between the user and the displayed virtual content.

在本申请实施例中，终端设备可以获取穿戴设备与终端设备之间的相对空间位置关系，以根据该相对空间位置关系，对虚拟内容进行显示。其中，该相对空间位置关系可以包括穿戴设备与终端设备之间的相对位置信息以及姿态信息等，姿态信息可以是穿戴设备相对终端设备的朝向及旋转角度。In this embodiment of the present application, the terminal device may acquire the relative spatial position relationship between the wearable device and the terminal device, so as to display virtual content according to the relative spatial position relationship. The relative spatial position relationship may include relative position information and attitude information between the wearable device and the terminal device, and the attitude information may be the orientation and rotation angle of the wearable device relative to the terminal device.

在一些实施例中，穿戴设备上可设置有标记物，如在智能手表的表盘上设置标记物。因此，在需要获取穿戴设备与终端设备之间的相对空间位置关系时，终端设备可以通过识别穿戴设备上的标记物，来得到穿戴设备与终端设备之间的相对空间位置关系。具体地，终端设备可以通过图像采集装置采集包含标记物的图像，然后对该图像中的标记物进行识别，可以得到标记物相对终端设备的空间位置信息，该空间位置信息可包括标记物相对终端设备的位置信息、旋转方向及旋转角度等。因此，终端设备可根据标记物在穿戴设备上的具体位置，以及标记物相对终端设备的空间位置信息，获取到穿戴设备与终端设备之间的相对空间位置关系。在一些方式中，也可直接将标记物相对终端设备的空间位置信息作为穿戴设备与终端设备之间的相对空间位置关系。In some embodiments, the wearable device may be provided with a marker, such as a marker on the dial of a smart watch. Therefore, when the relative spatial positional relationship between the wearable device and the terminal device needs to be obtained, the terminal device can obtain the relative spatial positional relationship between the wearable device and the terminal device by identifying the markers on the wearable device. Specifically, the terminal device can collect an image containing the marker through the image acquisition device, and then identify the marker in the image to obtain the spatial position information of the marker relative to the terminal device, and the spatial position information can include the marker relative to the terminal. The location information, rotation direction and rotation angle of the device, etc. Therefore, the terminal device can obtain the relative spatial positional relationship between the wearable device and the terminal device according to the specific position of the marker on the wearable device and the spatial position information of the marker relative to the terminal device. In some manners, the spatial position information of the marker relative to the terminal device can also be directly used as the relative spatial position relationship between the wearable device and the terminal device.

其中，终端设备采集包含标记物的图像，可以是通过调整终端设备在现实空间中的空间位置，也可以通过调整穿戴设备在现实空间中的空间位置，以使该穿戴设备上的标记物处于终端设备的图像采集装置的视野范围内，从而使终端设备可以对该标记物进行图像采集和图像识别。其中，图像采集装置的视野范围可以由视场角的大小决定。Among them, the terminal device collects the image containing the marker, either by adjusting the spatial position of the terminal device in the real space, or by adjusting the spatial position of the wearable device in the real space, so that the marker on the wearable device is in the terminal The image acquisition device of the device is within the field of view, so that the terminal device can perform image acquisition and image recognition on the marker. Wherein, the field of view of the image acquisition device may be determined by the size of the field of view.

在一些实施方式中，上述标记物可以包括若干个子标记物，子标记物可以是具有一定形状的图案。在一个实施例中，每个子标记物可具有一个或多个特征点，其中，特征点的形状不做限定，可以是圆点、圆环，也可以是三角形、其他形状。另外，不同标记物内的子标记物的分布规则不同，因此，每个标记物可具备不同的身份信息。终端设备通过识别标记物中包含的子标记物，可以获取与标记物对应的身份信息，该身份信息可以是编码等可用于唯一标识标记物的信息，但不限于此。In some embodiments, the above-mentioned marker may include several sub-markers, and the sub-markers may be a pattern with a certain shape. In one embodiment, each sub-marker may have one or more feature points, wherein the shape of the feature points is not limited, and may be a dot, a ring, or a triangle or other shapes. In addition, the distribution rules of sub-markers in different markers are different, so each marker can have different identity information. The terminal device can obtain identity information corresponding to the marker by identifying the sub-marker contained in the marker, and the identity information may be information such as code that can be used to uniquely identify the marker, but is not limited to this.

作为一种实施方式，标记物的轮廓可以为矩形，也可以为圆形，还可以与穿戴设备的实际形状相符，如与智能手表的表盘的实际形状相符的圆形，与智能手环的实际形状相符的方形。当然，标记物的形状也可以是其他形状，在此不做限定，形状区域以及该区域内的多个子标记物构成一个标记物。当然，上述标记物也可以为由光点构成的自身可以发光的物体等，光点标记物可以发射不同波段或不同颜色的光，终端设备通过识别光点标记物发出的光的波段或颜色等信息获取与标记物对应的身份信息。需要说明的是，具体的标记物的形状、样式、尺寸、颜色、特征点数量以及分布在本实施例中并不作为限定，仅需要标记物能被终端设备识别追踪即可。As an implementation manner, the outline of the marker may be a rectangle or a circle, and may also conform to the actual shape of the wearable device, such as a circle conforming to the actual shape of the dial of a smart watch, and conforming to the actual shape of the smart bracelet. A square that matches the shape. Of course, the shape of the marker may also be other shapes, which are not limited herein, and the shape area and the multiple sub-markers in the area constitute a marker. Of course, the above-mentioned markers can also be objects composed of light spots that can emit light by themselves, etc. The light spot markers can emit light in different wavelength bands or colors, and the terminal device can identify the wavelength band or color of the light emitted by the light spot markers, etc. The information acquires the identity information corresponding to the marker. It should be noted that the specific shape, style, size, color, number of feature points, and distribution of the specific marker are not limited in this embodiment, as long as the marker can be identified and tracked by the terminal device.

在一些实施例中，穿戴设备上的标记物可以为多个，且每个标记物相对穿戴设备的位置信息、旋转方向及旋转角度固定，也即，标记物与穿戴设备之间可具备刚性的结构关系。在一些实施方式中，终端设备采集到的图像中的标记物的个数可以是一个。作为一种方式，可以是通过识别图像中的一个标记物相对终端设备的空间位置信息，并根据预先存储的该标记物相对穿戴设备的位置信息、旋转方向及旋转角度，来得到穿戴设备与终端设备之间的相对空间位置关系。在另一些实施方式中，终端设备采集到的图像中的标记物的个数可以是多个。作为一种方式，可以是通过识别多个标记物中每个标记物相对终端设备的空间位置信息，并根据每个标记物相对穿戴设备的位置信息、旋转方向及旋转角度，得到穿戴设备与终端设备之间的相对空间位置关系。In some embodiments, there may be multiple markers on the wearable device, and the position information, rotation direction and rotation angle of each marker relative to the wearable device are fixed, that is, there may be a rigid relationship between the marker and the wearable device. structural relationship. In some embodiments, the number of markers in the image collected by the terminal device may be one. As a method, the wearable device and the terminal can be obtained by identifying the spatial position information of a marker in the image relative to the terminal device, and according to the pre-stored position information, rotation direction and rotation angle of the marker relative to the wearable device. The relative spatial positional relationship between devices. In other embodiments, the number of markers in the image collected by the terminal device may be multiple. As a method, the wearable device and the terminal can be obtained by identifying the spatial position information of each marker in the plurality of markers relative to the terminal device, and according to the position information, rotation direction and rotation angle of each marker relative to the wearable device. The relative spatial positional relationship between devices.

在一些实施方式中，标记物可以设置在穿戴设备的外壳上，如设置在智能手表的表盘的外圈(与表盘是固定的)，也可以采用图像的形式显示在穿戴设备的屏幕上，从而实现对穿戴设备进行定位及追踪。进一步的，当标记物设置在穿戴设备的外壳上时，可在标记物上设置一个滤光层，以对标记物进行隐藏。In some embodiments, the marker can be set on the casing of the wearable device, such as on the outer ring of the dial of the smart watch (fixed with the dial), or displayed on the screen of the wearable device in the form of an image, so as to Realize the positioning and tracking of wearable devices. Further, when the marker is arranged on the shell of the wearable device, a filter layer can be arranged on the marker to hide the marker.

进一步的，为了增大终端设备对穿戴设备的追踪范围，如用户翻转手腕时仍然可以获取到穿戴设备与终端设备之间的相对空间位置关系，可以在穿戴设备的靠近用户的一侧部位上设置标记物，比如，可以在智能手表靠近用户一侧的表带上设置标记物。也可以在穿戴设备的四周设置标记物，比如在穿戴设备佩戴时相对用户的左侧或右侧设置标记物等，但不限于此。Further, in order to increase the tracking range of the wearable device by the terminal device, for example, when the user turns the wrist, the relative spatial positional relationship between the wearable device and the terminal device can still be obtained, which can be set on the side of the wearable device near the user. Markers, for example, can be placed on the wristband of the smartwatch on the side of the smartwatch closer to the user. Markers can also be set around the wearable device, for example, when the wearable device is worn, markers are set on the left or right side of the user, etc., but not limited to this.

当然，获取穿戴设备与终端设备之间的相对空间位置关系的方式在本申请实施例中可以不作为限定。例如，也可以是通过穿戴设备的IMU的测量数据以及穿戴设备的轮廓识别数据，来获取上述穿戴设备与终端设备之间的相对空间位置关系，也可以是通过穿戴设备的IMU的测量数据以及穿戴设备的光点图像识别，来获取上述穿戴设备与终端设备之间的相对空间位置关系。Certainly, the manner of acquiring the relative spatial positional relationship between the wearable device and the terminal device may not be limited in this embodiment of the present application. For example, the relative spatial positional relationship between the above-mentioned wearable device and the terminal device can also be obtained through the measurement data of the IMU of the wearable device and the contour recognition data of the wearable device, or through the measurement data of the IMU of the wearable device and the wearable device. The light spot image recognition of the device is used to obtain the relative spatial positional relationship between the above-mentioned wearable device and the terminal device.

步骤S120：获取待显示的虚拟内容的内容数据。Step S120: Acquire content data of the virtual content to be displayed.

在本申请实施例中，终端设备在对虚拟内容进行显示时，终端设备需要获取待显示的虚拟内容的内容数据。其中，该内容数据可以包括待显示的虚拟内容的模型数据，模型数据为用于渲染虚拟内容的数据。例如，模型数据可以包括用于建立虚拟内容对应的颜色数据、顶点坐标数据、轮廓数据等。另外，待显示的虚拟内容的模型数据可以是存储于终端设备中，也可以是从穿戴设备、服务器等其他电子设备获取。In this embodiment of the present application, when a terminal device displays virtual content, the terminal device needs to acquire content data of the virtual content to be displayed. The content data may include model data of the virtual content to be displayed, where the model data is data used for rendering the virtual content. For example, the model data may include color data, vertex coordinate data, outline data, etc. for establishing the corresponding virtual content. In addition, the model data of the virtual content to be displayed may be stored in the terminal device, or may be obtained from other electronic devices such as a wearable device and a server.

在一些实施方式中，上述待显示的虚拟内容的内容数据可以根据穿戴设备上的标记物的身份信息获取，也就是说，可以通过标记物的身份信息读取对应的待显示的虚拟内容的内容数据，使显示的虚拟内容与穿戴设备上的标记物的身份信息对应。In some embodiments, the content data of the virtual content to be displayed can be obtained according to the identity information of the marker on the wearable device, that is, the content of the corresponding virtual content to be displayed can be read through the identity information of the marker data, so that the displayed virtual content corresponds to the identity information of the marker on the wearable device.

在一些实施方式中，上述待显示的虚拟内容的内容数据可以根据终端设备已经显示在虚拟空间的虚拟对象获取，也就是说，可以通过终端设备当前显示在虚拟空间的虚拟对象获取对应的待显示的虚拟内容的内容数据，其中，待显示的虚拟内容可以是虚拟对象的扩展内容。例如，为了不干扰用户的视野，AR眼镜中显示的虚拟通知信息会比较简单，若用户需要查看详细的通知信息，终端设备可以根据当前显示的虚拟通知信息，获取到虚拟的详细的通知信息，并将该虚拟的详细的通知信息的数据作为待显示的虚拟内容的内容数据。In some embodiments, the content data of the virtual content to be displayed can be obtained according to the virtual object that has been displayed in the virtual space by the terminal device, that is, the corresponding virtual object to be displayed can be obtained through the virtual object currently displayed in the virtual space by the terminal device The content data of the virtual content, wherein the virtual content to be displayed may be the extended content of the virtual object. For example, in order not to interfere with the user's field of vision, the virtual notification information displayed in the AR glasses is relatively simple. If the user needs to view the detailed notification information, the terminal device can obtain the virtual detailed notification information according to the currently displayed virtual notification information. The data of the virtual detailed notification information is used as the content data of the virtual content to be displayed.

当然，以上待显示的虚拟内容的内容数据的获取方式仅为举例，具体的虚拟内容的内容数据在本申请实施例中可以不作为限定。例如，也可以是根据采集的终端设备所处环境的场景图像获取待显示的虚拟内容的内容数据。Of course, the manner of acquiring the content data of the virtual content to be displayed above is only an example, and the specific content data of the virtual content may not be limited in the embodiments of the present application. For example, the content data of the virtual content to be displayed may also be acquired according to the collected scene image of the environment where the terminal device is located.

步骤S130：根据内容数据和相对空间位置关系，生成虚拟内容。Step S130: Generate virtual content according to the content data and the relative spatial positional relationship.

在本申请实施例中，终端设备获取到上述内容数据后，可以根据上述内容数据和上述相对空间位置关系，生成待显示的虚拟内容。其中，终端设备根据内容数据和相对空间位置关系，生成虚拟内容，可以是根据上述内容数据构建出虚拟内容，根据穿戴设备与终端设备之间的相对空间位置关系，获取到虚拟内容的渲染位置，从而根据该渲染位置渲染出虚拟内容。In this embodiment of the present application, after acquiring the above-mentioned content data, the terminal device may generate virtual content to be displayed according to the above-mentioned content data and the above-mentioned relative spatial positional relationship. Wherein, the terminal device generates virtual content according to the content data and the relative spatial positional relationship, which may be to construct virtual content according to the above-mentioned content data, and obtain the rendering position of the virtual content according to the relative spatial positional relationship between the wearable device and the terminal device, Thus, the virtual content is rendered according to the rendering position.

在一些实施方式中，由于终端设备已经得到穿戴设备与终端设备之间的相对空间位置关系，因此，终端设备可以获取到现实空间中的穿戴设备的空间位置坐标，将该空间位置坐标转换为虚拟空间中的空间坐标。其中，虚拟空间中可包括虚拟摄像头，该虚拟摄像头用于模拟用户的人眼，虚拟摄像头在虚拟空间中的位置可看作终端设备在虚拟空间中的位置。可根据虚拟空间中待显示的虚拟内容与穿戴设备的位置关系，以虚拟摄像头作为参照，则可以获取到虚拟内容相对虚拟摄像头的空间位置，从而得到虚拟内容在虚拟空间中的渲染坐标，即得到了虚拟内容的渲染位置，其中，该渲染位置可用作虚拟内容的渲染坐标，以实现虚拟内容渲染于渲染位置处。其中，上述渲染坐标可以是虚拟内容在虚拟空间中以虚拟摄像头为原点(也可看作是以人眼为原点)的三维空间坐标。In some embodiments, since the terminal device has obtained the relative spatial position relationship between the wearable device and the terminal device, the terminal device can obtain the spatial position coordinates of the wearable device in the real space, and convert the spatial position coordinates into virtual Spatial coordinates in space. Wherein, the virtual space may include a virtual camera, the virtual camera is used to simulate the user's human eyes, and the position of the virtual camera in the virtual space may be regarded as the position of the terminal device in the virtual space. According to the positional relationship between the virtual content to be displayed in the virtual space and the wearable device, and using the virtual camera as a reference, the spatial position of the virtual content relative to the virtual camera can be obtained, so as to obtain the rendering coordinates of the virtual content in the virtual space, that is, The rendering position of the virtual content is provided, wherein the rendering position can be used as the rendering coordinate of the virtual content, so as to realize the rendering of the virtual content at the rendering position. The above-mentioned rendering coordinates may be the three-dimensional space coordinates of the virtual content in the virtual space with the virtual camera as the origin (it may also be regarded as the human eye as the origin).

可以理解的是，终端设备得到用于虚拟空间中渲染虚拟内容的渲染坐标之后，终端设备可以根据获取到的待显示的虚拟内容对应的内容数据，构建三维的虚拟内容，以及根据上述渲染坐标渲染该虚拟内容，其中，渲染虚拟内容可以得到三维虚拟内容中各个顶点的RGB值及对应的坐标等。It can be understood that, after the terminal device obtains the rendering coordinates for rendering the virtual content in the virtual space, the terminal device can construct the three-dimensional virtual content according to the obtained content data corresponding to the virtual content to be displayed, and render the virtual content according to the above rendering coordinates. In the virtual content, the RGB values and corresponding coordinates of each vertex in the three-dimensional virtual content can be obtained by rendering the virtual content.

步骤S140：对虚拟内容进行显示。Step S140: Display the virtual content.

在本申请实施例中，终端设备生成虚拟内容后，可以对虚拟内容进行显示。具体地，终端设备构建并渲染出虚拟内容后，可以将渲染后的虚拟内容转化为显示画面，获取相应的显示数据，该显示数据可以包括显示画面中各个像素点的RGB值及对应的像素点坐标等，终端设备可根据该显示数据生成显示画面，并将显示画面通过显示屏或投射模组投射到显示镜片上，从而显示出虚拟内容。用户通过头戴显示装置的显示镜片，可以看到三维立体的虚拟内容叠加显示在真实世界中的穿戴设备上，实现增强现实的效果。从而实现根据随身佩戴的穿戴设备上的标记物的空间位置，在虚拟空间中显示相应的虚拟内容，使用户可以观察到虚拟内容叠加于真实世界的效果，提升虚拟内容的显示效果。In this embodiment of the present application, after the terminal device generates the virtual content, the virtual content may be displayed. Specifically, after the terminal device constructs and renders the virtual content, it can convert the rendered virtual content into a display screen, and obtain corresponding display data, where the display data may include the RGB values of each pixel in the display screen and the corresponding pixel points Coordinates, etc., the terminal device can generate a display screen according to the display data, and project the display screen onto the display lens through the display screen or the projection module, thereby displaying the virtual content. Through the display lens of the head-mounted display device, the user can see the three-dimensional virtual content superimposed and displayed on the wearable device in the real world to achieve the effect of augmented reality. Therefore, according to the spatial position of the marker on the wearable device, the corresponding virtual content is displayed in the virtual space, so that the user can observe the effect of the virtual content superimposed on the real world, and the display effect of the virtual content is improved.

例如，请参见图1，终端设备100为头戴显示装置，穿戴设备200为智能手表，用户可以通过佩戴的头戴显示装置，实时扫描穿戴设备200上的标记物201，可以看到三维的虚拟汽车模型叠加显示于现实空间的穿戴设备200上。又例如，请参见图4，为了不干扰用户的视野，终端设备100显示的虚拟通知信息300(虚拟通知信息)比较简单，当用户需要查看更详细的邮件信息时，用户可以通过佩戴的头戴显示装置，实时扫描穿戴设备200上的标记物201，可以看到详细的虚拟邮件信息402叠加显示于现实空间的穿戴设备200上，体现了虚拟内容的增强现实效果。For example, referring to FIG. 1 , theterminal device 100 is a head-mounted display device, and thewearable device 200 is a smart watch. The user can scan themarker 201 on thewearable device 200 in real time through the wearing head-mounted display device, and can see a three-dimensional virtual The car model is superimposed and displayed on thewearable device 200 in the real space. For another example, referring to FIG. 4, in order not to interfere with the user's field of vision, the virtual notification information 300 (virtual notification information) displayed by theterminal device 100 is relatively simple. When the user needs to view more detailed email information, the user can The display device scans themarker 201 on thewearable device 200 in real time, and can see that the detailedvirtual mail information 402 is superimposed and displayed on thewearable device 200 in the real space, reflecting the augmented reality effect of the virtual content.

在一些实施方式中，虚拟内容与穿戴设备的位置关系可以是固定的，如虚拟内容固定显示在穿戴设备上的预定区域，也可以和具体的虚拟内容有关，如虚拟内容为用户操作界面(User Interface，UI)时，虚拟内容显示在穿戴设备的前方，虚拟内容为邮件信息时，虚拟内容显示在穿戴设备的上方，在此不做限定。In some embodiments, the positional relationship between the virtual content and the wearable device may be fixed, for example, the virtual content is fixedly displayed in a predetermined area on the wearable device, or it may be related to the specific virtual content, for example, the virtual content is a user interface (User Interface, UI), the virtual content is displayed in front of the wearable device, and when the virtual content is email information, the virtual content is displayed above the wearable device, which is not limited herein.

步骤S150：根据穿戴设备检测到的操控动作参数以及相对空间位置关系的变化信息中的至少一种，对显示的虚拟内容进行控制。Step S150: Control the displayed virtual content according to at least one of the manipulation action parameters detected by the wearable device and the change information of the relative spatial positional relationship.

在本申请实施例中，终端设备可以利用穿戴设备控制显示的虚拟内容。具体地，终端设备可以根据穿戴设备检测到的操控动作参数以及相对空间位置关系的变化信息中的至少一种，对显示的虚拟内容进行控制。In this embodiment of the present application, the terminal device may use the wearable device to control the displayed virtual content. Specifically, the terminal device may control the displayed virtual content according to at least one of the manipulation action parameters detected by the wearable device and the change information of the relative spatial positional relationship.

其中，终端设备根据穿戴设备检测到的操控动作参数以及相对空间位置关系的变化信息中的至少一种，对显示的虚拟内容进行控制，可以理解为，终端设备可以根据穿戴设备检测到的操控动作参数，对显示的虚拟内容进行控制，也可以根据相对空间位置关系的变化信息，对显示的虚拟内容进行控制，还可以同时根据穿戴设备检测到的操控动作参数以及相对空间位置关系的变化信息，对显示的虚拟内容进行控制。在一些实施方式中，穿戴设备检测到的操控动作参数为多种时，终端设备可以根据其中的一种操控动作参数，对显示的虚拟内容进行控制，也可以同时根据其中的多种操控动作参数，对显示的虚拟内容进行控制。The terminal device controls the displayed virtual content according to at least one of the manipulation action parameters detected by the wearable device and the change information of the relative spatial position relationship. It can be understood that the terminal device can control the displayed virtual content according to the manipulation action detected by the wearable device. parameters, to control the displayed virtual content, or to control the displayed virtual content according to the change information of the relative spatial positional relationship, and also to control the displayed virtual content according to the control action parameters detected by the wearable device and the change information of the relative spatial positional relationship, Control the displayed virtual content. In some embodiments, when there are multiple manipulation action parameters detected by the wearable device, the terminal device may control the displayed virtual content according to one manipulation action parameter, or may simultaneously control the displayed virtual content according to multiple manipulation action parameters. , to control the displayed virtual content.

在本申请实施例中，上述操控动作参数具体是指用户在穿戴设备的操控区所做的操作动作信息，也就是说，该操控动作参数表征的是用户在穿戴设备的操控区所做的具体动作。例如，穿戴设备上的操控区中设有按键时，当用户在按键上进行按压动作时，操控动作参数可以为用户按压按键时产生的压力信号，穿戴设备通过压力信号则可获知用户对按键进行了按压动作。当然，上述操控动作参数仅为举例，具体的操控动作参数在本申请实施例中并不作为限定，仅需要操控动作参数与穿戴设备的操控区对应即可。其中，穿戴设备的操控区除了举例中的按键，还可以包括其他，如触摸屏、转盘等，具体的穿戴设备的操控区在本申请实施例中也不作为限定。In the embodiment of the present application, the above-mentioned manipulation action parameter specifically refers to the manipulation action information performed by the user in the manipulation area of the wearable device, that is to say, the manipulation action parameter represents the specific action performed by the user in the manipulation zone of the wearable device action. For example, when there is a button in the control area on the wearable device, when the user presses the button, the control action parameter can be the pressure signal generated when the user presses the button, and the wearable device can know the user's operation on the button through the pressure signal. press action. Of course, the above-mentioned manipulation action parameters are only examples, and the specific manipulation action parameters are not limited in the embodiments of the present application, and the manipulation action parameters only need to correspond to the manipulation area of the wearable device. The control area of the wearable device may also include other buttons, such as a touch screen, a turntable, etc., in addition to the buttons in the example, and the specific control area of the wearable device is not limited in the embodiments of the present application.

在一些实施例中，终端设备可以根据穿戴设备检测到的操控动作参数，生成操控动作参数对应的操控指令，并根据操控指令对显示的虚拟内容进行控制。在一些实施方式中，操控动作参数与操控指令的对应关系可以预先存储于终端设备中，该对应关系可以是用户自行设置，也可以是终端设备出厂时默认的，还可以是终端设备从服务器获取的。In some embodiments, the terminal device may generate a manipulation instruction corresponding to the manipulation action parameter according to the manipulation action parameter detected by the wearable device, and control the displayed virtual content according to the manipulation instruction. In some embodiments, the corresponding relationship between the manipulation action parameters and the manipulation instruction may be pre-stored in the terminal device, and the corresponding relationship may be set by the user, or the default when the terminal device leaves the factory, or the terminal device may obtain it from the server of.

终端设备在接收到穿戴设备发送的操控动作参数时，可以根据操控动作参数与操控指令的对应关系，对显示的虚拟内容进行控制。其中，不同的操控指令对应不同的控制效果，该控制效果可控制虚拟内容显示不同的效果。例如，请参见图5，用户转动智能手表的表盘上的转盘时，终端设备可根据转盘的转动参数，生成操控指令，控制虚拟内容400进行选项切换，又例如，用户在表盘的触控屏上操作不同的手势时，终端设备可根据触控屏上的不同手势生成不同的操控指令，以控制虚拟内容显示不同的效果。当然，以上控制效果仅为举例，具体的操控指令对应的对虚拟内容的控制效果在本申请实施例中可以不作为限定。When the terminal device receives the manipulation action parameters sent by the wearable device, it can control the displayed virtual content according to the corresponding relationship between the manipulation action parameters and the manipulation instruction. Wherein, different manipulation instructions correspond to different control effects, and the control effects can control the virtual content to display different effects. For example, referring to FIG. 5 , when the user rotates the dial on the dial of the smart watch, the terminal device can generate manipulation instructions according to the rotation parameters of the dial to control thevirtual content 400 to switch options. When operating different gestures, the terminal device can generate different manipulation instructions according to different gestures on the touch screen, so as to control the virtual content to display different effects. Of course, the above control effects are only examples, and the control effects on virtual content corresponding to specific manipulation instructions may not be limited in the embodiments of the present application.

在另一些实施例中，也可以是穿戴设备根据检测到的操控动作参数生成操控指令后，将该操控指令发送给终端设备，终端设备可根据接收到的操控指令对虚拟内容进行控制。In other embodiments, after the wearable device generates the manipulation command according to the detected manipulation action parameters, the manipulation command is sent to the terminal device, and the terminal device can control the virtual content according to the received manipulation command.

在本申请实施例中，上述相对空间位置关系的变化信息可包括穿戴设备相对终端设备的位置的变化信息和姿态的变化信息，可以由终端设备对穿戴设备进行识别和追踪而得到，例如，终端设备通过上述采集包含穿戴设备上的标记物的图像进行识别的方式，对穿戴设备进行识别和追踪。In the embodiment of the present application, the change information of the above-mentioned relative spatial position relationship may include change information of the position of the wearable device relative to the terminal device and change information of the posture, which may be obtained by identifying and tracking the wearable device by the terminal device. For example, the terminal The device identifies and tracks the wearable device through the above-mentioned method of collecting and identifying the image including the marker on the wearable device.

在一些实施例中，终端设备可以根据相对空间位置关系的变化信息，生成变化信息对应的操控指令，对显示的虚拟内容进行控制。具体地，终端设备可以根据穿戴设备的位置和/或姿态的具体变化，确定出穿戴设备的运动距离、运动方向等运动参数，可以并根据该运动参数确定出穿戴设备处于具体何种运动状态，例如翻转、移动等。从而根据运动状态与操控指令的对应关系，对显示的虚拟内容进行控制。例如，请参见图6，用户转动手腕时，终端设备可根据手腕上佩戴的穿戴设备的翻转状态，生成操控指令，控制虚拟内容400进行内容滚动，又例如，用户转动手腕至不同角度时，终端设备可根据穿戴设备的不同的翻转角度生成不同的操控指令，以控制虚拟内容显示不同的效果。In some embodiments, the terminal device may generate a manipulation instruction corresponding to the change information according to the change information of the relative spatial positional relationship, so as to control the displayed virtual content. Specifically, the terminal device can determine the movement parameters such as the movement distance and movement direction of the wearable device according to the specific changes in the position and/or posture of the wearable device, and can determine the specific movement state of the wearable device according to the movement parameters, Such as flipping, moving, etc. Thus, the displayed virtual content is controlled according to the corresponding relationship between the motion state and the manipulation instruction. For example, referring to FIG. 6 , when the user rotates the wrist, the terminal device can generate manipulation instructions according to the flip state of the wearable device on the wrist, and control thevirtual content 400 to scroll the content. For another example, when the user rotates the wrist to different angles, the terminal The device can generate different manipulation instructions according to different flip angles of the wearable device, so as to control the virtual content to display different effects.

在一些实施例中，终端设备还可以通过组合键的方式对显示的虚拟内容进行控制，也就是说，终端设备可以同时根据穿戴设备检测到的不同操控动作参数(如按键按压参数和转盘转动参数)，对显示的虚拟内容进行控制，也可以同时根据穿戴设备检测到的操控动作参数以及相对空间位置关系的变化信息，对显示的虚拟内容进行控制。例如，请参见图1和图7，用户在按住智能手表上的一个按键的同时旋转表盘上的转盘，可以实现对三维虚拟汽车模型401的放大。In some embodiments, the terminal device can also control the displayed virtual content by combining keys, that is, the terminal device can simultaneously control different action parameters (such as button pressing parameters and turntable rotation parameters) detected by the wearable device. ) to control the displayed virtual content, and also control the displayed virtual content according to the manipulation action parameters detected by the wearable device and the change information of the relative spatial positional relationship. For example, referring to FIG. 1 and FIG. 7 , the user can zoom in on the three-dimensionalvirtual car model 401 while holding down a button on the smart watch while rotating the dial on the dial.

本申请实施例提供的虚拟内容的控制方法，通过获取穿戴设备与终端设备之间的相对空间位置关系，以及获取待显示的虚拟内容的内容数据，并根据该内容数据和相对空间位置关系，生成虚拟内容，然后对虚拟内容进行显示，并根据穿戴设备检测到的操控动作参数以及相对空间位置关系的变化信息中的至少一种，对显示的虚拟内容进行控制，从而实现根据穿戴设备的空间位置，在虚拟空间中显示虚拟内容，并根据穿戴设备控制虚拟内容的显示，实现了穿戴设备与终端设备之间的交互，提高了用户与虚拟内容的交互性和交互便利性。The method for controlling virtual content provided by the embodiment of the present application, by acquiring the relative spatial positional relationship between the wearable device and the terminal device, and acquiring the content data of the virtual content to be displayed, and generating the content data and the relative spatial positional relationship according to the content data and the relative spatial positional relationship. virtual content, and then display the virtual content, and control the displayed virtual content according to at least one of the manipulation action parameters detected by the wearable device and the change information of the relative spatial position relationship, so as to realize the spatial position of the wearable device. , the virtual content is displayed in the virtual space, and the display of the virtual content is controlled according to the wearable device, the interaction between the wearable device and the terminal device is realized, and the interaction and interaction convenience between the user and the virtual content are improved.

请参阅图8，本申请另一实施例提供了一种虚拟内容的控制方法，可应用于终端设备，该虚拟内容的控制方法可以包括：Referring to FIG. 8, another embodiment of the present application provides a method for controlling virtual content, which can be applied to a terminal device. The method for controlling virtual content may include:

步骤S210：获取穿戴设备与终端设备之间的相对空间位置关系。Step S210: Obtain the relative spatial positional relationship between the wearable device and the terminal device.

在本申请实施例中，终端设备可以获取穿戴设备与终端设备之间的相对空间位置关系，以根据该相对空间位置关系，对虚拟内容进行显示。In this embodiment of the present application, the terminal device may acquire the relative spatial position relationship between the wearable device and the terminal device, so as to display virtual content according to the relative spatial position relationship.

在一些实施例中，终端设备还可以根据穿戴设备的惯性测量单元(Inertialmeasurement unit，IMU)的测量数据，获取到上述相对空间位置关系。具体地，请参见图9，上述获取穿戴设备与终端设备之间的相对空间位置关系，可以包括：In some embodiments, the terminal device may also acquire the above-mentioned relative spatial position relationship according to measurement data of an inertial measurement unit (Inertial measurement unit, IMU) of the wearable device. Specifically, referring to FIG. 9 , the above-mentioned acquisition of the relative spatial positional relationship between the wearable device and the terminal device may include:

步骤S211：获取包含穿戴设备的图像以及惯性测量传感器的感应数据。Step S211: Acquire the image including the wearable device and the sensing data of the inertial measurement sensor.

在一些实施例中，穿戴设备可以包括惯性测量传感器，惯性测量传感器包括惯性测量单元(Inertial measurement unit，IMU)。IMU可以检测穿戴设备的六自由度信息，也可以仅检测穿戴设备的三自由度信息。其中，三自由度信息可包括穿戴设备沿空间中三个直角坐标轴(X、Y、Z轴)的转动自由度，六自由度信息可包括穿戴设备沿空间中三个直角坐标轴的移动自由度和转动自由度，上述三个直角坐标轴对应的移动自由度可构成穿戴设备的位置信息，对应的转动自由度可构成穿戴设备的姿态信息。因此，终端设备可以通过接收穿戴设备发送的上述惯性测量传感器的感应数据，来得到IMU检测到的穿戴设备的姿态信息或者检测到的位置及姿态信息，进而获取到穿戴设备与终端设备之间的相对空间位置关系。In some embodiments, the wearable device may include an inertial measurement sensor including an inertial measurement unit (IMU). The IMU can detect the six-DOF information of the wearable device, or it can only detect the three-DOF information of the wearable device. The three-degree-of-freedom information may include the rotational degrees of freedom of the wearable device along the three rectangular coordinate axes (X, Y, and Z axes) in space, and the six-degree-of-freedom information may include the freedom of movement of the wearable device along the three rectangular coordinate axes in space. The movement degrees of freedom corresponding to the three rectangular coordinate axes can constitute the position information of the wearable device, and the corresponding rotation degrees of freedom can constitute the attitude information of the wearable device. Therefore, the terminal device can obtain the posture information of the wearable device detected by the IMU or the detected position and posture information by receiving the sensing data of the above-mentioned inertial measurement sensor sent by the wearable device, and then obtain the information between the wearable device and the terminal device. Relative spatial positional relationship.

在一些实施方式中，为精确地获取到穿戴设备的位置及姿态信息，终端设备可以获取包含穿戴设备的图像以及惯性测量传感器的感应数据，以根据图像的识别数据和IMU的检测数据，得到穿戴设备的位置及姿态信息。其中，终端设备获取包含穿戴设备的图像，可以是通过图像采集装置采集包含穿戴设备的图像，也可以是通过其他传感器装置采集包含穿戴设备的图像，例如图像传感器、光传感器等具有图像采集功能的传感器装置。In some embodiments, in order to accurately obtain the position and attitude information of the wearable device, the terminal device can obtain the image including the wearable device and the sensing data of the inertial measurement sensor, so as to obtain the wearable device according to the recognition data of the image and the detection data of the IMU. The location and attitude information of the device. Wherein, the terminal device acquires the image containing the wearable device, which may be acquired by the image acquisition device, or by other sensor devices, such as image sensors, light sensors, etc. with image acquisition functions. sensor device.

步骤S212：识别图像中的穿戴设备，得到穿戴设备的轮廓数据。Step S212: Identify the wearable device in the image, and obtain contour data of the wearable device.

在利用图像实现目标定位时，需要获取图像中目标对象的特征数据，以根据特征数据的位移变化，实现目标的实时追踪。因此，终端设备获取到上述包含穿戴设备的图像后，可以对该图像中的穿戴设备进行识别，得到穿戴设备的轮廓数据，即得到穿戴设备相对终端设备时所呈现的轮廓状态。可根据穿戴设备的轮廓数据获取穿戴设备相对终端设备的位置信息。When using an image to achieve target positioning, it is necessary to obtain the characteristic data of the target object in the image, so as to realize the real-time tracking of the target according to the displacement change of the characteristic data. Therefore, after acquiring the above-mentioned image including the wearable device, the terminal device can identify the wearable device in the image to obtain contour data of the wearable device, that is, obtain the contour state of the wearable device relative to the terminal device. The position information of the wearable device relative to the terminal device can be obtained according to the contour data of the wearable device.

在一些实施方式中，为实现对穿戴设备的主体的空间位置的实时追踪，上述轮廓数据可以是穿戴设备的主体的轮廓数据。例如，穿戴设备为智能手表时，上述轮廓数据可以为表盘的轮廓数据；穿戴设备为智能手环时，上述轮廓数据为手环本体的轮廓数据。In some embodiments, in order to realize real-time tracking of the spatial position of the body of the wearable device, the above-mentioned contour data may be contour data of the body of the wearable device. For example, when the wearable device is a smart watch, the above-mentioned contour data may be the contour data of the dial; when the wearable device is a smart wristband, the above-mentioned contour data is the contour data of the wristband body.

步骤S213：根据感应数据以及轮廓数据，获取穿戴设备与终端设备之间的相对空间位置关系。Step S213: Obtain the relative spatial positional relationship between the wearable device and the terminal device according to the sensing data and the contour data.

终端设备可根据穿戴设备的轮廓数据获取穿戴设备的位置信息，并根据IMU的感应数据得到穿戴设备的姿态信息，从而可以获取穿戴设备与终端设备之间的相对空间位置关系。The terminal device can obtain the position information of the wearable device according to the contour data of the wearable device, and obtain the posture information of the wearable device according to the sensing data of the IMU, so as to obtain the relative spatial position relationship between the wearable device and the terminal device.

在一些实施方式中，若由于上述感应数据可包括穿戴设备沿空间中三个直角坐标轴(X、Y、Z轴)的移动自由度和转动自由度，也可直接根据IMU的感应数据获取到穿戴设备与终端设备之间的相对空间位置关系。In some embodiments, if the above-mentioned sensing data may include the movement degrees of freedom and rotational degrees of freedom of the wearable device along the three Cartesian coordinate axes (X, Y, and Z axes) in space, they may also be obtained directly from the sensing data of the IMU. The relative spatial positional relationship between the wearable device and the terminal device.

在一些实施例中，穿戴设备上还可设置有光点，终端设备可以通过图像采集装置采集穿戴设备上的光点图像，以对光点图像中的光点进行识别，根据该光点图像确定穿戴设备与终端设备之间的相对空间位置关系，实现对穿戴设备的定位及追踪。穿戴设备上设置的光点可以是可见光点，也可以是红外光点等，当光点为红外光点时，终端设备上可设置有红外摄像头，用于采集该红外光点的光点图像。穿戴设备上设置的光点可以是一个，也可以是由多个光点组成的光点序列。In some embodiments, the wearable device can also be provided with light spots, and the terminal device can collect the light spot image on the wearable device through the image acquisition device, so as to identify the light spot in the light spot image, and determine the light spot according to the light spot image. The relative spatial position relationship between the wearable device and the terminal device realizes the positioning and tracking of the wearable device. The light spot set on the wearable device can be a visible light spot or an infrared light spot, etc. When the light spot is an infrared light spot, an infrared camera can be set on the terminal device to collect the light spot image of the infrared light spot. The light point set on the wearable device may be one light point or a light point sequence composed of multiple light points.

在一个实施例中，光点可设置在穿戴设备的外壳上，例如穿戴设备为智能手表时，光点可设置在表盘的周围。光点的设置方式可以是多种的，在此并不作限定。例如，为了可以实时获取穿戴设备的姿态信息，可以是穿戴设备的主体四周分别设置不同的光点，比如可以在智能手表的表盘四周设置不同数量的光点，或是不同颜色的光点等，从而使得终端设备根据光点图像中各个光点的分布，确定穿戴设备与终端设备之间的相对空间位置关系。In one embodiment, the light spots may be arranged on the casing of the wearable device, for example, when the wearable device is a smart watch, the light spots may be arranged around the dial. There may be various ways of setting the light spots, which are not limited here. For example, in order to obtain the posture information of the wearable device in real time, different light spots can be set around the main body of the wearable device. For example, different numbers of light spots or light spots of different colors can be set around the dial of the smart watch. Thus, the terminal device determines the relative spatial positional relationship between the wearable device and the terminal device according to the distribution of each light spot in the light spot image.

在一些实施方式中，终端设备也可以根据上述光点图像以及上述IMU的测量数据，精确地获取穿戴设备与终端设备之间的相对空间位置关系。In some embodiments, the terminal device can also accurately acquire the relative spatial positional relationship between the wearable device and the terminal device according to the above-mentioned light spot image and the above-mentioned measurement data of the IMU.

步骤S220：获取待显示的虚拟内容的内容数据。Step S220: Acquire content data of the virtual content to be displayed.

在本申请实施例中，终端设备在对虚拟内容进行显示时，终端设备需要获取待显示的虚拟内容的内容数据。In this embodiment of the present application, when a terminal device displays virtual content, the terminal device needs to acquire content data of the virtual content to be displayed.

在一些实施例中，上述待显示的虚拟内容的内容数据可以与穿戴设备对应，因此，请参见图10，上述获取待显示的虚拟内容的内容数据，可以包括：In some embodiments, the content data of the virtual content to be displayed may correspond to a wearable device. Therefore, referring to FIG. 10 , the obtaining of the content data of the virtual content to be displayed may include:

步骤S221：获取穿戴设备对应的身份标识。Step S221: Obtain the identity identifier corresponding to the wearable device.

其中，上述身份标识是一种能够唯一标识穿戴设备的标识物。具体的身份标识在此不做限定。例如，该身份标识可以是数字和/或英文构成的一串数据、也可以是二维码、条形码等具有一定形状的图案。在一些实施方式中，该身份标识可以存储于穿戴设备中，当穿戴设备与终端设备通信连接时，穿戴设备将该身份标识发送至终端设备，从而终端设备可以获取到穿戴设备对应的身份标识。Wherein, the above-mentioned identity identifier is an identifier that can uniquely identify the wearable device. The specific identification is not limited here. For example, the identity identifier may be a string of data composed of numbers and/or English, or may be a two-dimensional code, a barcode, or a pattern with a certain shape. In some embodiments, the identity can be stored in the wearable device, and when the wearable device is in communication with the terminal device, the wearable device sends the identity to the terminal device, so that the terminal device can obtain the identity corresponding to the wearable device.

在一些实施方式中，上述身份标识可以与穿戴设备上的标记物对应，因此终端设备可以通过识别穿戴设备上的标记物，来获取到穿戴设备对应的身份标识。具体地，终端设备对穿戴设备上的标记物进行识别后，可根据身份标识与穿戴设备上的标记物的对应关系，获取到与标记物对应的身份标识。在一些实施方式中，身份标识与穿戴设备上的标记物的对应关系可以存储于终端设备中，该对应关系可以是用户自行设置，也可以是终端设备出厂时默认的，还可以是终端设备从服务器获取的。In some embodiments, the above-mentioned identity identifier may correspond to a marker on the wearable device, so the terminal device may acquire the identity identifier corresponding to the wearable device by recognizing the marker on the wearable device. Specifically, after the terminal device identifies the marker on the wearable device, it can obtain the identity identifier corresponding to the marker according to the corresponding relationship between the identity identifier and the marker on the wearable device. In some embodiments, the corresponding relationship between the identity identifier and the marker on the wearable device can be stored in the terminal device, and the corresponding relationship can be set by the user, or the default when the terminal device leaves the factory, or the obtained by the server.

步骤S222：根据身份标识访问与身份标识对应的存储数据。Step S222: Access the stored data corresponding to the identity identifier according to the identity identifier.

在一些实施方式中，当存在少量的终端设备(例如头戴显示装置等AR设备)时，可以通过配置多个穿戴设备，使不同用户使用同一终端设备时，可以访问与自身相关的数据。具体地，终端设备可以根据上述身份标识，访问与身份标识对应的存储数据，其中，该身份标识可以理解为访问数据时需要的账号、密码等信息。这样，不同用户在使用同一终端设备时，可以通过各自的穿戴设备登录终端设备中，访问与自己相关的数据。In some embodiments, when there are a small number of terminal devices (such as AR devices such as head-mounted display devices), multiple wearable devices can be configured so that different users can access data related to themselves when using the same terminal device. Specifically, the terminal device can access the stored data corresponding to the identity identifier according to the above-mentioned identity identifier, where the identity identifier can be understood as account number, password and other information required for accessing the data. In this way, when different users use the same terminal device, they can log in to the terminal device through their respective wearable devices to access data related to themselves.

其中，上述存储数据可以是与穿戴设备有关的数据，如穿戴设备的电量信息、功能信息、规格参数等，也可以是与佩戴穿戴设备的用户有关的数据，如用户的行走步数、实时心率、来电数据、短信邮件信息等，也可以是用户自己存储的数据，比如文件、音频等数据。在一些实施方式中，该存储数据可以是终端设备中的存储数据，也可以是服务器中的存储数据。The above-mentioned stored data may be data related to the wearable device, such as battery information, function information, specification parameters, etc. of the wearable device, or data related to the user wearing the wearable device, such as the user's walking steps, real-time heart rate , call data, SMS and mail information, etc., or data stored by the user, such as files, audio and other data. In some embodiments, the stored data may be stored data in a terminal device, or may be stored data in a server.

在一些实施方式中，终端设备可以与云端服务器连接，终端设备可通过穿戴设备对应的身份标识登录云端服务器，访问云端服务器中对应存储空间的存储数据。In some embodiments, the terminal device can be connected to the cloud server, and the terminal device can log in to the cloud server through an identity corresponding to the wearable device, and access the stored data corresponding to the storage space in the cloud server.

步骤S223：从存储数据中获取待显示的虚拟内容的内容数据。Step S223: Acquire content data of the virtual content to be displayed from the stored data.

在一些实施方式中，终端设备需要显示虚拟内容时，可以从上述访问到的存储数据中，获取待显示的虚拟内容的内容数据，以将与自身相关的数据进行显示。其中，从存储数据中获取的待显示的虚拟内容的内容数据，可以是由用户进行选择的，也可以是终端设备默认获取的。In some embodiments, when the terminal device needs to display the virtual content, it can obtain the content data of the virtual content to be displayed from the stored data accessed above, so as to display the data related to itself. The content data of the virtual content to be displayed acquired from the stored data may be selected by the user, or may be acquired by the terminal device by default.

进一步的，终端设备可以根据用户的执行动作，判断是否需要进行上述内容数据的获取，因此，在上述获取待显示的虚拟内容的内容数据之前，该虚拟内容的控制方法还可以包括：Further, the terminal device can judge whether it is necessary to acquire the above-mentioned content data according to the execution action of the user. Therefore, before the above-mentioned acquisition of the content data of the virtual content to be displayed, the control method of the virtual content may further include:

根据相对空间位置关系，判断穿戴设备是否处于指定动作状态；当处于指定动作状态时，执行上述获取待显示的虚拟内容的内容数据的步骤。According to the relative spatial positional relationship, it is determined whether the wearable device is in the specified action state; when in the specified action state, the above step of acquiring the content data of the virtual content to be displayed is performed.

由于用户的手腕放下时，导致穿戴设备处于终端设备的图像采集装置的视野范围外，终端设备无法获取到穿戴设备的空间位置信息，从而无法根据穿戴设备的空间位置信息显示出虚拟内容。因此，当用户的手腕放下时，穿戴设备处于下降状态，终端设备可不对虚拟内容进行显示。When the user's wrist is put down, the wearable device is out of the field of view of the image acquisition device of the terminal device, and the terminal device cannot obtain the spatial location information of the wearable device, so that virtual content cannot be displayed according to the spatial location information of the wearable device. Therefore, when the user's wrist is put down, the wearable device is in a lowered state, and the terminal device may not display the virtual content.

在一些实施例中，终端设备可以仅在穿戴设备处于抬起状态时，对虚拟内容进行显示。具体地，终端设备可以根据上述相对空间位置关系，判断穿戴设备是否处于指定动作状态，当穿戴设备处于指定动作状态时，终端设备可以执行上述获取待显示的虚拟内容的内容数据的步骤，以进行后续的虚拟内容的显示，当穿戴设备不处于指定动作状态时，终端设备不会执行上述获取待显示的虚拟内容的内容数据的步骤，从而也不会进行虚拟内容的显示。当穿戴设备为处于指定动作状态时，执行上述获取待显示的虚拟内容的内容数据的步骤，从而实现在穿戴设备处于抬起状态时，终端设备才可以显示出虚拟内容，使用户可以通过保持手腕抬起，使穿戴设备处于抬起状态，从而控制虚拟内容的显示，提高了用户与虚拟内容之间的交互性。In some embodiments, the terminal device may display virtual content only when the wearable device is in a lifted state. Specifically, the terminal device can determine whether the wearable device is in the specified action state according to the above-mentioned relative spatial positional relationship, and when the wearable device is in the specified action state, the terminal device can perform the above-mentioned steps of acquiring the content data of the virtual content to be displayed to perform For subsequent display of virtual content, when the wearable device is not in the specified action state, the terminal device will not perform the above step of acquiring content data of the virtual content to be displayed, and thus will not display the virtual content. When the wearable device is in the designated action state, the above steps of acquiring the content data of the virtual content to be displayed are performed, so that the terminal device can display the virtual content only when the wearable device is in the lifted state, so that the user can display the virtual content by keeping the wrist Lifting makes the wearable device in a lifted state, so as to control the display of the virtual content and improve the interaction between the user and the virtual content.

在本申请实施例中，上述指定动作状态为终端设备获取上述内容数据时，穿戴设备需要保持的状态，可通过用户的指定动作实现。其中，该指定动作可以是抬腕动作，如用户将手腕从身侧抬至前方、将手腕从身侧抬至胸前等。可以理解的是，用户做出抬腕动作时，穿戴设备处于抬起状态，穿戴设备的主体部分处于终端设备的图像采集装置的视野范围内。In this embodiment of the present application, the above-mentioned designated action state is a state that the wearable device needs to maintain when the terminal device acquires the above-mentioned content data, which can be realized by a designated action of the user. Wherein, the specified action may be a wrist raising action, for example, the user lifts the wrist from the side to the front, lifts the wrist from the side to the chest, and the like. It can be understood that when the user lifts the wrist, the wearable device is in a lifted state, and the main body of the wearable device is within the field of view of the image acquisition device of the terminal device.

由于终端设备获取到的相对空间位置关系可包括穿戴设备相对终端设备的位置信息、旋转方向及旋转角度等，因此，在一些实施方式中，终端设备可以根据穿戴设备相对终端设备的位置信息、旋转方向及旋转角度，判断穿戴设备的主体部分是否在终端设备的图像采集装置的视野范围内，从而可以判断出穿戴设备是否处于抬起状态。可以理解的是，当穿戴设备的主体部分在终端设备的图像采集装置的视野范围内时，终端设备可以确定穿戴设备处于抬起状态，从而可以执行上述获取待显示的虚拟内容的内容数据的步骤。Since the relative spatial position relationship obtained by the terminal device may include the position information, rotation direction and rotation angle of the wearable device relative to the terminal device, etc., therefore, in some embodiments, the terminal device may The direction and rotation angle are used to determine whether the main part of the wearable device is within the field of view of the image acquisition device of the terminal device, so as to determine whether the wearable device is in a lifted state. It can be understood that, when the main body of the wearable device is within the field of view of the image acquisition device of the terminal device, the terminal device can determine that the wearable device is in a lifted state, so that the above steps of acquiring the content data of the virtual content to be displayed can be performed. .

在另一些实施例中，终端设备可以直接根据穿戴设备的传感器的感应数据，判断穿戴设备是否处于抬起状态。在一些实施方式中，该传感器可以是加速度传感器、重力传感器等，即可以是通过判断穿戴设备的加速度数值是否大于预设阈值，来判断穿戴设备是否处于抬起状态，也可以是通过判断穿戴设备的重力变化是否满足预设条件，来判断穿戴设备是否处于抬起状态。In other embodiments, the terminal device may directly determine whether the wearable device is in a lifted state according to the sensing data of the sensor of the wearable device. In some embodiments, the sensor may be an acceleration sensor, a gravity sensor, etc., that is, it may be determined whether the wearable device is in a lifted state by judging whether the acceleration value of the wearable device is greater than a preset threshold, or it may be determined by judging whether the wearable device is in a lifted state. Whether the change in the gravity of the wearable device meets the preset conditions to determine whether the wearable device is in a lifted state.

其中，上述预设阈值为一个大于9.8(重力加速度)的数值，如12、14等，该预设阈值为穿戴设备处于抬起状态，穿戴设备的加速度数值需要满足的最低数值。可以根据用户的具体使用情况进行设置，如可在初次使用穿戴设备时，采集并记录用户做出抬腕动作时的加速度信息，并根据该记录信息进行预设阈值的设定。可以理解的是，预设阈值设定得越大，用户触发此判断的抬手动作也需要越快。同理，上述预设条件为穿戴设备处于抬起状态时穿戴设备的重力变化范围，可以根据用户的具体使用情况进行设置。如可在初次使用穿戴设备时，采集并记录用户做出抬腕动作时的重力变化信息，并根据该记录信息进行预设条件的设定。The preset threshold is a value greater than 9.8 (gravitational acceleration), such as 12, 14, etc. The preset threshold is the minimum value that the wearable device needs to meet when the wearable device is in a lifted state. It can be set according to the specific usage of the user. For example, when the wearable device is used for the first time, the acceleration information when the user lifts the wrist can be collected and recorded, and the preset threshold can be set according to the recorded information. It can be understood that, the larger the preset threshold is set, the faster the user needs to raise his hand to trigger the judgment. Similarly, the above-mentioned preset condition is the gravity variation range of the wearable device when the wearable device is in a lifted state, which can be set according to the specific usage of the user. For example, when the wearable device is used for the first time, the gravity change information when the user lifts the wrist can be collected and recorded, and the preset conditions can be set according to the recorded information.

步骤S230：根据内容数据和相对空间位置关系，生成虚拟内容。Step S230: Generate virtual content according to the content data and the relative spatial positional relationship.

在一些实施例中，用户做出上述指定动作时，终端设备才可以获取到待显示的虚拟内容的内容数据，从而终端设备可以根据该内容数据构建出虚拟内容，根据穿戴设备与终端设备之间的相对空间位置关系，获取到虚拟内容的渲染位置，进而根据该渲染位置渲染出虚拟内容。In some embodiments, the terminal device can acquire the content data of the virtual content to be displayed only when the user performs the above specified action, so that the terminal device can construct the virtual content according to the content data, and according to the relationship between the wearable device and the terminal device The relative spatial position relationship of , obtains the rendering position of the virtual content, and then renders the virtual content according to the rendering position.

步骤S240：对虚拟内容进行显示。Step S240: Display the virtual content.

在一些实施例中，用户做出上述指定动作时，终端设备才可以获取到待显示的虚拟内容的内容数据，并根据该内容数据生成虚拟内容，从而将该虚拟内容显示于虚拟空间。实现用户抬腕时才可以看到虚拟内容显示在穿戴设备上，提高了虚拟内容的显示效果，提升了用户与虚拟内容之间的交互性。例如，请参见图1，用户抬起手腕至胸前时，用户佩戴的头戴显示装置可以实时扫描手腕上佩戴的穿戴设备200上的标记物201，可以看到虚拟汽车模型401叠加显示于现实空间的穿戴设备200上，体现了虚拟内容的增强现实的显示效果，体现了终端设备与穿戴设备之间的交互。In some embodiments, the terminal device can acquire the content data of the virtual content to be displayed, and generate virtual content according to the content data, so as to display the virtual content in the virtual space when the user performs the above-mentioned specified action. It is realized that the virtual content is displayed on the wearable device only when the user lifts the wrist, which improves the display effect of the virtual content and improves the interaction between the user and the virtual content. For example, referring to FIG. 1, when the user lifts the wrist to the chest, the head-mounted display device worn by the user can scan themarker 201 on thewearable device 200 worn on the wrist in real time, and thevirtual car model 401 can be seen superimposed and displayed on the real On thewearable device 200 in the space, the display effect of the augmented reality of the virtual content is embodied, and the interaction between the terminal device and the wearable device is embodied.

另外，在一些实施例中，终端设备可以与现实物体进行交互。具体地，终端设备可通过对现实空间中的实体对象进行识别，基于识别结果得到与实体对象对应的内容数据，然后根据该内容数据构建虚拟内容，并在虚拟空间中将与实体对象对应的虚拟内容进行显示。其中，实体对象可以是现实空间中的任一物理实体。例如，请参见图11，终端设备通过识别现实空间中智能台灯，在虚拟空间中显示出虚拟内容400，虚拟内容400为智能台灯的虚拟UI界面，用户通过头戴显示装置的显示镜片，可以看到与智能台灯对应的虚拟内容400叠加显示在真实世界中的智能手表上。Additionally, in some embodiments, the terminal device can interact with real objects. Specifically, the terminal device can identify the physical objects in the real space, obtain content data corresponding to the physical objects based on the recognition results, and then construct virtual content according to the content data, and convert the virtual content corresponding to the physical objects in the virtual space. content is displayed. The entity object can be any physical entity in the real space. For example, referring to FIG. 11 , the terminal device displaysvirtual content 400 in the virtual space by identifying the smart desk lamp in the real space. Thevirtual content 400 is the virtual UI interface of the smart desk lamp. Thevirtual content 400 corresponding to the smart desk lamp is superimposed and displayed on the smart watch in the real world.

在一些实施方式中，上述与实体对象对应的内容数据，可以是实体对象的UI数据，也可以是与实体对象相关的数据，如视频数据、图像数据等，在此不做限定。其中，内容数据可以是存储于终端设备中，也可以是从服务器等其他电子设备获取。当内容数据存储于服务器时，终端设备获取该内容数据，可以是在识别出实体对象的身份信息后，基于该身份信息从服务器中下载对应的内容数据。In some embodiments, the content data corresponding to the entity object may be UI data of the entity object, or data related to the entity object, such as video data, image data, etc., which are not limited herein. The content data may be stored in the terminal device or acquired from other electronic devices such as a server. When the content data is stored in the server, the terminal device may acquire the content data by downloading the corresponding content data from the server based on the identity information after identifying the identity information of the entity object.

在一些实施方式中，上述识别现实空间中的实体对象，可以是通过穿戴设备的摄像头采集包含有实体对象的图像，穿戴设备将该图像发送至终端设备，终端设备对该图像中的实体对象进行识别。也可以是终端设备直接采集包含有实体对象的图像，然后对该图像中的实体对象进行识别，在此不作限定。In some embodiments, the above-mentioned recognizing the entity object in the real space may be to collect an image containing the entity object through the camera of the wearable device, the wearable device sends the image to the terminal device, and the terminal device performs the operation on the entity object in the image. identify. It is also possible that the terminal device directly collects an image containing the entity object, and then identifies the entity object in the image, which is not limited herein.

进一步的，当现实空间中有多个实体对象时，终端设备需要选取一个实体对象进行交互。在一些实施方式中，终端设备可以利用穿戴设备进行实体对象的选取。Further, when there are multiple entity objects in the real space, the terminal device needs to select one entity object for interaction. In some embodiments, the terminal device can use the wearable device to select the entity object.

其中，终端设备利用穿戴设备进行实体对象的选取，可以是通过采集终端设备所处环境的场景图像，并根据穿戴设备相对终端设备的位置及姿态信息，获取场景图像中被选中的实体对象，从而终端设备可以识别被选中的实体对象，得到与被选中的实体对象对应的内容数据。在一些实施方式中，终端设备采集场景图像，可以是通过终端设备的图像采集装置进行采集，也可以是利用穿戴设备的摄像头进行采集后，从穿戴设备处获取到采集的场景图像。Among them, the terminal device uses the wearable device to select the entity object, which may be by collecting the scene image of the environment where the terminal device is located, and obtain the selected entity object in the scene image according to the position and attitude information of the wearable device relative to the terminal device. The terminal device can identify the selected entity object, and obtain content data corresponding to the selected entity object. In some embodiments, the terminal device collects the scene image, which may be collected by the image collection device of the terminal device, or the collected scene image may be obtained from the wearable device after the camera of the wearable device is used for collection.

由于终端设备采集到上述场景图像时，可以得到场景图像中的每个实体对象与终端设备之间的空间位置信息，因此，可以根据穿戴设备相对终端设备的位置及姿态信息，得到场景图像中的每个实体对象与穿戴设备之间的空间位置信息。由于穿戴设备所指向的方向相对穿戴设备固定，因此，可以根据穿戴设备的指向方向以及每个实体对象与穿戴设备之间的空间位置信息，确定场景图像中被选中的实体对象。Since the terminal device can obtain the spatial position information between each entity object in the scene image and the terminal device when the above-mentioned scene image is collected, the position and attitude information of the wearable device relative to the terminal device can be obtained. Spatial location information between each entity object and the wearable device. Since the pointing direction of the wearable device is fixed relative to the wearable device, the selected entity object in the scene image can be determined according to the pointing direction of the wearable device and the spatial position information between each entity object and the wearable device.

在一些实施方式中，穿戴设备上设置有光发射装置，用户可以通过移动穿戴设备改变光的指向，使光点可以显示在想选择的实体对象上。因此，终端设备可以通过判断场景图像中的实体对象上是否有光点，来确定该实体对象是否为被选中的实体对象。在一些实施方式中，光发射装置中的光可以是可见光，也可以是红外光，在此并不作限定。In some embodiments, the wearable device is provided with a light emitting device, and the user can change the direction of the light by moving the wearable device, so that the light point can be displayed on the desired physical object. Therefore, the terminal device can determine whether the entity object in the scene image is the selected entity object by judging whether the entity object in the scene image has light spots. In some embodiments, the light in the light emitting device may be visible light or infrared light, which is not limited herein.

当终端设备获取到场景图像中被选中的实体对象时，可以对该实体对象进行识别，得到与被选中的实体对象对应的内容数据，从而可以根据该内容数据生成虚拟内容。When the terminal device acquires the selected entity object in the scene image, it can identify the entity object and obtain content data corresponding to the selected entity object, so that virtual content can be generated according to the content data.

进一步的，上述虚拟内容可以显示在实体对象附近，以减少眼睛来回移动的次数。因此，在一些实施例中，可以通过获取虚拟内容与实体对象之间的第一空间位置关系，以及实体对象与终端设备之间的第二空间位置关系，然后根据内容数据、第一空间位置关系及第二空间位置关系，生成虚拟内容。从而实现将虚拟内容显示在实体对象附近。Further, the above-mentioned virtual content can be displayed near the physical object, so as to reduce the number of times the eyes move back and forth. Therefore, in some embodiments, the first spatial positional relationship between the virtual content and the physical object and the second spatial positional relationship between the physical object and the terminal device can be obtained, and then according to the content data, the first spatial positional relationship and the second spatial positional relationship to generate virtual content. In this way, the virtual content is displayed near the physical object.

可以理解的是，虚拟内容与实体对象之间的第一空间位置关系固定，即虚拟内容固定显示在实体对象附近。因此，终端设备可以获取虚拟内容与实体对象之间的第一空间位置关系，以及实体对象与终端设备之间的第二空间位置关系，以根据第一空间位置关系和第二空间位置关系，得到虚拟内容相对终端设备的空间位置，从而得到虚拟内容在虚拟空间中的渲染坐标。终端设备可以根据上述内容数据构建虚拟内容，根据该渲染坐标渲染出虚拟内容。It can be understood that the first spatial positional relationship between the virtual content and the physical object is fixed, that is, the virtual content is fixedly displayed near the physical object. Therefore, the terminal device can acquire the first spatial positional relationship between the virtual content and the physical object, and the second spatial positional relationship between the physical object and the terminal device, so as to obtain the The spatial position of the virtual content relative to the terminal device, thereby obtaining the rendering coordinates of the virtual content in the virtual space. The terminal device may construct virtual content according to the above-mentioned content data, and render the virtual content according to the rendering coordinates.

步骤S250：根据穿戴设备检测到的操控动作参数以及相对空间位置关系的变化信息中的至少一种，对显示的虚拟内容进行控制。Step S250: Control the displayed virtual content according to at least one of the manipulation action parameters detected by the wearable device and the change information of the relative spatial positional relationship.

在一些实施方式中，可以对显示的虚拟内容设定锁定功能，因此，上述根据相对空间位置关系的变化信息，对显示的虚拟内容进行控制，可以包括：In some embodiments, a locking function can be set for the displayed virtual content. Therefore, the above-mentioned control of the displayed virtual content according to the change information of the relative spatial position relationship may include:

获取相对空间位置关系的变化数值小于预设阈值的持续时长；当持续时长达到预设时长时，将虚拟内容固定于当前显示位置。The duration of the change in the relative spatial position relationship is obtained that is less than the preset threshold; when the duration reaches the preset duration, the virtual content is fixed at the current display position.

由于虚拟内容只有用户抬腕时才能进行显示，且长时间抬腕会造成的用户手臂酸痛，因此，当用户抬腕查看虚拟内容超过一定时间时，可以对显示的虚拟内容进行锁定，使得手腕被放下时，终端设备依然可以显示虚拟内容，且虚拟内容的显示位置固定。具体地，终端设备可以获取相对空间位置关系的变化数值小于预设阈值的持续时长，以根据持续时长确定用户抬腕查看虚拟内容时的抬腕时长，可通过判断该持续时长是否达到预设时长，来确定是否可以对虚拟内容进行锁定，并在判断出该持续时长达到预设时长时，终端设备可以将虚拟内容固定于当前显示位置，实现对虚拟内容的位置锁定。Since the virtual content can only be displayed when the user lifts the wrist, and the user’s arm is sore due to the long-time raising of the wrist, when the user lifts the wrist to view the virtual content for more than a certain period of time, the displayed virtual content can be locked, so that the wrist is blocked. When put down, the terminal device can still display the virtual content, and the display position of the virtual content is fixed. Specifically, the terminal device can obtain the duration of the change in the relative spatial position relationship that is smaller than the preset threshold, so as to determine the duration of the user's wrist raising when the user lifts his wrist to view the virtual content according to the duration, and can determine whether the duration reaches the preset duration by judging whether the duration reaches the preset duration. , to determine whether the virtual content can be locked, and when judging that the duration reaches a preset duration, the terminal device can fix the virtual content at the current display position to achieve position locking of the virtual content.

可以理解的是，当用户长时间保持抬腕动作时，穿戴设备与终端设备之间的相对空间位置关系也会保持稳定，并不会发生太大的变化，因此，在用户做出抬腕动作后，可以设定一个预设阈值，通过预设阈值判断用户是否有保持抬腕动作。其中，预设阈值为用户保持抬腕动作时，相对空间位置关系的最大变化数值。在用户做出抬腕动作后，如果相对空间位置关系的变化数值小于该预设阈值，则可以判断出用户保持着抬腕动作。如果相对空间位置关系的变化数值大于该预设阈值，则可以判断出用户做出了非抬腕动作。It is understandable that when the user keeps raising the wrist for a long time, the relative spatial positional relationship between the wearable device and the terminal device will also remain stable and will not change much. Afterwards, a preset threshold may be set, and whether the user keeps raising the wrist is determined by the preset threshold. The preset threshold is the maximum change value of the relative spatial positional relationship when the user maintains the wrist-lifting action. After the user lifts the wrist, if the change value of the relative spatial position relationship is less than the preset threshold, it can be determined that the user maintains the wrist lift. If the change value of the relative spatial position relationship is greater than the preset threshold, it can be determined that the user has made a non-lifting action.

进一步的，可以设定一个预设时长，以在用户保持抬腕动作到达预设时长时，可以对虚拟内容进行锁定。该预设时长为虚拟内容被锁定时，用户保持抬腕动作的最大时长。也可以理解为，在预设时长内，用户需要一直保持抬腕动作，终端设备才可以对虚拟内容进行锁定，即当相对空间位置关系的变化数值小于预设阈值的持续时长达到预设时长时，中设备可以将虚拟内容固定于当前显示位置。Further, a preset duration can be set, so that when the user keeps the wrist-lifting action for the preset duration, the virtual content can be locked. The preset duration is the maximum duration for which the user keeps the wrist-lifting action when the virtual content is locked. It can also be understood that within the preset time period, the user needs to keep raising the wrist all the time before the terminal device can lock the virtual content, that is, when the change value of the relative spatial positional relationship is smaller than the preset threshold value and the continuous time period reaches the preset time period. , the device can fix the virtual content to the current display position.

在一些实施方式中，终端设备将虚拟内容固定于当前显示位置，可以是通过获取当前虚拟内容的渲染坐标，并将该渲染坐标作为后续所有待显示的虚拟内容的渲染坐标，以根据该渲染坐标渲染后续的虚拟内容，实现将虚拟内容一直显示在当前显示位置。从而在用户放下手腕时，终端设备仍然可以对虚拟内容进行显示，且虚拟内容的显示位置固定。其中，该当前显示位置指的是虚拟内容在虚拟空间中的位置，该显示位置固定不变，可以是虚拟内容与虚拟摄像头(也可看作是人眼)的相对位置不变，也可以是虚拟内容相对于虚拟空间中的世界坐标原点不变。In some embodiments, the terminal device fixes the virtual content at the current display position, which may be by acquiring the rendering coordinates of the current virtual content, and using the rendering coordinates as the rendering coordinates of all subsequent virtual content to be displayed, so as to obtain the rendering coordinates according to the rendering coordinates. Render the subsequent virtual content, so that the virtual content is always displayed at the current display position. Therefore, when the user puts down the wrist, the terminal device can still display the virtual content, and the display position of the virtual content is fixed. Wherein, the current display position refers to the position of the virtual content in the virtual space, and the display position is fixed, which may be the relative position of the virtual content and the virtual camera (which can also be regarded as the human eye) unchanged, or it may be Virtual content is unchanged relative to the origin of world coordinates in virtual space.

在另一些实施方式中，上述根据相对空间位置关系的变化信息，对显示的虚拟内容进行控制，可以包括：In other embodiments, the above-mentioned control of the displayed virtual content according to the change information of the relative spatial positional relationship may include:

根据相对空间位置关系的变化信息，对显示的虚拟内容进行内容切换、移动、旋转、以及缩放比例调整中的至少一种。At least one of content switching, movement, rotation, and scaling adjustment is performed on the displayed virtual content according to the change information of the relative spatial positional relationship.

具体地，终端设备在根据穿戴设备与终端设备之间的相对空间位置关系，生成上述虚拟内容并将该虚拟内容进行显示之后，终端设备可以实时获取穿戴设备与终端设备之间的相对空间位置关系，以在穿戴设备与终端设备之间的相对空间位置关系发生变化时，根据相对空间位置关系的变化信息，控制显示的虚拟内容进行内容切换、移动、旋转、以及缩放比例调整中的至少一种。当然，也还可以对虚拟内容进行其他控制，如虚拟内容的拆分等，在此不作限定。其中，可以是通过转动手腕(例顺时针转动、逆时针转动)、挥动手腕(例如上下挥动、左右挥动、前后挥动)，改变穿戴设备与终端设备之间的相对空间位置关系。Specifically, after the terminal device generates and displays the virtual content according to the relative spatial positional relationship between the wearable device and the terminal device, the terminal device can acquire the relative spatial positional relationship between the wearable device and the terminal device in real time. , so that when the relative spatial positional relationship between the wearable device and the terminal device changes, according to the change information of the relative spatial positional relationship, the displayed virtual content is controlled to perform at least one of content switching, movement, rotation, and scaling adjustment . Of course, other controls can also be performed on the virtual content, such as splitting the virtual content, which is not limited here. The relative spatial positional relationship between the wearable device and the terminal device may be changed by rotating the wrist (eg, clockwise rotation, counterclockwise rotation) or waving the wrist (eg, up and down, left and right, front and rear).

作为一种实施方式，终端设备可以根据相对空间位置关系的变化信息，对显示的虚拟内容进行内容切换。在一些应用场景中，用户需要查看的虚拟内容比较多时，终端设备可能仅能显示其中的部分虚拟内容，因此，可以根据相对空间位置关系的变化信息，将未显示的部分虚拟内容逐渐显示，即对显示的虚拟内容进行内容切换。从而根据穿戴设备的位置及姿态信息的变化，实现虚拟内容的滚动显示。例如，请参见图6，用户通过转动手腕改变智能手表的位置及姿态信息时，用户通过头戴显示装置可以看到虚拟内容400在现实空间中的智能手表上滚动显示。当然，以上仅为举例，应用场景并不限定于此。As an implementation manner, the terminal device may perform content switching on the displayed virtual content according to the change information of the relative spatial positional relationship. In some application scenarios, when the user needs to view a lot of virtual content, the terminal device may only be able to display part of the virtual content. Therefore, the part of the virtual content that is not displayed can be gradually displayed according to the change information of the relative spatial position relationship, that is, Content switching is performed on the displayed virtual content. Therefore, according to the change of the position and posture information of the wearable device, the scrolling display of the virtual content is realized. For example, referring to FIG. 6 , when the user changes the position and posture information of the smart watch by rotating the wrist, the user can see thevirtual content 400 scrolled and displayed on the smart watch in the real space through the head-mounted display device. Of course, the above is just an example, and the application scenario is not limited to this.

作为另一种实施方式，终端设备可以根据相对空间位置关系的变化信息，对显示的虚拟内容进行内容移动。其中，内容移动可以是水平方向、垂直方向、或者自由方向等方向上的移动。在一些实施方式中，虚拟内容移动方向可以与穿戴设备移动的方向对应。例如，请参见图12，用户通过向上抬起手腕改变智能手表的位置及姿态信息时，用户通过头戴显示装置可以看到虚拟汽车模型401一直叠加显示于现实空间中的智能手表上，随着智能手表的移动方向而移动。As another implementation manner, the terminal device may perform content movement on the displayed virtual content according to the change information of the relative spatial positional relationship. The content movement may be movement in a horizontal direction, a vertical direction, or a free direction. In some implementations, the moving direction of the virtual content may correspond to the direction in which the wearable device moves. For example, referring to FIG. 12 , when the user changes the position and posture information of the smart watch by raising the wrist upward, the user can see through the head-mounted display device that thevirtual car model 401 has been superimposed and displayed on the smart watch in the real space. move according to the direction of movement of the smart watch.

作为又一种实施方式，终端设备可以根据相对空间位置关系的变化信息，对显示的虚拟内容进行内容旋转。在一些应用场景中，用户需要全方位地查看虚拟内容时，可以根据相对空间位置关系的变化信息，将虚拟内容的内容旋转，其中，内容旋转可以指在二维平面或者三维空间中将虚拟内容进行指定方向(例如，水平方向、垂直方向、或者自由方向等)上的转动，也就是将虚拟内容沿指定方向的旋转轴进行转动，使显示的虚拟内容的姿态(朝向方向等)发生变换。例如，请参见图1和图13，用户通过旋转手腕改变智能手表的位置及姿态信息时，用户通过头戴显示装置可以看到旋转后的虚拟汽车模型401叠加显示于现实空间中的智能手表上，以显示虚拟汽车模型401的不同视角。As another implementation manner, the terminal device may perform content rotation on the displayed virtual content according to the change information of the relative spatial positional relationship. In some application scenarios, when the user needs to view the virtual content in an all-round way, the content of the virtual content can be rotated according to the change information of the relative spatial position relationship. Rotate in a specified direction (for example, horizontal direction, vertical direction, or free direction, etc.), that is, rotate the virtual content along the rotation axis of the specified direction, so that the posture (orientation direction, etc.) of the displayed virtual content is transformed. For example, referring to FIG. 1 and FIG. 13 , when the user changes the position and posture information of the smart watch by rotating the wrist, the user can see the rotatedvirtual car model 401 displayed on the smart watch in the real space through the head-mounted display device. , to display different perspectives of thevirtual car model 401 .

进一步的，在一些实施方式中，虚拟内容旋转的方向，可以设定为指定方向，也可以与穿戴设备旋转的方向对应，即可以控制虚拟内容进行与穿戴设备的姿态对应方向上的转动。例如，请参见图14，用户通过旋转手腕改变智能手表的位置及姿态信息时，可以根据智能手表的旋转方向控制虚拟汽车模型401的旋转。Further, in some embodiments, the rotation direction of the virtual content may be set to a specified direction, or may correspond to the rotation direction of the wearable device, that is, the virtual content may be controlled to rotate in a direction corresponding to the posture of the wearable device. For example, referring to FIG. 14 , when the user changes the position and posture information of the smart watch by rotating the wrist, the rotation of thevirtual car model 401 can be controlled according to the rotation direction of the smart watch.

作为还一种实施方式，终端设备可以根据相对空间位置关系的变化信息，对显示的虚拟内容进行缩放比例调整。缩放比例调整可以指在将虚拟内容的模型进行放大比例或者缩小比例的调整，其中，放大比例以及缩小比例为显示的虚拟内容的大小相对于虚拟内容的原始大小的比例。在一些实施方式中，可以通过穿戴设备旋转的方向确定对虚拟内容的模型进行放大比例或者缩小比例的调整，例如，请参见图15，可以根据穿戴设备的顺时针旋转方向控制虚拟汽车模型401的放大，可以根据穿戴设备的逆时针旋转方向控制虚拟汽车模型401的缩小。另外，可以根据穿戴设备的旋转角度大小，确定对虚拟内容的模型进行放大或者缩小的比例，例如，穿戴设备的旋转角度越大，则放大或者缩小的比例越大。As another implementation manner, the terminal device may adjust the scaling ratio of the displayed virtual content according to the change information of the relative spatial positional relationship. The scaling ratio adjustment may refer to adjusting the enlargement ratio or the reduction ratio of the model of the virtual content, wherein the enlargement ratio and the reduction ratio are the ratios of the size of the displayed virtual content to the original size of the virtual content. In some embodiments, the scale of the virtual content model can be adjusted according to the rotation direction of the wearable device. For example, referring to FIG. 15 , the rotation direction of thevirtual car model 401 can be controlled according to the clockwise rotation direction of the wearable device. To zoom in, thevirtual car model 401 can be controlled to zoom out according to the counterclockwise rotation direction of the wearable device. In addition, the scale of the enlargement or reduction of the model of the virtual content may be determined according to the rotation angle of the wearable device. For example, the larger the rotation angle of the wearable device is, the larger the scale of enlargement or reduction is.

在一些实施例中，也可以根据穿戴设备上的操控区检测到的操控动作，对显示的虚拟内容进行控制，实现上述控制效果。具体地，上述穿戴设备检测到的操控动作参数，可以包括：In some embodiments, the displayed virtual content may also be controlled according to the manipulation action detected in the manipulation area on the wearable device, so as to achieve the above control effect. Specifically, the manipulation action parameters detected by the above-mentioned wearable device may include:

穿戴设备的按键检测到的按压参数、穿戴设备的触摸屏检测到的触摸参数、以及穿戴设备的转盘检测到的转动参数中的一种或多种。One or more of the pressing parameters detected by the keys of the wearable device, the touch parameters detected by the touch screen of the wearable device, and the rotation parameters detected by the turntable of the wearable device.

在一些实施方式中，穿戴设备可以包括至少一个物理按键，上述穿戴设备检测到的操控动作参数可以包括按键检测到的按压参数。其中，不同的按键可对应不同的控制效果，该控制效果可控制虚拟内容显示不同的效果。由于按键未被按下时，不会产生按压信号，每按一次按键，穿戴设备可检测到一个按压信号，因此，终端设备可根据按键检测到的按压参数，对显示的虚拟内容进行控制。其中，按压参数可以是按压信号，也可以是按压次数。在一些实施方式中，终端设备可以根据不同的按压次数，控制虚拟内容显示不同的效果。例如，按压一次，选中虚拟内容；按压两次，放大虚拟内容。In some embodiments, the wearable device may include at least one physical button, and the manipulation action parameters detected by the wearable device may include pressing parameters detected by the button. Wherein, different keys may correspond to different control effects, and the control effects may control the virtual content to display different effects. Since no pressing signal is generated when the button is not pressed, the wearable device can detect a pressing signal every time the button is pressed. Therefore, the terminal device can control the displayed virtual content according to the pressing parameters detected by the button. The pressing parameter may be a pressing signal or the number of pressings. In some embodiments, the terminal device may control the virtual content to display different effects according to different pressing times. For example, press once to select the virtual content; press twice to zoom in on the virtual content.

在一些实施方式中，穿戴设备可以包括触摸屏，上述穿戴设备检测到的操控动作参数可以包括触摸屏检测到的触摸参数。其中，触摸参数可包括用户在触摸屏上触控操作(如点击、滑动、长按等操作)。不同的触控操作可对应不同的控制效果，终端设备可根据触摸屏检测到的触摸参数，对显示的虚拟内容进行控制。例如，请参见图16，用户在智能手表的触控屏上左右划动时，可以旋转虚拟汽车模型401；又例如，在触控屏上两根手指按照相对方向划动时，可以放大虚拟内容；在触控屏上点击触控按键时，可以对虚拟内容进行选择、切换等。In some embodiments, the wearable device may include a touch screen, and the manipulation action parameters detected by the wearable device may include touch parameters detected by the touch screen. The touch parameters may include touch operations (such as click, slide, long press, etc.) performed by the user on the touch screen. Different touch operations can correspond to different control effects, and the terminal device can control the displayed virtual content according to the touch parameters detected by the touch screen. For example, referring to FIG. 16 , when the user swipes left and right on the touch screen of the smart watch, thevirtual car model 401 can be rotated; for another example, when two fingers swipe in opposite directions on the touch screen, the virtual content can be enlarged ; When the touch button is clicked on the touch screen, the virtual content can be selected and switched.

在一些实施方式中，穿戴设备的同一个按键对应多个功能时，可以通过点击触控屏上的触控按键，来切换按键当前的功能，即切换按键当前对应的控制效果。例如，在触控屏上单击触控按键时，按键的功能为切换虚拟选项，即按压穿戴设备的按键，可以切换选中的虚拟选项，在触控屏上双击触控按键时，按键的功能为切换虚拟选项列表，按压穿戴设备的按键，切换为新的虚拟选项列表。作为一种实施方式，也可以是每单击一次触控屏上的触控按键，就切换按键的一个功能，实现通过触控屏上的触控按键，切换按键的多种功能。In some embodiments, when the same button of the wearable device corresponds to multiple functions, the current function of the button can be switched by clicking the touch button on the touch screen, that is, the control effect currently corresponding to the button can be switched. For example, when a touch button is clicked on the touch screen, the function of the button is to switch virtual options, that is, pressing the button of the wearable device can switch the selected virtual option. When the touch button is double-clicked on the touch screen, the function of the button To switch the virtual option list, press the button of the wearable device to switch to the new virtual option list. As an implementation manner, each time a touch button on the touch screen is clicked, one function of the button is switched, so that multiple functions of the button can be switched through the touch button on the touch screen.

在一些实施方式中，穿戴设备可以包括转盘，上述穿戴设备检测到的操控动作参数可包括转盘检测到的转动参数。其中，转动参数可以是转动角度，也可是转动方向。因此，终端设备可根据转盘检测到的转动参数，对显示的虚拟内容进行控制。例如，通过转动转盘来调节虚拟内容的旋转，通过转动转盘来切换选中的虚拟选项，通过转动转盘来调整虚拟内容的缩放比例。In some embodiments, the wearable device may include a turntable, and the manipulation action parameters detected by the wearable device may include rotation parameters detected by the turntable. The rotation parameter may be a rotation angle or a rotation direction. Therefore, the terminal device can control the displayed virtual content according to the rotation parameters detected by the turntable. For example, the rotation of the virtual content can be adjusted by rotating the dial, the selected virtual option can be switched by rotating the dial, and the zoom ratio of the virtual content can be adjusted by rotating the dial.

同理，也可以通过点击触控屏上的触控按键，来切换转盘当前的功能，即切换转盘当前对应的控制效果。例如，在触控屏上单击触控按键时，转盘可用于旋转虚拟内容，双击触控按键时，转盘可用于切换显示的虚拟内容。Similarly, it is also possible to switch the current function of the turntable by clicking the touch button on the touch screen, that is, to switch the control effect currently corresponding to the turntable. For example, when a touch key is clicked on the touch screen, the turntable can be used to rotate the virtual content, and when the touch key is double-clicked, the turntable can be used to switch the displayed virtual content.

当穿戴设备检测到的操控动作参数包括转盘检测到的转动参数时，作为一种实施方式，上述根据穿戴设备检测到的操控动作参数，对显示的虚拟内容进行控制，可以包括：When the manipulation action parameters detected by the wearable device include the rotation parameters detected by the turntable, as an implementation manner, the above-mentioned control of the displayed virtual content according to the manipulation action parameters detected by the wearable device may include:

根据转动参数与操控指令的对应关系，生成转动参数对应的操控指令；根据操控指令，对显示的虚拟内容进行控制。According to the corresponding relationship between the rotation parameters and the operation instructions, the operation instructions corresponding to the rotation parameters are generated; according to the operation instructions, the displayed virtual content is controlled.

在一些实施方式中，穿戴设备的转盘检测到转动参数时，穿戴设备可将该转动参数发送至终端设备，终端设备接收到该转动参数后，根据转动参数与操控指令的对应关系，生成与该转动参数对应的操控指令，并根据该操控指令，对显示的虚拟内容进行控制。其中，不同的操控指令对应不同的控制效果，该控制效果可控制虚拟内容实现上述显示效果。In some embodiments, when the turntable of the wearable device detects a rotation parameter, the wearable device can send the rotation parameter to the terminal device, and after receiving the rotation parameter, the terminal device generates a corresponding relationship between the rotation parameter and the manipulation instruction according to the corresponding relationship between the rotation parameter and the control command. Rotate the manipulation instruction corresponding to the parameter, and control the displayed virtual content according to the manipulation instruction. Wherein, different manipulation instructions correspond to different control effects, and the control effects can control the virtual content to achieve the above-mentioned display effects.

在一些实施方式中，上述转动参数与操控指令的对应关系，可以是不同的转动角度对应不同的操控指令，如转盘转动不同角度时，终端设备可控制虚拟内容的旋转不同角度。也可以是不同的转动方向对应不同的操控指令，例如，顺时针转动转盘时，放大虚拟内容，逆时针转动转盘时，缩小虚拟内容。还可以是转动角度和转动方向的不同结合对应不同的操控指令，例如，顺时针转动转盘的角度逐渐增大时，虚拟内容的放大比例也逐渐增大。当前，上述转动参数与操控指令的对应关系，还可以是其他，如不同的转动圈数对应不同的操控指令，在此并不作限定。In some embodiments, the corresponding relationship between the above rotation parameters and manipulation commands may be that different rotation angles correspond to different manipulation commands. For example, when the turntable rotates at different angles, the terminal device can control the rotation of the virtual content at different angles. Different rotation directions may also correspond to different manipulation instructions. For example, when the turntable is rotated clockwise, the virtual content is enlarged, and when the turntable is rotated counterclockwise, the virtual content is reduced. Different combinations of rotation angles and rotation directions may also correspond to different manipulation instructions. For example, when the angle of rotating the turntable clockwise gradually increases, the magnification ratio of the virtual content also gradually increases. At present, the corresponding relationship between the above-mentioned rotation parameters and the manipulation command may be other, for example, different numbers of rotations correspond to different manipulation commands, which are not limited here.

在一些实施例中，穿戴设备检测到的触摸屏的触摸参数可以包括触摸轨迹，终端设备可以根据用户的触摸轨迹实现虚拟内容的文字输入。具体地，上述根据穿戴设备检测到的操控动作参数，对显示的虚拟内容进行控制，可以包括：In some embodiments, the touch parameters of the touch screen detected by the wearable device may include a touch track, and the terminal device may implement text input of virtual content according to the user's touch track. Specifically, the above-mentioned control of the displayed virtual content according to the manipulation action parameters detected by the wearable device may include:

根据触摸轨迹，生成与触摸轨迹匹配的至少一个虚拟字符；将至少一个虚拟字符叠加显示于虚拟内容；当任意一个虚拟字符处于选中状态且接收到对处于选中状态的虚拟字符的确认指令时，将处于选中状态的虚拟字符添加至虚拟内容中。According to the touch track, at least one virtual character matching the touch track is generated; the at least one virtual character is superimposed and displayed on the virtual content; when any virtual character is in the selected state and a confirmation instruction for the virtual character in the selected state is received, the The selected virtual character is added to the virtual content.

其中，上述触摸轨迹可以是用户在触摸屏上进行手写操作时，触摸屏检测到的手指滑动轨迹。上述虚拟字符可以是终端设备根据现有的字符数据库中的字符数据渲染生成的，该虚拟字符的字符数据可以存储于终端设备，也可从服务器下载。其中，虚拟字符可包括虚拟文字符号(如中文符号、英文符号)、虚拟标点符号、虚拟运算符号、或其他虚拟特殊符号等，例如！、@、#、？、$、％、&等符号，在此不作限定。The above-mentioned touch track may be a finger sliding track detected by the touch screen when the user performs a handwriting operation on the touch screen. The above-mentioned virtual characters may be rendered and generated by the terminal device according to the character data in the existing character database, and the character data of the virtual characters may be stored in the terminal device or downloaded from the server. Wherein, the virtual characters may include virtual character symbols (such as Chinese symbols, English symbols), virtual punctuation symbols, virtual operation symbols, or other virtual special symbols, etc., for example! , @, #, ? , $, %, & and other symbols are not limited here.

在一些实施方式中，终端设备可根据穿戴设备的触控屏检测到的触摸参数，得到用户的触摸轨迹，通过识别该触摸轨迹，从数据库中查找至少一个与该触摸轨迹具有相似轨迹的字符，以根据至少一个字符的数据，渲染生成对应的至少一个虚拟字符，并将该至少一个虚拟字符叠加显示于上述显示的虚拟内容。例如，在触控屏上手写“名”字，经识别后，在上述显示的虚拟内容上依序显示“名”、“各”、“吝”等相似的候选虚拟字符供用户选取。又例如，请参见图20，在触控屏上手写“大”字，用户可以通过佩戴的头戴显示装置，可以看到多个候选字符413叠加显示于虚拟编辑页面412上，体现了虚拟内容的显示效果。In some embodiments, the terminal device can obtain the user's touch trajectory according to the touch parameters detected by the touch screen of the wearable device, and by identifying the touch trajectory, find at least one character with a similar trajectory to the touch trajectory from the database, According to the data of at least one character, at least one corresponding virtual character is generated by rendering, and the at least one virtual character is superimposed and displayed on the above-mentioned displayed virtual content. For example, the word "name" is handwritten on the touch screen, and after recognition, similar candidate virtual characters such as "name", "each" and "miserable" are displayed in sequence on the displayed virtual content for the user to select. For another example, referring to FIG. 20 , the word "big" is written by hand on the touch screen, and the user can see a plurality ofcandidate characters 413 superimposed and displayed on thevirtual editing page 412 through the wearing head-mounted display device, reflecting the virtual content display effect.

其中，上述将至少一个虚拟字符叠加显示于虚拟内容，作为一种实施方式，可以是将虚拟内容的颜色调暗或者将虚拟内容的透明度调高，使用户通过佩戴的头戴显示设备，无法观察到该虚拟内容，减少虚拟内容对虚拟字符的干扰。作为另一种实施方式，可以是仅对虚拟内容中与虚拟字符的重叠区域的颜色调暗或者透明度调高，使用户通过佩戴的头戴显示设备，无法观察到该虚拟内容中与虚拟字符的重叠区域的内容，减少虚拟内容对虚拟字符的干扰。Wherein, the above-mentioned superimposing and displaying at least one virtual character on the virtual content, as an implementation manner, may be to darken the color of the virtual content or increase the transparency of the virtual content, so that the user cannot observe the virtual content through the wearing head-mounted display device. To the virtual content, the interference of the virtual content to the virtual characters is reduced. As another implementation, only the color of the overlapping area with the virtual characters in the virtual content may be dimmed or the transparency may be increased, so that the user cannot observe the overlap between the virtual content and the virtual characters through the wearing head-mounted display device. The content of the overlapping area reduces the interference of virtual content to virtual characters.

进一步的，终端设备将至少一个虚拟字符叠加显示于虚拟内容后，可以在虚拟内容中插入被选中的虚拟字符，实现虚拟内容的内容增添。具体地，当任意一个虚拟字符处于选中状态且接收到对处于选中状态的虚拟字符的确认指令时，终端设备可以将处于选中状态的虚拟字符添加至虚拟内容中。Further, after the terminal device superimposes and displays at least one virtual character on the virtual content, the selected virtual character can be inserted into the virtual content to implement content addition of the virtual content. Specifically, when any virtual character is in the selected state and a confirmation instruction for the virtual character in the selected state is received, the terminal device may add the virtual character in the selected state to the virtual content.

其中，虚拟字符处于选中状态可以是虚拟字符处于预设显示范围内，也可以是获取到对虚拟字符的选取操作，其中，对虚拟字符的选取操作可以是上述利用穿戴设备的虚拟指引方式进行选取，虚拟字符处于预设显示范围内可以是通过穿戴设备的操控动作参数，使虚拟字符的显示位置改变，从而使虚拟字符处于预设显示范围内。例如，转盘对应的虚拟字符为26个英文字母时，通过旋转转盘选取英文字母，按键对应的虚拟字符为文字候选词时，通过按下按键进行选中文字的切换。在一些实施方式中，预设显示范围可以是上述虚拟指引指向的目标区域，也可以是用户自行设定。当然，具体的预设显示范围在本申请实施例中可以不作为限定。Wherein, the virtual character being in the selected state may be that the virtual character is in the preset display range, or the selection operation of the virtual character may be obtained, wherein the selection operation of the virtual character may be the above-mentioned selection using the virtual guidance method of the wearable device. , the virtual character is within the preset display range may be through the manipulation action parameters of the wearable device to change the display position of the virtual character, so that the virtual character is within the preset display range. For example, when the virtual characters corresponding to the turntable are 26 English letters, the English letters are selected by rotating the turntable, and when the virtual characters corresponding to the keys are text candidates, the selected characters can be switched by pressing the keys. In some embodiments, the preset display range may be the target area pointed to by the virtual guide, or may be set by the user. Of course, the specific preset display range may not be limited in the embodiments of the present application.

终端设备获取到处于选中状态的虚拟字符后，可以进行虚拟字符的确认指令的接收，并在接收到对处于选中状态的虚拟字符的确认指令时，终端设备可以将处于选中状态的虚拟字符添加至虚拟内容中。例如，请参见图21A，用户在触控屏上写“大”字，且用户从多个候选字符413中选中的选中词414(“大”)，当用户确定该候选词时，又请参见图21B，用户通过佩戴的头戴显示设备，可以看到虚拟编辑页面412中的增加内容415(“大”)，实现了虚拟内容的内容增添。After the terminal device obtains the virtual character in the selected state, it can receive the confirmation instruction of the virtual character, and when receiving the confirmation instruction for the virtual character in the selected state, the terminal device can add the virtual character in the selected state to the in virtual content. For example, referring to FIG. 21A , the user writes the word "big" on the touch screen, and the user selects a selected word 414 ("big") from a plurality ofcandidate characters 413, when the user determines the candidate word, see also In FIG. 21B , the user can see the added content 415 (“large”) in thevirtual editing page 412 through the wearing head-mounted display device, which realizes the content addition of the virtual content.

作为一种实施方式，终端设备可以将处于选中状态的虚拟字符添加至虚拟内容中，可以根据处于选中状态的虚拟字符以及显示的虚拟内容，生成新虚拟内容，从而终端设备可以将新虚拟内容进行显示，其中，新虚拟内容包括显示的虚拟内容以及处于选中状态的虚拟字符，从而实现了虚拟内容的内容增添。As an implementation manner, the terminal device can add the virtual character in the selected state to the virtual content, and can generate new virtual content according to the virtual character in the selected state and the displayed virtual content, so that the terminal device can add the new virtual content to the virtual content. display, wherein the new virtual content includes the displayed virtual content and the virtual characters in the selected state, so as to realize the content addition of the virtual content.

在一些实施方式中，上述确认指令可以是穿戴设备根据检测到的操控动作参数进行生成的，例如，穿戴设备检测到双击触摸屏或按下按键时，生成确认指令。也可以是穿戴设备将检测到的操控动作参数发送至终端设备，终端设备根据该操控动作参数，生成确认指令，还可以是终端设备检测到虚拟字符停留于预设显示范围内的时长达到指定时长时，生成确认指令，在此不作限定。In some embodiments, the above-mentioned confirmation instruction may be generated by the wearable device according to the detected manipulation action parameters. For example, when the wearable device detects that the touch screen is double-clicked or a button is pressed, the confirmation instruction is generated. It can also be that the wearable device sends the detected manipulation action parameters to the terminal device, and the terminal device generates a confirmation instruction according to the manipulation action parameters, or the terminal device detects that the virtual character stays within the preset display range for a specified duration. , a confirmation command is generated, which is not limited here.

进一步的，终端设备还可以在虚拟内容中选取被选中的虚拟字符的插入位置。在一些实施方式中，该插入位置可以是穿戴设备的虚拟指引所指向的位置，同时为便于用户观察到该插入位置，在虚拟内容上可显示虚拟光标。终端设备可通过改变穿戴设备的位置及姿态信息，改变虚拟内容中虚拟光标的显示位置，从而改变被选中的虚拟字符的插入位置，实现虚拟内容的内容增添位置的选取。Further, the terminal device may also select the insertion position of the selected virtual character in the virtual content. In some embodiments, the insertion position may be the position pointed by the virtual guide of the wearable device, and at the same time, in order to facilitate the user to observe the insertion position, a virtual cursor may be displayed on the virtual content. The terminal device can change the display position of the virtual cursor in the virtual content by changing the position and posture information of the wearable device, thereby changing the insertion position of the selected virtual character, and realizing the selection of the content addition position of the virtual content.

可以理解的是，上述对显示的虚拟内容进行控制可以是上述实施方式中的一种或多种的组合，在此不作限定。例如，按住一个按键的同时旋转转盘，可以实现虚拟内容的缩放，又例如，按住一个按键的同时旋转手腕，可以实现将选中的虚拟选项在虚拟现象列表里上下移动。It can be understood that, the above-mentioned control of the displayed virtual content may be a combination of one or more of the above-mentioned embodiments, which is not limited herein. For example, rotating the turntable while holding down a button can realize the zooming of virtual content, and for example, rotating the wrist while holding down a button can realize moving the selected virtual option up and down in the virtual phenomenon list.

另外，在一些实施例中，终端设备对上述虚拟内容进行显示后，可以利用穿戴设备对虚拟内容中的不同虚拟选项进行选择。In addition, in some embodiments, after the terminal device displays the above-mentioned virtual content, the wearable device may be used to select different virtual options in the virtual content.

具体地，可以通过实时获取穿戴设备相对终端设备的位置及姿态信息，并根据上述虚拟内容的显示位置，确定虚拟内容中被选中的目标区域，进而实现根据穿戴设备的位置及姿态信息，对虚拟内容的选择。该目标区域为穿戴设备与虚拟内容相对时，虚拟内容中被穿戴设备所指向的区域，可与穿戴设备的位置及姿态信息对应。该目标区域可以根据用户的意愿进行确定，也就是说，用户可以通过改变穿戴设备的位置及姿态信息，来确定虚拟内容中被选中的目标区域。在一些实施方式中，终端设备获取穿戴设备相对终端设备的位置及姿态信息，可以参照上述获取穿戴设备与终端设备之间的相对空间位置关系的方式，在此不再赘述。Specifically, the position and posture information of the wearable device relative to the terminal device can be acquired in real time, and the selected target area in the virtual content can be determined according to the display position of the above-mentioned virtual content, so as to realize the position and posture information of the wearable device. choice of content. The target area is an area pointed by the wearable device in the virtual content when the wearable device is opposite to the virtual content, which may correspond to the position and posture information of the wearable device. The target area can be determined according to the user's will, that is, the user can determine the selected target area in the virtual content by changing the position and posture information of the wearable device. In some embodiments, for the terminal device to obtain the position and attitude information of the wearable device relative to the terminal device, reference may be made to the above-mentioned method for obtaining the relative spatial positional relationship between the wearable device and the terminal device, which will not be repeated here.

可以理解的是，穿戴设备指向的方向与穿戴设备所在的平面之间的角度固定，当终端设备获取到穿戴设备相对终端设备的位置及姿态信息时，可得到穿戴设备的空间位置信息，因此，可以根据指向的方向与穿戴设备的角度固定关系以及穿戴设备的空间位置信息，得到穿戴设备当前所指向的方向，从而可以根据虚拟内容的显示位置以及穿戴设备当前所指向的方向，得到虚拟内容中被选中的目标区域。It can be understood that the angle between the direction in which the wearable device points and the plane where the wearable device is located is fixed. When the terminal device obtains the position and attitude information of the wearable device relative to the terminal device, the spatial position information of the wearable device can be obtained. Therefore, According to the fixed relationship between the pointing direction and the angle of the wearable device and the spatial position information of the wearable device, the current direction of the wearable device can be obtained. The selected target area.

在一些实施方式中，终端设备可以通过穿戴设备的虚拟指引获取虚拟内容中被选中的目标区域。具体地，终端设备可以根据虚拟指引与穿戴设备之间的相对位置关系和穿戴设备相对终端设备的位置及姿态信息，生成虚拟指引并进行显示，该显示的虚拟指引可用于指示穿戴设备指向的方向。然后根据显示的虚拟指引和虚拟内容的显示位置，确定虚拟内容中与虚拟指引相交的相交区域，并将相交区域作为虚拟内容中被选中的目标区域。In some embodiments, the terminal device may acquire the selected target area in the virtual content through the virtual guidance of the wearable device. Specifically, the terminal device can generate and display the virtual guide according to the relative positional relationship between the virtual guide and the wearable device and the position and posture information of the wearable device relative to the terminal device, and the displayed virtual guide can be used to indicate the direction the wearable device is pointing. . Then, according to the displayed virtual guide and the display position of the virtual content, an intersecting area in the virtual content that intersects with the virtual guide is determined, and the intersecting area is used as a selected target area in the virtual content.

其中，上述虚拟指引可以是虚拟射线，也可以是虚拟曲线，在此不做限定，用于指示方向即可。其中，虚拟指引与穿戴设备之间的相对位置关系固定，可根据用户的操作习惯进行调整。在一些实施方式中，虚拟指引的指向方向可以与穿戴设备的平面平行，也可是一定角度，如斜上方45°，在此不作限定。Wherein, the above-mentioned virtual guide may be a virtual ray or a virtual curve, which is not limited here, and may be used to indicate a direction. The relative positional relationship between the virtual guide and the wearable device is fixed and can be adjusted according to the user's operating habits. In some embodiments, the pointing direction of the virtual guide may be parallel to the plane of the wearable device, or may be at a certain angle, such as 45° obliquely upward, which is not limited herein.

由于终端设备根据穿戴设备相对终端设备的位置及姿态信息，可得到穿戴设备在现实空间的空间位置坐标，将该空间位置坐标转换为虚拟空间中的空间坐标，并根据该空间坐标渲染虚拟指引。终端设备渲染出虚拟指引后，可对虚拟指引进行显示。用户通过头戴显示装置的显示镜片，可以看到虚拟指引叠加显示在真实世界中的穿戴设备上，实现增强现实的效果。Because the terminal device can obtain the spatial position coordinates of the wearable device in the real space according to the position and attitude information of the wearable device relative to the terminal device, convert the spatial position coordinates into the spatial coordinates in the virtual space, and render the virtual guide according to the spatial coordinates. After the terminal device renders the virtual guide, the virtual guide can be displayed. Through the display lens of the head-mounted display device, the user can see the virtual guide superimposed and displayed on the wearable device in the real world to achieve the effect of augmented reality.

终端设备可以根据虚拟内容的显示位置以及显示的虚拟指引，获取虚拟内容中与虚拟指引相交的相交区域，并将相交区域作为虚拟内容中被选中的目标区域。在一些实施方式中，上述获取虚拟内容中与虚拟指引相交的相交区域，可以是获取虚拟内容中与虚拟指引的坐标相同的坐标点区域，该坐标点区域可直接作为上述相交区域，也可以是根据坐标点区域获取到对应的虚拟内容所在的区域，将该虚拟内容所在的区域作为相交区域，即作为虚拟内容中被选中的目标区域。其中，根据坐标点区域获取到对应的虚拟内容所在的区域，可以是根据坐标点区域获取距离最近的虚拟内容所在的区域。这样，用户可以通过调整穿戴设备的位置及姿态信息，来改变虚拟指引的显示位置，使显示的虚拟指引可以与想选择的虚拟内容相交，从而可以在虚拟内容中准确选取到虚拟内容。例如，请参见图17，用户通过头戴显示设备可以看到，智能手表的斜上方发射了一条虚拟射线420，用户可以通过旋转手腕，改变虚拟射线的指向方向，从而可以将虚拟射线所指向的目标区域410作为虚拟内容中被选中的目标区域。The terminal device may acquire an intersection area in the virtual content that intersects with the virtual guide according to the display position of the virtual content and the displayed virtual guide, and use the intersection area as a selected target area in the virtual content. In some embodiments, the above-mentioned acquisition of the intersection area in the virtual content that intersects with the virtual guide may be to obtain the coordinate point area in the virtual content that is the same as the coordinates of the virtual guide, and the coordinate point area may be directly used as the above-mentioned intersection area, or may be The area where the corresponding virtual content is located is obtained according to the coordinate point area, and the area where the virtual content is located is taken as the intersection area, that is, as the selected target area in the virtual content. Wherein, the area where the corresponding virtual content is obtained according to the coordinate point area may be the area where the closest virtual content is obtained according to the coordinate point area. In this way, the user can change the display position of the virtual guide by adjusting the position and posture information of the wearable device, so that the displayed virtual guide can intersect with the desired virtual content, so that the virtual content can be accurately selected from the virtual content. For example, referring to Fig. 17, the user can see through the head-mounted display device that avirtual ray 420 is emitted diagonally above the smart watch. The user can rotate the wrist to change the pointing direction of the virtual ray, so that the Thetarget area 410 serves as a selected target area in the virtual content.

在游戏场景中，为了增强游戏体验感，可以在穿戴设备上显示虚拟瞄准器，整个穿戴设备可作为发射器以进行射击操作。在一些实施方式中，上述虚拟指引的指向方向可以与虚拟瞄准器的准心对齐，这样，当需要对虚拟内容中的某个虚拟内容进行选择时，用户就可以将虚拟瞄准器的准心对准该虚拟内容。例如，请参见图18，终端设备在智能手表上显示虚拟瞄准器600，虚拟瞄准器600的准心与虚拟射线420的指向方向对齐，用户通过头戴显示设备可以看，虚拟瞄准器600对准虚拟内容400中目标区域410。在一些实施方式中，可以将上述虚拟指引进行隐藏，使得用户通过头戴显示设备无法看到该虚拟指引，提高了游戏体验。In the game scene, in order to enhance the game experience, a virtual sight can be displayed on the wearable device, and the entire wearable device can be used as a launcher for shooting operations. In some embodiments, the pointing direction of the virtual guide can be aligned with the reticle of the virtual sight, so that when a certain virtual content needs to be selected in the virtual content, the user can align the reticle of the virtual sight to Approve the virtual content. For example, referring to FIG. 18 , the terminal device displays avirtual sight 600 on the smart watch, the reticle of thevirtual sight 600 is aligned with the pointing direction of thevirtual ray 420 , the user can see through the head-mounted display device, and thevirtual sight 600 is alignedTarget area 410 invirtual content 400 . In some embodiments, the above-mentioned virtual guide can be hidden, so that the user cannot see the virtual guide through the head-mounted display device, thereby improving the game experience.

进一步的，在一些实施方式中，终端设备在检测到穿戴设备相对终端设备的位置及姿态信息发生变化时，根据变化后的位置及姿态信息，获取虚拟内容中被选中的新的目标区域。Further, in some embodiments, when the terminal device detects that the position and posture information of the wearable device relative to the terminal device changes, it acquires a new target area selected in the virtual content according to the changed position and posture information.

在一些实施方式中，在根据虚拟内容的显示位置和穿戴设备的位置及姿态信息，确定虚拟内容中被选中的目标区域之后，可以实时检测穿戴设备相对终端设备的位置及姿态信息，以在穿戴设备相对终端设备的位置及姿态信息发生变化时，对虚拟内容中被选中的目标区域进行更新。也就是说，在检测到穿戴设备相对终端设备的位置及姿态信息发生变化时，根据变化后的位置及姿态信息，通过上述确定目标区域的方法，重新确定虚拟内容中被选中的目标区域，并进行与新的目标区域对应的处理操作。从而，可以使用户改变穿戴设备相对终端设备的空间位置，以对虚拟内容中的被选中区域进行移动调整等。可将穿戴设备作为控制器，对虚拟内容进行选取等操作。In some embodiments, after determining the selected target area in the virtual content according to the display position of the virtual content and the position and attitude information of the wearable device, the position and attitude information of the wearable device relative to the terminal device can be detected in real time, so as to detect the position and attitude information of the wearable device relative to the terminal device in real time. When the position and posture information of the device relative to the terminal device changes, the selected target area in the virtual content is updated. That is to say, when it is detected that the position and posture information of the wearable device relative to the terminal device changes, according to the changed position and posture information, the above-mentioned method for determining the target area is used to re-determine the selected target area in the virtual content, and The processing operation corresponding to the new target area is performed. Therefore, the user can change the spatial position of the wearable device relative to the terminal device to move and adjust the selected area in the virtual content. The wearable device can be used as a controller to select virtual content and other operations.

另外，由于上述步骤中，终端设备可以将实体对象的相关数据(如UI数据)进行增强现实显示以实现与现实物体的交互，因此，通过上述对虚拟内容的控制方式，终端设备可以进一步对实体对象对应的虚拟内容进行操作，以实现与现实物体的进一步交互。In addition, in the above steps, the terminal device can display the relevant data of the entity object (such as UI data) in augmented reality to realize the interaction with the real object. Therefore, through the above control method for the virtual content, the terminal device can further control the entity object. The virtual content corresponding to the object is operated to realize further interaction with the real object.

在一些实施例中，上述实体对象为智能家居设备时，终端设备还可以通过穿戴设备设定智能家居设备的状态，从而用户可通过穿戴设备与智能家居设备进行交互。具体地，终端设备可以获取目标区域对应的虚拟内容，根据该虚拟内容，生成执行指令，将执行指令传输至智能家居设备，执行指令用于指示智能家居设备执行设定操作。In some embodiments, when the above entity object is a smart home device, the terminal device can also set the state of the smart home device through the wearable device, so that the user can interact with the smart home device through the wearable device. Specifically, the terminal device can acquire virtual content corresponding to the target area, generate an execution instruction according to the virtual content, transmit the execution instruction to the smart home device, and the execution instruction is used to instruct the smart home device to perform the setting operation.

在一些实施方式中，终端设备可以对智能家居设备进行识别，以在虚拟空间中显示该智能家居设备的虚拟交互界面(虚拟UI)，进而用户可以通过调整穿戴设备的位置及姿态信息，对虚拟交互界面中的不同的虚拟内容进行选择，以进行对智能家居设备的状态的设定。In some embodiments, the terminal device can identify the smart home device to display the virtual interactive interface (virtual UI) of the smart home device in the virtual space, and then the user can adjust the position and posture information of the wearable device to display the virtual interactive interface (virtual UI) of the smart home device in the virtual space. Different virtual contents in the interactive interface are selected to set the state of the smart home device.

在一些实施方式中，终端设备显示出智能家居设备的虚拟交互界面时，可以通过上述实施例的目标区域的获取方式，获取到被选中的目标区域，从而可以根据目标区域，获取对应的被选中的虚拟内容。其中，该虚拟内容为显示的虚拟交互界面中的一部分。例如，请参见图11，终端设备显示出的虚拟内容400为智能台灯的虚拟交互界面，目标区域410为亮度设置，与目标区域410对应的虚拟内容为具体的亮度值。具体地，终端设备获取到被选中的目标区域时，可以获取到目标区域的空间位置，根据目标区域的空间位置，从虚拟交互界面中获取与该空间位置对应的虚拟内容。In some embodiments, when the terminal device displays the virtual interactive interface of the smart home device, the selected target area can be acquired through the acquisition method of the target area in the above-mentioned embodiment, so that the corresponding selected target area can be acquired according to the target area. virtual content. The virtual content is a part of the displayed virtual interactive interface. For example, referring to FIG. 11 , thevirtual content 400 displayed by the terminal device is the virtual interactive interface of the smart desk lamp, thetarget area 410 is the brightness setting, and the virtual content corresponding to thetarget area 410 is the specific brightness value. Specifically, when the terminal device obtains the selected target area, it can obtain the spatial position of the target area, and obtain virtual content corresponding to the spatial position from the virtual interactive interface according to the spatial position of the target area.

在一些实施方式中，终端设备可以根据获取到虚拟内容对智能家居设备的状态进行调整。具体地，终端设备可根据目标区域对应的虚拟内容，生成对应的执行指令，该执行指令用于将智能家居设备的状态调整为与虚拟内容对应的状态。例如，虚拟内容为亮度50时，终端设备可生成调整智能台灯亮度为50的执行指令。在一些实施方式中，虚拟内容可与执行指令对应，也就是说，当终端设备获取到目标区域对应的虚拟内容时，可根据虚拟内容与执行指令的对应关系，生成与虚拟内容对应的执行指令。其中，虚拟内容与执行指令的对应关系可存储于终端设备中，也可以从服务器中去获取。In some embodiments, the terminal device can adjust the state of the smart home device according to the acquired virtual content. Specifically, the terminal device can generate a corresponding execution instruction according to the virtual content corresponding to the target area, and the execution instruction is used to adjust the state of the smart home device to a state corresponding to the virtual content. For example, when the brightness of the virtual content is 50, the terminal device may generate an execution instruction to adjust the brightness of the smart desk lamp to 50. In some embodiments, the virtual content may correspond to the execution instruction, that is, when the terminal device obtains the virtual content corresponding to the target area, it may generate the execution instruction corresponding to the virtual content according to the corresponding relationship between the virtual content and the execution instruction . The corresponding relationship between the virtual content and the execution instruction may be stored in the terminal device, or may be acquired from the server.

终端设备在生成上述执行指令后，可以将该执行指令传输至智能家居设备，该执行指令用于指示智能家居设备执行设定操作。智能家居设备接收到该执行指令时，可以根据该执行指令进行设定操作，以将当前的状态调整为用户所设定的状态，即上述与虚拟内容对应的状态。从而实现了穿戴设备与智能家居设备之间的交互，提高了穿戴设备的智能化水平。After generating the above execution instruction, the terminal device may transmit the execution instruction to the smart home device, where the execution instruction is used to instruct the smart home device to perform the setting operation. When the smart home device receives the execution instruction, it can perform a setting operation according to the execution instruction, so as to adjust the current state to the state set by the user, that is, the above-mentioned state corresponding to the virtual content. Thus, the interaction between the wearable device and the smart home device is realized, and the intelligence level of the wearable device is improved.

进一步的，终端设备通过穿戴设备对目标区域进行选取后，可以进一步利用穿戴设备，对目标区域的虚拟内容进行控制显示。在一些实施例中，终端设备可以根据穿戴设备检测到的操控动作参数、终端设备检测到的用户手势、相对空间位置关系的变化信息中的至少一种，对目标区域的虚拟内容的显示进行控制。Further, after the terminal device selects the target area through the wearable device, the wearable device can be further used to control and display the virtual content of the target area. In some embodiments, the terminal device may control the display of the virtual content in the target area according to at least one of the manipulation action parameters detected by the wearable device, the user gesture detected by the terminal device, and the change information of the relative spatial positional relationship .

其中，根据穿戴设备检测到的操控动作参数和相对空间位置关系的变化信息中的至少一种，对目标区域的虚拟内容的显示进行控制，可以参考上述实施例中的对显示的虚拟内容进行控制的步骤，在此不再赘述。Wherein, according to at least one of the manipulation action parameters detected by the wearable device and the change information of the relative spatial positional relationship, the display of the virtual content in the target area is controlled, and reference may be made to the control of the displayed virtual content in the above embodiment. steps, which are not repeated here.

其中，上述终端设备检测到的用户手势，可以是通过图像采集装置实时扫描用户，采集用户的手势，也可以是通过穿戴设备的摄像头实时扫描用户，采集用户的手势。在一些实施方式中，终端设备可以根据用户的手势，生成手势对应的操控指令，对显示的虚拟内容进行控制。其中，用户的手势可以为上升、落下、左右挥手，也可以仅是手指的上下滑动等。例如，请参见图19，用户通过智能手表选中目标区域410时，终端设备可以根据用户食指的上下移动手势，控制虚拟选项411的切换。Wherein, the user gesture detected by the above-mentioned terminal device may be real-time scanning of the user through an image acquisition device to collect the user's gesture, or may be real-time scanning of the user through a camera of the wearable device to collect the user's gesture. In some embodiments, the terminal device may generate manipulation instructions corresponding to the gesture according to the user's gesture, and control the displayed virtual content. Wherein, the user's gesture may be rising, falling, waving left and right, or just sliding up and down of a finger. For example, referring to FIG. 19 , when the user selects thetarget area 410 through the smart watch, the terminal device can control the switching of thevirtual options 411 according to the up and down movement gesture of the user's index finger.

在一些实施例中，终端设备也可以同时根据穿戴设备检测到的操控动作参数以及终端设备检测到的用户手势，对显示的虚拟内容进行控制。例如，用户在按住智能手表上的一个按键的同时手指上下移动，可以实现将选中的虚拟内容进行上下移动。In some embodiments, the terminal device may also control the displayed virtual content according to the manipulation action parameters detected by the wearable device and the user gesture detected by the terminal device. For example, when a user moves a finger up and down while pressing a button on the smart watch, the selected virtual content can be moved up and down.

由于用户可以通过旋转手腕改变穿戴设备的位置及姿态信息，因此，在一些实施方式中，可以是不同旋转角度可对应不同的控制效果。具体地，终端设备可以根据位置及姿态的变化信息，判断穿戴设备是否处于旋转状态，当处于旋转状态时，获取穿戴设备的旋转角度，根据旋转角度与控制指令的对应关系，生成与旋转角度对应的控制指令。根据控制指令，对虚拟内容的显示进行控制。Since the user can change the position and posture information of the wearable device by rotating the wrist, in some embodiments, different rotation angles can correspond to different control effects. Specifically, the terminal device can determine whether the wearable device is in a rotating state according to the change information of the position and posture, and when it is in the rotating state, obtain the rotation angle of the wearable device, and generate a corresponding rotation angle according to the corresponding relationship between the rotation angle and the control command. control command. According to the control instruction, the display of the virtual content is controlled.

由于用户旋转手腕时，手腕上佩戴的穿戴设备也会进行旋转，即穿戴设备相对终端设备的位置及姿态信息也会发生改变。因此，终端设备可以根据位置及姿态的变化信息，确定出穿戴设备的运动距离、运动方向等运动参数，从而可以根据该运动参数判断穿戴设备是否处于翻转状态。When the user rotates the wrist, the wearable device worn on the wrist also rotates, that is, the position and posture information of the wearable device relative to the terminal device will also change. Therefore, the terminal device can determine the movement parameters such as the movement distance and movement direction of the wearable device according to the change information of the position and posture, so as to determine whether the wearable device is in a flipped state according to the movement parameters.

当终端设备根据位置及姿态的变化信息，判断出穿戴设备处于旋转状态时，可以获取穿戴设备的旋转角度，即获取穿戴设备的翻转角度。可以理解的是，穿戴设备的翻转角度可根据位置及姿态的变化信息获取。在一些实施方式中，上述旋转角度与控制指令对应，即穿戴设备翻转不同角度时，对应的控制指令也就不同，控制虚拟内容的显示效果就不同。终端设备在得到穿戴设备的旋转角度后，可根据预存的旋转角度与控制指令的对应关系，生成与当前旋转角度对应的控制指令。其中。该对应关系可以预存于终端设备中，也可以从服务器中下载。When the terminal device determines that the wearable device is in a rotating state according to the change information of the position and posture, the rotation angle of the wearable device can be obtained, that is, the flip angle of the wearable device can be obtained. It can be understood that the flip angle of the wearable device can be obtained according to the change information of the position and posture. In some embodiments, the above-mentioned rotation angles correspond to control instructions, that is, when the wearable device is flipped at different angles, the corresponding control instructions are also different, and the display effects of the control virtual content are different. After obtaining the rotation angle of the wearable device, the terminal device can generate a control instruction corresponding to the current rotation angle according to the pre-stored correspondence between the rotation angle and the control instruction. in. The corresponding relationship may be pre-stored in the terminal device or downloaded from the server.

例如，可以设定旋转穿戴设备90°(即旋转手腕90°)时，控制指令为选定虚拟指引指向的目标区域内的虚拟内容，也可以设定穿戴设备的旋转角度在-30°至30°区间时，控制指令为单击虚拟指引指向的目标区域内的虚拟内容，进一步的，还可以设定旋转角度为正时，控制指令为右键虚拟指引指向的目标区域内的虚拟内容，展示下一步可操作指令，旋转角度为负时，控制指令为左键虚拟指引指向的目标区域内的虚拟内容，直接进行默认操作。For example, when the wearable device is rotated 90° (that is, the wrist is rotated 90°), the control command can be set to select the virtual content in the target area pointed by the virtual guide, or the rotation angle of the wearable device can be set between -30° to 30° In the ° range, the control command is to click on the virtual content in the target area pointed by the virtual guide. Further, you can also set the rotation angle to be positive, and the control command is the virtual content in the target area pointed by the virtual guide by right-clicking. One-step operable command, when the rotation angle is negative, the control command is the virtual content in the target area pointed by the virtual guide of the left button, and the default operation is performed directly.

终端设备在生成控制指令后，可以根据该控制指令，对虚拟指引指向的目标区域内的虚拟内容的显示进行控制。其中，对虚拟指引指向的目标区域内的虚拟内容的显示进行控制，可以是对虚拟指引指向的目标区域内的虚拟内容进行内容切换、移动、旋转、以及缩放比例调整等，在此不作限定。After generating the control instruction, the terminal device can control the display of the virtual content in the target area pointed by the virtual guide according to the control instruction. Wherein, controlling the display of the virtual content in the target area pointed by the virtual guide may be content switching, moving, rotating, and scaling adjustment of the virtual content in the target area pointed by the virtual guide, which is not limited here.

在一些实施例中，终端设备还可以根据用户在穿戴设备的触控屏上的触摸手势，实现虚拟内容显示的快捷方式。具体地，该虚拟内容的控制方法还可包括：In some embodiments, the terminal device may also implement a shortcut for displaying virtual content according to the user's touch gesture on the touch screen of the wearable device. Specifically, the method for controlling virtual content may further include:

获取穿戴设备的触摸屏检测到的触摸轨迹；判断触摸轨迹是否与预设轨迹匹配；当与预设轨迹匹配时，根据预设轨迹与应用程序的对应关系，访问与预设轨迹对应的应用程序，并显示应用程序的虚拟界面。Obtain the touch track detected by the touch screen of the wearable device; determine whether the touch track matches the preset track; when it matches the preset track, access the application program corresponding to the preset track according to the corresponding relationship between the preset track and the application program, and display the virtual interface of the application.

在一些实施方式中，用户手指在触摸屏上滑动触摸时，穿戴设备的触摸屏可以检测到对应的触摸轨迹，穿戴设备可将该触摸轨迹的数据传输至终端设备，以使终端设备可以获取到该触摸轨迹。终端设备可以通过判断该触摸轨迹是否与预设轨迹匹配，确定是否进行应用程序的快捷方式。具体地，终端设备在判断出该触摸轨迹与预设轨迹匹配时，可以根据该预设轨迹与应用程序的对应关系，访问与预设轨迹对应的应用程序，并显示应用程序的虚拟界面。从而实现了虚拟内容显示的快捷方式，提高了穿戴设备与终端设备之间的交互性。例如，用户在穿戴设备上输入的“S”手势时，终端设备可以直接在虚拟空间中显示虚拟搜索页面。In some embodiments, when the user's finger slides on the touch screen, the touch screen of the wearable device can detect the corresponding touch track, and the wearable device can transmit the data of the touch track to the terminal device, so that the terminal device can acquire the touch trajectory. The terminal device can determine whether to perform a shortcut of the application by judging whether the touch track matches the preset track. Specifically, when judging that the touch track matches the preset track, the terminal device can access the application program corresponding to the preset track according to the corresponding relationship between the preset track and the application program, and display the virtual interface of the application program. Thus, a shortcut for virtual content display is realized, and the interactivity between the wearable device and the terminal device is improved. For example, when the user inputs the "S" gesture on the wearable device, the terminal device can directly display the virtual search page in the virtual space.

其中，该预设轨迹可以是终端设备出厂默认的，也可以是用户自行设定的，可以存储于终端设备，也可以从服务器下载，在此不作限定。同理，该预设轨迹与应用程序的对应关系也可以是终端设备出厂默认的，也可以是用户自行设定的，该对应关系可以存储于终端设备，可以从服务器下载，在此不作限定。The preset track may be factory default of the terminal device, or may be set by the user, may be stored in the terminal device, or downloaded from a server, which is not limited herein. Similarly, the corresponding relationship between the preset track and the application program can also be factory default of the terminal device, or can be set by the user, the corresponding relationship can be stored in the terminal device, and can be downloaded from the server, which is not limited here.

在一些实施例中，终端设备根据用户在穿戴设备的触控屏上的触摸手势，在虚拟空间中显示应用程序的虚拟界面后，可以根据穿戴设备检测到的操控动作参数以及穿戴设备与终端设备之间相对空间位置关系的变化信息中的至少一种，对显示的虚拟界面进行控制。其中，对显示的虚拟界面进行控制可以参考上述实施例中对显示的虚拟内容进行控制的对应步骤。In some embodiments, after the terminal device displays the virtual interface of the application program in the virtual space according to the user's touch gesture on the touch screen of the wearable device, the terminal device can display the virtual interface of the application program in the virtual space, and can determine the parameters of the manipulation action detected by the wearable device and the relationship between the wearable device and the terminal device. At least one of the change information of the relative spatial position relationship between them controls the displayed virtual interface. Wherein, for controlling the displayed virtual interface, reference may be made to the corresponding steps of controlling the displayed virtual content in the foregoing embodiment.

在一些实施例中，终端设备在显示上述虚拟内容后，可以根据用户在穿戴设备的触控屏上的触摸手势，快捷显示应用程序的虚拟界面，该虚拟界面可叠加显示于上述显示的虚拟内容。作为一种实施方式，虚拟界面可叠加显示于上述显示的虚拟内容，可以是取消虚拟内容的渲染生成以及显示，并将虚拟界面显示于虚拟内容的显示位置，实现显示内容切换的效果。当前，虚拟界面可叠加显示于上述显示的虚拟内容的实现方式还可以是参考上述虚拟字符叠加显示于显示的虚拟内容的方式。In some embodiments, after displaying the above virtual content, the terminal device can quickly display the virtual interface of the application program according to the user's touch gesture on the touch screen of the wearable device, and the virtual interface can be displayed superimposed on the above displayed virtual content . As an embodiment, the virtual interface can be superimposed and displayed on the displayed virtual content, and the rendering and display of the virtual content can be cancelled, and the virtual interface can be displayed on the display position of the virtual content to realize the effect of switching the displayed content. Currently, the implementation manner in which the virtual interface can be superimposed and displayed on the displayed virtual content may also be a manner of superimposing and displayed on the displayed virtual content with reference to the above-mentioned virtual characters.

在一些实施例中，穿戴设备上可设置有摄像头，该摄像头可用于采集用户的图像，当在聊天等场景中时，穿戴设备可以将摄像头采集到的图像传送给终端设备，终端设备将该图像实时传送给聊天对象，实现AR设备的视频通话功能。在一些实施方式中，该聊天对象可以佩戴有头戴显示装置，头戴显示装置接受到终端设备传送的图像后，可将该图像以AR的方式进行显示，实现AR设备之间的虚拟视频通话功能。在另一些实施方式中，该聊天对象也可以通过手机、电脑等移动终端接收终端设备传送的图像，移动终端再根据该图像生成视频，实现AR设备与其他移动终端的视频通过功能。In some embodiments, a camera may be set on the wearable device, and the camera may be used to capture images of the user. When in a scene such as a chat, the wearable device may transmit the image captured by the camera to the terminal device, and the terminal device will use the image Real-time transmission to the chat object to realize the video call function of the AR device. In some embodiments, the chat object may wear a head-mounted display device, and after receiving the image transmitted by the terminal device, the head-mounted display device may display the image in the form of AR to realize a virtual video call between AR devices Function. In other embodiments, the chat object can also receive an image transmitted by the terminal device through a mobile terminal such as a mobile phone or a computer, and the mobile terminal generates a video according to the image to realize the video passing function between the AR device and other mobile terminals.

另外，穿戴设备上的摄像头还可适用于人脸支付场景，在支付场景中，当需要采集人脸图像进行验证时，可直接通过摄像头采集用户图像进行人脸验证，在穿戴设备上实现人脸支付的功能。In addition, the camera on the wearable device can also be applied to the face payment scenario. In the payment scenario, when the face image needs to be collected for verification, the user image can be directly collected through the camera for face verification, and the face verification can be realized on the wearable device. payment function.

本申请实施例提供的虚拟内容的控制方法，通过判断穿戴设备是否处于指定动作状态，来确定是否获取待显示的虚拟内容的内容数据，以实现虚拟内容的抬腕显示，以及通过对虚拟内容的显示位置进行锁定，以在放下手腕时，终端设备仍然可以显示虚拟内容，降低用户手臂的负担，并在对虚拟内容进行显示后，可以根据穿戴设备检测到的操控动作参数以及相对空间位置关系的变化信息中的至少一种，对显示的虚拟内容进行控制，从而实现根据穿戴设备控制虚拟内容的显示，实现了穿戴设备与终端设备之间的交互，提高了用户与虚拟内容的交互性和交互便利性。The method for controlling virtual content provided by the embodiment of the present application determines whether to acquire the content data of the virtual content to be displayed by judging whether the wearable device is in a specified action state, so as to realize the wrist-lifting display of the virtual content, and through the control of the virtual content The display position is locked, so that when the wrist is lowered, the terminal device can still display the virtual content, which reduces the burden on the user's arm. At least one of the change information controls the displayed virtual content, so as to control the display of the virtual content according to the wearable device, realize the interaction between the wearable device and the terminal device, and improve the interactivity and interaction between the user and the virtual content Convenience.

请参阅图22，其示出了本申请实施例提供的一种虚拟内容的控制装置500的结构框图，应用于终端设备，该装置可以包括：位置获取模块510、数据获取模块520、内容生成模块530、显示控制模块540以及内容控制模块550。其中，位置获取模块510用于获取穿戴设备与终端设备之间的相对空间位置关系；数据获取模块520用于获取待显示的虚拟内容的内容数据；内容生成模块530用于根据内容数据和相对空间位置关系，生成虚拟内容；显示控制模块540用于对虚拟内容进行显示；内容控制模块550用于根据穿戴设备检测到的操控动作参数以及相对空间位置关系的变化信息中的至少一种，对显示的虚拟内容进行控制。Please refer to FIG. 22, which shows a structural block diagram of a virtual content control apparatus 500 provided by an embodiment of the present application, applied to a terminal device, the apparatus may include: a location acquisition module 510, a data acquisition module 520, and a content generation module 530 , a display control module 540 and a content control module 550 . Wherein, the position acquisition module 510 is used to acquire the relative spatial position relationship between the wearable device and the terminal device; the data acquisition module 520 is used to acquire the content data of the virtual content to be displayed; the content generation module 530 is used to obtain the content data and the relative space position relationship, to generate virtual content; the display control module 540 is used to display the virtual content; the content control module 550 is used to display according to at least one of the manipulation action parameters detected by the wearable device and the change information of the relative spatial positional relationship. control of virtual content.

在一些实施例中，该虚拟内容的控制装置500还可以包括：动作判断模块以及获取执行模块。其中，动作判断模块用于根据相对空间位置关系，判断穿戴设备是否处于指定动作状态；获取执行模块用于当处于指定动作状态时，执行获取待显示的虚拟内容的内容数据的步骤。In some embodiments, the virtual content control apparatus 500 may further include: an action judgment module and an acquisition execution module. The action judging module is used for judging whether the wearable device is in the specified action state according to the relative spatial position; the acquisition and execution module is used for executing the step of acquiring the content data of the virtual content to be displayed when it is in the specified action state.

作为一种实施方式，内容控制模块550可以具体用于：根据相对空间位置关系的变化信息，对显示的虚拟内容进行内容切换、移动、旋转、以及缩放比例调整中的至少一种。As an embodiment, the content control module 550 may be specifically configured to: perform at least one of content switching, movement, rotation, and scaling adjustment on the displayed virtual content according to the change information of the relative spatial positional relationship.

作为另一种实施方式，内容控制模块550可以具体用于：若在预设时长内，获取相对空间位置关系的变化数值均小于预设阈值的持续时长；当持续时长达到预设时长时，则将虚拟内容固定显示于当前显示位置。As another implementation manner, the content control module 550 may be specifically configured to: if within a preset time period, the acquired change values of the relative spatial position relationship are all less than the preset threshold value for the duration; when the duration reaches the preset time, then Pin the virtual content to the current display position.

在一些实施例中，上述穿戴设备检测到的操控动作参数，包括：穿戴设备的按键检测到的按压参数、穿戴设备的触摸屏检测到的触摸参数、以及穿戴设备的转盘检测到的转动参数中的一种或多种。In some embodiments, the manipulation action parameters detected by the wearable device include: pressing parameters detected by the buttons of the wearable device, touch parameters detected by the touch screen of the wearable device, and rotation parameters detected by the turntable of the wearable device. one or more.

作为又一种实施方式，上述操控动作参数包括触摸屏的触摸参数，触摸参数包括触摸轨迹，内容控制模块550可以具体用于：根据触摸轨迹，生成与触摸轨迹匹配的至少一个虚拟字符；将至少一个虚拟字符叠加显示于虚拟内容；当任意一个虚拟字符处于选中状态且接收到对处于选中状态的虚拟字符的确认指令时，将处于选中状态的虚拟字符添加至虚拟内容中。As another embodiment, the above-mentioned manipulation action parameters include touch parameters of the touch screen, and the touch parameters include a touch track. The content control module 550 may be specifically configured to: generate at least one virtual character matching the touch track according to the touch track; The virtual characters are superimposed and displayed on the virtual content; when any virtual character is in the selected state and a confirmation instruction for the virtual character in the selected state is received, the virtual character in the selected state is added to the virtual content.

在一些实施例中，上述操控动作参数包括转盘的转动参数，内容控制模块550可以具体用于：根据转动参数与操控指令的对应关系，生成转动参数对应的操控指令；根据操控指令，对显示的虚拟内容进行控制。In some embodiments, the above-mentioned manipulation action parameters include the rotation parameters of the turntable, and the content control module 550 may be specifically configured to: generate a manipulation command corresponding to the rotation parameter according to the corresponding relationship between the rotation parameter and the manipulation command; Virtual content for control.

在一些实施例中，该虚拟内容的控制装置500还可以包括：触摸轨迹获取模块、判断模块以及内容增加模块。其中，触摸轨迹获取模块用于获取穿戴设备的触摸屏检测到的触摸轨迹；判断模块用于判断触摸轨迹是否与预设轨迹匹配；内容增加模块用于当与预设轨迹匹配时，根据预设轨迹与应用程序的对应关系，访问与预设轨迹对应的应用程序，并显示应用程序的虚拟界面。In some embodiments, the virtual content control apparatus 500 may further include: a touch track acquisition module, a judgment module, and a content addition module. Wherein, the touch track acquisition module is used to obtain the touch track detected by the touch screen of the wearable device; the judgment module is used to judge whether the touch track matches the preset track; the content addition module is used to, when it matches the preset track, according to the preset track Correspondence with the application, access the application corresponding to the preset track, and display the virtual interface of the application.

在一些实施例中，数据获取模块520可以具体用于：获取穿戴设备对应的身份标识；根据身份标识访问与身份标识对应的存储数据；从存储数据中获取待显示的虚拟内容的内容数据。In some embodiments, the data acquisition module 520 may be specifically configured to: acquire an identity identifier corresponding to the wearable device; access stored data corresponding to the identity identifier according to the identity identifier; and acquire content data of virtual content to be displayed from the stored data.

在一些实施例中，上述穿戴设备包括惯性测量传感器，位置获取模块510可以具体用于：获取包含穿戴设备的图像以及惯性测量传感器的感应数据；识别图像中的穿戴设备，得到穿戴设备的轮廓数据；根据感应数据以及轮廓数据，获取穿戴设备与终端设备之间的相对空间位置关系。In some embodiments, the above-mentioned wearable device includes an inertial measurement sensor, and the position acquisition module 510 may be specifically configured to: acquire an image including the wearable device and the sensing data of the inertial measurement sensor; identify the wearable device in the image, and obtain contour data of the wearable device ; Obtain the relative spatial position relationship between the wearable device and the terminal device according to the sensing data and the contour data.

所属领域的技术人员可以清楚地了解到，为描述的方便和简洁，上述描述装置和模块的具体工作过程，可以参考前述方法实施例中的对应过程，在此不再赘述。Those skilled in the art can clearly understand that, for the convenience and brevity of description, for the specific working process of the above-described devices and modules, reference may be made to the corresponding processes in the foregoing method embodiments, which will not be repeated here.

在本申请所提供的几个实施例中，所显示或讨论的模块相互之间的耦合或直接耦合或通信连接可以是通过一些接口，装置或模块的间接耦合或通信连接，可以是电性，机械或其它的形式。In several embodiments provided in this application, the coupling or direct coupling or communication connection between the modules shown or discussed may be through some interfaces, and the indirect coupling or communication connection of devices or modules may be electrical, mechanical or otherwise.

另外，在本申请各个实施例中的各功能模块可以集成在一个处理模块中，也可以是各个模块单独物理存在，也可以两个或两个以上模块集成在一个模块中。上述集成的模块既可以采用硬件的形式实现，也可以采用软件功能模块的形式实现。In addition, each functional module in each embodiment of the present application may be integrated into one processing module, or each module may exist physically alone, or two or more modules may be integrated into one module. The above-mentioned integrated modules can be implemented in the form of hardware, and can also be implemented in the form of software function modules.

综上，本申请实施例提供的一种虚拟内容控制装置，应用于终端设备，通过获取穿戴设备与终端设备之间的相对空间位置关系，以及获取待显示的虚拟内容的内容数据，并根据该内容数据和相对空间位置关系，生成虚拟内容，然后对虚拟内容进行显示，并根据穿戴设备检测到的操控动作参数以及相对空间位置关系的变化信息中的至少一种，对显示的虚拟内容进行控制，从而实现根据穿戴设备的空间位置，在虚拟空间中显示虚拟内容，并根据穿戴设备控制虚拟内容的显示，实现了穿戴设备与终端设备之间的交互，提高了用户与虚拟内容的交互性和交互便利性。To sum up, a virtual content control device provided by the embodiment of the present application is applied to a terminal device. By acquiring the relative spatial position relationship between the wearable device and the terminal device, and acquiring the content data of the virtual content to be displayed, and according to the Content data and relative spatial position relationship, generate virtual content, then display the virtual content, and control the displayed virtual content according to at least one of the manipulation action parameters detected by the wearable device and the change information of the relative spatial position relationship , so as to display the virtual content in the virtual space according to the spatial position of the wearable device, and control the display of the virtual content according to the wearable device, realize the interaction between the wearable device and the terminal device, and improve the interaction between the user and the virtual content. Interaction convenience.

请参考图23，其示出了本申请实施例提供的一种终端设备的结构框图。该终端设备100可以是智能手机、平板电脑、头戴显示装置等能够运行应用程序的终端设备。本申请中的终端设备100可以包括一个或多个如下部件：处理器110、存储器120、图像采集装置130以及一个或多个应用程序，其中一个或多个应用程序可以被存储在存储器120中并被配置为由一个或多个处理器110执行，一个或多个程序配置用于执行如前述方法实施例所描述的方法。Please refer to FIG. 23 , which shows a structural block diagram of a terminal device provided by an embodiment of the present application. Theterminal device 100 may be a terminal device capable of running an application program, such as a smart phone, a tablet computer, a head-mounted display device, or the like. Theterminal device 100 in this application may include one or more of the following components: aprocessor 110, amemory 120, animage acquisition device 130, and one or more application programs, wherein one or more application programs may be stored in thememory 120 and Configured to be executed by one ormore processors 110, the one or more programs are configured to perform the method as described in the foregoing method embodiments.

处理器110可以包括一个或者多个处理核。处理器110利用各种接口和线路连接整个终端设备100内的各个部分，通过运行或执行存储在存储器120内的指令、程序、代码集或指令集，以及调用存储在存储器120内的数据，执行终端设备100的各种功能和处理数据。可选地，处理器110可以采用数字信号处理(Digital Signal Processing，DSP)、现场可编程门阵列(Field－Programmable Gate Array，FPGA)、可编程逻辑阵列(Programmable LogicArray，PLA)中的至少一种硬件形式来实现。处理器110可集成中央处理器(CentralProcessing Unit，CPU)、图像处理器(Graphics Processing Unit，GPU)和调制解调器等中的一种或几种的组合。其中，CPU主要处理操作系统、用户界面和应用程序等；GPU用于负责显示内容的渲染和绘制；调制解调器用于处理无线通信。可以理解的是，上述调制解调器也可以不集成到处理器110中，单独通过一块通信芯片进行实现。Theprocessor 110 may include one or more processing cores. Theprocessor 110 uses various interfaces and lines to connect various parts of the entireterminal device 100, and executes by running or executing the instructions, programs, code sets or instruction sets stored in thememory 120, and calling the data stored in thememory 120. Various functions of theterminal device 100 and processing data. Optionally, theprocessor 110 may employ at least one of a digital signal processing (Digital Signal Processing, DSP), a Field-Programmable Gate Array (Field-Programmable Gate Array, FPGA), and a Programmable Logic Array (Programmable Logic Array, PLA). implemented in hardware. Theprocessor 110 may integrate one or a combination of a central processing unit (Central Processing Unit, CPU), a graphics processing unit (Graphics Processing Unit, GPU), a modem, and the like. Among them, the CPU mainly handles the operating system, user interface and application programs, etc.; the GPU is used for rendering and drawing of the display content; the modem is used to handle wireless communication. It can be understood that, the above-mentioned modem may also not be integrated into theprocessor 110, and is implemented by a communication chip alone.

存储器120可以包括随机存储器(Random Access Memory，RAM)，也可以包括只读存储器(Read-Only Memory)。存储器120可用于存储指令、程序、代码、代码集或指令集。存储器120可包括存储程序区和存储数据区，其中，存储程序区可存储用于实现操作系统的指令、用于实现至少一个功能的指令(比如触控功能、声音播放功能、图像播放功能等)、用于实现下述各个方法实施例的指令等。存储数据区还可以存储终端设备100在使用中所创建的数据等。Thememory 120 may include random access memory (Random Access Memory, RAM), or may include read-only memory (Read-Only Memory).Memory 120 may be used to store instructions, programs, codes, sets of codes, or sets of instructions. Thememory 120 may include a stored program area and a stored data area, wherein the stored program area may store instructions for implementing an operating system, instructions for implementing at least one function (such as a touch function, a sound playback function, an image playback function, etc.) , instructions for implementing the following method embodiments, and the like. The storage data area may also store data and the like created by theterminal device 100 in use.

在本申请实施例中，图像采集装置130用于采集实体对象的图像以及采集目标场景的场景图像。图像采集装置130可以为红外摄像头，也可以是彩色摄像头，具体的摄像头类型在本申请实施例中并不作为限定。In this embodiment of the present application, theimage acquisition device 130 is configured to acquire images of physical objects and scene images of a target scene. Theimage acquisition device 130 may be an infrared camera or a color camera, and the specific camera type is not limited in this embodiment of the present application.

请参考图24，其示出了本申请实施例提供的一种计算机可读存储介质的结构框图。该计算机可读存储介质800中存储有程序代码，程序代码可被处理器调用执行上述方法实施例中所描述的方法。Please refer to FIG. 24 , which shows a structural block diagram of a computer-readable storage medium provided by an embodiment of the present application. The computer-readable storage medium 800 stores program codes, and the program codes can be invoked by the processor to execute the methods described in the above method embodiments.

该计算机可读存储介质800可以是诸如闪存、EEPROM(电可擦除可编程只读存储器)、EPROM、硬盘或者ROM之类的电子存储器。可选地，计算机可读存储介质800包括非易失性计算机可读介质(non-transitory computer-readable storage medium)。计算机可读存储介质800具有执行上述方法中的任何方法步骤的程序代码810的存储空间。这些程序代码可以从一个或者多个计算机程序产品中读出或者写入到这一个或者多个计算机程序产品中。程序代码810可以例如以适当形式进行压缩。The computer readable storage medium 800 may be an electronic memory such as flash memory, EEPROM (Electrically Erasable Programmable Read Only Memory), EPROM, hard disk, or ROM. Optionally, the computer-readable storage medium 800 includes a non-transitory computer-readable storage medium. Computer readable storage medium 800 has storage space forprogram code 810 to perform any of the method steps in the above-described methods. These program codes can be read from or written to one or more computer program products.Program code 810 may be compressed, for example, in a suitable form.

最后应说明的是：以上实施例仅用以说明本申请的技术方案，而非对其限制；尽管参照前述实施例对本申请进行了详细的说明，本领域的普通技术人员当理解：其依然可以对前述各实施例所记载的技术方案进行修改，或者对其中部分技术特征进行等同替换；而这些修改或者替换，并不驱使相应技术方案的本质脱离本申请各实施例技术方案的精神和范围。Finally, it should be noted that: the above embodiments are only used to illustrate the technical solutions of the present application, but not to limit them; although the present application has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand: it can still be Modifications are made to the technical solutions described in the foregoing embodiments, or some technical features thereof are equivalently replaced; and these modifications or replacements do not drive the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions in the embodiments of the present application.

Claims (13)

1. A control method of virtual content is applied to a terminal device, the terminal device is in communication connection with a wearable device, and the method comprises the following steps:

acquiring a relative spatial position relationship between the wearable device and the terminal device;

acquiring content data of virtual content to be displayed;

generating the virtual content according to the content data and the relative spatial position relation;

displaying the virtual content;

and controlling the displayed virtual content according to at least one of the control action parameters detected by the wearable device and the change information of the relative spatial position relationship.

2. The method according to claim 1, wherein prior to the obtaining content data of virtual content to be displayed, the method further comprises:

judging whether the wearable equipment is in a designated action state or not according to the relative spatial position relation;

and when the virtual content is in the designated action state, executing the step of acquiring the content data of the virtual content to be displayed.

3. The method according to claim 1, wherein the controlling the virtual content displayed according to the information about the change in the relative spatial position relationship comprises:

and performing at least one of content switching, moving, rotating and scaling on the displayed virtual content according to the change information of the relative spatial position relationship.

4. The method according to claim 1, wherein the controlling the virtual content displayed according to the information about the change in the relative spatial position relationship comprises:

acquiring the duration that the change value of the relative spatial position relation is smaller than a preset threshold;

and when the duration reaches a preset duration, fixing the virtual content at the current display position.

5. The method of claim 1, wherein the detected parameters of the manipulation action by the wearable device comprise:

the wearable device comprises one or more of a pressing parameter detected by a key of the wearable device, a touch parameter detected by a touch screen of the wearable device, and a rotation parameter detected by a turntable of the wearable device.

6. The method of claim 5, wherein the manipulation action parameters comprise touch parameters of the touch screen, the touch parameters comprise touch tracks, and the controlling the displayed virtual content according to the manipulation action parameters detected by the wearable device comprises:

generating at least one virtual character matched with the touch track according to the touch track;

displaying the at least one virtual character in an overlapping manner on the virtual content;

and when any virtual character is in a selected state and a confirmation instruction of the virtual character in the selected state is received, adding the virtual character in the selected state to the virtual content.

7. The method according to claim 5, wherein the manipulation action parameters include rotation parameters of the dial, and the controlling the displayed virtual content according to the manipulation action parameters detected by the wearable device includes:

generating a control instruction corresponding to the rotation parameter according to the corresponding relation between the rotation parameter and the control instruction;

and controlling the displayed virtual content according to the control instruction.

8. The method of claim 1, further comprising:

acquiring a touch track detected by a touch screen of the wearable device;

judging whether the touch track is matched with a preset track or not;

and when the preset track is matched, accessing the application program corresponding to the preset track according to the corresponding relation between the preset track and the application program, and displaying the virtual interface of the application program.

9. The method according to any one of claims 1 to 8, wherein the obtaining content data of the virtual content to be displayed comprises:

acquiring an identity corresponding to the wearable device;

accessing storage data corresponding to the identity according to the identity;

and acquiring the content data of the virtual content to be displayed from the storage data.

10. The method according to any one of claims 1-8, wherein the wearable device comprises an inertial measurement sensor, and the acquiring the relative spatial position relationship between the wearable device and the terminal device comprises:

acquiring an image containing the wearable device and sensing data of the inertial measurement sensor;

identifying the wearable equipment in the image to obtain profile data of the wearable equipment;

and acquiring the relative spatial position relation between the wearable device and the terminal device according to the induction data and the contour data.

11. The utility model provides a controlling means of virtual content which is characterized in that, is applied to terminal equipment, terminal equipment and wearing equipment communication connection, the device includes:

the position acquisition module is used for acquiring the relative spatial position relationship between the wearable device and the terminal device;

the data acquisition module is used for acquiring content data of virtual content to be displayed;

the content generation module is used for generating the virtual content according to the content data and the relative spatial position relation;

the display control module is used for displaying the virtual content;

and the content control module is used for controlling the displayed virtual content according to at least one of the control action parameters detected by the wearable device and the change information of the relative spatial position relationship.

12. A terminal device, comprising:

one or more processors;

a memory;

one or more applications, wherein the one or more applications are stored in the memory and configured to be executed by the one or more processors, the one or more programs configured to perform the method of any of claims 1-10.

13. A computer-readable storage medium, having stored thereon program code that can be invoked by a processor to perform the method according to any one of claims 1 to 10.

Images