CN111857457A

Movatterモバイル変換

Info

Publication number: CN111857457A
Application number: CN202010576184.1A
Authority: CN
Inventors: 刘智宇
Original assignee: Beijing Baidu Netcom Science and Technology Co Ltd
Current assignee: Beijing Baidu Netcom Science and Technology Co Ltd
Priority date: 2020-06-22
Filing date: 2020-06-22
Publication date: 2020-10-30

Abstract

Translated fromChinese

本申请公开了一种云手机的操控方法、装置、电子设备及可读存储介质，涉及云计算技术领域。具体实现方案为：输出显示设备上所运行的云手机应用中待操控应用的应用操控界面；响应于用户基于所述应用操控界面的操作，获取所在终端设备上的触控事件参数；基于与所述云手机应用之间的通信连接，将所述终端设备上的触控事件参数发送给所述云手机应用，以供所述云手机应用根据所述终端设备上的触控事件参数、所述显示设备的屏幕分辨率和所述待操控应用的页面属性，获得所述待操控应用的触控事件参数，根据所述待操控应用的触控事件参数，对所述待操控应用进行操控。能够实现对运行在显示设备上的云手机进行操控。

The present application discloses a control method, device, electronic device and readable storage medium of a cloud mobile phone, and relates to the technical field of cloud computing. The specific implementation scheme is: outputting the application control interface of the application to be controlled in the cloud mobile phone application running on the display device; in response to the user's operation based on the application control interface, obtaining the touch event parameters on the terminal device; The communication connection between the cloud mobile phone applications, and the touch event parameters on the terminal device are sent to the cloud mobile phone application, so that the cloud mobile phone application can use the touch event parameters on the terminal device, the The screen resolution of the display device and the page attribute of the application to be controlled are obtained, the touch event parameters of the application to be controlled are obtained, and the application to be controlled is controlled according to the touch event parameters of the application to be controlled. It can control the cloud mobile phone running on the display device.

Description

Translated fromChinese

云手机的操控方法、装置、电子设备及可读存储介质Control method, device, electronic device and readable storage medium of cloud mobile phone

技术领域technical field

涉及计算机技术领域，具体涉及云计算技术领域，尤其涉及云手机的操控方法、装置、电子设备及可读存储介质。It relates to the field of computer technology, in particular to the field of cloud computing technology, and in particular to a control method, device, electronic device and readable storage medium for a cloud mobile phone.

背景技术Background technique

云手机是一种基于云计算和虚拟化的虚拟手机，通过远程实时控制云手机，能够实现应用的云端运行。目前，云手机已经能够运行在电视、电脑等大屏幕显示设备上了。Cloud mobile phone is a virtual mobile phone based on cloud computing and virtualization. It can realize cloud operation of applications by remotely controlling the cloud mobile phone in real time. At present, cloud mobile phones have been able to run on large-screen display devices such as TVs and computers.

然而，在一些情况下，例如，云手机中的大部分应用都是基于触控屏开发的，而一般的显示设备却不具备触控功能等，使得无法直接通过显示设备的屏幕操控运行在其上的云手机中的应用。因此，亟需提供一种云手机的操控方法，用以实现对运行在显示设备上的云手机进行操控。However, in some cases, for example, most of the applications in the cloud phone are developed based on the touch screen, and the general display device does not have the touch function, etc., making it impossible to directly control the operation on the display device through the screen. application in the cloud phone on the . Therefore, there is an urgent need to provide a control method for a cloud phone, so as to control the cloud phone running on a display device.

发明内容SUMMARY OF THE INVENTION

本申请的多个方面提供云手机的操控方法、装置、电子设备及可读存储介质，用以实现对运行在显示设备上的云手机进行操控。Various aspects of the present application provide a control method, apparatus, electronic device, and readable storage medium for a cloud phone, so as to control a cloud phone running on a display device.

根据第一方面，提供了一种云手机的操控方法，包括：According to the first aspect, a method for controlling a cloud phone is provided, including:

输出显示设备上所运行的云手机应用中待操控应用的应用操控界面；Output and display the application control interface of the application to be controlled in the cloud mobile application running on the device;

响应于用户基于所述应用操控界面的操作，获取所在终端设备上的触控事件参数，所述触控事件参数包括所述终端设备上的触控事件的名称和该触控事件的坐标；In response to the user's operation based on the application control interface, obtain touch event parameters on the terminal device where the touch event parameters are located, where the touch event parameters include the name of the touch event on the terminal device and the coordinates of the touch event;

基于与所述云手机应用之间的通信连接，将所述终端设备上的触控事件参数发送给所述云手机应用，以供所述云手机应用根据所述终端设备上的触控事件参数、所述显示设备的屏幕分辨率和所述待操控应用的页面属性，获得所述待操控应用的触控事件参数，根据所述待操控应用的触控事件参数，对所述待操控应用进行操控，所述待操控应用的触控事件参数包括所述待操控应用的触控事件的名称和该触控事件的坐标。Based on the communication connection with the cloud mobile phone application, the touch event parameters on the terminal device are sent to the cloud mobile phone application for the cloud mobile phone application to use according to the touch event parameters on the terminal device , the screen resolution of the display device and the page attributes of the application to be controlled, obtain the touch event parameters of the application to be controlled, and perform the application to be controlled according to the touch event parameters of the application to be controlled. For manipulation, the touch event parameters of the application to be manipulated include the name of the touch event of the application to be manipulated and the coordinates of the touch event.

根据第二方面，提供了另一种云手机的操控方法，包括：According to the second aspect, another method for controlling a cloud phone is provided, including:

基于与虚拟手柄应用之间的通信连接，接收所述虚拟手柄应用发送的所述虚拟手柄应用所在终端设备上的触控事件参数，所述触控事件参数包括所述终端设备上的触控事件的名称和该触控事件的坐标；其中，所述终端设备上的触控事件参数为所述虚拟手柄应用输出显示设备上所运行的云手机应用中待操控应用的应用操控界面，响应于用户基于所述应用操控界面的操作获取的；Based on the communication connection with the virtual controller application, receive touch event parameters on the terminal device where the virtual controller application is located and sent by the virtual controller application, where the touch event parameters include touch events on the terminal device The name of the touch event and the coordinates of the touch event; wherein, the touch event parameter on the terminal device is the application control interface of the application to be controlled in the cloud mobile phone application running on the virtual handle application output display device, in response to the user Obtained based on the operation of the application control interface;

根据所述终端设备上的触控事件参数、所述显示设备的屏幕分辨率和所述待操控应用的页面属性，获得所述待操控应用的触控事件参数，所述待操控应用的触控事件参数包括所述待操控应用的触控事件的名称和该触控事件的坐标；According to the touch event parameters on the terminal device, the screen resolution of the display device, and the page attributes of the application to be controlled, the touch event parameters of the application to be controlled are obtained, and the touch event parameters of the application to be controlled are obtained. The event parameters include the name of the touch event of the application to be controlled and the coordinates of the touch event;

根据所述待操控应用的触控事件参数，对所述待操控应用进行操控。The application to be controlled is controlled according to the touch event parameters of the application to be controlled.

根据第三方面，提供了一种云手机的操控装置，包括：According to a third aspect, a control device for a cloud phone is provided, including:

输出单元，用于输出显示设备上所运行的云手机应用中待操控应用的应用操控界面；The output unit is used to output the application control interface of the application to be controlled in the cloud mobile application running on the display device;

获取单元，用于响应于用户基于所述应用操控界面的操作，获取所在终端设备上的触控事件参数，所述触控事件参数包括所述终端设备上的触控事件的名称和该触控事件的坐标；an obtaining unit, configured to obtain touch event parameters on the terminal device where the user operates based on the application control interface, where the touch event parameters include the name of the touch event on the terminal device and the touch event the coordinates of the event;

发送单元，用于基于与所述云手机应用之间的通信连接，将所述终端设备上的触控事件参数发送给所述云手机应用，以供所述云手机应用根据所述终端设备上的触控事件参数、所述显示设备的屏幕分辨率和所述待操控应用的页面属性，获得所述待操控应用的触控事件参数，根据所述待操控应用的触控事件参数，对所述待操控应用进行操控，所述待操控应用的触控事件参数包括所述待操控应用的触控事件的名称和该触控事件的坐标。a sending unit, configured to send the touch event parameters on the terminal device to the cloud mobile phone application based on the communication connection with the cloud mobile phone application, so that the cloud mobile phone application can The touch event parameters of the application to be controlled, the screen resolution of the display device and the page attributes of the application to be controlled are obtained, the touch event parameters of the application to be controlled are obtained, and the touch event parameters of the application to be controlled are obtained according to the touch event parameters of the application to be controlled. The application to be controlled performs the control, and the touch event parameters of the application to be controlled include the name of the touch event of the application to be controlled and the coordinates of the touch event.

根据第四方面，提供了另一种云手机的操控装置，包括：According to a fourth aspect, another control device for a cloud phone is provided, including:

接收单元，用于基于与虚拟手柄应用之间的通信连接，接收所述虚拟手柄应用发送的所述虚拟手柄应用所在终端设备上的触控事件参数，所述触控事件参数包括所述终端设备上的触控事件的名称和该触控事件的坐标；其中，所述终端设备上的触控事件参数为所述虚拟手柄应用输出显示设备上所运行的云手机应用中待操控应用的应用操控界面，响应于用户基于所述应用操控界面的操作获取的；a receiving unit, configured to receive, based on the communication connection with the virtual handle application, the touch event parameters sent by the virtual handle application on the terminal device where the virtual handle application is located, where the touch event parameters include the terminal device The name of the touch event on the terminal device and the coordinates of the touch event; wherein, the touch event parameter on the terminal device is the application control of the application to be controlled in the cloud phone application running on the virtual controller application output display device an interface, obtained in response to a user's operation based on the application manipulation interface;

映射单元，用于根据所述终端设备上的触控事件参数、所述显示设备的屏幕分辨率和所述待操控应用的页面属性，获得所述待操控应用的触控事件参数，所述待操控应用的触控事件参数包括所述待操控应用的触控事件的名称和该触控事件的坐标；a mapping unit, configured to obtain the touch event parameters of the application to be controlled according to the touch event parameters on the terminal device, the screen resolution of the display device and the page attributes of the application to be controlled, the The touch event parameters of the manipulation application include the name of the touch event of the application to be manipulated and the coordinates of the touch event;

响应单元，用于根据所述待操控应用的触控事件参数，对所述待操控应用进行操控。A response unit, configured to control the application to be controlled according to touch event parameters of the application to be controlled.

根据第五方面，提供一种电子设备，包括：According to a fifth aspect, an electronic device is provided, comprising:

至少一个处理器；以及at least one processor; and

与所述至少一个处理器通信连接的存储器；其中，a memory communicatively coupled to the at least one processor; wherein,

所述存储器存储有可被所述至少一个处理器执行的指令，所述指令被所述至少一个处理器执行，以使所述至少一个处理器能够执行如上所述的方面和任一可能的实现方式的方法。the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the aspects and any possible implementations described above way method.

根据第六方面，提供一种存储有计算机指令的非瞬时计算机可读存储介质，所述计算机指令用于使所述计算机执行如上所述的方面和任一可能的实现方式的方法。According to a sixth aspect, there is provided a non-transitory computer-readable storage medium storing computer instructions for causing the computer to perform the method of the aspect and any possible implementation as described above.

由上述技术方案可知，本申请实施例通过虚拟手柄应用输出显示设备上所运行的云手机应用中待操控应用的应用操控界面，进而，由虚拟手柄应用响应于用户基于所述应用操控界面的操作，获取所在终端设备上的触控事件参数，使得虚拟手柄应用能够基于与所述云手机应用之间的通信连接，将所述终端设备上的触控事件参数发送给所述云手机应用，这样，所述云手机应用就可以根据所述终端设备上的触控事件参数、所述显示设备的屏幕分辨率和所述待操控应用的页面属性，获得所述待操控应用的触控事件参数，根据所述待操控应用的触控事件参数，对所述待操控应用进行操控，从而实现了对运行在显示设备上的云手机进行操控。It can be seen from the above technical solutions that in the embodiment of the present application, the application control interface of the application to be controlled in the cloud mobile phone application running on the display device is output through the virtual handle application, and further, the virtual handle application responds to the user's operation based on the application control interface. , obtain the touch event parameters on the terminal device, so that the virtual controller application can send the touch event parameters on the terminal device to the cloud mobile phone application based on the communication connection with the cloud mobile phone application, so that , the cloud phone application can obtain the touch event parameters of the application to be controlled according to the touch event parameters on the terminal device, the screen resolution of the display device, and the page attributes of the application to be controlled, The application to be controlled is controlled according to the touch event parameters of the application to be controlled, so that the cloud mobile phone running on the display device is controlled.

另外，采用本申请所提供的技术方案，不再依赖显示设备是否具备触控功能，通过将具有触控功能的终端设备作为显示设备上所运行的云手机的触控输入设备，从而实现了对运行在显示设备上的云手机进行操控。同时，还能够有效提高云手机运行的可靠性和广泛性。In addition, using the technical solution provided by the present application, it no longer depends on whether the display device has a touch function, and by using the terminal device with touch function as the touch input device of the cloud mobile phone running on the display device, the It is controlled by the cloud mobile phone running on the display device. At the same time, it can also effectively improve the reliability and extensiveness of cloud mobile phone operation.

另外，采用本申请所提供的技术方案，能够有效地提高用户的体验。In addition, by adopting the technical solutions provided in the present application, the user experience can be effectively improved.

应当理解，本部分所描述的内容并非旨在标识本公开的实施例的关键或重要特征，也不用于限制本公开的范围。本公开的其它特征将通过以下的说明书而变得容易理解。It should be understood that what is described in this section is not intended to identify key or critical features of embodiments of the disclosure, nor is it intended to limit the scope of the disclosure. Other features of the present disclosure will become readily understood from the following description.

附图说明Description of drawings

为了更清楚地说明本申请实施例中的技术方案，下面将对实施例或现有技术描述中所需要使用的附图作一简单地介绍，显而易见地，下面描述中的附图是本申请的一些实施例，对于本领域普通技术人员来讲，在不付出创造性劳动性的前提下，还可以根据这些附图获得其他的附图。附图仅仅用于更好地理解本方案，不构成对本申请的限定。其中：In order to illustrate the technical solutions in the embodiments of the present application more clearly, the following briefly introduces the accompanying drawings that are required in the description of the embodiments or the prior art. Obviously, the drawings in the following description are from the present application. In some embodiments, for those of ordinary skill in the art, other drawings can also be obtained according to these drawings without any creative effort. The accompanying drawings are only for a better understanding of the present solution, and do not constitute a limitation to the present application. in:

图1A是根据本申请第一实施例的示意图；FIG. 1A is a schematic diagram according to a first embodiment of the present application;

图1B是本申请第一实施例中终端设备上的触控事件参数的可视化示意图；1B is a schematic diagram of a visualization of touch event parameters on a terminal device in the first embodiment of the present application;

图1C是本申请第一实施例中虚拟手柄应用所在的终端设备上的触控事件的坐标与云手机应用中所述待操控应用的触控事件的坐标之间的一映射示意图；1C is a schematic diagram of a mapping between the coordinates of the touch event on the terminal device where the virtual handle application is located in the first embodiment of the present application and the coordinates of the touch event of the application to be controlled in the cloud phone application;

图1D是本申请第一实施例中虚拟手柄应用所在的终端设备上的触控事件的坐标与云手机应用中所述待操控应用的触控事件的坐标之间的另一映射示意图；1D is another schematic diagram of mapping between the coordinates of the touch event on the terminal device where the virtual handle application is located in the first embodiment of the present application and the coordinates of the touch event of the application to be controlled in the cloud phone application;

图2是根据本申请第二实施例的示意图；2 is a schematic diagram according to a second embodiment of the present application;

图3是根据本申请第三实施例的示意图；3 is a schematic diagram according to a third embodiment of the present application;

图4是根据本申请第四实施例的示意图；4 is a schematic diagram according to a fourth embodiment of the present application;

图5是用来实现本申请实施例的云手机的操控方法的电子设备的示意图。FIG. 5 is a schematic diagram of an electronic device used to implement the control method for a cloud phone according to an embodiment of the present application.

具体实施方式Detailed ways

以下结合附图对本申请的示范性实施例做出说明，其中包括本申请实施例的各种细节以助于理解，应当将它们认为仅仅是示范性的。因此，本领域普通技术人员应当认识到，可以对这里描述的实施例做出各种改变和修改，而不会背离本申请的范围和精神。同样，为了清楚和简明，以下的描述中省略了对公知功能和结构的描述。Exemplary embodiments of the present application are described below with reference to the accompanying drawings, which include various details of the embodiments of the present application to facilitate understanding, and should be considered as exemplary only. Accordingly, those of ordinary skill in the art will recognize that various changes and modifications of the embodiments described herein can be made without departing from the scope and spirit of the present application. Also, descriptions of well-known functions and constructions are omitted from the following description for clarity and conciseness.

显然，所描述的实施例是本申请一部分实施例，而不是全部的实施例。基于本申请中的实施例，本领域普通技术人员在没有作出创造性劳动前提下所获得的全部其他实施例，都属于本申请保护的范围。Obviously, the described embodiments are some, but not all, embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by persons of ordinary skill in the art without creative efforts shall fall within the protection scope of the present application.

需要说明的是，本申请实施例中所涉及的终端设备可以包括但不限于手机、个人数字助理(PersonalDigitalAssistant，PDA)、无线手持设备、平板电脑(TabletComputer)等智能设备；显示设备可以包括但不限于个人电脑、电视等具有显示功能的设备。It should be noted that the terminal devices involved in the embodiments of the present application may include but are not limited to smart devices such as mobile phones, personal digital assistants (Personal Digital Assistants, PDAs), wireless handheld devices, and tablet computers (Tablet Computers); display devices may include but are not limited to smart devices. It is limited to devices with display functions such as personal computers and TVs.

另外，本文中术语“和/或”，仅仅是一种描述关联对象的关联关系，表示可以存在三种关系，例如，A和/或B，可以表示：单独存在A，同时存在A和B，单独存在B这三种情况。另外，本文中字符“/”，一般表示前后关联对象是一种“或”的关系。In addition, the term "and/or" in this article is only an association relationship to describe the associated objects, indicating that there can be three kinds of relationships, for example, A and/or B, it can mean that A exists alone, A and B exist at the same time, There are three cases of B alone. In addition, the character "/" in this document generally indicates that the related objects are an "or" relationship.

所谓的云手机就是指一种搭建在云服务器上的虚拟手机，云手机平台可以让使用者通过电脑、手机等操作终端来操控在云服务器上的虚拟云手机，这些虚拟云手机的功能与真正的手机基本相同，一般不包含基带模块没有电话卡功能。The so-called cloud mobile phone refers to a virtual mobile phone built on a cloud server. The cloud mobile phone platform allows users to control the virtual cloud mobile phone on the cloud server through operating terminals such as computers and mobile phones. The mobile phone is basically the same, generally does not contain a baseband module and no calling card function.

通常，云手机中的大部分应用都是基于触控屏开发的，而一般的显示设备却不具备触控功能，使得无法直接通过显示设备的屏幕操控运行在其上的云手机中的应用。Usually, most of the applications in the cloud phone are developed based on the touch screen, while the general display device does not have the touch function, making it impossible to directly control the applications running on the cloud phone through the screen of the display device.

本申请基于上述问题，提出一种云手机的操控方法，能够实现对运行在显示设备尤其是不具备触控功能的显示设备上的云手机进行操控。Based on the above problems, the present application proposes a control method for a cloud phone, which can realize the control of a cloud phone running on a display device, especially a display device without a touch function.

图1A是根据本申请第一实施例的示意图，如图1A所示。FIG. 1A is a schematic diagram according to a first embodiment of the present application, as shown in FIG. 1A .

101、输出显示设备上所运行的云手机应用中待操控应用的应用操控界面。101. Output and display an application control interface of the application to be controlled in the cloud mobile phone application running on the display device.

102、响应于用户基于所述应用操控界面的操作，获取所在终端设备上的触控事件参数。102. Acquire touch event parameters on the terminal device where the user is located in response to the user's operation based on the application manipulation interface.

其中，所述触控事件参数可以包括但不限于所述终端设备上的触控事件的名称和该触控事件的坐标，本实施例对此不进行特别限定。The touch event parameters may include, but are not limited to, the name of the touch event on the terminal device and the coordinates of the touch event, which are not particularly limited in this embodiment.

所述触控事件可以包括但不限于按下(Down)事件、离开(Up)事件和移动(Move)事件，用户的每个触控操作都可以由一个或多个触控事件组成。The touch events may include, but are not limited to, a down (Down) event, a leave (Up) event, and a move (Move) event, and each touch operation of the user may be composed of one or more touch events.

具体地，终端设备上的触控事件的坐标可以为绝对坐标，或者还可以为相对坐标，本实施例中不进行特别限定。Specifically, the coordinates of the touch event on the terminal device may be absolute coordinates, or may also be relative coordinates, which are not particularly limited in this embodiment.

本申请中，为了后续计算的简便，终端设备上的触控事件的坐标可以采用百分比形式的相对坐标，例如，终端设备处于竖直放置状态，将终端设备的屏幕的右上角的坐标设置为(0,0)，那么，终端设备的屏幕的右下角的坐标则可以记为(1,0)，终端设备的屏幕的左下角的坐标则可以记为(1,1)，终端设备的屏幕的左上角的坐标则可以记为(0,1)，例如，终端设备上的Down事件的坐标为(0.5,0.8)等。In this application, for the convenience of subsequent calculations, the coordinates of the touch event on the terminal device may be relative coordinates in the form of percentages. For example, if the terminal device is placed vertically, the coordinates of the upper right corner of the screen of the terminal device are set to ( 0,0), then the coordinates of the lower right corner of the screen of the terminal device can be recorded as (1,0), the coordinates of the lower left corner of the screen of the terminal device can be recorded as (1,1), and the coordinates of the screen of the terminal device can be recorded as (1,1). The coordinates of the upper left corner can be recorded as (0, 1), for example, the coordinates of the Down event on the terminal device are (0.5, 0.8) and so on.

103、基于与所述云手机应用之间的通信连接，将所述终端设备上的触控事件参数发送给所述云手机应用，以供所述云手机应用根据所述终端设备上的触控事件参数、所述显示设备的屏幕分辨率和所述待操控应用的页面属性，获得所述待操控应用的触控事件参数，根据所述待操控应用的触控事件参数，对所述待操控应用进行操控。103. Based on the communication connection with the cloud mobile phone application, send the touch event parameters on the terminal device to the cloud mobile phone application, so that the cloud mobile phone application can be used according to the touch control on the terminal device. event parameters, the screen resolution of the display device and the page attributes of the application to be manipulated, obtain the touch event parameters of the application to be manipulated, and according to the touch event parameters of the application to be manipulated, application to control.

其中，所述待操控应用的触控事件参数可以包括但不限于所述待操控应用的触控事件的名称和该触控事件的坐标，本实施例对此不进行特别限定。The touch event parameters of the application to be controlled may include, but are not limited to, the name of the touch event of the application to be controlled and the coordinates of the touch event, which are not particularly limited in this embodiment.

需要说明的是，101～103的执行主体的部分或全部可以为位于终端设备的虚拟手柄应用，或者还可以为设置在位于终端设备的虚拟手柄应用中的插件或SDK(SoftwareDevelopmentKit，软件开发工具包)等功能单元，本实施例对此不进行特别限定。It should be noted that some or all of the execution bodies of 101 to 103 may be a virtual handle application located in the terminal device, or may also be a plug-in or SDK (Software Development Kit, software development kit) set in the virtual handle application located in the terminal device ) and other functional units, which are not particularly limited in this embodiment.

可以理解的是，所述虚拟手柄应用可以是安装在终端设备上的本地程序(nativeApp)，或者还可以是终端设备上的浏览器的一个网页程序(webApp)，本实施例对此不进行限定。It can be understood that the virtual handle application may be a local program (nativeApp) installed on the terminal device, or may also be a web page program (webApp) of a browser on the terminal device, which is not limited in this embodiment. .

本申请中，所采用的显示设备可以为具备触控功能的显示设备，或者还可以为不具备的显示设备，本实施例对此不进行特别限定。In this application, the display device used may be a display device with a touch function, or may also be a display device without a touch function, which is not particularly limited in this embodiment.

本申请中，采用显示设备运行云手机应用，通过云手机应用与该云手机应用所对应的云服务器，实现云端运行的云手机。显示设备上所运行的云手机应用与该云手机应用所对应的云服务器之间，可以采用现有的云手机相关技术，通过网络进行信息交互。In this application, a display device is used to run a cloud mobile phone application, and a cloud mobile phone running in the cloud is realized through the cloud mobile phone application and the cloud server corresponding to the cloud mobile phone application. Between the cloud mobile phone application running on the display device and the cloud server corresponding to the cloud mobile phone application, the existing cloud mobile phone related technology can be used to exchange information through the network.

本申请中，在显示设备上所运行的云手机应用输出该云手机应用的应用界面之后，虚拟手柄应用与云手机应用可以采用多种连接方式，建立虚拟手柄应用与云手机应用之间的通信连接，可以为有线通信连接，或者还可以为无线通信连接，例如，蓝牙连接、红外连接、无线相容性认证(WirelessFidelity，WiFi)连接等无线通信连接等，本实施例对此不进行特别限定。In this application, after the cloud mobile phone application running on the display device outputs the application interface of the cloud mobile phone application, the virtual controller application and the cloud mobile phone application can adopt various connection methods to establish communication between the virtual controller application and the cloud mobile phone application The connection may be a wired communication connection, or may also be a wireless communication connection, for example, a wireless communication connection such as a Bluetooth connection, an infrared connection, a wireless compatibility authentication (Wireless Fidelity, WiFi) connection, etc., which is not particularly limited in this embodiment. .

以WiFi连接为例，假设运行云手机应用的显示设备记为A，运行虚拟手柄应用的终端设备记为B，举例说明A与B如何建立WiFi连接。Taking the WiFi connection as an example, suppose that the display device running the cloud phone application is marked as A, and the terminal device running the virtual handle application is marked as B, and an example is given to illustrate how A and B establish a WiFi connection.

方式1：A与B都在一个WiFi路由器下面，A启动的时候，通过输出二维码传递A的IP地址，B对二维码进行扫码，建立Socket连接，以连接到A。Method 1: Both A and B are under a WiFi router. When A starts, it transmits A's IP address by outputting a QR code, and B scans the QR code to establish a socket connection to connect to A.

方式2：A与B不在同一个WiFi路由器下面，A启动的时候，开启WiFi点对点(p2p)协议，启动WiFi点对点网络，B直接搜索WiFi点对点网络找到A，以连接到A。Method 2: A and B are not under the same WiFi router. When A starts, enable the WiFi peer-to-peer (p2p) protocol, start the WiFi peer-to-peer network, and B directly searches the WiFi peer-to-peer network to find A to connect to A.

方式3：A通过有线连接方式，连接到网络，A启动的时候，开启WiFi热点，通过输出二维码传递WiFi热点的名称和密码，B对二维码进行扫码，以连接到A的热点，进而连接到A。Method 3: A connects to the network through a wired connection. When A starts, it turns on the WiFi hotspot, and transmits the name and password of the WiFi hotspot by outputting a QR code. B scans the QR code to connect to A's hotspot. , which in turn connects to A.

在完成A与B之间的设备连接建立之后，则就建立了A上运行的云手机应用与B上运行的虚拟手柄应用之间的通信连接。After the establishment of the device connection between A and B is completed, the communication connection between the cloud phone application running on A and the virtual handle application running on B is established.

至此，云手机应用与虚拟手柄应用之间则可以基于所建立的通信连接进行信息交互。So far, the cloud phone application and the virtual handle application can exchange information based on the established communication connection.

可选地，在本实施例的一个可能的实现方式中，在101之前，还可以进一步确定所述云手机应用中所运行的待操控应用，进而，则可以根据所述待操控应用，获得所述应用操控界面。Optionally, in a possible implementation manner of this embodiment, beforestep 101, an application to be controlled running in the cloud mobile phone application may be further determined, and then, according to the application to be controlled, obtained The application control interface.

在一个具体的实现过程中，虚拟手柄应用具体可以获取所述云手机应用所提供的所述待操控应用的确定信息，进而，则可以根据所述待操控应用的确定信息，确定所述云手机应用中所运行的待操控应用。In a specific implementation process, the virtual controller application may specifically acquire the determination information of the application to be controlled provided by the cloud mobile phone application, and then, according to the determination information of the application to be controlled, determine the cloud mobile phone The application to be controlled running in the application.

在该实现过程中，虚拟手柄应用具体可以采用各种方式，获取所述云手机应用所提供的所述待操控应用的确定信息。In this implementation process, the virtual handle application may specifically acquire the determination information of the application to be controlled provided by the cloud mobile phone application in various ways.

例如，虚拟手柄应用具体可以对所述云手机应用所输出的二维码进行扫描，以获得所述云手机应用中当前所运行的待操控应用的确定信息，例如，应用名称等应用标识。For example, the virtual controller application may specifically scan the QR code output by the cloud mobile phone application to obtain the determination information of the application to be controlled currently running in the cloud mobile phone application, for example, the application identifier such as the application name.

或者，再例如，虚拟手柄应用具体可以基于所建立的通信连接，获取所述云手机应用所输出的该云手机应用中当前所运行的待操控应用的确定信息，例如，应用名称等应用标识。Or, for another example, the virtual handle application may specifically obtain the determination information of the application to be controlled currently running in the cloud mobile phone application output by the cloud mobile phone application based on the established communication connection, for example, the application identifier such as the application name.

此时，虚拟手柄应用通过确定云手机应用中所运行的待操控应用，进而，则可以根据所述待操控应用，获得所述应用操控界面，使得终端设备上所运行的虚拟手柄应用能够获得显示设备上所运行的云手机应用中待操控应用的应用操控界面，从而可以同步输出显示设备上所运行的云手机应用中待操控应用的应用操控界面。At this time, the virtual handle application determines the application to be controlled running in the cloud mobile phone application, and then, according to the application to be controlled, the application control interface can be obtained, so that the virtual handle application running on the terminal device can be displayed. The application control interface of the application to be controlled in the cloud mobile application running on the device can be synchronously output and displayed to display the application control interface of the application to be controlled in the cloud mobile application running on the device.

可选地，在本实施例的一个可能的实现方式中，在102之前，还可以进一步获取用户基于所述应用操控界面的操作。其中，该操作可以包括但不限于操作手势，所述操作手势可以包括但不限于下列操作手势中的至少一项：Optionally, in a possible implementation manner of this embodiment, before 102 , an operation of the user based on the application manipulation interface may be further acquired. The operation may include, but is not limited to, an operation gesture, and the operation gesture may include, but is not limited to, at least one of the following operation gestures:

用户对所述应用操控界面所对应的操作控件的操作手势；The operation gesture of the user on the operation control corresponding to the application control interface;

用户在所述应用操控界面上方的悬空操作手势；The user's hovering gesture above the application control interface;

用户在所述应用操控界面上的接触操作手势；以及The user's contact operation gesture on the application control interface; and

用户基于所述应用操控界面带动所述终端设备的运动趋势。The user drives the movement trend of the terminal device based on the application control interface.

所谓的用户对所述应用操控界面所对应的操作控件的操作手势，可以是指用户操作如点击操作终端设备的屏幕所显示的应用操控界面上的确认操作控件。所谓的操作控件，是用户可与之交互以输入或操作数据的对象，可以由一个或多个页面元素构成。其中，所谓的点击操作这一操作，可以为用户利用鼠标或键盘等外部输入设备控制光标所进行的触发操作，或者还可以为用户利用手指或输入笔等触控输入设备所进行的触碰操作，本实施例对此不进行特别限定。The so-called operation gesture of the user on the operation control corresponding to the application control interface may refer to the user's operation such as clicking the confirmation operation control on the application control interface displayed on the screen of the operation terminal device. The so-called operation controls are objects that users can interact with to input or manipulate data, and can be composed of one or more page elements. The so-called click operation may be a trigger operation performed by the user using an external input device such as a mouse or a keyboard to control the cursor, or may also be a touch operation performed by the user using a touch input device such as a finger or an input pen , which is not particularly limited in this embodiment.

例如，用户具体可以在应用操控界面所对应的方向操作控件上进行点击操作，在获取到用户对应用操控界面所对应的方向操作控件的点击操作手势之后，则可以获取所在终端设备上的触控事件参数。For example, the user can specifically perform a click operation on the direction operation control corresponding to the application control interface. After obtaining the user's click operation gesture on the direction operation control corresponding to the application control interface, the user can obtain the touch operation on the terminal device. Event parameters.

所谓的用户在所述应用操控界面上方的悬空操作手势，可以是指用户在终端设备的图像传感器的采集范围之内，在终端设备的屏幕所显示的应用操控界面上方的悬空滑动轨迹。其中，所述图像传感器可以为电荷耦合元件(ChargeCoupledDevice，CCD)传感器，或者还可以为金属氧化物半导体元件(ComplementaryMetal-OxideSemiconductor，CMOS)传感器，本实施例对此不进行特别限定。所述悬空滑动轨迹可以包括但不限于由若干个连续滑动事件对应的若干个停留点组成的直线或任意形状的曲线，本实施例对此不进行特别限定。The so-called hovering operation gesture of the user above the application control interface may refer to the hovering sliding trajectory of the user above the application control interface displayed on the screen of the terminal device within the acquisition range of the image sensor of the terminal device. The image sensor may be a charge coupled device (Charge Coupled Device, CCD) sensor, or may also be a metal oxide semiconductor (Complementary Metal-Oxide Semiconductor, CMOS) sensor, which is not particularly limited in this embodiment. The suspended sliding trajectory may include, but is not limited to, a straight line or a curve of any shape composed of a plurality of staying points corresponding to a plurality of continuous sliding events, which is not particularly limited in this embodiment.

所谓的用户在所述应用操控界面上的接触操作手势，可以是指用户在终端设备的屏幕所显示的应用操控界面上的接触滑动轨迹。通常，终端设备可以根据屏幕是否具有可触控的特性，分成两种类型，一种类型是触控式终端设备，另一种类型是非触控式终端设备。具体地，具体可以检测用户在触控式终端设备的触控屏所显示的应用操控界面上的接触滑动轨迹。所述接触滑动轨迹可以包括但不限于由若干个连续触摸事件对应的若干个触摸点组成的直线或任意形状的曲线，本实施例对此不进行特别限定。具体来说，具体可以为用户在应用操控界面上的任意区域的点击操作手势，或者还可以为用户在应用操控界面上的任意区域的长按操作手势，或者还可以为用户在应用操控界面上的任意区域的滑动操作手势，本实施例对此不进行特别限定。The so-called contact operation gesture of the user on the application control interface may refer to the contact sliding track of the user on the application control interface displayed on the screen of the terminal device. Generally, terminal devices can be divided into two types according to whether the screen has touch-ability characteristics, one type is a touch-type terminal device, and the other type is a non-touch-type terminal device. Specifically, the user's contact sliding track on the application control interface displayed on the touch screen of the touch terminal device can be detected. The contact sliding track may include, but is not limited to, a straight line or a curve of any shape composed of several touch points corresponding to several consecutive touch events, which is not particularly limited in this embodiment. Specifically, it can be the user's click operation gesture in any area on the application control interface, or it can also be the user's long-press operation gesture in any area on the application control interface, or it can also be the user's operation gesture on the application control interface. The sliding operation gesture in any area of is not particularly limited in this embodiment.

例如，用户具体可以在应用操控界面上的任意区域进行单击操作，在获取到用户对应用操控界面上的任意区域的单击操作手势之后，则可以获取所在终端设备上的触控事件参数。For example, the user can specifically perform a click operation on any area on the application control interface, and after obtaining the user's click operation gesture on any area on the application control interface, the touch event parameters on the terminal device where it is located can be obtained.

所谓的用户基于所述应用操控界面带动所述终端设备的运动趋势，可以是指用户手持终端设备，在终端设备的屏幕显示应用操控界面时，带动终端设备所进行运动的运动轨迹，例如，晃动、翻转等。The so-called movement trend that the user drives the terminal device based on the application control interface may refer to the movement trajectory of the terminal device that is driven by the user holding the terminal device when the application control interface is displayed on the screen of the terminal device, for example, shaking , flip, etc.

在一个具体的实现过程中，具体可以利用传感器设备，检测所述用户基于所述应用操控界面的操作手势。具体地，所述传感器设备可以包括但不限于重力传感器、加速度传感器、压力传感器、红外线传感器、距离传感器和图像传感器中的至少一个，本实施例对此不进行特别限定。In a specific implementation process, a sensor device may be specifically used to detect the operation gesture of the user based on the application manipulation interface. Specifically, the sensor device may include, but is not limited to, at least one of a gravity sensor, an acceleration sensor, a pressure sensor, an infrared sensor, a distance sensor, and an image sensor, which is not particularly limited in this embodiment.

其中，所述距离传感器可以为超声波距离传感器，或者还可以为红外距离传感器，或者还可以为激光距离传感器，或者还可以为微波距离传感器，本实施例对此不进行特别限定。这些距离传感器都是现有的成熟技术，详细描述可以参见现有技术中的相关内容，此处不再赘述。The distance sensor may be an ultrasonic distance sensor, or an infrared distance sensor, or a laser distance sensor, or a microwave distance sensor, which is not particularly limited in this embodiment. These distance sensors are all existing mature technologies. For detailed description, please refer to the relevant content in the prior art, which will not be repeated here.

其中，所述图像传感器可以为电荷耦合元件(ChargeCoupledDevice，CCD)传感器，或者还可以为金属氧化物半导体元件(ComplementaryMetal-OxideSemiconductor，CMOS)传感器，本实施例对此不进行特别限定。The image sensor may be a charge coupled device (Charge Coupled Device, CCD) sensor, or may also be a metal oxide semiconductor (Complementary Metal-Oxide Semiconductor, CMOS) sensor, which is not particularly limited in this embodiment.

在该实现过程中，所谓的检测所述用户基于所述应用操控界面的操作手势，具体可以是指检测所述用户基于所述应用操控界面的操作手势的起始点、结束点以及由起始点到结束点所形成的轨迹，或者还可以进一步检测所述轨迹所对应的弧度数据。In this implementation process, the so-called detection of the user's operation gesture based on the application control interface may specifically refer to detecting the start point, the end point, and the start point and the end point of the user's operation gesture based on the application control interface. The trajectory formed by the end point, or the radian data corresponding to the trajectory can be further detected.

为了实现上述功能，可选地，在本实施例的一个可能的实现方式中，在获取所述用户基于所述应用操控界面的第一操作之前，还可以进一步预先设置指定操作。只有当所获取的操作满足预先设置的指定操作时，才能够执行获取所在终端设备上的触控事件参数的步骤。In order to implement the above functions, optionally, in a possible implementation manner of this embodiment, before acquiring the first operation of the user based on the application manipulation interface, a specified operation may be further preset. Only when the acquired operation satisfies the preset specified operation, the step of acquiring the touch event parameters on the terminal device where it is located can be performed.

以所获取的操作包括操作手势为例，在获取到所述用户基于所述应用操控界面的操作手势之后，则可以将操作手势与预先设置的指定手势进行对比，只有当所获取的操作手势满足预先设置的指定手势时，才能够执行获取所在终端设备上的触控事件参数的步骤。Taking the obtained operation including operation gestures as an example, after obtaining the operation gestures of the user based on the application control interface, the operation gestures can be compared with the preset designated gestures, and only when the obtained operation gestures satisfy the preset operation gestures. Only when the specified gesture is set, the step of obtaining the touch event parameters on the terminal device can be performed.

具体来说，所述指定手势的数据可以存储在终端设备的存储设备中。Specifically, the data of the specified gesture may be stored in the storage device of the terminal device.

在一个具体的实现过程中，所述终端设备的存储设备可以慢速存储设备，具体可以为计算机系统的硬盘，或者还可以为手机的非运行内存即物理内存，例如，只读存储器(Read-OnlyMemory，ROM)和内存卡等，本实施例对此不进行特别限定。In a specific implementation process, the storage device of the terminal device may be a slow storage device, specifically a hard disk of a computer system, or may also be a non-running memory of a mobile phone, that is, a physical memory, for example, a read-only memory (Read-only memory). OnlyMemory, ROM) and a memory card, etc., which are not particularly limited in this embodiment.

在另一个具体的实现过程中，所述终端设备的存储设备还可以为快速存储设备，具体可以为计算机系统的内存，或者还可以为手机的运行内存即系统内存，例如，随机存储器(RandomAccessMemory，RAM)等，本实施例对此不进行特别限定。In another specific implementation process, the storage device of the terminal device may also be a fast storage device, specifically the memory of a computer system, or may also be the operating memory of the mobile phone, that is, the system memory, for example, random access memory (Random Access Memory, RAM), etc., which are not particularly limited in this embodiment.

在该实现方式中，响应于所述用户基于所述应用操控界面的操作，虚拟手柄应用还可以进一步对所述终端设备上的触控事件参数进行可视化处理，用以表示所述终端设备上的触控事件的触控位置，如图1B所示。在图1B中，在应用操控界面上示出了2个圆形图标，可以用以表示当前终端设备上的触控事件的触控位置，有几个圆形示意图标，则可以表示当前用户按了几个手指。In this implementation manner, in response to the user's operation based on the application control interface, the virtual handle application may further perform visual processing on the touch event parameters on the terminal device to represent the touch event parameters on the terminal device. The touch position of the touch event is shown in FIG. 1B . In FIG. 1B , two circular icons are shown on the application control interface, which can be used to represent the touch position of the touch event on the current terminal device. There are several circular icons, which can indicate that the current user presses a few fingers.

本实施例中，在虚拟手柄应用获得所述终端设备上的触控事件参数之后，则可以进一步将所述终端设备上的触控事件参数编码成特定的格式，例如，json格式或其他私有格式等，进而，则可以基于与所述云手机应用之间的通信连接，将编码之后的所述终端设备上的触控事件参数发送给云手机应用。In this embodiment, after the virtual handle application obtains the touch event parameters on the terminal device, the touch event parameters on the terminal device can be further encoded into a specific format, for example, json format or other private formats and so on, and further, based on the communication connection with the cloud mobile phone application, the encoded touch event parameters on the terminal device may be sent to the cloud mobile phone application.

在云手机应用基于云手机应用与虚拟手柄应用之间的通信连接，接收所述终端设备上的触控事件参数之后，所述云手机应用则可以先解码出所述终端设备上的触控事件参数。然后，所述云手机应用则可以根据所述终端设备上的触控事件的名称，获得所述待操控应用的触控事件的名称，再根据所述显示设备的屏幕分辨率和所述待操控应用的页面属性，将所述终端设备上的触控事件的坐标映射到所述显示设备的屏幕上，以获得所述待操控应用的触控事件的坐标，进而对所述待操控应用进行操控，从而实现了用户通过操作终端设备就能够操控显示设备上所运行的云手机应用的目的。After the cloud phone application receives the touch event parameters on the terminal device based on the communication connection between the cloud phone application and the virtual handle application, the cloud phone application can first decode the touch event on the terminal device parameter. Then, the cloud phone application can obtain the name of the touch event of the application to be controlled according to the name of the touch event on the terminal device, and then according to the screen resolution of the display device and the name of the to-be-controlled application The page attribute of the application, the coordinates of the touch event on the terminal device are mapped to the screen of the display device to obtain the coordinates of the touch event of the application to be manipulated, and then the application to be manipulated is manipulated , so that the user can control the cloud mobile phone application running on the display device by operating the terminal device.

由于虚拟手柄应用是基于终端设备的屏幕分辨率采集触控事件，而云手机应用是基于显示设备的屏幕分辨率来运行云手机的，两个屏幕分辨率通常是不一致，所以，云手机应用在解码出虚拟手柄应用所发送的所述终端设备上的触控事件参数之后，还需要进一步进行坐标转换。Since the virtual controller application collects touch events based on the screen resolution of the terminal device, and the cloud phone application runs the cloud phone based on the screen resolution of the display device, the two screen resolutions are usually inconsistent. After decoding the touch event parameters on the terminal device sent by the virtual handle application, further coordinate transformation needs to be performed.

由于终端设备可以通过任意旋转其屏幕而采用横屏方式或者竖屏方式，全屏运行虚拟手柄应用，因此，虚拟手柄应用的应用操控界面宽度和应用操控界面高度分别是所运行的终端设备的宽和高，分别是localWidth和localHeight；而显示设备则不能旋转其屏幕而只能采用固定屏幕运行云手机应用，因此，云手机应用中待操控应用的应用界面宽度和应用界面高度需要分别进行单独考虑，分别是screenWidth和screenHeight。Since the terminal device can run the virtual controller application in the full screen mode by arbitrarily rotating its screen in the horizontal or vertical screen mode, the width of the application control interface and the height of the application control interface of the virtual controller application are respectively the width and the height of the running terminal device. The height is localWidth and localHeight respectively; while the display device cannot rotate its screen and can only use a fixed screen to run the cloud mobile application. Therefore, the application interface width and application interface height of the application to be controlled in the cloud mobile application need to be considered separately. They are screenWidth and screenHeight respectively.

其中，应用界面高度就是云手机应用所运行的显示设备的高度，横屏应用的应用界面宽度就是云手机应用所运行的显示设备的宽度，竖屏应用的应用界面宽度就是云手机应用所运行的显示设备的宽度的中间一部分，例如，显示设备的宽度的中间三分之一。Among them, the height of the application interface is the height of the display device that the cloud mobile application runs on, the width of the application interface of the horizontal screen application is the width of the display device that the cloud mobile application runs, and the width of the application interface of the vertical screen application is the width of the cloud mobile application running. The middle portion of the width of the display device, eg, the middle third of the width of the display device.

虚拟手柄应用所运行的终端设备上的触控事件的坐标是xin和yin，对应的云手机应用中所述待操控应用的触控事件的坐标是xout和yout。为了计算简便，终端设备上的触控事件的坐标可以采用百分比形式的相对坐标，对应各设备在该方向的尺寸为1。The coordinates of the touch event on the terminal device on which the virtual handle application runs are xin and yin, and the coordinates of the touch event of the application to be controlled in the corresponding cloud phone application are xout and yout. In order to simplify the calculation, the coordinates of the touch events on the terminal device may be relative coordinates in the form of percentages, and the size corresponding to each device in this direction is 1.

如果云手机应用中所述待操控应用的页面属性是横屏页面，说明所述待操控应用是横屏应用，如图1C所示，那么，转换公式如下：If the page attribute of the application to be controlled in the cloud phone application is a landscape page, it means that the application to be controlled is a landscape application, as shown in FIG. 1C , then the conversion formula is as follows:

xout＝xin×screenWidth；xout=xin×screenWidth;

yout＝yin×screenHeight。yout=yin×screenHeight.

如果云手机应用中所述待操控应用的页面属性是竖屏页面，说明所述待操控应用是竖屏应用，如图1D所示，那么，转换公式如下：If the page attribute of the application to be controlled in the cloud phone application is a vertical screen page, it means that the application to be controlled is a vertical screen application, as shown in Figure 1D, then the conversion formula is as follows:

xout＝(1-yin)×screenWidth；xout=(1-yin)×screenWidth;

yout＝xin×screenHeight。yout=xin×screenHeight.

由于采用终端设备上的触控事件的相对坐标，使得虚拟手柄应用无需向云手机应用传递该终端设备的屏幕分辨率，云手机应用就能够根据终端设备上的触控事件的相对坐标和显示设备自身的屏幕分辨率，计算出所述云手机应用中待操控应用的触控事件的坐标，能够有效降低云手机运行的复杂度，同时能够进一步提高云手机运行的效率。Since the relative coordinates of the touch events on the terminal device are used, the virtual controller application does not need to transmit the screen resolution of the terminal device to the cloud mobile application, and the cloud mobile application can display the device according to the relative coordinates of the touch events on the terminal device. The coordinates of the touch event of the application to be controlled in the cloud mobile phone application can be calculated based on its own screen resolution, which can effectively reduce the operation complexity of the cloud mobile phone and further improve the operation efficiency of the cloud mobile phone.

可以理解的是，如果终端设备上的触控事件的坐标采用绝对位置形式的绝对坐标，那么，则可以采用相应的转换公式将终端设备上的触控事件的绝对坐标映射为待操控应用的应用界面的触控事件的坐标即可。It can be understood that if the coordinates of the touch event on the terminal device are absolute coordinates in the form of absolute positions, then the corresponding conversion formula can be used to map the absolute coordinates of the touch event on the terminal device to the application of the application to be controlled. The coordinates of the touch event of the interface are enough.

进一步地，在云手机应用获得所述待操控应用的触控事件参数之后，还可以进一步对所述待操控应用的触控事件参数进行可视化处理，在显示设备所输出的应用界面上示出所述待操控应用的触控事件的触控位置。Further, after the cloud mobile phone application obtains the touch event parameters of the application to be controlled, the touch event parameters of the application to be controlled may be further visualized, and displayed on the application interface output by the display device. Describes the touch position of the touch event of the application to be controlled.

这样，云手机应用通过对待操控应用的触控事件参数进行可视化处理，能够与虚拟手柄应用所输出的表示所述终端设备上的触控事件的触控位置一一对应，能够方便用户清楚获知该用户当前在所述云手机应用中待操控应用中的触控位置。In this way, the cloud phone application can visually process the touch event parameters of the application to be controlled, and can correspond one-to-one with the touch position representing the touch event on the terminal device output by the virtual handle application, which can facilitate the user to clearly know the touch event. The touch position in the application to be controlled by the user currently in the cloud phone application.

可选地，在本实施例的一个可能的实现方式中，所述终端设备上的触控事件参数还可以进一步包括所述终端设备的屏幕分辨率。Optionally, in a possible implementation manner of this embodiment, the touch event parameter on the terminal device may further include a screen resolution of the terminal device.

由于在不同屏幕分辨率所输出的应用操控界面上，界面布局可能不完全一致，此时，在所述云手机应用根据所述终端设备上的触控事件的名称、所述终端设备上的触控事件的坐标、所述显示设备的屏幕分辨率和所述待操控应用的页面属性，获得所述待操控应用的触控事件参数之后，还需要进一步根据所述终端设备的屏幕分辨率，对所获得的所述待操控应用的触控事件参数进行调整处理，能够适用于各种屏幕分辨率的终端设备，能够进一步有效提高云手机运行的可靠性和广泛性。Because the interface layout may not be completely consistent on the application control interface output by different screen resolutions, in this case, the cloud mobile phone application is based on the name of the touch event on the terminal device and the touch event on the terminal device. The coordinates of the control event, the screen resolution of the display device, and the page attributes of the application to be controlled are obtained. After obtaining the touch event parameters of the application to be controlled, it is necessary to further determine the screen resolution of the terminal device according to the screen resolution of the terminal device. The obtained touch event parameters of the application to be controlled can be adjusted and processed, which can be applied to terminal devices with various screen resolutions, and can further effectively improve the reliability and extensiveness of the operation of the cloud mobile phone.

采用本申请所提供的技术方案，通过虚拟手柄应用输出显示设备上所运行的云手机应用中待操控应用的应用操控界面，进而，由虚拟手柄应用响应于用户基于所述应用操控界面的操作，获取所在终端设备上的触控事件参数，使得虚拟手柄应用能够基于与所述云手机应用之间的通信连接，将所述终端设备上的触控事件参数发送给所述云手机应用，这样，所述云手机应用就可以根据所述终端设备上的触控事件参数、所述显示设备的屏幕分辨率和所述待操控应用的页面属性，获得所述待操控应用的触控事件参数，根据所述待操控应用的触控事件参数，对所述待操控应用进行操控，从而实现了对运行在显示设备上的云手机进行操控。该技术方案相当于把终端设备当作显示设备的触控输入设备来使用，所以可以适配所有的云手机中的应用，而且无需做额外的适配工作，因为云手机中的大部分应用都是基于触控屏开发的。Using the technical solution provided in this application, the application control interface of the application to be controlled in the cloud mobile phone application running on the display device is output through the virtual handle application, and then the virtual handle application responds to the user's operation based on the application control interface, Acquire the touch event parameters on the terminal device, so that the virtual handle application can send the touch event parameters on the terminal device to the cloud mobile phone application based on the communication connection with the cloud mobile phone application, so that, The cloud phone application can obtain the touch event parameters of the application to be controlled according to the touch event parameters on the terminal device, the screen resolution of the display device, and the page attributes of the application to be controlled, according to the The touch event parameter of the application to be controlled controls the application to be controlled, thereby realizing the control of the cloud phone running on the display device. This technical solution is equivalent to using the terminal device as the touch input device of the display device, so it can be adapted to all applications in the cloud phone, and no additional adaptation work is required, because most applications in the cloud phone are It is developed based on touch screen.

在电视或者电视盒子等不具备触控功能的显示设备上运行云手机应用时，就可以用手机等具备触控功能的终端设备上所运行的虚拟手柄应用来操控云手机应用中所运行的应用，而无需使用实体的蓝牙手柄，既节省了硬件成本，也增加了云手机适配应用的范围，不能适配蓝牙手柄的应用也可以正常应用。When running the cloud mobile phone application on a display device without touch function such as a TV or a TV box, the virtual controller application running on the terminal device with touch function such as a mobile phone can be used to control the application running in the cloud mobile phone application , instead of using a physical Bluetooth handle, which not only saves hardware costs, but also increases the scope of cloud phone adaptation applications, and applications that cannot be adapted to Bluetooth handles can also be used normally.

本实施例中，通过输出显示设备上所运行的云手机应用中待操控应用的应用操控界面，进而，响应于用户基于所述应用操控界面的操作，获取所在终端设备上的触控事件参数，使得能够基于与所述云手机应用之间的通信连接，将所述终端设备上的触控事件参数发送给所述云手机应用，以供所述云手机应用根据所述终端设备上的触控事件参数、所述显示设备的屏幕分辨率和所述待操控应用的页面属性，获得所述待操控应用的触控事件参数，根据所述待操控应用的触控事件参数，对所述待操控应用进行操控，从而实现了对运行在显示设备上的云手机进行操控。In this embodiment, by outputting the application control interface of the application to be controlled in the cloud mobile phone application running on the display device, and further, in response to the user's operation based on the application control interface, the touch event parameters on the terminal device where it is located are obtained, It enables to send touch event parameters on the terminal device to the cloud mobile application based on the communication connection with the cloud mobile application, so that the cloud mobile application can use the touch control on the terminal device according to the event parameters, the screen resolution of the display device and the page attributes of the application to be manipulated, obtain the touch event parameters of the application to be manipulated, and according to the touch event parameters of the application to be manipulated, The application is controlled, thereby realizing the control of the cloud mobile phone running on the display device.

图2是根据本申请第二实施例的示意图，如图2所示。FIG. 2 is a schematic diagram according to a second embodiment of the present application, as shown in FIG. 2 .

201、基于与虚拟手柄应用之间的通信连接，接收所述虚拟手柄应用发送的所述虚拟手柄应用所在终端设备上的触控事件参数，所述终端设备上的触控事件参数为所述虚拟手柄应用输出显示设备上所运行的云手机应用中待操控应用的应用操控界面，响应于用户基于所述应用操控界面的操作获取的。201. Based on the communication connection with the virtual handle application, receive a touch event parameter on the terminal device where the virtual handle application is located and sent by the virtual handle application, where the touch event parameter on the terminal device is the virtual handle The handle application output displays the application control interface of the application to be controlled in the cloud mobile phone application running on the device, and is obtained in response to the user's operation based on the application control interface.

202、根据所述终端设备上的触控事件参数、所述显示设备的屏幕分辨率和所述待操控应用的页面属性，获得所述待操控应用的触控事件参数。202. Obtain touch event parameters of the application to be manipulated according to the touch event parameters on the terminal device, the screen resolution of the display device, and the page attribute of the application to be manipulated.

203、根据所述待操控应用的触控事件参数，对所述待操控应用进行操控。203. Control the application to be controlled according to the touch event parameter of the application to be controlled.

需要说明的是，201～203的执行主体的部分或全部可以为位于显示设备的云手机应用，或者还可以为设置在位于显示设备的云手机应用中的插件或SDK(SoftwareDevelopmentKit，软件开发工具包)等功能单元，本实施例对此不进行特别限定。It should be noted that some or all of the execution subjects of 201 to 203 may be a cloud mobile phone application located on the display device, or may also be a plug-in or SDK (Software Development Kit, software development kit) set in the cloud mobile phone application located on the display device. ) and other functional units, which are not particularly limited in this embodiment.

可以理解的是，所述云手机应用可以是安装在显示设备上的本地程序(nativeApp)，或者还可以是显示设备上的浏览器的一个网页程序(webApp)，本实施例对此不进行限定。It can be understood that the cloud mobile phone application may be a local program (nativeApp) installed on the display device, or may also be a web page program (webApp) of a browser on the display device, which is not limited in this embodiment. .

以WiFi连接为例，假设运行云手机应用的显示设备记为A，运行虚拟手柄应用的终端设备记为B，举例说明云手机应用与虚拟手柄应用如何建立WiFi连接。Taking WiFi connection as an example, suppose that the display device running the cloud phone application is marked as A, and the terminal device running the virtual controller application is marked as B, and an example is given to illustrate how to establish a WiFi connection between the cloud phone application and the virtual controller application.

可选地，在本实施例的一个可能的实现方式中，在201之前，还可以进一步输出所述待操控应用的确定信息，以供虚拟手柄应用根据所述待操控应用的确定信息，确定所述云手机应用中所运行的待操控应用。Optionally, in a possible implementation manner of this embodiment, beforestep 201 , the determination information of the application to be manipulated may be further output, so that the virtual handle application can determine all the applications to be manipulated according to the determination information of the application to be manipulated. The application to be controlled running in the cloud mobile phone application.

在该实现方式中，在云手机应用输出所述待操控应用的确定信息之后，虚拟手柄应用具体可以采用各种方式，获取所述云手机应用所提供的所述待操控应用的确定信息。In this implementation manner, after the cloud mobile phone application outputs the determination information of the application to be controlled, the virtual handle application may specifically acquire the determination information of the application to be controlled provided by the cloud mobile phone application in various ways.

在虚拟手柄应用确定所述云手机应用中所运行的待操控应用之后，虚拟手柄应用则可以根据所述待操控应用，获得并输出所述应用操控界面。After the virtual handle application determines the application to be controlled running in the cloud phone application, the virtual handle application may obtain and output the application control interface according to the application to be controlled.

虚拟手柄应用具体可以获取用户基于所述应用操控界面的操作，进而，响应于用户基于所述应用操控界面的操作，获取所在终端设备上的触控事件参数。然后，虚拟手柄应用则可以基于所建立的通信连接，将所述终端设备上的触控事件参数发送给所述云手机应用。Specifically, the virtual handle application may acquire the user's operation based on the application control interface, and further, in response to the user's operation based on the application control interface, acquire touch event parameters on the terminal device where it is located. Then, the virtual handle application may send the touch event parameters on the terminal device to the cloud phone application based on the established communication connection.

其中，获取用户基于所述应用操控界面的操作，的详细描述可以参见第一实施例中的相关描述。Wherein, for a detailed description of acquiring the operation of the user based on the application manipulation interface, reference may be made to the relevant description in the first embodiment.

可选地，在本实施例的一个可能的实现方式中，在202中，具体可以根据所述终端设备上的触控事件的名称，获得所述待操控应用的触控事件的名称，进而，则可以根据所述显示设备的屏幕分辨率和所述待操控应用的页面属性，将所述终端设备上的触控事件的坐标映射到所述显示设备的屏幕上，以获得所述待操控应用的触控事件的坐标。Optionally, in a possible implementation manner of this embodiment, in 202, the name of the touch event of the application to be controlled may be obtained specifically according to the name of the touch event on the terminal device, and then, Then, according to the screen resolution of the display device and the page attribute of the application to be manipulated, the coordinates of the touch event on the terminal device can be mapped to the screen of the display device to obtain the application to be manipulated. The coordinates of the touch event.

该实现方式中，在云手机应用基于云手机应用与虚拟手柄应用之间的通信连接，接收所述终端设备上的触控事件参数之后，所述云手机应用则可以先解码出所述终端设备上的触控事件参数。然后，所述云手机应用则可以根据所述终端设备上的触控事件的名称，获得所述待操控应用的触控事件的名称，再根据所述显示设备的屏幕分辨率和所述待操控应用的页面属性，将所述终端设备上的触控事件的坐标映射到所述显示设备的屏幕上，以获得所述待操控应用的触控事件的坐标，进而对所述待操控应用进行操控，从而实现了用户通过操作终端设备就能够操控显示设备上所运行的云手机应用的目的。In this implementation, after the cloud mobile phone application receives the touch event parameters on the terminal device based on the communication connection between the cloud mobile phone application and the virtual handle application, the cloud mobile phone application can first decode the terminal device touch event parameters on . Then, the cloud phone application can obtain the name of the touch event of the application to be controlled according to the name of the touch event on the terminal device, and then according to the screen resolution of the display device and the name of the to-be-controlled application The page attribute of the application, the coordinates of the touch event on the terminal device are mapped to the screen of the display device to obtain the coordinates of the touch event of the application to be manipulated, and then the application to be manipulated is manipulated , so that the user can control the cloud mobile phone application running on the display device by operating the terminal device.

xout＝xin×screenWidth；xout=xin×screenWidth;

yout＝yin×screenHeight。yout=yin×screenHeight.

xout＝(1-yin)×screenWidth；xout=(1-yin)×screenWidth;

yout＝xin×screenHeight。yout=xin×screenHeight.

此时，在202中，所述云手机应用具体可以根据所述终端设备上的触控事件的名称、所述终端设备上的触控事件的坐标、所述显示设备的屏幕分辨率和所述待操控应用的页面属性，获得所述待操控应用的触控事件参数，进而，则可以根据所述终端设备的屏幕分辨率，对所获得的所述待操控应用的触控事件参数进行调整处理。At this time, in 202, the cloud mobile phone application may specifically be based on the name of the touch event on the terminal device, the coordinates of the touch event on the terminal device, the screen resolution of the display device and the The page properties of the application to be controlled are obtained, and the touch event parameters of the application to be controlled are obtained, and further, the obtained touch event parameters of the application to be controlled can be adjusted according to the screen resolution of the terminal device. .

本实施例中，通过基于与虚拟手柄应用之间的通信连接，接收所述虚拟手柄应用发送的所述虚拟手柄应用所在终端设备上的触控事件参数，所述终端设备上的触控事件参数为所述虚拟手柄应用输出显示设备上所运行的云手机应用中待操控应用的应用操控界面，响应于用户基于所述应用操控界面的操作获取的，进而，根据所述终端设备上的触控事件参数、所述显示设备的屏幕分辨率和所述待操控应用的页面属性，获得所述待操控应用的触控事件参数，使得能够根据所述待操控应用的触控事件参数，对所述待操控应用进行操控，从而实现了对运行在显示设备上的云手机进行操控。In this embodiment, based on the communication connection with the virtual handle application, the touch event parameters on the terminal device where the virtual handle application is located and sent by the virtual handle application are received, and the touch event parameters on the terminal device are Outputting the application control interface of the application to be controlled in the cloud mobile phone application running on the display device for the virtual handle application, obtained in response to the user's operation based on the application control interface, and further, according to the touch control on the terminal device event parameters, the screen resolution of the display device, and the page attributes of the application to be manipulated, and the touch event parameters of the application to be manipulated are obtained, so that the touch event parameters of the application to be manipulated can be The application to be controlled is controlled, thereby realizing the control of the cloud mobile phone running on the display device.

需要说明的是，对于前述的各方法实施例，为了简单描述，故将其都表述为一系列的动作组合，但是本领域技术人员应该知悉，本申请并不受所描述的动作顺序的限制，因为依据本申请，某些步骤可以采用其他顺序或者同时进行。其次，本领域技术人员也应该知悉，说明书中所描述的实施例均属于优选实施例，所涉及的动作和模块并不一定是本申请所必须的。It should be noted that, for the sake of simple description, the foregoing method embodiments are all expressed as a series of action combinations, but those skilled in the art should know that the present application is not limited by the described action sequence. Because in accordance with the present application, certain steps may be performed in other orders or simultaneously. Secondly, those skilled in the art should also know that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by the present application.

在上述实施例中，对各个实施例的描述都各有侧重，某个实施例中没有详述的部分，可以参见其他实施例的相关描述。In the above-mentioned embodiments, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference may be made to the relevant descriptions of other embodiments.

图3是根据本申请第三实施例的示意图，如图3所示。本实施例的云手机的操控装置300可以包括输出单元301、获取单元302和发送单元303。其中，输出单元301，用于输出显示设备上所运行的云手机应用中待操控应用的应用操控界面；获取单元302，用于响应于用户基于所述应用操控界面的操作，获取所在终端设备上的触控事件参数，所述触控事件参数包括所述终端设备上的触控事件的名称和该触控事件的坐标；发送单元303，用于基于与所述云手机应用之间的通信连接，将所述终端设备上的触控事件参数发送给所述云手机应用，以供所述云手机应用根据所述终端设备上的触控事件参数、所述显示设备的屏幕分辨率和所述待操控应用的页面属性，获得所述待操控应用的触控事件参数，根据所述待操控应用的触控事件参数，对所述待操控应用进行操控，所述待操控应用的触控事件参数包括所述待操控应用的触控事件的名称和该触控事件的坐标。FIG. 3 is a schematic diagram according to a third embodiment of the present application, as shown in FIG. 3 . Thecontrol apparatus 300 of the cloud phone in this embodiment may include anoutput unit 301 , anacquisition unit 302 and atransmission unit 303 . Wherein, theoutput unit 301 is used to output the application control interface of the application to be controlled in the cloud mobile phone application running on the display device; theacquisition unit 302 is used to respond to the user's operation based on the application control interface, obtain the information on the terminal device. touch event parameters, the touch event parameters include the name of the touch event on the terminal device and the coordinates of the touch event; the sendingunit 303 is configured to connect based on the communication with the cloud phone application , send the touch event parameters on the terminal device to the cloud mobile phone application, so that the cloud mobile phone application can use the touch event parameters on the terminal device, the screen resolution of the display device and the Page attributes of the application to be controlled, obtain the touch event parameters of the application to be controlled, and control the application to be controlled according to the touch event parameters of the application to be controlled, and the touch event parameters of the application to be controlled Including the name of the touch event of the application to be controlled and the coordinates of the touch event.

需要说明的是，本实施例所提供的云手机的操控装置的部分或全部可以为位于终端设备的虚拟手柄应用，或者还可以为设置在位于终端设备的虚拟手柄应用中的插件或SDK(SoftwareDevelopmentKit，软件开发工具包)等功能单元，本实施例对此不进行特别限定。It should be noted that part or all of the control device of the cloud phone provided in this embodiment may be a virtual handle application located in the terminal device, or may also be a plug-in or SDK (Software Development Kit) located in the virtual handle application located in the terminal device. , software development kit) and other functional units, which are not particularly limited in this embodiment.

可选地，在本实施例的一个可能的实现方式中，所述输出单元301，还可以进一步用于确定所述云手机应用中所运行的待操控应用；以及根据所述待操控应用，获得所述应用操控界面。Optionally, in a possible implementation manner of this embodiment, theoutput unit 301 may be further configured to determine the application to be controlled running in the cloud mobile phone application; and obtain the application to be controlled according to the application to be controlled the application control interface.

在一个具体的实现过程中，所述输出单元301，具体可以用于获取所述云手机应用所提供的所述待操控应用的确定信息；以及根据所述待操控应用的确定信息，确定所述云手机应用中所运行的待操控应用。In a specific implementation process, theoutput unit 301 may be specifically configured to obtain the determination information of the application to be controlled provided by the cloud mobile phone application; and determine the application to be controlled according to the determination information of the application to be controlled The application to be controlled running in the cloud mobile application.

可选地，在本实施例的一个可能的实现方式中，所述终端设备上的触控事件参数还可以进一步包括所述终端设备的屏幕分辨率，以供所述云手机应用根据所述终端设备的屏幕分辨率，对所获得的所述待操控应用的触控事件参数进行调整处理。Optionally, in a possible implementation manner of this embodiment, the touch event parameters on the terminal device may further include the screen resolution of the terminal device, so that the cloud mobile phone application can use the terminal device according to the screen resolution of the terminal device. The screen resolution of the device is used to adjust the obtained touch event parameters of the application to be controlled.

需要说明的是，图1A对应的实施例中的方法可以由本实施例提供的云手机的操控装置实现。详细描述可以参见图1A对应的实施例中的相关内容，此处不再赘述。It should be noted that, the method in the embodiment corresponding to FIG. 1A may be implemented by the control device of the cloud phone provided in this embodiment. For a detailed description, reference may be made to the relevant content in the embodiment corresponding to FIG. 1A , which will not be repeated here.

本实施例中，通过输出单元输出显示设备上所运行的云手机应用中待操控应用的应用操控界面，进而，由获取单元响应于用户基于所述应用操控界面的操作，获取所在终端设备上的触控事件参数，使得发送单元能够基于与所述云手机应用之间的通信连接，将所述终端设备上的触控事件参数发送给所述云手机应用，以供所述云手机应用根据所述终端设备上的触控事件参数、所述显示设备的屏幕分辨率和所述待操控应用的页面属性，获得所述待操控应用的触控事件参数，根据所述待操控应用的触控事件参数，对所述待操控应用进行操控，从而实现了对运行在显示设备上的云手机进行操控。In this embodiment, the application control interface of the application to be controlled in the cloud mobile phone application running on the display device is output through the output unit, and further, the acquisition unit responds to the user's operation based on the application control interface to obtain the information on the terminal device where it is located. touch event parameters, so that the sending unit can send the touch event parameters on the terminal device to the cloud mobile application based on the communication connection with the cloud mobile application, so that the cloud mobile application can touch event parameters on the terminal device, the screen resolution of the display device, and the page attributes of the application to be controlled, obtain the touch event parameters of the application to be controlled, and obtain the touch event parameters of the application to be controlled according to the touch event of the application to be controlled parameters to control the application to be controlled, thereby realizing the control of the cloud mobile phone running on the display device.

图4是根据本申请第四实施例的示意图，如图4所示。本实施例的云手机的操控装置400可以包括接收单元401、映射单元402和响应单元403。其中，接收单元401，用于基于与虚拟手柄应用之间的通信连接，接收所述虚拟手柄应用发送的所述虚拟手柄应用所在终端设备上的触控事件参数，所述触控事件参数包括所述终端设备上的触控事件的名称和该触控事件的坐标；其中，所述终端设备上的触控事件参数为所述虚拟手柄应用输出显示设备上所运行的云手机应用中待操控应用的应用操控界面，响应于用户基于所述应用操控界面的操作获取的；映射单元402，用于根据所述终端设备上的触控事件参数、所述显示设备的屏幕分辨率和所述待操控应用的页面属性，获得所述待操控应用的触控事件参数，所述待操控应用的触控事件参数包括所述待操控应用的触控事件的名称和该触控事件的坐标；响应单元403，用于根据所述待操控应用的触控事件参数，对所述待操控应用进行操控。FIG. 4 is a schematic diagram according to a fourth embodiment of the present application, as shown in FIG. 4 . Thecontrol apparatus 400 of the cloud phone in this embodiment may include a receivingunit 401 , amapping unit 402 and aresponse unit 403 . The receivingunit 401 is configured to receive, based on the communication connection with the virtual handle application, the touch event parameters sent by the virtual handle application on the terminal device where the virtual handle application is located, where the touch event parameters include all The name of the touch event on the terminal device and the coordinates of the touch event; wherein, the touch event parameter on the terminal device is the application to be controlled in the cloud phone application running on the virtual handle application output display device The application control interface is obtained in response to the user's operation based on the application control interface; themapping unit 402 is used for the touch event parameters on the terminal device, the screen resolution of the display device and the to-be-controlled The page attribute of the application, to obtain the touch event parameters of the application to be controlled, the touch event parameters of the application to be controlled include the name of the touch event of the application to be controlled and the coordinates of the touch event; theresponse unit 403 , which is used to control the application to be controlled according to the touch event parameters of the application to be controlled.

需要说明的是，本实施例所提供的云手机的操控装置的部分或全部可以为位于显示设备的云手机应用，或者还可以为设置在位于显示设备的云手机应用中的插件或SDK(SoftwareDevelopmentKit，软件开发工具包)等功能单元，本实施例对此不进行特别限定。It should be noted that part or all of the control device of the cloud phone provided in this embodiment may be a cloud phone application located on the display device, or may also be a plug-in or SDK (Software Development Kit) provided in the cloud phone application located on the display device. , software development kit) and other functional units, which are not particularly limited in this embodiment.

可选地，在本实施例的一个可能的实现方式中，4、根据权利要求13所述的装置，其中，所述接收单元401，还用于Optionally, in a possible implementation manner of this embodiment, 4. The apparatus according to claim 13, wherein the receivingunit 401 is further configured to

输出所述待操控应用的确定信息，以供虚拟手柄应用根据所述待操控应用的确定信息，确定所述云手机应用中所运行的待操控应用。The determination information of the application to be controlled is output, so that the virtual controller application determines the application to be controlled running in the cloud mobile phone application according to the determination information of the application to be controlled.

可选地，在本实施例的一个可能的实现方式中，所述映射单元402，具体可以用于根据所述终端设备上的触控事件的名称，获得所述待操控应用的触控事件的名称；以及根据所述显示设备的屏幕分辨率和所述待操控应用的页面属性，将所述终端设备上的触控事件的坐标映射到所述显示设备的屏幕上，以获得所述待操控应用的触控事件的坐标。Optionally, in a possible implementation manner of this embodiment, themapping unit 402 may be specifically configured to obtain the touch event information of the application to be controlled according to the name of the touch event on the terminal device. and according to the screen resolution of the display device and the page attribute of the application to be manipulated, map the coordinates of the touch event on the terminal device to the screen of the display device to obtain the to-be-operated application The coordinates of the applied touch event.

可选地，在本实施例的一个可能的实现方式中，所述终端设备上的触控事件参数还还可以进一步包括所述终端设备的屏幕分辨率；相应地，所述映射单元402，具体可以用于根据所述终端设备上的触控事件的名称、所述终端设备上的触控事件的坐标、所述显示设备的屏幕分辨率和所述待操控应用的页面属性，获得所述待操控应用的触控事件参数；以及根据所述终端设备的屏幕分辨率，对所获得的所述待操控应用的触控事件参数进行调整处理。Optionally, in a possible implementation manner of this embodiment, the touch event parameters on the terminal device may further include the screen resolution of the terminal device; correspondingly, themapping unit 402, specifically It can be used to obtain the to-be-controlled application according to the name of the touch event on the terminal device, the coordinates of the touch event on the terminal device, the screen resolution of the display device and the page attribute of the application to be manipulated. controlling the touch event parameters of the application; and adjusting and processing the obtained touch event parameters of the application to be controlled according to the screen resolution of the terminal device.

需要说明的是，图2对应的实施例中的方法可以由本实施例提供的云手机的操控装置实现。详细描述可以参见图2对应的实施例中的相关内容，此处不再赘述。It should be noted that, the method in the embodiment corresponding to FIG. 2 may be implemented by the control device of the cloud phone provided in this embodiment. For a detailed description, reference may be made to the relevant content in the embodiment corresponding to FIG. 2 , which will not be repeated here.

本实施例中，通过接收单元基于与虚拟手柄应用之间的通信连接，接收所述虚拟手柄应用发送的所述虚拟手柄应用所在终端设备上的触控事件参数，所述终端设备上的触控事件参数为所述虚拟手柄应用输出显示设备上所运行的云手机应用中待操控应用的应用操控界面，响应于用户基于所述应用操控界面的操作获取的，进而，由映射单元根据所述终端设备上的触控事件参数、所述显示设备的屏幕分辨率和所述待操控应用的页面属性，获得所述待操控应用的触控事件参数，使得响应单元能够根据所述待操控应用的触控事件参数，对所述待操控应用进行操控，从而实现了对运行在显示设备上的云手机进行操控。In this embodiment, based on the communication connection with the virtual handle application, the receiving unit receives the touch event parameters sent by the virtual handle application on the terminal device where the virtual handle application is located, and the touch event parameters on the terminal device are The event parameter is the application control interface of the application to be controlled in the cloud mobile phone application running on the virtual controller application output display device, and is obtained in response to the user's operation based on the application control interface, and further, the mapping unit according to the terminal The touch event parameters on the device, the screen resolution of the display device, and the page attributes of the application to be manipulated are obtained, and the touch event parameters of the application to be manipulated are obtained, so that the response unit can The control event parameters are used to control the application to be controlled, thereby realizing the control of the cloud mobile phone running on the display device.

根据本申请的实施例，本申请还提供了一种电子设备和一种存储有计算机指令的非瞬时计算机可读存储介质。According to the embodiments of the present application, the present application further provides an electronic device and a non-transitory computer-readable storage medium storing computer instructions.

如图5所示，是用来实现本申请实施例的云手机的操控方法的电子设备的示意图。电子设备旨在表示各种形式的数字计算机，诸如，膝上型计算机、台式计算机、工作台、个人数字助理、服务器、刀片式服务器、大型计算机、和其它适合的计算机。电子设备还可以表示各种形式的移动装置，诸如，个人数字处理、蜂窝电话、智能电话、可穿戴设备和其它类似的计算装置。本文所示的部件、它们的连接和关系、以及它们的功能仅仅作为示例，并且不意在限制本文中描述的和/或者要求的本申请的实现。As shown in FIG. 5 , it is a schematic diagram of an electronic device used to implement the control method of the cloud phone according to the embodiment of the present application. Electronic devices are intended to represent various forms of digital computers, such as laptops, desktops, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. Electronic devices may also represent various forms of mobile devices, such as personal digital processors, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are by way of example only, and are not intended to limit implementations of the application described and/or claimed herein.

如图5所示，该电子设备包括：一个或多个处理器501、存储器502，以及用于连接各部件的接口，包括高速接口和低速接口。各个部件利用不同的总线互相连接，并且可以被安装在公共主板上或者根据需要以其它方式安装。处理器可以对在电子设备内执行的指令进行处理，包括存储在存储器中或者存储器上以在外部输入/输出装置(诸如，耦合至接口的显示设备)上显示GUI(图形用户界面)的图形信息的指令。在其它实施方式中，若需要，可以将多个处理器和/或多条总线与多个存储器和多个存储器一起使用。同样，可以连接多个电子设备，各个设备提供部分必要的操作(例如，作为服务器阵列、一组刀片式服务器、或者多处理器系统)。图5中以一个处理器501为例。As shown in FIG. 5, the electronic device includes: one ormore processors 501, amemory 502, and interfaces for connecting various components, including a high-speed interface and a low-speed interface. The various components are interconnected using different buses and may be mounted on a common motherboard or otherwise as desired. The processor may process instructions for execution within the electronic device, including graphical information stored in or on memory to display GUI (graphical user interface) graphical information on external input/output devices such as a display device coupled to the interface instruction. In other embodiments, multiple processors and/or multiple buses may be used with multiple memories and multiple memories, if desired. Likewise, multiple electronic devices may be connected, each providing some of the necessary operations (eg, as a server array, a group of blade servers, or a multiprocessor system). Aprocessor 501 is taken as an example in FIG. 5 .

存储器502即为本申请所提供的非瞬时计算机可读存储介质。其中，所述存储器存储有可由至少一个处理器执行的指令，以使所述至少一个处理器执行本申请所提供的云手机的操控方法。本申请的非瞬时计算机可读存储介质存储计算机指令，该计算机指令用于使计算机执行本申请所提供的云手机的操控方法。Thememory 502 is the non-transitory computer-readable storage medium provided by the present application. Wherein, the memory stores instructions executable by at least one processor, so that the at least one processor executes the method for controlling a cloud phone provided by the present application. The non-transitory computer-readable storage medium of the present application stores computer instructions, and the computer instructions are used to make the computer execute the control method of the cloud phone provided by the present application.

存储器502作为一种非瞬时计算机可读存储介质，可用于存储非瞬时软件程序、非瞬时计算机可执行程序以及单元，如本申请实施例中的云手机的操控方法对应的程序指令/单元(例如，附图2所示的输出单元201、调整单元202和存储单元203)。处理器501通过运行存储在存储器502中的非瞬时软件程序、指令以及单元，从而执行服务器的各种功能应用以及数据处理，即实现上述方法实施例中的云手机的操控方法。As a non-transitory computer-readable storage medium, thememory 502 can be used to store non-transitory software programs, non-transitory computer-executable programs, and units, such as program instructions/units (eg, program instructions/units corresponding to the control method of the cloud phone in the embodiments of the present application). , theoutput unit 201, theadjustment unit 202 and thestorage unit 203 shown in FIG. 2). Theprocessor 501 executes various functional applications and data processing of the server by running the non-transitory software programs, instructions and units stored in thememory 502 , that is, implementing the control method of the cloud phone in the above method embodiments.

存储器502可以包括存储程序区和存储数据区，其中，存储程序区可存储操作系统、至少一个功能所需要的应用程序；存储数据区可存储根据实现本申请实施例提供的云手机的操控方法的电子设备的使用所创建的数据等。此外，存储器502可以包括高速随机存取存储器，还可以包括非瞬时存储器，例如至少一个磁盘存储器件、闪存器件、或其他非瞬时固态存储器件。在一些实施例中，存储器502可选包括相对于处理器501远程设置的存储器，这些远程存储器可以通过网络连接至实现本申请实施例提供的云手机的操控方法的电子设备。上述网络的实例包括但不限于互联网、企业内部网、局域网、移动通信网及其组合。Thememory 502 may include a stored program area and a stored data area, wherein the stored program area may store an operating system and an application program required by at least one function; Data created by the use of electronic equipment, etc. Additionally,memory 502 may include high-speed random access memory, and may also include non-transitory memory, such as at least one magnetic disk storage device, flash memory device, or other non-transitory solid-state storage device. In some embodiments, thememory 502 may optionally include a memory set remotely relative to theprocessor 501 , and the remote memory may be connected to an electronic device implementing the cloud phone control method provided by the embodiments of the present application through a network. Examples of such networks include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

云手机的操控方法的电子设备还可以包括：输入装置503和输出装置504。处理器501、存储器502、输入装置503和输出装置504可以通过总线或者其他方式连接，图5中以通过总线连接为例。The electronic device for the control method of the cloud phone may further include: aninput device 503 and anoutput device 504 . Theprocessor 501 , thememory 502 , theinput device 503 and theoutput device 504 may be connected by a bus or in other ways, and the connection by a bus is taken as an example in FIG. 5 .

输入装置503可接收输入的数字或字符信息，以及产生与实现本申请实施例提供的云手机的操控方法的电子设备的用户设置以及功能控制有关的键信号输入，例如触摸屏、小键盘、鼠标、轨迹板、触摸板、指示杆、一个或者多个鼠标按钮、轨迹球、操纵杆等输入装置。输出装置504可以包括显示设备、辅助照明装置(例如，LED)和触觉反馈装置(例如，振动电机)等。该显示设备可以包括但不限于，LCD(液晶显示器)、LED(发光二极管)显示器和等离子体显示器。在一些实施方式中，显示设备可以是触摸屏。Theinput device 503 can receive input digital or character information, and generate key signal input related to user settings and function control of the electronic device implementing the cloud phone control method provided in the embodiment of the present application, such as touch screen, keypad, mouse, Input devices such as trackpads, touchpads, pointing sticks, one or more mouse buttons, trackballs, joysticks, etc. Theoutput device 504 may include a display device, auxiliary lighting devices (eg, LEDs), haptic feedback devices (eg, vibration motors), and the like. The display devices may include, but are not limited to, LCD (Liquid Crystal Display), LED (Light Emitting Diode) displays, and plasma displays. In some implementations, the display device may be a touch screen.

此处描述的系统和技术的各种实施方式可以在数字电子电路系统、集成电路系统、ASIC(专用集成电路)、计算机硬件、固件、软件、和/或它们的组合中实现。这些各种实施方式可以包括：实施在一个或者多个计算机程序中，该一个或者多个计算机程序可在包括至少一个可编程处理器的可编程系统上执行和/或解释，该可编程处理器可以是专用或者通用可编程处理器，可以从存储系统、至少一个输入装置、和至少一个输出装置接收数据和指令，并且将数据和指令传输至该存储系统、该至少一个输入装置、和该至少一个输出装置。Various implementations of the systems and techniques described herein can be implemented in digital electronic circuitry, integrated circuit systems, ASICs (application specific integrated circuits), computer hardware, firmware, software, and/or combinations thereof. These various embodiments may include being implemented in one or more computer programs executable and/or interpretable on a programmable system including at least one programmable processor that The processor, which may be a special purpose or general-purpose programmable processor, may receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device an output device.

这些计算程序(也称作程序、软件、软件应用、或者代码)包括可编程处理器的机器指令，并且可以利用高级过程和/或面向对象的编程语言、和/或汇编/机器语言来实施这些计算程序。如本文使用的，术语“机器可读介质”和“计算机可读介质”指的是用于将机器指令和/或数据提供给可编程处理器的任何计算机程序产品、设备、和/或装置(例如，磁盘、光盘、存储器、PLD(可编程逻辑器件))，包括，接收作为机器可读信号的机器指令的机器可读介质。术语“机器可读信号”指的是用于将机器指令和/或数据提供给可编程处理器的任何信号。These computational programs (also referred to as programs, software, software applications, or codes) include machine instructions for programmable processors, and may be implemented using high-level procedural and/or object-oriented programming languages, and/or assembly/machine languages calculation program. As used herein, the terms "machine-readable medium" and "computer-readable medium" refer to any computer program product, apparatus, and/or apparatus for providing machine instructions and/or data to a programmable processor ( For example, a magnetic disk, an optical disk, a memory, a PLD (Programmable Logic Device)), including a machine-readable medium that receives machine instructions as a machine-readable signal. The term "machine-readable signal" refers to any signal used to provide machine instructions and/or data to a programmable processor.

为了提供与用户的交互，可以在计算机上实施此处描述的系统和技术，该计算机具有：用于向用户显示信息的显示装置(例如，CRT(阴极射线管)或者LCD(液晶显示器)监视器)；以及键盘和指向装置(例如，鼠标或者轨迹球)，用户可以通过该键盘和该指向装置来将输入提供给计算机。其它种类的装置还可以用于提供与用户的交互；例如，提供给用户的反馈可以是任何形式的传感反馈(例如，视觉反馈、听觉反馈、或者触觉反馈)；并且可以用任何形式(包括声输入、语音输入或者、触觉输入)来接收来自用户的输入。To provide interaction with a user, the systems and techniques described herein may be implemented on a computer having a display device (eg, a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user ); and a keyboard and pointing device (eg, a mouse or trackball) through which a user can provide input to the computer. Other kinds of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (eg, visual feedback, auditory feedback, or tactile feedback); and can be in any form (including acoustic input, voice input, or tactile input) to receive input from the user.

可以将此处描述的系统和技术实施在包括后台部件的计算系统(例如，作为数据服务器)、或者包括中间件部件的计算系统(例如，应用服务器)、或者包括前端部件的计算系统(例如，具有图形用户界面或者网络浏览器的用户计算机，用户可以通过该图形用户界面或者该网络浏览器来与此处描述的系统和技术的实施方式交互)、或者包括这种后台部件、中间件部件、或者前端部件的任何组合的计算系统中。可以通过任何形式或者介质的数字数据通信(例如，通信网络)来将系统的部件相互连接。通信网络的示例包括：LAN(局域网)、WAN(广域网)、互联网和区块链网络。The systems and techniques described herein may be implemented on a computing system that includes back-end components (eg, as a data server), or a computing system that includes middleware components (eg, an application server), or a computing system that includes front-end components (eg, a user's computer having a graphical user interface or web browser through which a user may interact with implementations of the systems and techniques described herein), or including such backend components, middleware components, Or any combination of front-end components in a computing system. The components of the system may be interconnected by any form or medium of digital data communication (eg, a communication network). Examples of communication networks include: LAN (Local Area Network), WAN (Wide Area Network), the Internet, and blockchain networks.

计算机系统可以包括客户端和服务器。客户端和服务器一般远离彼此并且通常通过通信网络进行交互。通过在相应的计算机上运行并且彼此具有客户端-服务器关系的计算机程序来产生客户端和服务器的关系。A computer system can include clients and servers. Clients and servers are generally remote from each other and usually interact through a communication network. The relationship of client and server arises by computer programs running on the respective computers and having a client-server relationship to each other.

根据本申请实施例的技术方案，通过虚拟手柄应用输出显示设备上所运行的云手机应用中待操控应用的应用操控界面，进而，由虚拟手柄应用响应于用户基于所述应用操控界面的操作，获取所在终端设备上的触控事件参数，使得虚拟手柄应用能够基于与所述云手机应用之间的通信连接，将所述终端设备上的触控事件参数发送给所述云手机应用，这样，所述云手机应用就可以根据所述终端设备上的触控事件参数、所述显示设备的屏幕分辨率和所述待操控应用的页面属性，获得所述待操控应用的触控事件参数，根据所述待操控应用的触控事件参数，对所述待操控应用进行操控，从而实现了对运行在显示设备上的云手机进行操控。According to the technical solutions of the embodiments of the present application, the application control interface of the application to be controlled in the cloud mobile phone application running on the display device is output through the virtual handle application, and then the virtual handle application responds to the user's operation based on the application control interface, Acquire the touch event parameters on the terminal device, so that the virtual handle application can send the touch event parameters on the terminal device to the cloud mobile phone application based on the communication connection with the cloud mobile phone application, so that, The cloud phone application can obtain the touch event parameters of the application to be controlled according to the touch event parameters on the terminal device, the screen resolution of the display device, and the page attributes of the application to be controlled, according to the The touch event parameter of the application to be controlled controls the application to be controlled, thereby realizing the control of the cloud phone running on the display device.

应该理解，可以使用上面所示的各种形式的流程，重新排序、增加或删除步骤。例如，本发申请中记载的各步骤可以并行地执行也可以顺序地执行也可以不同的次序执行，只要能够实现本申请公开的技术方案所期望的结果，本文在此不进行限制。It should be understood that steps may be reordered, added or deleted using the various forms of flow shown above. For example, the steps described in the present application can be performed in parallel, sequentially or in different orders, and as long as the desired results of the technical solutions disclosed in the present application can be achieved, no limitation is imposed herein.

上述具体实施方式，并不构成对本申请保护范围的限制。本领域技术人员应该明白的是，根据设计要求和其他因素，可以进行各种修改、组合、子组合和替代。任何在本申请的精神和原则之内所作的修改、等同替换和改进等，均应包含在本申请保护范围之内。The above-mentioned specific embodiments do not constitute a limitation on the protection scope of the present application. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions may occur depending on design requirements and other factors. Any modifications, equivalent replacements and improvements made within the spirit and principles of this application shall be included within the protection scope of this application.