技术领域technical field
本发明涉及远程桌面技术领域，尤其涉及一种远程桌面图像处理方法及装置。The invention relates to the technical field of remote desktops, in particular to a remote desktop image processing method and device.
背景技术Background technique
远程桌面技术使得通过网络连接并控制另一台开启了远程桌面连接功能的设备成为可能，远程桌面技术不仅包括远程的桌面图像的显示，还可包含远程声音的传送以及可插拔硬件的映射等等，因此可提供具有丰富保真度的桌面或应用程序体验。对于企业办公、营业厅、医院等的计算机或移动设备这样的客户端设备，可以通过局域网或广域网连接到拥有大量服务器的数据中心，数据中心的服务器开启远程桌面连接功能，与客户端设备搭建起远程桌面系统后，各客户端设备即可通过该远程桌面技术实时操作服务端设备，从而更方便灵活地使用数据中心的资源。Remote desktop technology makes it possible to connect and control another device with remote desktop connection function enabled through the network. Remote desktop technology includes not only the display of remote desktop images, but also the transmission of remote sound and the mapping of pluggable hardware, etc. etc., thus delivering a desktop or application experience with rich fidelity. For client devices such as computers or mobile devices in corporate offices, business halls, hospitals, etc., they can be connected to a data center with a large number of servers through a LAN or WAN. After the remote desktop system, each client device can operate the server device in real time through the remote desktop technology, so as to use the resources of the data center more conveniently and flexibly.
在传统的远程桌面系统搭建好后，服务端设备响应用户在客户端设备上的操作，将桌面显示图像进行统一地压缩以及缓存处理，并根据远程桌面协议封装形成图形指令后发送给客户端设备。客户端设备对接收到的图形指令进行解码并将图像数据渲染成一幅完整的桌面显示图像，从而在客户端设备上显示一个完整的桌面图像。After the traditional remote desktop system is set up, the server device responds to the user's operation on the client device, compresses and caches the desktop display image uniformly, and encapsulates it according to the remote desktop protocol to form a graphic command and sends it to the client device . The client device decodes the received graphics instruction and renders the image data into a complete desktop display image, so as to display a complete desktop image on the client device.
由于桌面显示图像的类型是多样的，现有技术中对各种类型的图像数据均进行统一地处理，并采用同一种压缩算法进行压缩，这势必导致对于某种或某些类型的桌面显示图像无法达到开销最小或者效率最优。Because the types of desktop display images are diverse, various types of image data are uniformly processed in the prior art, and the same compression algorithm is used to compress, which will inevitably lead to certain or certain types of desktop display images. It is impossible to achieve the minimum cost or the best efficiency.
发明内容Contents of the invention
发明要解决的问题The problem to be solved by the invention
本发明要解决的技术问题是：提供一种能够在对桌面显示图像进行处理之前，对桌面显示的图像进行识别处理的方法及装置。The technical problem to be solved by the present invention is to provide a method and device capable of identifying and processing images displayed on the desktop before processing the images displayed on the desktop.
用于解决问题的方案solutions to problems
为了解决上述问题，第一方面，本发明的实施例提供了一种远程桌面图像处理方法，包括：In order to solve the above problems, in a first aspect, an embodiment of the present invention provides a remote desktop image processing method, including:
响应于由用户操作所触发的输入事件，获取与用户操作相关的图形指令；In response to an input event triggered by a user operation, obtain a graphic instruction related to the user operation;
从所述图形指令中获取光标信息，根据所述光标信息判定光标类型；Acquiring cursor information from the graphics instruction, and determining a cursor type according to the cursor information;
根据所述光标信息以及所判定的光标类型识别所述用户操作的操作场景。An operation scene operated by the user is identified according to the cursor information and the determined cursor type.
结合第一方面，在第一种可能的实施方式中，该方法还包括：With reference to the first aspect, in a first possible implementation manner, the method further includes:
为与所述用户操作相关的图像数据设置操作场景标识，所述操作场景标识用于表示所识别出的操作场景的类型。An operation scene identifier is set for the image data related to the user operation, and the operation scene identifier is used to indicate the type of the recognized operation scene.
结合第一方面的第一种可能的实施方式，在第二种可能的实施方式中，还包括：In combination with the first possible implementation manner of the first aspect, in a second possible implementation manner, it further includes:
根据所述图像数据携带的所述操作场景标识，对所述图像数据进行处理。The image data is processed according to the operation scene identifier carried by the image data.
结合第一方面的第二种可能的实施方式，在第三种可能的实施方式中，所述根据所述图像数据携带的所述操作场景标识，对所述操作场景的图像数据进行处理，具体包括：With reference to the second possible implementation manner of the first aspect, in a third possible implementation manner, the image data of the operation scene is processed according to the operation scene identifier carried in the image data, specifically include:
根据所述操作场景标识选择相应的图像缓存机制和图像压缩算法，并根据所述图像缓存机制和所述图像压缩算法对所述图像数据进行处理。Select a corresponding image cache mechanism and image compression algorithm according to the operation scene identifier, and process the image data according to the image cache mechanism and the image compression algorithm.
结合第一方面的第二种或第三种可能的实施方式，在第四中可能的实施例方式中，该方法还包括：In combination with the second or third possible implementation manner of the first aspect, in a fourth possible embodiment manner, the method further includes:
根据远程桌面协议，封装处理后的图像数据并发送。According to the remote desktop protocol, the processed image data is encapsulated and sent.
结合第一方面，在第五种可能的实施方式中，所述光标信息包括光标的形状和光标的尺寸信息；With reference to the first aspect, in a fifth possible implementation manner, the cursor information includes cursor shape and cursor size information;
所述光标的形状和光标的尺寸信息从所述图形指令中的DrvSetPointerShape指令中获取。The shape of the cursor and the size information of the cursor are obtained from the DrvSetPointerShape command in the graphics command.
结合第一方面，在第六种可能的实施方式中，所述光标信息包括光标的位置以及光标是否隐藏的信息；With reference to the first aspect, in a sixth possible implementation manner, the cursor information includes information about the position of the cursor and whether the cursor is hidden;
所述光标的位置信息以及光标是否隐藏的信息从所述图形指令中的DrvMovePointer指令中获取。The position information of the cursor and the information of whether the cursor is hidden are obtained from the DrvMovePointer instruction in the graphics instruction.
结合第一方面，在第七种可能的实施方式中，所述图像数据包括光标的图像数据和远程桌面上显示的除所述光标外的图像数据。With reference to the first aspect, in a seventh possible implementation manner, the image data includes image data of a cursor and image data other than the cursor displayed on the remote desktop.
第二方面，本发明实施例提供了一种远程桌面图像处理装置，包括：In a second aspect, an embodiment of the present invention provides a remote desktop image processing device, including:
图形指令获取单元，用于响应于由用户操作所触发的输入事件，获取与所述用户操作相关的图形指令，并将所述图形指令发送至光标信息获取单元；a graphics instruction acquisition unit, configured to acquire a graphics instruction related to the user operation in response to an input event triggered by the user operation, and send the graphics instruction to the cursor information acquisition unit;
光标信息获取单元，用于接收来自所述图形指令获取单元发送的图形指令，从所述图形指令中获取光标信息，根据所述光标信息判定光标类型，并将所述光标信息和光标类型发送至识别单元；a cursor information acquisition unit, configured to receive the graphics instruction sent by the graphics instruction acquisition unit, acquire cursor information from the graphics instruction, determine the cursor type according to the cursor information, and send the cursor information and cursor type to identification unit;
识别单元，用于接收来自所述光标信息获取单元发送的光标信息和光标类型，根据所述光标信息以及所判定的光标类型识别所述操作场景的类型。The identification unit is configured to receive the cursor information and cursor type sent from the cursor information acquisition unit, and identify the type of the operation scene according to the cursor information and the determined cursor type.
结合第二方面，在第一种可能的实施方式中，该装置还包括：With reference to the second aspect, in a first possible implementation manner, the device further includes:
标记单元，用于根据所述识别单元识别的所述操作场景的类型，为与所述用户操作相关的图像数据设置操作场景标识，所述操作场景标识用于表示所识别出的操作场景的类型。A marking unit, configured to set an operation scene identifier for the image data related to the user operation according to the type of the operation scenario identified by the identification unit, and the operation scenario identifier is used to represent the identified type of the operation scenario .
结合第二方面的第一种可能的实施方式，在第二种可能的实施方式中，还包括：With reference to the first possible implementation manner of the second aspect, the second possible implementation manner further includes:
分类处理单元，用于根据所述图像数据携带的所述操作场景标识，对所述图像数据进行处理。A classification processing unit, configured to process the image data according to the operation scene identifier carried by the image data.
结合第二方面的第二种可能的实施方式，在第三种可能的实施方式中，所述处理单元具体用于根据所述操作场景标识选择图像缓存机制和图像压缩算法，并根据所述图像缓存机制和所述图像压缩算法对所述图像数据进行处理。With reference to the second possible implementation manner of the second aspect, in a third possible implementation manner, the processing unit is specifically configured to select an image caching mechanism and an image compression algorithm according to the operation scene identifier, and The cache mechanism and the image compression algorithm process the image data.
结合第二方面的第二种或第三种可能的实施方式，在第四种可能的实施方式中，还包括：In combination with the second or third possible implementation manner of the second aspect, the fourth possible implementation manner further includes:
封装单元，用于从所述分类处理单元接收处理后的图像数据，根据远程桌面协议封装所述处理后的图像数据并发送。An encapsulation unit, configured to receive the processed image data from the classification processing unit, encapsulate the processed image data according to the remote desktop protocol, and send the processed image data.
第三方面，本发明实施例还提供了一种远程桌面服务器，包括中央处理器和存储器，所述存储器存储执行指令，当所述远程桌面服务器运行时，所述中央处理器与所述存储器之间通信，所述中央处理器执行所述执行指令使得所述远程桌面服务器执行上述第一方面实施例中的任一方法。In the third aspect, the embodiment of the present invention also provides a remote desktop server, including a central processing unit and a memory, and the memory stores execution instructions. When the remote desktop server is running, the connection between the central processing unit and the memory Inter-communication, the central processing unit executes the execution instruction so that the remote desktop server executes any one of the methods in the embodiments of the first aspect above.
第四方面，本发明实施例还提供了一种计算机可读介质，包含计算机执行指令，所述计算机执行指令用于使远程桌面服务器执行上述第一方面实施例中的任一方法。In a fourth aspect, an embodiment of the present invention further provides a computer-readable medium, which includes computer-executable instructions, and the computer-executable instructions are used to enable the remote desktop server to execute any method in the above-mentioned embodiments of the first aspect.
发明的效果The effect of the invention
本发明实施例提供的方法及装置，能够在对桌面显示图像进行处理之前，通过响应于由用户操作所触发的输入事件，获取与用户操作相关的图形指令，进而获取光标信息，根据光标信息获取光标类型，结合光标信息和光标类型识别出操作场景的类型，进一步根据操作场景的类型选择相应的图像缓存机制和图像压缩算法，从而有效地减小了大量图像数据在发送过程中的不必要的开销，进而能够优化传输带宽。The method and device provided by the embodiments of the present invention can obtain graphics instructions related to user operations by responding to input events triggered by user operations before processing the desktop display images, and then obtain cursor information. Cursor type, combining cursor information and cursor type to identify the type of operation scene, and further select the corresponding image caching mechanism and image compression algorithm according to the type of operation scene, thus effectively reducing unnecessary unnecessary data loss during the sending process of a large amount of image data Overhead, and thus can optimize the transmission bandwidth.
附图说明Description of drawings
图1a为虚拟桌面基础架构场景示意图；Figure 1a is a schematic diagram of a virtual desktop infrastructure scenario;
图1b为远程桌面连接服务场景示意图；Figure 1b is a schematic diagram of a remote desktop connection service scenario;
图2a为实施例1的远程桌面图像处理方法的流程图；Fig. 2a is the flow chart of the remote desktop image processing method of embodiment 1;
图2b为实施例1的远程桌面图像处理方法的另一流程图；Fig. 2b is another flow chart of the remote desktop image processing method of embodiment 1;
图3a为实施例2的远程桌面图像处理装置的结构框图；Fig. 3 a is the structural block diagram of the remote desktop image processing device of embodiment 2;
图3b为实施例2的远程桌面图像处理装置的另一结构框图；Fig. 3b is another structural block diagram of the remote desktop image processing device of Embodiment 2;
图4a为实施例3虚拟桌面基础架构场景的远程桌面服务器的结构示意图；FIG. 4a is a schematic structural diagram of a remote desktop server in a virtual desktop infrastructure scenario in Embodiment 3;
图4b为实施例3远程桌面连接服务场景的远程桌面服务器的结构示意图。Fig. 4b is a schematic structural diagram of a remote desktop server in a remote desktop connection service scenario in Embodiment 3.
具体实施方式detailed description
本发明的远程桌面图像处理方法及装置结合附图及实施例详细说明如下。The remote desktop image processing method and device of the present invention are described in detail as follows with reference to the drawings and embodiments.
远程桌面技术可以实现通过网络远程地操作计算机，并将远程计算机桌面显示的内容（即远程桌面图像）在本地桌面上进行显示。一种远程桌面场景为虚拟桌面基础架构（VirtualDesktop Infrastructure，VDI）场景。虚拟桌面基础架构场景如图1a所示，本实施场景仅是一种示例性说明，但是并不是对本发明应用的具体限定，且图1a中只是给出了与本发明实施例相关的组成部分。图1a中左侧是虚拟桌面的客户端，通常称为瘦客户端（Thin Client，TC），客户端的设备形式可以是普通计算机、平板电脑、智能手机等。客户端通过网络使用远程桌面协议（Remote Desktop Protocol，RDP）访问远程桌面服务。服务器提供了远程桌面的载体，虚拟桌面（Virtual Desktop）以虚拟机（Virtual Machine，VM）的形式存在于服务器上。用户通过客户端访问服务器上分配给该用户的虚拟桌面（即虚拟机），分配给该用户的虚拟机将虚拟桌面（即远程桌面）传输到客户端进行显示。The remote desktop technology can realize the remote operation of the computer through the network, and display the content displayed on the remote computer desktop (that is, the remote desktop image) on the local desktop. One remote desktop scenario is a virtual desktop infrastructure (VirtualDesktop Infrastructure, VDI) scenario. The virtual desktop infrastructure scenario is shown in Figure 1a. This implementation scenario is only an exemplary description, but it does not specifically limit the application of the present invention, and Figure 1a only shows components related to the embodiment of the present invention. The left side of Figure 1a is the client of the virtual desktop, which is usually called a thin client (Thin Client, TC), and the device form of the client can be an ordinary computer, a tablet computer, a smart phone, and the like. The client uses the Remote Desktop Protocol (Remote Desktop Protocol, RDP) to access the remote desktop service through the network. The server provides the carrier of the remote desktop, and the virtual desktop (Virtual Desktop) exists on the server in the form of a virtual machine (Virtual Machine, VM). The user accesses the virtual desktop (that is, the virtual machine) assigned to the user on the server through the client, and the virtual machine assigned to the user transmits the virtual desktop (that is, the remote desktop) to the client for display.
远程桌面另一种场景为远程桌面连接服务。如图1b所示，当某台计算机开启了远程桌面连接功能后，就可以通过网络在另一端控制这台计算机，通过远程桌面功能可以实时的操作这台开启了远程桌面连接功能的计算机。这里，称开启了远程桌面连接功能的计算机为服务器，另一端控制这台服务器的计算机为客户端，客户端的设备形式可以是普通计算机、平板电脑、智能手机等。通过远程桌面连接服务，将服务器桌面（即远程桌面）传输到客户端显示。常见的为微软公司提供的远程桌面连接服务。Another scenario of remote desktop is the remote desktop connection service. As shown in Figure 1b, when a computer has enabled the remote desktop connection function, the computer can be controlled at the other end through the network, and the remote desktop function can be used to operate the computer with the remote desktop connection function enabled in real time. Here, the computer with the remote desktop connection function turned on is called the server, and the computer controlling the server at the other end is called the client, and the device form of the client can be an ordinary computer, a tablet computer, a smart phone, and the like. Through the remote desktop connection service, the server desktop (that is, the remote desktop) is transferred to the client display. common for microsoft The remote desktop connection service provided by the company.
实施例1Example 1
本实施例提供了一种实现桌面图像传输时的远程桌面图像处理方法，如图2a所示，该方法包括：This embodiment provides a remote desktop image processing method for implementing desktop image transmission, as shown in Figure 2a, the method includes:
S1.图形指令获取步骤：响应于由用户操作所触发的输入事件，获取与所述用户操作相关的图形指令。S1. Step of acquiring graphic instructions: in response to an input event triggered by a user operation, acquiring a graphic instruction related to the user operation.
当用户在客户端通过鼠标或键盘等输入设备进行输入操作时，服务器侧响应于由用户操作所触发的输入事件，获取由操作系统提供的基于图形设备接口（Graphics Device Interface，GDI）的图形指令，其中，图形指令包括矢量绘图命令；在响应于由用户操作所触发的输入时，还可以获取图像数据，图像数据包括光标的图像数据和远程桌面上显示的除所述光标外的图像数据。When the user performs an input operation on the client side through an input device such as a mouse or a keyboard, the server side responds to the input event triggered by the user operation and obtains the graphics instruction based on the Graphics Device Interface (GDI) provided by the operating system , wherein the graphics instruction includes a vector drawing command; when responding to an input triggered by a user operation, image data may also be acquired, and the image data includes image data of a cursor and image data other than the cursor displayed on the remote desktop.
S2.光标信息获取步骤，从所述图形指令中获取光标信息，并根据所获取的光标信息判定该用户操作相关的光标的类型。S2. The step of obtaining cursor information, obtaining cursor information from the graphic instruction, and determining the type of the cursor related to the user operation according to the obtained cursor information.
S3.识别步骤，根据所述光标信息、所判定的光标类型识别所述用户操作的操作场景的类型。其中，操作场景的类型能够反映用户在当前场景下进行用户操作所表现出的相关桌面图像变化的特征，不同类型的操作场景具有不同的桌面图像变化特征。S3. Identifying step, identifying the type of the operation scene operated by the user according to the cursor information and the determined cursor type. Wherein, the type of the operation scene can reflect the change characteristics of the relevant desktop image displayed by the user performing user operations in the current scene, and different types of operation scenes have different characteristics of the change of the desktop image.
本发明实施例提供的方法，能够在对桌面显示图像进行处理之前，通过响应于由用户操作所触发的输入事件，获取与用户操作相关的图形指令，进而获取光标信息，根据光标信息获取光标类型，结合光标信息、光标类型识别出操作场景的类型，从而可根据识别出的操作场景的类型选择相应的图像缓存机制和图像压缩算法。The method provided by the embodiment of the present invention can obtain the graphics instruction related to the user operation by responding to the input event triggered by the user operation before processing the desktop display image, and then obtain the cursor information, and obtain the cursor type according to the cursor information Combining the cursor information and cursor type to identify the type of operation scene, so that the corresponding image caching mechanism and image compression algorithm can be selected according to the identified type of operation scene.
本发明实施例提供的方法还可用于根据步骤S3对识别出操作类型的图像数据进一步处理，如图2b所示，包括：The method provided by the embodiment of the present invention can also be used to further process the image data of the identified operation type according to step S3, as shown in Figure 2b, including:
S4.标记步骤，为与所述用户操作相关的图像数据设置操作场景标识，所述操作场景标识用于表示该图像数据对应的操作场景的类型；S4. The marking step is to set an operation scene identifier for the image data related to the user operation, and the operation scene identifier is used to indicate the type of the operation scene corresponding to the image data;
S5.分类处理步骤，根据所述图像数据携带的所述操作场景标识，对所述图像数据进行处理；以及S5. A classification processing step, processing the image data according to the operation scene identifier carried by the image data; and
S6.封装步骤，根据远程桌面协议，封装经步骤S5处理后的图像数据及矢量绘图命令并发送。S6. Encapsulating step, encapsulating and sending the image data and vector drawing commands processed in step S5 according to the remote desktop protocol.
本发明实施例提供的方法进一步地，根据操作场景类型选择相应的图像缓存机制和图像压缩算法，从而有效地减小了大量图像数据在发送过程中的不必要的开销，进而能够优化传输带宽。The method provided by the embodiment of the present invention further selects the corresponding image caching mechanism and image compression algorithm according to the type of operation scene, thereby effectively reducing unnecessary overhead during the sending process of a large amount of image data, thereby optimizing the transmission bandwidth.
具言之，在步骤S2中，主要从GDI指令中的两种指令获取光标信息：一个是DrvSetPointerShape指令，另一个是DrvMovePointer指令。当用户在客户端上的操作引起远程桌面上光标形状的变换时，将触发该DrvSetPointerShape指令，从该指令中可以获取光标的具体信息，包括：光标的形状、尺寸（长宽）等；而用户操作光标在远程桌面上的移动，将触发该DrvMovePointer指令，这种指令记录远程桌面上光标的位置信息以及光标是否隐藏的信息。此外，在步骤S2中，还包括使用哈希（hash）算法计算远程桌面图像数据中所包含的图像（包括光标图像及除光标外的远程桌面上显示的图像）的哈希值。根据上述两种指令所获得的光标信息，结合光标图像的哈希值即可判定用户操作所对应的光标的类型。Specifically, in step S2, the cursor information is mainly obtained from two types of GDI commands: one is the DrvSetPointerShape command, and the other is the DrvMovePointer command. When the user's operation on the client causes the shape of the cursor on the remote desktop to change, the DrvSetPointerShape command will be triggered, and the specific information of the cursor can be obtained from the command, including: the shape, size (length and width) of the cursor, etc.; and the user The movement of the operating cursor on the remote desktop will trigger the DrvMovePointer command, which records the position information of the cursor on the remote desktop and information about whether the cursor is hidden. In addition, in step S2, a hash algorithm is also used to calculate the hash value of the images contained in the remote desktop image data (including the cursor image and images displayed on the remote desktop except the cursor). According to the cursor information obtained by the above two instructions, combined with the hash value of the cursor image, the type of the cursor corresponding to the user operation can be determined.
在本实施例的方法中，优选地将光标类型分为四种：Alpha光标、多色光标、单色光标以及透明光标。其中：In the method of this embodiment, the types of cursors are preferably divided into four types: Alpha cursors, multi-color cursors, monochrome cursors and transparent cursors. in:
Alpha光标在GDI指令中只包含一张彩色图像，远程桌面上显示的箭头，拖动及拉伸等操作对应的光标形状都属于这种光标。常见的Alpha光标例如远程桌面上的单向箭头形光标、窗口拉伸时的双向箭头形光标以及鼠标移动到链接附近时出现的手形光标，等等。The Alpha cursor only contains a color image in the GDI command, and the arrows displayed on the remote desktop, and the cursor shapes corresponding to operations such as dragging and stretching all belong to this kind of cursor. Common Alpha cursors include the one-way arrow-shaped cursor on the remote desktop, the two-way arrow-shaped cursor when the window is stretched, the hand-shaped cursor that appears when the mouse moves near a link, and so on.
多色光标在GDI指令中包含一张彩色图像和一张单色图像。图形指令对图像数据做渲染时会把单色图像作为掩码图，使得该种光标的彩色图像中只有一部分颜色显示出来。这种光标很不常见，只有在一些特殊的应用中会出现，比如在一些播放软件中点击播放媒体文件时的进度条所出现的手形光标，该手形光标与Alpha光标类型的手形光标不同，其出现在一些特殊的软件环境中，这种手形光标有可能是白色的，也有可能是其它特殊的手形形状。A multicolor cursor contains a color image and a monochrome image in the GDI command. When the graphics instruction renders the image data, the monochrome image will be used as a mask image, so that only a part of the color image of this type of cursor will be displayed. This kind of cursor is very uncommon and only appears in some special applications, such as the hand cursor that appears when you click the progress bar of a media file in some playback software. This hand cursor is different from the hand cursor of the Alpha cursor type. Appearing in some special software environments, this kind of hand cursor may be white, or it may be other special hand shapes.
单色光标在GDI指令中只包含一张单色图像。这种光标通常出现在文本编辑的时候，例如：在office软件、记事本中打字时，光标显示为一个“I”形，而进行移动操作时，光标将可能变成其它的形状。常见的单色光标例如打字时的“I”形光标及其各种变形、将鼠标移动到左侧空白处时的光标以及滚动鼠标滑轮所出现的特殊的竖直双向三角箭头形光标，等等。A monochrome cursor contains only a monochrome image in a GDI command. This kind of cursor usually appears when editing text, for example: when typing in office software or notepad, the cursor is displayed as an "I" shape, and when moving, the cursor may change into other shapes. Common monochromatic cursors such as the "I"-shaped cursor and its various deformations when typing, the cursor when the mouse is moved to the left blank space, and the special vertical two-way triangular arrow-shaped cursor that appears when the mouse wheel is rolled, etc. .
透明光标在GDI指令中不会包含任何图像，每当这种类型的光标发送到客户端时，客户端会使得光标透明显示。Transparent cursors do not contain any images in GDI commands, and whenever this type of cursor is sent to the client, the client will make the cursor transparent.
以下通过四种不同的操作场景对步骤S 3的识别过程进行说明。应该注意的是，本实施例的方法可根据需要应用于其它操作场景，并不仅限于这四种场景。该四种不同的操作场景分别为：打字场景、窗口拉伸场景、视频播放场景以及滚动场景。The identification process of step S3 will be described below through four different operation scenarios. It should be noted that the method in this embodiment can be applied to other operation scenarios as required, and is not limited to these four scenarios. The four different operation scenarios are respectively: a typing scenario, a window stretching scenario, a video playing scenario, and a scrolling scenario.
对于打字场景：根据光标信息进行判断，若光标类型为单色光标，此时，一定对应的是文本编辑的状态。在这种场景下，每当用户开始打字时，光标就会消失，DrvMovePointer指令中将出现光标隐藏信息。换言之，若单色光标类型且存在光标隐藏信息即可判定用户的操作场景为打字场景。For the typing scene: judge according to the cursor information, if the cursor type is a monochrome cursor, at this time, it must correspond to the state of text editing. In this scenario, whenever the user starts typing, the cursor will disappear, and the cursor hiding information will appear in the DrvMovePointer command. In other words, if the cursor type is monochrome and there is cursor hidden information, it can be determined that the user's operation scene is a typing scene.
对于窗口拉伸场景：在窗口拉伸的场景下，会出现光标位置的变化。因此，若光标类型为Alpha光标，光标图像的哈希值与斜向的双向箭头形的光标的哈希值吻合，且光标的位置发生了变化，即可判定用户的操作场景为窗口拉伸场景。For the window stretching scenario: In the window stretching scenario, the cursor position will change. Therefore, if the cursor type is Alpha cursor, the hash value of the cursor image matches the hash value of the oblique two-way arrow-shaped cursor, and the position of the cursor changes, it can be determined that the user's operation scene is a window stretching scene .
对于视频播放场景：在该场景下，静止操作状态持续一段时间后光标将隐藏，以在视频播放时去除光标的影响。因此，若光标类型为Alpha光标，且存在光标隐藏信息，即可判定用户操作的场景为视频播放场景。本实施例中所指的视频播放包括PPT以及通常意义上的视频播放。For the video playback scenario: In this scenario, the cursor will be hidden after the static operation state lasts for a period of time, so as to remove the influence of the cursor when the video is playing. Therefore, if the cursor type is Alpha cursor and there is cursor hidden information, it can be determined that the scene operated by the user is a video playback scene. The video playback referred to in this embodiment includes PPT and video playback in the usual sense.
对于滚动场景：当光标类型为单色光标，且光标图像的哈希值与特殊形状的双向三角箭头光标的哈希值吻合，即可判定用户操作的场景为鼠标滚轮的滚动场景。本实施例中所指的滚动场景包括文本编辑、PPT播放以及网页浏览等应用。For the scrolling scene: When the cursor type is a monochrome cursor, and the hash value of the cursor image matches the hash value of the specially shaped bidirectional triangular arrow cursor, it can be determined that the scene operated by the user is a scrolling scene of the mouse wheel. The scrolling scene referred to in this embodiment includes applications such as text editing, PPT playing, and web page browsing.
步骤S3完成对用户操作的操作场景的识别后，在步骤S4中对该用户操作相关的图像数据进行对应的操作场景的标记，以在后续的图像数据传输过程中，能够根据不同的场景采取不同的处理，进而减小图像数据传输过程中产生的不必要的开销，以优化传输带宽。After step S3 completes the identification of the operation scene operated by the user, in step S4, the image data related to the user operation is marked with the corresponding operation scene, so that in the subsequent image data transmission process, different scenes can be adopted according to different scenes. processing, thereby reducing unnecessary overhead generated during image data transmission to optimize transmission bandwidth.
具言之，在步骤S5中，根据图像数据携带的操作场景标识，对该操作场景的图像数据进行处理，具体包括：根据操作场景标识选择相应的图像缓存机制和图像压缩算法，以及根据图像缓存机制和图像压缩算法对该图像数据进行处理。仍以上述的四种操作场景为例：Specifically, in step S5, according to the operation scene identification carried by the image data, the image data of the operation scene is processed, specifically including: selecting the corresponding image caching mechanism and image compression algorithm according to the operation scene identification, and according to the image caching Mechanisms and image compression algorithms process the image data. Still take the above four operation scenarios as examples:
对于打字场景：该操作场景主要发生在word或记事本等编辑类应用中。在这种操作场景中，远程桌面显示图像主要以图形文字为主。对于这样的场景，本实施例的方法将整张较大的远程桌面显示图像分割成多个大小相同的小块图像进行缓存，另外，由于此操作场景下会存在多个区域为同一种颜色的空白区域，还存在一个图像块内的字和颜色与另一图像块完全相同的区域。因此，对于同样的小块图像将不再进行传输而只传输图像块中图像的哈希值。同时，对于此种操作场景下的图像数据，本实施例的方法采用适合压缩文字图片的压缩算法，比如字典算法、zlib算法，而像JPEG、H.264这类图像压缩算法如果用来压缩文字图片，不但会使得字体模糊难以辨认，而且无法达到合适的带宽范围，也很浪费服务端的性能。For typing scenarios: This operation scenario mainly occurs in editing applications such as Word or Notepad. In this operation scenario, the image displayed on the remote desktop is mainly based on graphics and text. For such a scenario, the method of this embodiment divides the entire larger remote desktop display image into multiple small images of the same size for caching. In addition, since there will be multiple regions of the same color in this operation scenario In the blank area, there is also an area where the words and colors in one image block are exactly the same as those in another image block. Therefore, for the same small image block, only the hash value of the image in the image block will be transmitted. Simultaneously, for the image data under this kind of operating scene, the method of this embodiment adopts the compression algorithm suitable for compressing text picture, such as dictionary algorithm, zlib algorithm, and image compression algorithm such as JPEG, H.264 if used to compress text Pictures will not only make the fonts blurry and difficult to read, but also fail to reach the appropriate bandwidth range, and waste the performance of the server.
对于窗口拉伸场景：在该操作场景下，远程桌面上窗口的范围会发生变化，其边框会根据一定比例放大缩小。但是同一种颜色的空白区域依然会大量存在，所以本实施例的方法中对于该操作场景下的图像数据仍采用切割成小图的分块缓存机制。但是，该操作场景下不存在大量的文字图片，而是有很多图标类的图像，因此，不适合采用文字类的压缩算法，对此，本实施例的方法采用压缩混合图像的算法对此场景下的图像数据进行压缩，例如sfalic算法。For the window stretching scenario: In this operation scenario, the scope of the window on the remote desktop will change, and its border will be enlarged or reduced according to a certain ratio. However, there will still be a large number of blank areas of the same color, so in the method of this embodiment, the block buffering mechanism of cutting into small images is still used for the image data in this operation scenario. However, there are not a large number of text pictures in this operation scene, but there are many icon-like images, so it is not suitable to use a text-based compression algorithm. For this, the method of this embodiment adopts an algorithm for compressing mixed images for this scene. Under the image data compression, such as sfalic algorithm.
对于视频播放场景：本实施例的方法优选地对该操作场景下的图像数据采用H.264的帧间预测对视频进行编码。For a video playing scene: the method of this embodiment preferably uses H.264 inter-frame prediction to encode the video for the image data in the operating scene.
对于滚动场景：在此操作场景下，当按下鼠标滚动键使得当前页面发生滚动时，整个页面实际上发生了整体的上移或下移，所以在更新页面（也即更新远程桌面显示图像）时实际上所进行的是根据需要按行增加图像的下面或上面几行像素，再消除掉图像的上面或下面几行像素，由于向上或向下偏移的距离不确定，因此，不适合采用分块的缓存机制。本实施例的方法对于此操作场景下的远程桌面图像数据进行按行分割再缓存的方式，这样更有利于接收端分割后图像的命中，该缓存机制也称作超级缓存（supercache）。For scrolling scenarios: In this operation scenario, when the mouse scroll key is pressed to scroll the current page, the entire page actually moves up or down as a whole, so when updating the page (that is, updating the remote desktop display image) What is actually done is to increase the pixels of the lower or upper rows of the image by rows according to the needs, and then eliminate the pixels of the upper or lower rows of the image. Since the distance of the upward or downward offset is uncertain, it is not suitable to use Block caching mechanism. The method of this embodiment divides and caches the remote desktop image data by row in this operation scenario, which is more conducive to the hit of the divided image at the receiving end. This caching mechanism is also called supercache (supercache).
实施例2Example 2
如图3a所示，本实施例还提供了一种远程桌面图像处理装置，该装置包括：图形指令获取单元100、光标信息获取单元200、以及识别单元300。在本实施例的装置中：As shown in FIG. 3 a , this embodiment also provides a remote desktop image processing device, which includes: a graphics command acquisition unit 100 , a cursor information acquisition unit 200 , and an identification unit 300 . In the device of this embodiment:
图形指令获取单元100，用于响应于由用户操作所触发的输入事件，获取与所述用户操作相关的图形指令，并将所述图形指令发送至光标信息获取单元200。The graphic instruction acquiring unit 100 is configured to acquire a graphic instruction related to the user operation in response to an input event triggered by the user operation, and send the graphic instruction to the cursor information acquiring unit 200 .
当用户在客户端通过鼠标或键盘等输入设备进行输入操作时，服务器侧响应于由用户操作所触发的输入事件，获取由操作系统提供的基于图形设备接口（Graphics Device Interface，GDI）的图形指令。其中，图形指令包含矢量绘图命令。在响应由用户操作所触发的输入时，还可以获取图像数据，图像数据包括光标的图像数据和远程桌面上显示的除所述光标外的图像数据。When the user performs an input operation on the client side through an input device such as a mouse or a keyboard, the server side responds to the input event triggered by the user operation and obtains the graphics instruction based on the Graphics Device Interface (GDI) provided by the operating system . Among them, graphics instructions include vector drawing commands. When responding to an input triggered by a user operation, image data may also be acquired, and the image data includes image data of the cursor and image data displayed on the remote desktop other than the cursor.
光标信息获取单元200用于接收来自图形指令获取单元100发送的图形指令，从该图形指令中获取光标信息，根据所获取的光标信息判定该用户操作相关的光标的类型，并将光标信息和光标类型发送至识别单元300。The cursor information acquisition unit 200 is configured to receive the graphics instruction sent from the graphics instruction acquisition unit 100, acquire cursor information from the graphics instruction, determine the type of the cursor related to the user operation according to the acquired cursor information, and combine the cursor information and the cursor The type is sent to the recognition unit 300 .
识别单元300用于接收来自光标信息获取单元200发送的光标信息和光标类型，根据光标信息、所判定的光标类型识别该用户操作的操作场景。The identification unit 300 is configured to receive the cursor information and cursor type sent from the cursor information acquisition unit 200, and identify the operation scene operated by the user according to the cursor information and the determined cursor type.
实施例提供的远程桌面图像处理装置，能够在对桌面显示图像进行处理之前，通过响应于由用户操作所触发的输入事件，获取与用户操作相关的图形指令，进而获取光标信息，根据光标信息获取光标类型，结合光标信息、光标类型识别出操作场景的类型，从而可根据识别出的操作场景的类型选择相应的图像缓存机制和图像压缩算法。The remote desktop image processing device provided in the embodiment can obtain the graphics instruction related to the user operation by responding to the input event triggered by the user operation before processing the desktop display image, and then obtain the cursor information, and obtain the cursor information according to the cursor information. Cursor type, combining the cursor information and cursor type to identify the type of operation scene, so that the corresponding image caching mechanism and image compression algorithm can be selected according to the identified type of operation scene.
在上述装置实施例基础上，本实施例还提供了另外一种远程桌面图像处理装置，如图3b所示，该装置还包括标记单元400。On the basis of the above device embodiments, this embodiment also provides another remote desktop image processing device, as shown in FIG. 3 b , the device further includes a marking unit 400 .
标记单元400，用于根据识别单元300识别的所述操作场景的类型，为与该用户操作相关的图像数据设置操作场景标识，从而在对该图像数据进行处理时，可以准确识别图像数据类型。The marking unit 400 is configured to set an operation scene identifier for the image data related to the user operation according to the type of the operation scene identified by the identification unit 300, so that the image data type can be accurately identified when the image data is processed.
本实施例提供的远程桌面图像处理装置，在包括标记单元400的基础上，还包括分类处理单元500，如图3b所示，分类处理单元500用于根据该图像数据携带的操作场景标识，对该图像数据进行处理。其中，分类处理单元500具体用于：根据该操作场景标识选择相应的图像缓存机制和图像压缩算法，并根据该图像缓存机制和该图像压缩算法对该图像数据进行处理。根据操作场景类型选择相应的图像缓存机制和图像压缩算法，从而有效地减小了大量图像数据在发送过程中的不必要的开销，进而能够优化传输带宽。The remote desktop image processing device provided in this embodiment further includes a classification processing unit 500 in addition to the marking unit 400. As shown in FIG. 3b, the classification processing unit 500 is used to identify the The image data is processed. Wherein, the classification processing unit 500 is specifically configured to: select a corresponding image cache mechanism and image compression algorithm according to the operation scene identifier, and process the image data according to the image cache mechanism and the image compression algorithm. Select the corresponding image caching mechanism and image compression algorithm according to the type of operation scene, thereby effectively reducing the unnecessary overhead of sending a large amount of image data, and then optimizing the transmission bandwidth.
本实施例提供的远程桌面图像处理装置，在包括分类处理单元500的基础上，还包括封装单元600，如图3b所示。封装单元600用于从分类处理单元500接收处理后的图像数据，根据远程桌面协议，封装该处理后的图像数据并发送。The remote desktop image processing apparatus provided in this embodiment further includes an encapsulation unit 600 in addition to the classification processing unit 500, as shown in FIG. 3b. The encapsulation unit 600 is used for receiving the processed image data from the classification processing unit 500, and encapsulating and sending the processed image data according to the remote desktop protocol.
实施例3Example 3
结合图1a，本发明实施例提供一种远程桌面服务器106a。如图4a所示，该远程桌面服务器106a由三大组成部分组成，分别为：硬件平台150、虚拟机监视器平台（Hypervisor）151以及虚拟机执行空间152。其中：Referring to FIG. 1a, the embodiment of the present invention provides a remote desktop server 106a. As shown in FIG. 4 a , the remote desktop server 106 a is composed of three major components, namely: a hardware platform 150 , a virtual machine monitor platform (Hypervisor) 151 and a virtual machine execution space 152 . in:
硬件平台150中包括远程桌面服务器106a物理结构中的硬盘160、CPU 161、网卡162、以及存储器RAM 163等。虚拟机监视器平台151和虚拟机执行空间152分别提供了虚拟机的硬件平台和软件平台。虚拟机监视器平台151包含了多个虚拟机监视器170a、170b…170n，虚拟机监视器平台151是物理硬件和虚拟机之间的中间层，负责协调各个虚拟机对服务器硬件的访问。虚拟机监视器（VMM）170a中实现虚拟机（VM）180a的虚拟硬件平台171，虚拟硬件平台171中包含了虚拟出来的硬盘172、CPU 173、存储器RAM 174、显卡175和网卡176等。虚拟机180a的软件系统运行在虚拟硬件平台171上。虚拟机执行空间152中运行了多个虚拟机180a、180b…180n，客户操作系统181运行于虚拟机180a中，客户操作系统181包含了一个设备驱动层182，安装了显卡驱动183、以及网卡驱动184等，当用户在客户端访问远程桌面时，虚拟机180a分配给该客户端的用户。The hardware platform 150 includes a hard disk 160, a CPU 161, a network card 162, and a memory RAM 163 in the physical structure of the remote desktop server 106a. The virtual machine monitor platform 151 and the virtual machine execution space 152 respectively provide the hardware platform and software platform of the virtual machine. The virtual machine monitor platform 151 includes multiple virtual machine monitors 170a, 170b...170n. The virtual machine monitor platform 151 is an intermediate layer between physical hardware and virtual machines, responsible for coordinating the access of each virtual machine to server hardware. The virtual machine monitor (VMM) 170a implements the virtual hardware platform 171 of the virtual machine (VM) 180a. The virtual hardware platform 171 includes a virtual hard disk 172, CPU 173, RAM 174, graphics card 175, and network card 176. The software system of the virtual machine 180 a runs on the virtual hardware platform 171 . A plurality of virtual machines 180a, 180b...180n are run in the virtual machine execution space 152, and the guest operating system 181 runs in the virtual machine 180a. The guest operating system 181 includes a device driver layer 182, and a graphics card driver 183 and a network card driver are installed. 184, etc., when the user accesses the remote desktop at the client, the virtual machine 180a is allocated to the user of the client.
使用本实施例的远程桌面服务器106a实现本发明实施例1的方法的过程为：当用户在客户端通过鼠标或键盘等输入设备进行输入操作时，由用户操作所触发的输入事件发送到远程桌面服务器侧后，分配给客户端用户的虚拟机180a响应于由用户操作所触发的该输入事件，显卡驱动183获取由操作系统181提供的基于图形设备接口（Graphics Device Interface，GDI）186的图形指令。其中，图形指令包含矢量绘图命令。在响应于由用户操作所触发的输入时，还可以获取图像数据，图像数据包括光标的图像数据和远程桌面上显示的除所述光标外的图像数据。显卡驱动183从与该用户操作相关的图形指令中获取光标信息，并根据所获取的光标信息判定该用户操作相关的光标的类型；根据光标信息、所判定的光标类型识别所述用户操作的操作场景，其中，操作场景的类型能够反映用户在当前场景下操作VM 180a所表现出的桌面图像变化的特征，不同类型的操作场景具有不同的桌面图像变化特征；为与该用户操作相关的图像数据设置操作场景标识，该操作场景标识用于表示该图像数据对应的操作场景的类型；以及将图像数据发送到桌面协议服务端187。桌面协议服务端187根据图像数据携带的操作场景标识，对该图像数据进行处理，具体包括：根据操作场景标识选择相应的图像缓存机制和图像压缩算法，以及根据图像缓存机制和图像压缩算法对该图像数据进行处理；根据远程桌面协议，封装经处理后的图像数据及矢量绘图命令并发送到客户端进行显示。具体实施过程请参见本发明实施例1的具体描述，在此不再赘述。The process of using the remote desktop server 106a of this embodiment to implement the method of Embodiment 1 of the present invention is: when the user performs an input operation on the client terminal through an input device such as a mouse or a keyboard, the input event triggered by the user operation is sent to the remote desktop After the server side, the virtual machine 180a assigned to the client user responds to the input event triggered by the user operation, and the graphics card driver 183 obtains the graphics instruction based on the graphics device interface (Graphics Device Interface, GDI) 186 provided by the operating system 181 . Among them, graphics instructions include vector drawing commands. When responding to an input triggered by a user operation, image data may also be acquired, and the image data includes image data of the cursor and image data displayed on the remote desktop other than the cursor. Graphics card driver 183 acquires cursor information from graphics instructions related to the user operation, and determines the type of cursor related to the user operation according to the acquired cursor information; identifies the operation of the user operation according to the cursor information and the determined cursor type Scenario, wherein, the type of the operation scenario can reflect the characteristics of the desktop image change shown by the user operating the VM 180a in the current scenario, and different types of operation scenarios have different characteristics of the desktop image change; it is the image data related to the user operation Set an operation scene identifier, which is used to indicate the type of operation scenario corresponding to the image data; and send the image data to the desktop protocol server 187 . The desktop protocol server 187 processes the image data according to the operation scene identifier carried by the image data, specifically including: selecting a corresponding image cache mechanism and image compression algorithm according to the operation scenario identifier, and selecting the corresponding image cache mechanism and image compression algorithm according to the image cache mechanism and image compression algorithm. The image data is processed; according to the remote desktop protocol, the processed image data and vector drawing commands are encapsulated and sent to the client for display. For the specific implementation process, please refer to the specific description of Embodiment 1 of the present invention, and details are not repeated here.
本发明实施例提供的远程桌面服务器106a的中央处理器CPU 161执行存储器163中存储的计算机指令，从而使该远程桌面服务器106a执行本发明实施例所描述的方法。该计算机指令可以存储在计算机可读介质中，对计算机可读介质类型本发明对此不作限定。本发明实施例及附图只给出了与本发明相关的组件与连接关系，其他未描述组件及连接关系的省略并不代表对本发明应用的具体限定；同时，本发明实施例只是对虚拟桌面基础架构一种场景的描述，对于其他架构类型的虚拟桌面场景本发明同样适用，在此不再赘述。The central processing unit CPU 161 of the remote desktop server 106a provided in the embodiment of the present invention executes the computer instructions stored in the memory 163, so that the remote desktop server 106a executes the method described in the embodiment of the present invention. The computer instructions may be stored in a computer-readable medium, and the type of the computer-readable medium is not limited in the present invention. The embodiments of the present invention and the accompanying drawings only show the components and connections related to the present invention, and the omission of other undescribed components and connections does not represent a specific limitation on the application of the present invention; at the same time, the embodiments of the present invention are only for virtual desktops The description of one scenario of the infrastructure is also applicable to the virtual desktop scenarios of other architecture types, and will not be repeated here.
结合图1b以及图4b所示的另一种远程桌面服务器106b，如图4b所示，即在远程桌面连接服务中的远程桌面服务器106b，使用本实施例提供的远程桌面服务器106b实现本发明实施例1的方法的过程为：当用户在客户端通过鼠标或键盘等输入设备进行输入操作时，由用户操作所触发的输入事件发送到远程桌面服务器106b后，远程桌面服务器106b响应于由用户操作所触发的该输入事件，显卡驱动204获取由操作系统202提供的基于图形设备接口（Graphics Device Interface，GDI）203的图形指令。其中，图形指令包含矢量绘图命令；在响应于由用户操作所触发的输入时，还可以获取图像数据，图像数据包括光标的图像数据和远程桌面上显示的除所述光标外的图像数据。显卡驱动204从与该用户操作相关的图形指令中获取光标信息，并根据所获取的光标信息判定该用户操作相关的光标的类型；根据光标信息、所判定的光标类型识别所述用户操作的操作场景，其中，操作场景的类型能够反映出用户在当前场景下操作远程桌面服务器106b所表现出的桌面图像变化的特征，不同类型的操作场景具有不同的桌面图像变化特征；为与该用户操作相关的图像数据设置操作场景标识，该操作场景标识用于表示该图像数据对应的操作场景的类型。根据图像数据携带的操作场景标识，对该图像数据进行处理，具体包括根据操作场景标识选择相应的图像缓存机制和图像压缩算法，以及根据图像缓存机制和图像压缩算法对该图像数据进行处理。根据远程桌面协议，封装经处理后的图像数据及矢量绘图命令并通过网卡4000发送到客户端进行显示。具体实施过程请参见本发明实施例1的具体描述，在此不再赘述。In combination with another remote desktop server 106b shown in FIG. 1b and FIG. 4b, as shown in FIG. 4b, that is, the remote desktop server 106b in the remote desktop connection service, use the remote desktop server 106b provided in this embodiment to realize the implementation of the present invention The process of the method in Example 1 is: when the user performs an input operation on the client terminal through an input device such as a mouse or a keyboard, after the input event triggered by the user operation is sent to the remote desktop server 106b, the remote desktop server 106b responds to the input event triggered by the user operation. When the input event is triggered, the graphics card driver 204 obtains a graphics instruction based on a graphics device interface (Graphics Device Interface, GDI) 203 provided by the operating system 202 . Wherein, the graphics instruction includes a vector drawing command; when responding to an input triggered by a user operation, image data may also be obtained, and the image data includes image data of a cursor and image data other than the cursor displayed on the remote desktop. Graphics card driver 204 acquires cursor information from graphics instructions related to the user operation, and determines the type of the cursor related to the user operation according to the acquired cursor information; identifies the operation of the user operation according to the cursor information and the determined cursor type Scenario, wherein, the type of the operation scenario can reflect the characteristics of the desktop image change shown by the user operating the remote desktop server 106b in the current scenario, and different types of operation scenarios have different characteristics of the desktop image change; it is related to the user operation An operation scene identifier is set for the image data, and the operation scene identifier is used to indicate the type of the operation scenario corresponding to the image data. Processing the image data according to the operation scene identifier carried by the image data, specifically including selecting a corresponding image cache mechanism and image compression algorithm according to the operation scenario identifier, and processing the image data according to the image cache mechanism and image compression algorithm. According to the remote desktop protocol, the processed image data and vector drawing commands are encapsulated and sent to the client through the network card 4000 for display. For the specific implementation process, please refer to the specific description of Embodiment 1 of the present invention, and details are not repeated here.
本发明实施例提供的远程桌面服务器106b的中央处理器CPU 3000执行存储器2000中存储的计算机指令，从而使该远程桌面服务器106b执行本发明实施例一所描述的方法。该计算机指令可以存储在计算机可读介质中，对计算机可读介质类型本发明对此不作限定。The central processing unit CPU 3000 of the remote desktop server 106b provided in the embodiment of the present invention executes the computer instructions stored in the memory 2000, so that the remote desktop server 106b executes the method described in the first embodiment of the present invention. The computer instructions may be stored in a computer-readable medium, and the type of the computer-readable medium is not limited in the present invention.
本领域普通技术人员可以意识到，结合本文中所公开的实施例描述的各示例的单元及算法步骤，能够以电子硬件、或者计算机软件和电子硬件的结合来实现。这些功能究竟以硬件还是软件方式来执行，取决于技术方案的特定应用和设计约束条件。专业技术人员可以对每个特定的应用来使用不同方法来实现所描述的功能，但是这种实现不应认为超出本发明的范围。Those skilled in the art can appreciate that the units and algorithm steps of the examples described in conjunction with the embodiments disclosed herein can be implemented by electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are executed by hardware or software depends on the specific application and design constraints of the technical solution. Skilled artisans may use different methods to implement the described functions for each specific application, but such implementation should not be regarded as exceeding the scope of the present invention.
所述功能如果以软件功能单元的形式实现并作为独立的产品销售或使用时，可以存储在一个计算机可读取存储介质中。基于这样的理解，本发明的技术方案本质上或者说对现有技术做出贡献的部分或者该技术方案的部分可以以软件产品的形式体现出来，该计算机软件产品存储在一个存储介质中，包括若干指令用以使得一台计算机设备（可以是个人计算机，服务器，或者网络设备等）执行本发明各个实施例所述方法的全部或部分步骤。而前述的存储介质包括：U盘、移动硬盘、只读存储器（ROM，Read-Only Memory）、随机存取存储器（RAM，RandomAccess Memory）、磁碟或者光盘等各种可以存储程序代码的介质。If the functions described above are realized in the form of software function units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the essence of the technical solution of the present invention or the part that contributes to the prior art or the part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including Several instructions are used to make a computer device (which may be a personal computer, a server, or a network device, etc.) execute all or part of the steps of the methods described in various embodiments of the present invention. The aforementioned storage media include: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), magnetic disk or optical disk, and other media capable of storing program codes.
以上实施方式仅用于说明本发明，而并非对本发明的限制，有关技术领域的普通技术人员，在不脱离本发明的精神和范围的情况下，还可以做出各种变化和变型，因此所有等同的技术方案也属于本发明的范畴，本发明的专利保护范围应由权利要求限定。The above embodiments are only used to illustrate the present invention, but not to limit the present invention. Those of ordinary skill in the relevant technical field can make various changes and modifications without departing from the spirit and scope of the present invention. Therefore, all Equivalent technical solutions also belong to the category of the present invention, and the scope of patent protection of the present invention should be defined by the claims.
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201210430561.6ACN103412701B (en) | 2012-11-01 | 2012-11-01 | remote desktop image processing method and device |
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201210430561.6ACN103412701B (en) | 2012-11-01 | 2012-11-01 | remote desktop image processing method and device |
| Publication Number | Publication Date |
|---|---|
| CN103412701A CN103412701A (en) | 2013-11-27 |
| CN103412701Btrue CN103412701B (en) | 2016-08-31 |
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201210430561.6AActiveCN103412701B (en) | 2012-11-01 | 2012-11-01 | remote desktop image processing method and device |
| Country | Link |
|---|---|
| CN (1) | CN103412701B (en) |
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104765636B (en)* | 2014-01-02 | 2018-05-04 | 华为技术有限公司 | A kind of synthetic method and device of remote desktop image |
| CN105227603A (en)* | 2014-06-23 | 2016-01-06 | 中兴通讯股份有限公司 | virtual desktop image processing method, device, virtual desktop server and thin terminal |
| CN104768009B (en)* | 2015-03-19 | 2018-01-05 | 中国科学院信息工程研究所 | A kind of image transfer method under desktop virtualization SPICE protocol |
| CN105978963A (en)* | 2016-05-09 | 2016-09-28 | 北京启迪思创科技有限公司 | Remote desktop display method, device and system |
| CN111708533B (en)* | 2016-08-31 | 2023-01-06 | 华为云计算技术有限公司 | Method and device for setting mouse display state in application thin client |
| CN108184118A (en)* | 2016-12-08 | 2018-06-19 | 中兴通讯股份有限公司 | Cloud desktop contents encode and coding/decoding method and device, system |
| CN107343026A (en)* | 2017-06-12 | 2017-11-10 | 苏睿 | Remote processing method, apparatus and equipment |
| CN107608787A (en)* | 2017-08-28 | 2018-01-19 | 西安万像电子科技有限公司 | Server |
| EP3570164B1 (en)* | 2018-05-14 | 2023-04-26 | Schneider Electric Industries SAS | Method and system for generating a mobile application from a desktop application |
| CN109701265A (en)* | 2018-12-04 | 2019-05-03 | 上海达龙信息科技有限公司 | Remote desktop cursor mode switching method, system, client and remote server |
| CN110392262B (en)* | 2019-07-03 | 2022-04-26 | 锐捷网络股份有限公司 | Method and device for compressing virtual desktop image |
| CN110740316A (en)* | 2019-09-09 | 2020-01-31 | 西安万像电子科技有限公司 | Data coding method and device |
| CN111447450A (en)* | 2020-03-16 | 2020-07-24 | 西安万像电子科技有限公司 | Data processing method and device |
| CN113625923B (en)* | 2020-05-06 | 2024-08-09 | 上海达龙信息科技有限公司 | Remote cloud desktop-based mouse processing method and device, storage medium and equipment |
| CN111913675B (en)* | 2020-07-30 | 2024-12-27 | 维沃移动通信有限公司 | Display control method and device, electronic device and computer-readable storage medium |
| CN113124986A (en)* | 2021-04-15 | 2021-07-16 | 天津港集装箱码头有限公司 | Remote virtual simulation method and system for truck scale |
| CN116048329A (en)* | 2022-12-30 | 2023-05-02 | 华为技术有限公司 | Cursor display method and related equipment |
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1538659A (en)* | 2003-04-17 | 2004-10-20 | �Ҵ���˾ | Remote support method and system for computer and other electronic device |
| CN1748196A (en)* | 2002-10-07 | 2006-03-15 | 米厄里戈有限公司 | Cursors for electronic devices |
| CN101506791A (en)* | 2006-08-23 | 2009-08-12 | 论理工场株式会社 | Control method for controlling remote computer |
| CN102027464A (en)* | 2008-05-12 | 2011-04-20 | 微软公司 | Virtual desktop view scrolling |
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20090113353A1 (en)* | 2007-10-31 | 2009-04-30 | Ravi Prakash Bansal | Cursor locator |
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1748196A (en)* | 2002-10-07 | 2006-03-15 | 米厄里戈有限公司 | Cursors for electronic devices |
| CN1538659A (en)* | 2003-04-17 | 2004-10-20 | �Ҵ���˾ | Remote support method and system for computer and other electronic device |
| CN101506791A (en)* | 2006-08-23 | 2009-08-12 | 论理工场株式会社 | Control method for controlling remote computer |
| CN102027464A (en)* | 2008-05-12 | 2011-04-20 | 微软公司 | Virtual desktop view scrolling |
| Publication number | Publication date |
|---|---|
| CN103412701A (en) | 2013-11-27 |
| Publication | Publication Date | Title |
|---|---|---|
| CN103412701B (en) | remote desktop image processing method and device | |
| EP1955187B1 (en) | Multi-user display proxy server | |
| US12067409B2 (en) | Streaming per-pixel transparency information using transparency-agnostic video codecs | |
| CN112486609B (en) | Virtual display card implementation method and device based on cloud desktop | |
| US7899864B2 (en) | Multi-user terminal services accelerator | |
| KR101773638B1 (en) | Methods for interfacing with a virtualized computing service over a network using a lightweight client | |
| US10331769B1 (en) | Interaction based prioritized retrieval of embedded resources | |
| CN102566910B (en) | Virtual desktop system and method for refreshing on-screen display of virtual desktop client | |
| CN108388460A (en) | Long-range real-time rendering platform construction method based on graphics cluster | |
| CN103312814B (en) | Method for establishing VNC hidden channel between cloud management platform and virtual machine terminal user | |
| US20160044115A1 (en) | Remote browsing session management | |
| US11044486B2 (en) | Split framebuffer encoding | |
| US10546038B2 (en) | Intelligent browser-based display tiling | |
| US11016653B2 (en) | Methods and system for locally generated gesture and transition graphics interaction with terminal control services | |
| TWI495330B (en) | System and method for efficiently streaming digital video | |
| CN104866318A (en) | Method and device for displaying label pages in multiple windows | |
| CN114968152B (en) | Method for reducing VIRTIO-GPU extra performance loss | |
| US9460481B2 (en) | Systems and methods for processing desktop graphics for remote display | |
| CN114064182A (en) | Low-cost desktop virtualization system and operation method | |
| CN113835816A (en) | Virtual machine desktop display method, device, equipment and readable storage medium | |
| Wen et al. | PaperWork: Exploring the Potential of Electronic Paper on Office Work | |
| Moravapalle et al. | Peek: A mobile-to-mobile remote computing protocol for smartphones and tablets | |
| CN120070147A (en) | Image processing method and device, electronic equipment and storage medium | |
| TWI598817B (en) | Multi-user computer system | |
| HK1173801A (en) | Multi-user terminal services accelerator |
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| C14 | Grant of patent or utility model | ||
| GR01 | Patent grant | ||
| TR01 | Transfer of patent right | ||
| TR01 | Transfer of patent right | Effective date of registration:20220215 Address after:550025 Huawei cloud data center, jiaoxinggong Road, Qianzhong Avenue, Gui'an New District, Guiyang City, Guizhou Province Patentee after:Huawei Cloud Computing Technologies Co.,Ltd. Address before:518129 Bantian HUAWEI headquarters office building, Longgang District, Guangdong, Shenzhen Patentee before:HUAWEI TECHNOLOGIES Co.,Ltd. |