CN108388460B - Remote real-time rendering platform construction method based on graphic cluster - Google Patents

Abstract

Translated fromChinese

本发明公开了一种基于图形集群的远程实时渲染平台构建方法，包括利用Docker架构在图形集群内部实现虚拟桌面系统的部署；在虚拟桌面系统中嵌入VNC服务端，VNC客户端访问所述VNC服务端后，通过VNC协议建立远程桌面连接；图形集群根据远程桌面连接情况进行动态负载均衡，建立图形集群动态任务调度与管理机制；通过图形集群动态任务调度与管理机制，在平台客户端与图形集群服务端间建立实时图像传输通道,综合集成基于图形集群的远程实时渲染平台。本发明针对大量远程用户基于不同需求的渲染应用问题，根据面向多用户的异步分布式工作模式，依托图形集群硬件环境，利用虚拟化手段来满足不同用户按需使用的需求，既方便管理又确保了数据安全。

The invention discloses a method for constructing a remote real-time rendering platform based on a graphics cluster. The method includes utilizing a Docker architecture to realize the deployment of a virtual desktop system inside the graphics cluster; embedding a VNC server in the virtual desktop system, and the VNC client accesses the VNC service After the terminal, a remote desktop connection is established through the VNC protocol; the graphics cluster performs dynamic load balancing according to the remote desktop connection, and establishes a dynamic task scheduling and management mechanism for the graphics cluster; through the dynamic task scheduling and management mechanism for the graphics cluster, the platform client and graphics cluster A real-time image transmission channel is established between servers, and a remote real-time rendering platform based on graphics cluster is integrated. Aiming at the rendering application problem of a large number of remote users based on different requirements, the invention relies on the multi-user-oriented asynchronous distributed work mode, relies on the graphics cluster hardware environment, and uses virtualization means to meet the needs of different users on-demand, which is convenient for management and ensures data security.

Description

Remote real-time rendering platform construction method based on graphic cluster

Technical Field

The invention belongs to the technical field of computer graphic rendering, and particularly relates to a remote real-time rendering platform construction method based on a graphic cluster, which is used for providing a remote real-time efficient graphic rendering environment for multiple users and is beneficial to the work in the aspects of military simulation, educational training, mobile office, autonomous controllable system popularization and the like.

Background

The graphics rendering system is the basis for implementing visual simulation, virtual reality, augmented reality and other technologies. The Graphics rendering solution is developed from an initial dedicated Graphics workstation based on SGI (Silicon Graphics, usa), to a Graphics cluster system with multiple parallel computers, and then to a multi-Graphics-card Graphics workstation using SLI (scalable Link Interface) of NVIDIA (NVIDIA, usa) or CF (cross fire) technology of AMD (Advanced Micro Devices, ultramicro semiconductors, usa), and the Graphics rendering application mode is developed by hardware technology to have a spiral rising posture. In order to solve the problem of large-scale, remote, and real-time rendering, and to effectively integrate graphics resources and improve resource utilization efficiency, the rendering mode based on the graphics cluster is the focus of research in the industry again.

Generally, a cluster rendering system used in cooperation with a graphics cluster is also called a distributed rendering environment or a rendering farm, and is composed of a plurality of computers running rendering software, and under the unified deployment of cluster rendering management software, the cluster rendering system coordinates to complete distributed rendering tasks in a parallel manner. From a real-time perspective, there are roughly two categories: one type is a non-real-time rendering system, namely, a management end issues a rendering instruction to a graphics cluster, the graphics cluster sends a rendering scene to the management end or a storage system after completing a rendering task within a certain time, and then a user checks and calls a rendering result. The rendering system is multipurpose for the fields of movie and television, animation production and the like; the other type is a real-time rendering system, namely, a management end issues a rendering instruction to a graphic cluster, the graphic cluster feeds back a rendering result to a user in real time after finishing a rendering task, and meanwhile, the interaction and control instruction of the user are responded. Such rendering systems are used in many applications, both for visual simulation and virtual reality systems.

Through investigation of relevant manufacturers and technologies of the current remote real-time rendering platform, it is found that core technologies covered by the platform mainly include VDI (Virtual Desktop Infrastructure), VMM (Virtual Machine Monitor), DCV (Desktop Cloud Visualization), vGPU (Virtual Graphics Processing Unit), and the like, and the oriented application conditions thereof are mainly embodied in the following three aspects.

1. Single-machine access single-station virtual desktop service acquisition

A user installs client software locally, a Remote graphic workstation node installs service software, and a Remote Desktop Protocol (RDP) is used for virtualizing a Desktop of the graphic workstation to the local so as to realize Remote access and display of the graphic software.

In fact, the Windows-based RDP protocol can effectively support file operations of a remote desktop and simple document processing, but is not suitable for processing videos and dynamic images. On the basis, HP proposes an RGS (Remote Graphics Software) solution, which is optimized on the basis of an RDP (Remote Graphics Software), and comprises the steps of using an image compression/decompression technology (the compression ratio can reach 170: 1) with an independent patent, greatly reducing network transmission load, and realizing the function of Remote multi-user shared desktop service through the expansion of RDP (Windows Remote desktop connection only allows one user to access at the same time, and the server becomes a screen locking state after the access; HP RGS allows a plurality of users to access the desktop of the server at the same time, and the desktop of the server can also be normally displayed). The scheme supports Windows, Linux and Unix operating systems, can basically meet the requirement of remote rendering tasks, but the server software requirement is required to be installed on a Hewlett packard graphics workstation, and no other optimization processing work is carried out on the allocation of graphics rendering resources.

2. Multi-machine access single station virtual desktop service acquisition

The scheme is suitable for multiple persons to cooperatively carry out the same work, the requirement that the multiple persons carry out different works by sharing the graphic resources cannot be met, and if each user can obtain an independent virtual desktop and can share the graphic resources, an independent desktop system, namely the deployment and the operation of a VDI (virtual desktop infrastructure) architecture, needs to be developed for each user on the graphic workstation node. At present, the mainstream VDI technology mainly comprises three solutions of Hyper-V + RGS, XenServer + HDX and NICE DCV, and the comparative analysis is as follows:

(1)Hyper-V+RGS

Hyper-V (Windows-based virtualization product from Microsoft corporation of America) is built in Windows Server 2008 and later operating system products, and has good compatibility but low speed compared with the past virtualization technology of Microsoft by adding one more operating system layer. Hyper-V allows the virtual machine to more directly use the hardware resources of the physical host, thereby increasing the performance of the virtual system. Hyper-V cuts into the field of server virtualization by virtue of the advantages of the Microsoft platform, but the Hyper-V has defects in the support of the Linux system. The function and deficiencies of RGS software are as described above.

(2)XenServer+HDX

The XenServer (a virtualization software product introduced by Sijie corporation in America) is a comprehensive and easily-managed server virtualization platform, can efficiently manage Windows and Linux virtual servers, can provide economic and efficient server integration and service continuity, has the functions of an operating system, and can be directly installed on a server to be started and operated in a guided manner. HDX (High Definition Experience, High Definition use Experience) of the samje corporation in usa is a remote High Definition interactive product which is launched aiming at desktop virtualization and application virtualization markets and can realize interaction of multimedia, voice, video and 3D graphics. However, in the field of engineering design, the product can only realize desktop virtualization interaction based on a Windows platform, and cannot realize sharing of GPU resources.

(3)KVM+NICE DCV

NICE DCV (DCV product by NICE corporation under amazon, usa) is an interactive product that enables remote access to 2D or 3D applications over standard networks, allowing the user to utilize remote 3D high-end graphics cards, fast I/O capabilities, and a large number of memory nodes. It is worth emphasizing that the performance of the software can be fully exerted by deploying NICE DCV on a KVM (Kernel-based Virtual Machine) virtualization platform. The scheme can support a Linux operating system, NICE DCV supports various GPU virtualization technologies on a hardware level, a special graphic card of NVIDIA company is matched with a vGPU driver, vGPU is distributed to each virtual machine, and GPU resource sharing can be achieved to a certain extent.

3. Multi-machine access graphics cluster to obtain virtual desktop services

The virtual desktop service mentioned in the two schemes is implemented on the basis of a single graphics workstation, and if the number of users is too large, the form of creating a virtual machine on the single graphics workstation cannot meet the application requirement, so that a plurality of graphics workstations need to be used to replace the single graphics workstation, which is the concept of graphics cluster. The third scheme is mainly based on the first two schemes, and mainly solves the problem of aggregation of the graphics clusters, namely how to organize and manage a plurality of current graphics workstations, so that the graphics workstations can conveniently and efficiently realize graphics services for multiple users like one machine.

If a plurality of independent graphics workstations are aggregated into a unified graphics Cluster system, rendering task management software is needed to allocate and schedule resources, typically VCM (Visualization Cluster Manager) and TechViz (Visualization software product of TechViz, france).

(1)VCM

The VCM introduced by Orad, Israel is cluster management software installed on a graph cluster management node, distributes rendering tasks by planning the connection mode of the graph nodes, and realizes integration and synchronization of images by using a special integration interface. The cluster management mode well realizes the distribution and management of rendering tasks, but the concept mainly provides services for a multi-channel display system, is more suitable for multi-channel synchronous output large-screen display, and is obviously not flexible and intelligent in application.

(2)TechViz

The TechViz adopts another idea in cluster management, a complete graphic application software is installed on a management node of a cluster, a local graphic card is directly called to perform rendering display after the graphic application software runs, however, the TechViz software distributes a high-resolution graphic rendering task to rendering nodes through a network in the form of OpenGL (Open Graphics Library) instruction stream, then performs rendering output by using the graphic card of the rendering nodes, and performs output synchronization between the rendering nodes through a synchronization card. This cluster management scheme does not require the installation of graphics application software on the rendering nodes and is also very flexible in the distribution of rendering tasks.

(3) Other software

Besides the above two kinds of software, there are some cluster management software for graphics rendering at home and abroad, such as Muster (cluster rendering software of Virtual version corporation, usa), Platform LSF (cluster management scheduling software of IBM corporation, usa) and EnFuzion (cluster rendering software of china troostics corporation), etc., which can manage and monitor the working condition of rendering nodes in real time, avoid the idle of the rendering nodes, and enable users to obtain final rendering products in the shortest time.

The graphics cluster system provides a good hardware basis in the aspect of providing remote real-time rendering services for multiple users, and the cluster rendering software system at the present stage focuses on the aspects of cluster architecture, cluster management, load balancing, parallel rendering and the like, so that although a faster and more efficient solution can be provided for the users in a synchronous parallel mode, the problem of rendering application of a large number of remote users based on different requirements is not solved.

Different from a traditional cluster rendering system, the technology provided by the invention mainly adopts a multi-user-oriented asynchronous distributed working mode, depends on a graphics cluster hardware environment, and utilizes a virtualization means to meet the requirements of different users for use as required. The core idea is that a user remotely accesses a graph cluster, the graph cluster provides a real-time and efficient running environment for the user, and responds to the local interactive operation of the user, so that the user obtains a working experience close to a graph workstation, and data resources and a computing program of the graph cluster are deployed at a far end, thereby facilitating management and ensuring data safety.

Disclosure of Invention

In order to solve the technical problems, the invention provides a remote real-time rendering platform construction method based on a graphic cluster, VDI is deployed and operated on the graphic cluster through virtual desktop system deployment based on a Docker framework, an independent virtual desktop system is opened up for a user, display driving is directly called under a Docker operation environment to realize GPU (Graphics Processing Unit) accelerated rendering, meanwhile, a VNC protocol is adopted to establish remote desktop connection, a real-time image transmission channel is established for the user, and an asynchronous distributed real-time rendering service is provided for multiple users by combining with a dynamic task scheduling and management mechanism of the graphic cluster.

The purpose of the invention is realized by the following technical scheme:

the remote real-time rendering platform construction method based on the graphic cluster comprises the following steps:

the method comprises the following steps that firstly, deployment of a virtual desktop system is achieved inside a graph cluster by means of a Docker framework;

embedding a VNC server in the virtual desktop system, and after the VNC client accesses the VNC server, establishing remote desktop connection through a VNC protocol;

thirdly, the graphics cluster performs dynamic load balancing according to the connection condition of the remote desktop, and establishes a dynamic task scheduling and managing mechanism of the graphics cluster;

and step four, establishing a real-time image transmission channel between the platform client and the graphic cluster server through a graphic cluster dynamic task scheduling and managing mechanism, and comprehensively integrating the remote real-time rendering platform based on the graphic cluster.

The method specifically comprises the following steps:

creating a Docker container on a graph cluster rendering node operating system;

making the packaged virtual desktop system into a Docker mirror image and storing the Docker mirror image into a shared storage system;

the Docker container realizes deployment of a virtual desktop system in the graphics cluster by loading Docker mirror images, and simultaneously realizes GPU rendering acceleration based on a Docker architecture in the virtual desktop system.

The step of implementing Graphics Processing Unit (GPU) rendering acceleration based on a Docker architecture in the virtual desktop system comprises the following steps:

mapping a Graphics card driver kernel file of a Graphics cluster rendering node operating system into an integrated environment of a Docker container, and calling a window extension plug-in of OpenGL (Open Graphics Library) to realize real-time rendering acceleration of a GPU (Graphics Processing Unit) in a virtual desktop system.

The display of the virtual desktop system adopts a virtual external output mode, and the mode comprises the following steps:

and distributing virtual display output for each Docker container, and outputting the graphics in the Docker container to a client through the virtual display output when a user accesses the Docker container, so that GPU rendering is independent of an actual graphics card output interface.

The second step specifically comprises:

the VNC server side screens the changed image areas by using a block matching area change detection algorithm, transmits the image compressed to the corresponding VNC client side, and establishes remote desktop connection based on a C/S framework through a VNC protocol;

the VNC server side utilizes a block matching area change detection algorithm to screen changed image areas, and the method comprises the following steps:

setting a unit area needing change detection;

acquiring information of a redrawing area of a screen of an intercepting system;

if the intercepted redrawing area of the system screen is smaller than the unit area, sending the position information of the area;

if the intercepted redrawing area of the system screen is larger than the unit area, the intercepted redrawing area of the system screen is split, the coordinate information of the split area is stored in a linked list, the coordinate information of each split area in the linked list is subjected to traversal detection to obtain the position information of the change area needing to be retransmitted, and the position information of the change area needing to be retransmitted is stored and transmitted.

In step three, the dynamic task scheduling and managing mechanism of the graphics cluster includes:

under a certain load balancing strategy, reasonably arranging rendering nodes by a cluster management node to provide service resources for users;

the dynamic task management and scheduling of the graph cluster relates to dynamic conversion among one or more of four conditions including but not limited to a single-user to single node, multiple-user to single node, a single-user to multiple nodes and/or multiple-user to multiple nodes.

The load balancing strategy comprises the following steps:

under the condition that a single user is opposite to a single node, a polling strategy is adopted to distribute rendering nodes;

under the condition that multiple nodes are subjected to multi-user pair, indexes such as resource utilization rate, network bandwidth and Input/Output (I/O) rate of rendering nodes are comprehensively considered, a single-target planning problem with rendering system load variance or standard deviation as a minimum value is solved by using a dynamic load balancing model, and an optimized graph cluster load balancing strategy is obtained.

The comprehensive integrated remote real-time rendering platform based on the graphic cluster comprises a platform client, a graphic cluster management node, a graphic cluster rendering node and a shared storage system;

the platform client is deployed at the user operation terminal and used for initiating an application request to the graphics cluster server;

the graphic cluster management node and the graphic cluster rendering node are deployed at a graphic cluster server and used for responding to an application request initiated by a platform client;

and the shared storage system is used for storing and managing user configuration files, virtual desktop system images and/or rendering model data information.

The fourth step specifically comprises:

the method comprises the steps that a VNC client side is embedded into a platform client side, a Docker container and a VNC server side are embedded into a graph cluster rendering node, a Docker mirror image is stored in a shared storage system, and a graph cluster management node performs cluster load balancing according to a dynamic task scheduling and management mechanism;

when an application request initiated by a platform client is received, a graphics cluster management node designates a rendering node for the platform client, the rendering node starts a Docker container and loads a Docker mirror image, and meanwhile, a remote real-time rendering channel is established by utilizing a VNC protocol, so that remote real-time rendering service is provided for a user.

Wherein the application mode of the remote real-time rendering platform comprises:

after the graphics cluster server receives an application request initiated by the platform client, the graphics cluster management node distributes a rendering task to the rendering nodes, and the rendering nodes provide rendering services for users by creating an independent virtual desktop system.

Aiming at the problem of rendering application of a large number of remote users based on different requirements, the invention provides a real-time and efficient running environment for the users by utilizing a virtualization means based on a Docker architecture and relying on a graphics cluster hardware environment, and responds to the local interactive operation of the users. The method is not only suitable for different hardware environments and operating system platforms, but also can realize that multiple users share GPU rendering resources, so that the users can obtain work experience close to a graphic workstation.

Drawings

FIG. 1 is a flow chart of a method for constructing a remote real-time rendering platform based on a graphics cluster.

Fig. 2 is a diagram comparing the technical mechanisms of a virtual machine and a Docker.

FIG. 3 is a diagram of an overall framework of a remote real-time rendering platform based on graphics clusters.

FIG. 4 is a functional module design diagram of a remote real-time rendering platform based on graphics clusters.

FIG. 5 is an exemplary diagram of an application flow for a user using a remote real-time rendering platform.

FIG. 6a is a host operating system rendering effect diagram; FIG. 6b is a Docker container rendering effect diagram.

FIG. 7a is a schematic diagram of a single node supporting two users; fig. 7b is a diagram of a single node supporting four users.

Detailed Description

In order to make the technical solutions of the present invention better understood, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.

Referring to fig. 1, fig. 1 is a flowchart illustrating an embodiment of a method for building a remote real-time rendering platform based on a graphics cluster according to the present invention, where the method includes: step one to step four.

In the first step, deployment of the virtual desktop system is achieved inside the graphics cluster by using a Docker architecture.

Firstly, VDIs are deployed and operated on a graph cluster, an independent virtual desktop system is opened up for users, and application requirements of different users can be better met. In consideration of the restriction of the virtual machine operation mode on the GPU rendering capability, the system adopts a VDI solution scheme based on a Docker (application container engine), and directly calls a display driver under a Docker operation environment, so that the GPU rendering efficiency is greatly improved.

Generally, a virtual desktop system based on a virtual machine realizes a display function by calling a display driver of a basic operating system, the display resource of the virtual desktop system is limited, and a processing mode mostly belongs to a serial working mode, so that the GPU rendering capability of a graphics cluster high-end display card cannot be fully exerted. The VDI solution based on Docker utilizes a GPU virtualization technology to penetrate through the display drive of a basic operating system, and graphics acceleration based on a GPU is realized inside a plurality of independent virtual machines, so that multi-user sharing of GPU resources is realized to a certain extent. Referring to fig. 2, fig. 2 is a schematic diagram comparing the technical mechanisms of the virtual machine and the Docker.

The method specifically comprises the following steps: creating a Docker container on a graph cluster rendering node operating system; making the packaged virtual desktop system into a Docker mirror image and storing the Docker mirror image into a shared storage system; the Docker container realizes deployment of a virtual desktop system in the graphics cluster by loading Docker mirror images, and simultaneously realizes GPU rendering acceleration based on a Docker architecture in the virtual desktop system.

The Docker is a high-level Container open source engine based on an LXC (Linux Container), and mainly includes three components, namely, a daemon (Docker daemon), an Image (Image) and a Container (Container), wherein the daemon is used as a server to receive and process a request from a client, the Image is a read-only template for the Container to run, and the Container includes an integrated environment required by the user application to run.

The rendering acceleration step specifically includes: mapping a Graphics card driver kernel file of a Graphics cluster rendering node operating system into an integrated environment of a Docker container, and calling a window extension plug-in of OpenGL (Open Graphics Library) to realize real-time rendering acceleration of a GPU (Graphics Processing Unit) in a virtual desktop system.

Because the Docker container shares the operating system kernel with the host operating system, the user application can directly call the hardware resources in the host operating system. However, in the actual application process, the user cannot realize effective GPU rendering by running the application in the integrated environment directly provided by the container. Through a large number of test experiments, the graphics card driver kernel file of the host operating system is mapped into the integrated environment of the container, and meanwhile, the X window extension plug-in of OpenGL is called, so that the rendering acceleration of the GPU can be well realized.

Further, the display of the virtual desktop system adopts a virtual external output mode, which includes:

and distributing virtual display output for each Docker container, and outputting the graphics in the Docker container to a client through the virtual display output when a user accesses the Docker container, so that GPU rendering is independent of an actual graphics card output interface.

In general, a GPU rendering pipeline performs on-screen rendering, and asynchronous access by multiple users cannot achieve screen sharing. Aiming at the problem, virtual external output is adopted to distribute virtual display output for each container, and when a user accesses the container, a graph outputs the display to a client, so that GPU rendering does not depend on an actual graph card output interface, and the problem of multi-user asynchronous application is well solved.

In step two, a VNC (Virtual Network Console) service end is embedded in the Virtual desktop system, and after the VNC client accesses the VNC service end, a remote desktop connection is established through a VNC protocol.

The VNC server side screens the changed image areas by using a block matching area change detection algorithm, the images are compressed and then sent to the corresponding VNC client side, and remote desktop connection based on a C/S framework is established through a VNC protocol.

The VNC server side utilizes a block matching area change detection algorithm to screen changed image areas, and the method comprises the following steps:

setting a unit area needing change detection;

acquiring information of a redrawing area of a screen of an intercepting system;

if the intercepted redrawing area of the system screen is smaller than the unit area, sending the position information of the area;

if the intercepted redrawing area of the system screen is larger than the unit area, the intercepted redrawing area of the system screen is split, the coordinate information of the split area is stored in a linked list, the coordinate information of each split area in the linked list is subjected to traversal detection to obtain the position information of the change area needing to be retransmitted, and the position information of the change area needing to be retransmitted is stored and transmitted.

Wherein the steps are as follows: the VNC mechanism intercepts redrawing area information of a system screen by adopting a message hook mechanism. Setting a unit area needing change detection, if the size of the unit area rgn is recorded as 32 multiplied by 32, if the redrawing area of the intercepted system screen is smaller than the unit area, directly sending the position information of the area; if the redrawing area of the intercepted system screen is larger than the unit area, splitting the redrawing area of the intercepted system screen to obtain a rectangular chain table (traversal chain table) for storing coordinate information of the split area, recording the rectangular chain table as array _ list, defining a change area as vnc _ rgn, specially storing area position information needing to be retransmitted, and calling the detection algorithm for each member rectangle of the array _ list to detect the change area.

A preferred embodiment:

(1) assuming that one member rectangle is ari, if the length and the width of ari are all less than 32 x 32, directly serving as a change area to be retransmitted, and storing the change area into vnc _ rgn, otherwise, analyzing and detecting the ari.

(2) Firstly, defining a new rectangle arrj (arri.left, 0, arri.right, 0), finding a changed line in arri, recording the y coordinate of the line, and recording arrj.top as y, and at the same time, increasing y + as 16 to find a non-changed line, and temporarily recording arrj.bottom as y; then, a comparison is made between the bottom y-1 of arrj, a line with changed content is found, and the y coordinate of the line is set to arrj. The final top, bottom coordinates of the variation rectangle arrj are determined so far. While setting the value of ari to ari (ari. left, arrj. bottom, ari. right, ari. bottom). After the analysis of arrj is completed, the analysis of arri is followed until the end.

(3) After the rectangle arrj is determined, the rectangle arrj is analyzed. Firstly, defining a rectangular arn (0, 0, 0, 0), searching a column which is changed firstly in arrj, recording left and top coordinates of a changed area after finding the column, wherein the left is x, and the arm is y, and analyzing the cyclic traversal of the x and y axes respectively (namely x + is 32, and y + is 32), if the screen pixel data of the column is changed, continuing to compare the x + with the x + 32; otherwise, recording the array.right ═ x, performing the same comparison in the y axis direction, y + ═ 32, and recording the array.bottom ═ y, finally obtaining the rectangular coordinates of the change region to be sent, storing the rectangle into the rectangular linked list finally sent, simultaneously recording the array (array.right, array.top, array.right, array.bottom), the array (array.left, array.bottom, array j.right, array j.bottom), and the array (array.j.bottom), assigning the array and the array to the array j respectively to perform similar analysis detection, and sequentially cycling the whole array j until finishing.

By the screen change area detection algorithm, the member rectangles of the whole array _ list linked list are traversed to obtain the coordinate information of the change area which needs to be sent again.

In addition, for the real-time transmission control of the image, the system adopts a C/S (Client/Server) framework to establish an image transmission channel between a Client and a Server, the Server collects and compresses the image and then sends the image to the Client, and the Client receives and decompresses the image and then displays the image to a user. The VNC protocol is adopted for image real-time transmission, the server side screens changed image areas by using a block matching area change detection algorithm, and the image areas are sent to the client side after being compressed, so that the network sending data volume is greatly reduced. The image refreshing speed of the client is autonomously controlled by a user, the server transmits images after receiving the refreshing request of the client, and when the client is busy or the network delay is large, the image refreshing speed is correspondingly reduced.

The method for enhancing the image transmission real-time property mainly comprises two aspects, namely, the efficiency of the region change detection algorithm is improved, and the image compression algorithm with higher compression rate and shorter encoding time is used. The VNC regional change detection algorithm is improved, and the image change region is subdivided through interlaced scanning, so that redundant unchanged image data are further eliminated. The system tests the algorithm, taking a 24-bit screen image of 1280 x 800 as an example, the refreshing interval of a client screen is 40ms, the data transmission amount in the applications of word processing, web browsing and the like is about 200kb/s, and the data amount can be reduced by about 10% on average by applying the algorithm; the data transmission amount exceeds 4Mb/s in high-dynamic image application, the influence of the application of the algorithm on the data amount is small, the CPU utilization rate is obviously improved, and therefore the method has weak influence on the real-time property of high-dynamic image transmission. The compression rate and the time spent on encoding are comprehensively considered, the system usually adopts an MJPEG (image compression algorithm) algorithm with higher efficiency to compress and encode the image, and experimental results show that the data transmission flow by adopting the MJPEG algorithm can be reduced to be less than 50% of that of a ZLib (image compression algorithm defaulted by a VNC protocol) algorithm under the same conditions, so that the problem of network congestion caused by overlarge data volume can be avoided to a certain extent.

In the third step, the graphics cluster performs dynamic load balancing according to the connection condition of the remote desktop, and establishes a dynamic task scheduling and management mechanism of the graphics cluster.

The graph cluster dynamic task scheduling and managing mechanism comprises:

under a certain load balancing strategy, reasonably arranging rendering nodes by a cluster management node to provide service resources for users; the dynamic task management and scheduling of the graph cluster relates to dynamic conversion among one or more of four conditions including but not limited to a single-user to single node, multiple-user to single node, a single-user to multiple nodes and/or multiple-user to multiple nodes.

The load balancing strategy comprises the following steps:

under the condition that a single user is opposite to a single node, a polling strategy is adopted to distribute rendering nodes; under the condition that multiple nodes are subjected to multi-user pair, indexes such as resource utilization rate, network bandwidth and Input/Output (I/O) rate of rendering nodes are comprehensively considered, a single-target planning problem with rendering system load variance or standard deviation as a minimum value is solved by using a dynamic load balancing model, and an optimized graph cluster load balancing strategy is obtained.

The task scheduling and management mechanism of the remote real-time rendering system is that under a certain load balancing strategy, the cluster management nodes reasonably arrange the rendering nodes to provide service resources for users. In general, cluster management involves four situations, namely single-user to single node, multi-user to single node, single-user to multi-node, and multi-user to multi-node, and dynamic switching of multiple situations may occur during system operation.

In the case of single-user to single-node, the rendering nodes can be allocated by adopting a polling strategy, and in the actual application, more than one user to multiple nodes are used. Under the condition that the number of users and the number of rendering nodes are fixed, indexes such as resource utilization rate, network bandwidth and I/O rate of the rendering nodes are comprehensively considered, and a load balancing strategy can be summarized as making a decision in the selection of the rendering nodes, namely solving a single-target planning problem that the load variance or standard deviation of a rendering system is a minimum value; if the number of users or the number of rendering nodes changes, the solution process may evolve into a complex dynamic programming problem in a period of time, but the solution process can be treated as a single-target programming problem at the present moment.

Let E_iAnd if the load value of the rendering node of the ith station is the load value of the rendering node of the ith station, the load value of the rendering node of the ith station is as follows:

E_i＝w₁C_i+w₂M_i+w₃G_i+w₄N_i+w₅IO_i (1)

in the formula, C_i、M_i、G_iRespectively the occupancy rates of the rendering node CPU, the memory and the GPU of the ith platform, N_iFor network utilization, IO_iIs the I/O rate, w_j(j-1, 2, …,5) is a weight value corresponding to the above 5 indexes, each index item and the load value E_iNormalization is required.

The total load mean of the rendering system is:

simplifying the solving of the rendering system load variance into solving of the minimum value of the difference value between the load of each rendering node and the total load mean value, namely:

after load balancing, the load value of each rendering node is generally stabilized above or below the average value of the total load of the system. In order to avoid the influence on the rendering experience of the current user due to excessive additional users, when the management node monitors that the rendering frame frequency of the current user of the rendering node is lower than a set value (default is 24 frames/second), the management node preferentially distributes the user request to other rendering nodes or rejects the access request of the user according to rules.

In the fourth step, a real-time image transmission channel is established between the platform client and the graphic cluster server through a graphic cluster dynamic task scheduling and managing mechanism, and a remote real-time rendering platform based on the graphic cluster is integrated comprehensively.

As shown in fig. 3, the steps specifically include:

when an application request initiated by a platform client is received, a VNC client is embedded into the platform client (the client n is short for term in figure 3, n is more than or equal to 1), a Docker container (the container n is short for term in figure 3, n is more than or equal to 1) and a VNC server are embedded into a graph cluster rendering node (the node is short for term in figure 3), a Docker mirror image (the mirror image is short for term in figure 3) is stored in a shared storage system, and cluster load balancing is carried out by a graph cluster management node according to a dynamic task scheduling and management mechanism;

the method comprises the steps that a graphic cluster management node designates a rendering node for a platform client, the rendering node starts a Docker container and loads a Docker mirror image, meanwhile, a VNC protocol is used for establishing a remote real-time rendering channel, and a remote real-time rendering platform based on a graphic cluster is integrated comprehensively.

The comprehensive integrated remote real-time rendering platform based on the graphic cluster comprises a platform client, a graphic cluster management node, a graphic cluster rendering node and a shared storage system; the platform client is deployed at the user operation terminal and used for initiating an application request to the graphics cluster server; the graphic cluster management node and the graphic cluster rendering node are deployed at a graphic cluster server and used for responding to an application request initiated by a platform client; and the shared storage system is used for storing and managing user configuration files, virtual desktop system images and/or rendering model data information.

The application mode of the remote real-time rendering platform comprises the following steps: after the graphics cluster server receives an application request initiated by the platform client, the graphics cluster management node distributes a rendering task to the rendering nodes, and the rendering nodes provide rendering services for users by creating an independent virtual desktop system.

In an embodiment, the remote real-time rendering system utilizes a Docker architecture to implement virtual desktop system deployment, and establishes a remote desktop connection using a VNC. And the remote user sends a Docker container management request to the cluster server, and establishes remote desktop connection with the VNC server by using the VNC client. At the graphic cluster server, the cluster management node manages and schedules the container through a Docker daemon, and a Docker registry is used for storing user-defined image files. After the server receives a container management request sent by the client, the Docker daemon searches the mirror image in the registry and creates a user container, and simultaneously starts the VNC server in the container.

After the graphic cluster receives an application request initiated by a user terminal, a management node distributes a rendering task to rendering nodes, and the rendering nodes provide rendering services for users by creating an independent virtual desktop system. The remote real-time rendering platform based on the graphic cluster mainly comprises a platform client, a graphic cluster management node, a graphic cluster rendering node and a shared storage system, wherein the platform client is deployed at a user operation terminal, the graphic cluster management node and the rendering node are deployed at a graphic cluster server, and the shared storage system mainly provides convenient and reliable storage management for user configuration files, virtual desktop system images, rendering model data and the like. The platform function module is shown in fig. 4.

The platform client sends a virtual desktop request to the graph cluster management node through a user registration/login mode, and the node to be managed is responsible for establishing remote desktop connection with the rendering node providing rendering service after the rendering node is appointed. The platform client can directly access the shared storage system and upload, download or modify a specific configuration file within the user authority range.

The graphic cluster management node mainly comprises 3 modules of initialization, task management and node management. After the system is started, the initialization functions of cluster internal connection, configuration file loading and the like are firstly completed; the task management module is mainly used for completing functions of task storage, scheduling, query and the like, after receiving a virtual desktop request of a platform client, querying the load condition of each rendering node in a cluster, and meanwhile, utilizing a dynamic load balancing algorithm to appoint the rendering node for the platform client, wherein if the virtual desktop request is in a new user registration mode, the rendering node newly opens a virtual desktop system, and if the virtual desktop request is in a user login mode, the rendering node loads a virtual desktop system file from a shared storage system; the node management mainly completes functions of adding and deleting, configuring, state query and the like of rendering nodes.

After the virtual desktop system is started, the graphics cluster rendering node establishes an independent data connection channel with the platform client through remote desktop management, and provides remote rendering service for a user. The rendering node mainly comprises two parts, namely virtual desktop system management and remote desktop system management, wherein the virtual desktop system management is responsible for creating and managing the virtual desktop system and providing real-time rendering service for a user, and the remote desktop system management is mainly used for remote display and interaction control.

The virtual desktop system management module is mainly used for creating a Docker container on a Linux operating system, making an encapsulated virtual desktop system into a Docker mirror image and storing the Docker mirror image into a shared storage system, realizing VDI (virtual device interface) deployment by loading the Docker mirror image when the virtual desktop system needs to be created, and solving the real-time rendering problem in the virtual desktop system by using a Docker-based GPU (graphics processing Unit) rendering technology.

The remote desktop management module mainly adopts a VNC solution, and since the VNC captures information of a redrawing area of a system screen by using a message hook mechanism, the remote desktop management module needs to create a display output interface map for a virtual desktop system of each user, that is, create a virtual "screen". And a VNC server is installed in the virtual desktop system, the platform client establishes remote desktop connection with the virtual desktop system by calling the VNC client, and the transmission instantaneity is optimized by adopting a VNC-based image real-time transmission control method. An example of an application flow for a user using a remote real-time rendering platform is shown in FIG. 5, which includes:

the platform client initiates a connection request, the graphic cluster management node receives the connection request, the graphic cluster management node selects a rendering node for a user, judges whether the connection is made by a new user, and if so, starts an application container engine and creates a virtual desktop system; if not, starting an application container engine and loading the mirror image of the virtual desktop system; the graph cluster rendering node responds to the user connection request, the graph cluster rendering node creates a VNC real-time rendering channel, the platform client completes remote desktop connection, and the user starts system application.

In order to test the application effect of the system, two graphic workstations are selected to form a graphic cluster, one of the two graphic workstations is used as a management node, a platform client is installed on a common PC, and the network environment is a gigabit local area network. The hardware configuration of the graphic cluster node is an Intel to strong quad-core processor, a 16G memory, an Nvidia Q7000 professional graphic card (a video memory 6G), and the operating system is a 64-bit Ubuntu Kylin 16.04 (a Kylin operating system in Wuban diagram). After logging in, the platform client establishes remote desktop connection with the graphics cluster rendering node, and respectively performs GPU rendering efficiency and multi-user asynchronous application experiments on 1280 multiplied by 800 resolution.

GPU rendering efficiency experiment: the same pattern test samples are respectively run in the host operating system and the Docker container environment, and due to the limitation of the screen refreshing frequency, the running effect is that the running frame frequency on the host operating system is not more than 60 frames/second as shown in FIG. 6a, the running frame frequency in the Docker container environment can reach 85 frames/second, and the running effect is shown in FIG. 6 b. Test results show that the efficiency of Docker-based GPU rendering is already substantially close to the efficiency of usage on the host operating system.

Multi-user asynchronous application experiment: after the multi-platform clients are started simultaneously, the rendering frame frequency is gradually reduced along with the increase of the number of the platform clients under the condition of stable operation of the system, the rendering frame frequency difference among the platform clients is not large, and the platform clients do not have obvious pause phenomenon. When a single rendering node supports 1 user, the average frame rate can reach 140 frames/second, when 2 users are supported, the average frame rate is about 120 frames/second, the display effect of the platform client is as shown in fig. 7a, when 4 users are supported, the average frame rate is about 75 frames/second, and the display effect of the platform client is as shown in fig. 7 b.

To sum up, the embodiment of the invention aims at the problem of rendering application of a large number of remote users based on different requirements, and meets the requirements of different users for use as required by relying on a graphics cluster hardware environment according to an asynchronous distributed working mode facing multiple users. The core idea is that a user remotely accesses a graph cluster, the graph cluster provides a real-time and efficient running environment for the user, and responds to the local interactive operation of the user, so that the user obtains a working experience close to a graph workstation, and data resources and a computing program of the graph cluster are deployed at a far end, thereby facilitating management and ensuring data safety.

Claims (6)

Translated fromChinese

1.基于图形集群的远程实时渲染平台构建方法，包括：1. Construction method of remote real-time rendering platform based on graphics cluster, including:步骤一、利用Docker架构在图形集群内部实现虚拟桌面系统的部署；Step 1, using the Docker architecture to realize the deployment of the virtual desktop system inside the graphics cluster;步骤二、在所述虚拟桌面系统中嵌入VNC服务端，VNC客户端访问所述VNC服务端后，通过VNC协议建立远程桌面连接；Step 2: Embed a VNC server in the virtual desktop system, and after the VNC client accesses the VNC server, establish a remote desktop connection through the VNC protocol;步骤三、图形集群根据远程桌面连接情况进行动态负载均衡，建立图形集群动态任务调度与管理机制；Step 3: The graphics cluster performs dynamic load balancing according to the connection status of the remote desktop, and establishes a dynamic task scheduling and management mechanism for the graphics cluster;步骤四、通过图形集群动态任务调度与管理机制，在平台客户端与图形集群服务端间建立实时图像传输通道,综合集成基于图形集群的远程实时渲染平台；Step 4: Establish a real-time image transmission channel between the platform client and the graphics cluster server through the dynamic task scheduling and management mechanism of the graphics cluster, and comprehensively integrate the remote real-time rendering platform based on the graphics cluster;步骤一，具体包括：在图形集群渲染节点操作系统上创建Docker容器；将封装后的虚拟桌面系统做成Docker镜像存入共享存储系统中；Docker容器通过加载Docker镜像，在图形集群内部实现虚拟桌面系统的部署，同时在虚拟桌面系统内实现基于Docker架构的GPU渲染加速；Step 1 specifically includes: creating a Docker container on the graphics cluster rendering node operating system; making the packaged virtual desktop system into a Docker image and storing it in a shared storage system; by loading the Docker image, the Docker container realizes a virtual desktop inside the graphics cluster The deployment of the system, and the GPU rendering acceleration based on the Docker architecture is realized in the virtual desktop system;所述在虚拟桌面系统内实现基于Docker架构的GPU渲染加速的步骤，包括：将图形集群渲染节点操作系统的显卡驱程内核文件映射到Docker容器的集成环境中，同时调用OpenGL的窗口扩展插件，实现在虚拟桌面系统内的GPU实时渲染加速；The described step of realizing GPU rendering acceleration based on Docker architecture in the virtual desktop system includes: mapping the graphics card driver kernel file of the graphics cluster rendering node operating system to the integrated environment of the Docker container, and simultaneously calling the OpenGL window extension plug-in, Realize GPU real-time rendering acceleration in virtual desktop system;所述虚拟桌面系统的显示采用虚拟外接输出的方式，该方式包括：为每一个Docker容器分配虚拟的显示输出，当用户访问该Docker容器时，Docker容器中的图形通过虚拟的显示输出，输出至客户端，使得GPU渲染不依赖于实际的图形卡输出接口；The display of the virtual desktop system adopts the mode of virtual external output, which includes: assigning a virtual display output to each Docker container, when the user accesses the Docker container, the graphics in the Docker container are output to the virtual display output through the virtual display output. Client, making GPU rendering independent of the actual graphics card output interface;所述远程实时渲染平台的应用模式包括：图形集群服务端在接到平台客户端发起的应用请求后，由图形集群管理节点将渲染任务分配至渲染节点，渲染节点通过创建独立的虚拟桌面系统为用户提供渲染服务。The application mode of the remote real-time rendering platform includes: after the graphics cluster server receives the application request initiated by the platform client, the graphics cluster management node assigns the rendering task to the rendering node, and the rendering node creates an independent virtual desktop system for The user provides rendering services.2.如权利要求1所述的基于图形集群的远程实时渲染平台构建方法，其特征在于，2. the method for constructing a remote real-time rendering platform based on graphics cluster as claimed in claim 1, is characterized in that,步骤二具体包括：Step 2 specifically includes:VNC服务端利用分块匹配的区域变化检测算法对发生变化的图像区域进行筛选，经图像压缩后发送给对应的VNC客户端，通过VNC协议建立基于C/S架构的远程桌面连接。The VNC server uses the block matching region change detection algorithm to filter the changed image area, and then sends the image to the corresponding VNC client after image compression, and establishes a remote desktop connection based on the C/S architecture through the VNC protocol.3.如权利要求2所述的基于图形集群的远程实时渲染平台构建方法，其特征在于，所述VNC服务端利用分块匹配的区域变化检测算法对发生变化的图像区域进行筛选，包括：3. the remote real-time rendering platform construction method based on graphics cluster as claimed in claim 2, is characterized in that, described VNC server utilizes the regional change detection algorithm of block matching to screen the changed image area, comprising:设定需要变化检测的单位区域；Set the unit area that needs change detection;获取截获系统屏幕重绘区域的信息；Obtain information about the screen redraw area of the intercepted system;若截获的系统屏幕重绘区域小于单位区域，发送该系统屏幕重绘区域的位置信息；If the captured system screen redraw area is smaller than the unit area, send the location information of the system screen redraw area;若截获的系统屏幕重绘区域大于单位区域，将截获系统屏幕重绘区域进行拆分，将拆分区域的坐标信息存储至链表中，对链表中每个拆分区域的坐标信息进行遍历检测后得到需要重新发送的变化区域位置信息，将需要重新发送的变化区域的位置信息进行存储并发送。If the intercepted system screen redraw area is larger than the unit area, split the intercepted system screen redraw area, store the coordinate information of the split area in the linked list, and traverse and detect the coordinate information of each split area in the linked list. The location information of the changed area that needs to be resent is obtained, and the location information of the changed area that needs to be resent is stored and sent.4.如权利要求1所述的基于图形集群的远程实时渲染平台构建方法，其特征在于，4. the remote real-time rendering platform construction method based on graphics cluster as claimed in claim 1, is characterized in that,步骤三中，所述图形集群动态任务调度与管理机制包括：In step 3, the dynamic task scheduling and management mechanism of the graphics cluster includes:在负载均衡策略下，由集群管理节点合理安排渲染节点为用户提供服务资源；Under the load balancing strategy, the cluster management node reasonably arranges the rendering nodes to provide service resources for users;所述图形集群动态任务管理与调度涉及到包括但不限于单用户对单节点、多用户对单节点、单用户对多节点和/或多用户对多节点四种情况中的一种或多种情况间动态转换；The dynamic task management and scheduling of the graph cluster involves, but is not limited to, one or more of four situations including, but not limited to, single-user-to-single-node, multi-user-to-single-node, single-user-to-multiple-node, and/or multi-user-to-multiple-node situations. dynamic transitions between situations;所述负载均衡策略包括：The load balancing strategy includes:在单用户对单节点情况下，采用轮询策略分配渲染节点；In the case of a single user versus a single node, a round-robin strategy is used to allocate rendering nodes;在多用户对多节点的情况，综合考虑渲染节点的资源利用率、网络带宽以及I/O速率指标，利用动态负载均衡模型求解渲染系统负载方差或标准差为极小值的单目标规划问题，得到优化后的图形集群负载均衡策略。In the case of multi-user versus multi-node, the resource utilization, network bandwidth and I/O rate indicators of the rendering node are comprehensively considered, and the dynamic load balancing model is used to solve the single-objective programming problem in which the load variance or standard deviation of the rendering system is a minimum value. Get the optimized graphics cluster load balancing strategy.5.如权利要求1所述的基于图形集群的远程实时渲染平台构建方法，其特征在于，5. the method for constructing a remote real-time rendering platform based on graphics cluster as claimed in claim 1, is characterized in that,所述综合集成的基于图形集群的远程实时渲染平台包括平台客户端、图形集群管理节点、图形集群渲染节点和共享存储系统四个部分；The comprehensively integrated remote real-time rendering platform based on graphics cluster includes four parts: platform client, graphics cluster management node, graphics cluster rendering node and shared storage system;平台客户端部署于用户操作终端，用于向图形集群服务端发起应用请求；The platform client is deployed on the user operating terminal to initiate application requests to the graphics cluster server;图形集群管理节点和图形集群渲染节点部署于图形集群服务端，用于响应平台客户端发起应用请求；The graphics cluster management node and the graphics cluster rendering node are deployed on the graphics cluster server to respond to application requests initiated by the platform client;共享存储系统，用于存储管理用户配置文件、虚拟桌面系统镜像和/或渲染模型数据信息。A shared storage system for storing and managing user profiles, virtual desktop system images, and/or rendering model data information.6.如权利要求5所述的基于图形集群的远程实时渲染平台构建方法，其特征在于，6. The method for constructing a remote real-time rendering platform based on graphics cluster as claimed in claim 5, characterized in that,步骤四具体包括：Step 4 specifically includes:将VNC客户端嵌入到平台客户端，Docker容器和VNC服务端嵌入到图形集群渲染节点，Docker镜像存储于共享存储系统中，由图形集群管理节点依据动态任务调度与管理机制进行集群负载均衡；The VNC client is embedded in the platform client, the Docker container and the VNC server are embedded in the graphics cluster rendering node, the Docker image is stored in the shared storage system, and the cluster load balancing is performed by the graphics cluster management node according to the dynamic task scheduling and management mechanism;接收到平台客户端发起应用请求时，图形集群管理节点为平台客户端指定渲染节点，渲染节点启动Docker容器并加载Docker镜像，同时利用VNC协议建立远程实时渲染通道，为用户提供远程实时渲染服务。When receiving an application request initiated by the platform client, the graphics cluster management node designates a rendering node for the platform client, the rendering node starts the Docker container and loads the Docker image, and uses the VNC protocol to establish a remote real-time rendering channel to provide users with remote real-time rendering services.

Images