CN112614202A - GUI rendering display method, terminal, server, electronic device and storage medium - Google Patents

Abstract

The embodiment of the application provides a method, a terminal, a server, electronic equipment and a storage medium for GUI rendering display, and relates to the technical field of software. The method comprises the following steps: the method comprises the steps of receiving interactive operation information sent by a terminal, updating a rendering instruction of each frame image and a timestamp corresponding to a screen updating period according to the interactive operation information, rendering the frame images to be rendered according to the rendering instructions by a server and the terminal respectively and independently to obtain rendered frame images, sending the rendered frame images to the terminal by the server, receiving the rendered frame images sent by the server by the terminal, and combining the rendered frame images of the terminal to be used for displaying a GUI of the system. The embodiment of the application can isolate the capability of processing the frame images of the terminal and the server, does not influence each other, fully exerts respective capability, synchronizes according to timestamp information, avoids time sequence errors, can improve the image processing speed, can also fully utilize hardware resources of the terminal and the server, improves the overall efficiency, and improves user experience.

Description

GUI rendering display method, terminal, server, electronic device and storage medium

Technical Field

The application relates to the technical field of software, in particular to a method, a terminal, a server, electronic equipment and a storage medium for GUI rendering display.

Background

With the development of cloud technology, virtualization, and internet technology, virtualized desktops represented by cloud desktops are widely used. A cloud Desktop Infrastructure (VDI) generally virtualizes a Desktop of a user by operating an operating system through a server of a data center. Users interface with the virtual desktop via a client computing protocol from a client device (client or home PC) and access their desktop is as if accessing a traditional locally installed desktop.

With the development of hardware technology and the wide application of VR, AR cloud rendering and cloud games, the hardware performance of a user terminal is generally improved, how a terminal participates in collaborative rendering of end and cloud GUI (computer operation user interface displayed in a graphical manner), the hardware performance of a cloud end and the terminal is fully exerted, and the high-performance end and cloud GUI collaborative rendering display capability is provided, which becomes a problem to be solved.

At present, methods mainly adopted for providing high-performance end and cloud GUI collaborative rendering display are as follows: the cloud end renders, directly maps the result to the virtual machine, and then directly displays the result through the client end; this kind of mode needs the high performance processing apparatus in the high in the clouds to aggravate the high in the clouds load easily, influence processing rate, another kind adopts end, cloud distributing type to participate in whole GUI and renders the flow, then shows through the client, this kind of mode mainly makes end, cloud render the same image of processing jointly, though improved image processing rate, when making end, cloud mutual influence easily, can not full play end, the hardware ability of cloud still can influence user experience.

Disclosure of Invention

Embodiments of the present invention provide a GUI rendering display method, a terminal, a server, an electronic device, and a storage medium that overcome the above problems or at least partially solve the above problems.

In a first aspect, a method for rendering and displaying a GUI is provided, which is applied to a server, and includes:

receiving interactive operation information sent by a terminal;

updating rendering instructions and timestamps corresponding to each frame image to be rendered according to the interactive operation information, and forming a rendering instruction sequence; the time stamp is used for representing the initiation time of the screen refreshing period corresponding to the frame image;

acquiring graphic processing capacity data of a terminal and a server for processing frame images to determine a first rendering instruction and a second rendering instruction from a rendering instruction sequence, wherein the first rendering instruction is a rendering instruction executed by the server, and the second rendering instruction is a rendering instruction executed by the terminal;

sending the second rendering instruction, the frame image to be rendered corresponding to the second rendering instruction and the timestamp corresponding to the second rendering instruction to the terminal, so that the terminal renders the frame image to be rendered corresponding to the second rendering instruction according to the second rendering instruction, and obtains a rendered second frame image;

rendering the frame image corresponding to the first rendering instruction according to the first rendering instruction, obtaining the rendered first frame image, and sending the rendered first frame image and the corresponding timestamp to the terminal, so that the terminal determines a GUI rendering result to be displayed according to the rendered first frame image and second frame image and the corresponding timestamps of the rendered first frame image and second frame image.

Further, updating the rendering instruction and the timestamp corresponding to each frame image to be rendered according to the interactive operation information, including:

updating and processing the frame image to be rendered and the rendering node in the frame image to be rendered according to the interactive operation information sent by the terminal;

and determining a rendering instruction corresponding to the frame image to be rendered according to the frame image to be rendered and a rendering node in the frame image to be rendered, and taking the starting time of a screen updating period corresponding to the frame image to be rendered as a timestamp corresponding to the frame image to be rendered.

Further, the method further includes sending the rendered first frame image and the corresponding timestamp to the terminal, and then:

and according to the timestamp corresponding to the rendered first frame image, identifying the rendered first frame image, and sending the identified first frame image to the terminal.

In a second aspect, a method for rendering and displaying a GUI is provided, which is applied to a terminal, and includes:

sending the interactive operation information to a server, so that the server updates a rendering instruction and a timestamp corresponding to each frame image to be rendered and a first frame image rendered by the server according to the interactive operation information, and sends a second rendering instruction, the frame image to be rendered corresponding to the second rendering instruction and the timestamp corresponding to the second rendering instruction to the terminal;

receiving a second rendering instruction sent by the server, a frame image to be rendered corresponding to the second rendering instruction and a timestamp corresponding to the second rendering instruction, and rendering the frame image to be rendered corresponding to the second rendering instruction according to the second rendering instruction to obtain a rendered second frame image;

and receiving the rendered first frame image and the corresponding timestamp sent by the server, and determining a GUI rendering result to be displayed by combining the rendered first frame image and second frame image and the corresponding timestamps of the rendered first frame image and second frame image.

Further, rendering the frame image to be rendered corresponding to the second rendering instruction according to the second rendering instruction, obtaining a rendered second frame image, and then:

and identifying the rendered second frame image according to the corresponding timestamp of the rendered second frame image to obtain the identified second frame image.

Further, determining a GUI rendering result to be displayed by combining the rendered first frame image and second frame image and the time stamps corresponding to the rendered first frame image and second frame image, including:

determining frame images to be displayed in the first frame image and the second frame image for the first frame image and the second frame image;

acquiring the current time, and determining the time interval of the frame image to be displayed according to the timestamp corresponding to the frame image to be displayed and the current time;

if the time interval of the frame images to be displayed is smaller than or equal to a preset threshold value, reserving the frame images to be displayed, and taking the frame images to be displayed as a GUI rendering result to be displayed;

and if the time interval of the frame images to be displayed is greater than the preset threshold value, clearing the frame images to be displayed.

In a third aspect, a server is provided, including:

the first receiving module is used for receiving the interactive operation information sent by the terminal;

the first processing module is used for updating rendering instructions and timestamps corresponding to each frame image to be rendered according to the interactive operation information and forming a rendering instruction sequence; the time stamp is used for representing the initiation time of a screen updating period corresponding to the rendering instruction of the frame image;

the second processing module is used for acquiring the graphic processing capacity data of the frame images processed by the terminal and the server so as to determine a first rendering instruction and a second rendering instruction from the rendering instruction sequence, wherein the first rendering instruction is a rendering instruction executed by the server, and the second rendering instruction is a rendering instruction pointed by the terminal;

the second sending module is used for sending the second rendering instruction, the frame image to be rendered corresponding to the second rendering instruction and the timestamp corresponding to the second rendering instruction to the terminal, so that the terminal renders the frame image to be rendered corresponding to the second rendering instruction according to the second rendering instruction, and the rendered second frame image is obtained;

and the third processing module is used for rendering the frame image corresponding to the first rendering instruction according to the first rendering instruction, obtaining the rendered first frame image, and sending the rendered first frame image and the corresponding timestamp to the terminal, so that the terminal determines a GUI rendering result to be displayed according to the rendered first frame image and second frame image and the corresponding timestamps of the rendered first frame image and second frame image.

In a fourth aspect, a terminal is provided, including:

the first sending module is used for sending the interactive operation information to the server so that the server updates the rendering instruction and the timestamp corresponding to each frame image to be rendered and the first frame image rendered by the server according to the interactive operation information, and sends the second rendering instruction, the frame image to be rendered corresponding to the second rendering instruction and the timestamp corresponding to the second rendering instruction to the terminal;

the second receiving module is used for receiving a second rendering instruction sent by the server, a frame image to be rendered corresponding to the second rendering instruction and a timestamp corresponding to the second rendering instruction, rendering the frame image to be rendered corresponding to the second rendering instruction according to the second rendering instruction, and obtaining a rendered second frame image;

and the third receiving module is used for receiving the rendered first frame image and the corresponding timestamp sent by the server, and determining a GUI rendering result to be displayed by combining the rendered first frame image and the rendered second frame image and the corresponding timestamps of the rendered first frame image and the rendered second frame image.

In a fifth aspect, an electronic device is provided, comprising a memory, a processor and a computer program stored on the memory and executable on the processor, the processor implementing the steps of the method as provided in the first or second aspect when executing the program.

A sixth aspect provides a computer readable storage medium having stored thereon a computer program which, when executed by a processor, performs the steps of the method as provided in the first or second aspect.

According to the GUI rendering display method, the terminal, the server, the electronic device and the storage medium, interactive operation information sent by the terminal is received, the rendering instruction of each frame image and the timestamp of the corresponding screen updating period are updated according to the interactive operation information, the server and the terminal respectively render the frame images to be rendered according to the rendering instructions to obtain rendered frame images, the server sends the rendered frame images to the terminal, and the terminal receives the rendered frame images sent by the server and is used for displaying the GUI of the system together with the rendered frame images of the terminal. According to the scheme, the capability of the terminal and the capability of the server for processing the frame images are isolated, the capability is not influenced mutually, the respective capability is fully exerted, synchronization is carried out according to timestamp information, time sequence errors are avoided, the image processing speed can be improved, hardware resources of the terminal and the server can be fully utilized, the overall efficiency is improved, and the user experience is improved.

Drawings

In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings used in the description of the embodiments of the present application will be briefly described below.

Fig. 1 is a schematic diagram of an application environment of a method for rendering and displaying a GUI according to an embodiment of the present application;

fig. 2 is a flowchart illustrating a method for rendering and displaying a GUI according to an embodiment of the present disclosure;

fig. 3 is a schematic diagram of rendering a frame image according to an embodiment of the present disclosure;

fig. 4 is a schematic diagram of a frame image combination according to an embodiment of the present disclosure;

fig. 5 is an interaction diagram of GUI rendering display between a terminal and a server according to an embodiment of the present application;

FIG. 6 is a schematic flowchart of another method for rendering and displaying a GUI according to an embodiment of the present disclosure;

fig. 7 is a schematic structural diagram of a server according to an embodiment of the present application;

fig. 8 is a schematic structural diagram of a terminal according to an embodiment of the present application;

fig. 9 is a schematic structural diagram of an electronic device according to an embodiment of the present application.

Detailed Description

Reference will now be made in detail to the embodiments of the present application, examples of which are illustrated in the accompanying drawings, wherein like or similar reference numerals refer to the same or similar elements or elements having the same or similar function throughout. The embodiments described below with reference to the drawings are exemplary only for the purpose of explaining the present application and are not to be construed as limiting the present invention.

As used herein, the singular forms "a", "an" and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. It will be further understood that the terms "comprises" and/or "comprising," when used in this specification, specify the presence of stated features, integers, steps, operations, elements, and/or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and/or groups thereof. It will be understood that when an element is referred to as being "connected" or "coupled" to another element, it can be directly connected or coupled to the other element or intervening elements may also be present. Further, "connected" or "coupled" as used herein may include wirelessly connected or wirelessly coupled. As used herein, the term "and/or" includes all or any element and all combinations of one or more of the associated listed items.

The application provides a GUI rendering display method, a GUI rendering display device, an electronic device and a storage medium, and aims to solve the technical problems in the prior art.

To make the objects, technical solutions and advantages of the present application more clear, embodiments of the present application will be described in further detail below with reference to the accompanying drawings.

Before describing the various embodiments shown herein, several concepts related to the present application will be described:

1) a GUI (Graphical User Interface) refers to a computer operation User Interface displayed in a Graphical manner, an Interface display format for human-computer communication, which allows a User to use an input device such as a mouse to manipulate icons or menu options on a screen to select commands, call files, start programs, or perform other daily tasks.

2) A GPU (Graphics Processing Unit), also called a display core, a visual processor, and a display chip, is a microprocessor that is specially used for image and Graphics related operations on a personal computer, a workstation, a game machine, and some mobile devices (such as a tablet computer, a smart phone, etc.).

3) A Central Processing Unit (CPU) is a final execution Unit for information Processing and program operation, and serves as an operation and control core of a computer system.

First, the embodiment of the present application may be applied to an application scene of GUI rendering display, for example: cloud desktops, VR, AR cloud rendering, cloud games, and the like, typically run an operating system through a server of a data center to virtualize a desktop of a user, the user connects to the virtual desktop through a client computing protocol from a client device (client or home PC), and the user accesses their desktop as if accessing a traditional locally installed desktop.

It should be understood that the method for GUI rendering and displaying provided by the present application may be applied to any system composed of a computer with a GUI graphics system and a server, and referring to fig. 1, fig. 1 is a schematic diagram of an application environment of a method for GUI rendering and displaying provided by an embodiment of the present application, and takes a method for GUI rendering and displaying jointly executed by a terminal and a server as an example. The GUI rendering and displaying method provided by the embodiment of the application comprises a terminal 10, a server 11 and the like; the terminal 10 may include a mobile phone, a tablet Computer, a notebook Computer, a Personal Computer (PC), or the like, and the server 11 may be an independent physical server, a server cluster or a distributed system configured by a plurality of physical servers, or a cloud server providing cloud computing services. The terminal 10 and the server 11 are connected via a network, such as a wired or wireless network connection. According to the scheme of the embodiment of the application, the timestamp and frame buffer pool technology is adopted, the frame buffer technology is respectively adopted on two sides of the terminal 10 and the server 11, the terminal 10 and the server 11 cooperatively render a graphic drawing instruction to complete the complete GUI drawing work, a frame image is generated and stored in the local frame buffer pools of the terminal 10 and the server 11, a timestamp information mark is added, and the server 11 sends the latest rendered frame image in the server 11 to the terminal 10 according to the timestamp; the terminal 10 receives the latest frame image sent by the server 11, compares the latest frame image with the local latest frame image according to the timestamp, and selects the latest frame image to display on the terminal 10. According to the method and the device, the capability of the terminal and the capability of the server for processing the frame images are isolated, the capability is not influenced, the respective capability is fully exerted, the synchronization is carried out according to the timestamp information, the error is avoided, the image processing speed can be improved, the hardware resources of the terminal and the server can be fully utilized, the overall efficiency is improved, and the user experience is improved.

The following describes the technical solutions of the present application and how to solve the above technical problems with specific embodiments. The following several specific embodiments may be combined with each other, and details of the same or similar concepts or processes may not be repeated in some embodiments. Embodiments of the present application will be described below with reference to the accompanying drawings.

In order to solve the above problem, an embodiment of the present application provides a method for rendering and displaying a GUI. Fig. 2 is a schematic flow diagram of a GUI rendering and displaying method provided in this embodiment, and as shown in fig. 2, the method includes:

s202, receiving the interactive operation information sent by the terminal.

Specifically, receiving interactive operation information sent by the terminal, where the interactive operation information includes movement of a mouse, input of a keyboard, execution logic of an application program, a menu instruction, switching of a window, and the like.

The server receives the transaction operation information sent by the terminal, and lays a foundation for realizing subsequent frame image rendering and selecting frame images meeting requirements.

And S203, updating the rendering instruction and the time stamp corresponding to each frame image to be rendered according to the interactive operation information, and forming a rendering instruction sequence.

After receiving the interactive operation information sent by the terminal, the embodiment of the application needs to determine the rendering instruction of each frame of image according to the interactive operation information, for example: when the terminal user performs interactive operation (moving a mouse, inputting a keyboard, etc.), the operating system of the server or the related application of the server determines the rendering instruction of each frame of image through the function by using the function similar to the response event, and the rendering instruction is generated by each function in the system according to the program logic in response to the external input. The method includes the steps that input is not performed by a terminal, a user does not perform interactive operation (operations such as a keyboard and a mouse are not performed) on the terminal, the server also needs to update each frame image to be rendered, refresh is performed, a rendering instruction and a timestamp corresponding to each frame image to be rendered are updated, then the frame image corresponding to the first rendering instruction is rendered according to the first rendering instruction, and the rendered first frame image is obtained.

The server of the embodiment of the application lays a foundation for realizing the rendering of the subsequent frame images and selecting the frame images meeting the requirements by updating the rendering instruction and the timestamp corresponding to each frame image to be rendered.

S204, acquiring the graphic processing capacity data of the frame images processed by the terminal and the server to determine a first rendering instruction and a second rendering instruction from the rendering instruction sequence, wherein the first rendering instruction is a rendering instruction executed by the server, and the second rendering instruction is a rendering instruction executed by the terminal.

The embodiment of the application needs to acquire the graphic processing capability data of the terminal and the server for processing the frame image, determining rendering tasks allocated to the terminal and the server according to the data analysis, wherein the first rendering instruction is a rendering instruction executed by the server, and the second rendering instruction is a rendering instruction executed by the terminal, and it should be understood that the server needs to acquire the capability of the terminal for processing the frame image, such as the number and the model of the GPUs owned by the terminal and the performance evaluation obtained by scoring with test software, and correspondingly allocating the rendering tasks according to the performance between the terminal and the server, that is, a part of the frame images rendered by the server in the frame image sequence is correspondingly allocated, for example, 10 processing tasks are provided in total, the hardware capability of the server is relatively good, 7 tasks can be processed, and the remaining 3 tasks are allocated to the terminal for processing, so that the processing speed can be increased, and the hardware resources of the terminal and the server can be fully utilized.

According to the method and the device, the performance data of the server and the terminal are obtained, the frame image to be rendered is distributed according to the performance data, the terminal and the server can independently process the frame image, the frame image processing speed is improved, and hardware resources of the terminal and the server can be fully utilized.

S205, sending the second rendering instruction, the frame image to be rendered corresponding to the second rendering instruction and the timestamp corresponding to the second rendering instruction to the terminal, so that the terminal renders the frame image to be rendered corresponding to the second rendering instruction according to the second rendering instruction, and obtains the rendered second frame image.

In the embodiment of the application, the server processes the second part of tasks according to the rendering tasks distributed by the terminal and the image processing performance data of the server, and the rendering tasks of the terminal are sent to the terminal, namely the second rendering instruction, the frame image to be rendered corresponding to the second rendering instruction and the corresponding timestamp, so that the terminal can render the frame image to be rendered corresponding to the second rendering instruction according to the second rendering instruction, obtain the rendered second frame image, and ensure that the required frame image can be rendered.

The second rendering instruction, the frame image to be rendered corresponding to the second rendering instruction and the corresponding timestamp are sent to the terminal, so that the frame image can be rendered by the terminal, hardware resources of the terminal can be fully utilized, and overall efficiency is improved.

S206, rendering the frame image corresponding to the first rendering instruction according to the first rendering instruction, obtaining the rendered first frame image, and sending the rendered first frame image and the corresponding timestamp to the terminal, so that the terminal determines a GUI rendering result to be displayed according to the rendered first frame image and second frame image and the corresponding timestamps of the rendered first frame image and second frame image.

In the embodiment of the application, the server processes the first rendering task according to the rendering task distributed by the image processing performance data of the terminal and the server, that is, the first rendering instruction, the frame image to be rendered corresponding to the first rendering instruction and the timestamp corresponding to the frame image to be rendered, the server renders the frame image corresponding to the first rendering instruction according to the first rendering instruction to obtain the rendered first frame image, and then determines the GUI rendering result to be displayed according to the rendered first frame image, the rendered second frame image, the rendered first frame image and the rendered timestamp corresponding to the rendered second frame image.

The GUI rendering and displaying method provided by the embodiment of the application updates the rendering instruction of each frame image and the timestamp corresponding to the screen updating period according to the interactive operation information sent by the receiving terminal, the server and the terminal respectively render the frame images according to the rendering instruction to obtain rendered frame images, the server sends the rendered frame images to the terminal, the terminal receives the frame images sent by the server, combines the rendered frame images of the terminal, selects the optimal frame images to be displayed on the terminal according to the timestamp, the scheme isolates the frame image processing capabilities of the terminal and the server from each other, fully exerts the respective capabilities, synchronizes according to the timestamp information, avoids the occurrence of timing sequence errors, can improve the image processing rate, and can fully utilize the hardware resources of the terminal and the server, the overall efficiency is improved, and the user experience is improved.

On the basis of the foregoing embodiments, as an optional embodiment, updating, according to the interaction operation information, a rendering instruction and a timestamp corresponding to each frame image to be rendered includes:

updating and processing the frame image to be rendered and the rendering node in the frame image to be rendered according to the interactive operation information sent by the terminal;

and determining a rendering instruction corresponding to the frame image to be rendered according to the frame image to be rendered and a rendering node in the frame image to be rendered, and taking the initiation time of a screen updating period corresponding to the frame image to be rendered as a timestamp corresponding to the frame image to be rendered.

After receiving the exchange operation information sent by the terminal, the server generates all frame images of a complete action according to the exchange operation information, determines an area where each frame image changes according to the change of the action, determines a place to be rendered according to the texture, the coordinate, the transparency and the color of the changed area, generates a rendering instruction, and simultaneously records the moment when the changed area of the frame images is converted into various rendering parameters, which is the initiation time of a response screen update period of the frame image to be rendered, and takes the moment as the timestamp of the frame image, such as: when the terminal user performs an interactive operation (moving a mouse, inputting a keyboard, etc.), the remote desktop is changed, and the operating system of the server or the related application of the server determines a rendering instruction of each frame of image through a function similar to a response event, wherein the rendering instruction is generated by each function in the system according to the program logic in response to an external input. The remote desktop typically includes multiple display interfaces (windows); each window is provided with a plurality of corresponding surfaces (window frame, title bar, menu bar, status bar and content view); each surface may in turn include rendering nodes (characters, primitives, textures); rendering instructions are corresponding to rendering nodes, areas which change each time are detected according to the positions and attributes of windows (whether other windows are covered or not, and whether the windows have transparency or not), areas to be rendered are determined (if the areas to be rendered are covered, drawing is not needed), and then the areas to be rendered are processed through a CPU to obtain rendering instructions, wherein the rendering instructions comprise specific rendering parameters (textures, transparency, colors and the like).

Fig. 3 is a schematic diagram of rendering frame images according to an embodiment of the present disclosure, as shown in fig. 3, where 1 part of the diagram shows an area where a frame image is not transformed, and 2 part of the diagram shows an area where a frame image is transformed, and after receiving interaction operation information sent by a terminal in the embodiment of the present disclosure, a rendering instruction and a timestamp of each frame image need to be updated, first, a time sequence of each frame image, i.e., where the frame image starts and ends, can be determined according to an arrangement position of each frame image, and then, a region where the frame image is transformed, such as a movement of a mouse, can be determined by comparing each frame image, and is established with a coordinate axis at a lower left corner of each frame image, so that a position coordinate of the mouse in each frame image can be found to be changed, and a transformed area in each frame image can be determined according to a change in a coordinate position, converting the area of each frame image into rendering parameters of each frame image through CPU conversion, wherein the rendering parameters comprise color, texture, transparency, coordinates and the like, for example, the area of mouse conversion has unchanged color, unchanged texture and unchanged transparency, and can be converted into coordinate parameters according to the conversion of position, and the rendering parameters of the frame image correspond to the rendering instructions of the frame image. In order to save resources and improve rendering rate, only the area converted in each frame image is converted into rendering parameters, and the area which is not converted in each frame image is reserved, so that the area which is not converted is directly multiplexed when the conversion area is rendered in the subsequent process, and time is saved. And simultaneously recording the time when the image transformation area of each frame is converted into the rendering instruction, and taking the time as the time stamp of each frame image, wherein the time stamp is a part which is used for determining that the interval time of each frame image cannot be lost.

After receiving the interactive operation information sent by the terminal, the embodiment of the application determines the rendering instruction and the timestamp corresponding to each frame image to be rendered, wherein the timestamp is set to ensure the feasibility of the subsequent steps and the time sequence of the frame images, and avoid disorder.

After the rendering instruction of each frame image is determined, rendering the frame image to be rendered corresponding to the rendering instruction according to the rendering instruction, and obtaining the rendered frame image, including:

rendering the rendering parameters corresponding to any frame image in a part of frame images to obtain a rendered transformation area of any frame image;

and combining the rendered transformation area of any frame image with the area which is not transformed in any frame image to obtain the rendered any frame image.

Fig. 4 is a schematic diagram of a frame image combination provided in an embodiment of the present application, as shown in fig. 4, where apart 1 in the diagram indicates an area of a frame image that is not transformed, apart 2 indicates a transformed area of the frame image that has been rendered, and twoparts 1 and 2 are combined into a rendered frame image, and according to a rendering instruction of each frame image, a frame image to be rendered is rendered to obtain a rendered transformed area of each frame image, and the rendered transformed area is combined with the area that is not transformed to obtain a rendered frame image, such as: and rendering a frame image according to the coordinate parameters of the mouse position, wherein only one part of the frame image is a region with the transformed mouse position, and the transformed region and the region without transformation are combined to form a complete frame image.

According to the method and the device, the corresponding frame image to be rendered is rendered according to the rendering instruction in real time, the area where each frame image is transformed is rendered, and the rendered transformed area and the area where the transformation does not occur are combined to form the rendered frame image, so that the rendering time can be saved, the consumption of system resources can be reduced, and the rendering speed is improved.

On the basis of the foregoing embodiments, as an optional embodiment, sending the rendered first frame image and the corresponding timestamp to the terminal, and then further including:

and according to the timestamp corresponding to the rendered first frame image, identifying the rendered first frame image, and sending the identified first frame image to the terminal.

According to the embodiment of the application, after the frame image rendering is completed, the rendered frame image is required to be correspondingly marked by the timestamp corresponding to the frame image, the sequence of the rendered frame image can be clearly known through the timestamp, the display time of the frame image can be conveniently determined according to the timestamp, the optimal frame image is selected, and a foundation is laid for the subsequent GUI rendering display.

Fig. 5 is an interaction diagram of GUI rendering display between a terminal and a server according to an embodiment of the present application; as shown in fig. 5, a terminal sends interoperation information to a server, the server receives the interoperation information, determines a rendering instruction and a timestamp corresponding to each frame of image to be rendered, and forms a rendering instruction sequence, acquires graphics processing capability data of the frame images processed by the terminal and the server, determines a first rendering instruction and a second rendering instruction from the rendering instruction sequence, sends the second rendering instruction, the frame images to be rendered corresponding to the second rendering instruction, and the timestamps corresponding to the frame images to be rendered to the terminal, then the server renders the frame images corresponding to the first rendering instruction according to the first rendering instruction, obtains a rendered first frame of image, and sends the rendered first frame of image and the corresponding timestamp to the terminal, the terminal receives the second rendering instruction sent by the server, the frame images to be rendered corresponding to the second rendering instruction, and the timestamps of the second rendering instruction, and rendering the frame image to be rendered corresponding to the second rendering instruction according to the second rendering instruction, obtaining the rendered second frame image, receiving the rendered first frame image and the corresponding timestamp sent by the server, and determining a GUI rendering result to be displayed by combining the rendered first frame image and second frame image and the corresponding timestamps of the rendered first frame image and second frame image.

The present application further provides a GUI rendering and displaying method, which is applied to a terminal, and fig. 6 is a schematic flowchart of another GUI rendering and displaying method provided in this embodiment, and as shown in fig. 6, the method includes:

s201, sending the interactive operation information to a server, so that the server updates a rendering instruction and a timestamp corresponding to each frame image to be rendered and a first frame image rendered by the server according to the interactive operation information, and sends a second rendering instruction, the frame image to be rendered corresponding to the second rendering instruction and the timestamp corresponding to the second rendering instruction to the terminal.

After the terminal sends the interactive operation information to the server, specifically, the terminal sends the specific interactive operation information of the user to the server according to the operation of the user on the interface, so that the server updates the rendering instruction and the timestamp corresponding to each frame image to be rendered according to the interactive operation information, then renders the frame image corresponding to the first rendering instruction according to the first rendering instruction, and obtains the rendered first frame image. If the user does not perform interactive operation on the terminal, the terminal does not send interactive operation information, but the GUI is still to be refreshed at this time, the rendering instruction and the timestamp corresponding to each frame image to be rendered are updated, and the frame image corresponding to the first rendering instruction is rendered according to the first rendering instruction to obtain the rendered first frame image.

According to the embodiment of the application, the terminal records the interactive operation of the user, the interactive operation information is sent to the server, and the server generates the rendering instruction and the timestamp of each frame of image, so that a foundation is laid for the subsequent realization of GUI rendering display.

S207, receiving a second rendering instruction sent by the server, the frame image to be rendered corresponding to the second rendering instruction and the timestamp corresponding to the second rendering instruction, rendering the frame image to be rendered corresponding to the second rendering instruction according to the second rendering instruction, and obtaining a rendered second frame image.

In the embodiment of the application, the server allocates the second rendering instruction, the frame image to be rendered corresponding to the second rendering instruction and the timestamp of the second rendering instruction to the terminal according to the performance of processing the image by the terminal and the server respectively, rendering the frame image to be rendered corresponding to the second rendering instruction according to the second rendering instruction sent by the server, the frame image to be rendered corresponding to the second rendering instruction and the timestamp of the second rendering instruction to obtain a second frame image, the specific rendering method process comprises the steps of rendering the image of the frame to be rendered through a second rendering instruction to obtain a region where the frame image to be rendered is transformed, combining the region where the frame image is transformed and the region where the frame image is not transformed into a complete rendered frame image, and storing the frame images into a preset buffer pool for buffering the frame images so as to avoid the loss of the frame images.

S208, receiving the rendered first frame image and the corresponding timestamp sent by the server, and determining a GUI rendering result to be displayed by combining the rendered first frame image and the rendered second frame image and the corresponding timestamps of the rendered first frame image and the rendered second frame image.

The embodiment of the application discloses a rendered first frame image and a corresponding timestamp sent by a server, the rendered first frame image and the rendered second frame image and the corresponding timestamps of the rendered first frame image and the rendered second frame image are combined, a GUI rendering result to be displayed needs to be determined according to the timestamps, and the method and the device are specific.

The GUI rendering and displaying method provided by the embodiment of the application updates the rendering instruction and the corresponding timestamp of each frame image by the server according to the interactive operation information by sending the interactive operation information, the server and the terminal respectively render the frame images to be rendered according to the rendering instruction to obtain rendered frame images, the server sends the rendered frame images to the terminal, the terminal receives the rendered frame images sent by the server, the rendered frame images are combined with the rendered frame images of the terminal and are commonly used for displaying the GUI of the system, the scheme isolates the frame image processing capabilities of the terminal and the server, does not influence each other, fully exerts the respective capabilities, synchronizes according to the timestamp information, avoids errors, can improve the image processing rate, can fully utilize the hardware resources of the terminal and the server, and improves the overall efficiency, the user experience is improved.

On the basis of the foregoing embodiments, as an optional embodiment, rendering a frame image to be rendered corresponding to a second rendering instruction according to the second rendering instruction to obtain a rendered second frame image, and then further including:

and identifying the rendered second frame image according to the corresponding timestamp of the rendered second frame image to obtain the identified second frame image.

According to the embodiment of the application, after the frame image rendering is completed, the rendered frame image is required to be correspondingly marked by the timestamp corresponding to the frame image, the sequence of the rendered frame image can be clearly known through the timestamp, the display time of the frame image can be conveniently determined according to the timestamp, the optimal frame image is selected, and a foundation is laid for the subsequent GUI rendering display.

On the basis of the foregoing embodiments, as an optional embodiment, determining a GUI rendering result to be displayed by combining the rendered first frame image and second frame image and timestamps corresponding to the rendered first frame image and second frame image includes:

determining frame images to be displayed in the first frame image and the second frame image for the first frame image and the second frame image;

acquiring the current time, and determining the time interval of the frame image to be displayed according to the timestamp corresponding to the frame image to be displayed and the current time;

if the time interval of the frame images to be displayed is smaller than or equal to a preset threshold value, reserving the frame images to be displayed, and taking the frame images to be displayed as a GUI rendering result to be displayed;

and if the time interval of the frame images to be displayed is greater than the preset threshold value, clearing the frame images to be displayed.

After receiving the first frame image and the second frame image, the embodiment of the application needs to determine the frame image to be displayed, obtain the current time, according to the current time and the timestamp of the frame image to be displayed, the time interval of any frame image can be determined, then the time interval of any frame image is compared with the preset time interval, if the time interval is less than or equal to the preset time interval, the any frame image meets the requirement, and is sent to the terminal, if the time interval is greater than the preset time interval, the any frame image is deleted, the buffer of the buffer pool is cleared, resource occupation is avoided, and other frame images are affected, for example: there are 3 available frame images, X1, X2, X3 in time sequence, and the time intervals of the three frame images are T1, T2, T3, respectively, if the time intervals are too long, the image display will be delayed, inconsistent, and the experience will be affected, if the image display is 60 frames/second, the time interval of each frame should be about 16.6666ms, if the delay of 2 frames between the frame images can be tolerated, the frame image near the time interval 49ms can be considered as the frame image meeting the requirement. Assuming that an interval T1 of X1 is 52ms, an interval T2 of X2 is 46ms, and an interval T3 of X3 is 16ms, in order to display more frames in a unit time and to display smoother frame images, X3 may be used as an optimal frame image, X3 is reserved, where X2 also corresponds to a preset time interval, X2 is also reserved, X2 is reserved as a spare frame image, in order to avoid a situation that when X3 has a problem or is lost, no frame image may be displayed and a null occurs, the time interval of X1 is greater than the preset interval, and if the requirement is not met, X1 is cleared, a memory resource occupied by X1 in a buffer pool is released, and other frame images may be provided, so that the resource is reasonably utilized.

Specifically, the server completes frame image 1 and frame image 2, sends the frame images to the terminal, and writes the frame images into a waiting available frame image queue; at this time, it may be possible for network reasons that only the order in which the frame image 1 and the frame image 2 can be inserted into the terminal available frame queue (in the case of only one network card for transmission) can be ensured, and at this time, the condition of the terminal at this time that the frame image 3 is present cannot be determined, and if rendering is not completed, only the frame image 1 and the frame image 2 are present in the terminal available frame queue; otherwise there are many possibilities that an available frame queue order may occur, which may be: frame image 3 (network busy, server frame image 1, 2 do not go over) or frame image 1, frame image 3 (server frame image 2 has not been transmitted to the terminal) or frame image 3, frame image 1 or frame image 1, frame image 2, frame image 3 or frame image 1, frame image 3, frame image 2 or frame image 3, frame image 2, frame image 1, etc., at this time, the terminal needs to take out an optimal frame image to display according to the timestamp of the frame image, if 3 is selected to refresh the screen, 1, 2 will be abandoned;

there is also a possibility that the server's GPU and network capabilities are too good, the server frame image 4 has already arrived at the terminal, and the frame image 3 task that the terminal is rendering (terminal GPU capabilities are too weak) may be abandoned, the screen is refreshed directly with frame image 4,

according to the frame image display method and device, the frame image meeting the preset requirement is determined according to the timestamp and the current moment of any frame image to be displayed, the display quality of the frame image can be improved, the smoothness of the whole display is ensured, and the overall working efficiency is improved.

An embodiment of the present application provides a server, and as shown in fig. 7, the server may include: thefirst receiving module 302, thefirst processing module 303, thesecond processing module 304, thesecond sending module 305, and thethird processing module 306 specifically:

thefirst receiving module 302 is configured to receive the interactive operation information sent by the terminal.

Thefirst processing module 303 is configured to update a rendering instruction and a timestamp corresponding to each frame image to be rendered according to the interactive operation information, and form a rendering instruction sequence; the time stamp is used for representing the initiation time of the screen updating period corresponding to the rendering instruction of the frame image.

Thesecond processing module 304 is configured to obtain graphics processing capability data of the terminal and the server for processing the frame image, so as to determine a first rendering instruction and a second rendering instruction from the rendering instruction sequence, where the first rendering instruction is a rendering instruction executed by the server, and the second rendering instruction is a rendering instruction pointed by the terminal.

Thesecond sending module 305 is configured to send the second rendering instruction, the frame image to be rendered corresponding to the second rendering instruction, and the timestamp corresponding to the second rendering instruction to the terminal, so that the terminal renders the frame image to be rendered corresponding to the second rendering instruction according to the second rendering instruction, and obtains a rendered second frame image.

Thethird processing module 306 is configured to render the frame image corresponding to the first rendering instruction according to the first rendering instruction, obtain a rendered first frame image, and send the rendered first frame image and the corresponding timestamp to the terminal, so that the terminal determines a GUI rendering result to be displayed according to the rendered first frame image and second frame image and the corresponding timestamps of the rendered first frame image and second frame image.

The server provided in this application embodiment specifically executes the flow of the method embodiment on the server side, and please refer to the content of the embodiment of the GUI rendering display method on the server side, which is not described herein again, in the server provided in this application embodiment, by receiving the interactive operation information sent by the terminal, updating the rendering instruction of each frame image and the timestamp of the corresponding screen update period according to the interactive operation information, the server and the terminal respectively render the frame images to be rendered according to the rendering instruction alone to obtain the rendered frame images, the server sends the rendered frame images to the terminal, the terminal receives the rendered frame images sent by the server, and the rendered frame images are used for the display of the system GUI in combination with the rendered frame images of the terminal, and the capability of the terminal and the server for processing the frame images is isolated and not influenced by each other, respective ability of full play to carry out the synchronization according to the timestamp information, avoid appearing the chronogenesis mistake, not only can improve image processing rate, can also make full use of the hardware resource of terminal and server, promote overall efficiency, improve user experience.

On the basis of the foregoing embodiments, as an alternative embodiment, thefirst processing module 303 includes:

the determining module is used for updating and processing the frame image to be rendered and the rendering node in the frame image to be rendered according to the interactive operation information sent by the terminal;

and determining a rendering instruction corresponding to the frame image to be rendered according to the frame image to be rendered and a rendering node in the frame image to be rendered, and taking the starting time of a screen updating period corresponding to the frame image to be rendered as a timestamp corresponding to the frame image to be rendered.

On the basis of the foregoing embodiments, as an alternative embodiment, thethird processing module 306 includes:

and the first identification module is used for identifying the rendered first frame image according to the corresponding timestamp of the rendered first frame image and sending the identified first frame image to the terminal.

An embodiment of the present application provides a terminal, as shown in fig. 8, the terminal may include: thefirst transmitting module 301, thesecond receiving module 307, and thethird receiving module 308, specifically,

thefirst sending module 301 is configured to send the interactive operation information to the server, so that the server updates, according to the interactive operation information, the rendering instruction and the timestamp corresponding to each frame of image to be rendered and the first frame of image rendered by the server, and sends the second rendering instruction, the frame of image to be rendered corresponding to the second rendering instruction, and the timestamp of the second rendering instruction to the terminal;

asecond receiving module 307, configured to receive a second rendering instruction sent by the server, the frame image to be rendered corresponding to the second rendering instruction, and a timestamp corresponding to the second rendering instruction, render the frame image to be rendered corresponding to the second rendering instruction according to the second rendering instruction, and obtain a rendered second frame image;

thethird receiving module 308 is configured to receive the rendered first frame image and the corresponding timestamp sent by the server, and determine a GUI rendering result to be displayed by combining the rendered first frame image and the rendered second frame image and the corresponding timestamps of the rendered first frame image and the rendered second frame image.

The terminal provided in this application embodiment, specifically executes the flow of the method embodiment at the terminal side, and please refer to the content of the GUI rendering display method at the terminal side in detail, which is not described herein any more, by sending the interactive operation information, the server updates the rendering instruction of each frame image and the timestamp corresponding to the screen update period according to the interactive operation information, the server and the terminal respectively and independently render the frame image to be rendered according to the rendering instruction to obtain the rendered frame image, the server sends the rendered frame image to the terminal, the terminal receives the rendered frame image sent by the server, compares the rendered frame image with the local latest frame image according to the timestamp, and displays the latest frame image at the terminal, which isolates the capability of the terminal for processing the frame image from the server, each other does not influence, full play is own ability to carry out the synchronization according to the timestamp information, avoid appearing the chronogenesis mistake, not only can improve image processing rate, can also make full use of the hardware resources of terminal and server, promote overall efficiency, improve user experience.

On the basis of the foregoing embodiments, as an alternative embodiment, thesecond receiving module 307 includes:

and the second identification module is used for identifying the rendered second frame image according to the corresponding timestamp of the rendered second frame image to obtain the identified second frame image.

On the basis of the foregoing embodiments, as an alternative embodiment, thethird receiving module 308 includes:

the screening module is used for determining frame images to be displayed in the first frame images and the second frame images;

acquiring the current time, and determining the time interval of the frame image to be displayed according to the timestamp corresponding to the frame image to be displayed and the current time;

if the time interval of the frame images to be displayed is smaller than or equal to a preset threshold value, reserving the frame images to be displayed, and taking the frame images to be displayed as a GUI rendering result to be displayed;

and if the time interval of the frame images to be displayed is greater than the preset threshold value, clearing the frame images to be displayed.

An embodiment of the present application provides an electronic device, including: a memory and a processor; at least one program stored in the memory for execution by the processor, which when executed by the processor, implements: the method comprises the steps of receiving interactive operation information sent by a terminal, updating a rendering instruction of each frame image and a timestamp corresponding to a screen updating period according to the interactive operation information, rendering the frame images to be rendered according to the rendering instructions by a server and the terminal respectively and independently to obtain rendered frame images, sending the rendered frame images to the terminal by the server, receiving the rendered frame images sent by the server by the terminal, combining the rendered frame images of the terminal, and being commonly used for the display scheme of a GUI (graphical user interface) of the system to isolate the frame image processing capacities of the terminal and the server from each other without influencing each other and fully exert the respective capacities, and the synchronization is carried out according to the timestamp information, so that the occurrence of time sequence errors is avoided, the image processing rate can be improved, the hardware resources of the terminal and the server can be fully utilized, the overall efficiency is improved, and the user experience is improved.

In an alternative embodiment, an electronic device is provided, as shown in fig. 9, theelectronic device 4000 shown in fig. 9 comprising: aprocessor 4001 and amemory 4003.Processor 4001 is coupled tomemory 4003, such as viabus 4002. Optionally, theelectronic device 4000 may further comprise atransceiver 4004. In addition, thetransceiver 4004 is not limited to one in practical applications, and the structure of theelectronic device 4000 is not limited to the embodiment of the present application.

TheProcessor 4001 may be a CPU (Central Processing Unit), a general-purpose Processor, a DSP (Digital Signal Processor), an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array) or other Programmable logic device, a transistor logic device, a hardware component, or any combination thereof. Which may implement or perform the various illustrative logical blocks, modules, and circuits described in connection with the disclosure. Theprocessor 4001 may also be a combination that performs a computational function, including, for example, a combination of one or more microprocessors, a combination of a DSP and a microprocessor, or the like.

Bus 4002 may include a path that carries information between the aforementioned components. Thebus 4002 may be a PCI (Peripheral Component Interconnect) bus, an EISA (Extended Industry Standard Architecture) bus, or the like. Thebus 4002 may be divided into an address bus, a data bus, a control bus, and the like. For ease of illustration, only one thick line is shown in FIG. 9, but this does not indicate only one bus or one type of bus.

TheMemory 4003 may be a ROM (Read Only Memory) or other types of static storage devices that can store static information and instructions, a RAM (Random Access Memory) or other types of dynamic storage devices that can store information and instructions, an EEPROM (Electrically Erasable Programmable Read Only Memory), a CD-ROM (Compact Disc Read Only Memory) or other optical Disc storage, optical Disc storage (including Compact Disc, laser Disc, optical Disc, digital versatile Disc, blu-ray Disc, etc.), a magnetic Disc storage medium or other magnetic storage devices, or any other medium that can be used to carry or store desired program code in the form of instructions or data structures and that can be accessed by a computer, but is not limited to these.

Thememory 4003 is used for storing application codes for executing the scheme of the present application, and the execution is controlled by theprocessor 4001.Processor 4001 is configured to execute application code stored inmemory 4003 to implement what is shown in the foregoing method embodiments.

The present application provides a computer-readable storage medium, on which a computer program is stored, which, when running on a computer, enables the computer to execute the corresponding content in the foregoing method embodiments. Compared with the prior art, the method updates the rendering instruction of each frame image and the timestamp corresponding to the screen updating period according to the interactive operation information sent by the receiving terminal, the server and the terminal respectively and independently render the frame images to be rendered according to the rendering instruction to obtain rendered frame images, the server sends the rendered frame images to the terminal, the terminal receives the rendered frame images sent by the server, and the rendered frame images of the terminal are combined to be commonly used for the display scheme of the GUI (graphical user interface) of the system to isolate the frame image processing capacity of the terminal and the frame image processing capacity of the server, without mutual influence, fully exert the respective capacity, and synchronize according to the timestamp information to avoid the occurrence of timing sequence errors, thereby not only improving the image processing rate, but also fully utilizing the hardware resources of the terminal and the server and improving the overall efficiency, the user experience is improved.

It should be understood that, although the steps in the flowcharts of the figures are shown in order as indicated by the arrows, the steps are not necessarily performed in order as indicated by the arrows. The steps are not performed in the exact order shown and may be performed in other orders unless explicitly stated herein. Moreover, at least a portion of the steps in the flow chart of the figure may include multiple sub-steps or multiple stages, which are not necessarily performed at the same time, but may be performed at different times, which are not necessarily performed in sequence, but may be performed alternately or alternately with other steps or at least a portion of the sub-steps or stages of other steps.

The foregoing is only a partial embodiment of the present invention, and it should be noted that, for those skilled in the art, various modifications and decorations can be made without departing from the principle of the present invention, and these modifications and decorations should also be regarded as the protection scope of the present invention.

Claims (10)

1. A method for GUI rendering display is applied to a server and is characterized by comprising the following steps:

receiving interactive operation information sent by a terminal;

updating rendering instructions and time stamps corresponding to each frame image to be rendered according to the interactive operation information, and forming a rendering instruction sequence; the time stamp is used for representing the initiation time of a screen updating period corresponding to the rendering instruction of the frame image;

acquiring graphics processing capacity data of a terminal and a server for processing frame images to determine a first rendering instruction and a second rendering instruction from the rendering instruction sequence, wherein the first rendering instruction is a rendering instruction executed by the server, and the second rendering instruction is a rendering instruction executed by the terminal;

sending the second rendering instruction, the frame image to be rendered corresponding to the second rendering instruction and the timestamp corresponding to the second rendering instruction to the terminal, so that the terminal renders the frame image to be rendered corresponding to the second rendering instruction according to the second rendering instruction, and obtains a rendered second frame image;

rendering the frame image corresponding to the first rendering instruction according to the first rendering instruction, obtaining a rendered first frame image, and sending the rendered first frame image and the corresponding timestamp to the terminal, so that the terminal determines a GUI rendering result to be displayed according to the rendered first frame image and second frame image and the corresponding timestamps of the rendered first frame image and second frame image.

2. The method for GUI rendering display according to claim 1, wherein the updating the rendering instruction and the timestamp corresponding to each frame image to be rendered according to the interactive operation information comprises:

updating and processing a frame image to be rendered and a rendering node in the frame image to be rendered according to interactive operation information sent by a terminal;

and for the frame image to be rendered, determining a rendering instruction corresponding to the frame image to be rendered according to the frame image to be rendered and a rendering node in the frame image to be rendered, and taking the initiation time of a screen updating period corresponding to the frame image to be rendered as a timestamp corresponding to the frame image to be rendered.

3. The method for GUI rendering display of claim 1, wherein the sending the rendered first frame image and the corresponding timestamp to the terminal further comprises:

and identifying the rendered first frame image according to the timestamp corresponding to the rendered first frame image, and sending the identified first frame image to the terminal.

4. A method for GUI rendering display is applied to a terminal, and is characterized by comprising the following steps:

sending the interactive operation information to a server, so that the server updates a rendering instruction and a timestamp corresponding to each frame image to be rendered and a first frame image rendered by the server according to the interactive operation information, and sends a second rendering instruction, the frame image to be rendered corresponding to the second rendering instruction and the timestamp corresponding to the second rendering instruction to the terminal;

receiving a second rendering instruction sent by a server, a frame image to be rendered corresponding to the second rendering instruction and a timestamp corresponding to the second rendering instruction, and rendering the frame image to be rendered corresponding to the second rendering instruction according to the second rendering instruction to obtain a rendered second frame image;

and receiving the rendered first frame image and the corresponding timestamp sent by the server, and determining a GUI rendering result to be displayed by combining the rendered first frame image and second frame image and the corresponding timestamps of the rendered first frame image and second frame image.

5. The GUI rendering display method according to claim 4, wherein the rendering the frame image to be rendered corresponding to the second rendering instruction according to the second rendering instruction to obtain a rendered second frame image, and then further comprising:

and identifying the rendered second frame image according to the corresponding timestamp of the rendered second frame image to obtain the identified second frame image.

6. The GUI rendering display method according to claim 4, wherein the determining the GUI rendering result to be displayed by combining the rendered first frame image and second frame image and the corresponding timestamps of the rendered first frame image and second frame image comprises:

determining frame images to be displayed in the first frame image and the second frame image for the first frame image and the second frame image;

acquiring the current time, and determining the time interval of the frame image to be displayed according to the timestamp corresponding to the frame image to be displayed and the current time;

if the time interval of the frame image to be displayed is smaller than or equal to a preset threshold value, reserving the frame image to be displayed, and taking the frame image to be displayed as a GUI rendering result to be displayed;

and if the time interval of the frame image to be displayed is greater than a preset threshold value, clearing the frame image to be displayed.

7. A server, comprising:

the first receiving module is used for receiving the interactive operation information sent by the terminal;

the first processing module is used for updating rendering instructions and timestamps corresponding to each frame image to be rendered according to the interactive operation information and forming a rendering instruction sequence; the time stamp is used for representing the initiation time of a screen updating period corresponding to the rendering instruction of the frame image;

a second processing module, configured to obtain graphics processing capability data of a terminal and a server for processing a frame image, so as to determine a first rendering instruction and a second rendering instruction from the rendering instruction sequence, where the first rendering instruction is a rendering instruction executed by the server, and the second rendering instruction is a rendering instruction pointed to by the terminal;

a second sending module, configured to send the second rendering instruction, the frame image to be rendered corresponding to the second rendering instruction, and the timestamp corresponding to the second rendering instruction to the terminal, so that the terminal renders the frame image to be rendered corresponding to the second rendering instruction according to the second rendering instruction, and obtains a rendered second frame image;

and the third processing module is used for rendering the frame image corresponding to the first rendering instruction according to the first rendering instruction, obtaining the rendered first frame image, and sending the rendered first frame image and the corresponding timestamp to the terminal, so that the terminal determines a GUI rendering result to be displayed according to the rendered first frame image and second frame image and the corresponding timestamps of the rendered first frame image and second frame image.

8. A terminal, comprising:

the first sending module is used for sending the interactive operation information to a server, so that the server updates a rendering instruction and a timestamp corresponding to each frame image to be rendered and a first frame image rendered by the server according to the interactive operation information, and sends the second rendering instruction, the frame image to be rendered corresponding to the second rendering instruction and the timestamp corresponding to the second rendering instruction to the terminal;

a second receiving module, configured to receive the second rendering instruction sent by the server, the frame image to be rendered corresponding to the second rendering instruction, and the timestamp corresponding to the second rendering instruction, and render the frame image to be rendered corresponding to the second rendering instruction according to the second rendering instruction to obtain a rendered second frame image;

and the third receiving module is used for receiving the rendered first frame image and the corresponding timestamp sent by the server, and determining a GUI rendering result to be displayed by combining the rendered first frame image and the rendered second frame image and the corresponding timestamps of the rendered first frame image and the rendered second frame image.

9. An electronic device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, wherein the steps of the method of rendering a display of a GUI according to any of claims 1 to 6 are implemented when the program is executed by the processor.

10. A computer readable storage medium storing computer instructions for causing a computer to perform the steps of a method of rendering a display according to the GUI of any one of claims 1 to 6.

Images