CN112235626B - Video rendering method and device, electronic equipment and storage medium - Google Patents

Abstract

The application discloses a video rendering method, a device, an electronic device and a storage medium, wherein the video rendering method is applied to the electronic device and comprises the following steps: obtaining layout parameters of at least two paths of videos in a preset window, wherein the preset window is a view window of a playing area corresponding to the at least two paths of videos; acquiring texture data corresponding to video images of each video in the at least two paths of videos; rendering texture data corresponding to the video images of each path of video based on the layout parameters to obtain a rendered image corresponding to the preset window; and displaying the rendered image. The method can realize the rendering of the multiple paths of videos through the same view window, and reduce the power consumption of the electronic equipment.

Description

Translated fromChinese

视频渲染方法、装置、电子设备及存储介质Video rendering method, device, electronic device and storage medium

技术领域technical field

本申请涉及视频技术领域，更具体地，涉及一种视频渲染方法、装置、电子设备及存储介质。The present application relates to the field of video technology, and more specifically, to a video rendering method, device, electronic equipment, and storage medium.

背景技术Background technique

随着科技水平的进步，投屏功能已经广泛应用到电子设备中。例如，用户可以将资源提供设备(如智能手机)中的显示界面投射至资源显示设备(如电视)中进行显示，方便用户在资源显示设备上查看资源提供设备中的显示内容。另外，随着投屏技术的发展，在投屏应用场景下，可将多个资源提供设备中的显示内容投射至同一资源显示设备中显示。但是，电子设备在显示多个设备投射的资源时，需要进行大量的渲染处理，造成电子设备的功耗较大。With the advancement of technology, the screen projection function has been widely used in electronic devices. For example, a user can project a display interface in a resource providing device (such as a smart phone) to a resource display device (such as a TV) for display, so that the user can view the display content in the resource providing device on the resource display device. In addition, with the development of screen projection technology, in a screen projection application scenario, the display content in multiple resource providing devices can be projected to the same resource display device for display. However, when an electronic device displays resources projected by multiple devices, it needs to perform a large amount of rendering processing, resulting in high power consumption of the electronic device.

发明内容Contents of the invention

鉴于上述问题，本申请提出了一种视频渲染方法、装置、电子设备及存储介质。In view of the above problems, the present application proposes a video rendering method, device, electronic equipment and storage medium.

第一方面，本申请实施例提供了一种视频渲染方法，应用于电子设备，该视频渲染方法包括：获取至少两路视频于预设窗口中的布局参数，所述预设窗口为所述至少两路视频对应的播放区域的视图窗口；获取所述至少两路视频中每路视频的视频图像对应的纹理数据；基于所述布局参数对所述每路视频的视频图像对应的纹理数据进行渲染，获得所述预设窗口对应的渲染图像；将所述渲染图像进行显示。In the first aspect, an embodiment of the present application provides a video rendering method, which is applied to an electronic device. The video rendering method includes: acquiring layout parameters of at least two channels of video in a preset window, and the preset window is the at least The view window of the playback area corresponding to the two channels of video; acquiring the texture data corresponding to the video image of each channel of the video in the at least two channels of video; rendering the texture data corresponding to the video image of each channel of video based on the layout parameters , obtaining a rendered image corresponding to the preset window; and displaying the rendered image.

第二方面，本申请实施例提供了一种视频渲染装置，应用于电子设备，该视频渲染装置包括：参数获取模块、纹理获取模块、纹理渲染模块以及图像显示模块，其中，所述参数获取模块用于获取至少两路视频于预设窗口中的布局参数，所述预设窗口为所述至少两路视频对应的播放区域的视图窗口；所述纹理获取模块用于获取所述至少两路视频中每路视频的视频图像对应的纹理数据；所述纹理渲染模块用于基于所述布局参数对所述每路视频的视频图像对应的纹理数据进行渲染，获得所述预设窗口对应的渲染图像；所述图像显示模块用于将所述渲染图像进行显示。In the second aspect, the embodiment of the present application provides a video rendering device, which is applied to electronic equipment. The video rendering device includes: a parameter acquisition module, a texture acquisition module, a texture rendering module, and an image display module, wherein the parameter acquisition module It is used to obtain layout parameters of at least two channels of video in a preset window, and the preset window is a view window of a play area corresponding to the at least two channels of video; the texture acquisition module is used to obtain the at least two channels of video The texture data corresponding to the video image of each channel of video; the texture rendering module is used to render the texture data corresponding to the video image of each channel of video based on the layout parameters, and obtain the rendered image corresponding to the preset window ; The image display module is used to display the rendered image.

第三方面，本申请实施例提供了一种电子设备，包括：一个或多个处理器；存储器；一个或多个应用程序，其中所述一个或多个应用程序被存储在所述存储器中并被配置为由所述一个或多个处理器执行，所述一个或多个程序配置用于执行上述第一方面提供的视频渲染方法。In a third aspect, the embodiment of the present application provides an electronic device, including: one or more processors; memory; one or more application programs, wherein the one or more application programs are stored in the memory and The one or more programs are configured to be executed by the one or more processors, and the one or more programs are configured to execute the video rendering method provided in the first aspect above.

第四方面，本申请实施例提供了一种计算机可读取存储介质，所述计算机可读取存储介质中存储有程序代码，所述程序代码可被处理器调用执行上述第一方面提供的视频渲染方法。In the fourth aspect, the embodiment of the present application provides a computer-readable storage medium, the computer-readable storage medium stores program codes, and the program codes can be invoked by a processor to execute the video file provided in the above-mentioned first aspect. rendering method.

本申请提供的方案，通过获取至少两路视频于预设窗口中的布局参数，该预设窗口为该至少两路视频对应的播放区域的视图窗口，获取该至少两路视频中每路视频的视频图像对应的纹理数据，然后基于布局参数对每路视频的视频图像对应的纹理数据进行渲染，获得该预设窗口对应的渲染图像，最后将渲染图像进行显示，从而实现通过同一个视图窗口对多路视频进行渲染，减少渲染次数，降低视频渲染的功耗。In the solution provided by the present application, by obtaining the layout parameters of at least two channels of video in a preset window, the preset window is the view window of the playback area corresponding to the at least two channels of video, and obtaining the layout parameters of each video in the at least two channels of video The texture data corresponding to the video image, and then render the texture data corresponding to the video image of each channel of video based on the layout parameters, obtain the rendered image corresponding to the preset window, and finally display the rendered image, so as to achieve the same view window through the same view window Multi-channel video is rendered, reducing the number of renderings and reducing the power consumption of video rendering.

附图说明Description of drawings

为了更清楚地说明本申请实施例中的技术方案，下面将对实施例描述中所需要使用的附图作简单地介绍，显而易见地，下面描述中的附图仅仅是本申请的一些实施例，对于本领域技术人员来讲，在不付出创造性劳动的前提下，还可以根据这些附图获得其他的附图。In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings that need to be used in the description of the embodiments will be briefly introduced below. Obviously, the drawings in the following description are only some embodiments of the present application. For those skilled in the art, other drawings can also be obtained based on these drawings without any creative effort.

图1示出了本申请实施例提供的应用环境的架构示意图。FIG. 1 shows a schematic diagram of an application environment provided by an embodiment of the present application.

图2示出了根据本申请一个实施例的视频渲染方法流程图。Fig. 2 shows a flowchart of a video rendering method according to an embodiment of the present application.

图3示出了根据本申请另一个实施例的视频渲染方法流程图。Fig. 3 shows a flowchart of a video rendering method according to another embodiment of the present application.

图4示出了本申请另一个实施例提供的显示效果示意图。FIG. 4 shows a schematic diagram of a display effect provided by another embodiment of the present application.

图5示出了根据本申请又一个实施例的视频渲染方法流程图。Fig. 5 shows a flowchart of a video rendering method according to another embodiment of the present application.

图6示出了本申请另一个实施例提供的视频渲染方法的原理示意图。Fig. 6 shows a schematic diagram of the principle of a video rendering method provided by another embodiment of the present application.

图7示出了根据本申请再一个实施例的视频渲染方法流程图。Fig. 7 shows a flowchart of a video rendering method according to yet another embodiment of the present application.

图8示出了根据本申请又另一个实施例的视频渲染方法流程图。Fig. 8 shows a flowchart of a video rendering method according to yet another embodiment of the present application.

图9示出了根据本申请一个实施例的视频渲染装置的一种框图。Fig. 9 shows a block diagram of a video rendering device according to an embodiment of the present application.

图10是本申请实施例的用于执行根据本申请实施例的视频渲染方法的电子设备的框图。Fig. 10 is a block diagram of an electronic device for executing the video rendering method according to the embodiment of the present application according to the embodiment of the present application.

图11是本申请实施例的用于保存或者携带实现根据本申请实施例的视频渲染方法的程序代码的存储单元。Fig. 11 is a storage unit for storing or carrying program codes for realizing the video rendering method according to the embodiment of the present application according to the embodiment of the present application.

具体实施方式Detailed ways

为了使本技术领域的人员更好地理解本申请方案，下面将结合本申请实施例中的附图，对本申请实施例中的技术方案进行清楚、完整地描述。In order to enable those skilled in the art to better understand the solutions of the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application.

目前，投屏功能广泛的应用于生活中，用户可以通过将一个设备中显示的内容投射至另一设备进行显示。例如，用户通过智能手机播放视频时，可以将移动终端上播放的视频投射至另一设备(例如智能电视等)上进行显示，以方便用户有更佳的视频观看体验；又例如，用户通过智能手表查看新闻网页时，可以将智能手表上显示的新闻网页投射到电脑上进行显示，以方便用户查看新闻内容。Currently, the screen projection function is widely used in daily life, and a user can display content displayed on one device to another device for display. For example, when a user plays a video on a smartphone, the video played on the mobile terminal can be projected to another device (such as a smart TV) for display, so that the user can have a better video viewing experience; When viewing the news webpage on the watch, the news webpage displayed on the smart watch can be projected to the computer for display, so that the user can view the news content conveniently.

逐渐的，一些电子设备可以实现多个设备投射的显示内容的显示，电子设备在实现多个设备投射的内容的显示时，通常是每一个设备的显示内容都需要单独进行渲染后再进行显示，例如，每个设备的显示内容都对应一个控件，各自将自己的内容渲染到依附的窗口中，然后提交到系统合成显示，但是在这样的方式中，需要针对每个设备的显示内容进行渲染，就需要多次的渲染，从而给电子设备带来了较大的功耗。并且，如果每个设备都对应一个控件，由于操作系统的平台界面窗口刷新机制在控件添加、删除或者尺寸变化的时候，会从当前窗口的控件根节点开始进行遍历每个控件计算每个控件在当前窗口的位置，这个过程可能会很耗时，而每一次的布局又会导致控件内部对应的图像缓存重新分配，从而使得在这一过程中视频的内容不能及时刷新显示。另外，电子设备在渲染多个设备投射的内容时，通常只能按照固定的布局方式进行显示，导致显示效果不佳，例如，在一些情况下，可能某个设备的内容不能进行显示，却还是会占据显示界面上的显示控件，导致显示界面不能充分的利用。Gradually, some electronic devices can realize the display of the display content projected by multiple devices. When an electronic device realizes the display of the content projected by multiple devices, usually the display content of each device needs to be rendered separately before displaying. For example, the display content of each device corresponds to a control, and each renders its own content into the attached window, and then submits it to the system for composite display. However, in this way, the display content of each device needs to be rendered. Multiple renderings are required, which brings greater power consumption to the electronic device. Moreover, if each device corresponds to a control, because the platform interface window refresh mechanism of the operating system will traverse each control from the control root node of the current window when the control is added, deleted or changed in size The position of the current window, this process may be time-consuming, and each layout will cause the corresponding image buffer inside the control to be reallocated, so that the video content cannot be refreshed and displayed in time during this process. In addition, when an electronic device renders the content projected by multiple devices, it usually can only display it according to a fixed layout, resulting in poor display effect. For example, in some cases, the content of a certain device may not be displayed, but it is still It will occupy the display controls on the display interface, causing the display interface not to be fully utilized.

针对上述问题，发明人提出了本申请实施例提供的视频渲染方法、装置、电子设备以及存储介质，可以实现通过同一个视图窗口对多路视频进行渲染，减少渲染次数，降低视频渲染的功耗。其中，具体的视频渲染方法在后续的实施例中进行详细的说明。In view of the above problems, the inventor proposes a video rendering method, device, electronic device, and storage medium provided by the embodiments of the present application, which can realize rendering of multiple channels of video through the same view window, reduce the number of rendering times, and reduce the power consumption of video rendering . Wherein, the specific video rendering method is described in detail in the following embodiments.

下面对本申请实施例提供的视频渲染方法的应用环境进行介绍。The application environment of the video rendering method provided by the embodiment of the present application is introduced below.

请参阅图1，图1示出了为本申请实施例应用场景的一种架构图，该应用场景可以包括电子设备100以及多个视频输入设备200。电子设备100可以为如图1中所示的智能电视等，当然，电子设备100也可以为其他具有显示功能的设备，例如，台式电脑、笔记本电脑等，在此不做限定。视频输入设备200可以为智能手机、平板电脑、智能手表、电脑等设备。在该应用场景中，视频输入设备200可以作为资源提供设备，将显示内容投射到电子设备100进行显示。当然，视频输入设备200还可以将音频内容传输至电子设备100进行播放。Please refer to FIG. 1 . FIG. 1 shows an architecture diagram of an application scenario according to an embodiment of the present application. The application scenario may include anelectronic device 100 and multiplevideo input devices 200 . Theelectronic device 100 may be a smart TV as shown in FIG. 1 , and of course, theelectronic device 100 may also be other devices with a display function, such as a desktop computer, a notebook computer, etc., which are not limited herein. Thevideo input device 200 may be a device such as a smart phone, a tablet computer, a smart watch, or a computer. In this application scenario, thevideo input device 200 may serve as a resource providing device, and project display content to theelectronic device 100 for display. Of course, thevideo input device 200 can also transmit audio content to theelectronic device 100 for playback.

在一些实施方式中，电子设备100与视频输入设备200可以进行通信，以完成数据交互。其中，电子设备100与视频输入设备200可以通过局域网(local area network，LAN)进行通信，也可以通过广域网(wide area network，WAN)互相通信。例如，电子设备100与视频输入设备200可以同时连接至一个路由器，电子设备100与视频输入设备200可以通过路由器所提供的局域网进行通信；又例如，电子设备100与视频输入设备200可以与云端进行通信，并通过云端实现两者之间的数据交互；再例如，电子设备100与视频输入设备200也可以通过蓝牙、Zigbee、WebRTC等通信方式，建立端到端的网络连接(即P2P网络连接)，并通过建立的网络连接进行通信。当然，电子设备100与视频输入设备200之间的通信方式可以不做限定。In some implementation manners, theelectronic device 100 and thevideo input device 200 can communicate to complete data interaction. Wherein, theelectronic device 100 and thevideo input device 200 may communicate through a local area network (local area network, LAN), or communicate with each other through a wide area network (wide area network, WAN). For example, theelectronic device 100 and thevideo input device 200 can be connected to a router at the same time, and theelectronic device 100 and thevideo input device 200 can communicate through the local area network provided by the router; communication, and realize the data interaction between the two through the cloud; for another example, theelectronic device 100 and thevideo input device 200 can also establish an end-to-end network connection (that is, a P2P network connection) through communication methods such as Bluetooth, Zigbee, and WebRTC. and communicate over the established network connection. Certainly, the communication method between theelectronic device 100 and thevideo input device 200 may not be limited.

在一些场景中，电子设备100可以为大屏设备(例如图1所示的智能电视等)，用户可以将智能手机、平板电脑、智能手表等小屏设备上的显示内容投射至大屏设备进行显示，从而提升用户查看显示内容时的体验。另外，电子设备100可以同时接收多个视频输入设备200投射的内容，并将多个视频输入设备200投射的内容进行显示，从而可以实现多个视频输入设备200投射内容至电子设备100进行显示，使用户可以同时查看多个视频输入设备200投屏的内容，极大地提升了用户体验。In some scenarios, theelectronic device 100 may be a large-screen device (such as a smart TV shown in FIG. Display, thereby improving the user experience when viewing the displayed content. In addition, theelectronic device 100 can simultaneously receive the content projected by multiplevideo input devices 200, and display the content projected by the multiplevideo input devices 200, so that the content projected by multiplevideo input devices 200 can be realized to be displayed on theelectronic device 100, The user can simultaneously view the content projected by multiplevideo input devices 200, which greatly improves the user experience.

在一些实施方式中，电子设备100可以根据接收的多个视频输入设备100的显示内容，对多个视频输入设备输入的显示内容进行显示布局(例如布局位置、显示区域的大小等)，然后基于显示布局，对多个视频输入设备输入的显示内容进行渲染后，在显示屏上进行显示。In some implementations, theelectronic device 100 can display the display content input by multiple video input devices according to the received display content of multiple video input devices 100 (such as layout position, size of the display area, etc.), and then based on Display layout, after rendering the display content input by multiple video input devices, display it on the display screen.

在一些实施方式中，电子设备100可以包括内容接收模块、第一能力协商模块、布局生成模块以及显示渲染模块；视频输入设备200可以包括内容采集模块、内容发送模块以及第二能力协商模块。In some implementations, theelectronic device 100 may include a content receiving module, a first capability negotiation module, a layout generation module, and a display rendering module; thevideo input device 200 may include a content collection module, a content sending module, and a second capability negotiation module.

其中，视频输入设备200可以首先通过内容采集模块采集内容作为待显示内容，再通过内容发送模块将待显示内容通过无线等方式传输给电子设备100，在电子设备100由内容接收模块负责接收。同时，视频输入设备200的第二能力协商模块和电子设备100的第一能力协商模块，负责协商双方的软硬件能力，同时在电子设备100上依据当前画面的路数即显示区域大小，动态协商出最优的显示参数(如分辨率/比特率/帧率)，并反馈给视频输入设备200进行自适应调优。电子设备100的布局生成模块根据当前的约束条件(如接入的输入设备的个数、输入设备自身的显示参数信息、电子设备的显示参数信息等)，综合考虑各方面的因素，依照布局原则，动态生成显示布局。生成好当前的布局方式之后，再交由显示渲染模块依据布局将待显示内容进行绘制并显示。Among them, thevideo input device 200 can first collect content as the content to be displayed through the content collection module, and then transmit the content to be displayed to theelectronic device 100 through the content sending module through wireless or other means, and theelectronic device 100 is responsible for receiving it by the content receiving module. At the same time, the second capability negotiation module of thevideo input device 200 and the first capability negotiation module of theelectronic device 100 are responsible for negotiating the software and hardware capabilities of both parties. At the same time, theelectronic device 100 dynamically negotiates according to the number of channels of the current picture, that is, the size of the display area. The optimal display parameters (such as resolution/bit rate/frame rate) are obtained, and fed back to thevideo input device 200 for adaptive optimization. The layout generation module of theelectronic device 100 comprehensively considers various factors according to the current constraints (such as the number of connected input devices, the display parameter information of the input device itself, the display parameter information of the electronic device, etc.), according to the layout principle , to dynamically generate the display layout. After the current layout mode is generated, the display rendering module draws and displays the content to be displayed according to the layout.

请参阅图1，图1示出了本申请一个实施例提供的视频渲染方法的流程示意图。在具体的实施例中，所述视频渲染方法应用于如图9所示的视频渲染装置400以及配置有所述视频渲染装置400的电子设备100(图10)。下面将以电子设备为例，说明本实施例的具体流程，当然，可以理解的，本实施例所应用的电子设备可以为智能手机、平板电脑、智能手表、智能眼镜、笔记本电脑等，在此不做限定。下面将针对图1所示的流程进行详细的阐述，所述视频渲染方法具体可以包括以下步骤：Please refer to FIG. 1 , which shows a schematic flowchart of a video rendering method provided by an embodiment of the present application. In a specific embodiment, the video rendering method is applied to a video rendering device 400 as shown in FIG. 9 and an electronic device 100 ( FIG. 10 ) configured with the video rendering device 400 . The following will take electronic equipment as an example to illustrate the specific process of this embodiment. Of course, it can be understood that the electronic equipment used in this embodiment can be smart phones, tablet computers, smart watches, smart glasses, notebook computers, etc., here No limit. The process shown in Figure 1 will be described in detail below, and the video rendering method may specifically include the following steps:

步骤S110：获取至少两路视频于预设窗口中的布局参数，所述预设窗口为所述至少两路视频对应的播放区域的视图窗口。Step S110: Obtain layout parameters of at least two videos in a preset window, where the preset window is a view window of a play area corresponding to the at least two videos.

在本申请实施例中，在至少两个视频输入设备同时输入内容至电子设备进行显示时，电子设备可以以同一窗口来进行输入的至少两路视频的显示，以减少电子设备渲染多路视频时的功耗。其中，该至少两路视频不仅限于传统的视频，也可以是视频输入设备显示的内容的屏幕流，例如显示的文档、游戏等。In the embodiment of the present application, when at least two video input devices simultaneously input content to the electronic device for display, the electronic device can use the same window to display at least two channels of input video, so as to reduce the time when the electronic device renders multiple channels of video. power consumption. Wherein, the at least two videos are not limited to traditional videos, but may also be screen streams of content displayed by the video input device, such as displayed documents, games, and the like.

在本申请实施例中，电子设备可以在渲染和显示至少两路视频时，可以获取至少两路视频于预设窗口中的布局参数，以便利用该预设窗口来完成该至少两路视频的渲染和显示。其中，该预设窗口可以为该至少两路视频对应的播放区域的视图窗口，该视图窗口可以实现多路视频作为同一个视图窗口进行渲染和显示，从而后续完成对该视图窗口的图像的渲染后，即可完成多路视频的渲染。In the embodiment of the present application, when rendering and displaying at least two videos, the electronic device can obtain the layout parameters of the at least two videos in the preset window, so as to use the preset window to complete the rendering of the at least two videos and display. Wherein, the preset window can be the view window of the playback area corresponding to the at least two videos, and the view window can realize the rendering and display of multiple videos as the same view window, so as to subsequently complete the rendering of the image of the view window After that, the rendering of the multi-channel video can be completed.

电子设备可以通过OpenGL来实现视频的渲染，OpenGL是一种图形应用程序编程接口，它是一种可以对图形硬件设备特性进行访问的软件库，OpenGL被设计为一个现代化的、硬件无关的接口，因此可以在不考虑计算机操作系统或窗口系统的前提下，在多种不同的图形硬件系统上，完全通过软件的方式实现OpenGL的接口。使用OpenGL程序渲染图像需要执行的主要操作如下：从OpenGL的几何图元中设置数据，用于构建形状；使用不同的着色器对输入的图元数据执行计算操作，判断它们的位置、颜色以及其他渲染属性；将图元数据的数学描述转换为与屏幕位置对应的像素片元(fragment)，这一步也称为光栅化(rasterization)。OpenGL的片元如果最终渲染为图像，那它就是像素；针对光栅化过程产生的片元，执行片元着色器(fragment shader)，从而决定这个片元的最终的颜色和位置。Electronic devices can achieve video rendering through OpenGL. OpenGL is a graphics application programming interface. It is a software library that can access the characteristics of graphics hardware devices. OpenGL is designed as a modern, hardware-independent interface. Therefore, on the premise of not considering the computer operating system or window system, the OpenGL interface can be realized completely through software on a variety of different graphics hardware systems. The main operations that need to be performed to render images using OpenGL programs are as follows: set data from OpenGL geometric primitives to build shapes; use different shaders to perform calculation operations on input primitive data, and determine their positions, colors, and others Rendering attributes; convert the mathematical description of the primitive data into pixel fragments (fragments) corresponding to the screen positions, this step is also called rasterization (rasterization). If the OpenGL fragment is finally rendered as an image, it is a pixel; for the fragment generated by the rasterization process, the fragment shader (fragment shader) is executed to determine the final color and position of the fragment.

电子设备安装有安卓(android)系统时，以上预设窗口可以为SurfaceView。其中，安卓平台提供了多种类型的控件用于展示视频流，SurfaceView是一个控件的同时也是一个窗口，当SurfaceView中的内容更新后可以直接提交到系统合成然后显示，而无需像其他控件需要将其内容渲染到依附的窗口中，然后再提交至系统合成后显示，从而可以有效减少渲染的次数，降低功耗。SurfaceView拥有独立的绘图表面，即它不与其宿主窗口共享同一个绘图表面，由于拥有独立的绘图表面，因此SurfaceView的UI就可以在一个独立的线程中进行绘制，又由于不会占用主线程资源，SurfaceView一方面可以实现复杂而高效的界面绘图，另一方面又不会导致用户输入得不到及时响应。When the electronic device is installed with an android system, the above preset window may be SurfaceView. Among them, the Android platform provides various types of controls for displaying video streams. SurfaceView is a control and also a window. When the content in SurfaceView is updated, it can be directly submitted to the system for synthesis and then displayed, without the need to Its content is rendered to the attached window, and then submitted to the system for synthesis and display, which can effectively reduce the number of renderings and reduce power consumption. SurfaceView has an independent drawing surface, that is, it does not share the same drawing surface with its host window. Because it has an independent drawing surface, the UI of SurfaceView can be drawn in an independent thread, and because it does not occupy the main thread resources, On the one hand, SurfaceView can realize complex and efficient interface drawing, and on the other hand, it will not cause the user input to be unresponsive in time.

在一些实施方式中，以上至少两路视频于预设窗口中的布局参数可以包括至少两路视频的展示区域在预设窗口中的位置、大小等，从而后续电子设备可以根据布局参数对至少两路视频进行渲染和显示后，能使得该至少两路视频能够以对应的显示布局进行显示。In some implementations, the above layout parameters of the at least two videos in the preset window may include the position, size, etc. After rendering and displaying the at least two channels of video, the at least two channels of video can be displayed in a corresponding display layout.

可选的，布局参数可以包括该至少两路视频对应的播放区域在预设窗口中的顶点坐标，后续根据顶点坐标即可对该至少两路视频进行渲染，从而完成对整个预设窗口对应的图像的渲染。Optionally, the layout parameters may include the vertex coordinates of the playback area corresponding to the at least two videos in the preset window, and then the at least two videos can be rendered according to the vertex coordinates, so as to complete the corresponding display of the entire preset window. The rendering of the image.

步骤S120：获取所述至少两路视频中每路视频的视频图像对应的纹理数据。Step S120: Obtain texture data corresponding to the video image of each of the at least two videos.

在本申请实施例中，电子设备还可以获取以上至少两路视频中每路视频的视频图像对应的纹理数据，以渲染视频图像。可以理解地，电子设备通过图形处理器(GPU，Graphics Processing Unit)对视频进行渲染时，需要将视频图像对应的纹理数据提交至GPU进行渲染，因此电子设备可以获取每路视频的视频图像对应的纹理数据。In this embodiment of the present application, the electronic device may also acquire texture data corresponding to a video image of each of the above at least two videos to render the video image. Understandably, when an electronic device renders a video through a graphics processing unit (GPU, Graphics Processing Unit), it needs to submit the texture data corresponding to the video image to the GPU for rendering, so the electronic device can obtain the texture data corresponding to the video image of each channel of video. texture data.

在本申请实施例中，OpenGL中包括两种着色器(shader)，即顶点着色器(vertexshader)和片段着色器(fragment shader)，vertex shader在每个顶点上都执行一次，通过不同世界的坐标系转化定位顶点的最终位置。它可以数据给fragment shader，如纹理坐标、顶点坐标、变换矩阵等。fragment shader在每个像素上都会执行一次，通过插值确定像素的最终显示颜色。In the embodiment of this application, OpenGL includes two kinds of shaders (shader), namely vertex shader (vertexshader) and fragment shader (fragment shader). The vertex shader is executed once on each vertex, through the coordinates of different worlds The final position of the transformed positioning vertex. It can give data to the fragment shader, such as texture coordinates, vertex coordinates, transformation matrices, etc. The fragment shader is executed once on each pixel to determine the final display color of the pixel through interpolation.

在一些实施方式中，电子设备获取每路视频图像的纹理数据，可以通过将视频图像转换为纹理，从而获得视频图像对应的纹理数据。其中，电子设备可以获取视频图像的图像数据，然后将视频图像的图像数据上传至GPU中与纹理绑定，从而得到视频图像对应的纹理数据。In some implementations, the electronic device acquires the texture data of each video image, and may obtain the texture data corresponding to the video image by converting the video image into a texture. Wherein, the electronic device can obtain image data of the video image, and then upload the image data of the video image to the GPU and bind it to the texture, so as to obtain the texture data corresponding to the video image.

步骤S130：基于所述布局参数对所述每路视频的视频图像对应的纹理数据进行渲染，获得所述预设窗口对应的渲染图像。Step S130: Render the texture data corresponding to the video image of each channel of video based on the layout parameters, and obtain the rendered image corresponding to the preset window.

在本申请实施例中，电子设备在获取到每路视频图像对应的纹理数据后，则可以基于以上布局参数对每路视频的视频图像对应的纹理数据进行渲染，从而获得预设窗口对应的渲染图像。其中，电子设备可以通过OpenGL中的EGL来对每路视频的视频图像的纹理数据进行渲染，从而获得预设窗口对应的渲染图像。In this embodiment of the application, after the electronic device acquires the texture data corresponding to each video image, it can render the texture data corresponding to the video image of each video based on the above layout parameters, so as to obtain the rendering corresponding to the preset window. image. Wherein, the electronic device can render the texture data of the video image of each channel of video through EGL in OpenGL, so as to obtain the rendered image corresponding to the preset window.

步骤S140：将所述渲染图像进行显示。Step S140: Display the rendered image.

在本申请实施例中，电子设备在渲染获得到以上渲染图像之后，则可以将渲染图像进行显示，由于以上渲染图像是对多路视频的视频图像，按照布局参数进行渲染后获得的图像，因此，渲染图像中不仅包括了多路视频的视频图像，而且多路视频的视频图像也是按照与布局参数对应的显示布局进行显示。其中，电子设备可以通过系统显示句柄(EGLDisplay)对以上渲染获得的渲染图像进行显示。In the embodiment of the present application, after the electronic device renders and obtains the above rendered image, it can display the rendered image. Since the above rendered image is a multi-channel video video image obtained after rendering according to the layout parameters, therefore , the rendered image not only includes the video image of the multi-channel video, but also displays the video image of the multi-channel video according to the display layout corresponding to the layout parameters. Wherein, the electronic device can display the rendered image obtained by the above rendering through the system display handle (EGLDisplay).

本申请实施例提供的视频渲染方法，通过获取至少两路视频于预设窗口中的布局参数，该预设窗口为该至少两路视频对应的播放区域的视图窗口，获取该至少两路视频中每路视频的视频图像对应的纹理数据，然后基于布局参数对每路视频的视频图像对应的纹理数据进行渲染，获得该预设窗口对应的渲染图像，最后将渲染图像进行显示，从而实现通过同一个视图窗口对多路视频进行渲染，由于是将多路视频的视频图像作为同一个视图窗口进行渲染，因此无需针对每路视频，分别采用一个视图窗口进行渲染，有效减少了渲染次数，减少了电子设备渲染多路视频时的功耗。另外，在需要调整多路视频的显示布局时，只需要对布局参数进行更改，即可实现显示布局的调整，提升了用户体验。In the video rendering method provided by the embodiment of the present application, by acquiring the layout parameters of at least two channels of video in a preset window, the preset window is the view window of the playback area corresponding to the at least two channels of video, and obtaining the layout parameters of the at least two channels of video The texture data corresponding to the video image of each channel of video, and then render the texture data corresponding to the video image of each channel of video based on the layout parameters, obtain the rendered image corresponding to the preset window, and finally display the rendered image, so as to achieve the same One view window renders multiple videos. Since the video images of multiple videos are rendered as the same view window, there is no need to use a view window for rendering each video, which effectively reduces the number of renderings and reduces the The power consumption of an electronic device when rendering multiple videos. In addition, when the display layout of the multi-channel video needs to be adjusted, the adjustment of the display layout can be realized only by changing the layout parameters, which improves the user experience.

请参阅图3，图3示出了本申请另一个实施例提供的视频渲染方法的流程示意图。该视频渲染方法应用于上述电子设备，下面将针对图3所示的流程进行详细的阐述，所述视频渲染方法具体可以包括以下步骤：Please refer to FIG. 3 , which shows a schematic flowchart of a video rendering method provided by another embodiment of the present application. The video rendering method is applied to the above-mentioned electronic device, and the process shown in FIG. 3 will be described in detail below. The video rendering method may specifically include the following steps:

步骤S210：获取所述至少两路视频于所述预设窗口中的显示布局。Step S210: Obtain the display layout of the at least two channels of video in the preset window.

在本申请实施例中，电子设备可以动态获取接收的至少两路视频的显示布局，以动态确定出该至少两路视频于预设窗口中的布局参数。其中，确定的显示布局可以至少包括：每路视频在预设窗口中的位置以及每路视频在预设窗口中的区域大小等，在此不做限定。In the embodiment of the present application, the electronic device may dynamically acquire the display layout of the received at least two videos, so as to dynamically determine the layout parameters of the at least two videos in the preset window. Wherein, the determined display layout may at least include: the position of each video in the preset window and the area size of each video in the preset window, etc., which are not limited here.

在一些实施方式中，电子设备获取至少两路视频于预设窗口中的显示布局，可以包括：获取所述至少两路视频的视频数量；基于所述视频数量，生成所述至少两路视频于所述预设窗口中的显示布局。In some implementations, the acquisition of the display layout of the at least two videos in the preset window by the electronic device may include: acquiring the number of videos of the at least two videos; based on the number of videos, generating the at least two videos in The display layout in the preset window.

在该实施方式中，电子设备可以基于该至少两路视频的视频数量，将预设窗口划分为多个区域，然后将多个区域中每个区域对应一路视频，从而生成该至少两路视频于预设窗口中的显示布局。其中，该至少两路视频在预设窗口中所占的面积之和与预设窗口的面积的比值可以最大。可以理解地，利用该布局方式对至少两路视频于预设窗口中的位置和大小进行布局，可以使得至少两路视频能够在预设窗口中所占的比例最大，使预设窗口中的区域充分利用。In this embodiment, the electronic device may divide the preset window into multiple areas based on the video numbers of the at least two channels of video, and then assign each of the multiple areas to one video, so as to generate the at least two channels of video in The display layout in the preset window. Wherein, the ratio of the sum of the areas occupied by the at least two channels of video in the preset window to the area of the preset window may be the largest. It can be understood that using this layout method to arrange the positions and sizes of at least two channels of video in the preset window can make the proportion of at least two channels of video in the preset window the largest, so that the area in the preset window Take advantage of.

在另一些实施方式中，电子设备也可以基于至少两路视频在其对应的视频输入设备中的显示参数，来生成显示布局。同样的，该至少两路视频在预设窗口中所占的面积之和与预设窗口的面积的比值可以最大。其中，显示参数可以包括每路视频在其视频输入设备中的尺寸信息以及显示状态，该尺寸信息可以包括“长”、“宽”、“长宽比”、“面积”等，显示状态可以包括横屏显示状态和竖屏显示状态。In some other implementation manners, the electronic device may also generate a display layout based on display parameters of at least two channels of video in its corresponding video input device. Likewise, the ratio of the sum of the areas occupied by the at least two channels of video in the preset window to the area of the preset window may be the largest. Among them, the display parameters can include the size information and display status of each video in its video input device, the size information can include "length", "width", "aspect ratio", "area", etc., and the display status can include Horizontal screen display status and vertical screen display status.

作为一种方式，视频对应的视频输入设备可以使用JavaScript判断其为横屏显示状态或者为竖屏显示状态，具体地，视频输入设备一般都支持window.orientation这个参数，则视频输入设备可以通过window.orientation判断出其显示状态，并将该显示状态发送给电子设备。另外，电子设备可以获取视频的尺寸信息，即电子设备可以获取视频的“长”、“宽”、“长宽比”、“面积”等尺寸信息。作为一种方式，视频输入设备可以使用JavaScript获取其可尺寸信息，并将该尺寸信息发送给电子设备。As a way, the video input device corresponding to the video can use JavaScript to judge whether it is in the horizontal screen display state or the vertical screen display state. Specifically, the video input device generally supports the window.orientation parameter, so the video input device can pass the window .orientation determines its display status and sends the display status to the electronic device. In addition, the electronic device can obtain size information of the video, that is, the electronic device can obtain size information such as "length", "width", "aspect ratio", and "area" of the video. As a way, the video input device can use JavaScript to obtain its size information, and send the size information to the electronic device.

在该实施方式中，电子设备通过获取多路视频的显示参数，确定显示布局，可以按照以上尺寸信息以及显示状态来对预设窗口进行划分为多个区域，并将每个区域分配至一路视频，使得不同视频被分配的区域的尺寸大小之间的关系能够与在视频输入设备中的尺寸信息之间的大小关系对应，例如，视频A在视频输入设备中的尺寸信息比视频比在视频输入设备中的尺寸信息大，则视频A所分配的区域的尺寸也比视频B所分配的尺寸大；并且，每路视频在预设窗口中的显示状态也与在视频输入设备中显示状态对应，例如，视频A在视频输入设备中是横屏显示，则分配的区域也应是横屏形状。In this embodiment, the electronic device determines the display layout by acquiring the display parameters of multiple channels of video, and can divide the preset window into multiple areas according to the above size information and display status, and assign each area to a channel of video , so that the relationship between the sizes of the areas where different videos are allocated can correspond to the size relationship between the size information in the video input device, for example, the size information of video A in the video input device is larger than that of the video input If the size information in the device is large, the size of the area allocated by video A is also larger than the size allocated by video B; and, the display status of each video in the preset window also corresponds to the display status in the video input device, For example, if video A is displayed in landscape on the video input device, the allocated area should also be in landscape.

在又一些实施方式中，电子设备也可以获取每路视频对应的视频输入设备与电子设备的相对位置关系，然后基于相对位置关系，来确定显示布局。其中，相对位置关系可以用于表征视频输入设备和电子设备的相对距离和相对角度。In still some implementation manners, the electronic device may also acquire the relative positional relationship between the video input device corresponding to each channel of video and the electronic device, and then determine the display layout based on the relative positional relationship. Wherein, the relative positional relationship can be used to characterize the relative distance and relative angle between the video input device and the electronic device.

在该实施方式中，视频输入设备可以位于电子设备的正前方、可以位于电子设备的左前方、可以位于电子设备的右前方、可以位于电子设备的上前方、也可以位于电子设备的下前方等，第二源设备可以位于电子设备的正前方、可以位于电子设备的左前方、可以位于电子设备的右前方、可以位于电子设备的上前方、也可以位于电子设备的下前方等，在此不做限定。In this embodiment, the video input device may be located directly in front of the electronic device, may be located in the left front of the electronic device, may be located in the right front of the electronic device, may be located in the upper front of the electronic device, or may be located in the lower front of the electronic device, etc. , the second source device may be located directly in front of the electronic device, may be located in the left front of the electronic device, may be located in the right front of the electronic device, may be located in the upper front of the electronic device, or may be located in the lower front of the electronic device, etc. Do limited.

作为第一种方式，电子设备可以包括超宽带传感器(ultra wide band，UWB)，视频输入设备也可以包括超宽带传感器，则电子设备可以通过两者的超宽带传感器获取视频输入设备与电子设备的相对位置关系。As a first method, the electronic device can include an ultra wide band sensor (ultra wide band, UWB), and the video input device can also include an ultra wide band sensor, and the electronic device can obtain the information of the video input device and the electronic device through the ultra wide band sensor of both. Relative positional relationship.

作为第二种方式，电子设备可以包括图像采集装置，则可以通过图像采集装置获取视频输入设备与电子设备的相对位置关系。As a second manner, the electronic device may include an image acquisition device, and then the relative positional relationship between the video input device and the electronic device may be obtained through the image acquisition device.

在本实施例中，在获取电子设备与视频输入设备的相对位置关系后，可以基于相对位置关系，确定显示布局。作为一种方式，当第一视频对应的第一视频输入设备在电子设备的左前方，第二视频对应的第二视频输入设备在电子设备的右前方时，则基于相对位置关系确定的显示布局可以包括：第一视频在预设窗口中的位置位于预设窗口的左部区域，第二视频在预设窗口中的位置位于预设窗口的右部区域等，在此不做限定。In this embodiment, after the relative positional relationship between the electronic device and the video input device is obtained, the display layout may be determined based on the relative positional relationship. As a method, when the first video input device corresponding to the first video is on the left front of the electronic device, and the second video input device corresponding to the second video is on the right front of the electronic device, the display layout determined based on the relative positional relationship It may include: the position of the first video in the preset window is located in the left area of the preset window, the position of the second video in the preset window is located in the right area of the preset window, etc., which are not limited here.

在以上实施方式中，电子设备也可以结合视频数量、每路视频的显示参数以及视频输入设备与电子设备的相对位置关系，来确定每路视频在预设窗口中的显示布局。In the above embodiments, the electronic device may also determine the display layout of each video in the preset window in combination with the number of videos, the display parameters of each video, and the relative positional relationship between the video input device and the electronic device.

步骤S220：基于所述显示布局，确定所述至少两路视频对应的播放区域于所述预设窗口中的顶点坐标作为所述布局参数。Step S220: Based on the display layout, determine the vertex coordinates of the playback area corresponding to the at least two videos in the preset window as the layout parameter.

在本申请实施例中，电子设备在确定显示布局之后，则可以根据显示布局，确定每路视频的播放区域在预设窗口中的顶点坐标，以便将顶点坐标作为布局参数，后续可以根据顶点坐标对各路视频的视频图像对应的纹理数据进行渲染，以得到与显示布局相同的渲染图像。In the embodiment of the present application, after the electronic device determines the display layout, it can determine the vertex coordinates of the playback area of each video in the preset window according to the display layout, so that the vertex coordinates can be used as the layout parameters. Render the texture data corresponding to the video image of each channel of video to obtain a rendered image that is the same as the display layout.

步骤S230：获取所述至少两路视频中每路视频的视频图像对应的纹理数据。Step S230: Obtain texture data corresponding to the video image of each of the at least two videos.

步骤S240：基于所述布局参数对所述每路视频的视频图像对应的纹理数据进行渲染，获得所述预设窗口对应的渲染图像。Step S240: Render the texture data corresponding to the video image of each channel of video based on the layout parameters, and obtain the rendered image corresponding to the preset window.

步骤S250：将所述渲染图像进行显示。Step S250: Display the rendered image.

在本申请实施例中，步骤S230至步骤S250可以参阅其他实施例的内容，在此不再赘述。In the embodiment of the present application, for steps S230 to S250, reference may be made to the contents of other embodiments, which will not be repeated here.

请参阅图4，当根据视频输入设备相对电子设备100的相对位置关系进行显示布局后，通过上述方式进行渲染获得渲染图像并进行显示后，若视频A1对应的视频输入设备在电子设备100的左前方，视频A2对应的视频输入设备在电子设备100的右前方时，则最终视频A1也显示在电子设备100的显示区域中的左部区域，最终视频A2也显示在电子设备100的显示区域中的右部区域。Please refer to FIG. 4. After the display layout is performed according to the relative positional relationship of the video input device relative to theelectronic device 100, and the rendered image is obtained and displayed through the above method of rendering, if the video input device corresponding to the video A1 is on the left side of theelectronic device 100 When the video input device corresponding to the video A2 is in the front right of theelectronic device 100, the final video A1 is also displayed in the left area of the display area of theelectronic device 100, and the final video A2 is also displayed in the display area of theelectronic device 100 the right area of .

步骤S260：在所述至少两路视频的数量更新时，重新获取更新后的至少两路视频于预设窗口中的布局参数。Step S260: when the quantity of the at least two videos is updated, reacquire the updated layout parameters of the at least two videos in the preset window.

在本申请实施例中，电子设备还可以检测视频输入设备所输入的视频的变化，当输入的以上至少两路视频的数量发生更新时，电子设备还可以重新对布局参数进行更新。其中，电子设备对布局参数进行更新，可以前述的确定布局参数的方式进行。In the embodiment of the present application, the electronic device can also detect the change of the video input by the video input device, and when the quantity of the above at least two videos inputted is updated, the electronic device can also re-update the layout parameters. The updating of the layout parameters by the electronic device may be performed in the manner of determining the layout parameters as described above.

当然，在一些方式中，当输入的以上至少两路视频的显示状态、尺寸信息、视频输入设备相对电子设备的相对位置关系中任一信息发生变化时，也可以对布局参数进行更新。从而可以使得布局参数能够与当前投屏的内容相适应，避免了相关技术中视频显示的布局固定所造成的显示效果不佳的问题。Certainly, in some manners, when any of the input display status, size information, and relative positional relationship between the video input device and the electronic device changes, the layout parameters may also be updated. Therefore, the layout parameters can be adapted to the content of the current projected screen, and the problem of poor display effect caused by the fixed layout of the video display in the related art is avoided.

本申请实施例提供的视频渲染方法，电子设备通过确定输入的至少两路视频在预设窗口中的显示布局，来确定布局参数，然后获取该至少两路视频中每路视频的视频图像对应的纹理数据，再基于布局参数对每路视频的视频图像对应的纹理数据进行渲染，获得该预设窗口对应的渲染图像，最后将渲染图像进行显示，从而实现通过同一个视图窗口对多路视频进行渲染，由于是将多路视频的视频图像作为同一个视图窗口进行渲染，因此无需针对每路视频，分别采用一个视图窗口进行渲染，有效减少了渲染次数，减少了电子设备渲染多路视频时的功耗。另外，在输入的至少两路视频发生变化时，能够动态的调整布局参数，使电子设备在展示多路视频时显示布局能够动态生成，提升了投屏时的显示效果。In the video rendering method provided by the embodiment of the present application, the electronic device determines the layout parameters by determining the display layout of at least two input videos in the preset window, and then obtains the corresponding video image of each video in the at least two videos Texture data, and then render the texture data corresponding to the video image of each channel of video based on the layout parameters, obtain the rendered image corresponding to the preset window, and finally display the rendered image, so as to achieve multi-channel video through the same view window Rendering, because the video images of multiple channels of video are rendered as the same view window, so there is no need to use a view window for rendering for each channel of video, which effectively reduces the number of renderings and reduces the time spent on electronic devices when rendering multiple channels of video. power consumption. In addition, when at least two input videos change, the layout parameters can be dynamically adjusted, so that the display layout can be dynamically generated when the electronic device displays multiple videos, and the display effect during screen projection is improved.

请参阅图5，图5示出了本申请又一个实施例提供的视频渲染方法的流程示意图。该视频渲染方法应用于上述电子设备，下面将针对图5所示的流程进行详细的阐述，所述视频渲染方法具体可以包括以下步骤：Please refer to FIG. 5 , which shows a schematic flowchart of a video rendering method provided in another embodiment of the present application. The video rendering method is applied to the above-mentioned electronic equipment, and the following will describe the process shown in Figure 5 in detail, and the video rendering method may specifically include the following steps:

步骤S310：获取至少两路视频于预设窗口中的布局参数，所述预设窗口为所述至少两路视频对应的播放区域的视图窗口。Step S310: Obtain layout parameters of at least two videos in a preset window, where the preset window is a view window of a play area corresponding to the at least two videos.

在本申请实施例中，步骤S310可以参阅前述实施例的内容，在此不再赘述。In the embodiment of the present application, for step S310, reference may be made to the contents of the foregoing embodiments, and details are not repeated here.

步骤S320：利用预设解码器对所述至少两路视频中每路视频的视频图像进行解码，获得所述每路视频的视频图像的解码数据。Step S320: Using a preset decoder to decode the video image of each video in the at least two video channels, to obtain decoded data of the video image of each video channel.

在本申请实施例中，请参阅图6，视频输入设备在投射视频至电子设备进行显示时，会与电子设备之间创建对应的网络连接，电子设备可以通过网络连接获取视频流数据。在一些实施方式中，为减少数据传输的带宽，视频输入设备可以以h264或者h264的格式对视频流进行编码，电子设备在本地以编码的视频帧为单位获取数据，将获取到的编码视频帧传递到预设解码器进行解码。作为一种方式，解码器可以选择FFmpeg或者android平台提供的原生解码器进行解码，其中，选择android平台提供的原生解码器进行解码，可以有效利用本地硬件解码的能力，从而获得更快的解码速度。In the embodiment of the present application, please refer to FIG. 6 , when the video input device projects video to the electronic device for display, it will establish a corresponding network connection with the electronic device, and the electronic device can obtain video stream data through the network connection. In some implementations, in order to reduce the bandwidth of data transmission, the video input device can encode the video stream in h264 or h264 format, and the electronic device acquires data locally in units of encoded video frames, and converts the acquired encoded video frames into Pass to the preset decoder for decoding. As a way, the decoder can choose FFmpeg or the native decoder provided by the android platform for decoding. Among them, choosing the native decoder provided by the android platform for decoding can effectively use the local hardware decoding ability to obtain faster decoding speed. .

步骤S330：根据所述每路视频的视频图像的解码数据，获取所述至少两路视频中每路视频的视频图像对应的纹理数据。Step S330: According to the decoded data of the video image of each video, obtain the texture data corresponding to the video image of each video in the at least two videos.

在本申请实施例中，电子设备在获取到每路视频的视频图像的解码数据后，则可以根据解码数据，获取少两路视频中每路视频的视频图像对应的纹理数据。In the embodiment of the present application, after the electronic device obtains the decoded data of the video image of each video, it can obtain the texture data corresponding to the video image of each video in the least two videos according to the decoded data.

在一些实施方式中，电子设备根据每路视频的视频图像的解码数据，获取至少两路视频中每路视频的视频图像的纹理数据，可以包括：In some implementations, the electronic device obtains the texture data of the video image of each video in at least two channels of video according to the decoded data of the video image of each video channel, which may include:

通过预设的纹理封装接口，将所述每路视频的视频图像的解码数据封装为预设格式的数据；将所述预设格式的数据上传至图形处理器中与纹理绑定，获得所述每路视频的视频图像对应的纹理数据。Through the preset texture encapsulation interface, the decoded data of the video image of each channel of video is encapsulated into the data of the preset format; the data of the preset format is uploaded to the graphics processor and bound with the texture, and the described The texture data corresponding to the video image of each channel of video.

在该实施方式中，解码后的视频数据，需要上传到GPU中作为纹理的数据，android平台解码器解码后的数据是和平台相关，它的数据内存布局对用户也是不透明的，而OpenGL的纹理只支持rgba、rgb和yuv等格式，所以无法使用标准的OpenGL接口上传纹理数据，android平台在这里对纹理数据上传格式做了拓展，屏蔽了平台实现差异，提供了纹理封装接口将数据封装为EGLImageKHR的格式，通过接口上传EGLImageKHR至GPU与纹理绑定，实现更新纹理数据，例如，如图6所示，通过将EGLImageKHR格式的数据传输至GPU来更新纹理。In this embodiment, the decoded video data needs to be uploaded to the GPU as texture data, and the data decoded by the android platform decoder is related to the platform, and its data memory layout is also opaque to the user, while the OpenGL texture Only supports rgba, rgb, and yuv formats, so the standard OpenGL interface cannot be used to upload texture data. The android platform expands the texture data upload format here, shielding platform implementation differences, and provides a texture encapsulation interface to encapsulate data as EGLImageKHR format, upload the EGLImageKHR to the GPU and bind the texture through the interface to update the texture data, for example, as shown in Figure 6, update the texture by transmitting the data in the EGLImageKHR format to the GPU.

步骤S340：基于所述布局参数，确定所述每路视频的视频图像对应的纹理坐标。Step S340: Based on the layout parameters, determine the texture coordinates corresponding to the video images of each channel of video.

在本申请实施例中，电子设备在基于布局参数对视频图像进行渲染时，可以基于布局参数中的顶点坐标，确定出每路视频的视频图像的纹理在预设窗口中的坐标，获得路视频的视频图像对应的纹理坐标。In the embodiment of the present application, when the electronic device renders the video image based on the layout parameters, it can determine the coordinates of the texture of the video image of each video in the preset window based on the vertex coordinates in the layout parameters, and obtain the video The texture coordinates corresponding to the video image.

步骤S350：通过所述预设窗口发起基于所述纹理坐标对所述每路视频的视频图像的纹理数据的渲染操作，获得所述预设窗口对应的渲染图像。Step S350: Initiate a rendering operation on the texture data of the video image of each channel of video based on the texture coordinates through the preset window, and obtain a rendered image corresponding to the preset window.

在本申请实施例中，上预设窗口可以为SurfaceView，在执行渲染之前，SurfaceView对应的渲染线程需要OpenGL环境初始化，其中最重要的是OpenGL中EGLSurface和预设窗口进行绑定。然后，通过与SurfaceView关联的EGLSurface，将纹理数据基于布局参数进行渲染，从而获得预设窗口对应的渲染图像。具体地，电子设备可以先将与SurfaceView关联的EGLSurface的Surface设置为预先创建的SurfaceTexture；然后通过将其Surface设置为预先创建的SurfaceTexture的EGLSurface，将纹理数据基于纹理坐标进行渲染。In the embodiment of this application, the upper preset window can be SurfaceView. Before performing rendering, the rendering thread corresponding to SurfaceView needs to be initialized by the OpenGL environment. The most important thing is that EGLSurface in OpenGL is bound to the preset window. Then, render the texture data based on the layout parameters through the EGLSurface associated with the SurfaceView, so as to obtain the rendered image corresponding to the preset window. Specifically, the electronic device may first set the Surface of the EGLSurface associated with the SurfaceView as a pre-created SurfaceTexture; and then render the texture data based on the texture coordinates by setting its Surface as the EGLSurface of the pre-created SurfaceTexture.

步骤S360：将所述渲染图像进行显示。Step S360: Display the rendered image.

在本申请实施例中，在渲染得到以上渲染图像之后，渲染图像缓存于预设窗口对应的缓冲区，该缓冲区可以为上述的EGLSurface，电子设备可以通过系统显示句柄(EGLDisplay)对以缓冲区中的渲染图像进行显示。In the embodiment of this application, after rendering the above rendered image, the rendered image is cached in the buffer corresponding to the preset window. The buffer can be the above-mentioned EGLSurface, and the electronic device can use the system display handle (EGLDisplay) to buffer the buffer The rendered image in .

本申请实施例提供的视频渲染方法，通过将多路视频作为一个视图窗口进行渲染，因此无需针对每路视频，分别采用一个视图窗口进行渲染，有效减少了渲染次数，减少了电子设备渲染多路视频时的功耗。The video rendering method provided by the embodiment of the present application renders multi-channel video as one view window, so it is not necessary to use a view window for rendering each channel of video, which effectively reduces the number of rendering times and reduces the number of electronic devices rendering multiple channels. Power consumption during video.

请参阅图7，图7示出了本申请再一个实施例提供的视频渲染方法的流程示意图。该视频渲染方法应用于上述电子设备，下面将针对图7所示的流程进行详细的阐述，所述视频渲染方法具体可以包括以下步骤：Referring to FIG. 7 , FIG. 7 shows a schematic flowchart of a video rendering method provided in another embodiment of the present application. The video rendering method is applied to the above-mentioned electronic device, and the process shown in FIG. 7 will be described in detail below. The video rendering method may specifically include the following steps:

步骤S410：获取至少两路视频于预设窗口中的布局参数，所述预设窗口为所述至少两路视频对应的播放区域的视图窗口。Step S410: Obtain layout parameters of at least two videos in a preset window, where the preset window is a view window of a play area corresponding to the at least two videos.

步骤S420：获取所述至少两路视频中每路视频的视频图像对应的纹理数据。Step S420: Obtain texture data corresponding to the video image of each of the at least two videos.

步骤S430：基于所述布局参数对所述每路视频的视频图像对应的纹理数据进行渲染，获得所述预设窗口对应的渲染图像。Step S430: Render the texture data corresponding to the video image of each channel of video based on the layout parameters, and obtain the rendered image corresponding to the preset window.

在本申请实施例中，步骤S410至步骤S430可以参阅前述实施例的内容，在此不再赘述。In the embodiment of the present application, reference may be made to the contents of the foregoing embodiments for steps S410 to S430, and details are not repeated here.

步骤S440：获取播放界面中其他窗口对应的渲染图像，所述其他窗口为视频播放界面中除所述至少两路视频的播放区域以外的其他区域所对应的视图窗口。Step S440: Obtain rendered images corresponding to other windows in the playback interface, where the other windows are view windows corresponding to areas other than the playback areas of the at least two videos in the video playback interface.

在本申请实施例中，预设窗口可以为SurfaceView，SurfaceView拥有独立的绘图表面，即它不与其宿主窗口共享同一个绘图表面。由于还存在宿主窗口、宿主窗口中存在的其他窗口等，例如，电子设备显示多路视频时，除了多路视频还可以显示其他内容，例如用于用户操作界面的控件、状态栏等，因此电子设备还可以获取其他窗口对应的渲染图像。In the embodiment of the present application, the default window may be SurfaceView, and SurfaceView has an independent drawing surface, that is, it does not share the same drawing surface with its host window. Since there are host windows, other windows in the host window, etc., for example, when an electronic device displays multiple videos, other content can be displayed in addition to the multi-channel videos, such as controls and status bars for the user interface, so the electronic The device can also obtain rendered images corresponding to other windows.

步骤S450：将所述预设窗口对应的渲染图像以及所述其他窗口对应的渲染图像进行合成，获得合成图像。Step S450: Synthesize the rendered image corresponding to the preset window and the rendered images corresponding to the other windows to obtain a composite image.

在本申请实施例中，SurfaceView可以提供一个Surface(表面)，嵌入到视图结构层次中，SurfaceView拥有自己的Surface，它与宿主窗口是分离的。因此，在获取到预设窗口对应的渲染图像以及其他窗口对应的渲染图像之后，则可以将这些窗口对应的渲染图像进行合成，从而获得合成图像。其中，电子设备可以按照窗口在屏幕中的位置，对这些窗口对应的渲染图像进行合成，从而获得的合成图像，即为电子设备所显示的整个用户界面，用户界面中包括各个窗口对应的内容。In the embodiment of this application, SurfaceView can provide a Surface (surface), which is embedded in the view structure hierarchy. SurfaceView has its own Surface, which is separated from the host window. Therefore, after the rendered image corresponding to the preset window and the rendered images corresponding to other windows are acquired, the rendered images corresponding to these windows may be synthesized to obtain a synthesized image. Wherein, the electronic device may synthesize the rendered images corresponding to these windows according to the positions of the windows on the screen, so that the obtained composite image is the entire user interface displayed by the electronic device, and the user interface includes content corresponding to each window.

步骤S460：将所述合成图像进行显示。Step S460: Display the synthesized image.

在本申请实施例中，电子设备在对各个窗口的渲染图像进行合成，获得合成图像之后，则可以将合成图像进行显示，从而完成用户界面的显示。In the embodiment of the present application, after the electronic device synthesizes the rendered images of each window to obtain the synthesized image, it may display the synthesized image, thereby completing the display of the user interface.

本申请实施例提供的视频渲染方法，不仅通过将多路视频作为一个视图窗口进行渲染，因此无需针对每路视频，分别采用一个视图窗口进行渲染，有效减少了渲染次数，减少了电子设备渲染多路视频时的功耗。另外，还提供了在显示多路视频时，渲染其他内容对应的窗口的渲染图像，最终将各个窗口对应的渲染图像进行合成后显示，实现在显示多路视频的同时，能够对其他显示内容也进行显示。The video rendering method provided in the embodiment of the present application not only renders multiple videos as one view window, but also does not need to use a view window for rendering each channel of video, which effectively reduces the number of renderings and reduces the number of renderings of electronic devices. Power consumption when streaming video. In addition, when displaying multi-channel video, it also provides rendering images of windows corresponding to other content, and finally synthesizes the rendering images corresponding to each window and displays them, so that other display content can be displayed while multi-channel video is displayed. to display.

请参阅图8，图8示出了本申请又另一个实施例提供的视频渲染方法的流程示意图。该视频渲染方法应用于上述电子设备，下面将针对图8所示的流程进行详细的阐述，所述视频渲染方法具体可以包括以下步骤：Please refer to FIG. 8 . FIG. 8 shows a schematic flowchart of a video rendering method provided in yet another embodiment of the present application. The video rendering method is applied to the above-mentioned electronic device, and the process shown in FIG. 8 will be described in detail below. The video rendering method may specifically include the following steps:

步骤S510：与所述至少两路视频中每路视频对应的视频输入设备建立网络连接。Step S510: Establish a network connection with the video input device corresponding to each of the at least two videos.

步骤S520：获取至少两路视频于预设窗口中的布局参数，所述预设窗口为所述至少两路视频对应的播放区域的视图窗口。Step S520: Obtain layout parameters of at least two videos in a preset window, where the preset window is a view window of a play area corresponding to the at least two videos.

在本申请实施例中，步骤S520可以参阅前述实施例的内容，在此不再赘述。In the embodiment of the present application, for step S520, reference may be made to the content of the foregoing embodiments, and details are not repeated here.

步骤S530：通过所述网络连接接收所述每路视频对应的视频输入设备传输的视频图像。Step S530: Receive the video image transmitted by the video input device corresponding to each channel of video through the network connection.

步骤S540：获取每个视频输入设备传输的视频图像对应的纹理数据。Step S540: Obtain texture data corresponding to video images transmitted by each video input device.

在本申请实施例中，视频渲染方法用于投屏场景时，电子设备可以与各个视频输入设备建立网络连接之后，通过建立的网络连接获取各个视频输入设备传输的视频图像，然后根据每个视频输入设备传输的视频图像，获取各个视频输入设备传输的视频图像对应的纹理数据，以便对多路视频的视频图像进行渲染。In the embodiment of this application, when the video rendering method is used in the screen projection scene, after the electronic device establishes a network connection with each video input device, it can obtain the video images transmitted by each video input device through the established network connection, and then according to each video The video images transmitted by the input device are input, and the texture data corresponding to the video images transmitted by each video input device is acquired, so as to render the video images of multiple channels of video.

步骤S550：基于所述布局参数对所述每路视频的视频图像对应的纹理数据进行渲染，获得所述预设窗口对应的渲染图像。Step S550: Render the texture data corresponding to the video image of each channel of video based on the layout parameters, and obtain the rendered image corresponding to the preset window.

步骤S560：将所述渲染图像进行显示。Step S560: Display the rendered image.

在本申请实施例中，步骤S550以及步骤S560可以参阅前述实施例的内容，在此不再赘述。In the embodiment of the present application, reference may be made to the contents of the foregoing embodiments for step S550 and step S560, and details are not repeated here.

本申请实施例提供的视频渲染方法，应用于多个视频输入设备的投屏场景中，电子设备可以通过同一视图窗口来实现对多个视频输入设备输入的视频的渲染，避免了单独针对每路视频采用一个视图窗口进行渲染，而带来的功耗较高的问题。The video rendering method provided by the embodiment of the present application is applied to the screen projection scene of multiple video input devices. The electronic device can render the video input by multiple video input devices through the same view window, avoiding the need The video uses one view window for rendering, which brings about the problem of high power consumption.

请参阅图9，其示出了本申请实施例提供的一种视频渲染装置400的结构框图。该视频渲染装置400应用上述的电子设备，该视频渲染装置400包括：参数获取模块410、纹理获取模块420、纹理渲染模块430以及图像显示模块440。其中，所述参数获取模块410用于获取至少两路视频于预设窗口中的布局参数，所述预设窗口为所述至少两路视频对应的播放区域的视图窗口；所述纹理获取模块420用于获取所述至少两路视频中每路视频的视频图像对应的纹理数据；所述纹理渲染模块430用于基于所述布局参数对所述每路视频的视频图像对应的纹理数据进行渲染，获得所述预设窗口对应的渲染图像；所述图像显示模块440用于将所述渲染图像进行显示。Please refer to FIG. 9 , which shows a structural block diagram of a video rendering apparatus 400 provided by an embodiment of the present application. The video rendering apparatus 400 applies the above-mentioned electronic equipment, and the video rendering apparatus 400 includes: a parameter acquisition module 410 , a texture acquisition module 420 , a texture rendering module 430 and an image display module 440 . Wherein, the parameter obtaining module 410 is used to obtain the layout parameters of at least two channels of video in a preset window, and the preset window is the view window of the playback area corresponding to the at least two channels of video; the texture acquiring module 420 The texture data corresponding to the video image of each video in the at least two video channels is acquired; the texture rendering module 430 is configured to render the texture data corresponding to the video image of each video channel based on the layout parameters, Obtain a rendered image corresponding to the preset window; the image display module 440 is configured to display the rendered image.

在一些实施方式中，参数获取模块410可以包括：布局获取单元以及参数确定单元。其中，布局获取单元用于获取所述至少两路视频于所述预设窗口中的显示布局；参数确定单元用于基于所述显示布局，确定所述至少两路视频对应的播放区域于所述预设窗口中的顶点坐标作为所述布局参数。In some implementations, the parameter acquisition module 410 may include: a layout acquisition unit and a parameter determination unit. Wherein, the layout acquisition unit is used to acquire the display layout of the at least two videos in the preset window; the parameter determination unit is used to determine the play area corresponding to the at least two videos in the said preset window based on the display layout. The vertex coordinates in the preset window are used as the layout parameters.

在该实施方式下，布局获取单元可以包括数量获取子单元以及布局生成子单元。其中，数量获取子单元用于获取所述至少两路视频的视频数量；布局生成子单元用于基于所述视频数量，生成所述至少两路视频于所述预设窗口中的显示布局。In this implementation manner, the layout acquisition unit may include a quantity acquisition subunit and a layout generation subunit. Wherein, the quantity acquisition subunit is used to acquire the video quantity of the at least two videos; the layout generation subunit is used to generate the display layout of the at least two videos in the preset window based on the video quantity.

在一些实施方式中，纹理获取模块420可以包括：解码数据获取单元以及纹理数据获取单元。其中，解码数据获取单元用于利用预设解码器对所述至少两路视频中每路视频的视频图像进行解码，获得所述每路视频的视频图像的解码数据；纹理数据获取单元用于根据所述每路视频的视频图像的解码数据，获取所述至少两路视频中每路视频的视频图像对应的纹理数据。In some implementations, the texture acquisition module 420 may include: a decoding data acquisition unit and a texture data acquisition unit. Wherein, the decoded data acquisition unit is used to decode the video image of each video in the at least two video channels by using a preset decoder to obtain the decoded data of the video image of each video channel; the texture data acquisition unit is used to decode the video image according to The decoded data of the video image of each channel of video obtains the texture data corresponding to the video image of each channel of video in the at least two channels of video.

在该实施方式下，纹理数据获取单元可以具体用于：通过预设的纹理封装接口，将所述每路视频的视频图像的解码数据封装为预设格式的数据；In this embodiment, the texture data acquisition unit may be specifically configured to: encapsulate the decoded data of the video image of each channel of video into data in a preset format through a preset texture encapsulation interface;

将所述预设格式的数据上传至图形处理器中与纹理绑定，获得所述每路视频的视频图像对应的纹理数据。Upload the data in the preset format to the graphics processor and bind it with the texture to obtain the texture data corresponding to the video image of each channel of video.

在一些实施方式中，纹理渲染模块430可以包括：纹理坐标获取单元以及渲染单元。其中，纹理坐标获取单元用于基于所述布局参数，确定所述每路视频的视频图像对应的纹理坐标；渲染单元用于通过所述预设窗口发起基于所述纹理坐标对所述每路视频的视频图像的纹理数据的渲染操作，获得所述预设窗口对应的渲染图像。In some implementations, the texture rendering module 430 may include: a texture coordinate acquisition unit and a rendering unit. Wherein, the texture coordinate acquisition unit is used to determine the texture coordinates corresponding to the video image of each channel of video based on the layout parameters; The rendering operation of the texture data of the video image to obtain the rendered image corresponding to the preset window.

在一些实施方式中，图像显示模块440可以具体用于：获取播放界面中其他窗口对应的渲染图像，所述其他窗口为视频播放界面中除所述至少两路视频的播放区域以外的其他区域所对应的视图窗口；将所述预设窗口对应的渲染图像以及所述其他窗口对应的渲染图像进行合成，获得合成图像；将所述合成图像进行显示。In some implementations, the image display module 440 may be specifically configured to: acquire rendered images corresponding to other windows in the playback interface, where the other windows are located in areas other than the playback areas of the at least two videos in the video playback interface The corresponding view window; combining the rendering image corresponding to the preset window and the rendering image corresponding to the other windows to obtain a composite image; and displaying the composite image.

在一些实施方式中，该视频渲染装置400还可以包括布局更新模块。布局更新模块用于在所述至少两路视频的数量更新时，重新获取更新后的至少两路视频于预设窗口中的布局参数。In some implementations, the video rendering apparatus 400 may further include a layout updating module. The layout updating module is used for reacquiring the updated layout parameters of the at least two videos in the preset window when the quantity of the at least two videos is updated.

在一些实施方式中，该视频渲染装置400还可以包括连接建立模块。连接建立模块用于在所述获取至少两路视频于预设窗口中的布局参数之前，与所述至少两路视频中每路视频对应的视频输入设备建立网络连接。纹理获取模块可以具体用于：通过所述网络连接接收所述每路视频对应的视频输入设备传输的视频图像；获取每个视频输入设备传输的视频图像对应的纹理数据。In some implementations, the video rendering apparatus 400 may further include a connection establishment module. The connection establishment module is used to establish a network connection with the video input device corresponding to each video in the at least two videos before acquiring the layout parameters of the at least two videos in the preset window. The texture acquisition module may be specifically configured to: receive the video image transmitted by the video input device corresponding to each video through the network connection; acquire the texture data corresponding to the video image transmitted by each video input device.

所属领域的技术人员可以清楚地了解到，为描述的方便和简洁，上述描述装置和模块的具体工作过程，可以参考前述方法实施例中的对应过程，在此不再赘述。Those skilled in the art can clearly understand that for the convenience and brevity of the description, the specific working process of the devices and modules described above can refer to the corresponding process in the foregoing method embodiments, which will not be repeated here.

在本申请所提供的几个实施例中，模块相互之间的耦合可以是电性，机械或其它形式的耦合。In several embodiments provided in the present application, the coupling between the modules may be electrical, mechanical or other forms of coupling.

另外，在本申请各个实施例中的各功能模块可以集成在一个处理模块中，也可以是各个模块单独物理存在，也可以两个或两个以上模块集成在一个模块中。上述集成的模块既可以采用硬件的形式实现，也可以采用软件功能模块的形式实现。In addition, each functional module in each embodiment of the present application may be integrated into one processing module, each module may exist separately physically, or two or more modules may be integrated into one module. The above-mentioned integrated modules can be implemented in the form of hardware or in the form of software function modules.

综上所述，本申请提供的方案，通过获取至少两路视频于预设窗口中的布局参数，该预设窗口为该至少两路视频对应的播放区域的视图窗口，获取该至少两路视频中每路视频的视频图像对应的纹理数据，然后基于布局参数对每路视频的视频图像对应的纹理数据进行渲染，获得该预设窗口对应的渲染图像，最后将渲染图像进行显示，从而实现通过同一个视图窗口对多路视频进行渲染，减少渲染次数，降低视频渲染的功耗。To sum up, the solution provided by this application obtains the layout parameters of at least two channels of video in a preset window, and the preset window is the view window of the playback area corresponding to the at least two channels of video, and obtains the layout parameters of the at least two channels of video The texture data corresponding to the video image of each channel in the video, and then based on the layout parameters, render the texture data corresponding to the video image of each channel of video, obtain the rendered image corresponding to the preset window, and finally display the rendered image, so as to achieve through The same view window renders multiple videos, reducing the number of renderings and reducing the power consumption of video rendering.

请参考图10，其示出了本申请实施例提供的一种电子设备的结构框图。该电子设备100可以是智能电视、笔记本电脑、台式电脑等能够运行应用程序的电子设备。本申请中的电子设备100可以包括一个或多个如下部件：处理器110、存储器120、显示屏130、以及一个或多个应用程序，其中一个或多个应用程序可以被存储在存储器120中并被配置为由一个或多个处理器110执行，一个或多个程序配置用于执行如前述方法实施例所描述的方法。Please refer to FIG. 10 , which shows a structural block diagram of an electronic device provided by an embodiment of the present application. Theelectronic device 100 may be an electronic device capable of running application programs, such as a smart TV, a notebook computer, and a desktop computer. Theelectronic device 100 in the present application may include one or more of the following components: a processor 110, amemory 120, adisplay screen 130, and one or more application programs, wherein one or more application programs may be stored in thememory 120 and It is configured to be executed by one or more processors 110, and one or more programs are configured to execute the methods described in the foregoing method embodiments.

处理器110可以包括一个或者多个处理核。处理器110利用各种接口和线路连接整个电子设备100内的各个部分，通过运行或执行存储在存储器120内的指令、程序、代码集或指令集，以及调用存储在存储器120内的数据，执行电子设备100的各种功能和处理数据。可选地，处理器110可以采用数字信号处理(Digital Signal Processing，DSP)、现场可编程门阵列(Field－Programmable Gate Array，FPGA)、可编程逻辑阵列(Programmable LogicArray，PLA)中的至少一种硬件形式来实现。处理器110可集成中央处理器(CentralProcessing Unit，CPU)、图像处理器(Graphics Processing Unit，GPU)和调制解调器等中的一种或几种的组合。其中，CPU主要处理操作系统、用户界面和应用程序等；GPU用于负责显示内容的渲染和绘制；调制解调器用于处理无线通信。可以理解的是，上述调制解调器也可以不集成到处理器110中，单独通过一块通信芯片进行实现。Processor 110 may include one or more processing cores. The processor 110 uses various interfaces and circuits to connect various parts of the entireelectronic device 100, and executes or executes instructions, programs, code sets or instruction sets stored in thememory 120, and calls data stored in thememory 120 to execute Various functions of theelectronic device 100 and processing data. Optionally, the processor 110 may use at least one of Digital Signal Processing (Digital Signal Processing, DSP), Field-Programmable Gate Array (Field-Programmable Gate Array, FPGA), and Programmable Logic Array (Programmable LogicArray, PLA). implemented in the form of hardware. The processor 110 may integrate one or a combination of a central processing unit (Central Processing Unit, CPU), an image processor (Graphics Processing Unit, GPU), a modem, and the like. Among them, the CPU mainly handles the operating system, user interface and application programs, etc.; the GPU is used to render and draw the displayed content; the modem is used to handle wireless communication. It can be understood that, the above-mentioned modem may also not be integrated into the processor 110, but implemented by a communication chip alone.

存储器120可以包括随机存储器(Random Access Memory，RAM)，也可以包括只读存储器(Read-Only Memory)。存储器120可用于存储指令、程序、代码、代码集或指令集。存储器120可包括存储程序区和存储数据区，其中，存储程序区可存储用于实现操作系统的指令、用于实现至少一个功能的指令(比如触控功能、声音播放功能、图像播放功能等)、用于实现下述各个方法实施例的指令等。存储数据区还可以存储电子设备100在使用中所创建的数据(比如电话本、音视频数据、聊天记录数据)等。Thememory 120 may include a random access memory (Random Access Memory, RAM), and may also include a read-only memory (Read-Only Memory). Thememory 120 may be used to store instructions, programs, codes, sets of codes, or sets of instructions. Thememory 120 may include a program storage area and a data storage area, wherein the program storage area may store instructions for implementing an operating system, instructions for implementing at least one function (such as a touch function, a sound playback function, an image playback function, etc.) , instructions for implementing the following method embodiments, and the like. The storage data area can also store data created during use of the electronic device 100 (such as phonebook, audio and video data, chat record data) and the like.

显示屏130用于显示由用户输入的信息、提供给用户的信息以及所述电子设备100的各种图形用户接口，这些图形用户接口可以由图形、文本、图标、数字、视频和其任意组合来构成，在一个实例中，该显示屏130可以为液晶显示器(Liquid Crystal Display，LCD)，也可以为有机发光二极管(Organic Light-Emitting Diode，OLED)，在此不做限定。Thedisplay screen 130 is used to display information input by the user, information provided to the user, and various graphical user interfaces of theelectronic device 100. These graphical user interfaces can be composed of graphics, text, icons, numbers, videos, and any combination thereof In one example, thedisplay screen 130 may be a liquid crystal display (Liquid Crystal Display, LCD) or an organic light-emitting diode (Organic Light-Emitting Diode, OLED), which is not limited herein.

请参考图11，其示出了本申请实施例提供的一种计算机可读存储介质的结构框图。该计算机可读介质800中存储有程序代码，所述程序代码可被处理器调用执行上述方法实施例中所描述的方法。Please refer to FIG. 11 , which shows a structural block diagram of a computer-readable storage medium provided by an embodiment of the present application. Program codes are stored in the computer-readable medium 800, and the program codes can be invoked by a processor to execute the methods described in the foregoing method embodiments.

计算机可读存储介质800可以是诸如闪存、EEPROM(电可擦除可编程只读存储器)、EPROM、硬盘或者ROM之类的电子存储器。可选地，计算机可读存储介质800包括非易失性计算机可读介质(non-transitory computer-readable storage medium)。计算机可读存储介质800具有执行上述方法中的任何方法步骤的程序代码810的存储空间。这些程序代码可以从一个或者多个计算机程序产品中读出或者写入到这一个或者多个计算机程序产品中。程序代码810可以例如以适当形式进行压缩。The computer readable storage medium 800 may be an electronic memory such as flash memory, EEPROM (Electrically Erasable Programmable Read Only Memory), EPROM, hard disk, or ROM. Optionally, the computer-readable storage medium 800 includes a non-transitory computer-readable storage medium (non-transitory computer-readable storage medium). The computer-readable storage medium 800 has a storage space forprogram code 810 for executing any method steps in the above-mentioned methods. These program codes can be read from or written into one or more computer program products.Program code 810 may, for example, be compressed in a suitable form.

最后应说明的是：以上实施例仅用以说明本申请的技术方案，而非对其限制；尽管参照前述实施例对本申请进行了详细的说明，本领域的普通技术人员当理解：其依然可以对前述各实施例所记载的技术方案进行修改，或者对其中部分技术特征进行等同替换；而这些修改或者替换，并不驱使相应技术方案的本质脱离本申请各实施例技术方案的精神和范围。Finally, it should be noted that: the above embodiments are only used to illustrate the technical solutions of the present application, but not to limit them; although the present application has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that: it can still Modifications are made to the technical solutions described in the foregoing embodiments, or equivalent replacements are made to some of the technical features; and these modifications or replacements do not drive the essence of the corresponding technical solutions away from the spirit and scope of the technical solutions of the various embodiments of the present application.

Claims (10)

1. A video rendering method, applied to an electronic device, the method comprising:

obtaining layout parameters of at least two paths of videos in a preset window, wherein the layout parameters are determined according to the generated display layout, the display layout is determined according to the relative position relation between video input equipment corresponding to each path of video in the at least two paths of videos and the electronic equipment, the position of each path of video in the preset window corresponds to the relative position relation, the preset window is a view window of a playing area corresponding to the at least two paths of videos, the preset window is a surface view in an android system, the surface view is a window at the same time of one control, the surface view is used for directly submitting the content of the surface view to the system for display after synthesis, and the surface view has an independent drawing surface and does not share the same drawing surface with a host window;

acquiring texture data corresponding to video images of each video in the at least two paths of videos;

determining texture coordinates corresponding to the video images of each path of video based on the layout parameters;

initiating a rendering operation of texture coordinates based on the texture data of the video images of each path of video through the preset window to obtain a rendering image corresponding to the preset window, wherein the rendering image is cached in a buffer area corresponding to the preset window, and the rendering image in the buffer area is displayed through a system display handle;

Acquiring rendered images corresponding to other windows in a playing interface, wherein the other windows are view windows corresponding to other areas except for playing areas of at least two paths of videos in a video playing interface;

synthesizing the rendering image corresponding to the preset window and the rendering images corresponding to the other windows to obtain a synthesized image;

and displaying the synthesized image.

2. The method of claim 1, wherein the obtaining layout parameters of at least two paths of video in a predetermined window comprises:

acquiring display layout of the at least two paths of videos in the preset window;

and determining vertex coordinates of play areas corresponding to the at least two paths of videos in the preset window as the layout parameters based on the display layout.

3. The method of claim 2, wherein the obtaining the display layout of the at least two paths of video in the preset window comprises:

acquiring the video quantity of the at least two paths of videos;

and generating display layouts of the at least two paths of videos in the preset window based on the video quantity.

4. The method according to claim 1, wherein the obtaining texture data corresponding to the video image of each of the at least two paths of video comprises:

Decoding video images of each path of video in the at least two paths of video by using a preset decoder to obtain decoded data of the video images of each path of video;

and acquiring texture data corresponding to the video image of each video in the at least two paths of videos according to the decoded data of the video image of each path of video.

5. The method according to claim 4, wherein the obtaining texture data corresponding to the video image of each of the at least two paths of video according to the decoded data of the video image of each path of video comprises:

packaging the decoded data of the video image of each path of video into data in a preset format through a preset texture packaging interface;

uploading the data in the preset format to a graphics processor to bind with textures, and obtaining texture data corresponding to the video images of each path of video.

6. The method according to any one of claims 1-5, further comprising:

and when the number of the at least two paths of videos is updated, the layout parameters of the at least two paths of updated videos in a preset window are acquired again.

7. The method according to any one of claims 1-5, wherein prior to the obtaining layout parameters of at least two paths of video in a predetermined window, the method further comprises:

Establishing network connection with video input equipment corresponding to each path of video in the at least two paths of videos;

the obtaining texture data corresponding to the video image of each video in the at least two paths of videos includes:

receiving video images transmitted by the video input equipment corresponding to each path of video through the network connection;

and acquiring texture data corresponding to the video image transmitted by each video input device.

8. A video rendering apparatus, characterized by being applied to an electronic device, the apparatus comprising: a parameter acquisition module, a texture rendering module and an image display module, wherein,

the parameter acquisition module is used for acquiring layout parameters of at least two paths of videos in a preset window, the layout parameters are determined according to the generated display layout, the display layout is determined according to the relative position relation between video input equipment corresponding to each path of video in the at least two paths of videos and the electronic equipment, the position of each path of video in the preset window corresponds to the relative position relation, the preset window is a view window of a playing area corresponding to the at least two paths of videos, the preset window is a surface view in an android system, the surface view is a control and is a window at the same time, the surface view is used for directly submitting the content of the surface view to a system for display after synthesis, and the surface view has an independent drawing surface and does not share the same drawing surface with a host window;

The texture acquisition module is used for acquiring texture data corresponding to video images of each path of video in the at least two paths of video;

the texture rendering module is used for determining texture coordinates corresponding to the video images of each path of video based on the layout parameters; initiating a rendering operation of texture coordinates based on the texture data of the video images of each path of video through the preset window to obtain a rendering image corresponding to the preset window, wherein the rendering image is cached in a buffer area corresponding to the preset window, and the rendering image in the buffer area is displayed through a system display handle;

the image display module is used for obtaining rendering images corresponding to other windows in the playing interface, wherein the other windows are view windows corresponding to other areas except the playing areas of the at least two paths of videos in the video playing interface; synthesizing the rendering image corresponding to the preset window and the rendering images corresponding to the other windows to obtain a synthesized image; and displaying the synthesized image.

9. An electronic device, comprising:

a display screen;

one or more processors;

a memory;

One or more applications, wherein the one or more applications are stored in the memory and configured to be executed by the one or more processors, the one or more applications configured to perform the method of any of claims 1-7.

10. A computer readable storage medium, characterized in that the computer readable storage medium has stored therein a program code, which is callable by a processor for executing the method according to any one of claims 1-7.

Images