CN114339401A - A kind of video background processing method and device - Google Patents
A kind of video background processing method and deviceDownload PDFInfo
- Publication number
- CN114339401A CN114339401ACN202111652168.7ACN202111652168ACN114339401ACN 114339401 ACN114339401 ACN 114339401ACN 202111652168 ACN202111652168 ACN 202111652168ACN 114339401 ACN114339401 ACN 114339401A
- Authority
- CN
- China
- Prior art keywords
- video
- background
- window
- display
- background image
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000003672processing methodMethods0.000titleclaimsabstractdescription28
- 230000004927fusionEffects0.000claimsabstractdescription26
- 238000007499fusion processingMethods0.000claimsabstractdescription21
- 238000000034methodMethods0.000claimsabstractdescription20
- 238000012545processingMethods0.000claimsdescription25
- 238000004891communicationMethods0.000claimsdescription13
- 238000000605extractionMethods0.000claimsdescription13
- 238000012986modificationMethods0.000description5
- 230000004048modificationEffects0.000description5
- 230000006870functionEffects0.000description4
- 230000008569processEffects0.000description4
- 238000009877renderingMethods0.000description3
- 230000003068static effectEffects0.000description3
- 230000008859changeEffects0.000description2
- 238000011161developmentMethods0.000description2
- 238000010586diagramMethods0.000description2
- 239000000203mixtureSubstances0.000description2
- 230000001133accelerationEffects0.000description1
- 239000002131composite materialSubstances0.000description1
- 238000005516engineering processMethods0.000description1
- 238000003286fusion draw glass processMethods0.000description1
- 238000012423maintenanceMethods0.000description1
- 230000003287optical effectEffects0.000description1
- 230000000750progressive effectEffects0.000description1
- 238000013519translationMethods0.000description1
Images
Landscapes
- Studio Circuits (AREA)
Abstract
Description
Translated fromChinese技术领域technical field
本发明涉及图像处理技术领域，具体而言，涉及一种视频背景处理方法及装置。The present invention relates to the technical field of image processing, and in particular, to a video background processing method and device.
背景技术Background technique
在视频直播课程或者视频会议过程中，经常需要对直播主播或直播讲师的摄像头拍摄背景进行替换，以达到丰富展示的目的。具体是将人像从复杂背景中抠出，再与背景进行合并展示，可以达到在视频应用场景中实时且丰富展示的目的。During a live video course or a video conference, it is often necessary to replace the camera shooting background of the live host or live lecturer to achieve the purpose of enriching the presentation. Specifically, the portrait is cut out from the complex background, and then combined with the background for display, which can achieve the purpose of real-time and rich display in the video application scene.
目前，只能将提取的人像与静态的图片进行融合，提取的人像无法与多个静态的图片或动态的视频进行融合。At present, only the extracted portrait can be fused with a static picture, and the extracted portrait cannot be fused with multiple static pictures or dynamic videos.
发明内容SUMMARY OF THE INVENTION
有鉴于此，本发明的目的在于提供一种视频背景处理方法及装置，能够实现定制化的多层背景场景，达到实时且丰富展示的目的。In view of this, the purpose of the present invention is to provide a video background processing method and device, which can realize customized multi-layer background scenes, and achieve the purpose of real-time and rich display.
第一方面，本发明提供了一种视频背景处理方法，该方法包括：In a first aspect, the present invention provides a video background processing method, the method comprising:
分别获取背景视频和背景图像以及分别确定所述背景视频中每个视频帧和所述背景图像在显示窗口上的窗口堆叠顺序和窗口坐标；Obtaining the background video and the background image respectively and respectively determining the window stacking order and window coordinates of each video frame in the background video and the background image on the display window;
基于所述背景视频中每个视频帧对应的窗口堆叠顺序和窗口坐标以及所述背景图像对应的窗口堆叠顺序和窗口坐标，将所述背景视频和所述背景图像进行融合处理得到融合视频；Based on the window stacking sequence and window coordinates corresponding to each video frame in the background video and the window stacking sequence and window coordinates corresponding to the background image, the background video and the background image are fused to obtain a fused video;
采集实时视频并对所述实时视频中的目标对象进行提取处理得到包含所述目标对象的提取视频；collecting real-time video and extracting the target object in the real-time video to obtain an extraction video containing the target object;
分别将所述提取视频和所述融合视频作为前景和背景进行融合处理。The extracted video and the fused video are respectively used as foreground and background for fusion processing.
在一实施例中，在所述确定所述背景视频中每个视频帧和所述背景图像在显示窗口上的窗口堆叠顺序和窗口坐标之后，还包括：In one embodiment, after the determining the window stacking sequence and window coordinates of each video frame in the background video and the background image on the display window, the method further includes:
对所述显示窗口上的窗口堆叠顺序或窗口坐标进行修改处理。Modifying the window stacking order or window coordinates on the display window.
在一实施例中，在所述确定所述背景视频中每个视频帧和所述背景图像在显示窗口上的窗口堆叠顺序和窗口坐标之后，还包括：In one embodiment, after the determining the window stacking sequence and window coordinates of each video frame in the background video and the background image on the display window, the method further includes:
分别确定所述背景视频在显示窗口上的显示区域和所述背景图像在显示窗口上的显示区域。The display area of the background video on the display window and the display area of the background image on the display window are respectively determined.
在一实施例中，在所述分别确定所述背景视频在显示窗口上的显示区域和所述背景图像在显示窗口上的显示区域之后，还包括：In an embodiment, after the said determining the display area of the background video on the display window and the display area of the background image on the display window respectively, the method further includes:
对所述背景视频在显示窗口上的显示区域或所述背景图像在显示窗口上的显示区域进行显示区域的缩放处理。The display area scaling process is performed on the display area of the background video on the display window or the display area of the background image on the display window.
在一实施例中，所述分别将所述提取视频和所述融合视频作为前景和背景进行融合处理，包括：In one embodiment, performing fusion processing on using the extracted video and the fusion video as foreground and background, respectively, includes:
确定所述提取视频的每一视频帧中所述目标对象上所有像素点的像素点透明度；determining the pixel transparency of all pixels on the target object in each video frame of the extracted video;
基于所述目标对象上所有像素点的像素点透明度将所述提取视频和所述融合视频进行融合处理；Perform fusion processing on the extracted video and the fusion video based on the pixel transparency of all pixels on the target object;
其中，所述像素点透明度为提取视频中所述目标对象对应的每个像素点在显示窗口上的透明程度。Wherein, the transparency of the pixel point is the degree of transparency of each pixel point corresponding to the target object in the extracted video on the display window.
第二方面，本发明提供了一种视频背景处理装置，该装置包括：In a second aspect, the present invention provides a video background processing device, the device comprising:
获取单元，用于分别获取背景视频和背景图像以及分别确定所述背景视频中每个视频帧和所述背景图像在显示窗口上的窗口堆叠顺序和窗口坐标；an acquisition unit, used to acquire a background video and a background image respectively and to determine the window stacking sequence and window coordinates of each video frame in the background video and the background image on the display window respectively;
背景单元，用于基于所述背景视频中每个视频帧对应的窗口堆叠顺序和窗口坐标以及所述背景图像对应的窗口堆叠顺序和窗口坐标，将所述背景视频和所述背景图像进行融合处理得到融合视频；The background unit is configured to perform fusion processing on the background video and the background image based on the window stacking sequence and window coordinates corresponding to each video frame in the background video and the window stacking sequence and window coordinates corresponding to the background image get the fusion video;
提取单元，用于采集实时视频并对所述实时视频中的目标对象进行提取处理得到包含所述目标对象的提取视频；an extraction unit, configured to collect real-time video and perform extraction processing on the target object in the real-time video to obtain an extraction video containing the target object;
融合单元，用于分别将所述提取视频和所述融合视频作为前景和背景进行融合处理。A fusion unit, configured to perform fusion processing on the extracted video and the fusion video as foreground and background respectively.
在一实施例中，还包括：In one embodiment, it also includes:
修改单元，用于对所述显示窗口上的窗口堆叠顺序或窗口坐标进行修改处理。A modification unit, configured to modify the window stacking order or the window coordinates on the display window.
在一实施例中，还包括：In one embodiment, it also includes:
调整单元，用于分别确定所述背景视频在显示窗口上的显示区域和所述背景图像在显示窗口上的显示区域。An adjustment unit, configured to respectively determine a display area of the background video on the display window and a display area of the background image on the display window.
第三方面，本发明提供了一种电子设备，包括：处理器、存储器、通信接口和通信总线；其中，处理器、通信接口和存储器通过通信总线完成相互间的通信；In a third aspect, the present invention provides an electronic device, comprising: a processor, a memory, a communication interface and a communication bus; wherein the processor, the communication interface and the memory communicate with each other through the communication bus;
处理器用于调用存储器中的计算机指令，以执行上述的视频背景处理方法的步骤。The processor is used for invoking computer instructions in the memory to execute the steps of the above-mentioned video background processing method.
第四方面，本发明提供了一种计算机可读存储介质，所述计算机可读存储介质存储计算机指令，所述计算机指令被执行时使所述计算机执行上述的视频背景处理方法的步骤。In a fourth aspect, the present invention provides a computer-readable storage medium, where the computer-readable storage medium stores computer instructions, and when the computer instructions are executed, the computer executes the steps of the above-mentioned video background processing method.
本发明提出的视频背景处理方法及装置，通过分别获取背景视频和背景图像以及分别确定所述背景视频中每个视频帧和所述背景图像在显示窗口上的窗口堆叠顺序和窗口坐标；基于所述背景视频中每个视频帧对应的窗口堆叠顺序和窗口坐标以及所述背景图像对应的窗口堆叠顺序和窗口坐标，将所述背景视频和所述背景图像进行融合处理得到融合视频；采集实时视频并对所述实时视频中的目标对象进行提取处理得到包含所述目标对象的提取视频；分别将所述提取视频和所述融合视频作为前景和背景进行融合处理。能够减弱了拼接痕迹，提高了融合后视频的视频质量，实现定制化的多层背景场景，达到实时且丰富展示的目的。The video background processing method and device proposed by the present invention obtain the background video and the background image respectively, and respectively determine the window stacking order and window coordinates of each video frame in the background video and the background image on the display window; In the background video, the window stacking sequence and window coordinates corresponding to each video frame and the window stacking sequence and window coordinates corresponding to the background image are performed, and the background video and the background image are fused to obtain a fusion video; real-time video is collected. and extracting and processing the target object in the real-time video to obtain an extraction video including the target object; respectively taking the extracted video and the fusion video as foreground and background for fusion processing. It can weaken the splicing traces, improve the video quality of the fused video, realize the customized multi-layer background scene, and achieve the purpose of real-time and rich display.
为使本发明的上述目的、特征和优点能更明显易懂，下文特举较佳实施例，并配合所附附图，作详细说明如下。In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, preferred embodiments are given below, and are described in detail as follows in conjunction with the accompanying drawings.
附图说明Description of drawings
为了更清楚地说明本发明实施例或现有技术中的技术方案，下面将对实施例或现有技术中所需要使用的附图作简单地介绍，应当理解，以下附图仅示出了本发明的某些实施例，因此不应被看作是对范围的限定，对于本领域普通技术人员来讲，在不付出创造性劳动的前提下，还可以根据这些附图获得其他相关的附图。In order to illustrate the embodiments of the present invention or the technical solutions in the prior art more clearly, the following briefly introduces the drawings required in the embodiments or the prior art. It should be understood that the following drawings only illustrate the present invention. Therefore, certain embodiments of the invention should not be regarded as limiting the scope. For those of ordinary skill in the art, other related drawings can also be obtained according to these drawings without creative efforts.
图1为本发明提供的一种视频背景处理方法的第一流程示意图；Fig. 1 is the first schematic flow chart of a kind of video background processing method provided by the present invention;
图2为本发明提供的一种视频背景处理方法的第二流程示意图；2 is a second schematic flowchart of a video background processing method provided by the present invention;
图3为本发明提供的一种视频背景处理方法的第三流程示意图；3 is a third schematic flowchart of a video background processing method provided by the present invention;
图4为本发明提供的一种视频背景处理方法的第四流程示意图；4 is a fourth schematic flowchart of a video background processing method provided by the present invention;
图5为本发明提供的一种视频背景处理装置的结构示意图；5 is a schematic structural diagram of a video background processing apparatus provided by the present invention;
图6是本发明提供的电子设备的结构示意图。FIG. 6 is a schematic structural diagram of an electronic device provided by the present invention.
具体实施方式Detailed ways
为使本发明实施例的目的、技术方案和优点更加清楚，下面将结合本发明实施例中附图，对本发明实施例中的技术方案进行清楚、完整地描述，显然，所描述的实施例仅仅是本发明一部分实施例，而不是全部的实施例。通常在此处附图中描述和示出的本发明实施例的组件可以以各种不同的配置来布置和设计。因此，以下对在附图中提供的本发明的实施例的详细描述并非旨在限制要求保护的本发明的范围，而是仅仅表示本发明的选定实施例。基于本发明的实施例，本领域技术人员在没有做出创造性劳动的前提下所获得的所有其他实施例，都属于本发明保护的范围。In order to make the purposes, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention. Obviously, the described embodiments are only These are some embodiments of the present invention, but not all embodiments. The components of the embodiments of the invention generally described and illustrated in the drawings herein may be arranged and designed in a variety of different configurations. Thus, the following detailed description of the embodiments of the invention provided in the accompanying drawings is not intended to limit the scope of the invention as claimed, but is merely representative of selected embodiments of the invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work fall within the protection scope of the present invention.
本发明实施例提供了一种视频背景处理方法，参见图1所示，具体包含有如下内容：An embodiment of the present invention provides a video background processing method, as shown in FIG. 1 , which specifically includes the following content:
S101：分别获取背景视频和背景图像以及分别确定所述背景视频中每个视频帧和所述背景图像在显示窗口上的窗口堆叠顺序和窗口坐标；S101: Respectively obtain a background video and a background image, and respectively determine the window stacking sequence and window coordinates of each video frame in the background video and the background image on the display window;
在本步骤中，获取的背景视频作为动态输入源，可以是一段已经录好的视频或者摄像头实时采集的视频。In this step, the acquired background video is used as a dynamic input source, which may be a recorded video or a video captured by a camera in real time.
通过OpengGL对应的sdk模块读取背景视频中每帧画面信息。其中，该每帧画面信息包括：该帧画面在内存中的图像内存地址、窗口堆叠顺序(z序)和窗口坐标。Read the picture information of each frame in the background video through the sdk module corresponding to OpenGL. Wherein, the picture information of each frame includes: the image memory address of the frame picture in the memory, the window stacking order (z order) and the window coordinates.
获取的背景图像作为动静态输入源，确定该背景图像对应的画面信息。背景图像对应的画面信息包括：背景图像在内存中的图像内存地址、窗口堆叠顺序(z序)和窗口坐标。The acquired background image is used as a dynamic and static input source, and the picture information corresponding to the background image is determined. The picture information corresponding to the background image includes: the image memory address of the background image in the memory, the window stacking order (z order) and the window coordinates.
需要说明的是，OpenGL(英语：Open Graphics Library，译名：开放图形库或者“开放式图形库”)是用于渲染2D、3D矢量图形的跨语言、跨平台的应用程序编程接口(API)。这个接口由近350个不同的函数调用组成，用来绘制从简单的图形比特到复杂的三维景象。It should be noted that OpenGL (English: Open Graphics Library, translation name: Open Graphics Library or "Open Graphics Library") is a cross-language and cross-platform application programming interface (API) for rendering 2D and 3D vector graphics. This interface consists of nearly 350 different function calls used to draw anything from simple graphic bits to complex three-dimensional scenes.
SDK模块即第三方的“软体开发工具包”,一般是一些被软件工程师用于为特定的软件包、软件框架、硬件平台、操作系统等建立应用软件的开发工具的集合。通俗点是指由第三方服务商提供的实现软件产品某项功能的工具包。The SDK module is a third-party "software development kit", which is generally a collection of development tools used by software engineers to build application software for specific software packages, software frameworks, hardware platforms, operating systems, etc. Popular point refers to a toolkit provided by a third-party service provider to implement a certain function of a software product.
窗口总是矩形的，它们沿着垂直于屏幕的假想直线相互堆叠在一起。堆叠在一起的窗口称为z序。每个窗口在z序中具有唯一的位置。在z序位置靠前的窗口位于位置靠后的窗口的前面或顶部。窗口在z序中的位置影响它的外观。Windows are always rectangular, and they are stacked on top of each other along an imaginary line perpendicular to the screen. Stacked windows are called z-order. Each window has a unique position in the z-order. The windows that are positioned earlier in the z-order position are in front of or on top of the windows that are positioned later. The position of the window in the z-order affects its appearance.
S102：基于所述背景视频中每个视频帧对应的窗口堆叠顺序和窗口坐标以及所述背景图像对应的窗口堆叠顺序和窗口坐标，将所述背景视频和所述背景图像进行融合处理得到融合视频；S102: Based on the window stacking sequence and window coordinates corresponding to each video frame in the background video and the window stacking sequence and window coordinates corresponding to the background image, perform fusion processing on the background video and the background image to obtain a fusion video ;
在本步骤中，将多个通道的背景图像元数据(元数据即由图像数据的内存地址获取的背景图片信息)，以及每个通道的背景图像大小，坐标，以及z序作为一个数据组合传输到OpengGL业务处理层。In this step, the background image metadata of multiple channels (the metadata is the background image information obtained from the memory address of the image data), and the background image size, coordinates, and z-order of each channel are transmitted as a data combination. to the OpenGL business processing layer.
OpengGL融合层根据层级关系以及每个图层坐标由下至上依次绘制每个背景图层。主要OpengGL融合流程包括：The OpenGL fusion layer draws each background layer sequentially from bottom to top according to the hierarchical relationship and the coordinates of each layer. The main OpenGL fusion process includes:
1，初始化OpengGL离屏环境，作为后台融合绘制的目标窗口，同时设置OpengGL的alpha blend相关参数。1. Initialize the OpenGL off-screen environment as the target window for background fusion drawing, and set the alpha blend related parameters of OpenGL.
2，融合层根据应用层传递的背景图像组合参数(z序，图像宽/高，坐标)，以及图像原始数据地址，分别依次在OpengGL离屏环境中进行渲染(OpengGL alpha blend)。2. The fusion layer performs rendering (OpengGL alpha blend) in the OpenGL off-screen environment in turn according to the background image combination parameters (z-order, image width/height, coordinates) passed by the application layer, and the original data address of the image.
3，从显存读取融合后的背景图像信息到内存中，返回给调用的应用层。此处可以使用双pbo(pixel buffer object)加快显存到内存的读取。3. Read the fused background image information from the video memory to the memory, and return it to the calling application layer. Here you can use double pbo (pixel buffer object) to speed up the reading from video memory to memory.
多通道背景融合使用OpengGL硬件加速的方式，由于OpengGL是一种跨平台的GPU渲染计算技术，开发的代码可以直接运行在Windows和Mac操作系统上，可以减少适配不同操作系统的开发时间和工作量以及后续维护的成本。Multi-channel background fusion uses OpenGL hardware acceleration. Since OpenGL is a cross-platform GPU rendering computing technology, the developed code can be directly run on Windows and Mac operating systems, which can reduce the development time and work of adapting to different operating systems. and the cost of subsequent maintenance.
S103：采集实时视频并对所述实时视频中的目标对象进行提取处理得到包含所述目标对象的提取视频；S103: Collect real-time video and perform extraction processing on the target object in the real-time video to obtain an extracted video containing the target object;
在本步骤中，通过AI抠图软件对实时视频中的目标对象进行提取处理得到包含所述目标对象的提取视频。In this step, the target object in the real-time video is extracted and processed by AI mapping software to obtain an extracted video including the target object.
本实施例中，目标对象为讲师或主讲人。In this embodiment, the target object is a lecturer or a presenter.
S104：分别将所述提取视频和所述融合视频作为前景和背景进行融合处理。S104: Perform fusion processing with the extracted video and the fusion video as foreground and background, respectively.
在本步骤中，获取到融合后的融合视频，再将AI抠图生成的包含scoremap的提取视频(scoremap包含摄像头画面中人像部分的像素点透明度以及人像背景像素点的透明度信息)将人像与合成后的背景图片进行叠加，融合，达到丰富展示的目的。In this step, the fused fused video is obtained, and then the extracted video containing the scoremap generated by AI matting (scoremap contains the pixel transparency of the portrait part in the camera screen and the transparency information of the background pixels of the portrait) is combined with the portrait and composite The background pictures are superimposed and merged to achieve the purpose of rich display.
从上述描述可知，本发明实施例提供的视频背景处理方法，通过分别获取背景视频和背景图像以及分别确定所述背景视频中每个视频帧和所述背景图像在显示窗口上的窗口堆叠顺序和窗口坐标；基于所述背景视频中每个视频帧对应的窗口堆叠顺序和窗口坐标以及所述背景图像对应的窗口堆叠顺序和窗口坐标，将所述背景视频和所述背景图像进行融合处理得到融合视频；采集实时视频并对所述实时视频中的目标对象进行提取处理得到包含所述目标对象的提取视频；分别将所述提取视频和所述融合视频作为前景和背景进行融合处理。能够减弱了拼接痕迹，提高了融合后视频的视频质量，实现定制化的多层背景场景，达到实时且丰富展示的目的。It can be seen from the above description that the video background processing method provided by the embodiment of the present invention obtains the background video and the background image respectively, and determines the window stacking order and the window stacking sequence of each video frame in the background video and the background image on the display window window coordinates; based on the window stacking sequence and window coordinates corresponding to each video frame in the background video and the window stacking sequence and window coordinates corresponding to the background image, the background video and the background image are fused to obtain fusion video; collect real-time video and perform extraction processing on the target object in the real-time video to obtain the extracted video including the target object; respectively use the extracted video and the fusion video as foreground and background for fusion processing. It can weaken the splicing traces, improve the video quality of the fused video, realize the customized multi-layer background scene, and achieve the purpose of real-time and rich display.
在本发明的一实施例中，参见图2，所述视频背景处理方法的实施例中步骤S101之后，还包含有步骤S105，具体包含有如下内容：In an embodiment of the present invention, referring to FIG. 2 , after step S101 in the embodiment of the video background processing method, step S105 is further included, which specifically includes the following contents:
S105：对所述显示窗口上的窗口堆叠顺序或窗口坐标进行修改处理。S105: Modify the window stacking order or the window coordinates on the display window.
在本实施例中，可以对显示窗口上的窗口堆叠顺序或窗口坐标进行修改处理。In this embodiment, the window stacking order or window coordinates on the display window can be modified.
其中，对窗口堆叠顺序的更改处理能够改变背景视频或背景图像在显示窗口上的堆叠顺序(Z序)，达到调整背景视频或背景图像在显示窗口上显示顺序。Wherein, the modification processing of the window stacking sequence can change the stacking sequence (Z sequence) of the background videos or background images on the display window, so as to adjust the display sequence of the background videos or background images on the display window.
对窗口坐标的更改处理能够改变背景视频或背景图像在显示窗口上的位置，避免背景视频或背景图像显示重叠的问题，进一步提高显示的丰富性。The modification processing of the window coordinates can change the position of the background video or background image on the display window, avoid the problem of overlapping background video or background image display, and further improve the richness of the display.
在本发明的一实施例中，参见图3，所述视频背景处理方法的实施例中步骤S101之后，还包含有步骤S106和步骤S107，具体包含有如下内容：In an embodiment of the present invention, referring to FIG. 3 , after step S101 in the embodiment of the video background processing method, step S106 and step S107 are further included, which specifically include the following contents:
S106：分别确定所述背景视频在显示窗口上的显示区域和所述背景图像在显示窗口上的显示区域。S106: Determine the display area of the background video on the display window and the display area of the background image on the display window, respectively.
S107：对所述背景视频在显示窗口上的显示区域或所述背景图像在显示窗口上的显示区域进行显示区域的缩放处理。S107: Perform display area scaling processing on the display area of the background video on the display window or the display area of the background image on the display window.
在本实施例中，可以对背景视频或背景图像在显示窗口上的显示区域，该显示区域指的是背景视频或背景图像占显示窗口的比例，或者背景视频或背景图像在显示窗口上的显示框。In this embodiment, the display area of the background video or background image on the display window may be defined, and the display area refers to the proportion of the background video or background image in the display window, or the display area of the background video or background image on the display window. frame.
对背景视频或背景图像在显示窗口上的显示区域进行缩放处理，能够进一步提高显示的丰富性。Scaling the display area of the background video or background image on the display window can further improve the richness of the display.
在本发明的一实施例中，提供所述视频背景处理方法中步骤S104的一种具体实现方式，参见图4，具体包含有如下内容：In an embodiment of the present invention, a specific implementation of step S104 in the video background processing method is provided, referring to FIG. 4 , which specifically includes the following content:
S1041：确定所述提取视频的每一视频帧中所述目标对象上所有像素点的像素点透明度；S1041: Determine the pixel transparency of all pixels on the target object in each video frame of the extracted video;
S1042：基于所述目标对象上所有像素点的像素点透明度将所述提取视频和所述融合视频进行融合处理；S1042: Perform fusion processing on the extracted video and the fusion video based on the pixel transparency of all pixels on the target object;
在本步骤中，确定目标对象对应的所有像素点在显示窗口上像素点透明度，像素点透明度为提取视频中所述目标对象对应的每个像素点在显示窗口上的透明程度。本步骤中，目标对象在在显示窗口上的像素点透明度为不透明。也就是说，显示窗口上能够完整显示目标对象。In this step, the transparency of all pixels corresponding to the target object on the display window is determined, and the pixel transparency is the degree of transparency of each pixel corresponding to the target object in the extracted video on the display window. In this step, the pixel point transparency of the target object on the display window is opaque. That is, the target object can be completely displayed on the display window.
为了提高了融合后视频的视频质量，确定目标对象与背景相接处的各个像素点的置信度；将该置信度确定为目标对象与背景相接处的各个像素点的像素点透明度。基于显示窗口上的像素点的比例(目标对象与背景相接处的各个像素点的置信度)确定为该像素点的目标对象与背景的比例。In order to improve the video quality of the fused video, the confidence level of each pixel point where the target object meets the background is determined; the confidence level is determined as the pixel transparency of each pixel point where the target object meets the background. Based on the ratio of the pixel points on the display window (the confidence level of each pixel point where the target object meets the background), the ratio of the target object to the background of the pixel point is determined.
本发明实施例提供一种能够实现所述视频背景处理方法中全部内容的视频背景处理装置的具体实施方式，参见图5，所述视频背景处理装置具体包括如下内容：An embodiment of the present invention provides a specific implementation of a video background processing apparatus capable of realizing all the contents in the video background processing method. Referring to FIG. 5 , the video background processing apparatus specifically includes the following contents:
获取单元10，用于分别获取背景视频和背景图像以及分别确定所述背景视频中每个视频帧和所述背景图像在显示窗口上的窗口堆叠顺序和窗口坐标；Obtaining
背景单元20，用于基于所述背景视频中每个视频帧对应的窗口堆叠顺序和窗口坐标以及所述背景图像对应的窗口堆叠顺序和窗口坐标，将所述背景视频和所述背景图像进行融合处理得到融合视频；The
提取单元30，用于采集实时视频并对所述实时视频中的目标对象进行提取处理得到包含所述目标对象的提取视频；
融合单元40，用于分别将所述提取视频和所述融合视频作为前景和背景进行融合处理。The
在本发明一实施例中，还包括：In an embodiment of the present invention, it also includes:
修改单元，用于对所述显示窗口上的窗口堆叠顺序或窗口坐标进行修改处理。A modification unit, configured to modify the window stacking order or the window coordinates on the display window.
在本发明一实施例中，还包括：In an embodiment of the present invention, it also includes:
调整单元，用于分别确定所述背景视频在显示窗口上的显示区域和所述背景图像在显示窗口上的显示区域。An adjustment unit, configured to respectively determine a display area of the background video on the display window and a display area of the background image on the display window.
在本发明一实施例中，还包括：In an embodiment of the present invention, it also includes:
缩放单元，用于对所述背景视频在显示窗口上的显示区域或所述背景图像在显示窗口上的显示区域进行显示区域的缩放处理。A scaling unit, configured to perform scaling processing on the display area of the background video on the display window or the display area of the background image on the display window.
在本发明一实施例中，融合单元40，包括：In an embodiment of the present invention, the
像素模块，用于确定所述提取视频的每一视频帧中所述目标对象上所有像素点的像素点透明度；a pixel module, configured to determine the pixel transparency of all pixels on the target object in each video frame of the extracted video;
融合模块，用于基于所述目标对象上所有像素点的像素点透明度将所述提取视频和所述融合视频进行融合处理；a fusion module, configured to perform fusion processing on the extracted video and the fusion video based on the pixel transparency of all pixels on the target object;
其中，所述像素点透明度为提取视频中所述目标对象对应的每个像素点在显示窗口上的透明程度。Wherein, the transparency of the pixel point is the degree of transparency of each pixel point corresponding to the target object in the extracted video on the display window.
本发明提供的视频背景处理装置的实施例具体可以用于执行上述实施例中的视频背景处理方法的实施例的处理流程，其功能在此不再赘述，可以参照上述方法实施例的详细描述。The embodiments of the video background processing apparatus provided by the present invention can be specifically used to execute the processing flow of the video background processing method embodiments in the foregoing embodiments, and the functions thereof are not repeated here, and reference may be made to the detailed descriptions of the foregoing method embodiments.
从上述描述可知，本发明实施例提供的视频背景处理装置，通过分别获取背景视频和背景图像以及分别确定所述背景视频中每个视频帧和所述背景图像在显示窗口上的窗口堆叠顺序和窗口坐标；基于所述背景视频中每个视频帧对应的窗口堆叠顺序和窗口坐标以及所述背景图像对应的窗口堆叠顺序和窗口坐标，将所述背景视频和所述背景图像进行融合处理得到融合视频；采集实时视频并对所述实时视频中的目标对象进行提取处理得到包含所述目标对象的提取视频；分别将所述提取视频和所述融合视频作为前景和背景进行融合处理。能够减弱了拼接痕迹，提高了融合后视频的视频质量，实现定制化的多层背景场景，达到实时且丰富展示的目的。It can be seen from the above description that the video background processing apparatus provided by the embodiment of the present invention obtains the background video and the background image respectively, and determines the window stacking sequence and the window stacking order of each video frame and the background image in the background video on the display window respectively. window coordinates; based on the window stacking sequence and window coordinates corresponding to each video frame in the background video and the window stacking sequence and window coordinates corresponding to the background image, the background video and the background image are fused to obtain fusion video; collect real-time video and perform extraction processing on the target object in the real-time video to obtain the extracted video including the target object; respectively use the extracted video and the fusion video as foreground and background for fusion processing. It can weaken the splicing traces, improve the video quality of the fused video, realize the customized multi-layer background scene, and achieve the purpose of real-time and rich display.
本发明实施例提供一种用于实现所述视频背景处理方法实施例中的全部或部分内容的电子设备的实施例，参见图6，所述电子设备具体包含有如下内容：An embodiment of the present invention provides an embodiment of an electronic device for implementing all or part of the content in the embodiment of the video background processing method. Referring to FIG. 6 , the electronic device specifically includes the following content:
处理器(processor)810、通信接口(Communications Interface)820、存储器(memory)830和通信总线840，其中，处理器810，通信接口820，存储器830通过通信总线840完成相互间的通信。处理器810可以调用存储器830中的计算机指令，以执行如下方法：A
分别获取背景视频和背景图像以及分别确定所述背景视频中每个视频帧和所述背景图像在显示窗口上的窗口堆叠顺序和窗口坐标；Obtaining the background video and the background image respectively and respectively determining the window stacking order and window coordinates of each video frame in the background video and the background image on the display window;
基于所述背景视频中每个视频帧对应的窗口堆叠顺序和窗口坐标以及所述背景图像对应的窗口堆叠顺序和窗口坐标，将所述背景视频和所述背景图像进行融合处理得到融合视频；Based on the window stacking sequence and window coordinates corresponding to each video frame in the background video and the window stacking sequence and window coordinates corresponding to the background image, the background video and the background image are fused to obtain a fused video;
采集实时视频并对所述实时视频中的目标对象进行提取处理得到包含所述目标对象的提取视频；collecting real-time video and extracting the target object in the real-time video to obtain an extraction video containing the target object;
分别将所述提取视频和所述融合视频作为前景和背景进行融合处理。The extracted video and the fused video are respectively used as foreground and background for fusion processing.
本发明实施例提供一种用于实现所述视频背景处理方法实施例中的全部或部分内容的计算机可读存储介质，所述计算机可读存储介质上存储有计算机指令，所述计算机指令被执行时使所述计算机执行上述实施例中的视频背景处理方法的全部步骤，例如，所述处理器执行所述计算机指令时实现下述步骤：An embodiment of the present invention provides a computer-readable storage medium for implementing all or part of the content in the video background processing method embodiment, where computer instructions are stored on the computer-readable storage medium, and the computer instructions are executed When the computer executes all the steps of the video background processing method in the above embodiment, for example, when the processor executes the computer instructions, the following steps are implemented:
分别获取背景视频和背景图像以及分别确定所述背景视频中每个视频帧和所述背景图像在显示窗口上的窗口堆叠顺序和窗口坐标；Obtaining the background video and the background image respectively and respectively determining the window stacking order and window coordinates of each video frame in the background video and the background image on the display window;
基于所述背景视频中每个视频帧对应的窗口堆叠顺序和窗口坐标以及所述背景图像对应的窗口堆叠顺序和窗口坐标，将所述背景视频和所述背景图像进行融合处理得到融合视频；Based on the window stacking sequence and window coordinates corresponding to each video frame in the background video and the window stacking sequence and window coordinates corresponding to the background image, the background video and the background image are fused to obtain a fused video;
采集实时视频并对所述实时视频中的目标对象进行提取处理得到包含所述目标对象的提取视频；collecting real-time video and extracting the target object in the real-time video to obtain an extraction video containing the target object;
分别将所述提取视频和所述融合视频作为前景和背景进行融合处理。The extracted video and the fused video are respectively used as foreground and background for fusion processing.
虽然本发明提供了如实施例或流程图所述的方法操作步骤，但基于常规或者无创造性的劳动可以包括更多或者更少的操作步骤。实施例中列举的步骤顺序仅仅为众多步骤执行顺序中的一种方式，不代表唯一的执行顺序。在实际中的装置或客户端产品执行时，可以按照实施例或者附图所示的方法顺序执行或者并行执行(例如并行处理器或者多线程处理的环境)。Although the present invention provides method operation steps as described in the embodiments or flow charts, more or less operation steps may be included based on routine or non-creative work. The sequence of steps enumerated in the embodiments is only one of the execution sequences of many steps, and does not represent the only execution sequence. When an actual device or client product is executed, the methods shown in the embodiments or the accompanying drawings may be executed sequentially or in parallel (for example, a parallel processor or a multi-threaded processing environment).
所述功能如果以软件功能单元的形式实现并作为独立的产品销售或使用时，可以存储在一个计算机可读取存储介质中。基于这样的理解，本发明的技术方案本质上或者说对现有技术做出贡献的部分或者该技术方案的部分可以以软件产品的形式体现出来，该计算机软件产品存储在一个存储介质中，包括若干指令用以使得一台计算机设备(可以是个人计算机，服务器，或者网络设备等)执行本发明各个实施例所述方法的全部或部分步骤。而前述的存储介质包括：U盘、移动硬盘、只读存储器(ROM，Read-Only Memory)、随机存取存储器(RAM，Random Access Memory)、磁碟或者光盘等各种可以存储程序代码的介质。The functions, if implemented in the form of software functional units and sold or used as independent products, may be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention can be embodied in the form of a software product in essence, or the part that contributes to the prior art or the part of the technical solution. The computer software product is stored in a storage medium, including Several instructions are used to cause a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, Read-Only Memory (ROM, Read-Only Memory), Random Access Memory (RAM, Random Access Memory), magnetic disk or optical disk and other media that can store program codes .
本说明书中的各个实施例均采用递进的方式描述，各个实施例之间相同相似的部分互相参见即可，每个实施例重点说明的都是与其他实施例的不同之处。尤其，对于装置(系统)实施例而言，由于其基本相似于方法实施例，所以描述的比较简单，相关之处参见方法实施例的部分说明即可。Each embodiment in this specification is described in a progressive manner, and the same and similar parts between the various embodiments may be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the apparatus (system) embodiments, since they are basically similar to the method embodiments, the description is relatively simple, and for related parts, please refer to the partial descriptions of the method embodiments.
在本文中，诸如第一和第二等之类的关系术语仅仅用来将一个实体或者操作与另一个实体或操作区分开来，而不一定要求或者暗示这些实体或操作之间存在任何这种实际的关系或者顺序。而且，术语“包括”、“包含”或者其任何其他变体意在涵盖非排他性的包含，从而使得包括一系列要素的过程、方法、物品或者设备不仅包括那些要素，而且还包括没有明确列出的其他要素，或者是还包括为这种过程、方法、物品或者设备所固有的要素。对于本领域的普通技术人员而言，可以根据具体情况理解上述术语在本发明中的具体含义。In this document, relational terms such as first and second, etc. are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such existence between these entities or operations. The actual relationship or sequence. Moreover, the terms "comprising", "comprising" or any other variation thereof are intended to encompass a non-exclusive inclusion such that a process, method, article or device that includes a list of elements includes not only those elements, but also includes not explicitly listed or other elements inherent to such a process, method, article or apparatus. For those of ordinary skill in the art, the specific meanings of the above terms in the present invention can be understood according to specific situations.
需要说明的是，在不冲突的情况下，本发明中的实施例及实施例中的特征可以相互组合。本发明并不局限于任何单一的方面，也不局限于任何单一的实施例，也不局限于这些方面和/或实施例的任意组合和/或置换。而且，可以单独使用本发明的每个方面和/或实施例或者与一个或更多其他方面和/或其实施例结合使用。It should be noted that the embodiments of the present invention and the features of the embodiments may be combined with each other under the condition of no conflict. The invention is not limited to any single aspect, nor to any single embodiment, nor to any combination and/or permutation of these aspects and/or embodiments. Furthermore, each aspect and/or embodiment of the invention may be used alone or in combination with one or more other aspects and/or embodiments thereof.
最后应说明的是：以上所述实施例，仅为本发明的具体实施方式，用以说明本发明的技术方案，而非对其限制，本发明的保护范围并不局限于此，尽管参照前述实施例对本发明进行了详细的说明，本领域的普通技术人员应当理解：任何熟悉本技术领域的技术人员在本发明揭露的技术范围内，其依然可以对前述实施例所记载的技术方案进行修改或可轻易想到变化，或者对其中部分技术特征进行等同替换；而这些修改、变化或者替换，并不使相应技术方案的本质脱离本发明实施例技术方案的精神和范围。都应涵盖在本发明的保护范围之内。因此，本发明的保护范围应所述以权利要求的保护范围为准。Finally, it should be noted that the above-mentioned embodiments are only specific implementations of the present invention, and are used to illustrate the technical solutions of the present invention, but not to limit them. The protection scope of the present invention is not limited thereto, although referring to the foregoing The embodiment has been described in detail the present invention, those of ordinary skill in the art should understand: any person skilled in the art who is familiar with the technical field within the technical scope disclosed by the present invention can still modify the technical solutions described in the foregoing embodiments. Changes can be easily conceived, or equivalent replacements are made to some of the technical features; and these modifications, changes or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present invention. All should be included within the protection scope of the present invention. Therefore, the protection scope of the present invention should be based on the protection scope of the claims.
Claims (10)
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111652168.7ACN114339401A (en) | 2021-12-30 | 2021-12-30 | A kind of video background processing method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111652168.7ACN114339401A (en) | 2021-12-30 | 2021-12-30 | A kind of video background processing method and device |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN114339401Atrue CN114339401A (en) | 2022-04-12 |
Family
ID=81018425
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111652168.7APendingCN114339401A (en) | 2021-12-30 | 2021-12-30 | A kind of video background processing method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114339401A (en) |
Citations (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2006060289A (en)* | 2004-08-17 | 2006-03-02 | Oki Electric Ind Co Ltd | Video image distributing system |
| US20080001970A1 (en)* | 2006-06-29 | 2008-01-03 | Jason Herrick | Method and system for mosaic mode display of video |
| CN201690532U (en)* | 2010-02-08 | 2010-12-29 | 深圳市同洲电子股份有限公司 | Video processing device and digital television receiving terminal |
| CN105744340A (en)* | 2016-02-26 | 2016-07-06 | 上海卓越睿新数码科技有限公司 | Real-time screen fusion method for live broadcast video and presentation file |
| US20170039867A1 (en)* | 2013-03-15 | 2017-02-09 | Study Social, Inc. | Mobile video presentation, digital compositing, and streaming techniques implemented via a computer network |
| EP3386204A1 (en)* | 2017-04-04 | 2018-10-10 | Thomson Licensing | Device and method for managing remotely displayed contents by augmented reality |
| CN110290425A (en)* | 2019-07-29 | 2019-09-27 | 腾讯科技（深圳）有限公司 | A kind of method for processing video frequency, device and storage medium |
| CN110996150A (en)* | 2019-11-18 | 2020-04-10 | 咪咕动漫有限公司 | Video fusion method, electronic device and storage medium |
| CN112261434A (en)* | 2020-10-22 | 2021-01-22 | 广州华多网络科技有限公司 | Interface layout control and processing method and corresponding device, equipment and medium |
| CN112839190A (en)* | 2021-01-22 | 2021-05-25 | 九天华纳(北京)科技有限公司 | Method for synchronously recording or live broadcasting video of virtual image and real scene |
- 2021
- 2021-12-30CNCN202111652168.7Apatent/CN114339401A/enactivePending
Patent Citations (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2006060289A (en)* | 2004-08-17 | 2006-03-02 | Oki Electric Ind Co Ltd | Video image distributing system |
| US20080001970A1 (en)* | 2006-06-29 | 2008-01-03 | Jason Herrick | Method and system for mosaic mode display of video |
| CN201690532U (en)* | 2010-02-08 | 2010-12-29 | 深圳市同洲电子股份有限公司 | Video processing device and digital television receiving terminal |
| US20170039867A1 (en)* | 2013-03-15 | 2017-02-09 | Study Social, Inc. | Mobile video presentation, digital compositing, and streaming techniques implemented via a computer network |
| US10515561B1 (en)* | 2013-03-15 | 2019-12-24 | Study Social, Inc. | Video presentation, digital compositing, and streaming techniques implemented via a computer network |
| CN105744340A (en)* | 2016-02-26 | 2016-07-06 | 上海卓越睿新数码科技有限公司 | Real-time screen fusion method for live broadcast video and presentation file |
| EP3386204A1 (en)* | 2017-04-04 | 2018-10-10 | Thomson Licensing | Device and method for managing remotely displayed contents by augmented reality |
| CN110290425A (en)* | 2019-07-29 | 2019-09-27 | 腾讯科技（深圳）有限公司 | A kind of method for processing video frequency, device and storage medium |
| CN110996150A (en)* | 2019-11-18 | 2020-04-10 | 咪咕动漫有限公司 | Video fusion method, electronic device and storage medium |
| CN112261434A (en)* | 2020-10-22 | 2021-01-22 | 广州华多网络科技有限公司 | Interface layout control and processing method and corresponding device, equipment and medium |
| CN112839190A (en)* | 2021-01-22 | 2021-05-25 | 九天华纳(北京)科技有限公司 | Method for synchronously recording or live broadcasting video of virtual image and real scene |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US20250086892A1 (en) | Methods and systems for producing content in multiple reality environments | |
| JP6655737B2 (en) | Multi-view scene segmentation and propagation | |
| JP7645917B2 (en) | A technique for capturing and editing dynamic depth images | |
| EP3883256A1 (en) | Live stream processing method in webrtc and stream pushing client | |
| US20210374972A1 (en) | Panoramic video data processing method, terminal, and storage medium | |
| CN108876887B (en) | Rendering method and device | |
| US10380726B2 (en) | Systems, devices, and methods for generating a social street view | |
| WO2019020061A1 (en) | Video dialogue processing method, video client, video server, and computer readable storage medium | |
| CN104618803A (en) | Information push method, information push device, terminal and server | |
| WO2022227868A1 (en) | Scene rendering method and apparatus, electronic device, and readable storage medium | |
| JP7213291B2 (en) | Method and apparatus for generating images | |
| CN107622504A (en) | Method and device for processing pictures | |
| CN114359081A (en) | Liquid material dissolving method, device, electronic device and storage medium | |
| US9143754B2 (en) | Systems and methods for modifying stereoscopic images | |
| WO2009106993A2 (en) | Method and system for generating online cartoon outputs | |
| CN114339401A (en) | A kind of video background processing method and device | |
| CN115311397B (en) | Method, apparatus, device and storage medium for image rendering | |
| CN109803163B (en) | Image display method and device and storage medium | |
| CN113691835B (en) | Video implantation method, apparatus, apparatus, and computer-readable storage medium | |
| CN118485733A (en) | Display method, device, electronic device and medium for objects to be displayed | |
| CN116503263A (en) | Rendering image noise reduction method, model rendering method, device, equipment and storage medium | |
| JP2023542598A (en) | Character display methods, devices, electronic devices, and storage media | |
| CN107038734A (en) | A kind of method of imaging importing text for Windows systems | |
| CN114820999B (en) | PPT enhancement display system based on augmented reality | |
| US20240412710A1 (en) | Overlaying displayed digital content with regional transparency and regional lossless compression transmitted over a communication network via processing circuitry |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication | Application publication date:20220412 |