CN112541858A - Video image enhancement method, device, equipment, chip and storage medium - Google Patents
Video image enhancement method, device, equipment, chip and storage mediumDownload PDFInfo
- Publication number
- CN112541858A CN112541858ACN201910895085.7ACN201910895085ACN112541858ACN 112541858 ACN112541858 ACN 112541858ACN 201910895085 ACN201910895085 ACN 201910895085ACN 112541858 ACN112541858 ACN 112541858A
- Authority
- CN
- China
- Prior art keywords
- viewing angle
- angle area
- video image
- enhancement
- terminal device
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/90—Dynamic range modification of images or parts thereof
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/77—Retouching; Inpainting; Scratch removal
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/80—Generation or processing of content or additional data by content creator independently of the distribution process; Content per se
- H04N21/81—Monomedia components thereof
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/80—Generation or processing of content or additional data by content creator independently of the distribution process; Content per se
- H04N21/81—Monomedia components thereof
- H04N21/8146—Monomedia components thereof involving graphical data, e.g. 3D object, 2D graphics
- H04N21/8153—Monomedia components thereof involving graphical data, e.g. 3D object, 2D graphics comprising still images, e.g. texture, background image
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10016—Video; Image sequence
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Computer Graphics (AREA)
- Two-Way Televisions, Distribution Of Moving Picture Or The Like (AREA)
Abstract
Translated fromChinese
Description
Translated fromChinese技术领域technical field
本申请涉及视频图像处理技术领域，尤其涉及一种视频图像的增强方法、装置、设备、芯片及存储介质。The present application relates to the technical field of video image processing, and in particular, to a video image enhancement method, apparatus, device, chip and storage medium.
背景技术Background technique
终端设备在播放虚拟现实(Virtual Reality，VR)360度片源时，若终端设备的配置较低，为了能够播放高分辨率的片源，终端设备可以将片源切分成若干个小块，并在片源播放时，只对视角区域内的部分块进行拼接和显示。另外为了避免视角区域中的数据流不能及时下载显示的问题，终端设备可以在背景部分叠加一层分辨率较低、码率较低的背景流，例如：背景流的分辨率是4K，视角区域中的数据流的分辨率是8K(即高清流)，而背景流和高清流的内容完全相同，二者投影后完全重叠，基于此，当用户视角切换到上述视角区域时，如果高清流不能得到及时显示，则用户看到的是背景流的内容。例如：图1为现有技术提供的片源被切分成若72块的示意图，如图1所示，视角区域1是用户在同一时刻实际能够看到的部分。终端设备在进行片源播放时，若终端设备可以及时下载到上述高清流，则对视角区域1采用高清流播放，若终端设备可以不能及时下载到上述高清流，则对视角区域1采用低清流(即背景流)播放，而对片源中的其他部分仍然使用低清流播放。When the terminal device is playing a 360-degree virtual reality (VR) source, if the configuration of the terminal device is low, in order to play the high-resolution source, the terminal device can divide the source into several small pieces, and When the video source is playing, only some blocks in the viewing angle area are spliced and displayed. In addition, in order to avoid the problem that the data stream in the viewing angle area cannot be downloaded and displayed in time, the terminal device can overlay a background stream with a lower resolution and a lower bit rate on the background part, for example: the resolution of the background stream is 4K, and the viewing angle area The resolution of the data stream is 8K (i.e. high-definition stream), and the content of the background stream and the high-definition stream is exactly the same, and the two completely overlap after projection. If it is displayed in time, what the user sees is the content of the background stream. For example, FIG. 1 is a schematic diagram of a slice source provided by the prior art being divided into 72 blocks. As shown in FIG. 1 , the viewing angle area 1 is the part that the user can actually see at the same time. When the terminal device is playing the film source, if the terminal device can download the above-mentioned high-definition stream in time, the high-definition stream is used for viewing area 1; (that is, the background stream), and still use the low-definition stream for other parts of the film source.
对于VR360片源等视频图像，其亮度、对比度、色彩饱和度值在编码时是针对整个视频图像的，因此如果终端设备在播放视角区域内容时，采用整个360度画面的亮度、对比度、色彩饱和度值，这将造成视角区域部分看起来比较暗，颜色不够鲜艳等显示效果不佳的问题。For video images such as VR360 film sources, the brightness, contrast, and color saturation values are for the entire video image during encoding. Therefore, if the terminal device plays the content in the viewing angle area, the brightness, contrast, and color saturation of the entire 360-degree screen are used. degree value, which will cause the viewing angle part to look dark, the color is not bright enough and other problems of poor display effect.
发明内容SUMMARY OF THE INVENTION
本申请提供一种视频图像的增强方法、装置、设备、芯片及存储介质。通过该方法可以提高视频图像显示效果。The present application provides a video image enhancement method, apparatus, device, chip and storage medium. By this method, the display effect of the video image can be improved.
第一方面，本申请提供一种视频图像增强方法，包括：服务器接收终端设备发送的请求消息，请求消息包括视频图像的标识。服务器根据视频图像的标识获取视频图像中的至少一个视角区域的画面参数。服务器向终端设备发送至少一个视角区域的画面参数。其中，至少一个视角区域的画面参数包括：至少一个视角区域分别对应的增强参数，至少一个视角区域中任一个视角区域对应的增强参数用于终端设备对该视角区域进行增强处理。In a first aspect, the present application provides a video image enhancement method, including: a server receiving a request message sent by a terminal device, where the request message includes an identifier of the video image. The server acquires picture parameters of at least one viewing angle area in the video image according to the identifier of the video image. The server sends picture parameters of at least one viewing angle area to the terminal device. The picture parameters of the at least one viewing angle area include: enhancement parameters corresponding to the at least one viewing angle area respectively, and the enhancement parameter corresponding to any viewing angle area in the at least one viewing angle area is used for the terminal device to perform enhancement processing on the viewing angle area.
即通过该方法可以提高视频图像显示效果。进一步地，本申请提供的技术方案可以将视频图像增强细化到各个视角区域或者主视角区域中，每个视角区域都有对应的增强参数，而不是针对整个视频图像进行视频图像增强，从而可以进一步地提高视频图像显示效果。That is, the video image display effect can be improved by this method. Further, the technical solution provided by the present application can refine the video image enhancement into each viewing angle area or the main viewing angle area, and each viewing angle area has corresponding enhancement parameters, instead of performing video image enhancement for the entire video image, so that it can be The video image display effect is further improved.
可选的，服务器接收终端设备发送的请求消息之前，还包括：服务器确定至少一个视角区域的画面参数。Optionally, before the server receives the request message sent by the terminal device, the method further includes: the server determines picture parameters of at least one viewing angle area.
可选的，服务器确定至少一个视角区域的画面参数，包括：若视频图像中各个像素的亮度分量均值小于亮度阈值，且第一像素的个数大于第一预设个数，则确定至少一个视角区域分别对应的亮度增强参数，第一像素为各个像素中亮度分量小于亮度分量均值的像素。和/或，若视频图像中各个像素的饱和度分量均值小于饱和度阈值，且第二像素的个数大于第二预设个数，则确定至少一个视角区域分别对应的饱和度增强参数，第二像素为各个像素中饱和度分量小于饱和度分量均值的像素。和/或，若视频图像中各个像素的对比度分量均值小于对比度阈值，且第三像素的个数大于第二预设个数，则确定至少一个视角区域分别对应的对比度增强参数，第三像素为各个像素中对比度分量小于对比度分量均值的像素。和/或，若视频图像中各个像素的锐化度分量均值小于锐化度阈值，且第四像素的个数大于第二预设个数，则确定至少一个视角区域分别对应的锐化度增强参数，第四像素为各个像素中锐化度分量小于锐化度分量均值的像素。Optionally, the server determines the picture parameters of at least one viewing angle area, including: if the mean value of the brightness components of each pixel in the video image is less than the brightness threshold, and the number of the first pixels is greater than the first preset number, then determining at least one viewing angle. The brightness enhancement parameters corresponding to the regions respectively, and the first pixel is a pixel whose brightness component is less than the mean value of the brightness components in each pixel. And/or, if the average value of the saturation components of each pixel in the video image is less than the saturation threshold, and the number of the second pixels is greater than the second preset number, then determine the saturation enhancement parameters corresponding to the at least one viewing angle region respectively, and the first Two pixels are pixels whose saturation components are smaller than the mean value of the saturation components in each pixel. And/or, if the average value of the contrast components of each pixel in the video image is less than the contrast threshold, and the number of the third pixels is greater than the second preset number, then determine the contrast enhancement parameters corresponding to at least one viewing angle region respectively, and the third pixel is Pixels whose contrast components are smaller than the mean value of the contrast components in each pixel. And/or, if the mean value of the sharpness component of each pixel in the video image is less than the sharpness threshold, and the number of the fourth pixel is greater than the second preset number, then determine the sharpness enhancement corresponding to at least one viewing angle area respectively. parameter, the fourth pixel is a pixel whose sharpness component is less than the mean value of the sharpness component in each pixel.
即当视频图像中各个像素满足上述条件时，服务器才确定至少一个视角区域的画面参数，当视频图像中各个像素不满足上述条件时，服务器不确定至少一个视角区域的画面参数，相应的，终端设备也不进行视频图像增强处理，一方面可以提高视频图像增强处理的灵活性，另一方面可以降低服务器和终端设备的功耗。可选的，上述多个视角区域覆盖视频图像。That is, when each pixel in the video image meets the above conditions, the server determines the picture parameters of at least one viewing angle area. When each pixel in the video image does not meet the above conditions, the server determines the picture parameters of at least one viewing angle area. Correspondingly, the terminal The device does not perform video image enhancement processing. On the one hand, the flexibility of the video image enhancement processing can be improved, and on the other hand, the power consumption of the server and the terminal device can be reduced. Optionally, the above-mentioned multiple viewing angle regions cover the video image.
可选的，编码器可以编码生成视频图像，可以将该视频图像注入到业务/媒体管理服务器，业务/媒体管理服务器生成该视频图像的唯一标识。编码器还可以将视频图像注入到流媒体服务器，即该流媒体服务器用于存储视频图像。其中，终端设备和图像增强服务器可以从业务/ 媒体管理服务器获取视频图像的标识。当终端设备需要播放该视频图像时，终端设备向图像增强服务器发送携带有视频图像的标识的请求消息，以请求获取上述至少一个视角区域对应的画面参数。并且终端设备可以从流媒体服务器获取视频图像。Optionally, the encoder may generate a video image by encoding, and the video image may be injected into the service/media management server, and the service/media management server may generate a unique identifier of the video image. The encoder can also inject video images into a streaming media server, which is used to store video images. Wherein, the terminal device and the image enhancement server can obtain the identification of the video image from the service/media management server. When the terminal device needs to play the video image, the terminal device sends a request message carrying the identification of the video image to the image enhancement server, so as to request to acquire the picture parameters corresponding to the at least one viewing angle area. And the terminal device can obtain video images from the streaming media server.
或者，上述业务/媒体管理服务器、流媒体服务器和图像增强服务器为一个服务器，假设将该服务器还被称为图像增强服务器，这种情况下，编码器可以编码生成视频图像，可以将该视频图像注入到图像增强服务器，图像增强服务器生成该视频图像的唯一标识。编码器还可以将视频图像注入到图像增强服务器，即该图像增强服务器用于存储视频图像。其中，终端设备可以从图像增强服务器获取视频图像的标识。当终端设备需要播放该视频图像时，终端设备向图像增强服务器发送携带有视频图像的标识的请求消息，以请求获取上述至少一个视角区域对应的画面参数和视频图像。Alternatively, the above-mentioned service/media management server, streaming media server and image enhancement server are one server, and it is assumed that the server is also called an image enhancement server. In this case, the encoder can encode and generate video images, which can It is injected into the image enhancement server, and the image enhancement server generates the unique identification of the video image. The encoder can also inject video images into an image enhancement server, which is used to store video images. Wherein, the terminal device may obtain the identification of the video image from the image enhancement server. When the terminal device needs to play the video image, the terminal device sends a request message carrying the identification of the video image to the image enhancement server, so as to request to obtain the picture parameters and the video image corresponding to the at least one viewing angle area.
或者，上述业务/媒体管理服务器和流媒体服务器为一个服务器，假设将该服务器称为媒体服务器，编码器可以编码生成视频图像，可以将该视频图像注入到媒体服务器，媒体服务器生成该视频图像的唯一标识。编码器还可以将视频图像注入到媒体服务器，即该媒体服务器用于存储视频图像。其中，终端设备和图像增强服务器可以从媒体服务器获取视频图像的标识。当终端设备需要播放该视频图像时，终端设备向图像增强服务器发送携带有视频图像的标识的请求消息，以请求获取上述至少一个视角区域对应的画面参数。并且终端设备可以从媒体服务器获取视频图像。Alternatively, the above-mentioned service/media management server and streaming media server are one server, assuming that the server is called a media server, the encoder can encode and generate a video image, and the video image can be injected into the media server, and the media server generates the video image. Uniquely identifies. The encoder can also inject video images into the media server, which is used to store the video images. Wherein, the terminal device and the image enhancement server can obtain the identification of the video image from the media server. When the terminal device needs to play the video image, the terminal device sends a request message carrying the identification of the video image to the image enhancement server, so as to request to acquire the picture parameters corresponding to the at least one viewing angle area. And the terminal device can acquire the video image from the media server.
可选的，上述至少一个视角区域的画面参数还包括：至少一个视角区域分别对应的时间戳和空间范围。Optionally, the picture parameters of the at least one viewing angle area further include: timestamps and spatial ranges respectively corresponding to the at least one viewing angle area.
下面将提供终端设备侧所执行的视频图像的增强方法，以及视频图像的增强装置、设备、芯片、存储介质及计算机程序产品，其效果可参考上述视频图像的增强方法对应的效果，下面对此不再赘述。The following will provide a video image enhancement method performed by the terminal device side, as well as a video image enhancement device, device, chip, storage medium and computer program product. This will not be repeated here.
第二方面，本申请提供一种视频图像的增强方法，包括：终端设备向服务器发送请求消息，请求消息包括视频图像的标识，视频图像的标识用于服务器获取视频图像中的至少一个视角区域的画面参数。终端设备接收服务器发送的至少一个视角区域的画面参数。终端设备根据至少一个视角区域的画面参数对视频图像进行增强处理。其中，至少一个视角区域的画面参数包括：至少一个视角区域分别对应的增强参数，至少一个视角区域中任一个视角区域对应的增强参数用于终端设备对该视角区域进行增强处理。In a second aspect, the present application provides a method for enhancing a video image, comprising: a terminal device sending a request message to a server, where the request message includes an identifier of the video image, and the identifier of the video image is used by the server to obtain the information of at least one viewing angle area in the video image. screen parameters. The terminal device receives the picture parameters of at least one viewing angle area sent by the server. The terminal device performs enhancement processing on the video image according to the picture parameters of the at least one viewing angle area. The picture parameters of the at least one viewing angle area include: enhancement parameters corresponding to the at least one viewing angle area respectively, and the enhancement parameter corresponding to any viewing angle area in the at least one viewing angle area is used for the terminal device to perform enhancement processing on the viewing angle area.
可选的，至少一个视角区域的画面参数还包括：至少一个视角区域分别对应的时间戳和空间范围。相应的，终端设备根据至少一个视角区域的画面参数对视频图像进行增强处理，包括：终端设备根据目标视角区域的时间戳和空间范围，以及，至少一个视角区域分别对应的时间戳和空间范围确定目标视角区域对应的增强参数。根据目标视角区域对应的增强参数，对目标视角区域进行增强处理。其中，目标视角区域为至少一个视角区域的部分或者全部。Optionally, the picture parameters of the at least one viewing angle area further include: timestamps and spatial ranges respectively corresponding to the at least one viewing angle area. Correspondingly, the terminal device performs enhancement processing on the video image according to the picture parameters of the at least one viewing angle area, including: the terminal device determines according to the timestamp and the spatial range of the target viewing angle area, and the timestamp and the spatial scope corresponding to the at least one viewing angle area respectively. Enhancement parameters corresponding to the target viewing area. The target viewing angle area is enhanced according to the enhancement parameter corresponding to the target viewing angle area. Wherein, the target viewing angle area is part or all of at least one viewing angle area.
可选的，任一个视角区域对应的增强参数包括以下至少一项：该视角区域的亮度增强参数、饱和度增强参数、对比度增强参数、锐化度增强参数。Optionally, the enhancement parameter corresponding to any viewing angle region includes at least one of the following: a brightness enhancement parameter, a saturation enhancement parameter, a contrast enhancement parameter, and a sharpness enhancement parameter of the viewing angle region.
第三方面，本申请提供一种视频图像的增强装置，包括：In a third aspect, the present application provides a device for enhancing a video image, including:
接收模块，用于接收终端设备发送的请求消息，请求消息包括视频图像的标识。The receiving module is used for receiving a request message sent by the terminal device, where the request message includes the identifier of the video image.
获取模块，用于根据视频图像的标识获取视频图像中的至少一个视角区域的画面参数。The acquiring module is configured to acquire picture parameters of at least one viewing angle area in the video image according to the identifier of the video image.
发送模块，用于向终端设备发送至少一个视角区域的画面参数。The sending module is configured to send the picture parameters of at least one viewing angle area to the terminal device.
其中，至少一个视角区域的画面参数包括：至少一个视角区域分别对应的增强参数，至少一个视角区域中任一个视角区域对应的增强参数用于终端设备对该视角区域进行增强处理。The picture parameters of the at least one viewing angle area include: enhancement parameters corresponding to the at least one viewing angle area respectively, and the enhancement parameter corresponding to any viewing angle area in the at least one viewing angle area is used for the terminal device to perform enhancement processing on the viewing angle area.
第四方面，本申请提供一种一种视频图像的增强装置，包括：In a fourth aspect, the present application provides a device for enhancing a video image, comprising:
发送模块，用于向服务器发送请求消息，请求消息包括视频图像的标识，视频图像的标识用于服务器获取视频图像中的至少一个视角区域的画面参数。The sending module is configured to send a request message to the server, where the request message includes an identifier of the video image, and the identifier of the video image is used by the server to obtain picture parameters of at least one viewing angle area in the video image.
接收模块，用于接收服务器发送的至少一个视角区域的画面参数。The receiving module is configured to receive the picture parameters of at least one viewing angle area sent by the server.
处理模块，用于根据至少一个视角区域的画面参数对视频图像进行增强处理。The processing module is configured to perform enhancement processing on the video image according to the picture parameters of the at least one viewing angle area.
其中，至少一个视角区域的画面参数包括：至少一个视角区域分别对应的增强参数，至少一个视角区域中任一个视角区域对应的增强参数用于终端设备对该视角区域进行增强处理。The picture parameters of the at least one viewing angle area include: enhancement parameters corresponding to the at least one viewing angle area respectively, and the enhancement parameter corresponding to any viewing angle area in the at least one viewing angle area is used for the terminal device to perform enhancement processing on the viewing angle area.
第五方面，本申请提供一种服务器，包括：存储器和处理器。存储器用于存储计算机指令，以使处理器执行计算机指令，以实现如第一方面或第一方面的可选方式的视频图像的增强方法。In a fifth aspect, the present application provides a server, including: a memory and a processor. The memory is used to store computer instructions to cause the processor to execute the computer instructions to implement a method of enhancing a video image as in the first aspect or an alternative of the first aspect.
第六方面，本申请提供一种终端设备，包括：存储器和处理器。存储器用于存储计算机指令，以使处理器执行计算机指令，以实现如第二方面或第二方面的可选方式的视频图像的增强方法。In a sixth aspect, the present application provides a terminal device, including: a memory and a processor. The memory is used to store computer instructions to cause the processor to execute the computer instructions to implement a method of enhancing a video image as in the second aspect or an alternative of the second aspect.
第七方面，本申请提供一种芯片，芯片用于实现如第一方面或第一方面的可选方式的视频图像的增强方法。In a seventh aspect, the present application provides a chip for implementing the video image enhancement method according to the first aspect or an optional manner of the first aspect.
第八方面，本申请提供一种芯片，芯片用于实现如第二方面或第二方面的可选方式的视频图像的增强方法。In an eighth aspect, the present application provides a chip for implementing the video image enhancement method according to the second aspect or an optional manner of the second aspect.
第十方面，本申请提供一种计算机存储介质，包括：计算机可执行指令，计算机可执行指令用于实现如第一方面或第一方面的可选方式的视频图像的增强方法。In a tenth aspect, the present application provides a computer storage medium, comprising: computer-executable instructions, where the computer-executable instructions are used to implement the video image enhancement method according to the first aspect or an optional manner of the first aspect.
第十一方面，本申请提供一种计算机存储介质，包括：计算机可执行指令，计算机可执行指令用于实现如第二方面或第二方面的可选方式的视频图像的增强方法。In an eleventh aspect, the present application provides a computer storage medium, comprising: computer-executable instructions, where the computer-executable instructions are used to implement the video image enhancement method according to the second aspect or an optional manner of the second aspect.
第十二方面，本申请提供一种计算机程序产品，包括：计算机可执行指令，计算机可执行指令用于实现如第一方面或第一方面的可选方式的视频图像的增强方法。In a twelfth aspect, the present application provides a computer program product, comprising: computer-executable instructions, where the computer-executable instructions are used to implement the video image enhancement method according to the first aspect or an optional manner of the first aspect.
第十三方面，本申请提供一种计算机程序产品，包括：计算机可执行指令，计算机可执行指令用于实现如第二方面或第二方面的可选方式的视频图像的增强方法。In a thirteenth aspect, the present application provides a computer program product, comprising: computer-executable instructions, where the computer-executable instructions are used to implement the video image enhancement method according to the second aspect or an optional manner of the second aspect.
本申请提供一种视频图像的增强方法、装置、设备、芯片及存储介质。该方法包括：服务器接收终端设备发送的请求消息，请求消息包括视频图像的标识。服务器根据视频图像的标识获取视频图像中的至少一个视角区域的画面参数。服务器向终端设备发送至少一个视角区域的画面参数。其中，至少一个视角区域的画面参数包括：至少一个视角区域分别对应的增强参数，至少一个视角区域中任一个视角区域对应的增强参数用于终端设备对该视角区域进行增强处理。通过该方法可以提高视频图像显示效果。The present application provides a video image enhancement method, apparatus, device, chip and storage medium. The method includes: the server receives a request message sent by a terminal device, where the request message includes an identifier of a video image. The server acquires picture parameters of at least one viewing angle area in the video image according to the identifier of the video image. The server sends picture parameters of at least one viewing angle area to the terminal device. The picture parameters of the at least one viewing angle area include: enhancement parameters corresponding to the at least one viewing angle area respectively, and the enhancement parameter corresponding to any viewing angle area in the at least one viewing angle area is used for the terminal device to perform enhancement processing on the viewing angle area. By this method, the display effect of the video image can be improved.
附图说明Description of drawings
图1为现有技术提供的片源被切分成若72块的示意图；Fig. 1 is the schematic diagram that the slice source provided by the prior art is divided into 72 pieces;
图2为本申请一实施例提供的本申请技术方案的应用场景图；FIG. 2 is an application scenario diagram of the technical solution of the present application provided by an embodiment of the present application;
图3为本申请一实施例提供的视频图像的增强方法的交互流程图；3 is an interactive flowchart of a method for enhancing a video image provided by an embodiment of the present application;
图4为本申请一实施例提供的视频图像的示意图；4 is a schematic diagram of a video image provided by an embodiment of the present application;
图5A至图5C为本申请另一实施例提供的视频图像的示意图；5A to 5C are schematic diagrams of video images provided by another embodiment of the present application;
图6为本申请一实施例提供的亮度增强示意图；FIG. 6 is a schematic diagram of brightness enhancement provided by an embodiment of the present application;
图7为本申请另一实施例提供的视频图像的增强方法的流程图；7 is a flowchart of a video image enhancement method provided by another embodiment of the present application;
图8为本申请再一实施例提供的视频图像的增强方法的交互流程图；FIG. 8 is an interactive flowchart of a method for enhancing a video image provided by yet another embodiment of the present application;
图9为本申请又一实施例提供的视频图像的增强方法的流程图；9 is a flowchart of a method for enhancing a video image provided by another embodiment of the present application;
图10为本申请一实施例提供的一种视频图像的增强装置的示意图；10 is a schematic diagram of an apparatus for enhancing a video image provided by an embodiment of the present application;
图11为本申请另一实施例提供的一种视频图像的增强装置的示意图；11 is a schematic diagram of an apparatus for enhancing a video image provided by another embodiment of the present application;
图12为本申请一实施例提供的一种服务器的示意图；FIG. 12 is a schematic diagram of a server according to an embodiment of the present application;
图13为本申请一实施例提供的一种终端设备的示意图。FIG. 13 is a schematic diagram of a terminal device according to an embodiment of the application.
具体实施方式Detailed ways
如上所述，对于VR360片源等视频图像，其亮度、对比度、色彩饱和度值在编码时是针对整个360度画面的，因此如果终端设备在播放视角区域内容时，采用整个360度画面的亮度、对比度、色彩饱和度值，这将造成视角区域部分看起来比较暗，颜色不够鲜艳等显示效果不佳的问题。为了解决该技术问题，本申请提供一种视频图像的增强方法、装置、设备、芯片及存储介质。As mentioned above, for video images such as VR360 film sources, the brightness, contrast, and color saturation values are for the entire 360-degree screen during encoding. Therefore, if the terminal device plays the content in the viewing angle area, the brightness of the entire 360-degree screen is used. , contrast, color saturation value, which will cause the viewing angle area to look darker, the color is not bright enough and other problems of poor display effect. In order to solve this technical problem, the present application provides a video image enhancement method, apparatus, device, chip and storage medium.
示例性的，图2为本申请一实施例提供的本申请技术方案的应用场景图，如图2所示，本申请技术方案涉及的网元包括：编码器21、业务/媒体管理服务器22、终端设备23、流媒体服务器24和图像增强服务器25。其中，编码器21分别与业务/媒体管理服务器22、流媒体服务器24和图像增强服务器25连接，并且业务/媒体管理服务器22、流媒体服务器24和图像增强服务器25分别与终端设备23连接。其中，编码器21用于编码生成视频图像，并将该视频图像分别发送给业务/媒体管理服务器22、流媒体服务器24和图像增强服务器25。业务/媒体管理服务器22生成该视频图像的标识。业务/媒体管理服务器22将该视频图像的标识发送给终端设备23。图像增强服务器25确定视频图像中的至少一个视角区域对应的画面参数，终端设备23可以从图像增强服务器25获取该画面参数，并且从流媒体服务器24获取视频图像，以通过该画面参数对视频图像进行增强处理。Exemplarily, FIG. 2 is an application scenario diagram of the technical solution of the present application provided by an embodiment of the present application. As shown in FIG. 2, the network elements involved in the technical solution of the present application include: an
其中，本申请中涉及的终端设备可以是指具有视频或者视频图像播放功能的设备，其可以是具有无线连接功能的手持式设备、或连接到无线调制解调器的其它处理设备。该终端设备可以经无线接入网(Radio Access Network，RAN)与至少一个核心网进行通信。该终端设备可以是移动终端，如移动电话(或称为“蜂窝”电话)和计算机，例如，可以是便携式、袖珍式、手持式、计算机内置的或者车载的移动装置，它们与无线接入网交换数据。终端设备也可以称为用户单元(Subscriber Unit)、用户站(Subscriber Station)，移动站(Mobile Station)、移动台(Mobile Station)、远程站(Remote Station)、接入点(AccessPoint)、远程终端(Remote Terminal)、接入终端(Access Terminal)、用户终端(UserTerminal)、用户代理(User Agent) 或用户设备(User Equipment)，在此不作限定。综上，只要终端设备具有视频或者视频图像播放功能的设备，并且可以和服务器之间实现通信即可，本申请对终端设备的形式不做限制。The terminal device involved in this application may refer to a device with a video or video image playback function, which may be a handheld device with a wireless connection function, or other processing device connected to a wireless modem. The terminal device can communicate with at least one core network via a radio access network (Radio Access Network, RAN). The terminal equipment may be a mobile terminal such as a mobile phone (or "cellular" phone) and a computer, for example, may be a portable, pocket-sized, hand-held, computer-built or vehicle-mounted mobile device, which is connected to a wireless access network exchange data. The terminal equipment may also be called a Subscriber Unit, a Subscriber Station, a Mobile Station, a Mobile Station, a Remote Station, an Access Point, a Remote Terminal (Remote Terminal), access terminal (Access Terminal), user terminal (UserTerminal), user agent (User Agent) or user equipment (User Equipment), which is not limited here. To sum up, as long as the terminal device has a video or video image playback function and can communicate with the server, the present application does not limit the form of the terminal device.
示例性的，上述的至少一个视角区域可以是视频图像中的多个视角区域或者主视角区域，下面分别针对这两种情况对视频图像增强方法进行详细说明：Exemplarily, the above-mentioned at least one viewing angle area may be a plurality of viewing angle areas or a main viewing angle area in the video image, and the video image enhancement method will be described in detail below for these two cases:
图3为本申请一实施例提供的视频图像的增强方法的交互流程图，该方法涉及的网元包括：服务器的部分或者全部，和，终端设备的部分或者全部。其中，服务器的部分可以是服务器中的处理器或者具有处理功能的芯片等，终端设备的部分可以是终端设备中的处理器或者具有处理功能的芯片等。为了将该服务器和本申请中的其他服务器区分开来，在本申请中，可以将该服务器称为图像增强服务器。下面以图像增强服务器和终端设备为执行主体，对视频图像增强方法进行说明。如图3所示，该方法包括如下步骤：FIG. 3 is an interaction flowchart of a video image enhancement method provided by an embodiment of the present application. The network elements involved in the method include: part or all of a server, and part or all of a terminal device. The part of the server may be a processor in the server or a chip with a processing function, etc., and the part of the terminal device may be a processor in the terminal device or a chip with a processing function, or the like. In order to distinguish this server from other servers in this application, in this application, this server may be referred to as an image enhancement server. The following describes the video image enhancement method by taking the image enhancement server and the terminal device as the execution subject. As shown in Figure 3, the method includes the following steps:
可选步骤S301：图像增强服务器确定视频图像中的多个视角区域对应的画面参数。Optional step S301: The image enhancement server determines picture parameters corresponding to multiple viewing angle regions in the video image.
步骤S302：终端设备向图像增强服务器发送请求消息，该请求消息包括：上述视频图像的标识。Step S302: The terminal device sends a request message to the image enhancement server, where the request message includes: the identifier of the above-mentioned video image.
步骤S303：图像增强服务器根据视频图像的标识向终端设备发送上述多个视角区域对应的画面参数。Step S303: The image enhancement server sends the picture parameters corresponding to the above-mentioned multiple viewing angle regions to the terminal device according to the identification of the video image.
步骤S304：终端设备根据上述多个视角区域对应的画面参数对视频图像进行增强处理。Step S304: The terminal device performs enhancement processing on the video image according to the picture parameters corresponding to the above-mentioned multiple viewing angle regions.
针对步骤S301进行说明：Describe step S301:
视频图像被划分为多个视角区域，示例性的，图4为本申请一实施例提供的视频图像的示意图，该视频图像可以为VR360片源，但不限于，如图4所示，视频图像被划分为5个视角区域，分别为视角区域1、视角区域2、视角区域3、视角区域4和视角区域5，每个视角区域为一个矩形区域，这5个视角区域两两相交部分超过120度，且这5个视角区域覆盖视频图像。需要说明的是，视频图像实际是一个环形片源，即如图4所示的第一列内容和最后一列内容相邻。第一列内容包括：数据块1、数据块13、数据块25、数据块37、数据块49、数据块61。最后一列内容包括：数据块12、数据块24、数据块36、数据块48、数据块60、数据块72。A video image is divided into multiple viewing angle regions. Exemplarily, FIG. 4 is a schematic diagram of a video image provided by an embodiment of the application. The video image may be a VR360 film source, but is not limited to, as shown in FIG. It is divided into 5 viewing angle areas, namely viewing angle area 1, viewing angle area 2, viewing angle area 3, viewing angle area 4 and viewing angle area 5, each viewing angle area is a rectangular area, and the intersection of these five viewing angle areas exceeds 120 degrees, and these 5 viewing angle areas cover the video image. It should be noted that the video image is actually a ring source, that is, as shown in FIG. 4 , the content of the first column and the content of the last column are adjacent. The content of the first column includes: data block 1, data block 13, data block 25, data block 37, data block 49, and data block 61. The content of the last column includes: data block 12, data block 24, data block 36, data block 48, data block 60, data block 72.
图像增强服务器可以生成多个视角区域对应的画面参数，针对上述多个视角区域中的任一个视角区域，则多个视角区域对应的画面参数包括：该视角区域对应的增强参数，该增强参数用于终端设备对该视角区域进行增强处理。可选的，该视角区域的增强参数包括以下至少一项：该视角区域的亮度增强参数、饱和度增强参数、对比度增强参数、锐化度增强参数。The image enhancement server can generate picture parameters corresponding to multiple viewing angle regions, and for any one viewing angle region in the above-mentioned multiple viewing angle regions, the picture parameters corresponding to the multiple viewing angle regions include: enhancement parameters corresponding to the viewing angle regions, and the enhancement parameters use The terminal device performs enhancement processing on the viewing angle area. Optionally, the enhancement parameter of the viewing angle region includes at least one of the following: a brightness enhancement parameter, a saturation enhancement parameter, a contrast enhancement parameter, and a sharpness enhancement parameter of the viewing angle region.
可选的，多个视角区域对应的画面参数还包括：任一个视角区域的时间戳和空间范围。Optionally, the picture parameters corresponding to the multiple viewing angle regions further include: timestamp and spatial range of any viewing angle region.
其中，图像增强服务器可以经过训练得到亮度增强参数、饱和度增强参数、对比度增强参数、锐化度增强参数分别对应的增强算法模型，例如：亮度增强参数对应的增强算法模型如下：The image enhancement server can be trained to obtain enhancement algorithm models corresponding to brightness enhancement parameters, saturation enhancement parameters, contrast enhancement parameters, and sharpness enhancement parameters. For example, the enhancement algorithm models corresponding to brightness enhancement parameters are as follows:
Y(new)＝A*Y(original)+B(当Y(original)<200)Y(new)=A*Y(original)+B (when Y(original)<200)
Y(new)＝C*Y(original)+D(当Y(original)>＝200)Y(new)=C*Y(original)+D (when Y(original)>=200)
其中，针对任一视角区域中的任一像素，Y(original)表示在对该像素进行增强处理之前的亮度分量，Y(new)表示在对该像素进行增强处理之后的亮度分量，A、B、C和D为亮度增强参数。Among them, for any pixel in any viewing angle area, Y(original) represents the brightness component before the enhancement processing is performed on the pixel, Y(new) represents the brightness component after the enhancement processing on the pixel, A, B , C and D are brightness enhancement parameters.
进一步地，图像增强服务器可以确定多个视角区域中的每个视角区域分别对应的增强参数。需要说明的是，本申请对图像增强服务器如何训练得到增强算法模型以及如何确定每个视角区域对应的增强参数不做限制。Further, the image enhancement server may determine an enhancement parameter corresponding to each viewing angle region in the multiple viewing angle regions. It should be noted that, the present application does not limit how the image enhancement server trains to obtain the enhancement algorithm model and how to determine the enhancement parameter corresponding to each viewing angle region.
由于视频图像存在多个视角区域，为了令终端设备在播放该视频图像时，能够确定各个视角区域的增强参数，如上所述，每个视角区域的参数还包括该视角区域的时间戳和空间范围。该时间戳为该视角区域的播放时间，时间戳一般以毫秒(millisecond，ms)为单位，视频播放的时候实际是以一帧帧连续的画面按照以固定的时间间隔平稳的展示出来，比如帧率为25帧的视频，意思是每秒播放25帧画面，每帧间隔为40毫秒(1秒/25帧)，时间戳的定义比如从0开始，则第一帧画面的时间戳为0，第二帧画面的时间戳为40，第三帧画面的时间戳为80等。空间范围是按照经纬度的定义的。Since there are multiple viewing angle areas in a video image, in order to enable the terminal device to determine the enhancement parameters of each viewing angle area when playing the video image, as mentioned above, the parameters of each viewing angle area also include the time stamp and spatial range of the viewing angle area . The timestamp is the playback time of the viewing angle area. The timestamp is generally in milliseconds (ms). When the video is played, it is actually displayed as a continuous frame of pictures at a fixed time interval. For example, frame A video with a rate of 25 frames means that 25 frames of pictures are played per second, and the interval between each frame is 40 milliseconds (1 second/25 frames). The timestamp of the second frame is 40, the timestamp of the third frame is 80, and so on. The spatial extent is defined in terms of latitude and longitude.
例如：某个视角区域的参数为：133|(0,90)-(72,-90)|1,1.15,5.5,1.05,3.5。其中，133表示该视角区域的时间戳为133ms，空间范围为(0,90)-(72,-90)，1表示过暗增强场景，即1是场景标识，该标识为可选参数。1.15,5.5,1.05,3.5分别表示A＝1.15，B＝5.5，C＝1.05D＝3.5。For example, the parameters of a certain viewing angle area are: 133|(0,90)-(72,-90)|1,1.15,5.5,1.05,3.5. Among them, 133 indicates that the time stamp of the viewing angle area is 133ms, the spatial range is (0,90)-(72,-90), and 1 indicates that the dark enhancement scene is too dark, that is, 1 is the scene identifier, which is an optional parameter. 1.15, 5.5, 1.05, and 3.5 represent A=1.15, B=5.5, C=1.05, and D=3.5, respectively.
需要说明的是，图像增强服务器还可以为是否确定多个视角区域对应的画面参数设置条件：例如：若视频图像中各个像素的亮度分量均值小于亮度阈值，如255*0.3，且第一像素的个数大于第一预设个数，则确定上述多个视角区域分别对应的亮度增强参数，第一像素为各个像素中亮度分量小于亮度分量均值的像素。和/或，若视频图像中各个像素的饱和度分量均值小于饱和度阈值，且第二像素的个数大于第二预设个数，则确定上述多个视角区域分别对应的饱和度增强参数，第二像素为各个像素中饱和度分量小于饱和度分量均值的像素。和/或，若视频图像中各个像素的对比度分量均值小于对比度阈值，且第三像素的个数大于第二预设个数，则确定上述多个视角区域分别对应的对比度增强参数，第三像素为各个像素中对比度分量小于对比度分量均值的像素。和/或，若视频图像中各个像素的锐化度分量均值小于锐化度阈值，且第四像素的个数大于第二预设个数，则确定上述多个视角区域分别对应的锐化度增强参数，第四像素为各个像素中锐化度分量小于锐化度分量均值的像素。It should be noted that the image enhancement server can also set conditions for whether to determine the picture parameters corresponding to multiple viewing angle regions: for example: if the average value of the luminance component of each pixel in the video image is less than the luminance threshold, such as 255*0.3, and the first pixel If the number is greater than the first preset number, the brightness enhancement parameters corresponding to the plurality of viewing angle regions are determined, and the first pixel is a pixel whose brightness component is less than the mean value of the brightness components in each pixel. And/or, if the average value of the saturation component of each pixel in the video image is less than the saturation threshold, and the number of the second pixels is greater than the second preset number, then determine the saturation enhancement parameters corresponding to the above-mentioned multiple viewing angle regions respectively, The second pixel is a pixel whose saturation component is smaller than the mean value of the saturation component in each pixel. And/or, if the average value of the contrast components of each pixel in the video image is less than the contrast threshold, and the number of third pixels is greater than the second preset number, then determine the contrast enhancement parameters corresponding to the above-mentioned multiple viewing angle regions respectively, and the third pixel is determined. is the pixel whose contrast component is less than the mean value of the contrast component in each pixel. And/or, if the average value of the sharpness component of each pixel in the video image is less than the sharpness threshold, and the number of the fourth pixel is greater than the second preset number, then determine the sharpness corresponding to the above-mentioned multiple viewing angle regions respectively. Enhancement parameter, the fourth pixel is a pixel whose sharpness component is less than the mean value of the sharpness component in each pixel.
针对步骤S302进行说明：Describe step S302:
示例性地，编码器可以编码生成视频图像，可以将该视频图像注入到业务/媒体管理服务器，业务/媒体管理服务器生成该视频图像的唯一标识，如：A894093847378394，该唯一标识用于标识视频图像。编码器还可以将视频图像注入到流媒体服务器，即该流媒体服务器用于存储视频图像。其中，终端设备和图像增强服务器可以从业务/媒体管理服务器获取视频图像的标识。当终端设备需要播放该视频图像时，终端设备向图像增强服务器发送携带有视频图像的标识的请求消息，以请求获取多个视角区域对应的画面参数。并且终端设备可以从流媒体服务器获取视频图像。Exemplarily, the encoder can encode and generate a video image, and the video image can be injected into the service/media management server, and the service/media management server generates a unique identifier of the video image, such as: A894093847378394, which is used to identify the video image. . The encoder can also inject video images into a streaming media server, which is used to store video images. Wherein, the terminal device and the image enhancement server can obtain the identification of the video image from the service/media management server. When the terminal device needs to play the video image, the terminal device sends a request message carrying the identification of the video image to the image enhancement server, so as to request to acquire picture parameters corresponding to multiple viewing angle regions. And the terminal device can obtain video images from the streaming media server.
或者，or,
示例性地，上述业务/媒体管理服务器、流媒体服务器和图像增强服务器为一个服务器，假设将该服务器还被称为图像增强服务器，这种情况下，编码器可以编码生成视频图像，可以将该视频图像注入到图像增强服务器，图像增强服务器生成该视频图像的唯一标识，如： A894093847378394，该唯一标识用于标识视频图像。编码器还可以将视频图像注入到图像增强服务器，即该图像增强服务器用于存储视频图像。其中，终端设备可以从图像增强服务器获取视频图像的标识。当终端设备需要播放该视频图像时，终端设备向图像增强服务器发送携带有视频图像的标识的请求消息，以请求获取多个视角区域对应的画面参数和视频图像。Exemplarily, the above-mentioned service/media management server, streaming media server, and image enhancement server are one server, and it is assumed that the server is also called an image enhancement server. The video image is injected into the image enhancement server, and the image enhancement server generates a unique identifier of the video image, such as: A894093847378394, which is used to identify the video image. The encoder can also inject video images into an image enhancement server, which is used to store video images. Wherein, the terminal device may obtain the identification of the video image from the image enhancement server. When the terminal device needs to play the video image, the terminal device sends a request message carrying the identification of the video image to the image enhancement server, so as to request to acquire picture parameters and video images corresponding to multiple viewing angle regions.
或者，or,
示例性地，上述业务/媒体管理服务器和流媒体服务器为一个服务器，假设将该服务器称为媒体服务器，编码器可以编码生成视频图像，可以将该视频图像注入到媒体服务器，媒体服务器生成该视频图像的唯一标识，如：A894093847378394，该唯一标识用于标识视频图像。编码器还可以将视频图像注入到媒体服务器，即该媒体服务器用于存储视频图像。其中，终端设备和图像增强服务器可以从媒体服务器获取视频图像的标识。当终端设备需要播放该视频图像时，终端设备向图像增强服务器发送携带有视频图像的标识的请求消息，以请求获取多个视角区域对应的画面参数。并且终端设备可以从媒体服务器获取视频图像。Exemplarily, the above-mentioned service/media management server and streaming media server are one server, assuming that the server is called a media server, the encoder can encode and generate a video image, and the video image can be injected into the media server, and the media server generates the video image. The unique identifier of the image, such as: A894093847378394, the unique identifier is used to identify the video image. The encoder can also inject video images into the media server, which is used to store the video images. Wherein, the terminal device and the image enhancement server can obtain the identification of the video image from the media server. When the terminal device needs to play the video image, the terminal device sends a request message carrying the identification of the video image to the image enhancement server, so as to request to acquire picture parameters corresponding to multiple viewing angle regions. And the terminal device can acquire the video image from the media server.
结合步骤S303和步骤S304进行说明：Described in conjunction with step S303 and step S304:
如上所述，视频图像包括多个视角区域，因此当图像增强服务器获取到视频图像的标识后，可以获取上述多个视角区域对应的画面参数。并将多个视角区域对应的画面参数发送给终端设备，终端设备根据目标视角区域的时间戳和空间范围，以及，上述多个视角区域分别对应的时间戳和空间范围确定所述目标视角区域对应的增强参数；并根据所述目标视角区域对应的增强参数，对所述目标视角区域进行增强处理；其中，所述目标视角区域为上述多个视角区域的部分或者全部。例如：图5A至图5C为本申请另一实施例提供的视频图像的示意图，如图5A所示，由于目标视角区域51属于视角区域1，因此目标视角区域51对应的增强参数即为视角区域1对应的增强参数。如图5B所示，由于目标视角区域52属于视角区域1 和视角区域2，因此目标视角区域51对应的增强参数即为视角区域1或者视角区域2对应的增强参数。如图5C所示，由于目标视角区域53属于视角区域3，因此目标视角区域53对应的增强参数即为视角区域3对应的增强参数。As described above, the video image includes multiple viewing angle regions, so after the image enhancement server obtains the identification of the video image, it can obtain the picture parameters corresponding to the above-mentioned multiple viewing angle regions. Send the picture parameters corresponding to the multiple viewing angle areas to the terminal device, and the terminal device determines the corresponding time stamps and spatial ranges of the target viewing angle areas according to the timestamps and spatial ranges of the target viewing angle areas, and the timestamps and spatial ranges corresponding to the multiple viewing angle areas respectively. and performing enhancement processing on the target viewing angle area according to the enhancement parameter corresponding to the target viewing angle area; wherein, the target viewing angle area is part or all of the above-mentioned multiple viewing angle areas. For example, FIGS. 5A to 5C are schematic diagrams of video images provided by another embodiment of the present application. As shown in FIG. 5A , since the target viewing angle area 51 belongs to the viewing angle area 1, the enhancement parameter corresponding to the target viewing angle area 51 is the viewing angle area. 1 corresponds to the enhancement parameter. As shown in FIG. 5B , since the target viewing angle area 52 belongs to the viewing angle area 1 and the viewing angle area 2, the enhancement parameter corresponding to the target viewing angle area 51 is the enhancement parameter corresponding to the viewing angle area 1 or the viewing angle area 2. As shown in FIG. 5C , since the target viewing angle area 53 belongs to the viewing angle area 3 , the enhancement parameter corresponding to the target viewing angle area 53 is the enhancement parameter corresponding to the viewing angle area 3 .
假设终端设备获取到的亮度增强参数A＝1.15，B＝5.5，C＝1.05D＝3.5。则终端设备对每个像素可以进行如下增强处理：Assume that the brightness enhancement parameters obtained by the terminal device are A=1.15, B=5.5, C=1.05D=3.5. Then the terminal device can perform the following enhancement processing on each pixel:
Y(new)＝1.15*Y(original)+5.5(当Y(original)<200)Y(new)=1.15*Y(original)+5.5 (when Y(original)<200)
Y(new)＝1.05*Y(original)+3.5(当200<＝Y(original)<230)Y(new)=1.05*Y(original)+3.5 (when 200<=Y(original)<230)
其中，针对任一视角区域中的任一像素，Y(original)表示在对该像素进行增强处理之前的亮度分量，Y(new)表示在对该像素进行增强处理之后的亮度分量，A、B、C和D为亮度增强参数。图6为本申请一实施例提供的亮度增强示意图，如图6所示，横坐标表示各个像素在进行增强处理之前的亮度分量，纵坐标表示各个像素在进行增强处理之后的亮度分量。Among them, for any pixel in any viewing angle area, Y(original) represents the brightness component before the enhancement processing is performed on the pixel, Y(new) represents the brightness component after the enhancement processing on the pixel, A, B , C and D are brightness enhancement parameters. FIG. 6 is a schematic diagram of brightness enhancement provided by an embodiment of the present application. As shown in FIG. 6 , the abscissa represents the brightness component of each pixel before the enhancement process is performed, and the ordinate represents the brightness component of each pixel after the enhancement process.
可选的，终端设备在对视频图像做完增强处理之后，还可以对视频图像进行渲染。Optionally, after the terminal device performs enhancement processing on the video image, the terminal device may further render the video image.
综上，本申请提供一种视频图像增强方法，包括：终端设备向图像增强服务器发送请求消息，请求消息包括：上述视频图像的标识。图像增强服务器根据视频图像的标识向终端设备发送上述多个视角区域对应的画面参数。终端设备根据上述多个视角区域对应的画面参数对视频图像进行增强处理。即通过该方法可以提高视频图像显示效果。进一步地，本申请提供的技术方案可以将视频图像增强细化到各个视角区域中，每个视角区域都有对应的增强参数，而不是针对整个视频图像进行视频图像增强，从而可以进一步地提高视频图像显示效果。To sum up, the present application provides a video image enhancement method, which includes: a terminal device sends a request message to an image enhancement server, where the request message includes: the identifier of the above-mentioned video image. The image enhancement server sends the picture parameters corresponding to the above-mentioned multiple viewing angle regions to the terminal device according to the identification of the video image. The terminal device performs enhancement processing on the video image according to the picture parameters corresponding to the above-mentioned multiple viewing angle regions. That is, the video image display effect can be improved by this method. Further, the technical solution provided by the present application can refine the video image enhancement into each viewing angle area, and each viewing angle area has corresponding enhancement parameters, instead of performing video image enhancement for the entire video image, so that the video image can be further improved. Image display effect.
如上所述，编码器可以编码生成视频图像，可以将该视频图像注入到业务/媒体管理服务器，业务/媒体管理服务器生成该视频图像的唯一标识，该唯一标识用于标识视频图像。编码器还可以将视频图像注入到流媒体服务器，即该流媒体服务器用于存储视频图像。其中，终端设备和图像增强服务器可以从业务/媒体管理服务器获取视频图像的标识。当终端设备需要播放该视频图像时，终端设备向图像增强服务器发送携带有视频图像的标识的请求消息，以请求获取多个视角区域对应的画面参数。并且终端设备可以从流媒体服务器获取视频图像。下面对这种方案进行详细说明：As described above, the encoder can encode and generate a video image, and the video image can be injected into the service/media management server, and the service/media management server generates a unique identifier of the video image, and the unique identifier is used to identify the video image. The encoder can also inject video images into a streaming media server, which is used to store video images. Wherein, the terminal device and the image enhancement server can obtain the identification of the video image from the service/media management server. When the terminal device needs to play the video image, the terminal device sends a request message carrying the identification of the video image to the image enhancement server, so as to request to acquire picture parameters corresponding to multiple viewing angle regions. And the terminal device can obtain video images from the streaming media server. This scheme is described in detail below:
图7为本申请另一实施例提供的视频图像的增强方法的流程图，如图7所示，该方法包括如下步骤：FIG. 7 is a flowchart of a video image enhancement method provided by another embodiment of the present application. As shown in FIG. 7 , the method includes the following steps:
步骤S701：编码器编码生成视频图像。Step S701: The encoder encodes and generates a video image.
步骤S702：编码器将该视频图像发送给业务/媒体管理服务器。Step S702: the encoder sends the video image to the service/media management server.
步骤S703：业务/媒体管理服务器生成该视频图像的标识。Step S703: The service/media management server generates an identifier of the video image.
步骤S704：业务/媒体管理服务器将该视频图像的标识发送给终端设备。Step S704: The service/media management server sends the identification of the video image to the terminal device.
步骤S705：编码器将该视频图像发送给流媒体服务器。Step S705: the encoder sends the video image to the streaming media server.
步骤S706：编码器将该视频图像发送给图像增强服务器。Step S706: the encoder sends the video image to the image enhancement server.
步骤S707：图像增强服务器确定上述多个视角区域对应的画面参数。Step S707: The image enhancement server determines the picture parameters corresponding to the above-mentioned multiple viewing angle regions.
步骤S708：终端设备向图像增强服务器发送请求消息，该请求消息包括：视频图像的标识。Step S708: The terminal device sends a request message to the image enhancement server, where the request message includes: an identifier of the video image.
步骤S709：图像增强服务器向终端设备发送响应消息，该响应消息包括：视频图像对应的第一存储地址和各个视角区域对应的参数对应的第二存储地址。Step S709: The image enhancement server sends a response message to the terminal device, where the response message includes: a first storage address corresponding to the video image and a second storage address corresponding to parameters corresponding to each viewing angle area.
其中，第二存储地址包括：上述视频图像的标识，例如：第二存储地址为： http://10.135.13.12/A894093847378394_enhance.dat，A894093847378394为上述视频图像的标识。The second storage address includes: the identifier of the above-mentioned video image, for example, the second storage address is: http://10.135.13.12/A894093847378394_enhance.dat, and A894093847378394 is the identifier of the above-mentioned video image.
步骤S710：终端设备根据该第一存储地址从流媒体服务器获取视频图像。Step S710: The terminal device acquires the video image from the streaming media server according to the first storage address.
步骤S711：终端设备根据该第二存储地址从图像增强服务器获取上述多个视角区域对应的画面参数。Step S711: The terminal device acquires the picture parameters corresponding to the above-mentioned multiple viewing angle regions from the image enhancement server according to the second storage address.
步骤S712：终端设备根据上述多个视角区域对应的画面参数对视频图像进行增强处理。Step S712: The terminal device performs enhancement processing on the video image according to the picture parameters corresponding to the above-mentioned multiple viewing angle regions.
需要说明的是，本实施例主要应用于存在业务/媒体管理服务器、流媒体服务器、图像增强服务器的情况，它们分别用于确定视频图像的标识、存储视频图像和获取视频图像的各个视角区域对应的参数，本实施例的细节可参考上一实施例，在此不再赘述。It should be noted that this embodiment is mainly applied to the existence of a service/media management server, a streaming media server, and an image enhancement server, which are respectively used to determine the identification of the video image, store the video image, and obtain the corresponding viewing angle regions of the video image. For details of this embodiment, reference may be made to the previous embodiment, which will not be repeated here.
图8为本申请再一实施例提供的视频图像的增强方法的交互流程图，该方法涉及的网元包括：服务器的部分或者全部，和，终端设备的部分或者全部。其中，服务器的部分可以是服务器中的处理器或者具有处理功能的芯片等，终端设备的部分可以是终端设备中的处理器或者具有处理功能的芯片等。为了将该服务器和本申请中的其他服务器区分开来，在本申请中，可以将该服务器称为图像增强服务器。下面以图像增强服务器和终端设备为执行主体，对视频图像增强方法进行说明。如图8所示，该方法包括如下步骤：FIG. 8 is an interactive flowchart of a video image enhancement method provided by still another embodiment of the present application. The network elements involved in the method include: part or all of a server, and part or all of a terminal device. The part of the server may be a processor in the server or a chip with a processing function, etc., and the part of the terminal device may be a processor in the terminal device or a chip with a processing function, or the like. In order to distinguish this server from other servers in this application, in this application, this server may be referred to as an image enhancement server. The video image enhancement method will be described below by taking the image enhancement server and the terminal device as the execution subject. As shown in Figure 8, the method includes the following steps:
可选步骤S801：图像增强服务器确定视频图像中的主视角区域对应的画面参数。Optional step S801: The image enhancement server determines picture parameters corresponding to the main viewing angle area in the video image.
步骤S802：终端设备向图像增强服务器发送请求消息，请求消息包括：上述视频图像的标识。Step S802: The terminal device sends a request message to the image enhancement server, where the request message includes: the identifier of the above-mentioned video image.
步骤S803：图像增强服务器根据视频图像的标识向终端设备发送主视角区域对应的画面参数。Step S803: The image enhancement server sends the picture parameters corresponding to the main viewing angle area to the terminal device according to the identification of the video image.
步骤S804：终端设备根据主视角区域对应的画面参数对视频图像进行增强处理。Step S804: The terminal device performs enhancement processing on the video image according to the picture parameters corresponding to the main viewing angle area.
针对步骤S801进行说明：Describe step S801:
其中，针对上述视频图像，主视角区域是用户在同一时刻实际能够看到的部分。Wherein, for the above video image, the main viewing angle area is the part that the user can actually see at the same time.
图像增强服务器可以生成主视角区域对应的画面参数，该主视角区域对应的画面参数包括：该主视角区域的增强参数，主视角区域的增强参数用于终端设备对主视角区域进行增强处理。主视角区域的增强参数包括以下至少一项：主视角区域的亮度增强参数、饱和度增强参数、对比度增强参数、锐化度增强参数。The image enhancement server can generate picture parameters corresponding to the main perspective area, where the picture parameters corresponding to the main perspective area include: enhancement parameters of the main perspective area, and the enhancement parameters of the main perspective area are used for the terminal device to perform enhancement processing on the main perspective area. The enhancement parameters of the main viewing angle area include at least one of the following: a brightness enhancement parameter, a saturation enhancement parameter, a contrast enhancement parameter, and a sharpness enhancement parameter of the main viewing angle area.
可选的，该主视角区域对应的画面参数还包括：该主视角区域的时间戳和空间范围，其中，图像增强服务器可以经过训练得到亮度增强参数、饱和度增强参数、对比度增强参数、锐化度增强参数分别对应的增强算法模型，例如：亮度增强参数对应的增强算法模型如下：Optionally, the picture parameters corresponding to the main viewing angle area further include: a timestamp and a spatial range of the main viewing angle area, wherein the image enhancement server can obtain brightness enhancement parameters, saturation enhancement parameters, contrast enhancement parameters, sharpening parameters through training. The enhancement algorithm models corresponding to the brightness enhancement parameters, for example, the enhancement algorithm models corresponding to the brightness enhancement parameters are as follows:
Y(new)＝A*Y(original)+B(当Y(original)<200)Y(new)=A*Y(original)+B (when Y(original)<200)
Y(new)＝C*Y(original)+D(当Y(original)>＝200)Y(new)=C*Y(original)+D (when Y(original)>=200)
其中，针对主视角区域中的任一像素，Y(original)表示在对该像素进行增强处理之前的亮度分量，Y(new)表示在对该像素进行增强处理之后的亮度分量，A、B、C和D为亮度增强参数。Among them, for any pixel in the main viewing angle area, Y(original) represents the brightness component before the enhancement processing is performed on the pixel, Y(new) represents the brightness component after the enhancement processing on the pixel, A, B, C and D are brightness enhancement parameters.
进一步地，图像增强服务器可以确定主视角区域对应的增强参数。需要说明的是，本申请对图像增强服务器如何训练得到增强算法模型以及如何确定主视角区域对应的增强参数不做限制。Further, the image enhancement server may determine enhancement parameters corresponding to the main viewing angle area. It should be noted that this application does not limit how the image enhancement server is trained to obtain the enhancement algorithm model and how to determine the enhancement parameters corresponding to the main viewing angle area.
由于视频图像存在多个视角区域，为了令终端设备在播放该视频图像时，能够确定主视角区域的增强参数，如上所述，主视角区域的参数还包括主视角区域的时间戳和空间范围。Since there are multiple viewing angle regions in a video image, in order to enable the terminal device to determine the enhancement parameters of the main viewing angle region when playing the video image, as mentioned above, the parameters of the main viewing angle region also include the timestamp and spatial range of the main viewing angle region.
需要说明的是，图像增强服务器还可以为是否确定主视角区域对应的画面参数设置条件：例如：若视频图像中各个像素的亮度分量均值小于亮度阈值，如255*0.3，且第一像素的个数大于第一预设个数，则确定视频图像中的主视角区域对应的亮度增强参数，第一像素为各个像素中亮度分量小于亮度分量均值的像素。和/或，若视频图像中各个像素的饱和度分量均值小于饱和度阈值，且第二像素的个数大于第二预设个数，则确定视频图像中的主视角区域对应的饱和度增强参数，第二像素为各个像素中饱和度分量小于饱和度分量均值的像素。和/或，若视频图像中各个像素的对比度分量均值小于对比度阈值，且第三像素的个数大于第二预设个数，则确定视频图像中的主视角区域对应的对比度增强参数，第三像素为各个像素中对比度分量小于对比度分量均值的像素。和/或，若视频图像中各个像素的锐化度分量均值小于锐化度阈值，且第四像素的个数大于第二预设个数，则确定视频图像中的主视角区域对应的锐化度增强参数或者多个视角区域分别对应的锐化度增强参数，第四像素为各个像素中锐化度分量小于锐化度分量均值的像素。It should be noted that the image enhancement server can also set conditions for whether to determine the picture parameters corresponding to the main viewing angle area: for example: if the average value of the luminance components of each pixel in the video image is less than the luminance threshold, such as 255*0.3, and the first pixel If the number is greater than the first preset number, the brightness enhancement parameter corresponding to the main viewing angle area in the video image is determined, and the first pixel is a pixel whose brightness component is less than the mean value of the brightness components in each pixel. And/or, if the average value of the saturation component of each pixel in the video image is less than the saturation threshold, and the number of the second pixels is greater than the second preset number, then determine the saturation enhancement parameter corresponding to the main viewing angle area in the video image. , and the second pixel is a pixel whose saturation component is less than the mean value of the saturation component in each pixel. And/or, if the average value of the contrast components of each pixel in the video image is less than the contrast threshold, and the number of the third pixels is greater than the second preset number, then determine the contrast enhancement parameter corresponding to the main viewing angle area in the video image, and the third A pixel is a pixel whose contrast component is smaller than the mean value of the contrast components in each pixel. And/or, if the average value of the sharpness component of each pixel in the video image is less than the sharpness threshold, and the number of the fourth pixel is greater than the second preset number, then determine the sharpening corresponding to the main viewing angle area in the video image. The sharpness enhancement parameter or the sharpness enhancement parameter corresponding to the multiple viewing angle regions respectively, and the fourth pixel is the pixel whose sharpness component is smaller than the mean value of the sharpness component in each pixel.
针对步骤S802进行说明：Describe step S802:
示例性地，编码器可以编码生成视频图像，可以将该视频图像注入到业务/媒体管理服务器，业务/媒体管理服务器生成该视频图像的唯一标识，如：A894093847378394，该唯一标识用于标识视频图像。编码器还可以将视频图像注入到流媒体服务器，即该流媒体服务器用于存储视频图像。其中，终端设备和图像增强服务器可以从业务/媒体管理服务器获取视频图像的标识。当终端设备需要播放该视频图像时，终端设备向图像增强服务器发送携带有视频图像的标识的请求消息，以请求获取主视角区域对应的画面参数。并且终端设备可以从流媒体服务器获取视频图像。Exemplarily, the encoder can encode and generate a video image, and the video image can be injected into the service/media management server, and the service/media management server generates a unique identifier of the video image, such as: A894093847378394, which is used to identify the video image. . The encoder can also inject video images into a streaming media server, which is used to store video images. Wherein, the terminal device and the image enhancement server can obtain the identification of the video image from the service/media management server. When the terminal device needs to play the video image, the terminal device sends a request message carrying the identification of the video image to the image enhancement server, so as to request to obtain the picture parameters corresponding to the main viewing angle area. And the terminal device can obtain video images from the streaming media server.
或者，or,
示例性地，上述业务/媒体管理服务器、流媒体服务器和图像增强服务器为一个服务器，假设将该服务器还被称为图像增强服务器，这种情况下，编码器可以编码生成视频图像，可以将该视频图像注入到图像增强服务器，图像增强服务器生成该视频图像的唯一标识，如：A894093847378394，该唯一标识用于标识视频图像。编码器还可以将视频图像注入到图像增强服务器，即该图像增强服务器用于存储视频图像。其中，终端设备可以从图像增强服务器获取视频图像的标识。当终端设备需要播放该视频图像时，终端设备向图像增强服务器发送携带有视频图像的标识的请求消息，以请求获取主视角区域对应的画面参数和视频图像。Exemplarily, the above-mentioned service/media management server, streaming media server, and image enhancement server are one server, and it is assumed that the server is also called an image enhancement server. The video image is injected into the image enhancement server, and the image enhancement server generates a unique identifier of the video image, such as: A894093847378394, which is used to identify the video image. The encoder can also inject video images into an image enhancement server, which is used to store video images. Wherein, the terminal device may obtain the identification of the video image from the image enhancement server. When the terminal device needs to play the video image, the terminal device sends a request message carrying the identification of the video image to the image enhancement server, so as to request to obtain the picture parameters and the video image corresponding to the main viewing angle area.
或者，or,
示例性地，上述业务/媒体管理服务器和流媒体服务器为一个服务器，假设将该服务器称为媒体服务器，编码器可以编码生成视频图像，可以将该视频图像注入到媒体服务器，媒体服务器生成该视频图像的唯一标识，如：A894093847378394，该唯一标识用于标识视频图像。编码器还可以将视频图像注入到媒体服务器，即该媒体服务器用于存储视频图像。其中，终端设备和图像增强服务器可以从媒体服务器获取视频图像的标识。当终端设备需要播放该视频图像时，终端设备向图像增强服务器发送携带有视频图像的标识的请求消息，以请求获取主视角区域对应的画面参数。并且终端设备可以从媒体服务器获取视频图像。Exemplarily, the above-mentioned service/media management server and streaming media server are one server, assuming that the server is called a media server, the encoder can encode and generate a video image, and the video image can be injected into the media server, and the media server generates the video image. The unique identifier of the image, such as: A894093847378394, the unique identifier is used to identify the video image. The encoder can also inject video images into the media server, which is used to store the video images. Wherein, the terminal device and the image enhancement server can obtain the identification of the video image from the media server. When the terminal device needs to play the video image, the terminal device sends a request message carrying the identification of the video image to the image enhancement server, so as to request to obtain the picture parameters corresponding to the main viewing angle area. And the terminal device can acquire the video image from the media server.
结合步骤S803和步骤S804进行说明：Description will be given in conjunction with step S803 and step S804:
如上所述，视频图像包括主视角区域，因此当图像增强服务器获取到视频图像的标识后，可以获取上述主视角区域对应的画面参数。并将主视角区域对应的画面参数发送给终端设备，终端设备根据目标视角区域的时间戳和空间范围，以及，主视角区域对应的时间戳和空间范围确定所述目标视角区域对应的增强参数；并根据所述目标视角区域对应的增强参数，对所述目标视角区域进行增强处理；其中，所述目标视角区域为主视角区域中的部分或者全部。As described above, the video image includes the main viewing angle area, so after the image enhancement server obtains the identification of the video image, it can obtain the picture parameters corresponding to the above-mentioned main viewing angle area. Sending the picture parameters corresponding to the main viewing angle area to the terminal device, and the terminal device determines the enhancement parameter corresponding to the target viewing angle area according to the timestamp and spatial range of the target viewing angle area, and the timestamp and spatial range corresponding to the main viewing angle area; and performing enhancement processing on the target viewing angle area according to the enhancement parameter corresponding to the target viewing angle area; wherein, the target viewing angle area is part or all of the main viewing angle area.
假设终端设备获取到的亮度增强参数A＝1.15，B＝5.5，C＝1.05D＝3.5。则终端设备对每个像素可以进行如下增强处理：Assume that the brightness enhancement parameters obtained by the terminal device are A=1.15, B=5.5, C=1.05D=3.5. Then the terminal device can perform the following enhancement processing on each pixel:
Y(new)＝1.15*Y(original)+5.5(当Y(original)<200)Y(new)=1.15*Y(original)+5.5 (when Y(original)<200)
Y(new)＝1.05*Y(original)+3.5(当200<＝Y(original)<230)Y(new)=1.05*Y(original)+3.5 (when 200<=Y(original)<230)
其中，针对主视角区域中的任一像素，Y(original)表示在对该像素进行增强处理之前的亮度分量，Y(new)表示在对该像素进行增强处理之后的亮度分量，A、B、C和D为亮度增强参数。可选的，终端设备在对视频图像做完增强处理之后，还可以对视频图像进行渲染。Among them, for any pixel in the main viewing angle area, Y(original) represents the brightness component before the enhancement processing is performed on the pixel, Y(new) represents the brightness component after the enhancement processing on the pixel, A, B, C and D are brightness enhancement parameters. Optionally, after the terminal device performs enhancement processing on the video image, the terminal device may further render the video image.
综上，本申请提供一种视频图像增强方法，包括：终端设备向图像增强服务器发送请求消息，请求消息包括：上述视频图像的标识。图像增强服务器根据视频图像的标识向终端设备发送上述主视角区域对应的画面参数。终端设备根据上述主视角区域对应的画面参数对视频图像进行增强处理。即通过该方法可以提高视频图像显示效果。进一步地，本申请提供的技术方案可以将视频图像增强细化到主视角区域中，主视角区域具有对应的画面参数，而不是针对整个视频图像进行视频图像增强，从而可以进一步地提高视频图像显示效果。To sum up, the present application provides a video image enhancement method, which includes: a terminal device sends a request message to an image enhancement server, where the request message includes: the identifier of the above-mentioned video image. The image enhancement server sends the picture parameter corresponding to the above-mentioned main viewing angle area to the terminal device according to the identification of the video image. The terminal device performs enhancement processing on the video image according to the picture parameter corresponding to the above-mentioned main viewing angle area. That is, the video image display effect can be improved by this method. Further, the technical solution provided by the present application can refine the video image enhancement into the main viewing angle area, and the main viewing angle area has corresponding picture parameters, instead of performing video image enhancement for the entire video image, so that the video image display can be further improved. Effect.
如上所述，编码器可以编码生成视频图像，可以将该视频图像注入到业务/媒体管理服务器，业务/媒体管理服务器生成该视频图像的唯一标识，该唯一标识用于标识视频图像。编码器还可以将视频图像注入到流媒体服务器，即该流媒体服务器用于存储视频图像。其中，终端设备和图像增强服务器可以从业务/媒体管理服务器获取视频图像的标识。当终端设备需要播放该视频图像时，终端设备向图像增强服务器发送携带有视频图像的标识的请求消息，以请求获取主视角区域对应的画面参数。并且终端设备可以从流媒体服务器获取视频图像。下面对这种方案进行详细说明：As described above, the encoder can encode and generate a video image, and the video image can be injected into the service/media management server, and the service/media management server generates a unique identifier of the video image, and the unique identifier is used to identify the video image. The encoder can also inject video images into a streaming media server, which is used to store video images. Wherein, the terminal device and the image enhancement server can obtain the identification of the video image from the service/media management server. When the terminal device needs to play the video image, the terminal device sends a request message carrying the identification of the video image to the image enhancement server, so as to request to obtain the picture parameters corresponding to the main viewing angle area. And the terminal device can obtain video images from the streaming media server. This scheme is described in detail below:
图9为本申请又一实施例提供的视频图像的增强方法的流程图，如图9所示，该方法包括如下步骤：FIG. 9 is a flowchart of a video image enhancement method provided by another embodiment of the present application. As shown in FIG. 9 , the method includes the following steps:
步骤S901：编码器编码生成视频图像。Step S901: the encoder encodes and generates a video image.
步骤S902：编码器将该视频图像发送给业务/媒体管理服务器。Step S902: the encoder sends the video image to the service/media management server.
步骤S903：业务/媒体管理服务器生成该视频图像的标识。Step S903: The service/media management server generates an identifier of the video image.
步骤S904：业务/媒体管理服务器将该视频图像的标识发送给终端设备。Step S904: The service/media management server sends the identification of the video image to the terminal device.
步骤S905：编码器将该视频图像发送给流媒体服务器。Step S905: the encoder sends the video image to the streaming media server.
步骤S906：编码器将该视频图像发送给图像增强服务器。Step S906: the encoder sends the video image to the image enhancement server.
步骤S907：图像增强服务器确定视频图像中的主视角区域对应的画面参数。Step S907: The image enhancement server determines the picture parameters corresponding to the main viewing angle area in the video image.
步骤S908：终端设备向图像增强服务器发送请求消息，该请求消息包括：视频图像的标识。Step S908: The terminal device sends a request message to the image enhancement server, where the request message includes: an identifier of the video image.
步骤S909：图像增强服务器向终端设备发送响应消息，该响应消息包括：视频图像对应的第一存储地址和主视角区域对应的参数对应的第二存储地址。Step S909: The image enhancement server sends a response message to the terminal device, where the response message includes: the first storage address corresponding to the video image and the second storage address corresponding to the parameter corresponding to the main viewing angle area.
其中，第二存储地址包括：上述视频图像的标识，例如：第二存储地址为： http://10.135.13.12/A894093847378394_enhance.dat，A894093847378394为上述视频图像的标识。The second storage address includes: the identifier of the above-mentioned video image, for example, the second storage address is: http://10.135.13.12/A894093847378394_enhance.dat, and A894093847378394 is the identifier of the above-mentioned video image.
步骤S910：终端设备根据该第一存储地址从流媒体服务器获取视频图像。Step S910: The terminal device acquires the video image from the streaming media server according to the first storage address.
步骤S911：终端设备根据该第二存储地址从图像增强服务器获取主视角区域对应的画面参数。Step S911: The terminal device acquires the picture parameters corresponding to the main viewing angle area from the image enhancement server according to the second storage address.
步骤S912：终端设备根据上述主视角区域分别对应的画面参数对视频图像进行增强处理。Step S912: The terminal device performs enhancement processing on the video image according to the picture parameters corresponding to the above-mentioned main viewing angle regions respectively.
需要说明的是，本实施例主要应用于存在业务/媒体管理服务器、流媒体服务器、图像增强服务器的情况，它们分别用于确定视频图像的标识、存储视频图像和获取视频图像的各个视角区域对应的参数，本实施例的细节可参考上一实施例，在此不再赘述。It should be noted that this embodiment is mainly applied to the existence of a service/media management server, a streaming media server, and an image enhancement server, which are respectively used to determine the identification of the video image, store the video image, and obtain the corresponding viewing angle regions of the video image. For details of this embodiment, reference may be made to the previous embodiment, which will not be repeated here.
图10为本申请一实施例提供的一种视频图像的增强装置的示意图，其中该视频图像的增强装置可以为服务器的部分或者全部，如图10所示，该装置包括：FIG. 10 is a schematic diagram of an apparatus for enhancing a video image provided by an embodiment of the application, where the apparatus for enhancing a video image may be part or all of a server. As shown in FIG. 10 , the apparatus includes:
接收模块1001，用于接收终端设备发送的请求消息，请求消息包括视频图像的标识。The
获取模块1002，用于根据视频图像的标识获取视频图像中的至少一个视角区域的画面参数。The obtaining
发送模块1003，用于向终端设备发送至少一个视角区域的画面参数。The sending
其中，至少一个视角区域的画面参数包括：至少一个视角区域分别对应的增强参数，至少一个视角区域中任一个视角区域对应的增强参数用于终端设备对该视角区域进行增强处理。The picture parameters of the at least one viewing angle area include: enhancement parameters corresponding to the at least one viewing angle area respectively, and the enhancement parameter corresponding to any viewing angle area in the at least one viewing angle area is used for the terminal device to perform enhancement processing on the viewing angle area.
可选的，该装置还包括：确定模块1004，用于在接收模块接收终端设备发送的请求消息之前，确定至少一个视角区域的画面参数。Optionally, the apparatus further includes: a determining
可选的，确定模块1004具体用于：若视频图像中各个像素的亮度分量均值小于亮度阈值，且第一像素的个数大于第一预设个数，则确定至少一个视角区域分别对应的亮度增强参数，第一像素为各个像素中亮度分量小于亮度分量均值的像素。和/或，若视频图像中各个像素的饱和度分量均值小于饱和度阈值，且第二像素的个数大于第二预设个数，则确定至少一个视角区域分别对应的饱和度增强参数，第二像素为各个像素中饱和度分量小于饱和度分量均值的像素。和/或，若视频图像中各个像素的对比度分量均值小于对比度阈值，且第三像素的个数大于第二预设个数，则确定至少一个视角区域分别对应的对比度增强参数，第三像素为各个像素中对比度分量小于对比度分量均值的像素。和/或，若视频图像中各个像素的锐化度分量均值小于锐化度阈值，且第四像素的个数大于第二预设个数，则确定至少一个视角区域分别对应的锐化度增强参数，第四像素为各个像素中锐化度分量小于锐化度分量均值的像素。Optionally, the determining
可选的，至少一个视角区域的画面参数还包括：至少一个视角区域分别对应的时间戳和空间范围。Optionally, the picture parameters of the at least one viewing angle area further include: timestamps and spatial ranges respectively corresponding to the at least one viewing angle area.
本申请提供的视频图像的增强装置可用于执行上述图像增强服务器所执行的视频图像的增强方法，其内容和效果可参考方法实施例部分，对此不再赘述。The video image enhancement apparatus provided by the present application can be used to execute the video image enhancement method executed by the above-mentioned image enhancement server. For the content and effect, please refer to the method embodiment section, which will not be repeated here.
图11为本申请另一实施例提供的一种视频图像的增强装置的示意图，其中该视频图像的增强装置可以为服务器的部分或者全部，如图11所示，该装置包括：FIG. 11 is a schematic diagram of an apparatus for enhancing a video image provided by another embodiment of the present application, where the apparatus for enhancing a video image may be part or all of a server. As shown in FIG. 11 , the apparatus includes:
发送模块1101，用于向服务器发送请求消息，请求消息包括视频图像的标识，视频图像的标识用于服务器获取视频图像中的至少一个视角区域的画面参数。The sending
接收模块1102，用于接收服务器发送的至少一个视角区域的画面参数。The
处理模块1103，用于根据至少一个视角区域的画面参数对视频图像进行增强处理。The
其中，至少一个视角区域的画面参数包括：至少一个视角区域分别对应的增强参数，至少一个视角区域中任一个视角区域对应的增强参数用于终端设备对该视角区域进行增强处理。The picture parameters of the at least one viewing angle area include: enhancement parameters corresponding to the at least one viewing angle area respectively, and the enhancement parameter corresponding to any viewing angle area in the at least one viewing angle area is used for the terminal device to perform enhancement processing on the viewing angle area.
可选的，至少一个视角区域的画面参数还包括：至少一个视角区域分别对应的时间戳和空间范围。相应的，处理模块1103具体用于：根据目标视角区域的时间戳和空间范围，以及，至少一个视角区域分别对应的时间戳和空间范围确定目标视角区域对应的增强参数。根据目标视角区域对应的增强参数，对目标视角区域进行增强处理。其中，目标视角区域为至少一个视角区域的部分或者全部。Optionally, the picture parameters of the at least one viewing angle area further include: timestamps and spatial ranges respectively corresponding to the at least one viewing angle area. Correspondingly, the
可选的，任一个视角区域对应的增强参数包括以下至少一项：该视角区域的亮度增强参数、饱和度增强参数、对比度增强参数、锐化度增强参数。Optionally, the enhancement parameter corresponding to any viewing angle region includes at least one of the following: a brightness enhancement parameter, a saturation enhancement parameter, a contrast enhancement parameter, and a sharpness enhancement parameter of the viewing angle region.
本申请提供的视频图像的增强装置可用于执行上述终端设备所执行的视频图像的增强方法，其内容和效果可参考方法实施例部分，对此不再赘述。The video image enhancement apparatus provided by the present application can be used to execute the video image enhancement method executed by the above-mentioned terminal device. For the content and effect, please refer to the method embodiment section, which will not be repeated here.
图12为本申请一实施例提供的一种服务器的示意图，如图12所示，该服务器包括：存储器1201和处理器1202。存储器用于存储计算机指令，以使处理器执行计算机指令，以实现服务器所执行的视频图像的增强方法，其内容和效果可参考方法实施例部分，对此不再赘述。可选的，该服务器还包括收发器1203，用于实现与终端设备或者其他设备之间的数据传输。FIG. 12 is a schematic diagram of a server according to an embodiment of the present application. As shown in FIG. 12 , the server includes: a
图13为本申请一实施例提供的一种终端设备的示意图，如图13所示，该终端设备包括：存储器1301和处理器1302。存储器用于存储计算机指令，以使处理器执行计算机指令，以实现终端设备所执行的视频图像的增强方法，其内容和效果可参考方法实施例部分，对此不再赘述。可选的，该终端设备还包括收发器1303，用于实现与服务器或者其他设备之间的数据传输。FIG. 13 is a schematic diagram of a terminal device according to an embodiment of the application. As shown in FIG. 13 , the terminal device includes: a
本申请提供一种芯片，该芯片用于实现服务器所执行的视频图像的增强方法。其内容和效果可参考方法实施例部分，对此不再赘述。The present application provides a chip for implementing a video image enhancement method executed by a server. For the content and effect, please refer to the method embodiment section, which will not be repeated here.
本申请提供一种芯片，该芯片用于实现终端设备所执行的视频图像的增强方法。其内容和效果可参考方法实施例部分，对此不再赘述。The present application provides a chip for implementing a video image enhancement method performed by a terminal device. For the content and effect, please refer to the method embodiment section, which will not be repeated here.
本申请提供一种计算机存储介质，包括：计算机可执行指令，计算机可执行指令用于实现服务器所执行的视频图像的增强方法。其内容和效果可参考方法实施例部分，对此不再赘述。The present application provides a computer storage medium, including: computer-executable instructions, where the computer-executable instructions are used to implement a method for enhancing a video image executed by a server. For the content and effect, please refer to the method embodiment section, which will not be repeated here.
本申请提供一种计算机存储介质，包括：计算机可执行指令，计算机可执行指令用于实现终端设备所执行的视频图像的增强方法。其内容和效果可参考方法实施例部分，对此不再赘述。The present application provides a computer storage medium, including: computer-executable instructions, where the computer-executable instructions are used to implement a video image enhancement method executed by a terminal device. For the content and effect, please refer to the method embodiment section, which will not be repeated here.
Claims (16)
Translated fromChinesePriority Applications (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910895085.7ACN112541858B (en) | 2019-09-20 | 2019-09-20 | Video image enhancement method, device, equipment, chip and storage medium |
| PCT/CN2020/106526WO2021052040A1 (en) | 2019-09-20 | 2020-08-03 | Video image enhancement method, device, apparatus, chip, and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910895085.7ACN112541858B (en) | 2019-09-20 | 2019-09-20 | Video image enhancement method, device, equipment, chip and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN112541858Atrue CN112541858A (en) | 2021-03-23 |
| CN112541858B CN112541858B (en) | 2025-02-25 |
Family
ID=74883967
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910895085.7AActiveCN112541858B (en) | 2019-09-20 | 2019-09-20 | Video image enhancement method, device, equipment, chip and storage medium |
Country Status (2)
| Country | Link |
|---|---|
| CN (1) | CN112541858B (en) |
| WO (1) | WO2021052040A1 (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114979487A (en)* | 2022-05-27 | 2022-08-30 | 联想(北京)有限公司 | Image processing method, image processing device, electronic equipment and storage medium |
| CN116567287A (en)* | 2022-01-27 | 2023-08-08 | 腾讯科技（深圳）有限公司 | Image quality enhancement method, device, and electronic device |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106454321A (en)* | 2016-10-26 | 2017-02-22 | 深圳市道通智能航空技术有限公司 | Panoramic video processing method, device and system |
| US20170332117A1 (en)* | 2016-05-16 | 2017-11-16 | Google Inc. | Method and apparatus for spatial enhanced adaptive bitrate live streaming for 360 degree video playback |
| CN108965847A (en)* | 2017-05-27 | 2018-12-07 | 华为技术有限公司 | A kind of processing method and processing device of panoramic video data |
| CN109788359A (en)* | 2017-11-14 | 2019-05-21 | 腾讯科技（深圳）有限公司 | A kind of video data handling procedure and relevant apparatus |
| CN109996069A (en)* | 2018-01-03 | 2019-07-09 | 华为技术有限公司 | Video image encoding and decoding method and equipment |
Family Cites Families (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US9210413B2 (en)* | 2012-05-15 | 2015-12-08 | Imagine Mobile Augmented Reality Ltd | System worn by a moving user for fully augmenting reality by anchoring virtual objects |
| CN106919248A (en)* | 2015-12-26 | 2017-07-04 | 华为技术有限公司 | It is applied to the content transmission method and equipment of virtual reality |
| EP3475785A4 (en)* | 2016-04-22 | 2020-05-13 | SZ DJI Technology Co., Ltd. | SYSTEMS AND METHODS FOR PROCESSING IMAGE DATA BASED ON A USER'S INTEREST |
| CN107395984A (en)* | 2017-08-25 | 2017-11-24 | 北京佰才邦技术有限公司 | A kind of method and device of transmission of video |
| CN110223228A (en)* | 2019-05-16 | 2019-09-10 | 北京奇艺世纪科技有限公司 | Image processing method, device, terminal device and storage medium |
- 2019
- 2019-09-20CNCN201910895085.7Apatent/CN112541858B/enactiveActive
- 2020
- 2020-08-03WOPCT/CN2020/106526patent/WO2021052040A1/ennot_activeCeased
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20170332117A1 (en)* | 2016-05-16 | 2017-11-16 | Google Inc. | Method and apparatus for spatial enhanced adaptive bitrate live streaming for 360 degree video playback |
| CN106454321A (en)* | 2016-10-26 | 2017-02-22 | 深圳市道通智能航空技术有限公司 | Panoramic video processing method, device and system |
| CN108965847A (en)* | 2017-05-27 | 2018-12-07 | 华为技术有限公司 | A kind of processing method and processing device of panoramic video data |
| CN109788359A (en)* | 2017-11-14 | 2019-05-21 | 腾讯科技（深圳）有限公司 | A kind of video data handling procedure and relevant apparatus |
| CN109996069A (en)* | 2018-01-03 | 2019-07-09 | 华为技术有限公司 | Video image encoding and decoding method and equipment |
Non-Patent Citations (1)
| Title |
|---|
| 程鹏润: "增强现实原理及标识识别算法", 科技展望, no. 03, 10 March 2015 (2015-03-10)* |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116567287A (en)* | 2022-01-27 | 2023-08-08 | 腾讯科技（深圳）有限公司 | Image quality enhancement method, device, and electronic device |
| CN114979487A (en)* | 2022-05-27 | 2022-08-30 | 联想(北京)有限公司 | Image processing method, image processing device, electronic equipment and storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN112541858B (en) | 2025-02-25 |
| WO2021052040A1 (en) | 2021-03-25 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US10979663B2 (en) | Methods and apparatuses for image processing to optimize image resolution and for optimizing video streaming bandwidth for VR videos | |
| EP3633995B1 (en) | Method and device for chroma prediction | |
| US12112460B2 (en) | Image processing method and apparatus | |
| CN107040794A (en) | Video broadcasting method, server, virtual reality device and panoramic virtual reality play system | |
| WO2018068236A1 (en) | Video stream transmission method, related device and system | |
| WO2023202095A1 (en) | Point cloud media encoding method and apparatus, point cloud media decoding method and apparatus, and electronic device and storage medium | |
| CN113839829A (en) | Cloud game delay testing method, device and system and electronic equipment | |
| CN110290398B (en) | Video issuing method and device, storage medium and electronic equipment | |
| CN110740316A (en) | Data coding method and device | |
| US11910054B2 (en) | Method and apparatus for decoding a 3D video | |
| CN112541858A (en) | Video image enhancement method, device, equipment, chip and storage medium | |
| CN115293994B (en) | Image processing method, image processing device, computer equipment and storage medium | |
| CN111885417B (en) | VR video playing method, device, equipment and storage medium | |
| CN113099237A (en) | Video processing method and device | |
| US12425657B2 (en) | Method and apparatus for decoding point cloud media, and method and apparatus for encoding point cloud media | |
| CN112929703A (en) | Method and device for processing code stream data | |
| KR102735086B1 (en) | Methods for transmitting and rendering 3D scenes, methods for generating patches, and corresponding devices and computer programs | |
| CN110941413A (en) | Display screen generation method and related device | |
| WO2021018223A1 (en) | Video caching method and apparatus | |
| CN117692706B (en) | Video processing method, device, equipment, readable storage medium and product | |
| CN105721887B (en) | Video playback method, device and system | |
| CN114786015B (en) | Screen projection method, terminal and storage medium | |
| CN120568148A (en) | Point cloud data processing method and device and storage medium | |
| Börner et al. | Performance of Augmented Reality Remote Rendering via Mobile Network | |
| Sormain | Offline H. 264 encoding method for omnidirectional videos with empirical region-of-interest |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |