CN116389763A - Video coding method and device based on multiple encoders - Google Patents

Abstract

The application provides a video coding method and device based on various encoders, wherein the method comprises the following steps: s1, processing a video signal through an encoding tool of a first encoder to obtain encoding information of the video signal; s2, encoding the video signal by a second encoder by using the encoding information to generate an encoded video code stream, wherein the encoding standard of the first encoder is different from that of the second encoder. According to the technical scheme, when the video is encoded through the second encoder, the encoding information provided by the encoding tool which is not supported by the second encoder is utilized, so that the encoding efficiency of the second encoder on the video signal is improved.

Description

Video coding method and device based on multiple encoders

Technical Field

The present application relates to the field of video coding, and in particular, to a video coding method and apparatus based on multiple encoders.

Background

In order to achieve higher coding efficiency, video encoders typically encode video signals two or more times. In such two or more coding schemes, the final first coding produces the final output code stream, while the previous coding is to pre-analyze the video content to select optimal coding parameters for the final first coding.

Fig. 1 is a schematic diagram of a two-pass coding scheme according to the prior art. As shown in fig. 1, during the first encoding process, some information of the current frame, such as the average value, variance, integral pixel value distribution map, SAD, SATD, etc. of each 64×64 blocks of pixels are encoded. The second encoding may be later than the first encoding by a number of frames to ensure that there is enough first encoding information to help select the optimal second encoding parameters.

In the second or multiple coding schemes currently common, the same coding standard is used for each coding process. For example, for an HEVC encoder, the first encoding, the second encoding, and the multiple encodings all use the same HEVC encoding standard, and the final coding quality efficiency is limited by the combination of encoding tools supported by the current encoder.

Accordingly, it is desirable to provide a video encoding method capable of improving the encoding efficiency of the existing second encoding or multiple encodings.

Disclosure of Invention

In view of the foregoing, the present application provides a video encoding method and apparatus based on multiple encoders, so as to solve the above technical problems in the prior art.

According to one aspect of the present application, there is provided a video encoding method based on a plurality of encoders, the plurality of encoders including a first encoder and a second encoder, the method comprising:

s1, processing a video signal through an encoding tool of a first encoder to obtain encoding information of the video signal;

s2, encoding the video signal by a second encoder by using the encoding information to generate an encoded video code stream,

wherein the coding standard of the first encoder is different from the coding standard of the second encoder.

According to some embodiments of the present application, the encoding tool used by the first encoder is different from the encoding tool used by the second encoder.

According to some embodiments of the present application, the first encoder comprises a plurality of encoders according to different encoding standards.

According to some embodiments of the present application, the first encoder comprises an AV1 encoder and the second encoder is an h.264 or HEVC encoder.

According to some embodiments of the present application, the encoding tool used by the first encoder comprises a screen content encoding tool, and the encoded information comprises screen content judgment information.

According to some embodiments of the present application, the encoding tool used by the first encoder comprises a compound wedge prediction tool, and the encoded information comprises intra-frame inter-frame compound prediction information.

According to some embodiments of the present application, the encoding tool used by the first encoder comprises a CDEF filtering tool, and the encoding information comprises block boundary information of the video signal.

According to some embodiments of the present application, the first encoder comprises an HEVC or AV1 encoder and the second encoder is an h.264 encoder.

According to some embodiments of the present application, the encoding tool used by the first encoder comprises a block partitioning tool, and the encoding information comprises block partitioning information.

According to an aspect of the present application, there is provided a video encoding apparatus based on a plurality of encoders, the video encoding apparatus including:

the first encoder is used for processing the video signal through an encoding tool of the first encoder to obtain encoding information of the video signal;

a second encoder for encoding the video signal according to the encoding information to generate an encoded video code stream,

wherein the coding standard of the first encoder is different from the coding standard of the second encoder.

According to one aspect of the present application, there is provided an electronic device including:

one or more processors;

a memory for storing executable instructions;

the one or more processors are configured to perform the method described above according to the executable instructions.

According to one aspect of the present application, there is provided a computer-readable storage medium having stored thereon a computer program which, when executed by a processor, causes the processor to perform the method described above.

According to still another aspect of the present application, there is provided an electronic device including: one or more processors and a memory, wherein the memory is to store executable instructions; the one or more processors are configured to implement the methods described above via executable instructions.

From the above technical solution, the technical solution provided in the present application has at least the following advantages: when the video is encoded by the second encoder, the encoding information provided by the encoding tool which is not supported by the second encoder is utilized, so that the encoding efficiency of the video signal by the second encoder is improved.

Drawings

The accompanying drawings are included to provide a further understanding of the technical aspects of the present application and are incorporated in and constitute a part of this specification, illustrate the technical aspects of the present application and together with the examples of the present application, but do not constitute a limitation of the technical aspects of the present application.

Fig. 1 shows a schematic diagram of a two-pass coding scheme according to the prior art;

FIG. 2 illustrates a flow chart of a video encoding method based on multiple encoders provided by exemplary embodiments of the present application;

FIG. 3 illustrates a schematic diagram of a video encoding method based on multiple encoders provided by an exemplary embodiment of the present application;

fig. 4 shows a block diagram of a video encoding apparatus based on various encoders according to an exemplary embodiment of the present application;

fig. 5 shows a block diagram of an electronic device according to an exemplary embodiment of the present application.

Detailed Description

Various exemplary embodiments of the present application will be described in detail below with reference to the accompanying drawings. The description of the exemplary embodiments is merely illustrative and is not intended to be limiting of the application and its uses or uses. This application may be embodied in many different forms and is not limited to the embodiments described herein. These embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the disclosure to those skilled in the art.

Unless the context clearly indicates otherwise, if the number of elements is not specifically limited, the elements may be one or more. As used in this specification, the term "plurality/s/these" means two or more, and the term "based on/according to" should be interpreted as "based at least in part on/according to". Furthermore, the term "and/or" and "at least one of …" encompasses any and all possible combinations of the listed items.

Fig. 2 shows a flowchart of a video encoding method based on multiple encoders provided in an exemplary embodiment of the present application. As shown in fig. 2, a video encoding method based on various encoders according to an exemplary embodiment of the present application includes:

s1, processing a video signal through an encoding tool of a first encoder to obtain encoding information of the video signal;

s2, encoding the video signal by a second encoder by using the encoding information to generate an encoded video code stream,

wherein the coding standard of the first encoder is different from the coding standard of the second encoder.

In this embodiment, the plurality of encoders includes a first encoder and a second encoder. In particular, the various encoders may include encoders employing different encoding standards, such as an AV1 encoder, an h.264 encoder, and an HEVC encoder.

In step S1, the video signal is processed by the encoding tool of the first encoder to obtain encoded information of the video signal. Encoders of different coding standards provide different coding tools. The purpose of step S1 is to obtain the encoded information of the video signal, so that it is not necessary to actually generate an encoded video stream.

In some embodiments, the first encoder may include only one encoder, and in step S1, the video signal to be encoded is processed using one or more encoding tools in the encoder, thereby generating one or more pieces of encoding information corresponding to the encoding tools. In other embodiments, the first encoder may comprise a plurality of encoders of different encoding standards, and in the different S1 each encoder processes the video signal to be encoded by one or more encoding tools, thereby generating one or more pieces of encoded information corresponding to the encoding tools. In the case where the first encoders may include encoders of a plurality of different encoding standards, the processing of the video signal by each first encoder may be performed in parallel or may be performed in any order, which is not limited in this application.

S2, encoding the video signal by a second encoder by using the encoding information to generate an encoded video code stream.

Since the coding standard of the second encoder is different from that of the first encoder, the second encoder does not have a partial coding tool of the first encoder. Thus, the encoded information generated by the encoding tool of the first encoder is not available to the second encoder by its own encoding tool. The coding tool of the second coder adopts the additional auxiliary information obtained by the first coder to code the video in the coding process, thereby improving the coding efficiency.

Fig. 3 is a schematic diagram of a method provided in an exemplary embodiment of the present application.

In the upper part of fig. 3, a process (corresponding to step S1) of processing a video by a first encoder is shown, mainly including processes of block division, predictive coding, transformation, quantization, inverse transformation, deblocking filtering, entropy coding, and the like, each of which needs to be performed using an encoder to provide a corresponding encoding tool. In the corresponding process of the encoding tool, the encoding tool generates corresponding encoding information including, for example, block division information, prediction mode, transformation information, quantization information, code stream information, filtering information, and the like. The purpose of the first encoder processing the video is to generate encoded information and does not require the actual output of the encoded video stream, so the output of the encoded video stream is optional and not necessary.

In the lower part of fig. 3, a process (corresponding to step S2) of processing the video by the second encoder is shown, which mainly includes processes of block division, predictive coding, transformation, quantization, inverse transformation, deblocking filtering, entropy coding, and the like, each of which needs to be performed using an encoder to provide a corresponding coding tool. Since the encoding information including the block division information, the prediction mode, the transform information, the quantization information, the code stream information, the filtering information, and the like generated by the first encoder is transferred to the second encoder, in the encoding process of the second encoder, the encoding tool performing the processes of the block division, the prediction encoding, the transform, the quantization, the inverse transform, the deblocking filtering, the entropy encoding, and the like can perform the block division fast decision using the block division information, the prediction mode fast decision using the prediction mode information, the transform selection using the transform information, the quantization parameter control using the quantization information, the code rate control using the code stream information, and the filtering control using the filtering information, thereby improving the encoding efficiency of the second encoder.

In embodiments of the present application, the first encoder and the second encoder employ different encoding standards, thereby enabling the first encoder to process the video feed signal with encoding tools that the second encoder does not have to generate additional side information for use by the second encoder.

According to some embodiments of the present application, the first encoder comprises an AV1 encoder and the second encoder is an encoder of a different coding standard than the AV1 encoder, such as an HEVC encoder or an h.264 encoder. The encoding tool of the AV1 encoder may provide the following encoded information:

-partition information up to 128x128, down to 4x4 size blocks;

up to 56 directions of prediction information;

-three smooths prediction modes, including "vertical", "horizontal", "vertical/horizontal combination";

-Screen Content Coding (SCC) tools Palette mode, intra Block Copy;

-predicting a chrominance tool Cfl from the gray scale;

-intra inter-frame composite prediction mode information;

in-loop filtering may provide deblocking filtering, limiting directivity enhanced filtering information;

up to 4 transform coding schemes and the longitudinal transform and the transverse transform may be different transform coding schemes.

The second encoder, such as an HEVC encoder or an h.264 encoder, may encode the video signal using the above information provided by the encoding tool of the first encoder (AV 1 encoder) to thereby increase the encoding efficiency of the second encoder.

For example, tools for screen content detection are not included in the h.264 and HEVC standards, and screen content encoding (SCC) of the AV1 encoder may provide the second encoder with determination information of whether the current frame is screen content when processing the video signal. Since there is a great difference in video properties between the screen content and the normal camera shooting content, the optimal encoding strategies for the screen content and the normal camera shooting content are also very different. With this information as to whether it is screen content, the second encoder (e.g., h.264 or HEVC encoder) can more specifically optimize the encoding selection during encoding, such as predictive encoding, with preference for intra prediction for low frequency regions and preference for inter prediction for high frequency regions.

For example, loop filtering in the h.264 standard is simply deblocking filtering, while more complex filtering modules such as SAO, CDEF, etc. are added to the updated HEVC and AV1 standards. If the first encoder (AV 1 encoder) determines that the block boundary is a strong edge through the CDEF filtering tool of AV1, the second encoder (H.264 encoder) can dynamically adjust the deblocking filter strength of H.264 according to the strong edge, so as to achieve better coding quality. If the CDEF filter is not disabled in the first encoder, the second encoder (HEVC encoder) may turn on the SAO filter with a greater probability. If deblocking filtering is not enabled for some coding blocks in the first encoder, the second encoder may also directly not enable deblocking filtering for corresponding blocks, thereby improving coding efficiency.

For example, a compound wedge prediction tool of a first encoder (AV 1 encoder) may provide intra-frame inter-compound prediction information to a second encoder (e.g., HEVC encoder) for guiding a directional selection of a prediction mode of the second encoder. Also for example, a CDEF filtering tool of a first encoder (AV 1 encoder) may provide block boundary information to a second encoder (e.g., an h.264 or HEVC encoder), which may adjust block boundaries based on the block boundary information.

According to some embodiments of the present application, the first encoder comprises an HEVC encoder and the second encoder is an encoder of a different coding standard than the AV1 encoder, such as an AV1 encoder. The coding tool of the HEVC encoder may provide the following coding information:

partition information up to 64x64, as small as 4x4 size blocks;

up to 33 direction prediction information;

-intra DC mode and intra Planar mode information;

-inter Merge prediction mode, inter AMVP prediction mode;

-selection of transform coded DCT and DST.

The second encoder, e.g., an AV1 encoder, may encode the video signal using the above information provided by the encoding tool of the first encoder (HEVC encoder) to increase the encoding efficiency of the second encoder.

For example, a block partitioning tool of a first encoder (HEVC encoder) may provide block partitioning information to a second encoder (e.g., AV1 encoder), which may adjust the block partitioning based on the block partitioning information. If the first encoder (HEVC encoder) is at the maximum block case of 64x64 for block partitioning, then the second encoder (AV 1 encoder) can scale to a larger 128x128 block partition size, if the first encoder's block partitioning is smaller than 64x64, then the second encoder does not have to try 128x128 block partitions, starting encoding directly from 64x64 block partitions.

For example, for a coding direction selected by a first encoder (HEVC encoder), a second encoder (AV 1 encoder) may divide more finely in a peripheral direction of the coding direction, while for a coding direction not selected by the first encoder, the second encoder may crop the coding direction.

For example, if the intra prediction encoding of the first encoder (HEVC encoder) directly selects the DC mode or Planar mode, the second encoder (AV 1 encoder) may prefer the mode selected by the first encoder.

Fig. 4 is a block diagram of a video encoding apparatus based on various encoders according to an exemplary embodiment of the present application. As shown in fig. 4, the apparatus 20 includes:

a first encoder 201 for processing the video signal by an encoding tool of the first encoder to obtain encoded information of the video signal;

a second encoder 202 for encoding the video signal according to the encoding information, to generate an encoded video code stream,

wherein the coding standard of the first encoder is different from the coding standard of the second encoder.

It should be appreciated that the various modules of the multiple encoder based video encoding apparatus shown in fig. 4 may correspond to the various steps in the multiple encoder based video encoding method described with reference to fig. 1. Thus, the operations, features and advantages described above with respect to the multiple encoder-based video encoding methods are equally applicable to the multiple encoder-based video encoding apparatus and the modules comprised thereby. For brevity, certain operations, features and advantages are not described in detail herein.

Although specific functions are discussed above with reference to specific modules, it should be noted that the functions of the various modules discussed herein may be divided into multiple modules and/or at least some of the functions of the multiple modules may be combined into a single module. The particular module performing the actions discussed herein includes the particular module itself performing the actions, or alternatively the particular module invoking or otherwise accessing another component or module that performs the actions (or performs the actions in conjunction with the particular module). Thus, a particular module that performs an action may include that particular module itself that performs the action and/or another module that the particular module invokes or otherwise accesses that performs the action.

It should also be appreciated that various techniques may be described herein in the general context of software hardware elements or program modules. The various modules described above with respect to fig. 4 may be implemented in hardware or in hardware in combination with software and/or firmware. For example, the modules may be implemented as computer program code/instructions configured to be executed in one or more processors and stored in a computer-readable storage medium. Alternatively, these modules may be implemented as hardware logic/circuitry. For example, in some embodiments, one or more of the various modules depicted in fig. 2 may be implemented together in a System on Chip (SoC). The SoC may include one or more components in an integrated circuit chip including a processor, such as a central processing unit (Central Processing Unit, CPU), microcontroller, microprocessor, digital signal processor (Digital Signal Processor, DSP), etc., memory, one or more communication interfaces, and/or other circuitry, and may optionally execute received program code and/or include embedded firmware to perform functions.

In addition to the technical scheme, the application also provides electronic equipment, which comprises one or more processors and a memory for storing executable instructions. Wherein the one or more processors are configured to implement the above-described methods via executable instructions.

The present application also provides a computer readable storage medium having stored thereon a computer program which, when executed by a processor, causes the processor to perform the above-described method.

In the following part of the present description, illustrative examples of the aforementioned electronic device, non-transitory computer readable storage medium, and computer program product will be described in connection with fig. 5.

Fig. 5 shows a block diagram of an electronic device according to an exemplary embodiment of the present application. The system provided by the present application may also be implemented, in whole or in part, by electronic device 300 or a similar device or system.

The electronic device 300 may be a variety of different types of devices. Examples of electronic device 300 include, but are not limited to: desktop, server, notebook or netbook computers, mobile devices, wearable devices, entertainment devices, televisions or other display devices, automotive computers, and the like.

Electronic device 300 may include at least oneprocessor 302, memory 304, communication interface(s) 309,display device 301, other input/output (I/O)devices 310, and one or more mass storage devices 303, capable of communicating with each other, such as through a system bus 311 or other suitable connection.

Processor 302 may be a single processing unit or multiple processing units, all of which may include a single or multiple computing units or multiple cores.Processor 302 may be implemented as one or more microprocessors, microcomputers, microcontrollers, digital signal processors, central processing units, state machines, logic circuitries, and/or any devices that manipulate signals based on operational instructions. Theprocessor 302 may be configured to, among other capabilities, obtain and execute computer-readable instructions stored in the memory 304, mass storage device 303, or other computer-readable medium, such as program code of theoperating system 305, program code of theapplication programs 306, program code ofother programs 307, and so forth.

Memory 304 and mass storage device 303 are examples of computer-readable storage media for storing instructions that are executed byprocessor 302 to implement the various functions as previously described. For example, memory 304 may generally include volatile memory and nonvolatile memory. In addition, mass storage device 303 may typically include hard disk drives, solid state drives, removable media, and the like. Memory 304 and mass storage device 303 may both be referred to collectively in this application as memory or a computer-readable storage medium, and may be non-transitory media capable of storing computer-readable, processor-executable program instructions as computer program code that may be executed byprocessor 302 as a particular machine configured to implement the operations and functions described in the examples of this application.

A number of programs may be stored on the mass storage device 303. These programs include anoperating system 305, one ormore application programs 306,other programs 307, andprogram data 308, and they may be loaded into memory 304 for execution. Examples of such application programs or program modules may include, for example, computer program logic (e.g., computer program code or instructions) for implementing the following components/functions: the methods provided herein (including any suitable steps of the methods) and/or the additional embodiments described herein.

Although illustrated in fig. 5 as being stored in memory 304 of electronic device 300,operating system 305, one ormore application programs 306,other programs 307, andprogram data 308, or portions thereof, may be implemented using any form of computer readable media accessible by electronic device 300. Herein, a computer-readable medium may be any available computer-readable storage medium or communication medium that can be accessed by a computer.

Communication media includes, for example, computer readable instructions, data structures, program modules, or other data in a communication signal that is transferred from one system to another system. The communication medium may include a conductive transmission medium, as well as a wireless medium capable of propagating energy waves. Computer readable instructions, data structures, program modules, or other data may be embodied as a modulated data signal, for example, in a wireless medium. The modulation may be analog, digital or hybrid modulation techniques.

By way of example, computer-readable storage media may include volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storage of information such as computer-readable instructions, data structures, program modules or other data. For example, computer-readable storage media include, but are not limited to, volatile memory, such as random access memory; and nonvolatile memory such as flash memory, various read only memories, magnetic and ferromagnetic/ferroelectric memory; magnetic and optical storage devices; or other known media or later developed computer-readable information/data that may be stored for use by a computer system.

One ormore communication interfaces 309 are used to exchange data with other devices, such as via a network, direct connection, or the like. Such communication interfaces may be one or more of the following: any type of network interface, wired or wireless interface, wi-MAX interface, ethernet interface, universal serial bus interface, cellular network interface, bluetooth interface, NFC interface, etc. Thecommunication interface 309 may facilitate communication within a variety of network and protocol types, including wired and wireless networks, the internet, and the like. Thecommunication interface 309 may also provide communication with external storage devices (not shown) such as in a storage array, network attached storage, storage area network, or the like.

In some examples, adisplay device 301, such as a monitor, may be included for displaying information and images to a user. Other I/O devices 310 may be devices that receive various inputs from a user and provide various outputs to the user, and may include touch input devices, gesture input devices, cameras, keyboards, remote controls, mice, printers, audio input/output devices, and so on. The technical solutions described herein may be supported by these various configurations of the electronic device 300, and are not limited to the specific examples of technical solutions described herein.

While the application has been illustrated and described in detail in the drawings and foregoing description, such illustration and description are to be considered illustrative and schematic and not restrictive; it will be evident to those skilled in the art that the present application is not limited to the details of the foregoing illustrative embodiments, and that the present application may be embodied in other specific forms without departing from the spirit or essential characteristics thereof.

The scope of the invention being indicated by the appended claims rather than by the foregoing description, and all changes which come within the meaning and range of equivalency of the claims are therefore intended to be embraced therein. Any reference sign in a claim should not be construed as limiting the claim concerned. Furthermore, it is evident that the word "comprising" does not exclude other elements or steps, and that the singular does not exclude a plurality. A plurality of units or means recited in the apparatus claims can also be implemented by means of one unit or means in software or hardware.

Claims (12)

1. A method of video encoding based on a plurality of encoders, the plurality of encoders including a first encoder and a second encoder, the method comprising:

s1, processing a video signal through an encoding tool of the first encoder to obtain encoding information of the video signal;

s2, encoding the video signal by the second encoder by using the encoding information to generate an encoded video code stream,

wherein the coding standard of the first encoder is different from the coding standard of the second encoder.

2. The video coding method of claim 1, wherein the first encoder uses a different coding tool than the second encoder.

3. The video coding method of claim 1, wherein the first encoder comprises a plurality of encoders according to different coding standards.

4. The video coding method of claim 1, wherein the first encoder comprises an AV1 encoder and the second encoder is an h.264 or HEVC encoder.

5. The video coding method of claim 4, wherein the coding tool used by the first encoder comprises a screen content coding tool, and the coding information comprises screen content judgment information.

6. The method of video encoding according to claim 4, wherein the encoding tool used by the first encoder comprises a compound wedge prediction tool and the encoding information comprises intra-frame inter-frame compound prediction information.

7. The method of video coding according to claim 4, wherein the coding tool used by the first encoder comprises a CDEF filtering tool, and the coding information comprises block boundary information of the video signal.

8. The video coding method of claim 1, wherein the first encoder comprises an HEVC or AV1 encoder and the second encoder is an h.264 encoder.

9. The video coding method of claim 8, wherein the coding tool used by the first encoder comprises a block partitioning tool, and wherein the coding information comprises block partitioning information.

10. A video encoding apparatus based on a plurality of encoders, the video encoding apparatus comprising:

a first encoder for processing a video signal by an encoding tool of the first encoder to obtain encoding information of the video signal;

a second encoder for encoding the video signal based on the encoding information to generate an encoded video stream,

wherein the coding standard of the first encoder is different from the coding standard of the second encoder.

11. An electronic device, the electronic device comprising:

one or more processors;

a memory for storing executable instructions;

the one or more processors are configured to implement the method of any of claims 1 to 9 via the executable instructions.

12. A computer readable storage medium having stored thereon a computer program which, when executed by a processor, causes the processor to perform the method of any of claims 1 to 9.

Images