Disclosure of Invention
The embodiment of the invention provides a coding method and a coding device of a long-term reference code stream, which are used for solving the problems that most code stream generating equipment do not support a long-term reference coding mode in the prior art and the long-term reference coding mode in the prior art is high in cost.
The embodiment of the invention provides the following specific technical scheme:
a coding method of a long-term reference code stream comprises the following steps:
the code stream generating equipment respectively carries out short-term reference coding on the acquired original YUV data corresponding to each instant decoding refreshing IDR frame to generate corresponding short-term reference code stream data;
the code stream generation equipment selects an IDR frame as a current long-term reference frame, and respectively carries out P frame coding on original YUV data corresponding to each IDR frame except the current long-term reference frame on the basis of the current long-term reference frame, and obtains corresponding target P frame code stream data;
and the code stream generating equipment respectively replaces the short-term reference code stream data corresponding to each IDR frame except the current long-term reference frame with corresponding target P frame code stream data.
Preferably, the code stream generating device performs P-frame coding on the original YUV data corresponding to any one of the IDR frames except the current long-term reference frame, and obtains corresponding target P-frame code stream data, including:
the code stream generating equipment acquires a quantization parameter QP value and a QP variable quantity of any one IDR frame;
and the code stream generating equipment performs P frame coding on the original YUV data corresponding to any one IDR frame according to the obtained QP value and the QP variable quantity on the basis of the current long-term reference frame to obtain corresponding target P frame code stream data.
Preferably, the obtaining, by the stream generation device, the QP variation of the arbitrary IDR frame includes:
the code stream generating equipment further obtains short-term reference reconstruction YUV data corresponding to any IDR frame; wherein the short-term reference reconstruction YUV data of any one IDR frame is: the original YUV data corresponding to any one IDR frame is generated during short-term reference coding;
the code stream generating equipment calculates a first peak signal-to-noise ratio (PSNR) between the short-term reference reconstruction YUV data of any one IDR frame and corresponding original YUV data;
and the code stream generating equipment acquires the QP variable quantity of any one IDR frame based on the first PSNR.
Preferably, the code stream generating device performs P-frame coding on the original YUV data corresponding to the any IDR frame according to the obtained QP value and QP variation based on the current long-term reference frame, and obtains corresponding target P-frame code stream data, including:
the code stream generating equipment carries out P frame coding on the original YUV data corresponding to any one IDR frame for multiple times based on the current long-term reference frame according to the obtained QP value and the QP variable quantity to obtain a plurality of corresponding P frame code stream data;
and the code stream generating equipment selects one P frame code stream data from the plurality of P frame code stream data as the target P frame code stream data of any one IDR frame.
Preferably, the code stream generating device performs, based on the current long-term reference frame, multiple P-frame coding on the original YUV data corresponding to the arbitrary IDR frame according to the obtained QP value and QP variation, to obtain corresponding multiple P-frame code stream data, where the method includes at least two of the following operations:
the code stream generating equipment carries out P frame coding on the original YUV data corresponding to any one IDR frame according to the obtained QP value based on the current long-term reference frame to obtain corresponding first P frame code stream data;
the code stream generating equipment carries out P frame coding on the original YUV data corresponding to any one IDR frame according to the difference value between the QP value and the QP variable quantity on the basis of the current long-term reference frame to obtain corresponding second P frame code stream data;
and the code stream generating equipment performs P frame coding on the original YUV data corresponding to any one IDR frame according to the sum of the QP value and the QP variable quantity on the basis of the current long-term reference frame to obtain corresponding third P frame code stream data.
Preferably, the code stream generating device selects one P-frame code stream data from the plurality of P-frame code stream data as the target P-frame code stream data of the arbitrary IDR frame, and includes:
the code stream generating device further obtains a plurality of reconstructed YUV data of the P frame corresponding to the any one IDR frame, wherein the reconstructed YUV data of the P frame corresponding to the any one IDR frame is: the original YUV data corresponding to any one IDR frame is generated when P-frame coding is carried out for one time, and one P-frame reconstructed YUV data corresponds to one P-frame code stream data;
the code stream generating equipment respectively calculates a second PSNR between each P frame reconstruction YUV data corresponding to any one IDR frame and short-term reference reconstruction YUV data corresponding to any one IDR frame, and a plurality of second PSNRs corresponding to any one IDR frame are obtained;
the code stream generation equipment selects a target second PSNR from the plurality of second PSNRs;
and the code stream generating equipment selects P frame code stream data corresponding to the target second PSNR from the plurality of P frame code stream data as target P frame code stream data of any one IDR frame.
Preferably, the code stream generating device selects an IDR frame as the current long-term reference frame, and the method includes:
the code stream generating equipment takes the collected first IDR frame as a current long-term reference frame; or,
and when the code stream generating equipment determines that any one target second PSNR is not larger than a preset threshold value, taking an IDR frame corresponding to the any one target second PSNR as a current long-term reference frame.
Preferably, the code stream generating device replaces the short-term reference code stream data corresponding to each of the IDR frames except the current long-term reference frame with the corresponding target P frame code stream data, respectively, and includes:
the code stream generating equipment screens all IDR frames with the target second PSNR larger than a preset threshold value from all other IDR frames except the current long-term reference frame;
and the code stream generation equipment respectively replaces the short-term reference code stream data corresponding to each screened IDR frame with corresponding target P frame code stream data.
Preferably, the code stream generating device replaces the short-term reference code stream data corresponding to any one of the screened IDR frames with the corresponding target P frame code stream data, and includes:
the code stream generating equipment converts syntax elements corresponding to target P frame code stream data of any one of the screened IDR frames according to a preset long-term reference coding mode, and replaces short-term reference code stream data corresponding to the IDR frame with the converted target P frame code stream data; or,
and the code stream generating equipment directly replaces the short-term reference code stream data corresponding to any one of the screened IDR frames with corresponding target P frame code stream data.
An encoding apparatus of a long-term reference code stream, comprising:
the first coding unit is used for respectively carrying out short-term reference coding on the acquired original YUV data corresponding to each instant decoding refreshing IDR frame to generate corresponding short-term reference code stream data;
the second coding unit is used for selecting one IDR frame as a current long-term reference frame, respectively carrying out P frame coding on original YUV data corresponding to each IDR frame except the current long-term reference frame on the basis of the current long-term reference frame, and obtaining corresponding target P frame code stream data;
and the replacing unit is used for respectively replacing the short-term reference code stream data corresponding to each IDR frame except the current long-term reference frame with corresponding target P frame code stream data.
Preferably, when P-frame coding is performed on the original YUV data corresponding to any one of the IDR frames except the current long-term reference frame, and corresponding target P-frame code stream data is obtained, the second coding unit is configured to:
obtaining a quantization parameter QP value and a QP variable of any one IDR frame;
and performing P frame coding on the original YUV data corresponding to any IDR frame according to the obtained QP value and the QP variable quantity based on the current long-term reference frame to obtain corresponding target P frame code stream data.
Preferably, when the QP variation of any one of the IDR frames is obtained, the second encoding unit is configured to:
further obtaining short-term reference reconstruction YUV data corresponding to any one IDR frame; wherein the short-term reference reconstruction YUV data of any one IDR frame is: the original YUV data corresponding to any one IDR frame is generated during short-term reference coding;
calculating a first peak signal-to-noise ratio (PSNR) between the short-term reference reconstruction YUV data of any one IDR frame and corresponding original YUV data;
and acquiring the QP variable quantity of any one IDR frame based on the first PSNR.
Preferably, based on the current long-term reference frame, when performing P-frame coding on the original YUV data corresponding to the arbitrary IDR frame according to the obtained QP value and the obtained QP variation, and obtaining corresponding target P-frame code stream data, the second encoding unit is configured to:
performing P frame coding on the original YUV data corresponding to any one IDR frame for multiple times based on the current long-term reference frame according to the obtained QP value and the QP variable quantity to obtain corresponding multiple P frame code stream data;
and selecting one P frame code stream data from the plurality of P frame code stream data as the target P frame code stream data of any one IDR frame.
Preferably, based on the current long-term reference frame, when performing multiple P-frame coding on the original YUV data corresponding to the arbitrary IDR frame according to the obtained QP value and QP variation, and obtaining multiple corresponding P-frame code stream data, the second encoding unit is configured to perform at least two operations as follows:
performing P frame coding on the original YUV data corresponding to any one IDR frame according to the obtained QP value based on the current long-term reference frame to obtain corresponding first P frame code stream data;
based on the current long-term reference frame, performing P frame coding on the original YUV data corresponding to any one IDR frame according to the difference value between the QP value and the QP variable quantity to obtain corresponding second P frame code stream data;
and performing P frame coding on the original YUV data corresponding to any one IDR frame according to the sum of the QP value and the QP variable quantity on the basis of the current long-term reference frame to obtain corresponding third P frame code stream data.
Preferably, when one P-frame code stream data is selected from the plurality of P-frame code stream data and used as the target P-frame code stream data of any one IDR frame, the second encoding unit is configured to:
further obtaining a plurality of reconstructed YUV data of the P frame corresponding to the any one IDR frame, wherein the reconstructed YUV data of the P frame corresponding to the any one IDR frame is: the original YUV data corresponding to any one IDR frame is generated when P-frame coding is carried out for one time, and one P-frame reconstructed YUV data corresponds to one P-frame code stream data;
respectively calculating second PSNR between each P frame reconstruction YUV data corresponding to any one IDR frame and short-term reference reconstruction YUV data corresponding to any one IDR frame to obtain a plurality of second PSNR corresponding to any one IDR frame;
selecting a target second PSNR from the plurality of second PSNRs;
and selecting P frame code stream data corresponding to the target second PSNR from the plurality of P frame code stream data as target P frame code stream data of any one IDR frame.
Preferably, when an IDR frame is selected as the current long-term reference frame, the second coding unit is configured to:
taking the collected first IDR frame as a current long-term reference frame; or,
and when determining that any one target second PSNR is not larger than a preset threshold value, taking an IDR frame corresponding to the any one target second PSNR as a current long-term reference frame.
Preferably, when short-term reference code stream data corresponding to each of the IDR frames except the current long-term reference frame is replaced with corresponding target P-frame code stream data, the replacing unit is configured to:
screening all IDR frames with the target second PSNR larger than a preset threshold value from all other IDR frames except the current long-term reference frame;
and respectively replacing the short-term reference code stream data corresponding to each screened IDR frame with corresponding target P frame code stream data.
Preferably, when the short-term reference code stream data corresponding to any one of the screened IDR frames is replaced with the corresponding target P frame code stream data, the replacing unit is configured to:
converting syntax elements corresponding to target P frame code stream data of any one of the screened IDR frames according to a preset long-term reference coding mode, and replacing short-term reference code stream data corresponding to any one of the IDR frames with converted target P frame code stream data; or,
and directly replacing the short-term reference code stream data corresponding to any one screened IDR frame with corresponding target P frame code stream data.
The embodiment of the invention has the following beneficial effects:
in the embodiment of the invention, the original code stream generating equipment is utilized to carry out P frame coding on the original YUV data corresponding to each IDR frame except the current long-term reference frame without any hardware modification or coding protocol modification on the current code stream generating equipment, thereby realizing the synthesis of the long-term reference code stream, having simple and efficient realization process and not occupying excessive internal resources.
Furthermore, the code stream generating device performs P frame coding on the original YUV data corresponding to the IDR frame for multiple times according to the obtained QP value and the QP variable quantity, so that the excessive loss caused by P frame coding is reduced, and only the short-term reference code stream data of the IDR frame with the target second PSNR larger than the preset threshold value is replaced by the target P frame code stream data, so that the coding code rate is effectively reduced and the storage space is saved while the image quality is ensured.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
In order to solve the problems that most of existing code stream generating devices do not support a long-term reference coding mode in the prior art and the cost for synthesizing a long-term reference coding code stream is high, in the embodiment of the invention, after short-term reference coding is carried out on each acquired IDR frame by using the existing code stream generating devices, decoding is not needed, one IDR frame can be directly selected as a current long-term reference frame, P frame coding is respectively carried out on each IDR frame except the current long-term reference frame based on the current long-term reference frame, corresponding target P frame code stream data is generated, and short-term reference code stream data corresponding to each IDR frame except the current long-term reference frame is respectively replaced by the corresponding target P frame code stream data.
The present invention will be described in detail with reference to specific examples, but it is to be understood that the present invention is not limited to the examples.
In the embodiment of the present invention, a code stream generating device may collect monitored Data frames (where the Data frames include IDR frames and non-IDR frames) in real time, and perform short-term reference coding on the collected Data frames, and since most of current code stream generating devices do not support a long-term reference coding manner, a format of a generated coded code stream is generally an encoded code stream format of idrpp. The method comprises the steps of firstly obtaining related coded data from a DDR memory, then carrying out short-term reference coding on original YUV data corresponding to each collected data frame based on the obtained related coded data, and storing corresponding short-term reference code stream data and short-term reference reconstruction YUV data generated after the short-term reference coding into the DDR memory.
Referring to fig. 3, in the embodiment of the present invention, a specific flow of a method for encoding a long-term reference code stream is as follows:
step 300: and the code stream generating equipment respectively carries out short-term reference coding on the acquired original YUV data corresponding to each IDR frame to generate corresponding short-term reference code stream data.
Specifically, when step 300 is executed, the following manners may be adopted, but not limited to:
the code stream generating device collects and monitors original YUV data corresponding to each data frame (the data frames include an IDR frame and a non-IDR frame) in real time, and stores the collected original YUV data corresponding to each data frame, and when the code stream generating device performs short-term reference coding on the collected original YUV data corresponding to each data frame, the following three conditions exist but are not limited to:
in the first case: when the code stream generating equipment determines that the current data frame subjected to short-term reference coding is an IDR frame, the short-term reference coding is directly carried out on the original YUV data corresponding to the IDR frame to obtain corresponding short-term reference code stream data, and further obtain corresponding short-term reference reconstruction YUV data.
In the second case: the code stream generating equipment determines that the data frame currently carrying out short-term reference coding is as follows: and when the first P frame between two adjacent IDR frames is used, reconstructing YUV data based on short-term reference of the first IDR frame in the two adjacent IDR frames, performing short-term reference coding on the original YUV data corresponding to the first P frame to obtain corresponding short-term reference code stream data, and further obtaining corresponding short-term reference reconstruction YUV data.
In the third case: the code stream generating equipment determines that the data frame currently carrying out short-term reference coding is as follows: when any one of other P frames between two adjacent IDR frames except the first P frame is in the next P frame, based on the short-term reference reconstruction YUV data of the previous P frame of the any one P frame, the short-term reference coding is carried out on the original YUV data corresponding to the any one P frame, so as to obtain corresponding short-term reference code stream data, and further obtain corresponding short-term reference reconstruction YUV data.
For example: the code stream generating device 1 collects the monitored original YUV data corresponding to each data frame in real time and stores the collected original YUV data corresponding to each data frame.
The code stream generating device 1 performs short-term reference coding on the original YUV data corresponding to each acquired data frame, and the following description will be given by taking as an example that the code stream generating device 1 performs short-term reference coding on a data frame between two adjacent IDR frames, and it is assumed that there are 5P frames (P frame 1-P frame 5 in sequence) between the IDR frame a and the IDR frame B.
When the code stream generating device 1 determines that the current data frame subjected to short-term reference coding is an IDR frame A, short-term reference coding is carried out on the basis of the original YUV data A of the IDR frame A to generate short-term reference code stream data A and short-term reference reconstruction YUV data A.
When the code stream generating device 1 determines that the current data frame subjected to short-term reference coding is a P frame 1 (namely, the first P frame between an IDR frame a and an IDR frame B), short-term reference reconstruction YUV data a is performed on the basis of the short-term reference of the IDR frame a, and short-term reference coding is performed on the original YUV data 1 of the P frame 1 to generate short-term reference code stream data 1 and short-term reference reconstruction YUV data 1.
When the code stream generating device 1 determines that the current data frame subjected to short-term reference coding is a P frame 2 (namely any one of all other P frames except a first P frame between an IDR frame a and an IDR frame B), short-term reference reconstruction YUV data 1 based on the P frame 1 is performed, short-term reference coding is performed on the original YUV data 2 of the P frame 2, and short-term reference code stream data 2 and short-term reference reconstruction YUV data 2 are generated.
When the code stream generating device 1 determines that the current data frame subjected to short-term reference coding is a P frame 3 (which is any one of all other P frames except a first P frame between an IDR frame a and an IDR frame B), short-term reference reconstruction YUV data 2 based on the P frame 2 is performed, and short-term reference coding is performed on the original YUV data 3 of the P frame 3 to generate short-term reference code stream data 3 and short-term reference reconstruction YUV data 3.
When the code stream generating device 1 determines that the current data frame subjected to short-term reference coding is a P frame 4 (which is any one of all other P frames except a first P frame between an IDR frame a and an IDR frame B), short-term reference reconstruction YUV data 3 based on the P frame 3 is performed, and short-term reference coding is performed on the original YUV data 4 of the P frame 4 to generate short-term reference code stream data 4 and short-term reference reconstruction YUV data 4.
When the code stream generating device 1 determines that the current data frame subjected to short-term reference coding is a P frame 5 (which is any one of all other P frames except a first P frame between an IDR frame a and an IDR frame B), short-term reference reconstruction YUV data 4 based on the P frame 4 is performed, and short-term reference coding is performed on the original YUV data 5 of the P frame 5 to generate short-term reference code stream data 5 and short-term reference reconstruction YUV data 5.
When the code stream generating device 1 determines that the current data frame subjected to short-term reference coding is an IDR frame B, short-term reference coding is carried out on the basis of the original YUV data B of the IDR frame B to generate short-term reference code stream data B and short-term reference reconstruction YUV data B.
Step 310: the code stream generating device selects an IDR frame as a current long-term reference frame, and respectively carries out P frame coding on original YUV data corresponding to each IDR frame except the current long-term reference frame based on the current long-term reference frame, and obtains corresponding target P frame code stream data.
Specifically, when step 310 is executed, the following steps may be adopted, but not limited to:
firstly, the code stream generating device selects an IDR frame as a current long-term reference frame, respectively calculates short-term reference reconstruction YUV data of each IDR frame except the current long-term reference frame and a first PSNR between the short-term reference reconstruction YUV data and corresponding original YUV data, respectively calculates QP variable quantity of each IDR frame except the current long-term reference frame based on all the obtained first PSNRs, and directly obtains the QP value of each IDR frame except the current long-term reference frame from a coding standard.
Then, the code stream generating device performs P-frame coding on the original YUV data corresponding to each of the IDR frames except the current long-term reference frame for multiple times respectively according to the obtained QP value and the corresponding QP variation of each of the IDR frames except the current long-term reference frame, based on the current long-term reference frame, and generates a plurality of corresponding P-frame code stream data.
Preferably, the code stream generating device performs P-frame coding on the original YUV data corresponding to any one of the IDR frames except the current long-term reference frame for multiple times, and when generating a plurality of corresponding P-frame code stream data, may perform, but is not limited to, at least two of the following steps:
and the code stream generating equipment performs P frame coding on the original YUV data corresponding to any one IDR frame according to the obtained QP value based on the current long-term reference frame to obtain corresponding first P frame code stream data and further obtain corresponding P frame reconstruction YUV data.
And the code stream generating equipment performs P frame coding on the original YUV data corresponding to any one IDR frame according to the difference value between the QP value and the QP variable quantity on the basis of the current long-term reference frame to obtain corresponding second P frame code stream data and further obtain corresponding P frame reconstruction YUV data.
And the code stream generating equipment performs P frame coding on the original YUV data corresponding to any one IDR frame according to the sum of the QP value and the QP variable quantity on the basis of the current long-term reference frame to obtain corresponding third P frame code stream data and further obtain corresponding P frame reconstruction YUV data.
Secondly, the code stream generating device obtains a plurality of P frame reconstruction YUV data corresponding to each other IDR frame except the current long-term reference frame, and respectively calculates a plurality of second PSNR corresponding to each other IDR frame except the current long-term reference frame, wherein the plurality of second PSNR corresponding to one IDR frame are as follows: and a second PSNR between each P frame reconstructed YUV data generated after P frame coding is carried out on the IDR frame for multiple times and short-term reference reconstructed YUV data corresponding to the IDR frame.
Finally, the code stream generating device selects target P frame code stream data for each of the IDR frames except the current long-term reference frame from the plurality of P frame code stream data and the plurality of P frame reconstructed YUV data corresponding to each of the IDR frames except the current long-term reference frame, and preferably, the target P frame code stream data may be but is not limited to: and the target second PSNR (which may be, but is not limited to, the maximum second PSNR) corresponds to P frame code stream data.
Preferably, when the code stream generating device selects an IDR frame as the current long-term reference frame, it may select, but is not limited to, the following two IDR frames as the current long-term reference frame:
the first selection mode is as follows: and selecting the first acquired IDR frame as a current long-term reference frame.
The second selection mode is as follows: and when determining that any one target second PSNR (which can be but is not limited to the maximum second PSNR) is not greater than a preset threshold value, selecting an IDR frame corresponding to the any one target second PSNR as a current long-term reference frame.
For example: continuing to use the above example, assuming that the code stream generating device 1 selects the first collected IDR frame (i.e., IDR frame a) as the current long-term reference frame, and the data frame currently subjected to P-frame coding is an IDR frame B, the following description will be given by taking as an example that the code stream generating device 1 performs P-frame coding on the original YUV data of the IDR frame B, where the code stream generating device 1 performs P-frame coding on the original YUV data corresponding to each IDR frame collected in real time, respectively.
The code stream generating device 1 selects the first acquired IDR frame (i.e. IDR frame a) as the current long-term reference frame, and calculates the short-term reference of the IDR frame B to reconstruct YUV data, and a first PSNR2 between the short-term reference and the corresponding original YUV data.
The code stream generating device 1 calculates a QP variation 2 of the IDR frame B based on the obtained first PSNR2, and acquires a QP value 2 of the IDR frame B from the encoding standard.
The code stream generating device 1 performs P-frame coding on an IDR frame B3 times based on the IDR frame a and according to QP value 2, the sum of QP value 2 and QP variation 2, and the difference between QP value 2 and QP variation 2, respectively, to obtain corresponding 3P-frame code stream data (P-frame code stream data B1, P-frame code stream data B2, and P-frame code stream data B3), and further obtain corresponding 3P-frame reconstruction YUV data (P-frame reconstruction YUV data B1, P-frame reconstruction YUV data B2, and P-frame reconstruction YUV data B3).
The code stream generating device 1 calculates a second PSNR B1, a second PSNR B2 and a second PSNR B3 corresponding to the IDR frame B, respectively, based on the P-frame reconstructed YUV data B1 and the short-term reference reconstructed YUV data B corresponding to the IDR frame B, the P-frame reconstructed YUV data B2 and the short-term reference reconstructed YUV data B, and the P-frame reconstructed YUV data B3 and the short-term reference reconstructed YUV data B.
The code stream generating device 1 selects the second PSNR B1 with the largest value as the target second PSNR according to the calculated 3 second PSNR (the second PSNR 1, the second PSNR B2, and the second PSNR B3) corresponding to the IDR frame B, and selects the P frame code stream data B1 corresponding to the second PSNR B1 (i.e., the target second PSNR) as the target P frame code stream data of the IDR frame B.
Step 320: and the code stream generating equipment respectively replaces the short-term reference code stream data corresponding to each IDR frame except the current long-term reference frame with corresponding target P frame code stream data.
Specifically, when step 320 is executed, the following steps may be adopted, but not limited to:
first, the code stream generating device screens out all IDR frames, of which the maximum second PSNR (i.e., the target second PSNR) is greater than a preset threshold value, from all IDR frames except the current long-term reference frame.
And then, the code stream generating equipment respectively replaces the short-term reference code stream data corresponding to each screened IDR frame with corresponding target P frame code stream data.
Preferably, in order to ensure the image quality of the monitored image to be clear, the code stream generating device only replaces the short-term reference code stream data corresponding to the IDR frame of which the maximum second PSNR (i.e., the target second PSNR) is greater than the preset threshold value with the corresponding target P frame code stream data, and does not replace the short-term reference code stream data corresponding to the IDR frame of which the maximum second PSNR (i.e., the target second PSNR) is not greater than the preset threshold value.
It is worth mentioning that when the code stream generation device respectively replaces the short-term reference code stream data corresponding to each screened IDR frame with the corresponding target P frame code stream data, the following two methods can be adopted, but are not limited to:
the first mode is as follows: the P frame hard coding mode is a coding mode which is completed by a special chip with an integrated decoding function and does not need an operating system and corresponding software to participate. Specifically, the code stream generation device converts syntax elements corresponding to target P frame code stream data of each screened IDR frame according to a preset long-term reference coding mode, and replaces short-term reference code stream data corresponding to each IDR frame with corresponding converted target P frame code stream data.
The method for hard coding the P frame only relates to the slice header transcoding of target P frame code stream data, decoding and re-coding of short-term reference code stream data are not needed, the implementation process is simple and efficient, excessive internal resources are not occupied, the performance requirement on code stream generating equipment is low, and the main stream code stream generating equipment can complete the synthesis of long-term reference coding code streams.
The second mode is as follows: the P-frame soft coding mode is a coding mode which is completed by the CPU, needs the support of the operating system and has the participation of related software. Specifically, the code stream generation device directly replaces the short-term reference code stream data corresponding to each screened IDR frame with the corresponding target P frame code stream data.
The above P-frame soft coding mode does not need to transcode syntax elements, only refers to coding for a single frame, and codes a frame only within several seconds, and has low requirements on the performance of code stream generating equipment, and the mainstream code stream generating equipment can be realized.
For example: continuing to use the above example, when the code stream generation device 1 determines that the second PSNR B1 (i.e., the target second PSNR) of the IDR frame B is greater than the preset threshold value, transcoding the syntax element corresponding to the target P frame code stream data corresponding to the IDR frame B according to the long-term reference coding method, and replacing the short-term reference code stream data B corresponding to the IDR frame B with the transcoded target P frame code stream data.
Further, the code stream generating device 1 replaces the short-term reference code stream data B corresponding to the IDR frame B with transcoded target P frame code stream data, and then, by using the above-mentioned P frame coding method, performs P frame coding on each acquired IDR frame, and replaces the short-term reference code stream data of all IDR frames whose maximum second PSNR (i.e. target second PSNR) is greater than the preset threshold value with the corresponding transcoded target P frame code stream data, respectively.
For example: the code stream generating device 1 performs P-frame coding on the acquired original YUV data of the IDR frame C by using the above P-frame coding method after replacing the short-term reference code stream data B corresponding to the IDR frame B with transcoded target P-frame code stream data, does not replace the short-term reference code stream data C corresponding to the IDR frame C when determining that the maximum second PSNR (i.e., target second PSNR) of the IDR frame C is smaller than a preset threshold value, and can select the IDR frame C as the current long-term reference frame when performing corresponding P-frame coding on the original YUV data D corresponding to the acquired next IDR frame (assumed to be an IDR frame D).
In the following, the above embodiment is further described in detail by using a specific application scenario, and referring to fig. 4, in the embodiment of the present invention, a specific flow of the encoding method of the long-term reference code stream is as follows:
step 400: the code stream generating device 1 collects the monitored original YUV data corresponding to each data frame in real time and stores the collected original YUV data corresponding to each data frame.
Step 401: the code stream generating device 1 performs short-term reference coding on the original YUV data corresponding to each acquired data frame.
In the following, the code stream generation apparatus 1 will be described by taking as an example only short-term reference encoding of a data frame between two adjacent IDR frames, assuming that there are 5P frames (P frame 1 to P frame 5 in order) between the IDR frame a and the IDR frame B.
When the code stream generating device 1 determines that the current data frame subjected to short-term reference coding is an IDR frame A, short-term reference coding is carried out on the basis of the original YUV data A of the IDR frame A to generate short-term reference code stream data A and short-term reference reconstruction YUV data A.
When the code stream generating device 1 determines that the current data frame subjected to short-term reference coding is a P frame 1 (namely, the first P frame between an IDR frame a and an IDR frame B), short-term reference reconstruction YUV data a is performed on the basis of the short-term reference of the IDR frame a, and short-term reference coding is performed on the original YUV data 1 of the P frame 1 to generate short-term reference code stream data 1 and short-term reference reconstruction YUV data 1.
When the code stream generating device 1 determines that the current data frame subjected to short-term reference coding is a P frame 2 (namely any one of all other P frames except a first P frame between an IDR frame a and an IDR frame B), short-term reference reconstruction YUV data 1 based on the P frame 1 is performed, short-term reference coding is performed on the original YUV data 2 of the P frame 2, and short-term reference code stream data 2 and short-term reference reconstruction YUV data 2 are generated.
When the code stream generating device 1 determines that the current data frame subjected to short-term reference coding is a P frame 3 (which is any one of all other P frames except a first P frame between an IDR frame a and an IDR frame B), short-term reference reconstruction YUV data 2 based on the P frame 2 is performed, and short-term reference coding is performed on the original YUV data 3 of the P frame 3 to generate short-term reference code stream data 3 and short-term reference reconstruction YUV data 3.
When the code stream generating device 1 determines that the current data frame subjected to short-term reference coding is a P frame 4 (which is any one of all other P frames except a first P frame between an IDR frame a and an IDR frame B), short-term reference reconstruction YUV data 3 based on the P frame 3 is performed, and short-term reference coding is performed on the original YUV data 4 of the P frame 4 to generate short-term reference code stream data 4 and short-term reference reconstruction YUV data 4.
When the code stream generating device 1 determines that the current data frame subjected to short-term reference coding is a P frame 5 (which is any one of all other P frames except a first P frame between an IDR frame a and an IDR frame B), short-term reference reconstruction YUV data 4 based on the P frame 4 is performed, and short-term reference coding is performed on the original YUV data 5 of the P frame 5 to generate short-term reference code stream data 5 and short-term reference reconstruction YUV data 5.
When the code stream generating device 1 determines that the current data frame subjected to short-term reference coding is an IDR frame B, short-term reference coding is carried out on the basis of the original YUV data B of the IDR frame B to generate short-term reference code stream data B and short-term reference reconstruction YUV data B.
Step 402: (in the following, only the code stream generating device 1 performs P-frame coding on the raw YUV data of the IDR frame B, where the code stream generating device 1 performs P-frame coding on the raw YUV data corresponding to each IDR frame acquired in real time) the code stream generating device 1 selects the first acquired IDR frame (i.e., the IDR frame a) as the current long-term reference frame, and calculates a short-term reference of the IDR frame B to reconstruct YUV data, and a first PSNR2 between the short-term reference and the corresponding raw YUV data.
Step 403: the code stream generation device 1 calculates a QP variation 2 of the IDR frame B based on the obtained first PSNR2, and obtains a QP value 2 of the IDR frame B.
Step 404: the code stream generating device 1 performs P-frame coding on an IDR frame B3 times based on the IDR frame a according to QP value 2, the sum of QP value 2 and QP variation 2, and the difference between QP value 2 and QP variation 2, respectively, to generate corresponding 3 pieces of P-frame code stream data (P-frame code stream data B1, P-frame code stream data B2, P-frame code stream data B3) and 3 pieces of P-frame reconstructed YUV data (P-frame reconstructed YUV data B1, P-frame reconstructed YUV data B2, P-frame reconstructed YUV data B3).
Step 405: the code stream generating device 1 calculates a second PSNR B1, a second PSNR B2 and a second PSNR B3 corresponding to the IDR frame B, respectively, based on the P-frame reconstructed YUV data B1 and the short-term reference reconstructed YUV data B corresponding to the IDR frame B, the P-frame reconstructed YUV data B2 and the short-term reference reconstructed YUV data B, and the P-frame reconstructed YUV data B3 and the short-term reference reconstructed YUV data B.
Step 406: the code stream generating device 1 selects the second PSNR 1 with the largest value as the target second PSNR according to the calculated 3 second PSNR (the second PSNR B1, the second PSNR B2, and the second PSNR B3) corresponding to the IDR frame B, and selects the P frame code stream data B1 corresponding to the second PSNR B1 (i.e., the target second PSNR) as the target P frame code stream data of the IDR frame B.
Step 407: when the code stream generation device 1 determines that the maximum second PSNR B1 (i.e., the target second PSNR) of the IDR frame B is greater than the preset threshold value, transcoding the syntax element corresponding to the target P frame code stream data of the IDR frame B according to a long-term reference coding method.
Step 408: the code stream generation device 1 replaces the short-term reference code stream data B corresponding to the IDR frame B with the transcoded target P frame code stream data.
Step 409: the code stream generation device 1 performs P-frame coding on each of the acquired IDR frames by using the above P-frame coding method, and replaces short-term reference code stream data of all IDR frames with a maximum second PSNR (i.e., a target second PSNR) greater than a preset threshold value with corresponding transcoded target P-frame code stream data.
The code stream generating device 1 performs P-frame coding on the acquired original YUV data of the IDR frame C by using the above P-frame coding method after replacing the short-term reference code stream data B corresponding to the IDR frame B with transcoded target P-frame code stream data, does not replace the short-term reference code stream data C corresponding to the IDR frame C when determining that the maximum second PSNR of the IDR frame C is smaller than a preset threshold value, and can select the IDR frame C as the current long-term reference frame when performing corresponding P-frame coding on the acquired original YUV data D corresponding to the next IDR frame (assumed to be an IDR frame D).
Based on the foregoing embodiments, as shown in fig. 5, in an embodiment of the present invention, an encoding apparatus for a long-term reference code stream at least includes:
the first encoding unit 500 is configured to perform short-term reference encoding on the acquired original YUV data corresponding to each instant decoding and refreshing IDR frame, and generate corresponding short-term reference code stream data;
a second encoding unit 510, configured to select an IDR frame as a current long-term reference frame, and perform P-frame encoding on original YUV data corresponding to each IDR frame except the current long-term reference frame based on the current long-term reference frame, and obtain corresponding target P-frame code stream data;
a replacing unit 520, configured to replace short-term reference code stream data corresponding to each IDR frame except the current long-term reference frame with corresponding target P frame code stream data.
Preferably, when performing P-frame coding on the original YUV data corresponding to any one of the IDR frames except the current long-term reference frame and obtaining corresponding target P-frame code stream data, the second encoding unit 510 is configured to:
obtaining a quantization parameter QP value and a QP variable of any one IDR frame;
and performing P frame coding on the original YUV data corresponding to any one IDR frame according to the obtained QP value and the QP variable quantity based on the current long-term reference frame to obtain corresponding target P frame code stream data.
Preferably, when obtaining the QP variation of any one of the IDR frames, the second encoding unit 510 is configured to:
further obtaining short-term reference reconstruction YUV data corresponding to any one IDR frame; wherein, the short-term reference reconstruction YUV data of any one of the IDR frames is: the original YUV data corresponding to any one IDR frame is generated during short-term reference coding;
calculating a first peak signal-to-noise ratio (PSNR) between the short-term reference reconstruction YUV data of any one IDR frame and corresponding original YUV data;
and acquiring the QP variable quantity of any one IDR frame based on the first PSNR.
Preferably, when performing P-frame coding on the original YUV data corresponding to the any IDR frame according to the obtained QP value and the obtained QP variation based on the current long-term reference frame, and obtaining corresponding target P-frame code stream data, the second encoding unit 510 is configured to:
performing P frame coding on the original YUV data corresponding to any one IDR frame for multiple times based on the current long-term reference frame according to the obtained QP value and the QP variable quantity to obtain corresponding multiple P frame code stream data;
and selecting one P frame code stream data from the plurality of P frame code stream data as the target P frame code stream data of any one IDR frame.
Preferably, based on the current long-term reference frame, when performing multiple P-frame coding on the original YUV data corresponding to the any IDR frame according to the obtained QP value and the obtained QP variation to obtain multiple corresponding P-frame code stream data, the second encoding unit 510 is configured to perform at least two operations as follows:
performing P frame coding on the original YUV data corresponding to any one IDR frame according to the obtained QP value based on the current long-term reference frame to obtain corresponding first P frame code stream data;
performing P frame coding on the original YUV data corresponding to any one IDR frame according to the difference value between the QP value and the QP variable quantity on the basis of the current long-term reference frame to obtain corresponding second P frame code stream data;
and performing P frame coding on the original YUV data corresponding to any one IDR frame according to the sum of the QP value and the QP variable quantity on the basis of the current long-term reference frame to obtain corresponding third P frame code stream data.
Preferably, when one P-frame code stream data is selected from the plurality of P-frame code stream data to serve as the target P-frame code stream data of any one of the IDR frames, the second encoding unit 510 is configured to:
further obtaining a plurality of reconstructed YUV data of the P frame corresponding to the any one IDR frame, wherein the reconstructed YUV data of the P frame corresponding to the any one IDR frame is: the original YUV data corresponding to any one IDR frame is generated when P-frame coding is carried out for one time, and one P-frame reconstructed YUV data corresponds to one P-frame code stream data;
respectively calculating a second PSNR between each P frame reconstruction YUV data corresponding to any one IDR frame and short-term reference reconstruction YUV data corresponding to any one IDR frame to obtain a plurality of second PSNRs corresponding to any one IDR frame;
selecting a target second PSNR from the plurality of second PSNRs;
and selecting the P frame code stream data corresponding to the target second PSNR from the plurality of P frame code stream data as the target P frame code stream data of any one IDR frame.
Preferably, when an IDR frame is selected as the current long-term reference frame, the second encoding unit 510 is configured to:
taking the collected first IDR frame as a current long-term reference frame; or,
and when determining that any one target second PSNR is not larger than a preset threshold value, taking an IDR frame corresponding to the any one target second PSNR as a current long-term reference frame.
Preferably, when the short-term reference code stream data corresponding to each IDR frame except the current long-term reference frame is replaced with the corresponding target P frame code stream data, the replacing unit 520 is configured to:
screening all IDR frames with the target second PSNR larger than a preset threshold value from all other IDR frames except the current long-term reference frame;
and respectively replacing the short-term reference code stream data corresponding to each screened IDR frame with corresponding target P frame code stream data.
Preferably, when the short-term reference code stream data corresponding to any one of the screened IDR frames is replaced with the corresponding target P frame code stream data, the replacing unit 520 is configured to:
converting syntax elements corresponding to target P frame code stream data of any one of the screened IDR frames according to a preset long-term reference coding mode, and replacing short-term reference code stream data corresponding to any one of the IDR frames with converted target P frame code stream data; or,
and directly replacing the short-term reference code stream data corresponding to any one screened IDR frame with corresponding target P frame code stream data.
In summary, in the embodiment of the present invention, after performing short-term reference coding on each collected IDR frame, the code stream generation device selects one IDR frame as a current long-term reference frame, and performs P-frame coding on each IDR frame except the current long-term reference frame based on the current long-term reference frame, so as to generate corresponding target P-frame code stream data, and replace short-term reference code stream data corresponding to each IDR frame except the current long-term reference frame with corresponding target P-frame code stream data. Therefore, the original YUV data corresponding to each IDR frame except the current long-term reference frame can be subjected to P frame coding by using the original code stream generating equipment without any hardware modification or coding protocol modification on the current code stream generating equipment, so that the synthesis of the long-term reference code stream is realized, the realization process is simple and efficient, and excessive internal resources are not occupied.
Furthermore, the code stream generating device performs P frame coding on the original YUV data corresponding to the IDR frame for multiple times according to the obtained QP value and the QP variable quantity, so that the excessive loss caused by P frame coding is reduced, and only the short-term reference code stream data of the IDR frame with the target second PSNR larger than the preset threshold value is replaced by the target P frame code stream data, so that the coding code rate is effectively reduced and the storage space is saved while the image quality is ensured.
As will be appreciated by one skilled in the art, embodiments of the present invention may be provided as a method, system, or computer program product. Accordingly, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present invention may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.
The present invention is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each flow and/or block of the flow diagrams and/or block diagrams, and combinations of flows and/or blocks in the flow diagrams and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
While preferred embodiments of the present invention have been described, additional variations and modifications in those embodiments may occur to those skilled in the art once they learn of the basic inventive concepts. Therefore, it is intended that the appended claims be interpreted as including preferred embodiments and all such alterations and modifications as fall within the scope of the invention.
It will be apparent to those skilled in the art that various modifications and variations can be made in the embodiments of the present invention without departing from the spirit or scope of the embodiments of the invention. Thus, if such modifications and variations of the embodiments of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to encompass such modifications and variations.