CN102547265B - Interframe prediction method and device - Google Patents

Abstract

The invention provides an interframe prediction method which comprises the following steps: step 1: determining the relationship between a first reference frame and the current coding frame; step 2: if a lens zooms in, processing the first reference frame to obtain a second reference frame, and setting the current reference frame as the second reference frame; entering into the step 3; if the lens zooms out, processing the first reference frame to obtain a fourth reference frame, and setting the current reference frame as the fourth reference frame; entering into the step 3: if the current reference frame is the first reference frame, entering into the step 3; step 3: carrying out interframe prediction on the current coding frame by adopting the current reference frame.

Description

A kind of inter-frame prediction method, device

Technical field

The present invention relates to field of video encoding, relate in particular to a kind of inter-frame prediction method, device.

Background technology

At present, in video coding process, conventionally carry out the spatial redundancies of removal of images with intra-frame prediction method, eliminate temporal redundancy with inter-frame prediction method.Due to the temporal redundancy of the interframe of video source, to be compared to spatial redundancies in frame much bigger, and this just makes inter-frame prediction method in Video coding, seem very important so.

Inter prediction is divided into according to the difference of prediction direction: the prediction of P frame and the prediction of B frame.The frame that the forward direction of the P frame Forecasting Methodology of main flow employing has at present been encoded, as the reference frame of current encoded frame, utilizes similitude between the two, the information of compression current encoded frame.This,, in the time that reference frame and current encoded frame have the high degree of approximation, have good effect, but along with the reduction of the degree of approximation between the two, compression effectiveness also can sharply decline.Especially in the time that the film sources such as low frame per second, overall track in are encoded, this problem will significantly show especially.

Summary of the invention

The object of the embodiment of the present invention is to propose a kind of inter-frame prediction method, be intended to solve the reference frame and the current encoded frame degree of approximation that in prior art, run into less, especially in the time that the film sources such as low frame per second, overall track in are encoded, cause compressing the problem that present encoding effect frame is lower.

The invention provides a kind inter-frame prediction method,, described method comprises:

Step 1: the relation of determining the first reference frame and current encoded frame;

Step 2: the first reference frame is processed if camera lens furthers, obtained the second reference frame, and current reference frame is set to the second reference frame; Enter step 3;

If camera lens zooms out the first reference frame is processed, obtain the 4th reference frame, and current reference frame is set to the 4th reference frame; Enter step 3;

If current reference frame is the first reference frame, enter step 3;

Step 3: adopt current reference frame to carry out inter prediction to current encoded frame.

The present invention also provides a kind of inter prediction device, and this device comprises:

Judging unit, for determining the relation of the first reference frame and current encoded frame;

The camera lens unit that furthers, for the first reference frame being processed, obtain the second reference frame in the time that camera lens furthers, and current reference frame is set to the second reference frame;

Camera lens extension unit, for the first reference frame being processed, obtain the 4th reference frame in the time zooming out for camera lens, and current reference frame is set to the 4th reference frame;

Predicting unit, for adopting current reference frame to carry out inter prediction to current encoded frame.

The present invention proposes a kind of inter-frame prediction method and device.The method and device are by determining the relation of current reference frame and current encoded frame, the mode zooming out according to camera lens respectively or further is carried out upper and lower sampling processing, improve the similarity of current reference frame and coded frame, thereby reached the compression effectiveness of optimizing current encoded frame.In the time of low frame per second, the shot change between frame and frame is larger, now adopts frame Forecasting Methodology of the present invention, and the lifting of compression performance will be more remarkable.

Brief description of the drawings

Fig. 1 is the method flow diagram of the embodiment of the present invention 1;

Fig. 2 is the method flow diagram of the embodiment of the present invention 2.

Fig. 3 is the structural representation of the embodiment of the present invention 3.

Embodiment

In order to make object of the present invention, technical scheme and advantage clearer, below in conjunction with drawings and Examples, the present invention is further elaborated, for convenience of explanation, only show the part relevant to the embodiment of the present invention.Should be appreciated that the specific embodiment that this place is described, only for explaining the present invention, not in order to limit the present invention.

The present invention proposes a kind of inter-frame prediction method of new P frame.The method is by determining the relation of current reference frame and current encoded frame, zooms out respectively or the mode that furthers is carried out upper and lower sampling processing according to camera lens, improved the similarity of current reference frame and coded frame, thereby reached the compression effectiveness of optimization current encoded frame.In the time of low frame per second, the shot change between frame and frame is larger, now adopts frame Forecasting Methodology of the present invention, and the lifting of compression performance will be more remarkable.

Embodiment 1, referring to Fig. 1, the method is mainly used in the prediction of P frame, is specially:

Step 101: the relation of determining the first reference frame and current encoded frame; Namely determine i reference frame ref_iwith the relation of current encoded frame frame, select different P frame Forecasting Methodologies; If furthering, camera lens performs step 102; If zooming out, camera lens performs step 103; If do not exist camera lens further or zoom out, the first reference frame is that current reference frame performs step 104;

If (camera lens draws in) enters step 102;

Else if (camera lens pushes away far) enters step 103;

Else curr_ref_i=ref_i, enter 104;

Namely i reference frame ref of the first reference frame described herein_i; Curr_ref_ifor the current reference frame after upgrading;

Step 102: the first reference frame is processed if camera lens furthers, obtained the second reference frame, and current reference frame is set to the second reference frame; Enter step 104;

This step method is specially: to ref_icarry out up-sampling, obtain new reference frame the second reference frame ref_i'; Curr_ref_i=ref_i';

Step 103: if camera lens zooms out, the first reference frame is processed, obtained the 4th reference frame, and current reference frame is set to the 4th reference frame; Enter step 104;

This step method is specially: to ref_icarry out down-sampling, obtain new reference frame the 4th reference frame ref_i" ', curr_ref_i=ref_i" ';

Step 104: adopt current reference frame to carry out inter prediction to current encoded frame.

The method is by determining the relation of current reference frame and current encoded frame, zooms out respectively or the mode that furthers is carried out upper and lower sampling processing according to camera lens, improved the similarity of current reference frame and coded frame, thereby reached the compression effectiveness of optimization current encoded frame.In the time of low frame per second, the shot change between frame and frame is larger, now adopts frame Forecasting Methodology of the present invention, and the lifting of compression performance will be more remarkable.

Embodiment 2, referring to Fig. 2, the method for the present invention is mainly used in the prediction of P frame, is specially:

Step 201: the relation of determining the first reference frame and current encoded frame; If furthering, camera lens performs step 202; If zooming out, camera lens performs step 203; If do not exist camera lens further or zoom out, the first reference frame is that current reference frame performs step 204;

If (camera lens draws in) enters step 202;

Else if (camera lens pushes away far) enters step 203;

Else curr_ref_i=ref_i, enter 204;

Namely i reference frame ref of the first reference frame described herein_i; Curr_ref_ifor the current reference frame after upgrading;

Step 202: the first reference frame is processed if camera lens furthers, obtained the second reference frame, and the second reference frame is processed, obtain the 3rd reference frame; Current encoded frame is set to the 3rd reference frame;

This step is specially:

Step 2021: to the first reference frame ref_icarry out up-sampling, obtain new reference frame the second reference frame ref_i';

Step 2022: to the second reference frame ref_i' carry out boundary pixel deletion, obtain the 3rd reference frame ref_i"; Thereby make the 3rd reference frame ref_i" with the first reference frame ref_ithere is identical resolution.(the second reference frame has different resolution from the 3rd reference frame, and the 3rd reference frame carries out after boundary pixel is deleted obtaining to the second reference frame herein)

To the second reference frame ref_i' carry out boundary pixel delet method and be:

The 3rd reference frame ref_i" (m, n)=ref_i' (m+d_heifht', n+d_width')

Wherein o_width, o_height are ref_icolumns and line number, m_width', m_height' are ref_i' columns and line number, m, n are line label and the row label of reference frame pixel;

d_width'=(m_width'-o_width)/2，

d_height'=(m_height'-o_height)/2

Step 2023:curr_ref_i=ref_i"

Step 203: if camera lens zooms out, the first reference frame is processed, obtained the 4th reference frame, and the 4th reference frame is processed, obtain the 5th reference frame; And current reference frame is set to the 5th reference frame.

This step implementation method is specially:

Step 2031: to the first reference frame ref_icarry out down-sampling, obtain new reference frame the 4th reference frame ref_i" ';

Step 2032: to the 4th reference frame ref_i" ' carry out boundary pixel is filled expansion, obtains the 5th reference frame ref_i" ", thus make the 5th reference frame ref_i" ' with the first reference frame ref_ithere is identical resolution;

To the 4th reference frame ref_i" ' carry out boundary pixel is filled extended method:

Row are filled:

${{\mathrm{ref}}_i}^{\&prime;\&prime;\&prime;\&prime;}(m,n)=\left\{\begin{array}{cc}{{\mathrm{ref}}_i}^{\&prime;\&prime;\&prime;}(m,0),& 0\&le;n<d\_{\mathrm{width}}^{\&prime;\&prime;\&prime;}\\ {{\mathrm{ref}}_i}^{\&prime;\&prime;\&prime;}(m,n),& d\_{\mathrm{width}}^{\&prime;\&prime;\&prime;}\&le;n<o\_\mathrm{width}-d\_{\mathrm{width}}^{\&prime;\&prime;\&prime;}\\ {{\mathrm{ref}}_i}^{\&prime;\&prime;\&prime;}(m,o\_\mathrm{width}-1),& o\_\mathrm{width}-d\_{\mathrm{width}}^{\&prime;\&prime;\&prime;}\&le;n<o\_\mathrm{width}\end{array}\right.$

Row is filled:

${{\mathrm{ref}}_i}^{\&prime;\&prime;\&prime;\&prime;}(m,n)=\left\{\begin{array}{cc}{{\mathrm{ref}}_i}^{\&prime;\&prime;\&prime;}(0,n),& 0\&le;m<d\_{\mathrm{height}}^{\&prime;\&prime;\&prime;}\\ {{\mathrm{ref}}_i}^{\&prime;\&prime;\&prime;}(m,n),& d\_{\mathrm{height}}^{\&prime;\&prime;\&prime;}\&le;n<o\_\mathrm{height}-d\_{\mathrm{height}}^{\&prime;\&prime;\&prime;}\\ {{\mathrm{ref}}_i}^{\&prime;\&prime;\&prime;}(o\_\mathrm{height}-1,n),& o\_\mathrm{height}-d\_{\mathrm{height}}^{\&prime;\&prime;\&prime;}\&le;n<o\_\mathrm{height}\end{array}\right.$

${{\mathrm{ref}}_i}^{\&prime;\&prime;\&prime;\&prime;}(m,n)=\left\{\begin{array}{cc}{{\mathrm{ref}}_i}^{\&prime;\&prime;\&prime;}(0,n),& 0\&le;m<d\_{\mathrm{height}}^{\&prime;\&prime;\&prime;}\\ {{\mathrm{ref}}_i}^{\&prime;\&prime;\&prime;}(m,n),& d\_{\mathrm{height}}^{\&prime;\&prime;\&prime;}\&le;m<o\_\mathrm{height}-d\_{\mathrm{height}}^{\&prime;\&prime;\&prime;}\\ {{\mathrm{ref}}_i}^{\&prime;\&prime;\&prime;}(o\_\mathrm{height}-1,n),& o\_\mathrm{height}-d\_{\mathrm{height}}^{\&prime;\&prime;\&prime;}\&le;m<o\_\mathrm{height}\end{array}\right.$

Wherein o_width, o_height are ref_icolumns and line number, m_width " ', m_height " ' is ref_i" ' columns and line number, m, n are line label and the row label of reference frame pixel;

d_width″′=(o_width-m_width″′)/2，

d_height″′=(o_height-m_height″′)/2

Step 2033:curr_ref_i=ref_i" "

Step 204: adopt current reference frame to carry out inter prediction to current encoded frame.

The method is by determining the relation of current reference frame and current encoded frame, the mode zooming out according to camera lens respectively or further is carried out upper and lower sampling processing, and further the second reference frame is carried out to pixel deletion, the 4th reference frame has been carried out to pixel-expansion, make current reference frame there is identical resolution with the first reference frame, avoid redistributing of internal memory, be convenient to the compatibility of code.Thereby further improve the similarity of current reference frame and coded frame, reached the compression effectiveness of optimizing current encoded frame.In the time of low frame per second, the shot change between frame and frame is larger, now adopts frame Forecasting Methodology of the present invention, and the lifting of compression performance will be more remarkable.

Embodiment 3, the present invention also provides a kind of inter prediction device of P frame corresponding to embodiment 1, and referring to Fig. 3, this device comprises:

Judging unit 301, for determining the relation of the first reference frame and current encoded frame;

The camera lens unit 302 that furthers, for the first reference frame being processed, obtain the second reference frame in the time that camera lens furthers, and current reference frame is set to the second reference frame;

Camera lens extension unit 303, for the first reference frame being processed, obtain the 4th reference frame in the time zooming out for camera lens, and current reference frame is set to the 4th reference frame;

Predicting unit 304, for adopting current reference frame to carry out inter prediction to current encoded frame.

Wherein camera lens furthers unit for the first reference frame is processed, and obtains the second reference frame and is specially: the first reference frame is carried out to up-sampling, obtain the second reference frame.

Corresponding to embodiment 2, the described camera lens unit that furthers is further used for obtaining, after described the second reference frame, further the second reference frame being processed, and obtains the 3rd reference frame, and current reference frame is set to the 3rd reference frame.

Wherein camera lens furthers unit for the second reference frame is processed, and obtains the 3rd reference frame and is specially:

The second reference frame is carried out to boundary pixel deletion, obtain the 3rd reference frame, make the 3rd reference frame there is identical resolution with the first reference frame;

Wherein, describedly the second reference frame carried out to boundary pixel delet method be specially:

ref_i″(m,n)=ref_i'(m+d_heifht',n+d_width')

Wherein ref_i' be the second reference frame, ref_i' ' be the 3rd reference frame, o_width, o_height are ref_ithe columns of current reference frame and line number, m_width', m_height' are ref_i' columns and line number, m, n are line label and the row label of reference frame pixel;

d_width'=(m_width'-o_width)/2，

d_height'=(m_height'-o_height)/2

Wherein, described camera lens extension unit, for the first reference frame is processed, obtains the 4th reference frame and is specially: the first reference frame is carried out to down-sampling, obtain the 4th reference frame.

Corresponding to embodiment 2, described camera lens extension unit is further used for obtaining, after the 4th described reference frame, further the 4th reference frame being processed, and obtains the 5th reference frame; Accordingly, current reference frame is set to the 5th reference frame;

Wherein, described camera lens extension unit is for processing the 4th reference frame, obtaining the 5th reference frame is specially: the 4th reference frame is carried out to boundary pixel and fill expansion, obtain the 5th reference frame, make the 5th reference frame have identical resolution with the first reference frame;

This is specially wherein the 4th reference frame to be carried out to boundary pixel filling expansion:

Row are filled:

${{\mathrm{ref}}_i}^{\&prime;\&prime;\&prime;\&prime;}(m,n)=\left\{\begin{array}{cc}{{\mathrm{ref}}_i}^{\&prime;\&prime;\&prime;}(m,0),& 0\&le;n<d\_{\mathrm{width}}^{\&prime;\&prime;\&prime;}\\ {{\mathrm{ref}}_i}^{\&prime;\&prime;\&prime;}(m,n),& d\_{\mathrm{width}}^{\&prime;\&prime;\&prime;}\&le;n<o\_\mathrm{width}-d\_{\mathrm{width}}^{\&prime;\&prime;\&prime;}\\ {{\mathrm{ref}}_i}^{\&prime;\&prime;\&prime;}(m,o\_\mathrm{width}-1),& o\_\mathrm{width}-d\_{\mathrm{width}}^{\&prime;\&prime;\&prime;}\&le;n<o\_\mathrm{width}\end{array}\right.$

Row is filled:

${{\mathrm{ref}}_i}^{\&prime;\&prime;\&prime;\&prime;}(m,n)=\left\{\begin{array}{cc}{{\mathrm{ref}}_i}^{\&prime;\&prime;\&prime;}(0,n),& 0\&le;m<d\_{\mathrm{height}}^{\&prime;\&prime;\&prime;}\\ {{\mathrm{ref}}_i}^{\&prime;\&prime;\&prime;}(m,n),& d\_{\mathrm{height}}^{\&prime;\&prime;\&prime;}\&le;m<o\_\mathrm{height}-d\_{\mathrm{height}}^{\&prime;\&prime;\&prime;}\\ {{\mathrm{ref}}_i}^{\&prime;\&prime;\&prime;}(o\_\mathrm{height}-1,n),& o\_\mathrm{height}-d\_{\mathrm{height}}^{\&prime;\&prime;\&prime;}\&le;m<o\_\mathrm{height}\end{array}\right.$

Wherein ref_i" ' be the 4th reference frame, o_width, o_height are ref_ithe columns of current reference frame and line number, m_width " ', m_height " ' is ref_i" columns of the ' the four reference frame and line number, m, n are line label and the row label of reference frame pixel;

d_width″′=(o_width-m_width″′)/2，

d_height″′=(o_height-m_height″′)/2

Those having ordinary skill in the art will appreciate that, the all or part of step realizing in above-described embodiment method can complete by program command related hardware, described program can be stored in a computer read/write memory medium, and described storage medium can be ROM, RAM, disk, CD etc.

The foregoing is only preferred embodiment of the present invention, not in order to limit the present invention, all any amendments of doing within the spirit and principles in the present invention, be equal to and replace and improvement etc., within all should being included in protection scope of the present invention.

Claims (14)

1. an inter-frame prediction method, is characterized in that, described method comprises:

Step 1: the relation of determining the first reference frame and current encoded frame;

Step 2: the first reference frame is processed if camera lens furthers, obtained the second reference frame, and current reference frame is set to the second reference frame; Enter step 3;

If camera lens zooms out the first reference frame is processed, obtain the 4th reference frame, and current reference frame is set to the 4th reference frame; Enter step 3;

If current reference frame is the first reference frame, enter step 3;

Step 3: adopt current reference frame to carry out inter prediction to current encoded frame;

The first reference frame is processed if described camera lens furthers, is obtained the second reference frame and be specially:

The first reference frame is carried out to up-sampling, obtain the second reference frame.

2. inter-frame prediction method according to claim 1, is characterized in that, obtains, after described the second reference frame, further the second reference frame being processed, and obtains the 3rd reference frame;

Accordingly, current reference frame is set to the 3rd reference frame, enters step 3.

3. inter-frame prediction method according to claim 2, is characterized in that, described further processes the second reference frame, obtains the 3rd reference frame and is specially:

The second reference frame is carried out to boundary pixel deletion, obtain the 3rd reference frame, make the 3rd reference frame there is identical resolution with the first reference frame.

4. inter-frame prediction method according to claim 3, is characterized in that, described the second reference frame is carried out to boundary pixel delet method be specially:

ref_i″(m,n)=ref_i'(m+d_heifht',n+d_width')

Wherein ref_i' be the second reference frame, ref_i" be the 3rd reference frame, o_width, o_height are ref_ithe columns of current reference frame and line number, m_width', m_height' are ref_i' columns and line number, m, n are line label and the row label of reference frame pixel;

d_width'=(m_width'-o_width)/2，

d_height'=(m_height'-o_height)/2。

5. inter-frame prediction method according to claim 1, is characterized in that, described processes the first reference frame, obtains the 4th reference frame and is specially:

The first reference frame is carried out to down-sampling, obtain the 4th reference frame.

6. inter-frame prediction method according to claim 1, is characterized in that, obtains, after the 4th described reference frame, further the 4th reference frame being processed, and obtains the 5th reference frame;

Accordingly, current reference frame is set to the 5th reference frame, enters step 3.

7. inter-frame prediction method according to claim 6, is characterized in that, described processes the 4th reference frame, obtains the 5th reference frame and is specially:

The 4th reference frame is carried out to boundary pixel and fill expansion, obtain the 5th reference frame, make the 5th reference frame there is identical resolution with the first reference frame.

8. inter-frame prediction method according to claim 7, is characterized in that, described the 4th reference frame is carried out to boundary pixel fill expansion and is specially:

Row are filled:

${{\mathrm{ref}}_i}^{\&prime;\&prime;\&prime;\&prime;}(m,n)=\left\{\begin{array}{cc}{{\mathrm{ref}}_i}^{\&prime;\&prime;\&prime;}(m,0),& 0\&le;n<d\_{\mathrm{width}}^{\&prime;\&prime;\&prime;}\\ {{\mathrm{ref}}_i}^{\&prime;\&prime;\&prime;}(m,n),& d\_{\mathrm{width}}^{\&prime;\&prime;\&prime;}\&le;n<o\_\mathrm{width}-d\_{\mathrm{width}}^{\&prime;\&prime;\&prime;}\\ {{\mathrm{ref}}_i}^{\&prime;\&prime;\&prime;}(m,o\_\mathrm{width}-1),& o\_\mathrm{width}-d\_{\mathrm{width}}^{\&prime;\&prime;\&prime;}\&le;n<o\_\mathrm{width}\end{array}\right.$

Row is filled:

${{\mathrm{ref}}_i}^{\&prime;\&prime;\&prime;\&prime;}(m,n)=\left\{\begin{array}{cc}{{\mathrm{ref}}_i}^{\&prime;\&prime;\&prime;}(0,n),& 0\&le;m<d\_{\mathrm{height}}^{\&prime;\&prime;\&prime;}\\ {{\mathrm{ref}}_i}^{\&prime;\&prime;\&prime;}(m,n),& d\_{\mathrm{height}}^{\&prime;\&prime;\&prime;}\&le;m<o\_\mathrm{height}-d\_{\mathrm{height}}^{\&prime;\&prime;\&prime;}\\ {{\mathrm{ref}}_i}^{\&prime;\&prime;\&prime;}(o\_\mathrm{height}-1,n),& o\_\mathrm{height}-d\_{\mathrm{height}}^{\&prime;\&prime;\&prime;}\&le;m<o\_\mathrm{height}\end{array}\right.$

Wherein ref_i" ' be the 4th reference frame, o_width, o_height are ref_ithe columns of current reference frame and line number, m_width " ', m_height " ' is ref_i" columns of the ' the four reference frame and line number, m, n are line label and the row label of reference frame pixel;

d_width″′=(o_width-m_width″′)/2，

d_height″′=(o_height-m_height″′)/2。

9. an inter prediction device, is characterized in that, this device comprises:

Judging unit, for determining the relation of the first reference frame and current encoded frame;

The camera lens unit that furthers, for the first reference frame being processed, obtain the second reference frame in the time that camera lens furthers, and current reference frame is set to the second reference frame;

Camera lens extension unit, for the first reference frame being processed, obtain the 4th reference frame in the time zooming out for camera lens, and current reference frame is set to the 4th reference frame;

Predicting unit, for adopting current reference frame to carry out inter prediction to current encoded frame;

Described camera lens furthers unit for the first reference frame is processed, and obtains the second reference frame and is specially: the first reference frame is carried out to up-sampling, obtain the second reference frame.

10. inter prediction device according to claim 9, it is characterized in that, the described camera lens unit that furthers is further used for obtaining, after described the second reference frame, further the second reference frame being processed, obtain the 3rd reference frame, current reference frame is set to the 3rd reference frame.

11. inter prediction devices according to claim 10, is characterized in that, described camera lens furthers unit for the second reference frame is processed, and obtain the 3rd reference frame and are specially:

The second reference frame is carried out to boundary pixel deletion, obtain the 3rd reference frame, make the 3rd reference frame there is identical resolution with the first reference frame;

Wherein, describedly the second reference frame carried out to boundary pixel delet method be specially:

ref_i″(m,n)=ref_i'(m+d_heifht',n+d_width')

Wherein ref_i' be the second reference frame, ref_i" be the 3rd reference frame, o_width, o_height are ref_ithe columns of current reference frame and line number, m_width', m_height' are ref_i' columns and line number, m, n are line label and the row label of reference frame pixel;

d_width'=(m_width'-o_width)/2，

d_height'=(m_height'-o_height)/2。

12. inter prediction devices according to claim 9, is characterized in that, described camera lens extension unit, for the first reference frame is processed, obtains the 4th reference frame and is specially: the first reference frame is carried out to down-sampling, obtain the 4th reference frame.

13. inter prediction devices according to claim 9, is characterized in that, described camera lens extension unit is further used for obtaining, after the 4th described reference frame, further the 4th reference frame being processed, and obtains the 5th reference frame; Accordingly, current reference frame is set to the 5th reference frame.

14. inter prediction devices according to claim 13, it is characterized in that, described camera lens extension unit is for processing the 4th reference frame, obtaining the 5th reference frame is specially: the 4th reference frame is carried out to boundary pixel and fill expansion, obtain the 5th reference frame, make the 5th reference frame there is identical resolution with the first reference frame;

Wherein the 4th reference frame being carried out to boundary pixel filling expansion is specially:

Row are filled:

${{\mathrm{ref}}_i}^{\&prime;\&prime;\&prime;\&prime;}(m,n)=\left\{\begin{array}{cc}{{\mathrm{ref}}_i}^{\&prime;\&prime;\&prime;}(m,0),& 0\&le;n<d\_{\mathrm{width}}^{\&prime;\&prime;\&prime;}\\ {{\mathrm{ref}}_i}^{\&prime;\&prime;\&prime;}(m,n),& d\_{\mathrm{width}}^{\&prime;\&prime;\&prime;}\&le;n<o\_\mathrm{width}-d\_{\mathrm{width}}^{\&prime;\&prime;\&prime;}\\ {{\mathrm{ref}}_i}^{\&prime;\&prime;\&prime;}(m,o\_\mathrm{width}-1),& o\_\mathrm{width}-d\_{\mathrm{width}}^{\&prime;\&prime;\&prime;}\&le;n<o\_\mathrm{width}\end{array}\right.$

Row is filled:

${{\mathrm{ref}}_i}^{\&prime;\&prime;\&prime;\&prime;}(m,n)=\left\{\begin{array}{cc}{{\mathrm{ref}}_i}^{\&prime;\&prime;\&prime;}(0,n),& 0\&le;m<d\_{\mathrm{height}}^{\&prime;\&prime;\&prime;}\\ {{\mathrm{ref}}_i}^{\&prime;\&prime;\&prime;}(m,n),& d\_{\mathrm{height}}^{\&prime;\&prime;\&prime;}\&le;m<o\_\mathrm{height}-d\_{\mathrm{height}}^{\&prime;\&prime;\&prime;}\\ {{\mathrm{ref}}_i}^{\&prime;\&prime;\&prime;}(o\_\mathrm{height}-1,n),& o\_\mathrm{height}-d\_{\mathrm{height}}^{\&prime;\&prime;\&prime;}\&le;m<o\_\mathrm{height}\end{array}\right.$

Wherein ref_i" ' be the 4th reference frame, o_width, o_height are ref_ithe columns of current reference frame and line number, m_width " ', m_height " ' is ref_i" columns of the ' the four reference frame and line number, m, n are line label and the row label of reference frame pixel;

d_width″′=(o_width-m_width″′)/2，

d_height″′=(o_height-m_height″′)/2。