技术领域technical field
本发明属于图像处理技术领域，涉及视频图像隔行扫描格式至逐行扫描格式的转换，可用于隔行扫描图像至逐行扫描图像的去隔行处理，是近年来视频图像去隔行的研究热点之一。The invention belongs to the technical field of image processing, relates to the conversion of video image from interlaced scanning format to progressive scanning format, can be used for deinterlacing processing from interlaced scanning image to progressive scanning image, and is one of the research hotspots of video image deinterlacing in recent years.
背景技术Background technique
目前随着各种新型显示设备、新的电视广播格式的发展，在视频领域存在大量不同的格式标准。为了实现不同格式视频信号间的交流，视频格式转变不可或缺。当今电视广播系统中，绝大部分视频信号源采用隔行扫描，此方式可以减少带宽需求，但会引起爬行、画面闪烁、边缘模糊及锯齿现象。去隔行技术是格式转换的一项关键技术，也是其他格式转换技术的基础。所谓“去隔行”，即从隔行扫描格式向逐行扫描格式的转换。With the development of various new display devices and new TV broadcast formats, there are a large number of different format standards in the video field. In order to realize the exchange between video signals of different formats, video format conversion is indispensable. In today's TV broadcasting systems, most video signal sources use interlacing, which can reduce bandwidth requirements, but can cause crawling, flickering, blurred edges, and jagged edges. De-interlacing technology is a key technology of format conversion and the basis of other format conversion technologies. The so-called "de-interlacing" refers to the conversion from an interlaced scanning format to a progressive scanning format.
为了克服隔行图像转换为逐行图像时带来的模糊、锯齿等现象，人们提出了许多不同的去隔行方法，按照去隔行技术发展的时间顺序，依次为：线性去隔行方法，包括空域线性滤波、时域线性滤波、时空域线性滤波；非线性去隔行方法，包括基于边缘自适应的去隔行方法、基于中值滤波的去隔行方法、基于运动自适应的去隔行方法；基于运动补偿的去隔行方法，为目前最先进的去隔行方法。In order to overcome the blurring, jaggedness and other phenomena brought about when interlaced images are converted to progressive images, many different deinterlacing methods have been proposed. According to the chronological order of deinterlacing technology development, they are: linear deinterlacing methods, including spatial linear filtering , time-domain linear filtering, time-space linear filtering; non-linear de-interlacing methods, including edge-adaptive de-interlacing methods, median filtering-based de-interlacing methods, motion-adaptive de-interlacing methods; motion compensation-based de-interlacing methods The interlacing method is the most advanced deinterlacing method at present.
运动补偿去隔行方法有两个重要的过程：运动估计与运动补偿。对于视频图像序列，相邻帧与帧之间存在着较强的相关性，运动估计就是对多帧之间的运动信息的估计。运动补偿是根据运动估计的结果做插值运算。运动估计和运动补偿被广泛应用于数字视频处理中。例如H.261，H.263，H.264和MPEG-1，MPEG-2，MPEG-4压缩标准中，都用到了运动估计与运动补偿。在去隔行视频处理中，运动估计应用前后场的运动信息，得到运动矢量MV，然后沿着运动轨迹进行插值来重构帧。一般来说，如果得到的运动矢量足够准确，运动补偿方法将得到最好的去隔行效果。这种去隔行方法能够很好地保持图像的垂直清晰度。There are two important processes in the motion compensation deinterlacing method: motion estimation and motion compensation. For video image sequences, there is a strong correlation between adjacent frames, and motion estimation is the estimation of motion information between multiple frames. Motion compensation is an interpolation operation based on the result of motion estimation. Motion estimation and motion compensation are widely used in digital video processing. For example, H.261, H.263, H.264, MPEG-1, MPEG-2, and MPEG-4 compression standards all use motion estimation and motion compensation. In deinterlacing video processing, motion estimation applies the motion information of the front and rear fields to obtain the motion vector MV, and then interpolates along the motion trajectory to reconstruct the frame. Generally speaking, the motion compensation method will get the best deinterlacing effect if the obtained motion vectors are accurate enough. This deinterlacing method does a good job of maintaining the vertical sharpness of the image.
运动估计是运动补偿方法的基础，也是运动补偿去隔行方法最重要的一步。目前已经有多种运动估计方法，常用的有块匹配法，贝叶斯运动估计等。块匹配法是最常用的一种运动估计方法，其基本思想是将图像划分成N×N个固定大小、互不重叠的图像块，并假设子块内所有像素均具有相同的运动参数，即每个子块的运动为刚体平移，对每块分别进行处理。当前场的N×N个子块在参考场对应的子块邻域窗口内根据一定的匹配准则搜索到与之最匹配的子块，当前子块与匹配块在二维平面上的位移即为运动估计得到的运动矢量。块匹配运动估计的优势在于计算简单、计算量小和硬件实现简单，精度与复杂度方面有较好的折衷。Motion estimation is the basis of the motion compensation method, and it is also the most important step of the motion compensation deinterlacing method. At present, there are many motion estimation methods, commonly used are block matching method, Bayesian motion estimation and so on. The block matching method is the most commonly used motion estimation method. Its basic idea is to divide the image into N×N fixed-size, non-overlapping image blocks, and assume that all pixels in the sub-blocks have the same motion parameters, that is, The motion of each sub-block is rigid body translation, and each block is processed separately. The N×N sub-blocks of the current field search for the most matching sub-block according to a certain matching criterion in the sub-block neighborhood window corresponding to the reference field, and the displacement between the current sub-block and the matching block on the two-dimensional plane is the motion Estimated motion vector. The advantage of block matching motion estimation lies in its simple calculation, small amount of calculation, simple hardware implementation, and a good compromise between accuracy and complexity.
运动补偿是根据运动估计得到的运动矢量所描述的运动轨迹进行图像插值，这样就相当于将一个运动序列虚拟地变成一个静止序列，因此也就可以利用静止图像插值的方法处理运动图像。常用的运动补偿有前向运动补偿、后向运动补偿、双向运动补偿和多假设运动补偿等方法。Motion compensation is to perform image interpolation according to the motion trajectory described by the motion vector obtained by motion estimation, which is equivalent to turning a motion sequence into a static sequence virtually, so the still image interpolation method can be used to process moving images. Commonly used motion compensation methods include forward motion compensation, backward motion compensation, two-way motion compensation and multi-hypothesis motion compensation.
目前，基于运动估计和运动补偿的去隔行方法主要由整像素运动估计、运动补偿构成。其中运动估计采用整像素块匹配快速运动估计方法，运动补偿采用的是双向运动补偿方式，这种方法的不足之处是：整像素运动估计搜索范围内的运动矢量位移都是整像素级别的，如果视频序列中物体存在分数像素级运动，现有的整像素运动估计搜索精度不足，无法准确得到真实运动的运动矢量，只能在整像素级搜索范围内寻找相对最优的匹配块，从而导致明显的块效应，对隔行视频图像去隔行处理效果不佳。At present, the de-interlacing method based on motion estimation and motion compensation mainly consists of integer pixel motion estimation and motion compensation. Among them, the motion estimation adopts the whole-pixel block matching fast motion estimation method, and the motion compensation adopts the two-way motion compensation method. The disadvantage of this method is that the motion vector displacement within the search range of the whole-pixel motion estimation is at the whole-pixel level, If the object in the video sequence has fractional pixel-level motion, the existing integer-pixel motion estimation search accuracy is insufficient, and the motion vector of the real motion cannot be accurately obtained, and the relatively optimal matching block can only be found within the integer-pixel-level search range, resulting in Obvious block effect, poor deinterlacing effect on interlaced video images.
发明内容Contents of the invention
本发明的目的在于针对上述现有技术的不足，提出基于运动估计与运动补偿的1/4像素精度块匹配图像去隔行方法，以提高块匹配运动估计的搜索准确度，提高隔行视频图像去隔行处理效果。The purpose of the present invention is to address the above-mentioned deficiencies in the prior art, and propose a 1/4 pixel precision block matching image deinterlacing method based on motion estimation and motion compensation, to improve the search accuracy of block matching motion estimation, and to improve the deinterlacing of interlaced video images processing effect.
为实现上述目的，本发明技术方案包括如下步骤：To achieve the above object, the technical solution of the present invention comprises the following steps:
(1)读入一个大小为352×288的raw格式的100场的实际隔行视频序列，选取一个插值场和两个参考场，其中选取的参考场为插值场的前向场和后向场；(1) read in an actual interlaced video sequence of 100 fields in raw format with a size of 352 × 288, select an interpolation field and two reference fields, wherein the reference field selected is the forward field and the backward field of the interpolation field;
(2)在前向和后向参考场的整像素点之间产生半像素点：(2) Generate half-pixels between the integer pixels of the forward and backward reference fields:
应用相邻整像素点进行内插得到半像素点，其中内插方式为应用一个六阶有限冲击响应滤波器对相邻整像素点进行加权，滤波器权值为：Apply adjacent integer pixels to interpolate to obtain half pixels, where the interpolation method is to apply a sixth-order finite impulse response filter to weight adjacent integer pixels, and the filter weight is:
(1/32,-5/32,5/8,5/8,-5/32,1/32)；(1/32,-5/32,5/8,5/8,-5/32,1/32);
(3)在得到所有的半像素位置像素点后，由邻近整像素点和半像素点进行线性插补，得到1/4像素位置的像素点；(3) After obtaining all half-pixel position pixels, linear interpolation is performed by adjacent integer pixel points and half-pixel points to obtain the pixel points at 1/4 pixel positions;
(4)运用有效的三步搜索法E3SS对参考场整像素、1/2像素、1/4像素位置进行块匹配运动估计，寻找绝对误差和SAD最小的最佳匹配块，当位移矢量为(i,j)时，绝对误差和SAD定义如下：(4) Use the effective three-step search method E3SS to perform block matching motion estimation on the reference field full pixel, 1/2 pixel, and 1/4 pixel positions, and find the best matching block with the smallest absolute error and SAD. When the displacement vector is ( i, j), the absolute error and SAD are defined as follows:
式中，fk和fk-1分别是当前场和前向场对应像素点的灰度值，M×N为宏块大小；In the formula, fk and fk-1 are the gray values of corresponding pixels in the current field and the forward field respectively, and M×N is the size of the macroblock;
(5)对前向和后向参考场进行双向运动估计，得到两个运动矢量(dx1,dy1)、(dx2,dy2)，取其中绝对误差和SAD最小的一个运动矢量作为当前宏块的运动矢量(dx,dy)；(5) Perform two-way motion estimation on the forward and backward reference fields to obtain two motion vectors (dx1 , dy1 ), (dx2 , dy2 ), and take the motion vector with the smallest absolute error and SAD as the current The motion vector (dx, dy) of the macroblock;
(6)根据当前宏块的运动矢量(dx,dy)，应用前向和后向参考场中相应的参考宏块，线性插值得到插值宏块F(x,y;k)：(6) According to the motion vector (dx, dy) of the current macroblock, apply the corresponding reference macroblocks in the forward and backward reference fields, and linearly interpolate to obtain the interpolated macroblock F(x, y; k):
其中，(x,y)为宏块的坐标，F(x-dx,y-dy;k-1)为前向参考场应用运动矢量(dx,dy)得到的参考宏块，F(x+dx,y+dy;k+1)为后向参考场应用运动矢量(dx,dy)得到的参考宏块；Among them, (x,y) is the coordinates of the macroblock, F(x-dx,y-dy;k-1) is the reference macroblock obtained by applying the motion vector (dx,dy) to the forward reference field, F(x+ dx, y+dy; k+1) is the reference macroblock obtained by applying the motion vector (dx, dy) to the backward reference field;
(7)合并插值宏块F(x,y;k)与当前宏块，得到去隔行后的逐行图像宏块。(7) Merging the interpolation macroblock F(x, y; k) and the current macroblock to obtain a deinterlaced progressive image macroblock.
本发明由于采用邻近整像素点和半像素点进行线性插补，得到1/4像素位置的像素点，故能有效地提高运动估计搜索精度；同时由于本发明采用六阶有限响应滤波器对整像素点进行插值，得到插值效果更优的半像素点，能有效消除图像噪声引起的像素点突变对插值效果的影响；此外由于本发明对整数、分数像素位置运用有效的三步搜索法E3SS进行块匹配运动估计搜索，因而降低了计算复杂度，并且对运动剧烈图像差值效果好，能在保证插值计算复杂度适当的前提下，有效地提高了插值后的图像视觉效果。The present invention can effectively improve the search precision of motion estimation because the present invention uses adjacent integer pixels and half-pixels for linear interpolation to obtain pixel points at 1/4 pixel positions; Pixels are interpolated to obtain half-pixels with better interpolation effects, which can effectively eliminate the impact of pixel mutations caused by image noise on interpolation effects; in addition, because the present invention uses an effective three-step search method E3SS for integer and fractional pixel positions Block matching motion estimation search, thus reducing the computational complexity, and has a good effect on the difference value of violently moving images, and can effectively improve the visual effect of the interpolated image under the premise of ensuring that the interpolation calculation complexity is appropriate.
附图说明Description of drawings
图1为本发明的流程图；Fig. 1 is a flowchart of the present invention;
图2为本发明使用的测试视频图像；Fig. 2 is the test video image that the present invention uses;
图3为本发明应用的一种有效的三步搜索法E3SS的搜索模板；Fig. 3 is the search template of a kind of effective three-step search method E3SS that the present invention applies;
图4为本发明与现有整像素运动补偿去隔行法对football序列进行去隔行处理的实验对比图；Fig. 4 is the experimental contrast figure that the present invention and existing integer pixel motion compensation deinterlacing method carry out deinterlacing process to football sequence;
图5为本发明与现有整像素运动补偿去隔行法对soccer序列进行去隔行处理的实验对比图。FIG. 5 is an experimental comparison diagram of deinterlacing processing on soccer sequences between the present invention and the existing integer pixel motion compensation deinterlacing method.
具体实施方式detailed description
下面结合附图对本发明做进一步的详细描述。The present invention will be described in further detail below in conjunction with the accompanying drawings.
参照图1，本发明的实施步骤如下：With reference to Fig. 1, the implementation steps of the present invention are as follows:
步骤1,选取插值场和参考场：Step 1, select interpolation field and reference field:
读入一个大小为352×288的raw格式的100场的实际隔行视频序列，插值场按照时间顺序从起始场到结束场依次选取，选取一个场作为插值场后，两个参考场分别为选取的插值场的前向场和后向场。Read in an actual interlaced video sequence of 100 fields in raw format with a size of 352×288. The interpolation fields are selected sequentially from the start field to the end field in time order. After one field is selected as the interpolation field, two reference fields are selected respectively. The forward and backward fields of the interpolated field.
步骤2，在前向和后向参考场的整像素点之间产生半像素点。In step 2, half pixels are generated between the integer pixels of the forward and backward reference fields.
根据待插值半像素点与相邻整像素点距离的不同，应用不同的像素点插值得到半像素点，其实现步骤如下：According to the difference between the half pixel point to be interpolated and the adjacent integer pixel point, apply different pixel point interpolation to obtain the half pixel point, and the implementation steps are as follows:
(2a)对于临近水平方向整数位置的半像素点，是通过应用一个六阶有限冲击响应滤波器对左右邻近整像素点进行加权，内插得到插值半像素点，其中滤波器权值为：(1/32,-5/32,5/8,5/8,-5/32,1/32)；(2a) For the half-pixel points adjacent to the integer position in the horizontal direction, the left and right adjacent integer pixels are weighted by applying a sixth-order finite impulse response filter, and the interpolated half-pixel points are obtained by interpolation, where the filter weight is: ( 1/32,-5/32,5/8,5/8,-5/32,1/32);
(2b)对于临近垂直方向整数位置的半像素点，是通过应用一个六阶有限冲击响应滤波器对上下邻近的整像素点进行加权，内插得到插值半像素点，其中滤波器权值为：(1/32,-5/32,5/8,5/8,-5/32,1/32)；(2b) For the half-pixel points adjacent to the integer position in the vertical direction, the upper and lower adjacent integer pixels are weighted by applying a sixth-order finite impulse response filter, and the interpolated half-pixel points are obtained by interpolation, where the filter weight is: (1/32,-5/32,5/8,5/8,-5/32,1/32);
(2c)对于余下的半像素位置的像素点，是通过应用一个六阶有限冲击响应滤波器对六个水平或者垂直方向已知的邻近半像素点进行加权，内插得到插值半像素点，其中滤波器权值为：(1/32,-5/32,5/8,5/8,-5/32,1/32)。(2c) For the pixels at the remaining half-pixel positions, weight the six adjacent half-pixels known in the horizontal or vertical direction by applying a sixth-order finite impulse response filter, and interpolate to obtain interpolated half-pixels, where The filter weights are: (1/32,-5/32,5/8,5/8,-5/32,1/32).
步骤3，在参考场中得到所有的半像素位置像素点后，根据待插值1/4像素点与相邻整像素点、半像素点距离的不同，应用不同的整像素点和半像素点插值得到1/4像素点，其实现步骤如下：Step 3: After obtaining all half-pixel position pixels in the reference field, apply different integer and half-pixel interpolation according to the difference between the 1/4 pixel to be interpolated and the adjacent integer and half-pixel distances To get 1/4 pixel, the implementation steps are as follows:
(3a)对于临近水平方向整像素和半像素位置的四分之一像素，是由左右邻近的整像素点和半像素点进行线性插补得到；(3a) For the quarter pixel adjacent to the integer pixel and half pixel position in the horizontal direction, it is obtained by performing linear interpolation on the left and right adjacent integer pixels and half pixels;
(3b)对于临近垂直方向整像素和半像素位置的四分之一像素，是由上下邻近的整像素点和半像素点进行线性插补得到；(3b) For the quarter pixel adjacent to the integer pixel and half pixel position in the vertical direction, it is obtained by performing linear interpolation on the upper and lower adjacent integer pixels and half pixels;
(3c)对于余下的四分之一像素位置的像素，是用相邻对角线位置的两个半像素点进行线性插补得到。(3c) For the pixels at the remaining quarter-pixel positions, it is obtained by performing linear interpolation with two half-pixel points at adjacent diagonal positions.
步骤4，，运用有效的三步搜索法E3SS对参考场的整像素、1/2像素、1/4像素位置进行块匹配运动估计。Step 4, use the effective three-step search method E3SS to perform block matching motion estimation on the integer pixel, 1/2 pixel, and 1/4 pixel positions of the reference field.
参照图3，本步骤的具体实现如下：Referring to Figure 3, the specific implementation of this step is as follows:
(4a)对整像素位置，搜索该E3SS模板上的13个检测点，如果最小误差点为搜索窗口中心检测点，则该最小误差点对应的参考宏块为最佳匹配参考宏块，搜索结束，得到一个最佳匹配参考宏块对应的运动矢量(iint,jint)；如果最小误差点处于13个检测点的外8个检测点，采用三步搜索法3SS继续进行块匹配运动估计搜索；如果最小误差点处于13个检测点的内菱形的4个检测点，将此时的最小误差点作为下一步搜索内菱形的4个检测点的中心点，继续搜索对应的内菱形4个检测点，直到最小误差点为内菱形4个检测点的中心点或搜索到达搜索窗口边界位置；结束搜索后，得到最佳匹配参考宏块对应的运动矢量(iint,jint)；(4a) For the whole pixel position, search for 13 detection points on the E3SS template, if the minimum error point is the detection point at the center of the search window, then the reference macroblock corresponding to the minimum error point is the best matching reference macroblock, and the search ends , to obtain a motion vector (iint ,jint ) corresponding to the best matching reference macroblock; if the minimum error point is in the 8 detection points out of the 13 detection points, use the three-step search method 3SS to continue the block matching motion estimation search ; If the minimum error point is in the 4 detection points of the inner rhombus with 13 detection points, use the minimum error point at this time as the center point of the 4 detection points of the inner rhombus in the next step, and continue to search for the corresponding 4 detection points of the inner rhombus point until the minimum error point is the center point of the 4 detection points of the inner diamond or the search reaches the boundary position of the search window; after the search is completed, the motion vector (iint , jint ) corresponding to the best matching reference macroblock is obtained;
(4b)对1/2像素位置，搜索该E3SS模板上的13个检测点，如果最小误差点为搜索窗口中心检测点，则该最小误差点对应的参考宏块为最佳匹配参考宏块，搜索结束，得到一个最佳匹配参考宏块对应的运动矢量(i1/2,j1/2)；如果最小误差点处于13个检测点的外8个检测点，采用三步搜索法3SS继续进行块匹配运动估计搜索；如果最小误差点处于13个检测点的内菱形的4个检测点，将此时的最小误差点作为下一步搜索内菱形的4个检测点的中心点，继续搜索对应的内菱形4个检测点，直到最小误差点为内菱形4个检测点的中心点或搜索到达搜索窗口边界位置；结束搜索后，得到最佳匹配参考宏块对应的运动矢量(i1/2,j1/2)；(4b) For the 1/2 pixel position, search for 13 detection points on the E3SS template, if the minimum error point is the detection point at the center of the search window, then the reference macroblock corresponding to the minimum error point is the best matching reference macroblock, After the search is over, a motion vector (i1/2 , j1/2 ) corresponding to the best matching reference macroblock is obtained; if the minimum error point is at the 8 detection points out of the 13 detection points, use the three-step search method 3SS to continue Carry out block matching motion estimation search; if the minimum error point is in the 4 detection points of the inner rhombus of 13 detection points, use the minimum error point at this time as the center point of the 4 detection points of the inner rhombus in the next search, and continue to search for the corresponding The 4 detection points of the inner rhombus until the minimum error point is the center point of the 4 detection points of the inner rhombus or the search reaches the boundary position of the search window; after the search is completed, the motion vector corresponding to the best matching reference macroblock (i1/2 ,j1/2 );
(4c)对1/4像素位置，搜索该E3SS模板上的13个检测点，如果最小误差点为搜索窗口中心检测点，则该最小误差点对应的参考宏块为最佳匹配参考宏块，搜索结束，得到一个最佳匹配参考宏块对应的运动矢量(i1/4,j1/4)；如果最小误差点处于13个检测点的外8个检测点，采用三步搜索法3SS继续进行块匹配运动估计搜索；如果最小误差点处于13个检测点的内菱形的4个检测点，将此时的最小误差点作为下一步搜索内菱形的4个检测点的中心点，继续搜索对应的内菱形4个检测点，直到最小误差点为内菱形4个检测点的中心点或搜索到达搜索窗口边界位置；结束搜索后，得到最佳匹配参考宏块对应的运动矢量(i1/4,j1/4)；(4c) For the 1/4 pixel position, search for 13 detection points on the E3SS template, if the minimum error point is the detection point at the center of the search window, then the reference macroblock corresponding to the minimum error point is the best matching reference macroblock, After the search is over, a motion vector (i1/4 , j1/4 ) corresponding to the best matching reference macroblock is obtained; if the minimum error point is in the outer 8 detection points of the 13 detection points, use the three-step search method 3SS to continue Carry out block matching motion estimation search; if the minimum error point is in the 4 detection points of the inner rhombus of 13 detection points, use the minimum error point at this time as the center point of the 4 detection points of the inner rhombus in the next search, and continue to search for the corresponding The 4 detection points of the inner rhombus until the minimum error point is the center point of the 4 detection points of the inner rhombus or the search reaches the boundary position of the search window; after the search is completed, the motion vector corresponding to the best matching reference macroblock (i1/4 ,j1/4 );
(4d)从整像素、1/2像素、1/4像素位置搜索得到的运动矢量(iint,jint)、(i1/2,j1/2)、(i1/4,j1/4)中，选取绝对误差和SAD最小的运动矢量，作为应用有效的三步搜索法E3SS得到的最佳运动矢量(i,j)。(4d) Motion vectors (iint , jint ), (i1/2 , j1/2 ), (i1/4 , j1 ) obtained from searches of integer pixel, 1/2 pixel, and 1/4 pixel positions/4 ), the motion vector with the minimum absolute error and SAD is selected as the optimal motion vector (i, j) obtained by applying the effective three-step search method E3SS.
步骤5，对前向和后向参考宏块的整像素、1/2像素、1/4像素位置分别进行搜索比较，得到前向参考宏块的运动矢量(dx1,dy1)和后向参考宏块的运动矢量(dx2,dy2)，取其中绝对误差和SAD最小的一个运动矢量作为当前宏块的运动矢量(dx,dy)。Step 5, search and compare the integer pixel, 1/2 pixel, and 1/4 pixel positions of the forward and backward reference macroblocks respectively, and obtain the motion vector (dx1 , dy1 ) and the backward motion vector of the forward reference macroblock Refer to the motion vector (dx2 , dy2 ) of the macroblock, and take the motion vector with the smallest absolute error and SAD as the motion vector (dx, dy) of the current macroblock.
步骤6，根据当前宏块的运动矢量(dx,dy)，应用前向和后向参考场中相应的参考宏块，线性插值得到插值宏块F(x,y;k)：Step 6, according to the motion vector (dx, dy) of the current macroblock, apply the corresponding reference macroblocks in the forward and backward reference fields, and linearly interpolate to obtain the interpolated macroblock F(x, y; k):
其中，k为插值场在图像序列中的序号，(x,y)为宏块的坐标，F(x,y;k)为第k个插值场中以(x,y)为坐标的插值宏块，F(x-dx,y-dy;k-1)为前向参考场应用运动矢量(dx,dy)得到的参考宏块，F(x+dx,y+dy;k+1)为后向参考场应用运动矢量(dx,dy)得到的参考宏块。Among them, k is the serial number of the interpolation field in the image sequence, (x, y) is the coordinate of the macroblock, F(x, y; k) is the interpolation macro with (x, y) as the coordinate in the kth interpolation field Block, F(x-dx,y-dy;k-1) is the reference macroblock obtained by applying the motion vector (dx,dy) to the forward reference field, F(x+dx,y+dy;k+1) is The reference macroblock obtained by applying the motion vector (dx, dy) backward to the reference field.
步骤7，将插值宏块F(x,y;k)逐行插入当前宏块，形成一个行数为当前宏块行数二倍的图像宏块，该图像宏块为去隔行后的逐行图像宏块。Step 7, the interpolation macroblock F (x, y; k) is inserted into the current macroblock line by line to form an image macroblock whose number of lines is twice the number of lines of the current macroblock, and the image macroblock is progressive after deinterlacing image macroblock.
本发明的效果可以通过以下仿真实验和对比客观评价指标进一步说明：Effect of the present invention can be further illustrated by following simulation experiment and contrast objective evaluation index:
1)仿真实验条件：1) Simulation experiment conditions:
本发明的实验环境是Microsoft Visual Studio 2010，实验原始数据是两个大小为352×288的raw格式的100场的实际隔行视频序列，为经典的测试序列football和soccer序列，图2(a)为football序列的快照，图2(b)为soccer序列的快照。The experimental environment of the present invention is Microsoft Visual Studio 2010, and the original data of the experiment is the actual interlaced video sequence of 100 fields of two sizes in the raw format of 352 * 288, which is a classic test sequence football and soccer sequence, and Fig. 2 (a) is A snapshot of the football sequence, and Figure 2(b) is a snapshot of the soccer sequence.
本发明使用的客观评价指标为峰值信噪比PSNR，定义如下：The objective evaluation index that the present invention uses is peak signal-to-noise ratio PSNR, is defined as follows:
假定：两幅图像的大小为X×Y，令f(x,y)表示原始图像，表示插值后的图像，客观评价指标峰值信噪比PSNR定义为：Assumption: the size of the two images is X×Y, let f(x,y) represent the original image, Indicates the interpolated image, and the objective evaluation index peak signal-to-noise ratio PSNR is defined as:
2)仿真实验内容：2) Simulation experiment content:
仿真实验1，对图2(a)所示的football序列，运用整像素运动补偿去隔行方法进行实验仿真得到一个峰值信噪比PSNR曲线，运用本发明的1/4像素精度运动补偿去隔行方法进行实验仿真得到一个峰值信噪比PSNR曲线，将两个峰值信噪比PSNR曲线进行对比，结果如图4所示；Simulation experiment 1, to the football sequence shown in Fig. 2 (a), use whole pixel motion compensation deinterlacing method to carry out experimental simulation and obtain a peak signal-to-noise ratio PSNR curve, use 1/4 pixel precision motion compensation deinterlacing method of the present invention Perform experimental simulation to obtain a peak signal-to-noise ratio PSNR curve, compare the two peak signal-to-noise ratio PSNR curves, and the results are shown in Figure 4;
仿真实验2，对图2(b)所示的soccer序列，运用整像素运动补偿去隔行方法进行实验仿真得到一个峰值信噪比PSNR曲线，运用本发明的1/4像素精度运动补偿去隔行方法进行实验仿真得到一个峰值信噪比PSNR曲线，将两个峰值信噪比PSNR曲线进行对比，结果如图5所示。Simulation experiment 2, to the soccer sequence shown in Fig. 2 (b), use whole pixel motion compensation deinterlacing method to carry out experimental simulation and obtain a peak signal-to-noise ratio PSNR curve, use 1/4 pixel precision motion compensation deinterlacing method of the present invention Perform experimental simulation to obtain a peak signal-to-noise ratio PSNR curve, compare the two peak signal-to-noise ratio PSNR curves, and the result is shown in Figure 5.
3)实验结果分析：3) Analysis of experimental results:
从图4比较结果可以看出，对于存在剧烈运动的经典测试序列football，相比整像素快速运动估计去隔行效果，运用本发明提出的1/4像素精度运动补偿去隔行方法对图像去隔行处理的效果有明显的提升；As can be seen from the comparison results in Figure 4, for the classic test sequence football with violent motion, compared with the deinterlacing effect of the whole pixel fast motion estimation, the 1/4 pixel precision motion compensation deinterlacing method proposed by the present invention is used to deinterlace the image The effect has been significantly improved;
从图5比较结果可以看出，对于存在场景、人物综合运动的的经典测试序列soccer，相比整像素快速运动估计去隔行效果，运用本发明提出的1/4像素精度运动补偿去隔行方法对图像去隔行处理的效果有明显的提升；As can be seen from the comparison results in Fig. 5, for the classic test sequence soccer in which there are scenes and comprehensive movements of characters, compared with the deinterlacing effect of the whole pixel fast motion estimation, the 1/4 pixel precision motion compensation deinterlacing method proposed by the present invention is used for the deinterlacing effect. The effect of image deinterlacing processing has been significantly improved;
综上，本发明提出的基于运动补偿的1/4像素精度块匹配图像去隔行方法，在现有的整像素精度运动估计去隔行方法的基础上加入了1/2像素精度和1/4像素精度图像插值，使得运动估计搜索范围扩大，运动矢量搜索的精度提高，相对于整像素精度运动估计去隔行方法，本发明的图像PSNR评价标准有大幅度提升，在保证计算复杂度的前提下，能有效提高去隔行后的视频图像的视觉效果。In summary, the 1/4 pixel precision block matching image deinterlacing method based on motion compensation proposed by the present invention adds 1/2 pixel precision and 1/4 pixel precision to the existing integer pixel precision motion estimation deinterlacing method. High-precision image interpolation makes the search range of motion estimation expand, and the precision of motion vector search is improved. Compared with the whole pixel precision motion estimation de-interlacing method, the image PSNR evaluation standard of the present invention has been greatly improved. On the premise of ensuring the computational complexity, It can effectively improve the visual effect of the deinterlaced video image.
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201310030625.8ACN103051857B (en) | 2013-01-25 | 2013-01-25 | Motion compensation-based 1/4 pixel precision video image deinterlacing method |
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201310030625.8ACN103051857B (en) | 2013-01-25 | 2013-01-25 | Motion compensation-based 1/4 pixel precision video image deinterlacing method |
| Publication Number | Publication Date |
|---|---|
| CN103051857A CN103051857A (en) | 2013-04-17 |
| CN103051857Btrue CN103051857B (en) | 2015-07-15 |
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201310030625.8AExpired - Fee RelatedCN103051857B (en) | 2013-01-25 | 2013-01-25 | Motion compensation-based 1/4 pixel precision video image deinterlacing method |
| Country | Link |
|---|---|
| CN (1) | CN103051857B (en) |
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105516726B (en)* | 2015-11-27 | 2019-04-09 | 传线网络科技（上海）有限公司 | The motion compensation matching process and system of Video coding |
| CN106303340B (en)* | 2016-09-20 | 2019-02-15 | 天津大学 | A Deinterlacing Method for Bidirectional Motion Estimation |
| CN106998437B (en)* | 2017-03-31 | 2020-07-31 | 武汉斗鱼网络科技有限公司 | Method and device for reconstructing video image |
| CN109831672B (en)* | 2019-01-09 | 2020-10-27 | 北京大学 | Motion estimation method of space-time pulse array, electronic equipment and storage medium |
| CN109951713B (en)* | 2019-03-11 | 2023-02-28 | 深圳信息职业技术学院 | A motion estimation compensation circuit and method for video deinterlacing |
| CN110263699B (en)* | 2019-06-17 | 2021-10-22 | 睿魔智能科技（深圳）有限公司 | Video image processing method, device, equipment and storage medium |
| CN113246859B (en)* | 2021-05-28 | 2023-04-07 | 深圳天联星科技有限公司 | Electronic rearview mirror with driving auxiliary system warning function |
| CN116016831B (en)* | 2022-12-13 | 2023-12-05 | 湖南快乐阳光互动娱乐传媒有限公司 | Low-time-delay image de-interlacing method and device |
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| EP1234447A1 (en)* | 1999-11-05 | 2002-08-28 | Demografx | System and method for motion compensation and frame rate conversion |
| CN1846437A (en)* | 2003-09-07 | 2006-10-11 | 微软公司 | Innovations in encoding and decoding macroblocks and motion information for interlaced and progressive video |
| CN101340539A (en)* | 2007-07-06 | 2009-01-07 | 北京大学软件与微电子学院 | Deinterlacing video processing method and system by moving vector and image edge detection |
| CN101510985A (en)* | 2009-02-27 | 2009-08-19 | 西安交通大学 | Self-adapting de-interleave method for movement compensation accessory movement |
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US8160160B2 (en)* | 2005-09-09 | 2012-04-17 | Broadcast International, Inc. | Bit-rate reduction for multimedia data streams |
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| EP1234447A1 (en)* | 1999-11-05 | 2002-08-28 | Demografx | System and method for motion compensation and frame rate conversion |
| CN1846437A (en)* | 2003-09-07 | 2006-10-11 | 微软公司 | Innovations in encoding and decoding macroblocks and motion information for interlaced and progressive video |
| CN101340539A (en)* | 2007-07-06 | 2009-01-07 | 北京大学软件与微电子学院 | Deinterlacing video processing method and system by moving vector and image edge detection |
| CN101510985A (en)* | 2009-02-27 | 2009-08-19 | 西安交通大学 | Self-adapting de-interleave method for movement compensation accessory movement |
| Publication number | Publication date |
|---|---|
| CN103051857A (en) | 2013-04-17 |
| Publication | Publication Date | Title |
|---|---|---|
| CN103051857B (en) | Motion compensation-based 1/4 pixel precision video image deinterlacing method | |
| Kang et al. | Motion compensated frame rate up-conversion using extended bilateral motion estimation | |
| Ha et al. | Motion compensated frame interpolation by new block-based motion estimation algorithm | |
| Wang et al. | Frame rate up-conversion using trilateral filtering | |
| TW480885B (en) | Fast motion-compensated video frame interpolator | |
| KR100530223B1 (en) | Frame interpolation method and apparatus at frame rate conversion | |
| CN101106685B (en) | An deinterlacing method and device based on motion detection | |
| CN104219533B (en) | A kind of bi-directional motion estimation method and up-conversion method of video frame rate and system | |
| US20090110075A1 (en) | Method and System for Motion Compensated Picture Rate Up-Conversion of Digital Video Using Picture Boundary Processing | |
| CN102665061A (en) | Motion vector processing-based frame rate up-conversion method and device | |
| CN1422074A (en) | Interpolating picture element data selection for motion compensation and its method | |
| CN102025960A (en) | Motion compensation de-interlacing method based on adaptive interpolation | |
| CN101299818B (en) | N-level sub-pixel search method based on integer pixel search results | |
| CN101867759A (en) | Adaptive Motion Compensation Frame Rate Improvement Method Based on Scene Detection | |
| CN104811728A (en) | Video content self-adaptation motion search method | |
| KR20060135770A (en) | Reduction of artifacts in scan-rate conversion of image signals by combining interpolation and extrapolation of images | |
| CN103369208A (en) | Self-adaptive de-interlacing method and device | |
| CN108574844B (en) | Multi-strategy video frame rate improving method for space-time significant perception | |
| CN101969568A (en) | Frame rate up conversion-oriented motion estimation method | |
| CN105872559A (en) | Frame rate up-conversion method based on mixed matching of chromaticity | |
| CN102447870A (en) | Stationary object detection method and motion compensation device | |
| CN104065975B (en) | Based on the frame per second method for improving that adaptive motion is estimated | |
| CN102364933A (en) | An Adaptive Deinterlacing Method Based on Motion Classification | |
| CN106303340B (en) | A Deinterlacing Method for Bidirectional Motion Estimation | |
| CN106303339B (en) | A New Block Overlapped Motion Compensation Deinterlacing Method |
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| C14 | Grant of patent or utility model | ||
| GR01 | Patent grant | ||
| CF01 | Termination of patent right due to non-payment of annual fee | Granted publication date:20150715 Termination date:20200125 | |
| CF01 | Termination of patent right due to non-payment of annual fee |