CN101877790A - A Fast Global Motion Estimation Method for Panoramic Video Coding - Google Patents

Abstract

Translated fromChinese

本发明公开了一种面向全景视频编码的快速全局运动估计方法。该方法分为两步：第一步，用一个基于统计特性与阈值均值相结合的亮度残差阈值模型来大致划分局部运动区域与全局运动区域，得到全局运动估计区域的近似集合；第二步，使用一种运动矢量残差分级阈值技术在能量残差函数最小化迭代过程中逐步细化全局运动像素点集合，最后分离出完整的全局运动区域，从而实现快速运动估计的目的。本发明的优点是：兼顾了全局运动估计的准确性和鲁棒性，在保证重构图象质量不下降，而且还有略微提升的前提下，大大节省运动估计计算开销。对全景视频序列的全局运动估计具有算法复杂度低，运动参数估计精确的效果。The invention discloses a fast global motion estimation method for panoramic video coding. The method is divided into two steps: the first step is to roughly divide the local motion area and the global motion area with a luminance residual threshold model based on the combination of statistical characteristics and the threshold value, and obtain an approximate set of global motion estimation areas; the second step , using a motion vector residual grading threshold technique to gradually refine the global motion pixel set in the iterative process of energy residual function minimization, and finally separate the complete global motion region, so as to achieve the purpose of fast motion estimation. The advantages of the present invention are: the accuracy and robustness of the global motion estimation are taken into account, and the motion estimation calculation cost is greatly saved under the premise of ensuring that the quality of the reconstructed image is not lowered, but also slightly improved. The global motion estimation of panoramic video sequence has the effect of low algorithm complexity and accurate motion parameter estimation.

Description

A kind of quick global motion estimation method towards panorama video code

Technical field

The present invention relates to a kind of image video encoding compression technology, specifically a kind of quick global motion estimation method towards panorama video code.

Background technology

Panoramic video is widely used in the application such as sports cast, three-dimensional movie, multipart video-meeting.One width of cloth panoramic picture is to do rotation or convergent-divergent motion by the flake video camera of six or octahedral around fixing axle, the same time domain of scene around the video camera is taken the photo of different azimuth, seamless spliced these photos with some " stitching " technology, again according to cylinder or the sphere mapping algorithm is mapped to cylinder figure or spherical diagram forms.Therefore, the resolution of panoramic picture is in general than higher, and the motion details is abundanter.And because panoramic video be in order to allow the user have reality scene to experience, the camera motion of general virtual support reappears, and therefore, the staggered feature of global motion and local motion is more obvious.Moreover cylinder and sphere mapping algorithm can cause the object of which movement distortion, make the difficulty of estimation strengthen.But simultaneously, we can find from the analysis of two dimensional surface coordinate mapping cylinder areal coordinate and spherical coordinate, the motion deformation of panoramic picture object on cylinder or sphere mainly shows as the deformation that motion is cut in video camera rotation, convergent-divergent, mistake, therefore, we can use global motion model to come panoramic video is carried out the overall motion estimation compensation.

But, in the past studies show that overall motion estimation exists the big problem of amount of calculation.The overall motion estimation key is the estimation to global motion model parameters.In theory, for a definite global motion parameter model, as affine six movement parameter models, only the motion vector of 3 background pixel points of need can be tried to achieve the value of six kinematic parameters.But general motion vector method can only obtain the limited motion vector of precision, if only estimate global motion model parameters with seldom several pixel motion vector, the precision as a result that obtains is very low, therefore needs more pixel motion vector to participate in global motion model parameters and estimates to improve accuracy.Like this, parameter Estimation has just become a problem of separating the inconsistent equation group, and available least square method is found the solution, as Gauss's Newton iteration method.By the several iterative computation group of solving an equation, to the last the global motion model parameters value converges on a comparatively stable value.But this iterative process is the highest part of computation complexity in the overall motion estimation, especially when the global motion model of need handling comparatively complicated, under the more situation of iterations.

Moreover, most video scene all includes global motion and local motion, if allow all pixel motion vector all participate in overall motion estimation, pixel and the bigger pixel of estimation of motion vectors error of then making local motion can produce very big interference to overall motion estimation, make motion model parameters estimate just to be converged on a stationary value many times by iteration.Not only reduce the precision of overall motion estimation, and made computational complexity raise greatly.

Pixel in the local motion zone, we are called exterior point, the global motion pixel, we are called interior point.The emphasis of the quick global motion estimation method that we will study effectively splits the local motion zone from the view picture image exactly, makes the global motion zone comprise interior point as much as possible and the least possible exterior point.Key and the difficult point of distinguishing exterior point and interior point are determining of threshold value T.In some document, T is set to a fixing value (being referred to as the fixed threshold method), but often be difficult to find a suitable fixed threshold T to be competent at: if T is too little in whole iterative process, iterative process converges on the part that needs the local motion zone probably at the very start, when especially the local motion zone accounts for the large percentage of whole scene; When iteration finished if T is too big, the global motion pixel is misjudged some was the local motion point, thereby influences the accuracy of parameter Estimation greatly.Some document is mentioned with percentage threshold r% and is replaced threshold value T (being referred to as the fixed percentage threshold method), even the exterior point set is the set of the bigger r% pixel of residual values.But because to account for the scale of whole scene all inequality in the global motion zone in the different video, therefore this method or can only obtain the global motion zone of part, or in interior some set, can comprise a lot of exterior points.There is the scholar to propose a kind of improved method: to make the average (be referred to as residual error average threshold method) of threshold value T for the residual error of all pixels, but the residual error averaging method might not be portrayed the difference of global motion zone and local moving region well, and experimental result shows that also the some set converges on the local motion zone in this method may make in some cases.

Above-mentioned these inner regions determine that method is difficult to take into account the accuracy and the robustness of parameter Estimation, and often need to select a desirable threshold value in advance at different video scenes, and adaptability is relatively poor.

Summary of the invention

The purpose of this invention is to provide a kind of global motion estimating method of cutting apart global motion zone and local moving region fast towards panorama video code.

The technical scheme that the present invention solves the problems of the technologies described above is:

A kind of quick global motion estimation method, this method towards panorama video code be divided into two the step carry out:

1. the first step is used a luminance threshold model that combines with the threshold value average based on statistical property, roughly divides local motion zone and global motion zone by block matching motion estimation, motion compensation, thresholding brightness residual.

2. second go on foot, owing to obtained the approximate solution of global motion pixel set in step 1, here, we use a kind of classification thresholds method based on the motion vector residual error to come further refinement global motion pixel set, finally isolate complete global motion zone and realize the quick global estimation.Second step is a thresholding algorithm in conjunction with the constraint set linear decrease of least square method iterative process, by motion model parameters estimation, global motion vector estimation, calculating kinematical vector residual error and thresholding motion vector residual error, effectively be partitioned into the foreground object object, eliminate of the influence of local motion zone overall motion estimation.

Quick global estimation specific algorithm disclosed in this invention as shown in Figure 1.

The present invention's advantage compared with the prior art is: taken into account the accuracy and the robustness of overall motion estimation, guaranteed that the reconstructed image quality does not descend, and also having under the prerequisite that promotes slightly, saved the estimation computing cost greatly.It is low that the overall motion estimation of panoramic video sequence is had an algorithm complex, and kinematic parameter is estimated accurate effect.

Description of drawings

Fig. 1 is the quick global motion estimation algorithm flow chart of the present invention towards panorama video code.

Embodiment

Below in conjunction with embodiment the present invention is further described:

The concrete enforcement of the present invention is carried out in two steps:

1. the first step is carried out the brightness residual threshold method.

When the utilization recurrent least square method was estimated global motion, the initialization input value was the motion vector that each macro block obtains after estimating through block matching motion in the image.The motion vector that is in the macro block in local motion zone does not belong to the set of point in the global motion, it is the exterior point that the global motion parameter Estimation is caused interference, will consume a large amount of operation time. therefore, if when using block matching algorithm to carry out estimation, just can carry out anticipation to the attribute of macro block estimation, find in advance may disturb the macro block of overall motion estimation and be rejected, will inevitably reduce the amount of calculation of overall motion estimation greatly.

Analysis to video image can find that the texture in global motion zone is than horn of plenty; And local motion generally is positioned at texture gray scale unanimity or the extremely level and smooth zone of gray scale.This is that comparatively speaking, the brightness residual between primitive frame and the reference frame corresponding blocks also seldom because the movable information that the local motion zone is comprised is fewer.Therefore, the monochrome information of the present invention after by the macro block motion compensation comes each macro block is carried out preanalysis, to judge whether this macro block belongs to the local motion macro block, if, then this macro block was rejected away from the global motion set of macroblocks before carrying out overall motion estimation, to save the computing cost of overall motion estimation, improve the precision of overall motion estimation.

This algorithm is divided into following four steps:

1) entire image is divided into the piece of non-overlapping copies, block size is 16 * 16, to the estimation of taking exercises of each piece, obtains the motion vector (Δ x, Δ y) of each piece with block matching algorithm.In theory, block size is more little, and motion estimation result is accurate more, an opposite extreme situations, and when block size is a pixel, then motion estimation result is the most accurate.But so, the complexity of estimation will raise greatly, need the motion vector bit number of coding transmission also more.This algorithm adopts 16 * 16, takes into account and considers computation complexity and estimation accuracy.Simultaneously, estimate that what we adopted is that three step search quick motions are estimated, further reduces the complexity of estimation for block matching motion.The motion vector (Δ x, Δ y) of each piece that obtains by block matching algorithm needs preserve, imports as the initialization of the second link motion vector residual error classification thresholds method of execution.

2) use motion vector (Δ x, Δ y) that the previous step block matching algorithm obtains to the compensation of taking exercises of each piece of present frame, obtain the predicted value I of each pixel of present frame_kBecause (Δ x, Δ y) representative is because the caused interframe local motion displacement MV of object motion itself₁, do after the inter motion compensation, then can passing threshold current frame pixel (i, brightness I j)_kWith in the next frame ((i, j)+MV₁) brightness of position pixel

Difference judge that (i j) is global motion area pixel or local motion area pixel to current frame pixel.

3) calculate all pixels in the current block gradient mean value S (i, j), wherein,

And in the corresponding blocks in reference frame all pixels gradient mean value S ' (i, j), wherein,Relatively S (i, j) and S ' (i, the j) size of residual absolute value also is | S (i, j)-S ' (i, j).Be shown below:

$G_B=\underset{i=1}{\overset{m}{\mathrm{\Σ}}}\underset{j=1}{\overset{n}{\mathrm{\Σ}}}|S_{i,j}-S_{i,j}^{\′}|/m\×n---\left(1\right)$

In formula, m * n is a block size.

4) calculated threshold C_gThreshold value C_gDetermine according to following formula:

In the formula, N is the number that image is divided into the piece of non-overlapping copies, G_iBe the brightness residual (calculating) of i piece in the image according to formula 1,

Represent the brightness residual average of present frame.If the brightness residual G of current block_BLess than C_g, then current block is judged as the local motion region unit, rejects away from the global motion set of macroblocks.Consider threshold value critical point C_gValue excessively will cause global motion parameter Estimation failure, we determine that the value of k in the formula 2 is: under low code check situation, as the video sequence of encode CIF, QCIF form, k=1.1; Under high code check situation, as coding 4CIF, high definition, panoramic video sequence, k=1.2 also promptly is shown below:

C_g=1.1G_Av, under the low code check situation (3)

C_g=1.2G_Av, under the high code check situation

2. in second step, carry out motion vector residual error classification thresholds method.

Motion vector residual error ratio juris mainly is based on the difference of global motion vector and local motion vector, is the key point of this method to the tolerance of this difference.The present invention adopts a kind of classification thresholds method to realize the accurate division of this tolerance.The main thought of this algorithm is: in the starting stage of iteration, remove the exterior point of the overwhelming majority earlier with a big percentage threshold.Because at the beginning, the ratio that the exterior point that differs greatly with interior point accounts in image is maximum (local motion zone particularly, often comparatively significantly kinetic property difference is arranged) with the global motion zone, adopt a big percentage threshold can remove the big exterior point of these othernesses rapidly, in case got rid of the interference of the bigger exterior point of otherness earlier, then whole iterative process restrains sooner; For the first time after the iteration, obtained a rough overall motion estimation result, the pending exterior point that be left in the image this time is and the interior some pixel that difference is less, the ratio that accounts in image is little, in ensuing iteration several times, we can classification reduce the value of percentage threshold, get rid of the little exterior point of these othernesses gradually, the accurately set of point in the refinement, some set in exterior point as much as possible is all excluded.The specific algorithm step is as follows:

1) use block matching algorithm to estimate the local motion displacement of each piece of present frame.This step is carried out in the threshold method of a last link-based on brightness residual.

2) estimate motion model parameters.Consider that the global motion zone generally concentrates on image background regions, edge of image part just, we choose the motion vector of central block (block size is 16 * 16) of the image upper left corner, the upper right corner, three 48 * 48 image-regions in the lower left corner as the initialization input value of Gauss's Newton iteration estimation model parametric procedure first time, by Gauss's Newton iterative calculation, can draw initialized several motion model parameters values.

3) global motion vector of each piece (Δ x ', Δ y ') in the computed image.In the image, the piece motion of background area belongs to global motion, and the motion of the piece of foreground area belongs to local motion, but we still can be by in the motion model parameters value substitution Gauss again Newton iteration that obtains in rapid previous step, coordinate (x according to each piece central pixel point, y) value calculates respectively " global motion vector " of each piece.For the local motion piece, this motion vector is pseudo-global motion vector.

4) calculating kinematical vector residual error.For same, the motion vector (Δ x ', Δ y ') of this piece that draws in the motion vector (Δ x, Δ y) of this piece that step (1) is drawn and the step (3) is made variance, as shown in Equation 4.The purpose of calculating kinematical vector residual error is for the actual motion vector of judging this block of pixels and the difference of global motion vector, if the motion vector residual error is bigger, illustrate that this piece probably is the local motion piece, otherwise this piece probably is the global motion piece.

5) percentage threshold is set and gets rid of the local motion piece.We adopt the percentage threshold P of a hierarchical subtractive clustering to come progressively meticulous global motion zone, and the motion vector threshold residual value T that a hierarchical subtractive clustering is set simultaneously judges whether this iteration can finish.In first time iterative process, the initial value that we are provided with P is 50.It is 0.04 that T is set simultaneously, that is:

$\sqrt{{(\mathrm{\Δx}-{\mathrm{\Δx}}^{\′})}^2+{(\mathrm{\Δy}-{\mathrm{\Δy}}^{\′})}^2}>T---\left(4\right)$

The piece of the P percentage that is no more than image block number summation at most that satisfies the following formula condition is labeled as the local motion piece, and the remaining global motion piece that is labeled as is as the set of blocks of overall motion estimation iterative computation next time.If the motion vector residual error of all pieces does not satisfy the following formula condition in this overall motion estimation set of blocks, then finish this iteration, generate final motion model parameters, simultaneously can be partitioned into the local motion zone according to the local motion set of blocks of each iteration institute mark.The rule of successively decreasing of P and T: before at every turn entering the iterative computation of a new round, P is updated to 1/4th (P=P/4) of P, promptly in the iterative process second time, P is made as 12.5, and in the iterative process, P is made as 3.125 for the third time ... this is because of the increase along with iterations, interior some set more and more tends towards stability, the exterior point that is comprised is fewer and feweri, so the P value needs linear decrease, in order to avoid iteration convergence is in the part of interior some set.Simultaneously T is updated to T=T-0.01, because after each iteration end, the motion vector residual error of remaining piece is littler, wherein some is the local motion piece littler with the global motion attribute difference for this, need differentiate them with a littler T.When the iteration result tends to stable, can reach the quickening iteration convergence simultaneously with the T that successively decreases, lower computation complexity and guarantee the progressively meticulous purpose of overall motion estimation result.After Gauss's Newton iteration finishes for the first time, a new cover motion model parameters value will be obtained.The input value of this cover motion model parameters value as next round Gauss Newton iteration.

6) repeating step 3), 4), 5), all equal 4 up to the motion vector residual error of all pieces, then the finishing iteration process less than T or iterations.Experiment showed, that working as iterations surpasses 4 times, the accuracy increase of separating of model parameter is extremely limited, that is to say, 4th, the result of the 5th, the 6th iteration is very approximate, even is as good as, model parameter separate a limit that has reached this classification thresholds iterative method accuracy.

It should be noted last that above embodiment is only unrestricted in order to technical scheme of the present invention to be described.Although the present invention is had been described in detail with reference to embodiment, those of ordinary skill in the art is to be understood that, technical scheme of the present invention is made amendment or is equal to replacement, do not break away from the spirit and scope of technical solution of the present invention, it all should be encompassed in the claim of the present invention.

Claims (1)

Translated fromChinese

1.一种面向全景视频编码的快速全局运动估计方法，其特征在于：该方法分为两步进行：1. A fast global motion estimation method for panoramic video coding, characterized in that: the method is divided into two steps:1)第一步，使用一个基于统计特性与阈值均值相结合的亮度阈值模型，通过块匹配运动估计、运动补偿、阈值化亮度残差来大致划分局部运动区域与全局运动区域；1) In the first step, a brightness threshold model based on the combination of statistical characteristics and threshold mean is used to roughly divide the local motion area and the global motion area through block matching motion estimation, motion compensation, and thresholded brightness residual;2)第二步，由于在步骤1)已经取得了全局运动像素点集合的近似解，在这里，我们使用一种基于运动矢量残差的分级阈值法来进一步细化全局运动像素点集合，最终分离出完整的全局运动区域并实现快速全局运动估计；第二步骤是一个结合最小二乘法迭代过程的约束集线性递减的阈值算法，通过运动模型参数估计、全局运动矢量估计、计算运动矢量残差以及阈值化运动矢量残差，有效的分割出前景物体对象，消除局部运动区域对全局运动估计的影响。2) In the second step, since the approximate solution of the global motion pixel point set has been obtained in step 1), here, we use a classification threshold method based on the motion vector residual to further refine the global motion pixel point set, and finally Separate the complete global motion area and achieve fast global motion estimation; the second step is a threshold algorithm that combines the linear decrease of the constraint set with the iterative process of the least squares method, through motion model parameter estimation, global motion vector estimation, and calculation of motion vector residuals And the thresholded motion vector residual can effectively segment the foreground object and eliminate the influence of the local motion area on the global motion estimation.