技术领域technical field
本发明属于图像处理技术领域，更具体涉及一种面向移动终端的抖动模糊图像复原方法。The invention belongs to the technical field of image processing, and more specifically relates to a mobile terminal-oriented method for restoring shaken and blurred images.
背景技术Background technique
移动终端的内置相机在拍摄图片过程中，不可避免的会由于各种原因的抖动而导致图像产生模糊，而多数时候拍摄瞬间具有不可重现性，因此对拍摄的模糊图像进行复原就十分有必要。市场上绝大部分的移动终端都内置陀螺仪，这使得获取移动终端在相机快门开启时间内的运动信息成为可能，根据移动终端的运动信息就能获取该时间内的抖动PSF模板，然后根据图像复原算法和上述的抖动PSF模板对该模糊图片进行复原处理，就可以得到复原图像。During the process of taking pictures, the built-in camera of the mobile terminal will inevitably blur the image due to various reasons of shaking, and most of the time the shooting moment is irreproducible, so it is very necessary to restore the blurred image taken . Most of the mobile terminals on the market have built-in gyroscopes, which makes it possible to obtain the motion information of the mobile terminal during the opening time of the camera shutter. According to the motion information of the mobile terminal, the jitter PSF template within this time can be obtained, and then according to the image The restoration algorithm and the above-mentioned dithering PSF template perform restoration processing on the blurred picture to obtain a restored image.
图像复原在图像处理领域非常重要，其复原过程实际上是一种估计过程，根据图像模糊的因素构建相应的复原模型，从而采取相应的图像复原算法对模糊图像进行复原，提高图像的清晰度，改善其视觉效果。Image restoration is very important in the field of image processing. The restoration process is actually an estimation process. The corresponding restoration model is constructed according to the blurred factors of the image, so that the corresponding image restoration algorithm is used to restore the blurred image and improve the clarity of the image. Improve its visual effects.
模糊图像复原一直是图像处理领域的研究热点和难点，根据PSF是否已知，模糊图像复原分为盲复原和非盲复原，非盲复原相比盲复原主要多出PSF的估计过程，这往往是图像复原的难点。目前有很多从模糊图像中直接提取PSF的方法，此类方法复杂程度高，计算时间较长，提取效果受图像内容影响较大，适应性一般。Blurred image restoration has always been a research hotspot and difficulty in the field of image processing. According to whether the PSF is known, blurred image restoration can be divided into blind restoration and non-blind restoration. Compared with blind restoration, non-blind restoration mainly requires PSF estimation process, which is often Difficulties in image restoration. At present, there are many methods for directly extracting PSF from blurred images. These methods are highly complex, take a long time to calculate, the extraction effect is greatly affected by the image content, and the adaptability is general.
发明内容Contents of the invention
针对现有技术的缺陷，本发明提供了一种面向移动终端的抖动模糊图像复原方法，其目的在于利用终端内置的陀螺仪获取PSF，然后将其应用到非盲复原算法中对模糊图像进行复原，提高图像的清晰度。Aiming at the deficiencies of the prior art, the present invention provides a mobile terminal-oriented restoration method for shaking and blurred images, the purpose of which is to use the gyroscope built in the terminal to obtain PSF, and then apply it to a non-blind restoration algorithm to restore blurred images , to improve image clarity.
本发明提供了一种面向移动终端的抖动模糊图像复原方法，包括下述步骤：The invention provides a mobile terminal-oriented method for restoring a shaking and blurred image, comprising the following steps:
(1)获得一幅移动终端抖动时的模糊图像；(1) Obtain a blurred image when the mobile terminal shakes;
(2)通过陀螺仪获得相机快门开启时间内，与移动终端抖动轨迹相关的输出序列；(2) Obtain the output sequence related to the shaking track of the mobile terminal within the opening time of the camera shutter through the gyroscope;
其中，输出序列为m×n组数据，数据格式为(Gxi,Gyi,Gzi,ti)，m为陀螺仪的采样频率，n为相机的快门开启时间，下标i表示输出序列号，取值为1，2，……，m×n，Gxi表示第i组绕X轴旋转的角速度，Gyi表示第i组绕Y轴旋转的角速度，Gzi表示第i组绕Z轴旋转的角速度，ti表示第i组数据的采样时刻，X轴为相机的视轴，Y轴为像平面的纵轴，Z轴为像平面的横轴；Among them, the output sequence is m×n groups of data, the data format is (Gxi , Gyi , Gzi , ti ), m is the sampling frequency of the gyroscope, n is the shutter opening time of the camera, and the subscript i represents the output sequence No., the value is 1, 2,..., m×n, Gxi represents the angular velocity of the i-th group rotating around the X-axis, Gyi represents the angular velocity of the i-th group rotating around the Y-axis, Gzi represents the i-th group rotating around the Z The angular velocity of axis rotation, ti represents the sampling moment of the i-th group of data, the X-axis is the viewing axis of the camera, the Y-axis is the vertical axis of the image plane, and the Z-axis is the horizontal axis of the image plane;
(3)对所述输出序列进行积分处理，获得相机在其所在XYZ坐标系中Y轴和Z轴的转动角度，并将所述转动角度进行坐标映射，获得抖动PSF模板；(3) Carry out integral processing to described output sequence, obtain the rotation angle of camera Y axis and Z axis in its XYZ coordinate system, and carry out coordinate mapping to described rotation angle, obtain shaking PSF template;
(4)根据所述模糊图像和所述抖动PSF模板，采用图像复原算法进行复原处理，获得复原图像。(4) According to the blurred image and the dithered PSF template, an image restoration algorithm is used to perform restoration processing to obtain a restored image.
更进一步地，获得抖动PSF模板的步骤具体为：Furthermore, the steps for obtaining the jitter PSF template are as follows:
(1)初始化一个空链表L，令输出序列号i的初始值为1；(1) Initialize an empty linked list L, so that the initial value of the output sequence number i is 1;
(2)对所述输出序列进行积分处理，获得相机在快门开启时间内ti时刻在其所在XYZ坐标系中Y轴的转动角度θyi和Z轴的转动角度θzi；(2) Integral processing is carried out to described output sequence, obtains the angle of rotation θyi of the Y axis and the angle of rotation θzi of the Z axis in the XYZ coordinate system where the camera is located at the time ti of the shutter opening time;
(3)将相机在Y轴的转动角度θyi和Z轴的转动角度θzi映射到像平面坐标系中，得到像平面中ti时刻的一个位置点(yi,zi,△ti)，并将它添加到链表L中；(3) Map the rotation angle θyi of the camera on the Y axis and the rotation angle θzi of the Z axis to the image plane coordinate system, and obtain a position point (yi , zi, Δti ), and add it to the linked list L;
其中，yi为ti时刻Y轴的偏移位置，zi为ti时刻Z轴的偏移位置，△ti为相机姿态在该位置的保持时间，yi＝h×tan(θyi)，zi＝h×tan(θzi)，△ti＝ti+1-ti，h表示相机镜头焦距；Among them, yi is the offset position of the Y axis at the time ti , zi is the offset position of the Z axis at the time ti , △ti is the holding time of the camera attitude at this position, yi =h×tan(θyi ), zi ＝h×tan(θzi ), △ti ＝ti+1 -ti , h represents the focal length of the camera lens;
(4)判断i是否等于m×n，若是，则进入步骤(5)；若否，则i加1，并返回至步骤(2)；(4) judge whether i is equal to m×n, if so, then enter step (5); if not, add 1 to i, and return to step (2);
(6)初始化一个(2×|y|max+1)行，(2×|z|max+1)列的全0矩阵M，并令数据序列号j的初始值为1；(6) Initialize a (2×|y|max +1) row, (2×|z|max +1) column all-0 matrix M, and set the initial value of the data sequence number j to 1;
(7)取出链表L中的第j组数据(yj,zj,△tj)，将矩阵M中第(yj+|y|max)行第(zj+|z|max)列的元素值加△tj；(7) Take out the jth group of data (yj ,zj ,△tj ) in the linked list L, and put the (yj +|y|max ) row and (zj +|z|max ) column in the matrix M Add △tj to the element value of ;
(8)判断j是否等于m×n，若是，则进入步骤(9)；若否，则j加1，并返回至步骤(7)；(8) judge whether j is equal to m×n, if so, then enter step (9); if not, add 1 to j, and return to step (7);
(9)将矩阵M进行归一化，即得到抖动PSF模板。(9) Normalize the matrix M to obtain the dithering PSF template.
本发明与现有技术相比的优点在于：The advantage of the present invention compared with prior art is:
(1)本发明为了提高模糊图像的清晰度，对其进行非盲复原处理，可以估计出模糊图像的PSF；具体地利用陀螺仪输出序列准确计算相机在快门开启时间内的运动轨迹，然后通过坐标映射就可以估计出模糊图像的PSF，再将该PSF应用到非盲复原算法中对模湖图像进行复原处理，获得复原图像，从而提高了图像的清晰度。(1) In order to improve the clarity of the blurred image, the present invention performs non-blind restoration processing on it, and the PSF of the blurred image can be estimated; specifically, the gyroscope output sequence is used to accurately calculate the motion track of the camera within the shutter opening time, and then pass Coordinate mapping can estimate the PSF of the blurred image, and then apply the PSF to the non-blind restoration algorithm to restore the model lake image to obtain the restored image, thereby improving the clarity of the image.
(2)本发明只与终端相机的状态有关，而与所拍摄图片的内容无关；而现有的PSF提取方法大多在分析模糊图像的内容后，根据先验知识对PSF进行估计和提取，这些方法计算量大且使用范围多有局限性，本发明只需要根据终端相机姿态变化的输出序列进行模糊图像的PSF提取，与拍摄内容无关，适用性强。(2) The present invention is only related to the state of the terminal camera, and has nothing to do with the content of the captured picture; and most of the existing PSF extraction methods estimate and extract the PSF according to the prior knowledge after analyzing the content of the blurred image. The calculation amount of the method is large and the scope of use is limited. The present invention only needs to extract the PSF of the blurred image according to the output sequence of the terminal camera posture change, which has nothing to do with the shooting content and has strong applicability.
(3)本发明根据移动终端内置的陀螺仪的输出序列直接计算模糊图像的PSF，而这些数据量比较有限，因此本发明在PSF提取上复杂程度很低，计算量很小。(3) The present invention directly calculates the PSF of the blurred image according to the output sequence of the built-in gyroscope in the mobile terminal, and the amount of these data is relatively limited, so the present invention has a very low complexity in PSF extraction, and the amount of calculation is very small.
(4)纯数字化处理，不需要添加额外的硬件，成本低廉。通过上面的描述可以看出，本发明只根据移动终端内置的陀螺仪就能得到理想的结果，复原操作纯数字化，完全不需要其他辅助硬件设备，成本低廉，具有广泛的实用价值。(4) Pure digital processing, no need to add additional hardware, low cost. It can be seen from the above description that the present invention can obtain ideal results only based on the built-in gyroscope of the mobile terminal, and the recovery operation is purely digital, without any other auxiliary hardware equipment, low cost, and has extensive practical value.
附图说明Description of drawings
图1是本发明实施例提供的相机XYZ坐标系图，其中X轴为相机的视轴，Y轴为像平面的纵轴，Z轴为像平面的横轴；Fig. 1 is the XYZ coordinate system diagram of the camera provided by the embodiment of the present invention, wherein the X axis is the visual axis of the camera, the Y axis is the longitudinal axis of the image plane, and the Z axis is the horizontal axis of the image plane;
图2是本发明实施例提供的复原流程图；Fig. 2 is a recovery flowchart provided by an embodiment of the present invention;
图3是本发明实施例提供的输出序列到像平面的坐标关系图；Fig. 3 is a coordinate relationship diagram from an output sequence to an image plane provided by an embodiment of the present invention;
图4是本发明实施例提供的PSF的提取流程图；Fig. 4 is the extraction flowchart of the PSF provided by the embodiment of the present invention;
图5是本发明实施例提供的实际复原效果对比图，其中图(a)、(b)为两幅模糊图像，图(c)是图(a)的复原效果图，图(d)是图(b)的复原效果图。Fig. 5 is a comparison diagram of the actual restoration effect provided by the embodiment of the present invention, wherein Fig. (a) and (b) are two blurred images, Fig. (c) is the restoration effect diagram of Fig. (a), and Fig. (d) is (b) Restoration rendering.
具体实施方式Detailed ways
为了使本发明的目的、技术方案及优点更加清楚明白，以下结合附图及实施例，对本发明进行进一步详细说明。应当理解，此处所描述的具体实施例仅仅用以解释本发明，并不用于限定本发明。In order to make the object, technical solution and advantages of the present invention clearer, the present invention will be further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described here are only used to explain the present invention, not to limit the present invention.
相机的抖动分为平动和旋转，平动对图像质量的影响较小，可以不考虑，旋转对图像质量的影响很大，是造成图像模糊的主要原因。图1是本实施例提供的相机XYZ坐标系图，其中X轴为相机的视轴，Y轴为像平面的纵轴，Z轴为像平面的横轴。相机的旋转就是绕着X轴、Y轴和Z轴的旋转。而绕X轴的旋转对成像的影响比较小，所以也可以不考虑。Camera shake is divided into translation and rotation. Translation has little impact on image quality and can be ignored. Rotation has a great impact on image quality and is the main cause of image blur. FIG. 1 is a diagram of the XYZ coordinate system of the camera provided in this embodiment, wherein the X axis is the viewing axis of the camera, the Y axis is the longitudinal axis of the image plane, and the Z axis is the horizontal axis of the image plane. The rotation of the camera is the rotation around the X-axis, Y-axis and Z-axis. The rotation around the X-axis has relatively little impact on imaging, so it can be ignored.
本发明提供了一种面向移动终端的抖动模糊图像复原方法，能够估计出相机快门开启时间内的抖动PSF模板，并用此PSF模板进行抖动模糊图像的数字化复原。本发明利用移动终端内置的陀螺仪获得相机绕所述X轴、Y轴和Z轴旋转的输出序列，然后估计出相机在快门开启时间内的抖动轨迹，再通过坐标映射得到模糊图像的PSF，最后将此PSF应用到图像复原算法中对模糊图像进行数字复原，从而提高图像的清晰度，改善其视觉效果。由于相机和陀螺仪都内置于移动终端，因此相机和陀螺仪拥有相同的抖动轨迹，所以可以用陀螺仪在相机快门开启时间内的输出序列得到相机的运动信息。The invention provides a mobile terminal-oriented method for restoring a shake-blurred image, which can estimate a shake PSF template within the opening time of a camera shutter, and use the PSF template to perform digital restoration of the shake-blur image. The present invention utilizes the built-in gyroscope of the mobile terminal to obtain the output sequence of the camera rotating around the X-axis, Y-axis and Z-axis, then estimates the shaking track of the camera within the shutter opening time, and then obtains the PSF of the blurred image through coordinate mapping, Finally, the PSF is applied to the image restoration algorithm to digitally restore the blurred image, thereby improving the clarity of the image and improving its visual effect. Since both the camera and the gyroscope are built into the mobile terminal, the camera and the gyroscope have the same shaking trajectory, so the motion information of the camera can be obtained by using the output sequence of the gyroscope within the opening time of the camera shutter.
在本发明提供的面向移动终端的抖动模糊图像复原方法中，最后需要将抖动PSF模板应用到图像复原算法中才能对模糊图像进行复原，本实施例选用的图像复原算法为Richardson-Lucy算法。图2是本发明实施例提供的复原流程图，具体包括下述步骤：In the mobile terminal-oriented jitter-blurred image restoration method provided by the present invention, the blurred image can only be restored by applying the jitter PSF template to the image restoration algorithm. The image restoration algorithm selected in this embodiment is the Richardson-Lucy algorithm. Fig. 2 is a recovery flowchart provided by an embodiment of the present invention, which specifically includes the following steps:
(1)获得一幅移动终端抖动时的模糊图像；(1) Obtain a blurred image when the mobile terminal shakes;
(2)通过陀螺仪获得相机在快门开启时间内绕所述X轴、Y轴和Z轴旋转的输出序列。(2) Obtain an output sequence of the camera rotating around the X-axis, Y-axis and Z-axis within the shutter opening time through the gyroscope.
(3)对上述输出序列进行积分处理，得到快门开启时间内各时刻相机Y轴和Z轴的转动角度。(3) Integrate the above output sequence to obtain the Y-axis and Z-axis rotation angles of the camera at each moment within the shutter opening time.
(4)将步骤(3)中求得的转动角度通过坐标转换映射到像平面坐标系，得到抖动PSF模板。(4) Map the rotation angle obtained in step (3) to the image plane coordinate system through coordinate transformation, and obtain the dithering PSF template.
(5)利用Richardson-Lucy算法对模糊图像进行复原。(5) Use Richardson-Lucy algorithm to restore the blurred image.
Richardson-Lucy复原算法是一种采用迭代方式的模糊图像复原方法，其核心是利用得到的高信噪比PSF模板进行模糊图像的复原，步骤(4)中已经得到了模糊图像的PSF，从而利用Richardson-Lucy算法可以实现对移动终端拍摄的模糊图像的复原。The Richardson-Lucy restoration algorithm is an iterative fuzzy image restoration method. Its core is to restore the fuzzy image by using the PSF template with high signal-to-noise ratio. In step (4), the PSF of the fuzzy image has been obtained, thus using The Richardson-Lucy algorithm can realize the restoration of blurred images captured by mobile terminals.
对于光学成像系统，PSF被定义为以二维冲击函数作为输入的系统输出。其中二维冲击函数被定义为如下所示的形式：For optical imaging systems, the PSF is defined as the system output with a 2D shock function as input. where the two-dimensional shock function is defined as follows:
广义的抖动定义为光学成像系统PSF中心位置随时间变化产生的随机偏移。对点目标来说，抖动表现为图像中目标像点在各个有效采样时刻对其理想位置的偏离，也就是说抖动是一种随机影响造成的运动噪声，表现为目标位置在图像帧内的随机偏移。此处讨论的抖动为广义上的抖动，需要注意的是，当成像相机快门开启时间足够短时，抖动不会造成PSF在图像帧内的随机运动，也就是说抖动不影响瞬时PSF的空间分布特征。不考虑散焦等原因造成的图像模糊，抖动的图像在采样瞬间是清晰的，但是时间平均的结果会造成图像的模糊。In a broad sense, jitter is defined as the random shift of the PSF center position of the optical imaging system over time. For a point target, the jitter is manifested as the deviation of the target image point in the image from its ideal position at each effective sampling time, that is to say, the jitter is a kind of motion noise caused by random influence, which is manifested as the random variation of the target position in the image frame. offset. The jitter discussed here is jitter in a broad sense. It should be noted that when the shutter opening time of the imaging camera is short enough, the jitter will not cause the random movement of the PSF within the image frame, that is to say, the jitter will not affect the spatial distribution of the instantaneous PSF feature. Regardless of the image blur caused by defocus and other reasons, the jittered image is clear at the sampling moment, but the result of time averaging will cause the image to be blurred.
假设快门的开启和关闭所用的时间非常短，那么光学成像过程不会受到图像抖动的干扰。原始图像f(x,y)通过退化函数H的作用，得到模糊图像g(x,y)：Assuming the shutter is opened and closed for a very short time, the optical imaging process will not be disturbed by image shake. The original image f(x,y) is degraded by the function H to obtain a blurred image g(x,y):
g(x,y)＝H[f(x,y)] (2)g(x,y)=H[f(x,y)] (2)
若退化函数H是线性的、空间不变的，则退化图像在空间域通过下式给出：If the degradation function H is linear and space invariant, the degraded image is given by the following formula in the spatial domain:
g(x,y)＝h(x,y)*f(x,y) (3)g(x,y)=h(x,y)*f(x,y) (3)
其中h(x,y)为退化函数的空间表示，也称PSF，“*”表示卷积，对上式两边进行傅里叶变换可以得到Among them, h(x,y) is the space representation of the degradation function, also known as PSF, "*" means convolution, and performing Fourier transform on both sides of the above formula can be obtained
G(u,v)＝H(u,v)F(u,v) (4)G(u,v)=H(u,v)F(u,v) (4)
其中(u,v)为频域坐标，G(u,v)、H(u,v)和F(u,v)分别对应模糊图像g(x,y)、退化函数h(x,y)和原始图像f(x,y)的频域值。Where (u,v) is the frequency domain coordinates, G(u,v), H(u,v) and F(u,v) correspond to the blurred image g(x,y) and the degradation function h(x,y) respectively and the frequency domain value of the original image f(x,y).
步骤(2)中，通过陀螺仪获得移动终端在快门开启时间内绕所述X轴、Y轴和Z轴旋转的角速度，输出序列为m×n组数据，数据格式为(Gxi,Gyi,Gzi,ti)，m为陀螺仪的采样频率，n为相机的快门开启时间，下标i表示输出序列号，取值为1，2，……，m×n，Gxi表示第i组绕X轴旋转的角速度，Gyi表示第i组绕Y轴旋转的角速度，Gzi表示第i组绕Z轴旋转的角速度，ti表示第i组数据的采样时刻。In step (2), the angular velocity of the mobile terminal rotating around the X-axis, Y-axis and Z-axis within the shutter opening time is obtained through the gyroscope, and the output sequence is m×n sets of data, and the data format is (Gxi , Gyi , Gzi , ti ), m is the sampling frequency of the gyroscope, n is the shutter opening time of the camera, the subscript i represents the output sequence number, and the value is 1, 2,..., m×n, Gxi represents the first The angular velocity of group i rotating around the X-axis, Gyi represents the angular velocity of the i-th group rotating around the Y-axis, Gzi represents the angular velocity of the i-th group rotating around the Z-axis, and ti represents the sampling time of the i-th group of data.
步骤(3)中，相机绕X的旋转对成像影响较小，所以可以不考虑。图3是本发明实施例提供的输出序列到像平面的坐标关系图，该图中θy、θz分别是相机Y轴、Z轴的转动角度，可见相机绕Y轴旋转影响的是Z轴的转动角度θz，相机绕Z轴旋转影响的是Y轴的转动角度θy。定义θyi是ti时刻Y轴的转动角度，θzi是ti时刻Z轴的转动角度，它们的计算如下In step (3), the rotation of the camera around X has little effect on imaging, so it can be ignored. Fig. 3 is a coordinate relationship diagram from the output sequence to the image plane provided by the embodiment of the present invention, in which θy and θz are the rotation angles of the Y-axis and Z-axis of the camera respectively, it can be seen that the rotation of the camera around the Y-axis affects the Z-axis The rotation angle θz of the camera around the Z axis affects the rotation angle θy of the Y axis. Definition θyi is the rotation angle of the Y axis at the time ti , and θzi is the rotation angle of the Z axis at the time ti , and their calculations are as follows
步骤(4)中，根据上述的角度通过坐标映射到像平面坐标系，得到抖动PSF模板。图3中a和b分别为拍摄图像的宽度和长度，h为相机镜头焦距，y和z分别为冲击函数在θy和θz影响下于像平面YOZ中Y轴和Z轴的偏移位置。定义yi为ti时刻Y轴的偏移位置，zi为ti时刻Z轴的偏移位置，△ti为相机姿态在该位置的保持时间。像平面YOZ中点(yi,zi,△ti)的计算公式如下In step (4), the dithering PSF template is obtained by mapping the coordinates to the image plane coordinate system according to the above angle. In Figure 3, a and b are the width and length of the captured image, h is the focal length of the camera lens, y and z are the offset positions of the impact function on the Y-axis and Z-axis in the image plane YOZ under the influence of θy and θz , respectively . Define yi as the offset position of the Y axis at the time ti , zi as the offset position of the Z axis at the time ti , and Δti as the holding time of the camera attitude at this position. The formula for calculating the midpoint (yi , zi , △ti ) of the image plane YOZ is as follows
图4是上述抖动PSF模板的提取流程图，具体包括以下步骤：Fig. 4 is the extraction flowchart of above-mentioned dithering PSF template, specifically comprises the following steps:
(1)初始化一个空链表L，令输出序列号i为1；(1) Initialize an empty linked list L, and set the output sequence number i to 1;
(2)对所述输出序列进行积分处理，获得相机在快门开启时间内ti时刻在其所在XYZ坐标系中Y轴的转动角度θyi和Z轴的转动角度θzi；(2) Integral processing is carried out to described output sequence, obtains the angle of rotation θyi of the Y axis and the angle of rotation θzi of the Z axis in the XYZ coordinate system where the camera is located at the time ti of the shutter opening time;
(3)将相机在Y轴的转动角度θyi和Z轴的转动角度θzi映射到像平面坐标系中，得到像平面中ti时刻的位置点(yi,zi,△ti)，并将它添加到链表L中；(3) Map the rotation angle θyi of the camera on the Y axis and the rotation angle θzi of the Z axis to the image plane coordinate system, and obtain the position point (yi , zi , △ ti ) in the image plane at time ti , and add it to the linked list L;
(4)判断i是否等于m×n，若是，则进入步骤(5)；若否，则i加1，并返回至步骤(2)；(4) judge whether i is equal to m×n, if so, then enter step (5); if not, add 1 to i, and return to step (2);
(5)计算yi的绝对值最大值|y|max和zi的绝对值最大值|z|max；(5) Calculate the absolute value maximum value |y|max of yi and the absolute value maximum value |z|max of zi ;
(6)初始化一个(2×|y|max+1)行，(2×|z|max+1)列的全0矩阵M，并令数据序列号j等于1；(6) Initialize a matrix M with (2×|y|max +1) rows and (2×|z|max +1) columns of all 0s, and set the data serial number j equal to 1;
(7)取出链表L中的第j组数据(yj,zj,△tj)，将矩阵M中第(yj+|y|max)行第(zj+|z|max)列的元素值加△tj；(7) Take out the jth group of data (yj ,zj ,△tj ) in the linked list L, and put the (yj +|y|max ) row and (zj +|z|max ) column in the matrix M Add △tj to the element value of ;
(8)判断j是否等于m×n，若是，则进入步骤(9)；若否，则j加1，并返回至步骤(7)；(8) judge whether j is equal to m×n, if so, then enter step (9); if not, add 1 to j, and return to step (7);
(9)将矩阵M进行归一化，即得到抖动PSF模板。(9) Normalize the matrix M to obtain the dithering PSF template.
在复原流程的步骤(5)中，Richardson-Lucy算法具体为：In step (5) of the recovery process, the Richardson-Lucy algorithm is specifically:
Richardson-Lucy算法是由Richardson和Lucy共同提出的一种用于模糊图像复原的算法。最初用于纠正由于哈勃望远镜的镜头偏差所造成的图像模糊，目前该算法是一种应用广泛的图像复原算法。The Richardson-Lucy algorithm is an algorithm for blurred image restoration jointly proposed by Richardson and Lucy. Originally used to correct image blur caused by Hubble's lens aberration, the algorithm is currently a widely used image restoration algorithm.
Richardson-Lucy算法的基本思想是通过不断的迭代使得输出为理想数据的一个最大似然值。将此算法应用到模糊图像复原时，前提是PSF必须是已知的，或者至少是可以估计得到的，然后再通过一定的迭代次数，使得输出图像收敛到同真实图像相近的最大似然图像，从而改善模糊图像的显示效果。The basic idea of the Richardson-Lucy algorithm is to make the output a maximum likelihood value of ideal data through continuous iteration. When applying this algorithm to blurred image restoration, the premise is that the PSF must be known, or at least can be estimated, and then through a certain number of iterations, the output image converges to the maximum likelihood image similar to the real image. Thereby improving the display effect of blurred images.
Richardson-Lucy算法假设图像噪声服从泊松分布，模糊图像g＝h*f+β，f和g分别表示原始清晰图像和模糊图像，h表示PSF，β表示图像噪声，“*”表示卷积运算。移项之后有β＝g-h*f，得到图像f的似然概率p(g|f)与噪声的似然概率p(β)的概率分布相同，都服从泊松分布，用最大似然估计求解时，条件概率分布：The Richardson-Lucy algorithm assumes that the image noise obeys the Poisson distribution, the blurred image g=h*f+β, f and g represent the original clear image and the blurred image, h represents the PSF, β represents the image noise, and "*" represents the convolution operation . After the transposition, there is β=g-h*f, and the probability distribution of the likelihood probability p(g|f) of the image f and the likelihood probability p(β) of the noise is the same, and both obey the Poisson distribution, and the maximum likelihood estimation is used to solve the problem , the conditional probability distribution:
为了求取最大值，根据最大似然估计方法，将上式取对数后两端对f(x,y)求导并令其为零，则有：In order to obtain the maximum value, according to the maximum likelihood estimation method, take the logarithm of the above formula and then take the derivative of f(x,y) at both ends and make it zero, then:
其中h*为h的伴随矩阵，即h*(x,y)＝h(-x,-y)。上式两端同时乘以f(x,y)，可以得到：Where h* is the accompanying matrix of h, that is, h* (x, y) = h(-x, -y). Multiplying both sides of the above formula by f(x,y) at the same time, we can get:
加入迭代算法对其求解得：Add iterative algorithm to solve it:
其中，α表示迭代次数。上式变换到频域如下：Among them, α represents the number of iterations. The above formula is transformed into the frequency domain as follows:
其中(u,v)为频域坐标，Fα(u,v)为第α-1次迭代后的复原图像的频域值，Fα+1(u,v)为第α次迭代后复原图像的频域值，G(u,v)、H(u,v)分别对应模糊图像g(x,y)、PSF h(x,y)的频域值，H*(u,v)为H(u,v)的共轭矩阵，并且F0(u,v)＝G(u,v)。Where (u,v) is the frequency domain coordinates, Fα (u,v) is the frequency domain value of the restored image after the α-1th iteration, Fα+1 (u,v) is the restored image after the αth iteration The frequency domain value of the image, G(u,v) and H(u,v) correspond to the frequency domain value of the fuzzy image g(x,y) and PSF h(x,y) respectively, and H* (u,v) is The conjugate matrix of H(u,v), and F0 (u,v)=G(u,v).
这就是标准Richardson-Lucy算法，该方法没有提供迭代终止条件，随着迭代次数α的增加，复原图像逐渐收敛于原始清晰图像。该算法在使用小尺寸PSF时，通常较少的迭代次数就能达到稳定的解决方案。当已知PSF但图像噪声信息未知时，也可以用这种恢复方法进行有效恢复。This is the standard Richardson-Lucy algorithm. This method does not provide iteration termination conditions. As the iteration number α increases, the restored image gradually converges to the original clear image. The algorithm usually achieves a stable solution with fewer iterations when using a small PSF size. When the PSF is known but the image noise information is unknown, this restoration method can also be used for effective restoration.
利用Richardson-Lucy算法进行模糊图像复原的特点有以下几点：The characteristics of blurred image restoration using Richardson-Lucy algorithm are as follows:
(1)给定一幅原始图像，若PSF选择适当，只需要较少的迭代次数，就能够得到较好的复原效果；(1) Given an original image, if the PSF is properly selected, only a small number of iterations is required to obtain a better restoration effect;
(2)只要PSF和输入图像的像素点非负，则经过迭代处理后所得到的复原图像的像素点的值也是非负的；(2) As long as the PSF and the pixels of the input image are non-negative, the values of the pixels of the restored image obtained after iterative processing are also non-negative;
(3)若模糊图像与原始图像有较大的偏差，只要给出准确的PSF，同样可以用较少的迭代次数将其复原。(3) If there is a large deviation between the blurred image and the original image, as long as the accurate PSF is given, it can also be restored with fewer iterations.
在本方法中根据复原流程的步骤(2)到步骤(4)得到移动终端拍摄模糊图像的PSF后，首先设定迭代次数α并将迭代方程中f0(x,y)设为g(x,y)，即将初次迭代的结果初始化为模糊图像自身，然后将式(11)中h置为复原流程的步骤(4)所求得的PSF模板，最后根据式(11)进行迭代得到复原图像。In this method, after obtaining the PSF of the blurred image taken by the mobile terminal according to steps (2) to (4) of the restoration process, first set the number of iterations α and set f0 (x, y) in the iteration equation as g(x , y), that is, initialize the result of the first iteration as the blurred image itself, then set h in formula (11) as the PSF template obtained in step (4) of the restoration process, and finally iterate according to formula (11) to obtain the restored image .
图5展示了实施上述方法的效果对比图，其中图5(a)、(b)是两幅模糊图像，图(c)是图(a)的复原效果图，图(d)是图(b)的复原效果图，图(c)、(d)中的右下角是本方法提取的PSF。通过图5我们可以看到使用本发明得到模糊图像的PSF之后采用Richardson-Lucy算法进行复原能够得到细节清晰的复原图像，图像清晰度得到提高。Figure 5 shows the effect comparison diagram of implementing the above method, in which Figure 5(a) and (b) are two blurred images, Figure (c) is the restoration effect diagram of Figure (a), and Figure (d) is the image (b) ) restoration effect diagram, the lower right corners in (c) and (d) are the PSF extracted by this method. From Fig. 5, we can see that after the PSF of the blurred image is obtained by using the present invention, the Richardson-Lucy algorithm is used for restoration to obtain a restored image with clear details, and the image definition is improved.
本领域的技术人员容易理解，以上所述仅为本发明的较佳实施例而已，并不用以限制本发明，凡在本发明的精神和原则之内所作的任何修改、等同替换和改进等，均应包含在本发明的保护范围之内。It is easy for those skilled in the art to understand that the above descriptions are only preferred embodiments of the present invention, and are not intended to limit the present invention. Any modifications, equivalent replacements and improvements made within the spirit and principles of the present invention, All should be included within the protection scope of the present invention.
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201510459486.XACN105069753B (en) | 2015-07-30 | 2015-07-30 | A kind of shake Restoration method of blurred image of facing moving terminal |
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201510459486.XACN105069753B (en) | 2015-07-30 | 2015-07-30 | A kind of shake Restoration method of blurred image of facing moving terminal |
| Publication Number | Publication Date |
|---|---|
| CN105069753A CN105069753A (en) | 2015-11-18 |
| CN105069753Btrue CN105069753B (en) | 2018-06-26 |
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201510459486.XAExpired - Fee RelatedCN105069753B (en) | 2015-07-30 | 2015-07-30 | A kind of shake Restoration method of blurred image of facing moving terminal |
| Country | Link |
|---|---|
| CN (1) | CN105069753B (en) |
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106778730B (en)* | 2016-12-29 | 2020-07-07 | 深圳爱拼信息科技有限公司 | Self-adaptive method and system for rapidly generating OCR training samples |
| CN110678898B (en)* | 2017-06-09 | 2022-03-18 | 厦门美图之家科技有限公司 | Video anti-shake method and mobile device |
| CN107395961A (en)* | 2017-07-07 | 2017-11-24 | 青岛海信移动通信技术股份有限公司 | The restored method and device of a kind of view data |
| CN109949234B (en)* | 2019-02-25 | 2020-10-02 | 华中科技大学 | Video restoration model training method and video restoration method based on deep network |
| CN111578839B (en)* | 2020-05-25 | 2022-09-20 | 阿波罗智联(北京)科技有限公司 | Obstacle coordinate processing method and device, electronic equipment and readable storage medium |
| CN113049082B (en)* | 2021-02-08 | 2022-03-08 | 杭州富加镓业科技有限公司 | Detection method of seed crystal rod jitter detection device and crystal growth method |
| CN113014823B (en)* | 2021-03-25 | 2023-06-06 | 苏州科达科技股份有限公司 | Anti-shake processing method, system, equipment and storage medium of image pickup device |
| CN113139949B (en)* | 2021-04-30 | 2023-04-07 | 逻腾（杭州）科技有限公司 | Robot image ambiguity detection method |
| CN116934654B (en) | 2022-03-31 | 2024-08-06 | 荣耀终端有限公司 | Method for determining image blur and related device |
| CN116300294B (en)* | 2022-10-25 | 2024-04-12 | 荣耀终端有限公司 | A method and device for simulating human body shaking |
| CN118797214B (en)* | 2024-09-10 | 2024-11-22 | 电子科技大学 | Unmanned system flexible collaboration method for intelligent seal detection scene |
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101742050A (en)* | 2009-12-03 | 2010-06-16 | 浙江大学 | TDICCD image restoration method for motion blur kernel space shift |
| CN103947184A (en)* | 2011-11-14 | 2014-07-23 | 三星电子株式会社 | Photographing apparatus and image processing apparatus using coded light, and method thereof |
| US9049356B2 (en)* | 2010-08-27 | 2015-06-02 | Canon Kabushiki Kaisha | Image processing method, image processing apparatus and image processing program |
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101742050A (en)* | 2009-12-03 | 2010-06-16 | 浙江大学 | TDICCD image restoration method for motion blur kernel space shift |
| US9049356B2 (en)* | 2010-08-27 | 2015-06-02 | Canon Kabushiki Kaisha | Image processing method, image processing apparatus and image processing program |
| CN103947184A (en)* | 2011-11-14 | 2014-07-23 | 三星电子株式会社 | Photographing apparatus and image processing apparatus using coded light, and method thereof |
| Title |
|---|
| Motion Blurred Image Restoration;Shuai Jia等;《2013 6th International Congress on Image and Signal Processing》;20131218;384-389* |
| 一种遥感图像振动模糊复原方法;徐沪萍等;《华中科技大学学报(自然科学版)》;20140307;第42卷(第3期);第28-32页* |
| Publication number | Publication date |
|---|---|
| CN105069753A (en) | 2015-11-18 |
| Publication | Publication Date | Title |
|---|---|---|
| CN105069753B (en) | A kind of shake Restoration method of blurred image of facing moving terminal | |
| CN109753971B (en) | Correction method and device for distorted text lines, character recognition method and device | |
| Su et al. | Deep video deblurring for hand-held cameras | |
| Klingner et al. | Street view motion-from-structure-from-motion | |
| Lee et al. | Simultaneous localization, mapping and deblurring | |
| Su et al. | Rolling shutter motion deblurring | |
| US20220222849A1 (en) | Pose Tracking Method, Pose Tracking Device and Electronic Device | |
| US9824486B2 (en) | High resolution free-view interpolation of planar structure | |
| US8447140B1 (en) | Method and apparatus for estimating rotation, focal lengths and radial distortion in panoramic image stitching | |
| CN108564617A (en) | Three-dimensional reconstruction method and device of multi-eye camera, VR camera and panoramic camera | |
| Gao et al. | Atmospheric turbulence removal using convolutional neural network | |
| EP3296952B1 (en) | Method and device for blurring a virtual object in a video | |
| CN105635588B (en) | A kind of digital image stabilization method and device | |
| US8411980B1 (en) | Removing motion blur from unaligned multiple blurred images | |
| CN111192308B (en) | Image processing method and device, electronic equipment and computer storage medium | |
| CN106910208A (en) | A kind of scene image joining method that there is moving target | |
| KR101839617B1 (en) | Method and apparatus for removing non-uniform motion blur using multiframe | |
| CN110660098A (en) | Positioning method and device based on monocular vision | |
| CN114742866A (en) | Image registration method and device, storage medium and electronic equipment | |
| CN108073909A (en) | Method and apparatus, computer installation and the storage medium of the fuzzy facial image of synthesis | |
| US9838572B2 (en) | Method and device for determining movement between successive video images | |
| Zhang et al. | Combining inertial measurements with blind image deblurring using distance transform | |
| CN110060208B (en) | Method for improving reconstruction performance of super-resolution algorithm | |
| CN116883235A (en) | Distributed photoelectric oriented image stitching method and device | |
| CN107093165A (en) | The fast display method and device of a kind of recursive image |
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| CF01 | Termination of patent right due to non-payment of annual fee | ||
| CF01 | Termination of patent right due to non-payment of annual fee | Granted publication date:20180626 |