CN112529792A - Distortion correction method for distortion-free model camera - Google Patents

Abstract

Translated fromChinese

本发明公开了一种免畸变模型的相机畸变校正方法，首先计算机生成纵向和横向两组标准条纹图像作为校正模板并在平板显示器显示，相机拍摄这些图像，获得畸变图像；然后分别对两组畸变图像进行相移相位求解和相位解包裹，获得畸变图像的相位分布；获取畸变图像的瞬时空间频率分布，确定畸变中心条纹瞬时频率；接着提取畸变中心附近连续多点的畸变图像相位值，进行线性拟合，获取无畸变图像的相位值；将畸变图像与无畸变图像的相位分布相减，得到畸变图像畸变相位分布，再获得畸变图像畸变量分布；再获取校正坐标系下的畸变量分布后和校正映射图谱；最终利用校正映射图谱对相机拍摄的图像进行畸变校正。本发明无需畸变模型即可校正畸变相机图像。

The invention discloses a camera distortion correction method for a distortion-free model. First, a computer generates two sets of vertical and horizontal standard fringe images as correction templates and displays them on a flat panel display. The camera shoots these images to obtain distortion images; The image is phase-shifted and phase-unwrapped to obtain the phase distribution of the distorted image; the instantaneous spatial frequency distribution of the distorted image is obtained to determine the instantaneous frequency of the fringe at the center of the distortion; Fit to obtain the phase value of the undistorted image; subtract the phase distribution of the distorted image and the undistorted image to obtain the distorted image distortion phase distribution, and then obtain the distortion value distribution of the distorted image; and then obtain the distortion value distribution in the corrected coordinate system. and the correction map; finally, the correction map is used to correct the distortion of the image captured by the camera. The present invention can correct a distorted camera image without a distortion model.

Description

Translated fromChinese

一种免畸变模型的相机畸变校正方法A Distortion-Free Camera Distortion Correction Method

技术领域technical field

本发明涉及计算机视觉的畸变图像校正与光学测量技术领域，特别涉及一种免畸变模型的相机畸变校正方法。The invention relates to the technical field of computer vision distortion image correction and optical measurement, in particular to a distortion-free camera distortion correction method.

背景技术Background technique

计算机视觉是通过各种摄像镜头和设备捕获图像信息，对其进行分析处理，模仿人类实现对图像的辨识和场景的重建。为满足大视角的景物信息的捕获，具有宽视场成像特性的镜头被广泛应用于各个领域。但是利用宽视场镜头拍摄的图像存在各种程度的畸变，其中最为显著的是径向畸变。径向畸变是一种以畸变中心沿径向产生位置偏移，畸变导致无法精准确定图像中物体的几何位置和尺寸，甚至在图像识别过程中造成误判。因此，畸变图像的精确校准对于任何涉及计算机视觉任务的技术都是至关重要的，是保证计算机视觉后续各种处理任务顺利进行的必要条件。目前国内外对径向畸变的检测与估算的方法主要可以分为传统的视觉测量校正和机器学习校正。Computer vision is to capture image information through various camera lenses and equipment, analyze and process it, and imitate humans to realize image recognition and scene reconstruction. In order to meet the capture of scene information with a large viewing angle, lenses with wide-field imaging characteristics are widely used in various fields. However, images captured with wide-field lenses have various degrees of distortion, the most prominent of which is radial distortion. Radial distortion is a kind of positional offset along the radial direction with the center of distortion. The distortion leads to the inability to accurately determine the geometric position and size of the object in the image, and even causes misjudgment in the image recognition process. Therefore, accurate calibration of distorted images is crucial for any technology involving computer vision tasks, and is a necessary condition to ensure the smooth progress of various subsequent processing tasks in computer vision. At present, the methods of detecting and estimating radial distortion at home and abroad can be mainly divided into traditional vision measurement correction and machine learning correction.

传统的视觉测量校正方法可分为三类。第一类是模板测量法，通过对一个已知的模板进行畸变参量的测量以实现图像校正，如直线检测，特征点检测和平面模板等。第二类是多视图校正法，通过不同角度畸变图像中的点对应关系来计算畸变参量的，该方法无需特定的模板，可实现自动校正，但需要进行多角度拍摄。第三类是铅垂线算法，根据直线畸变圆弧的特征进行畸变参量的检测，该方法可通过单幅畸变图像实现校正，但需要在原视场中存在一定量的直线结构。另外，由于图像中的一些不可靠圆弧及非畸变圆弧会导致检测的错误，因此该方法常需要进行人工干预。这些方法均是建立在一定的畸变模型上，通过对有限的特征点或者特征线进行检测，实现对特定畸变模型的畸变参量的估算。Traditional vision measurement correction methods can be divided into three categories. The first type is the template measurement method, which realizes image correction by measuring the distortion parameters of a known template, such as line detection, feature point detection and plane template. The second type is the multi-view correction method, which calculates the distortion parameters through the correspondence of points in the distorted images at different angles. This method does not require a specific template and can achieve automatic correction, but requires multi-angle shooting. The third type is the plumb line algorithm, which detects the distortion parameters according to the characteristics of the straight line distortion arc. This method can achieve correction through a single distorted image, but requires a certain amount of straight line structures in the original field of view. In addition, since some unreliable arcs and non-distorted arcs in the image can cause detection errors, this method often requires manual intervention. These methods are all based on a certain distortion model, and realize the estimation of the distortion parameters of a specific distortion model by detecting limited feature points or feature lines.

机器学习的校正方法是基于卷积神经网络对畸变图像进行学习，然后基于某一特定畸变模型对畸变参量进行估算；或对多种不同畸变模型形成的畸变图像综合训练集进行学习，获得畸变位移量的分布，从而实现畸变图像的校正。但是，在机器学习的方法中，基于某一特定畸变模型训练的网络推广至其它畸变模型下的图像校正会出现较大的错误。多畸变模型下形成的图像训练集在一定程度上改善了网络的推广应用。The correction method of machine learning is to learn the distorted image based on the convolutional neural network, and then estimate the distortion parameter based on a specific distortion model; The distribution of the amount, so as to achieve the correction of the distorted image. However, in the method of machine learning, a network trained based on a specific distortion model is extended to image correction under other distortion models, and there will be large errors. The image training set formed under the multi-distortion model improves the generalization and application of the network to a certain extent.

但是，目前还没有任何一种畸变模型可以满足所有的摄像镜头的畸变，而且在实际的镜头制作与相机安装中，还可能存在非圆心对称的畸变。这类基于特定圆心对称畸变模型的校正方法对于未知类别摄像镜头的校正是存在缺陷的。However, there is currently no distortion model that can satisfy all the distortions of the camera lens, and in the actual lens fabrication and camera installation, there may also be non-centrosymmetric distortions. Such correction methods based on a specific center-symmetric distortion model are defective for the correction of unknown types of camera lenses.

发明内容SUMMARY OF THE INVENTION

本发明的目的在于提出一种不依赖于畸变模型的相机畸变校正方法，以克服目前利用畸变模型进行相机畸变校正的缺陷。The purpose of the present invention is to propose a camera distortion correction method that does not depend on the distortion model, so as to overcome the defect of using the distortion model to correct the camera distortion at present.

本发明的技术方案如下：一种免畸变模型的相机畸变校正方法，包括以下步骤：The technical scheme of the present invention is as follows: a distortion-free camera distortion correction method, comprising the following steps:

S1、获得校正模板和校正模板的畸变图像：利用计算机生成纵向和横向两组无畸变的标准正弦分布条纹图像，将这两组图像作为校正模板并依次在平板显示器显示；利用待校正相机依次拍摄这些无畸变图像，获得纵向和横向两组畸变图像；S1. Obtain the correction template and the distortion image of the correction template: use a computer to generate two sets of standard sinusoidal distribution fringe images without distortion in the vertical and horizontal directions, use the two sets of images as the correction templates and display them on the flat panel display in turn; use the cameras to be corrected in turn Take these undistorted images to obtain two sets of vertical and horizontal distorted images;

S2、获取畸变图像的相位分布：分别对两组畸变图像进行相移相位求解和相位解包裹，获得纵向和横向两个方向的畸变图像的相位分布；S2. Obtain the phase distribution of the distorted image: perform phase shift phase solution and phase unwrapping on the two sets of distorted images respectively, and obtain the phase distribution of the distorted images in the vertical and horizontal directions;

S3、确定畸变图像的畸变中心和畸变中心的瞬时频率：由纵向和横向畸变图像的相位分布分别获得纵向和横向畸变图像的瞬时空间频率分布，找出瞬时频率的最小值位置，由该最小值位置确定畸变图像的畸变中心及该畸变中心对应的瞬时频率；S3. Determine the distortion center of the distorted image and the instantaneous frequency of the distortion center: obtain the instantaneous spatial frequency distribution of the vertical and horizontal distorted images respectively from the phase distributions of the vertical and horizontal distorted images, and find the minimum position of the instantaneous frequency. The position determines the distortion center of the distorted image and the instantaneous frequency corresponding to the distortion center;

S4、获取无畸变图像的相位分布：提取畸变中心附近连续多点的畸变图像相位值，对这些选取点的相位值进行线性拟合，从而获得畸变中心的相位和通过畸变中心位置的纵向和横向方向所有点的相位值，根据这些相位值计算纵向和横向无畸变图像所有点的相位值；S4. Obtain the phase distribution of the undistorted image: extract the phase values of the distorted image at multiple consecutive points near the distortion center, and perform linear fitting on the phase values of these selected points, so as to obtain the phase of the distortion center and the vertical and horizontal directions passing through the position of the distortion center. The phase values of all points in the direction, and the phase values of all points of the vertical and horizontal undistorted images are calculated according to these phase values;

S5、获取畸变图像畸变量分布：将纵向和横向两个方向的畸变图像的相位分布分别与纵向和横向无畸变图像的相位分布相减，得到纵向和横向畸变图像的畸变相位分布，根据这些畸变相位分布计算畸变图像的畸变量分布；S5. Obtain the distortion amount distribution of the distorted image: subtract the phase distribution of the distorted image in the vertical and horizontal directions from the phase distribution of the vertical and horizontal undistorted images, respectively, to obtain the distortion phase distribution of the vertical and horizontal distorted images. According to these distortions The phase distribution calculates the distortion amount distribution of the distorted image;

S6、获取校正坐标下的畸变量分布和校正映射图谱：根据畸变图像的畸变位置坐标以及其与畸变中心的欧氏距离，结合畸变图像畸变量分布，计算得到校正位置与畸变中心的欧氏距离以及相应的校正位置坐标；将畸变图像畸变量分布作为校正坐标下的畸变量分布，根据校正位置与畸变中心的欧氏距离和校正坐标下的畸变量分布获得校正映射图谱；S6. Obtain the distribution of the distortion amount and the correction map under the correction coordinates: According to the distortion position coordinates of the distorted image and the Euclidean distance from the distortion center, combined with the distortion value distribution of the distorted image, calculate the Euclidean distance between the correction position and the distortion center and the corresponding correction position coordinates; take the distortion value distribution of the distorted image as the distortion value distribution under the correction coordinates, and obtain the correction map according to the Euclidean distance between the correction position and the distortion center and the distortion value distribution under the correction coordinates;

S7、利用校正映射图谱对相机拍摄的相机图像进行畸变校正，获取畸变校正后的相机图像。S7. Perform distortion correction on the camera image captured by the camera by using the correction map to obtain a camera image after distortion correction.

优选的，待校正相机依次拍摄校正模板图像时，相机的成像透镜光轴和平板显示器的图像显示平面垂直；Preferably, when the camera to be calibrated sequentially shoots the calibration template images, the optical axis of the imaging lens of the camera is perpendicular to the image display plane of the flat panel display;

纵向和横向两组校正模板的空间周期一致，每一组包含相位差为δ₀的多幅无畸变图像，表示为：The spatial periods of the vertical and horizontal correction templates are the same, and each group contains multiple undistorted images with a phase difference of δ₀ , which is expressed as:

纵向和横向两组畸变图像均包含相位差为δ₀的多幅图像，表示为：Both vertical and horizontal distorted images contain multiple images with a phase difference of δ₀ , which are expressed as:

和

and

其中，(x_d,y_d)表示相机拍摄的畸变图像的位置坐标；(x,y)表示平板显示器显示的无畸变图像坐标；a₀表示标准正弦分布条纹的背景光强；b₀表示标准正弦分布条纹的调制幅度；f₀′表示标准正弦分布条纹图像的初始频率；a(x_d,y_d)表示畸变条纹的背景光强；b(x_d,y_d)表示畸变条纹的调制幅度；f₀表示畸变条纹的载频频率；φ_x(x_d,y_d)和φ_y(x_d,y_d)分别表示纵向畸变图像和横向畸变图像的初始相位；a₀、b₀、a(x_d,y_d)、b(x_d,y_d)和f₀′、f₀均为大于零的实数，δ₀为实数；Among them, (x_d , y_d ) represents the position coordinates of the distorted image captured by the camera; (x, y) represents the coordinates of the undistorted image displayed by the flat panel display; a₀ represents the background light intensity of standard sinusoidal distribution stripes; b₀ represents the standard The modulation amplitude of the sinusoidal distribution fringes; f₀ ′ represents the initial frequency of the standard sinusoidal distribution fringe image; a(x_d , y_d ) represents the background light intensity of the distorted fringes; b(x_d , y_d ) represents the modulation amplitude of the distorted fringes ; f₀ represents the carrier frequency of the distorted fringes; φ_x (x_d , y_d ) and φ_y (x_d , y_d ) represent the initial phases of the longitudinally distorted image and the transversely distorted image, respectively; a₀ , b₀ , a (x_d , y_d ), b(x_d , y_d ) and f₀ ′, f₀ are all real numbers greater than zero, and δ₀ is a real number;

在步骤S2中，纵向畸变图像的相位分布为：

横向畸变图像的相位分布为：

In step S2, the phase distribution of the longitudinally distorted image is:

The phase distribution of the laterally distorted image is:

更进一步的，步骤S1中的相位差δ₀为π/2，纵向和横向两组四幅校正模板的畸变图像如下：Further, the phase difference δ₀ in step S1 is π/2, and the distortion images of the two groups of four vertical and horizontal correction templates are as follows:

和

and

在步骤S2中，利用四步相移公式获取获得畸变图像对应的包裹在(-π,π)内的相位分布：In step S2, the phase distribution wrapped in (-π, π) corresponding to the obtained distorted image is obtained by using the four-step phase shift formula:

和

and

然后利用解包裹算法对

和

进行解包裹处理，获得纵向和横向畸变图像的相位分布。Then use the unwrapping algorithm to

and

An unwrapping process is performed to obtain the phase distribution of the longitudinally and laterally distorted images.

更进一步的，步骤S3的畸变中心和畸变瞬时频率确定过程如下：Further, the determination process of the distortion center and the distortion instantaneous frequency in step S3 is as follows:

S31、分别沿x方向和y方向对相位分布

和

求偏导，获得纵向和横向畸变图像的瞬时空间频率分布：

和

S31, distribute the phase along the x direction and the y direction respectively

and

Find the partial derivative to obtain the instantaneous spatial frequency distribution of the longitudinally and laterally distorted images:

and

S32、根据瞬时频率的分布状况找到瞬时频率的最小值位置，由该最小值位置确定畸变图像的畸变中心(x₀,y₀)；S32, find the minimum value position of the instantaneous frequency according to the distribution of the instantaneous frequency, and determine the distortion center (x₀ , y₀ ) of the distorted image by the minimum value position;

S33、计算畸变中心的畸变瞬时频率：

其中，

S33. Calculate the distortion instantaneous frequency of the distortion center:

in,

更进一步的，步骤S32具体为：根据瞬时频率的分布状况找到瞬时频率的最小值位置，将该最小值位置作为畸变图像的畸变中心(x₀,y₀)。Further, step S32 is specifically: find the minimum value position of the instantaneous frequency according to the distribution of the instantaneous frequency, and use the minimum value position as the distortion center (x₀ , y₀ ) of the distorted image.

更进一步的，步骤S32具体为：分别提取图像中心所在行的纵向畸变图像的瞬时空间频率分布和图像中心所在列的横向畸变图像的瞬时空间频率分布，并采用二次项多项式分别对两组相应数据进行拟合，得到两条拟合曲线，然后分别检测两条拟合曲线的最小值位置来获得畸变中心所在的列即x₀和所在的行即y₀，由此确定畸变中心坐标(x₀,y₀)。Further, step S32 is specifically as follows: respectively extracting the instantaneous spatial frequency distribution of the longitudinally distorted image in the row where the image center is located and the instantaneous spatial frequency distribution of the horizontally distorted image in the column where the image center is located, and using quadratic term polynomials for the corresponding two groups. The data is fitted to obtain two fitted curves, and then the minimum position of the two fitted curves is detected to obtain the column where the distortion center is located, namely x₀ and the row where it is located, namely y₀ , thereby determining the coordinates of the distortion center (x₀ ,y₀ ).

更进一步的，步骤S4的无畸变图像的相位分布获取过程如下：Further, the phase distribution acquisition process of the undistorted image in step S4 is as follows:

S41、分别沿x方向和y方向，提取畸变中心位置(x₀,y₀)附近2N+1个连续点的

和

值：

和

其中，N为整数；S41 , along the x direction and the y direction respectively, extract the 2N+1 continuous points near the distortion center position (x₀ , y₀ )

and

value:

and

Among them, N is an integer;

S42、对这些选取点的相位值进行线性拟合，获得通过畸变中心位置(x₀,y₀)的x方向和y方向的所有点的相位分布：

和

S42, perform linear fitting on the phase values of these selected points, and obtain the phase distribution of all points in the x-direction and y-direction passing through the distortion center position (x₀ , y₀ ):

and

S43、再获取纵向和横向无畸变图像所有点的相位值：S43, then obtain the phase values of all points of the vertical and horizontal undistorted images:

其中，

和

分别为

和

的取值，

和

分别为纵向和横向畸变图像位相

的取值；in,

and

respectively

and

value of ,

and

are the vertical and horizontal distorted image phases, respectively

value of ;

步骤S5的畸变图像畸变量分布获取过程如下：The process of obtaining the distortion value distribution of the distorted image in step S5 is as follows:

将纵向和横向两个方向的畸变图像的相位分布分别与纵向和横向无畸变图像的相位分布相减，得到纵向和横向的畸变图像畸变相位分布：

和

Subtract the phase distributions of the distorted images in the vertical and horizontal directions from the phase distributions of the vertical and horizontal undistorted images, respectively, to obtain the distorted phase distributions of the vertical and horizontal distorted images:

and

再获得所有方向的畸变相位分布：

那么畸变图像畸变量分布为：

Then obtain the distorted phase distribution in all directions:

Then the distortion distribution of the distorted image is:

更进一步的，步骤S6的校正坐标下的畸变量分布和校正映射图谱获取过程如下：Further, the process of obtaining the distortion variable distribution and the correction map at the correction coordinates in step S6 is as follows:

S61、对于每一点畸变坐标(x_d,y_d)所对应的畸变量分布Δr(x_d,y_d)，首先根据畸变位置和校正位置与畸变中心间的欧氏距离关系r_u＝r_d+Δr(x_d,y_d)，计算每一点畸变位置对应的校正位置与畸变中心间的欧氏距离r_u；S61. For the distortion variable distribution Δr(x_d , y_d ) corresponding to the distortion coordinates (x_d , y_d ) of each point, first, according to the Euclidean distance relationship between the distortion position and the correction position and the distortion center_ru =r_d +Δr(x_d , y_d ), calculate the Euclidean distance r_u between the correction position corresponding to the distortion position of each point and the distortion center;

S62、然后根据坐标转换关系

计算对应的校正位置坐标(x_u,y_u)，建立校正位置坐标(x_u,y_u)所对应的畸变量分布Δr(x_u,y_u)＝Δr(x_d,y_d)，其中

S62, and then convert the relationship according to the coordinates

Calculate the corresponding correction position coordinates (x_u , y_u ), and establish the distortion variable distribution Δr(x_u , y_u ) corresponding to the correction position coordinates (x_u , y_u ) = Δr(x_d , y_d ), where

S63、如果计算所得的校正位置坐标(x_u,y_u)为整数值(m,n)，则畸变量分布Δr(x_u,y_u)＝Δr(m,n)，欧氏距离r_u为r_u(m,n)，计算并建立校正映射图谱

S63. If the calculated corrected position coordinates (x_u , y_u ) are integer values (m, n), then the distribution of distortion variables Δr(x_u , y_u )=Δr(m, n), the Euclidean distance r_u is r_u (m,n), calculate and build the calibration map

如果计算所得的校正位置坐标(x_u,y_u)为非整数值，则对Δr(x_u,y_u)进行插值计算，获得校正位置坐标为整数值(m,n)所对应的畸变量分布Δr(m,n)，计算并建立校正映射图谱

其中，(m,n)是校正整数值位置坐标，m、n为整数，r_u(m,n)为校正整数值位置坐标(m,n)与畸变中心间的欧氏距离，

为与校正整数值位置坐标(m,n)对应的校正前畸变位置坐标与畸变中心间的欧氏距离。If the calculated correction position coordinates (x_u , y_u ) are non-integer values, perform interpolation calculation on Δr (x_u , y_u ) to obtain the distortion value corresponding to the correction position coordinates as integer values (m, n). Distribution Δr(m,n), calculate and build calibration map

Among them, (m,n) is the corrected integer value position coordinate, m, n are integers, r_u (m,n) is the Euclidean distance between the corrected integer value position coordinate (m,n) and the distortion center,

is the Euclidean distance between the pre-correction distortion position coordinates corresponding to the corrected integer-valued position coordinates (m, n) and the distortion center.

更进一步的，在步骤S7中，利用校正映射图谱对相机拍摄的相机图像进行畸变校正的过程如下：Further, in step S7, the process of performing distortion correction on the camera image captured by the camera using the correction map is as follows:

S71、利用步骤S6获得的校正映射图谱

根据坐标转换关系

计算校正图像中整数值位置坐标(m,n)在校正前所对应的相机图像的畸变位置坐标

S71, using the calibration map obtained in step S6

According to the coordinate transformation relationship

Calculate the distortion position coordinates of the camera image corresponding to the integer value position coordinates (m, n) in the corrected image before correction

S72、将畸变位置坐标

的像素值

赋给校正图像整数值位置坐标(m,n)的像素值I_u(m,n)，即

S72. Set the distortion position coordinates

pixel value of

The pixel value I_u (m, n) assigned to the integer value position coordinate (m, n) of the corrected image, namely

其中，如果这里计算所得的畸变位置坐标

为非整数值，则利用计算所得的畸变位置坐标

周围的整数点位置的相机图像像素值，进行插值得到

的像素值，然后再对校正图像整数值位置坐标(m,n)的像素值进行赋值；Among them, if the distortion position coordinates calculated here are

is a non-integer value, use the calculated distortion position coordinates

The pixel value of the camera image around the integer point position, which is obtained by interpolation

The pixel value of , and then assign the pixel value of the integer value position coordinate (m, n) of the corrected image;

S73、按照上述赋值过程遍历校正图像上所有坐标点位置，获得畸变校正后相机图像。S73 , traverse all coordinate point positions on the corrected image according to the above assignment process, and obtain a camera image after distortion correction.

优选的，平板显示器为液晶平板显示器，或有机发光二极管平板显示器，或等离子体平板显示器。Preferably, the flat panel display is a liquid crystal flat panel display, or an organic light emitting diode flat panel display, or a plasma flat panel display.

本发明提供的技术方案与现有利用畸变模型的相机畸变校正技术方案相比，其显著优点包括：Compared with the existing technical solution for camera distortion correction using the distortion model, the technical solution provided by the present invention has the following significant advantages:

(1)畸变图像中所有点的畸变量都可以通过测量获得，无需依赖任何的畸变模型，从而避免了精确获取畸变模型参数的困难。(1) The distortion value of all points in the distorted image can be obtained by measurement without relying on any distortion model, thus avoiding the difficulty of accurately obtaining the parameters of the distortion model.

(2)根据测量模板载频条纹的瞬时频率分布特征可以自动获取畸变中心位置，从而避免了由于畸变中心和畸变图像中心不重合导致的校正误差，提高了校正精度。(2) The position of the distortion center can be automatically obtained according to the instantaneous frequency distribution characteristics of the carrier frequency stripe of the measurement template, thereby avoiding the correction error caused by the misalignment of the distortion center and the center of the distorted image, and improving the correction accuracy.

(3)本发明构建的校正映射图谱可应用于各类畸变相机图像的畸变校正，可见，本发明方法的适用范围广。(3) The correction map constructed by the present invention can be applied to the distortion correction of various distorted camera images. It can be seen that the method of the present invention has a wide application range.

附图说明Description of drawings

图1为本发明免畸变模型的相机畸变校正方法的流程图。FIG. 1 is a flow chart of a camera distortion correction method for a distortion-free model according to the present invention.

图2为纵向和横向条纹校正模板的畸变图像。Figure 2 is a distorted image of the vertical and horizontal stripe correction templates.

图3为采用四步相移分析得到的包裹相位分布图，其中，(a)图为纵向条纹校正模板的包裹相位图，(b)图为横向条纹校正模板的包裹相位图。Figure 3 shows the wrapped phase distribution obtained by the four-step phase shift analysis, in which (a) is the wrapped phase map of the vertical stripe correction template, and (b) is the wrapped phase map of the horizontal stripe correction template.

图4为畸变图像的相位分布图，其中，(a)图为纵向畸变图像的相位分布图，Figure 4 is a phase distribution diagram of a distorted image, wherein (a) is a phase distribution diagram of a longitudinally distorted image,

(b)图为横向畸变图像的相位分布图。(b) is the phase distribution of the laterally distorted image.

图5(a)为图像中心所在的第1024行的纵向条纹瞬时频率分布图。Figure 5(a) is a graph of the instantaneous frequency distribution of the vertical stripes in the 1024th row where the center of the image is located.

图5(b)为图像中心所在的第1224列的横向条纹瞬时频率分布图。Figure 5(b) is the instantaneous frequency distribution diagram of the horizontal fringes in the 1224th column where the center of the image is located.

图6为无畸变条纹图像相位分布图，其中，(a)图为纵向无畸变条纹图像相位分布图，(b)图为横向无畸变条纹图像相位分布图。FIG. 6 is a phase distribution diagram of an undistorted fringe image, wherein (a) is a longitudinal undistorted fringe image phase distribution diagram, and (b) is a horizontal undistorted fringe image phase distribution diagram.

图7为畸变相位分布图。Fig. 7 is a distortion phase distribution diagram.

图8为畸变量分布图，其中，(a)图为畸变坐标下畸变量分布Δr(x_d,y_d)的示意图；(b)图为校正坐标下畸变量Δr(m,n)的示意图。Fig. 8 is a distribution diagram of the distortion amount, wherein, (a) is a schematic diagram of the distortion amount distribution Δr(x_d , y_d ) in the distortion coordinates; (b) is a schematic diagram of the distortion amount Δr(m, n) in the corrected coordinates .

图9为相机图像畸变校正的对比图，其中，(a)图为校正前的相机图像，(b)Figure 9 is a comparison diagram of camera image distortion correction, wherein (a) is the camera image before correction, (b)

图为校正后的相机图像。Pictured is the corrected camera image.

具体实施方式Detailed ways

为了使本发明的目的、技术方案及优点更加清楚明白，以下结合附图及实施例，对本发明内容进一步详细说明。应当理解，此处所描述的具体实施例仅用以解释本发明，并不用于限定本发明。In order to make the objectives, technical solutions and advantages of the present invention clearer, the content of the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention, but not to limit the present invention.

实施例Example

本实施例公开了一种免畸变模型的相机畸变校正方法，包括以下步骤：This embodiment discloses a distortion-free camera distortion correction method, which includes the following steps:

S1、获得校正模板和校正模板的畸变图像：S1. Obtain the correction template and the distorted image of the correction template:

利用计算机生成纵向和横向两组无畸变的标准正弦分布条纹图像，纵向和横向代表条纹方向，将这两组图像作为校正模板并依次在平板显示器显示；利用待校正相机依次拍摄这些无畸变图像，获得纵向和横向两组畸变图像。Use a computer to generate two sets of undistorted standard sinusoidal fringe images in the vertical and horizontal directions. The vertical and horizontal directions represent the fringe directions. These two sets of images are used as correction templates and displayed on the flat panel display in turn; these undistorted images are sequentially captured by the camera to be corrected. , two sets of distorted images in vertical and horizontal directions are obtained.

这里，平板显示器可以采用液晶平板显示器，或有机发光二极管平板显示器，或等离子体平板显示器。待校正相机依次拍摄校正模板图像时，相机的成像透镜光轴和平板显示器的图像显示平面垂直。纵向和横向两组校正模板的空间周期一致，每一组包含相位差为δ₀的多幅无畸变图像，表示如下(等式左边均表示无畸变图像)：Here, the flat panel display may adopt a liquid crystal flat panel display, or an organic light emitting diode flat panel display, or a plasma flat panel display. When the camera to be calibrated sequentially shoots calibration template images, the optical axis of the imaging lens of the camera is perpendicular to the image display plane of the flat panel display. The spatial periods of the vertical and horizontal correction templates are the same, and each group contains multiple undistorted images with a phase difference of δ₀ , which is expressed as follows (the left side of the equation represents undistorted images):

和

and

纵向和横向两组畸变图像均包含相位差为δ₀的多幅图像，表示如下(等式左边均表示畸变图像)：Both vertical and horizontal two sets of distorted images contain multiple images with a phase difference of δ₀ , which are expressed as follows (the left side of the equation represents the distorted image):

和

and

其中，(x_d,y_d)表示相机拍摄的畸变图像的位置坐标；(x,y)表示平板显示器显示的无畸变图像坐标；a₀表示标准正弦分布条纹的背景光强；b₀表示标准正弦分布条纹的调制幅度；f₀′表示标准正弦分布条纹图像的初始频率；a(x_d,y_d)表示畸变条纹的背景光强；b(x_d,y_d)表示畸变条纹的调制幅度；f₀表示畸变条纹的载频频率；φ_x(x_d,y_d)和φ_y(x_d,y_d)分别表示纵向畸变图像和横向畸变图像的初始相位；a₀、b₀、a(x_d,y_d)、b(x_d,y_d)和f₀′、f₀均为大于零的实数，δ₀为实数。Among them, (x_d , y_d ) represents the position coordinates of the distorted image captured by the camera; (x, y) represents the coordinates of the undistorted image displayed by the flat panel display; a₀ represents the background light intensity of standard sinusoidal distribution stripes; b₀ represents the standard The modulation amplitude of the sinusoidal distribution fringes; f₀ ′ represents the initial frequency of the standard sinusoidal distribution fringe image; a(x_d , y_d ) represents the background light intensity of the distorted fringes; b(x_d , y_d ) represents the modulation amplitude of the distorted fringes ; f₀ represents the carrier frequency of the distorted fringes; φ_x (x_d , y_d ) and φ_y (x_d , y_d ) represent the initial phases of the longitudinally distorted image and the transversely distorted image, respectively; a₀ , b₀ , a (x_d , y_d ), b(x_d , y_d ), f₀ ′, f₀ are all real numbers greater than zero, and δ₀ is a real number.

S2、获取畸变图像的相位分布：S2. Obtain the phase distribution of the distorted image:

分别对两组畸变图像进行相移相位求解和相位解包裹，获得纵向方向的畸变图像的相位分布：

以及获得横向方向的畸变图像的相位分布：

The phase shift phase solution and phase unwrapping are performed on the two sets of distorted images respectively, and the phase distribution of the distorted images in the longitudinal direction is obtained:

and obtain the phase distribution of the distorted image in the lateral direction:

在本实施例中，步骤S1中的相位差δ₀设为π/2，因此，纵向和横向两组四幅校正模板的畸变图像如下：In this embodiment, the phase difference δ₀ in step S1 is set to π/2, therefore, the distorted images of the two sets of four vertical and horizontal correction templates are as follows:

和

and

在步骤S2中，具体是利用四步相移公式获取获得畸变图像对应的包裹在(-π,π)内的相位分布：In step S2, the phase distribution wrapped in (-π, π) corresponding to the obtained distorted image is obtained by using the four-step phase shift formula:

和

and

然后利用解包裹算法对

和

进行解包裹处理，获得纵向和横向畸变图像的相位分布。Then use the unwrapping algorithm to

and

An unwrapping process is performed to obtain the phase distribution of the longitudinally and laterally distorted images.

S3、确定畸变图像的畸变中心和畸变中心的瞬时频率：S3. Determine the distortion center of the distorted image and the instantaneous frequency of the distortion center:

S31、分别沿x方向和y方向对相位分布

和

求偏导，获得纵向和横向畸变图像的瞬时空间频率分布：

和

S31, distribute the phase along the x direction and the y direction respectively

and

Find the partial derivative to obtain the instantaneous spatial frequency distribution of the longitudinally and laterally distorted images:

and

S32、根据瞬时频率的分布状况找到瞬时频率的最小值位置，由该最小值位置确定畸变图像的畸变中心(x₀,y₀)；S32, find the minimum value position of the instantaneous frequency according to the distribution of the instantaneous frequency, and determine the distortion center (x₀ , y₀ ) of the distorted image by the minimum value position;

其中，可以是直接将该最小值位置作为畸变图像的畸变中心(x₀,y₀)，也可以是先分别提取图像中心所在行的纵向畸变图像的瞬时空间频率分布和图像中心所在列的横向畸变图像的瞬时空间频率分布，并采用二次项多项式分别对两组相应数据进行拟合，得到两条拟合曲线，然后分别检测两条拟合曲线的最小值位置来获得畸变中心所在的列即x₀和所在的行即y₀，由此确定畸变中心坐标(x₀,y₀)。Among them, the position of the minimum value can be directly used as the distortion center (x₀ , y₀ ) of the distorted image, or the instantaneous spatial frequency distribution of the longitudinally distorted image in the row where the image center is located and the horizontal direction of the column where the image center is located can be extracted separately. The instantaneous spatial frequency distribution of the distorted image, and the quadratic term polynomial is used to fit the two sets of corresponding data to obtain two fitting curves, and then detect the minimum position of the two fitting curves to obtain the column where the distortion center is located. That is, x₀ and the row where it is located are y₀ , thereby determining the coordinates of the distortion center (x₀ , y₀ ).

S33、计算畸变中心的畸变瞬时频率：

其中，

f₀即是图像基频。S33. Calculate the distortion instantaneous frequency of the distortion center:

in,

f₀ is the fundamental frequency of the image.

S4、获取无畸变图像的相位分布：S4. Obtain the phase distribution of the undistorted image:

S41、分别沿x方向和y方向，提取畸变中心位置(x₀,y₀)附近2N+1个连续点的

和

值：

和

其中，N为整数；S41 , along the x direction and the y direction respectively, extract the 2N+1 continuous points near the distortion center position (x₀ , y₀ )

and

value:

and

Among them, N is an integer;

S42、对这些选取点的相位值进行线性拟合，获得通过畸变中心位置(x₀,y₀)的x方向和y方向的所有点的相位分布：

和

S42, perform linear fitting on the phase values of these selected points, and obtain the phase distribution of all points in the x-direction and y-direction passing through the distortion center position (x₀ , y₀ ):

and

S43、再获取纵向和横向无畸变图像所有点的相位值：S43, then obtain the phase values of all points of the vertical and horizontal undistorted images:

其中，

和

分别为

和

的取值，

和

分别为纵向和横向畸变图像位相

的取值。in,

and

respectively

and

value of ,

and

are the vertical and horizontal distorted image phases, respectively

value of .

S5、获取畸变图像畸变量分布：S5. Obtain the distortion value distribution of the distorted image:

将纵向和横向两个方向的畸变图像的相位分布分别与纵向和横向无畸变图像的相位分布相减，得到纵向和横向的畸变图像畸变相位分布：

和

Subtract the phase distributions of the distorted images in the vertical and horizontal directions from the phase distributions of the vertical and horizontal undistorted images, respectively, to obtain the distorted phase distributions of the vertical and horizontal distorted images:

and

再获得所有方向的畸变相位分布：

那么畸变图像畸变量分布为：

Then obtain the distorted phase distribution in all directions:

Then the distortion distribution of the distorted image is:

S6、获取校正坐标下的畸变量分布和校正映射图谱：S6. Obtain the distortion variable distribution and the correction mapping map under the correction coordinates:

S61、对于每一点畸变坐标(x_d,y_d)所对应的畸变量分布Δr(x_d,y_d)，首先根据畸变位置和校正位置与畸变中心间的欧氏距离关系r_u＝r_d+Δr(x_d,y_d)，计算每一点畸变位置对应的校正位置与畸变中心间的欧氏距离r_u；S61. For the distortion variable distribution Δr(x_d , y_d ) corresponding to the distortion coordinates (x_d , y_d ) of each point, first, according to the Euclidean distance relationship between the distortion position and the correction position and the distortion center_ru =r_d +Δr(x_d , y_d ), calculate the Euclidean distance r_u between the correction position corresponding to the distortion position of each point and the distortion center;

S62、然后根据坐标转换关系

计算对应的校正位置坐标(x_u,y_u)，建立校正位置坐标(x_u,y_u)所对应的畸变量分布Δr(x_u,y_u)＝Δr(x_d,y_d)，其中

S62, and then convert the relationship according to the coordinates

Calculate the corresponding correction position coordinates (x_u , y_u ), and establish the distortion variable distribution Δr(x_u , y_u ) corresponding to the correction position coordinates (x_u , y_u ) = Δr(x_d , y_d ), where

S63、如果计算所得的校正位置坐标(x_u,y_u)为整数值(m,n)，则畸变量分布Δr(x_u,y_u)＝Δr(m,n)，欧氏距离r_u为r_u(m,n)，计算并建立校正映射图谱

S63. If the calculated corrected position coordinates (x_u , y_u ) are integer values (m, n), then the distribution of distortion variables Δr(x_u , y_u )=Δr(m, n), the Euclidean distance r_u is r_u (m,n), calculate and build the calibration map

如果计算所得的校正位置坐标(x_u,y_u)为非整数值，则对Δr(x_u,y_u)进行插值计算，获得校正位置坐标为整数值(m,n)所对应的畸变量分布Δr(m,n)，计算并建立校正映射图谱

其中，(m,n)是校正整数值位置坐标，m、n为整数，r_u(m,n)为校正整数值位置坐标(m,n)与畸变中心间的欧氏距离，

为与校正整数值位置坐标(m,n)对应的校正前畸变位置坐标与畸变中心间的欧氏距离。If the calculated correction position coordinates (x_u , y_u ) are non-integer values, perform interpolation calculation on Δr (x_u , y_u ) to obtain the distortion value corresponding to the correction position coordinates as integer values (m, n). Distribution Δr(m,n), calculate and build calibration map

Among them, (m,n) is the corrected integer value position coordinate, m, n are integers, r_u (m,n) is the Euclidean distance between the corrected integer value position coordinate (m,n) and the distortion center,

is the Euclidean distance between the pre-correction distortion position coordinates corresponding to the corrected integer-valued position coordinates (m, n) and the distortion center.

S7、利用校正映射图谱对相机拍摄的相机图像进行畸变校正，获取畸变校正后的相机图像：S7. Use the correction map to perform distortion correction on the camera image captured by the camera, and obtain the camera image after distortion correction:

S71、利用步骤S6获得的校正映射图谱

根据坐标转换关系

计算校正图像中整数值位置坐标(m,n)在校正前所对应的相机图像的畸变位置坐标

S71, using the calibration map obtained in step S6

According to the coordinate transformation relationship

Calculate the distortion position coordinates of the camera image corresponding to the integer value position coordinates (m, n) in the corrected image before correction

S72、将畸变位置坐标

的像素值

赋给校正图像整数值位置坐标(m,n)的像素值I_u(m,n)，即

S72. Set the distortion position coordinates

pixel value of

The pixel value I_u (m, n) assigned to the integer value position coordinate (m, n) of the corrected image, namely

其中，如果这里计算所得的畸变位置坐标

为非整数值，则利用计算所得的畸变位置坐标

周围的整数点位置的相机图像像素值，进行插值得到

的像素值，然后再对校正图像整数值位置坐标(m,n)的像素值进行赋值；Among them, if the distortion position coordinates calculated here are

is a non-integer value, use the calculated distortion position coordinates

The pixel value of the camera image around the integer point position, which is obtained by interpolation

The pixel value of , and then assign the pixel value of the integer value position coordinate (m, n) of the corrected image;

S73、按照上述赋值过程遍历校正图像上所有坐标点位置，获得畸变校正后相机图像。S73 , traverse all coordinate point positions on the corrected image according to the above assignment process, and obtain a camera image after distortion correction.

为更好地描述本实施例，下面以一具体实例加以说明。In order to better describe this embodiment, a specific example is used for description below.

利用本发明对一个装有视角为137度广角镜头的相机(2048×2448像素，每个像素3.45μm)进行畸变校正，如图1所示。Distortion correction is performed on a camera (2048×2448 pixels, each pixel 3.45 μm) equipped with a wide-angle lens with a viewing angle of 137 degrees by using the present invention, as shown in FIG. 1 .

步骤一、计算机生成纵向和横向两组相移δ₀为π/2的标准正弦分布条纹图像，并在一个49英寸平板液晶显示器上显示，显示的标准正弦分布条纹图像作为校正模板(也可称为测量模板)。Step 1. The computer generates two sets of standard sinusoidal distribution fringe images with a vertical and horizontal phase shift δ₀ of π/2, and displays them on a 49-inch flat-panel liquid crystal display. The displayed standard sinusoidal distribution fringe image is used as a correction template (also called for the measurement template).

设置待畸变校正相机的成像透镜光轴和平板显示器的图像显示平面垂直，然后通过相机拍摄液晶平板显示器显示的条纹图像，获得两组大小为2048×2448像数的畸变图像，如图2所示，位于上方的一组为纵向条纹校正模板所对应的纵向畸变图像，位于下方的一组为横向条纹校正模板所对应的横向畸变图像。每一组包括四幅初相位分别为0、π/2、π、3π/2的图像。Set the optical axis of the imaging lens of the camera to be distortion corrected to be perpendicular to the image display plane of the flat panel display, and then shoot the fringe image displayed by the LCD flat panel display with the camera to obtain two sets of 2048×2448 images of distortion, as shown in Figure 2 , the upper group is the longitudinally distorted image corresponding to the vertical stripe correction template, and the lower group is the horizontally distorted image corresponding to the horizontal stripe correction template. Each group includes four images with initial phases of 0, π/2, π, and 3π/2, respectively.

步骤二、分别对纵向和横向两组相移条纹图像进行相移相位分析，根据四步相移公式获取包裹在范围(-π,π)的纵向和横向畸变图像对应的相位分布

和

如图3所示，左边的(a)图为纵向畸变图像的包裹相位分布，右边的(b)图为横向畸变图像的包裹相位分布。Step 2: Perform phase-shift phase analysis on the longitudinal and lateral phase-shift fringe images respectively, and obtain the phase distribution corresponding to the longitudinal and lateral distortion images wrapped in the range (-π, π) according to the four-step phase-shift formula

and

As shown in Figure 3, (a) on the left is the wrapped phase distribution of the longitudinally distorted image, and (b) on the right is the wrapped phase distribution of the laterally distorted image.

然后根据解包裹算法，扫描相邻的像素点，检验相邻点的包裹相位的差值，当差值在范围中时(-π,π)，相位值不变；当差值大于π时，相位值减去2π的整数倍；当差值小于-π，相位值加上2π的整数倍，直到得到的连续的相位值，从而获得纵向和横向畸变图像相位分布

和

如图4的(a)和(b)图所示，其中，

和

Then according to the unwrapping algorithm, scan the adjacent pixels to check the difference of the wrapping phase of the adjacent points. When the difference is in the range (-π, π), the phase value remains unchanged; when the difference is greater than π, The phase value is subtracted from an integer multiple of 2π; when the difference value is less than -π, the phase value is added to an integer multiple of 2π until a continuous phase value is obtained, so as to obtain the phase distribution of longitudinal and lateral distortion images

and

As shown in (a) and (b) of Figure 4, where,

and

步骤三、分别沿x方向和y方向对相位

和

求偏导，得到瞬时频率

和

Step 3. Align the phases along the x-direction and the y-direction respectively

and

Find the partial derivative to get the instantaneous frequency

and

考虑实验拍摄畸变图像的畸变中心落在图像的中心区域附近，因此以图像的中心位置点所在的行列进行分析。分别提取图像中心所在的第1024行的纵向畸变图像的瞬时空间频率分布和图像中心所在的第1224列的横向畸变图像的瞬时空间频率分布，并采用二次项多项式分别对两组相应数据进行拟合，得到如图5(a)和图5(b)所示的瞬时空间频率分布的拟合曲线。Considering that the distortion center of the experimentally captured distorted image is located near the center area of the image, the analysis is performed based on the row and column where the center point of the image is located. Extract the instantaneous spatial frequency distribution of the longitudinally distorted image in the 1024th row where the image center is located and the instantaneous spatial frequency distribution of the horizontally distorted image in the 1224th column where the image center is located, and use quadratic term polynomials to fit the two sets of corresponding data respectively. Combined, the fitting curves of the instantaneous spatial frequency distribution as shown in Figure 5(a) and Figure 5(b) are obtained.

检测第1024行的纵向畸变图像的瞬时空间频率分布的拟合曲线的最小值位置，获得畸变中心所在列为第1224列，如图5(a)所示；检测第1224列的横向畸变图像的瞬时空间频率分布的拟合曲线的最小值位置获得畸变中心所在行为第1008行，如图5(b)所示，即畸变中心坐标为(1224,1008)。Detect the minimum position of the fitting curve of the instantaneous spatial frequency distribution of the longitudinally distorted image in the 1024th row, and obtain the column where the distortion center is located in the 1224th column, as shown in Figure 5(a); The minimum position of the fitting curve of the instantaneous spatial frequency distribution obtains the row 1008 where the distortion center is located, as shown in Figure 5(b), that is, the coordinates of the distortion center are (1224, 1008).

根据畸变中心的畸变瞬时频率计算公式

其中，

和

即可计算出该畸变中心点(1224,1008)的畸变条纹瞬时频率为基频值f₀＝0.0133。According to the calculation formula of the distortion instantaneous frequency of the distortion center

in,

and

It can be calculated that the instantaneous frequency of the distortion fringes at the distortion center point (1224, 1008) is the fundamental frequency value f₀ =0.0133.

步骤四、分别沿x方向和y方向，提取畸变中心位置(1224,1008)附近2N+1,(N＝12)个连续点的

和

值：

和

Step 4: Extract the 2N+1, (N=12) consecutive points near the distortion center position (1224, 1008) along the x and y directions respectively.

and

value:

and

对这些选取点的相位值进行线性拟合，获得通过畸变中心位置(1224,1008)的x方向和y方向所有点的相位分布

和

再获取纵向和横向无畸变图像所有点的相位值

和

如图6中的(a)和(b)图所示。Perform linear fitting on the phase values of these selected points to obtain the phase distribution of all points in the x-direction and y-direction through the distortion center position (1224, 1008)

and

Then obtain the phase values of all points in the vertical and horizontal undistorted images

and

As shown in (a) and (b) of Figure 6 .

步骤五、将纵向和横向两个方向的畸变图像的相位分布分别与纵向和横向无畸变图像的相位分布相减，得到纵向和横向的畸变图像畸变相位分布：

和

再获得所有方向的畸变相位分布：

如图7所示。根据

获得畸变坐标下的畸变量分布，如图8中的(a)图所示。Step 5. Subtract the phase distributions of the distorted images in the vertical and horizontal directions from the phase distributions of the vertical and horizontal undistorted images, respectively, to obtain the distortion phase distributions of the vertical and horizontal distorted images:

and

Then obtain the distorted phase distribution in all directions:

As shown in Figure 7. according to

The distortion amount distribution under the distortion coordinates is obtained, as shown in (a) in Fig. 8 .

步骤六、对于每一点畸变坐标(x_d,y_d)所对应的畸变量分布Δr(x_d,y_d)，首先根据畸变位置和校正位置与畸变中心间的欧氏距离关系r_u＝r_d+Δr(x_d,y_d)，计算每一点畸变位置对应的校正位置与畸变中心间的欧氏距离r_u；Step 6. For the distortion variable distribution Δr(x_d , y_d ) corresponding to the distortion coordinates (x_d , y_d ) of each point, first, according to the Euclidean distance relationship between the distortion position and the correction position and the distortion center r_u =r_d +Δr(x_d , y_d ), calculate the Euclidean distance r_u between the correction position corresponding to the distortion position of each point and the distortion center;

然后根据坐标转换关系

计算对应的校正位置坐标(x_u,y_u)；建立校正坐标点(x_u,y_u)所对应的畸变量分布Δr(x_u,y_u)＝Δr(x_d,y_d)，其中

Then convert the relationship according to the coordinates

Calculate the corresponding corrected position coordinates (x_u , y_u ); establish the distortion variable distribution Δr(x_u , y_u ) corresponding to the corrected coordinate points (x_u , y_u ) = Δr(x_d , y_d ), where

如果计算所得的校正坐标点(x_u,y_u)为非整数值，对Δr(x_u,y_u)进行插值计算，获得校正位置坐标为整数值(m,n)所对应的畸变量分布Δr(m,n)，计算并获得校正映射图谱

其中，(m,n)是校正整数值位置坐标，m、n为整数，r_u(m,n)为校正整数值位置坐标(m,n)与畸变中心间的欧氏距离，

为与校正整数值位置坐标(m,n)对应的校正前畸变位置坐标与畸变中心间的欧氏距离，这里采用的是双三次样条插值算法。If the calculated correction coordinate point (x_u , y_u ) is a non-integer value, perform interpolation calculation on Δr(x_u , y_u ) to obtain the distortion value distribution corresponding to the integer value (m, n) of the corrected position coordinate Δr(m,n), calculate and obtain the corrected map

Among them, (m,n) is the corrected integer value position coordinate, m, n are integers, r_u (m,n) is the Euclidean distance between the corrected integer value position coordinate (m,n) and the distortion center,

is the Euclidean distance between the pre-correction distortion position coordinate and the distortion center corresponding to the corrected integer-valued position coordinate (m, n), and the bicubic spline interpolation algorithm is used here.

以畸变图像第600行第500列像素点为例。对于畸变图像第600行第500列像素点，该畸变位置点坐标为(500，600)，与畸变中心位置点(1224，1008)的欧氏距离r_d＝831.05像素，计算所得的畸变量为Δr＝83.40像素。因此有，校正坐标下畸变量为Δr＝83.40像素所对应校正点与畸变中心的欧氏距离为r_u＝r_d+Δr＝914.45像素，通过坐标转换得到对应的校正位置坐标点为(427.3，559.1)；通过插值算法分别得到相邻四个整点位置的畸变量分别为“83.48像素(427，559)”，“83.19像素(427，560)”，“83.27像素(428，559)”和“81.68像素(428，560)”，坐标为校正整数值位置坐标值，坐标前的数值为校正整数值位置坐标所对应的畸变量分布，即Δr(m,n)。Δr(m,n)如图8中的(b)图所示。由

即可得到校正映射图谱

Take the pixel point of the 600th row and 500th column of the distorted image as an example. For the pixel point in the 600th row and 500th column of the distorted image, the coordinates of the distortion position point are (500, 600), and the Euclidean distance r_d = 831.05 pixels from the distortion center position point (1224, 1008), the calculated distortion value is Δr=83.40 pixels. Therefore, under the correction coordinates, the distortion amount is Δr=83.40 pixels, and the Euclidean distance between the corresponding correction point and the distortion center is r_u =r_d +Δr=914.45 pixels, and the corresponding correction position coordinate point obtained by coordinate transformation is (427.3, 559.1); through the interpolation algorithm, the distortion values of the four adjacent integer point positions are obtained as "83.48 pixels (427, 559)", "83.19 pixels (427, 560)", "83.27 pixels (428, 559)" and "81.68 pixels (428, 560)", the coordinates are the coordinates of the corrected integer value position, and the value before the coordinates is the distortion distribution corresponding to the corrected integer value position coordinates, that is, Δr(m,n). Δr(m,n) is shown in (b) of FIG. 8 . Depend on

The calibration map can be obtained

步骤七、相机拍摄的相机图像如图9中的(a)图所示。利用上述步骤获得的校正映射图谱

根据坐标转换关系

计算校正图像上整数值位置坐标(m,n)在校正前所对应的相机图像上的位置坐标

将

的像素值

赋给校正图像整数值位置坐标(m,n)的像素值I_u(m,n)，即

Step 7. The camera image captured by the camera is shown in (a) of FIG. 9 . Correction map obtained using the above steps

According to the coordinate transformation relationship

Calculate the position coordinates on the camera image corresponding to the integer value position coordinates (m, n) on the corrected image before the correction

Will

pixel value of

The pixel value I_u (m, n) assigned to the integer value position coordinate (m, n) of the corrected image, namely

如果这里计算所得的畸变坐标点

为非整数值，利用计算所得的畸变图像位置

周围的整数点位置的相机图像像素值，进行插值得到非整数位置

点的像素值，然后再对校正图像整数值位置坐标(m,n)的像素值进行赋值；If the distortion coordinate point calculated here

is a non-integer value, using the calculated distorted image position

Camera image pixel values around integer point positions, interpolated to get non-integer positions

The pixel value of the point, and then assign the pixel value of the integer value position coordinate (m, n) of the corrected image;

遍历校正图像上所有坐标点位置，获得畸变校正后的相机图像，如图9中的(b)图所示。这里的是采用双线性插值算法。Traverse the positions of all coordinate points on the corrected image to obtain a camera image after distortion correction, as shown in (b) in Figure 9 . Here is the use of bilinear interpolation algorithm.

可通过各种手段实施本发明描述的技术。举例来说，这些技术可实施在硬件、固件、软件或其组合中。对于硬件实施方案，处理模块可实施在一个或一个以上专用集成电路(ASIC)、数字信号处理器(DSP)、可编程逻辑装置(PLD)、现场可编辑逻辑门阵列(FPGA)、处理器、控制器、微控制器、电子装置、其他经设计以执行本发明所描述的功能的电子单元或其组合内。The techniques described herein can be implemented by various means. For example, these techniques may be implemented in hardware, firmware, software, or a combination thereof. For a hardware implementation, a processing module may be implemented in one or more Application Specific Integrated Circuits (ASICs), Digital Signal Processors (DSPs), Programmable Logic Devices (PLDs), Field Programmable Logic Gate Arrays (FPGAs), processors, Within a controller, microcontroller, electronic device, other electronic unit designed to perform the functions described herein, or a combination thereof.

对于固件和/或软件实施方案，可用执行本文描述的功能的模块(例如，过程、步骤、流程等)来实施所述技术。固件和/或软件代码可存储在存储器中并由处理器执行。存储器可实施在处理器内或处理器外部。For firmware and/or software implementations, the techniques may be implemented in modules (eg, procedures, steps, flows, etc.) that perform the functions described herein. Firmware and/or software codes may be stored in memory and executed by a processor. The memory may be implemented within the processor or external to the processor.

本领域普通技术人员可以理解：实现上述方法实施例的全部或部分步骤可以通过程序指令相关的硬件来完成，前述的程序可以存储在一计算机可读取存储介质中，该程序在执行时，执行包括上述方法实施例的步骤；而前述的存储介质包括：ROM、RAM、磁碟或者光盘等各种可以存储程序代码的介质。Those of ordinary skill in the art can understand that all or part of the steps of implementing the above method embodiments may be completed by program instructions related to hardware, the aforementioned program may be stored in a computer-readable storage medium, and when the program is executed, execute It includes the steps of the above method embodiments; and the aforementioned storage medium includes: ROM, RAM, magnetic disk or optical disk and other media that can store program codes.

以上所述仅为本发明的较佳实施例而已，并不用以限制本发明，凡在本发明的精神和原则之内所作的任何修改、等同替换和改进等，均应包含在本发明的保护范围之内。The above descriptions are only preferred embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent replacements and improvements made within the spirit and principles of the present invention shall be included in the protection of the present invention. within the range.

Claims (10)

Translated fromChinese

1.一种免畸变模型的相机畸变校正方法，其特征在于，包括以下步骤：1. a camera distortion correction method of a distortion-free model, is characterized in that, comprises the following steps:S1、获得校正模板和校正模板的畸变图像：利用计算机生成纵向和横向两组无畸变的标准正弦分布条纹图像，将这两组图像作为校正模板并依次在平板显示器显示；利用待校正相机依次拍摄这些无畸变图像，获得纵向和横向两组畸变图像；S1. Obtain the correction template and the distortion image of the correction template: use a computer to generate two sets of standard sinusoidal distribution fringe images without distortion in the vertical and horizontal directions, use the two sets of images as the correction templates and display them on the flat panel display in turn; use the cameras to be corrected in turn Take these undistorted images to obtain two sets of vertical and horizontal distorted images;S2、获取畸变图像的相位分布：分别对两组畸变图像进行相移相位求解和相位解包裹，获得纵向和横向两个方向的畸变图像的相位分布；S2. Obtain the phase distribution of the distorted image: perform phase shift phase solution and phase unwrapping on the two sets of distorted images respectively, and obtain the phase distribution of the distorted images in the vertical and horizontal directions;S3、确定畸变图像的畸变中心和畸变中心的瞬时频率：由纵向和横向畸变图像的相位分布分别获得纵向和横向畸变图像的瞬时空间频率分布，找出瞬时频率的最小值位置，由该最小值位置确定畸变图像的畸变中心及该畸变中心对应的瞬时频率；S3. Determine the distortion center of the distorted image and the instantaneous frequency of the distortion center: obtain the instantaneous spatial frequency distribution of the vertical and horizontal distorted images respectively from the phase distributions of the vertical and horizontal distorted images, and find the minimum position of the instantaneous frequency. The position determines the distortion center of the distorted image and the instantaneous frequency corresponding to the distortion center;S4、获取无畸变图像的相位分布：提取畸变中心附近连续多点的畸变图像相位值，对这些选取点的相位值进行线性拟合，从而获得畸变中心的相位和通过畸变中心位置的纵向和横向方向所有点的相位值，根据这些相位值计算纵向和横向无畸变图像所有点的相位值；S4. Obtain the phase distribution of the undistorted image: extract the phase values of the distorted image at multiple consecutive points near the distortion center, and perform linear fitting on the phase values of these selected points, so as to obtain the phase of the distortion center and the vertical and horizontal directions passing through the position of the distortion center. The phase values of all points in the direction, and the phase values of all points of the vertical and horizontal undistorted images are calculated according to these phase values;S5、获取畸变图像畸变量分布：将纵向和横向两个方向的畸变图像的相位分布分别与纵向和横向无畸变图像的相位分布相减，得到纵向和横向畸变图像的畸变相位分布，根据这些畸变相位分布计算畸变图像的畸变量分布；S5. Obtain the distortion amount distribution of the distorted image: subtract the phase distribution of the distorted image in the vertical and horizontal directions from the phase distribution of the vertical and horizontal undistorted images, respectively, to obtain the distortion phase distribution of the vertical and horizontal distorted images. According to these distortions The phase distribution calculates the distortion amount distribution of the distorted image;S6、获取校正坐标下的畸变量分布和校正映射图谱：根据畸变图像的畸变位置坐标以及其与畸变中心的欧氏距离，结合畸变图像畸变量分布，计算得到校正位置与畸变中心的欧氏距离以及相应的校正位置坐标；将畸变图像畸变量分布作为校正坐标下的畸变量分布，根据校正位置与畸变中心的欧氏距离和校正坐标下的畸变量分布获得校正映射图谱；S6. Obtain the distribution of the distortion amount and the correction map under the correction coordinates: According to the distortion position coordinates of the distorted image and the Euclidean distance from the distortion center, combined with the distortion value distribution of the distorted image, calculate the Euclidean distance between the correction position and the distortion center and the corresponding correction position coordinates; take the distortion value distribution of the distorted image as the distortion value distribution under the correction coordinates, and obtain the correction map according to the Euclidean distance between the correction position and the distortion center and the distortion value distribution under the correction coordinates;S7、利用校正映射图谱对相机拍摄的相机图像进行畸变校正，获取畸变校正后的相机图像。S7. Perform distortion correction on the camera image captured by the camera by using the correction map to obtain a camera image after distortion correction.2.根据权利要求1所述的免畸变模型的相机畸变校正方法，其特征在于：待校正相机依次拍摄校正模板图像时，相机的成像透镜光轴和平板显示器的图像显示平面垂直；2. The camera distortion correction method of the distortion-free model according to claim 1, characterized in that: when the camera to be corrected shoots the correction template images in turn, the imaging lens optical axis of the camera is perpendicular to the image display plane of the flat panel display;纵向和横向两组校正模板的空间周期一致，每一组包含相位差为δ₀的多幅无畸变图像，表示为：The spatial periods of the vertical and horizontal correction templates are the same, and each group contains multiple undistorted images with a phase difference of δ₀ , which is expressed as:

纵向和横向两组畸变图像均包含相位差为δ₀的多幅图像，表示为：Both vertical and horizontal distorted images contain multiple images with a phase difference of δ₀ , which are expressed as:

和

and

其中，(x_d,y_d)表示相机拍摄的畸变图像的位置坐标；(x,y)表示平板显示器显示的无畸变图像坐标；a₀表示标准正弦分布条纹的背景光强；b₀表示标准正弦分布条纹的调制幅度；f₀′表示标准正弦分布条纹图像的初始频率；a(x_d,y_d)表示畸变条纹的背景光强；b(x_d,y_d)表示畸变条纹的调制幅度；f₀表示畸变条纹的载频频率；φ_x(x_d,y_d)和φ_y(x_d,y_d)分别表示纵向畸变图像和横向畸变图像的初始相位；a₀、b₀、a(x_d,y_d)、b(x_d,y_d)和f₀′、f₀均为大于零的实数，δ₀为实数；Among them, (x_d , y_d ) represents the position coordinates of the distorted image captured by the camera; (x, y) represents the coordinates of the undistorted image displayed by the flat panel display; a₀ represents the background light intensity of standard sinusoidal distribution stripes; b₀ represents the standard The modulation amplitude of the sinusoidal distribution fringes; f₀ ′ represents the initial frequency of the standard sinusoidal distribution fringe image; a(x_d , y_d ) represents the background light intensity of the distorted fringes; b(x_d , y_d ) represents the modulation amplitude of the distorted fringes ; f₀ represents the carrier frequency of the distorted fringes; φ_x (x_d , y_d ) and φ_y (x_d , y_d ) represent the initial phases of the longitudinally distorted image and the transversely distorted image, respectively; a₀ , b₀ , a (x_d , y_d ), b(x_d , y_d ) and f₀ ′, f₀ are all real numbers greater than zero, and δ₀ is a real number;在步骤S2中，纵向畸变图像的相位分布为：

横向畸变图像的相位分布为：

In step S2, the phase distribution of the longitudinally distorted image is:

The phase distribution of the laterally distorted image is:

3.根据权利要求2所述的免畸变模型的相机畸变校正方法，其特征在于：步骤S1中的相位差δ₀为π/2，纵向和横向两组四幅校正模板的畸变图像如下：3. the camera distortion correction method of the distortion-free model according to claim 2, is characterized in that: the phase difference δ₀ in the step S1 is π/2, and the distortion images of two groups of vertical and horizontal correction templates are as follows:

和

and

在步骤S2中，利用四步相移公式获取获得畸变图像对应的包裹在(-π,π)内的相位分布：In step S2, the phase distribution wrapped in (-π, π) corresponding to the obtained distorted image is obtained by using the four-step phase shift formula:

和

and

然后利用解包裹算法对

和

进行解包裹处理，获得纵向和横向畸变图像的相位分布。Then use the unwrapping algorithm to

and

An unwrapping process is performed to obtain the phase distribution of the longitudinally and laterally distorted images.4.根据权利要求2所述的免畸变模型的相机畸变校正方法，其特征在于：步骤S3的畸变中心和畸变瞬时频率确定过程如下：4. The camera distortion correction method of the distortion-free model according to claim 2, is characterized in that: the determination process of the distortion center and distortion instantaneous frequency of step S3 is as follows:S31、分别沿x方向和y方向对相位分布

和

求偏导，获得纵向和横向畸变图像的瞬时空间频率分布：

和

S31, distribute the phase along the x direction and the y direction respectively

and

Find the partial derivative to obtain the instantaneous spatial frequency distribution of the longitudinally and laterally distorted images:

and

S32、根据瞬时频率的分布状况找到瞬时频率的最小值位置，由该最小值位置确定畸变图像的畸变中心(x₀,y₀)；S32, find the minimum value position of the instantaneous frequency according to the distribution of the instantaneous frequency, and determine the distortion center (x₀ , y₀ ) of the distorted image by the minimum value position;S33、计算畸变中心的畸变瞬时频率：

其中，

S33. Calculate the distortion instantaneous frequency of the distortion center:

in,

5.根据权利要求4所述的免畸变模型的相机畸变校正方法，其特征在于：步骤S32具体为：根据瞬时频率的分布状况找到瞬时频率的最小值位置，将该最小值位置作为畸变图像的畸变中心(x₀,y₀)。5. The camera distortion correction method for a distortion-free model according to claim 4, wherein step S32 is specifically: find the minimum value position of the instantaneous frequency according to the distribution of the instantaneous frequency, and use the minimum value position as the value of the distorted image. Distortion center (x₀ , y₀ ).6.根据权利要求4所述的免畸变模型的相机畸变校正方法，其特征在于：步骤S32具体为：分别提取图像中心所在行的纵向畸变图像的瞬时空间频率分布和图像中心所在列的横向畸变图像的瞬时空间频率分布，并采用二次项多项式分别对两组相应数据进行拟合，得到两条拟合曲线，然后分别检测两条拟合曲线的最小值位置来获得畸变中心所在的列即x₀和所在的行即y₀，由此确定畸变中心坐标(x₀,y₀)。6. The camera distortion correction method for a distortion-free model according to claim 4, wherein step S32 is specifically: extracting the instantaneous spatial frequency distribution of the longitudinally distorted image of the row where the image center is located and the horizontal distortion of the column where the image center is located. The instantaneous spatial frequency distribution of the image, and the quadratic term polynomial is used to fit the two sets of corresponding data to obtain two fitting curves, and then detect the minimum position of the two fitting curves to obtain the column where the distortion center is located, namely The row where x₀ is located is y₀ , thereby determining the coordinates of the distortion center (x₀ , y₀ ).7.根据权利要求4所述的免畸变模型的相机畸变校正方法，其特征在于：步骤S4的无畸变图像的相位分布获取过程如下：7. The camera distortion correction method of the distortion-free model according to claim 4, wherein: the phase distribution acquisition process of the distortion-free image in step S4 is as follows:S41、分别沿x方向和y方向，提取畸变中心位置(x₀,y₀)附近2N+1个连续点的

和

值：

和

其中，N为整数；S41 , along the x direction and the y direction respectively, extract the 2N+1 continuous points near the distortion center position (x₀ , y₀ )

and

value:

and

Among them, N is an integer;S42、对这些选取点的相位值进行线性拟合，获得通过畸变中心位置(x₀,y₀)的x方向和y方向的所有点的相位分布：

和

S42, perform linear fitting on the phase values of these selected points, and obtain the phase distribution of all points in the x-direction and y-direction passing through the distortion center position (x₀ , y₀ ):

and

S43、再获取纵向和横向无畸变图像所有点的相位值：S43, then obtain the phase values of all points of the vertical and horizontal undistorted images:

其中，

和

分别为

和

的取值，

和

分别为纵向和横向畸变图像位相

的取值；in,

and

respectively

and

value of ,

and

are the vertical and horizontal distorted image phases, respectively

value of ;步骤S5的畸变图像畸变量分布获取过程如下：The process of obtaining the distortion value distribution of the distorted image in step S5 is as follows:将纵向和横向两个方向的畸变图像的相位分布分别与纵向和横向无畸变图像的相位分布相减，得到纵向和横向的畸变图像畸变相位分布：

和

Subtract the phase distributions of the distorted images in the vertical and horizontal directions from the phase distributions of the vertical and horizontal undistorted images, respectively, to obtain the distorted phase distributions of the vertical and horizontal distorted images:

and

再获得所有方向的畸变相位分布：

那么畸变图像畸变量分布为：

Then obtain the distorted phase distribution in all directions:

Then the distortion distribution of the distorted image is:

8.根据权利要求7所述的免畸变模型的相机畸变校正方法，其特征在于：步骤S6的校正坐标下的畸变量分布和校正映射图谱获取过程如下：8. The camera distortion correction method of the distortion-free model according to claim 7 is characterized in that: the distortion variable distribution under the correction coordinates of step S6 and the acquisition process of the correction map are as follows:S61、对于每一点畸变坐标(x_d,y_d)所对应的畸变量分布Δr(x_d,y_d)，首先根据畸变位置和校正位置与畸变中心间的欧氏距离关系r_u＝r_d+Δr(x_d,y_d)，计算每一点畸变位置对应的校正位置与畸变中心间的欧氏距离r_u；S61. For the distortion variable distribution Δr(x_d , y_d ) corresponding to the distortion coordinates (x_d , y_d ) of each point, first, according to the Euclidean distance relationship between the distortion position and the correction position and the distortion center_ru =r_d +Δr(x_d , y_d ), calculate the Euclidean distance r_u between the correction position corresponding to the distortion position of each point and the distortion center;S62、然后根据坐标转换关系

计算对应的校正位置坐标(x_u,y_u)，建立校正位置坐标(x_u,y_u)所对应的畸变量分布Δr(x_u,y_u)＝Δr(x_d,y_d)，其中

S62, and then convert the relationship according to the coordinates

Calculate the corresponding correction position coordinates (x_u , y_u ), and establish the distortion variable distribution Δr(x_u , y_u ) corresponding to the correction position coordinates (x_u , y_u ) = Δr(x_d , y_d ), where

S63、如果计算所得的校正位置坐标(x_u,y_u)为整数值(m,n)，则畸变量分布Δr(x_u,y_u)＝Δr(m,n)，欧氏距离r_u为r_u(m,n)，计算并建立校正映射图谱

S63. If the calculated corrected position coordinates (x_u , y_u ) are integer values (m, n), then the distribution of distortion variables Δr(x_u , y_u )=Δr(m, n), the Euclidean distance r_u is r_u (m,n), calculate and build the calibration map

如果计算所得的校正位置坐标(x_u,y_u)为非整数值，则对Δr(x_u,y_u)进行插值计算，获得校正位置坐标为整数值(m,n)所对应的畸变量分布Δr(m,n)，计算并建立校正映射图谱

其中，(m,n)是校正整数值位置坐标，m、n为整数，r_u(m,n)为校正整数值位置坐标(m,n)与畸变中心间的欧氏距离，

为与校正整数值位置坐标(m,n)对应的校正前畸变位置坐标与畸变中心间的欧氏距离。If the calculated correction position coordinates (x_u , y_u ) are non-integer values, perform interpolation calculation on Δr (x_u , y_u ) to obtain the distortion value corresponding to the correction position coordinates as integer values (m, n). Distribution Δr(m,n), calculate and build calibration map

Among them, (m,n) is the corrected integer value position coordinate, m, n are integers, r_u (m,n) is the Euclidean distance between the corrected integer value position coordinate (m,n) and the distortion center,

is the Euclidean distance between the pre-correction distortion position coordinates corresponding to the corrected integer-valued position coordinates (m, n) and the distortion center.9.根据权利要求8所述的免畸变模型的相机畸变校正方法，其特征在于：在步骤S7中，利用校正映射图谱对相机拍摄的相机图像进行畸变校正的过程如下：9. The camera distortion correction method of the distortion-free model according to claim 8, characterized in that: in step S7, the process of performing distortion correction on the camera image captured by the camera using the correction map is as follows:S71、利用步骤S6获得的校正映射图谱

根据坐标转换关系

计算校正图像中整数值位置坐标(m,n)在校正前所对应的相机图像的畸变位置坐标

S71, using the calibration map obtained in step S6

According to the coordinate transformation relationship

Calculate the distortion position coordinates of the camera image corresponding to the integer value position coordinates (m, n) in the corrected image before correction

S72、将畸变位置坐标

的像素值

赋给校正图像整数值位置坐标(m,n)的像素值I_u(m,n)，即

S72. Set the distortion position coordinates

pixel value of

The pixel value I_u (m, n) assigned to the integer value position coordinate (m, n) of the corrected image, namely

其中，如果这里计算所得的畸变位置坐标

为非整数值，则利用计算所得的畸变位置坐标

周围的整数点位置的相机图像像素值，进行插值得到

的像素值，然后再对校正图像整数值位置坐标(m,n)的像素值进行赋值；Among them, if the distortion position coordinates calculated here are

is a non-integer value, use the calculated distortion position coordinates

The pixel value of the camera image around the integer point position, which is obtained by interpolation

The pixel value of , and then assign the pixel value of the integer value position coordinate (m, n) of the corrected image;S73、按照上述赋值过程遍历校正图像上所有坐标点位置，获得畸变校正后相机图像。S73 , traverse all coordinate point positions on the corrected image according to the above assignment process, and obtain a camera image after distortion correction.10.根据权利要求1所述的免畸变模型的相机畸变校正方法，其特征在于：平板显示器为液晶平板显示器，或有机发光二极管平板显示器，或等离子体平板显示器。10 . The camera distortion correction method for a distortion-free model according to claim 1 , wherein the flat panel display is a liquid crystal flat panel display, an organic light emitting diode flat panel display, or a plasma flat panel display. 11 .

Images