CN103530880B

Movatterモバイル変換

Info

Publication number: CN103530880B
Application number: CN201310482789.4A
Authority: CN
Inventors: 贾振元; 刘巍; 李明星; 刘阳; 杨景豪; 张驰
Original assignee: Dalian University of Technology
Current assignee: Dalian University of Technology
Priority date: 2013-10-16
Filing date: 2013-10-16
Publication date: 2016-04-06
Anticipated expiration: 2033-10-16
Also published as: CN103530880A

Abstract

本发明基于投影高斯网格图案的摄像机标定方法属于图像处理和计算机视觉检测领域，特别涉及大型锻件尺寸测量系统中摄像机内、外参数的现场标定方法。摄像机标定方法利用高斯网格图案中横、纵光条在宽度方向上的灰度呈高斯分布的特性，通过拟合高斯曲线可以高精度地获取光条中心线上的点的图像坐标，进而拟合出横、纵光条的中心线方程，横、纵光条中心线的交点即为标定特征点，依据拍摄得到的高斯网格图案的图像中提供的标定特征点的图像坐标，分步获取摄像机的内、外参数。本发明具有高的实时性、鲁棒性及较高的标定精度，分步标定可以获取高精度的摄像机参数，避免将所有摄像机参数同时求解时的耦合性问题，适用于锻造现场对摄像机进行在线标定。

The invention relates to a camera calibration method based on a projected Gaussian grid pattern, which belongs to the field of image processing and computer vision detection, and particularly relates to a method for on-site calibration of internal and external parameters of a camera in a size measurement system for large forgings. The camera calibration method utilizes the characteristics of the Gaussian distribution of the horizontal and vertical light bars in the width direction of the Gaussian grid pattern. By fitting the Gaussian curve, the image coordinates of the points on the center line of the light bar can be obtained with high precision, and then simulated. Combine the centerline equations of the horizontal and vertical light strips, and the intersection of the centerlines of the horizontal and vertical light strips is the calibration feature point, which is obtained step by step according to the image coordinates of the calibration feature points provided in the Gaussian grid pattern image obtained The internal and external parameters of the camera. The present invention has high real-time performance, robustness and high calibration accuracy, step-by-step calibration can obtain high-precision camera parameters, and avoids the coupling problem when solving all camera parameters at the same time, and is suitable for on-line monitoring of cameras at forging sites calibration.

Description

Translated fromChinese

基于投影高斯网格图案的摄像机标定方法Camera Calibration Method Based on Projected Gaussian Grid Pattern

技术领域technical field

本发明属于图像处理和计算机视觉检测领域，特别涉及大型锻件尺寸测量系统中摄像机内、外参数的现场标定方法。The invention belongs to the field of image processing and computer vision detection, in particular to a method for on-site calibration of internal and external parameters of a camera in a size measurement system for large forgings.

背景技术Background technique

计算机视觉处理的基本任务之一是根据二维图像信息恢复物体的三维几何信息。要实现利用图像点求取相应的空间物体表面点的任务，需要确定摄像机成像几何模型，该几何模型的参数称为摄像机参数。摄像机参数分为内、外参数，内参数为摄像机自身的与几何和光学特性有关的参数，外参数为摄像机相对于某一世界坐标系的三维位置和方向。确定摄像机内、外参数的过程称为摄像机标定，标定方法的精度直接影响到计算机视觉测量的精度。因此，对摄像机进行快速、简捷、精准标定的研究无疑具有重大的意义。One of the basic tasks of computer vision processing is to recover the 3D geometric information of objects from 2D image information. To achieve the task of using image points to obtain the corresponding surface points of space objects, it is necessary to determine the camera imaging geometric model, and the parameters of the geometric model are called camera parameters. The camera parameters are divided into internal and external parameters. The internal parameters are parameters related to the geometric and optical characteristics of the camera itself, and the external parameters are the three-dimensional position and direction of the camera relative to a certain world coordinate system. The process of determining the internal and external parameters of the camera is called camera calibration, and the accuracy of the calibration method directly affects the accuracy of computer vision measurement. Therefore, the research on fast, simple and accurate calibration of cameras is undoubtedly of great significance.

传统的摄像机标定方法按照标定参照物的不同可以分为基于3D立体靶标标定方法，基于2D平面靶标标定方法(以张正友提出的棋盘格靶标标定方法为代表)和基于1D靶标的标定方法。这些传统的标定方法均需要标定参照物，而对于大视场摄像机的标定，标定参照物的特征点能否均匀布满于整个视场，直接影响着标定的精度。一方面，制作高精度的大尺寸标定靶标造价昂贵、维护困难。另一方面，对于不适用于在线与不可能使用标定参照物的场合也不适用。在锻件锻造车间高温环境下，标定块、标定板与粘贴靶点的方法均不能应用。因此，传统摄像机标定方法不能满足大型锻件在线尺寸参数测量的要求。另外，自标定方法虽然不利用任何标定物，仅利用摄像机内参数自身存在的约束，根据图像间图像点的对应关系就能估计出摄像机内参数。该类方法操作上较为灵活，但是精度不太高，鲁棒性不足。According to different calibration reference objects, traditional camera calibration methods can be divided into calibration methods based on 3D stereo targets, calibration methods based on 2D plane targets (represented by the checkerboard target calibration method proposed by Zhang Zhengyou) and calibration methods based on 1D targets. These traditional calibration methods all require a calibration reference object, and for the calibration of a large field of view camera, whether the feature points of the calibration reference object can evenly cover the entire field of view directly affects the calibration accuracy. On the one hand, making large-scale calibration targets with high precision is expensive and difficult to maintain. On the other hand, it is not suitable for occasions where it is not suitable for online and it is not possible to use calibration reference objects. In the high temperature environment of the forging workshop, the methods of calibration block, calibration plate and sticking target points cannot be applied. Therefore, the traditional camera calibration method cannot meet the requirements of online dimensional parameter measurement of large forgings. In addition, although the self-calibration method does not use any calibration objects, it only uses the constraints of the internal parameters of the camera itself, and the internal parameters of the camera can be estimated according to the corresponding relationship between image points between images. This type of method is more flexible in operation, but the accuracy is not high and the robustness is insufficient.

通过采用投影仪投射靶标可以解决上述问题，投影的特征图案理论上边缘应为阶跃变化，但是实际上由于扩散效应，图案边缘呈渐变趋势，边缘会向黑色背景一侧偏移。以投影仪投射圆形特征光斑阵列为例，圆形光斑的中心即为用于标定的特征点，由于每个光斑会向周围产生不同程度的扩散，因此很难根据二值化后的图像采用形心法获取圆斑的精确中心，利用相关算法进行圆形特征边界提取并进行圆形(或椭圆形)拟合获得高精度的圆斑中心也并非易事。同理，利用投影仪投影一般的光条组合图案，提取光条的中心作为特征线，精度也难以保证。The above problems can be solved by using a projector to project the target. The edge of the projected characteristic pattern should be a step change in theory, but in fact, due to the diffusion effect, the edge of the pattern shows a gradual trend, and the edge will shift to the black background side. Take the circular feature spot array projected by the projector as an example. The center of the circular spot is the feature point used for calibration. Since each spot will diffuse to different degrees around it, it is difficult to use it according to the binarized image. The centroid method obtains the precise center of the circular spot, and it is not easy to use the relevant algorithm to extract the circular feature boundary and perform circular (or elliptical) fitting to obtain the high-precision circular spot center. In the same way, it is difficult to guarantee the accuracy of using a projector to project a general combination pattern of light stripes and extracting the center of the light stripes as a feature line.

发明内容Contents of the invention

本发明所要解决的技术问题是克服现有技术的不足，针对在锻造现场，传统的标定方法存在精度低、非实时，甚至不能应用以及自标定方法存在精度不太高，鲁棒性不足等问题，发明一种基于投影高斯网格图案的大视场摄像机标定方法，利用高斯网格图案中横、纵光条在宽度方向上的灰度呈高斯分布的特性，通过拟合高斯曲线可以高精度地获取光条中心线上的点的图像坐标，进而拟合出横、纵光条的中心线方程，横、纵光条中心线的交点即为标定特征点，依据拍摄得到的高斯网格图案的图像中提供的标定特征点的图像坐标，分步获取摄像机的内、外参数。The technical problem to be solved by the present invention is to overcome the deficiencies of the prior art. For the forging site, the traditional calibration method has low precision, non-real-time, and even cannot be applied, and the self-calibration method has problems such as low precision and insufficient robustness. , invented a large-field-of-view camera calibration method based on projected Gaussian grid patterns, using the characteristics of the Gaussian distribution of the horizontal and vertical light bars in the width direction of the Gaussian grid pattern, and by fitting Gaussian curves, high-precision Accurately obtain the image coordinates of the points on the centerline of the light strip, and then fit the centerline equation of the horizontal and vertical light strips. The intersection of the centerlines of the horizontal and vertical light strips is the calibration feature point. According to the Gaussian grid pattern obtained by shooting The image coordinates of the calibration feature points provided in the image, and the internal and external parameters of the camera are obtained step by step.

本发明采取的技术方案是一种基于投影高斯网格图案的摄像机标定方法，其特征在于，利用高斯网格图案中横、纵光条在宽度方向上的灰度呈高斯分布的特性，通过拟合高斯曲线可以高精度地获取光条中心线上的点的图像坐标，进而拟合出横、纵光条的中心线方程，横、纵光条中心线的交点即为标定特征点，依据拍摄得到的高斯网格图案的图像中提供的标定特征点的图像坐标，分步获取摄像机的内、外参数；具体步骤如下：The technical solution adopted by the present invention is a camera calibration method based on projected Gaussian grid pattern, which is characterized in that the gray scale of the horizontal and vertical light bars in the Gaussian grid pattern is Gaussian distributed in the width direction, through the simulated Gaussian grid pattern. The Gaussian curve can obtain the image coordinates of the points on the centerline of the light strip with high precision, and then fit the centerline equation of the horizontal and vertical light strips. The intersection point of the centerlines of the horizontal and vertical light strips is the calibration feature point. According to the shooting The image coordinates of the calibration feature points provided in the obtained Gaussian grid pattern image, and the internal and external parameters of the camera are obtained step by step; the specific steps are as follows:

步骤1：搭建摄像机标定系统。将左侧四维电控平台2a、右侧四维电控平台2b及投影仪3安装在平台1的台面上，将左侧摄像机4a固定在左侧四维电控平台2a上，将右侧摄像机4b固定在右侧四维电控平台2b上。Step 1: Build a camera calibration system. Install the left four-dimensional electric control platform 2a, the right four-dimensional electric control platform 2b and the projector 3 on the platform 1, fix the left camera 4a on the left four-dimensional electric control platform 2a, and fix the right camera 4b On the right four-dimensional electric control platform 2b.

步骤2：投影高斯网格图案、拍摄并获取交点坐标。通过投影仪3向厂房内光整的平板或墙面5上投影由多条平行的横光条与多条平行的纵光条组成高斯网格图案6，各光条在宽度方向上的灰度均呈高斯分布，其中横、纵光条交点A_i,j为标定特征点，i为横光条的编号，按照从上至下的次序，j为纵光条的编号，按照从左至右的次序。由于横、纵光条交点处光强叠加，对左侧摄像机4a与右侧摄像机4b拍摄得到的图像进行二值化处理后，获得的图像中只剩下网格交点处的亮斑，即一个个孤立的连通区域。利用形心法可以获取连通区域的质心坐标(u0_i,j,v0_i,j)，作为特征点A_i,j的粗略位置。将以该粗略位置为圆心，以Δ个像素为半径的圆形区域作为搜索范围，然后在[u0_i,j-Δ,u0_i,j+Δ]范围内每隔Δ/n沿宽度方向搜索一次横光条，按照高斯分布特性进行拟合，将高斯分布峰值点作为横光条中心线上的点，因此可以得到2n+1个中心线上的点P_i,j,s，下标s为1,2,3,…,2n+1，进而拟合出直线l_h,i,j。同样地，在[v0_i,j-Δ,v0_i,j+Δ]范围内每隔Δ/n沿宽度方向搜索一次纵光条，按照高斯分布特性进行拟合，将高斯分布峰值点作为纵光条中心线上的点，可以得到2n+1个中心线上的点Q_i,j,t，下标t为1,2,3,…,2n+1，进而拟合出直线l_v,i,j。最终，通过求取同一搜索范围内两条相交直线的交点作为标定特征点A_i,j，其坐标为(u_i,j,v_i,j)。Step 2: Project the Gaussian grid pattern, photograph and obtain the intersection coordinates. Through the projector 3, the Gaussian grid pattern 6 composed of multiple parallel horizontal light strips and multiple parallel vertical light strips is projected onto the smooth flat panel or wall 5 in the factory building. The gray scale of each light strip in the width direction is uniform. Gaussian distribution, where the horizontal and vertical light bar intersection A_i,j is the calibration feature point, i is the number of the horizontal light bar, in order from top to bottom, and j is the number of the vertical light bar, in the order from left to right . Due to the superposition of the light intensity at the intersection of the horizontal and vertical light strips, after binarizing the images captured by the left camera 4a and the right camera 4b, only the bright spot at the intersection of the grid remains in the obtained image, that is, a an isolated connected region. Using the centroid method, the centroid coordinates (u0_i,j ,v0_i,j ) of the connected area can be obtained as the rough position of the feature point A_i,j . Use the rough position as the center and a circular area with a radius of Δ pixels as the search range, and then search along the width direction every Δ/n within the range of [u0_i,j -Δ,u0_i,j +Δ] A horizontal light bar is fitted according to the Gaussian distribution characteristics, and the peak point of the Gaussian distribution is used as a point on the center line of the horizontal light bar, so 2n+1 points P_i,j,s on the center line can be obtained, and the subscript s is 1 ,2,3,…,2n+1, and then fit the straight line l_h,i,j . Similarly, within the range of [v0_i,j -Δ,v0_i,j +Δ], the vertical light strip is searched every Δ/n along the width direction, fitted according to the Gaussian distribution characteristics, and the peak point of the Gaussian distribution is used as the longitudinal Points on the center line of the light bar can get 2n+1 points Q_i,j,t on the center line, and the subscript t is 1,2,3,...,2n+1, and then fit the straight line l_{v, i,j} . Finally, the intersection point of two intersecting straight lines within the same search range is obtained as the calibration feature point A_i,j , and its coordinates are (u_i,j ,v_i,j ).

步骤3：获取主点的粗略坐标。利用左侧摄像机4a或右侧摄像机4b在两种不同焦距下拍摄同一个投影的高斯网格图案6，特征点A_i,j的图像坐标分别为(u1_i,j,v1_i,j)和(u2_i,j,v2_i,j)，主点坐标为(u₀,v₀)，则有：Step 3: Get the rough coordinates of the principal point. Using the left camera 4a or the right camera 4b to shoot the same projected Gaussian grid pattern 6 at two different focal lengths, the image coordinates of the feature points A_i,j are (u1_i,j ,v1_i,j ) and (u2_i,j ,v2_i,j ), the principal point coordinates are (u₀ ,v₀ ), then:

$\frac{u u 22_{i i,, j j} - - {u u}_{00}}{u u 11_{i i,, j j} - - {u u}_{00}} = = \frac{v v 22_{i i,, j j} - - {v v}_{00}}{v v 11_{i i,, j j} - - {v v}_{00}} - - - - - - ((22))$

可以利用上式求出主点的粗略位置的坐标(u₀,v₀)。The coordinates (u₀ , v₀ ) of the rough position of the main point can be obtained by using the above formula.

步骤4：求取畸变系数与优化的主点坐标。根据畸变模型可以推出实际拍摄的交点A_i,j的坐标p_i,j＝(u_i,j,v_i,j,1)^T与理想的交点坐标q_i,j＝(u'_i,j,v'_i,j,1)^T的转换关系如下：Step 4: Calculate the distortion coefficient and the optimized principal point coordinates. According to the distortion model, it can be deduced that the coordinate p_i,j =(u_i,j ,v_i,j ,1)^T of the actually photographed intersection point A_i,j and the ideal intersection point coordinate q_i,j =(u'_i,j ,v'_i,j ,1) The conversion relationship of^T is as follows:

$[\begin{matrix} {u u}^{' '}_{i i,, j j} \\ {v v}^{' '}_{i i,, j j} \end{matrix}] = = [\begin{matrix} {u u}_{i i,, j j} \\ {v v}_{i i,, j j} \end{matrix}] + + [\begin{matrix} {\overset{~ ~}{u u}}_{i i,, j j} (({k k}_{11} {r r}_{i i,, j j}^{22} + + {k k}_{22} {r r}_{i i,, j j}^{44})) +2 +2 {p p}_{11} {\overset{~ ~}{u u}}_{i i,, j j} {\overset{~ ~}{v v}}_{i i,, j j} + + {p p}_{22} (({r r}_{i i,, j j}^{22} + + 22 {\overset{~ ~}{u u}}_{i i,, j j}^{22})) \\ {\overset{~ ~}{v v}}_{i i,, j j} (({k k}_{11} {r r}_{i i,, j j}^{22} + + {k k}_{22} {r r}_{i i,, j j}^{44})) + + {p p}_{11} (({r r}_{i i,, j j}^{22} + + 22 {\overset{~ ~}{v v}}_{i i,, j j}^{22})) + + 22 {p p}_{22} {\overset{~ ~}{u u}}_{i i,, j j} {\overset{~ ~}{v v}}_{i i,, j j} \end{matrix}] - - - - - - ((33))$

其中， ${\tilde{u}}_{i, j} = u_{i, j} - u_{0}, {\tilde{v}}_{i, j} = v_{i, j} - v_{0}, r_{i, j} = \sqrt{{\tilde{u}}_{i, j}^{2} + {\tilde{v}}_{i, j}^{2}},$ k₁与k₂为径向畸变系数，p₁与p₂为切向畸变系数。in, ${\tilde{u}}_{i, j} = u_{i, j} - u_{0}, {\tilde{v}}_{i, j} = v_{i, j} - v_{0}, r_{i, j} = \sqrt{{\tilde{u}}_{i, j}^{2} + {\tilde{v}}_{i, j}^{2}},$ k₁ and k₂ are radial distortion coefficients, p₁ and p₂ are tangential distortion coefficients.

另外，以横、纵光条数目相同的网格图案为例，交点总数为num，那么每行上有个交点，根据直线的保线性，即在同一条光条上的点共线的性质，结合三点共线的充要条件可以列出优化目标函数如下：In addition, taking the grid pattern with the same number of horizontal and vertical light bars as an example, the total number of intersection points is num, then there are intersection point, according to the linearity of the straight line, that is, the collinear nature of the points on the same light strip, combined with the necessary and sufficient conditions for three points to be collinear, the optimization objective function can be listed as follows:

$\begin{matrix} min min & {Σ Σ}_{i i = = 11}^{\sqrt{n no u u m m}} (({Σ Σ}_{j j = = \sqrt{n no u u m m} ((i i - - 11)) + + 11}^{\sqrt{n no u u m m} ((i i - - 11)) + + \sqrt{n no u u m m} - - 22} | | {q q}_{i i,, j j + + 11}^{T T} {[[{q q}_{i i,, j j}]]}_{\times \times} {q q}_{i i,, j j + + 22} | |)) - - - - - - ((44)) \end{matrix}$

其中，为三点A_i,j，A_i,j+1，A_i,j+2共线的充要条件。[q_i,j]_×代表q_i,j的反对称矩阵，即：in, It is a necessary and sufficient condition for three points A_i,j , A_i,j+1 and A_i,j+2 to be collinear. [q_i,j ]_× represents the antisymmetric matrix of q_i,j , namely:

${[[{q q}_{i i,, j j}]]}_{\times \times} = = [\begin{matrix} 00 & - - 11 & - - {v v}^{' '}_{i i,, j j} \\ 11 & 00 & - - {u u}^{' '}_{i i,, j j} \\ - - {v v}^{' '}_{i i,, j j} & {u u}^{' '}_{i i,, j j} & 00 \end{matrix}] - - - - - - ((55))$

通过Levenberg-Marquardt非线性优化算法进行优化，使式(4)的目标函数的值最小，可以获取畸变系数k₁、k₂、p₁与p₂以及优化后的主点坐标(u'₀,v'₀)。然后利用式(3)将所有的交点坐标修正为理想的坐标。Through the optimization of the Levenberg-Marquardt nonlinear optimization algorithm, the value of the objective function of formula (4) can be minimized, and the distortion coefficients k₁ , k₂ , p₁ and p₂ and the optimized principal point coordinates (u'₀ , v'₀ ). Then use formula (3) to correct all intersection coordinates to ideal coordinates.

步骤5：求取摄像机其余的内部参数。利用修正后的理想交点作为特征点，采用主动视觉方法，利用左侧四维电控平台2a带动左侧摄像机4a作两组正交运动，在每组正交运动的三个始末位置上分别拍摄一张投影的高斯网格图案6的图像，最终左侧摄像机4a拍摄得到6张图像。Step 5: Obtain the remaining internal parameters of the camera. Using the corrected ideal intersection point as a feature point, using the active vision method, using the left four-dimensional electric control platform 2a to drive the left camera 4a to perform two sets of orthogonal movements, and take a picture at the three starting and ending positions of each set of orthogonal movements images of the projected Gaussian grid pattern 6, and finally the left camera 4a captures 6 images.

平行直线与无穷远平面相交于同一个无穷远点，即隐消点。而一组正交运动含有两次平移，一次平移运动的始末位置上拍摄的两幅图像上对应交点的连线为一组空间平行线，而且两次平移是相互垂直的，因此我们可以获得一组正交的隐消点对e_i1(u_i1,v_i1)和e_i2(u_i2,v_i2)，其中i＝1,2，代表正交运动的次序，并有Oe_i1·Oe_i2＝0，其中O为摄像机的主点。利用两组正交隐消点对，可以通过求解下述方程组分别求解左侧摄像机4a的内参数矩阵K中的尺度因子α_x与α_y：Parallel lines intersect the infinite plane at the same infinite point, which is the vanishing point. A set of orthogonal motions contains two translations, and the lines corresponding to the intersection points on the two images taken at the start and end positions of a translation motion are a set of spatial parallel lines, and the two translations are perpendicular to each other, so we can obtain a A set of orthogonal blanking point pairs e_i1 (u_i1 ,v_i1 ) and e_i2 (u_i2 ,v_i2 ), where i=1,2 represents the order of orthogonal motion, and Oe_i1 ·Oe_i2 = 0, where O is the principal point of the camera. Using two sets of orthogonal blanking point pairs, the scale factors α_x and α_y in the internal parameter matrix K of the left camera 4a can be solved respectively by solving the following equations:

$\{\begin{matrix} (({u u}_{1111} - - {u u}^{' '}_{00})) (({u u}_{1212} - - {u u}^{' '}_{00})) / / {α α}_{x x}^{22} + + (({v v}_{1111} - - {v v}^{' '}_{00})) (({v v}_{1212} - - {v v}^{' '}_{00})) / / {α α}_{y the y}^{22} + + 11 = = 00 \\ (({u u}_{21 twenty one} - - {u u}^{' '}_{00})) (({u u}_{22 twenty two} - - {u u}^{' '}_{00})) / / {α α}_{x x}^{22} + + (({v v}_{21 twenty one} - - {v v}^{' '}_{00})) (({v v}_{22 twenty two} - - {v v}^{' '}_{00})) / / {α α}_{y the y} + + 11 = = 00 \end{matrix} - - - - - - ((66))$

同样地，可以获取右侧摄像机4b的内参数矩阵K中的尺度因子α_x与α_y。Likewise, the scale factors α_x and α_y in the internal parameter matrix K of the right camera 4 b can be obtained.

步骤6：获取摄像机的外部参数。左侧摄像机4a与右侧摄像机4b拍摄同一投影的高斯网格图案6，将世界坐标系建立在左侧摄像机4a的像机坐标系上，利用左、右摄像机4a、4b拍摄图像修正后的匹配点计算出基本矩阵F。利用已求取的内参数和基本矩阵可在相差一个比例因子s的情况下计算出本质矩阵E。分解本质矩阵E后可在相差一个比例因子的情况下确定外部参数(旋转矩阵R'与平移向量t')。Step 6: Obtain the external parameters of the camera. The left camera 4a and the right camera 4b shoot the Gaussian grid pattern 6 of the same projection, establish the world coordinate system on the camera coordinate system of the left camera 4a, and use the left and right cameras 4a and 4b to shoot the corrected images. Point to calculate the fundamental matrix F. The essential matrix E can be calculated by using the calculated internal parameters and the fundamental matrix with a difference of a scale factor s. After decomposing the essential matrix E, the external parameters (rotation matrix R' and translation vector t') can be determined with a difference of a scale factor.

利用投影仪3投影平行的高斯光条至实际长度L₀已精确测量的量块上，利用光条的高斯特性拟合出亚像素光条中心线，利用灰度骤变的点作为量块的边界点，根据上述求取的内、外参数重建量块的长度L'₀，可以获得比例因子为：s＝L₀/L'₀。因此，可以得到摄像机实际的外部参数(旋转矩阵R＝R'与平移向量t＝s*t')。至此，完成了摄像机的标定过程。Use the projector 3 to project parallel Gaussian light strips onto the gauge block whose actual length_L0 has been accurately measured, use the Gaussian characteristics of the light strips to fit the center line of the sub-pixel light strips, and use the points with sudden changes in gray scale as the gauge block For the boundary point, the length L'₀ of the gauge block can be reconstructed according to the internal and external parameters obtained above, and the scale factor can be obtained as: s=L₀ /L'₀ . Therefore, the actual external parameters of the camera (rotation matrix R=R' and translation vector t=s*t') can be obtained. So far, the calibration process of the camera is completed.

本发明的有益效果是采用投影仪投影的高斯网格图案的交点作为标定特征点，避免了标定块、标定板与粘贴标记点的使用，便于实现在锻造现场等复杂环境下进行摄像机的实时标定。根据光条宽度方向上灰度呈高斯分布特性可高精度地确定标定特征点的位置信息，具有高的鲁棒性，分步标定可以获取高精度的摄像机参数，该方法同时可以避免将所有摄像机参数同时求解时的耦合性问题。The beneficial effect of the present invention is that the intersection of the Gaussian grid pattern projected by the projector is used as the calibration feature point, which avoids the use of calibration blocks, calibration plates and pasted marking points, and facilitates the real-time calibration of cameras in complex environments such as forging sites . According to the Gaussian distribution characteristic of the gray level in the width direction of the light strip, the position information of the calibration feature points can be determined with high precision, which has high robustness. The step-by-step calibration can obtain high-precision camera parameters, and this method can avoid all cameras at the same time Coupling problem when parameters are solved simultaneously.

附图说明Description of drawings

图1为本发明的标定系统示意图。其中：1-隔震平台，2a-左侧四维电控平台，2b-右侧四维电控平台，3-投影仪，4a-左侧摄像机，4b-右侧摄像机，5-光整的平板或墙面，6-高斯网格图案。Fig. 1 is a schematic diagram of the calibration system of the present invention. Among them: 1-shock isolation platform, 2a-left four-dimensional electric control platform, 2b-right four-dimensional electric control platform, 3-projector, 4a-left camera, 4b-right camera, 5-light flat panel or Wall surface, 6-Gauss grid pattern.

图2为本发明由摄像机拍摄得到的网格图案的图像。Fig. 2 is an image of a grid pattern captured by a camera in the present invention.

图3为本发明对网格图案的图像进行二值化处理并获取交点的粗略位置。Fig. 3 shows the rough position of the intersecting point obtained by binarizing the image of the grid pattern according to the present invention.

图4为本发明的高斯光条阵列重建量块尺寸求取比例因子。Fig. 4 shows the calculation scale factor for Gaussian light strip array reconstruction gauge block size of the present invention.

具体实施方式detailed description

下面结合附图和技术方案进一步详细说明本发明的具体实施方式。The specific implementation manner of the present invention will be further described in detail below in conjunction with the accompanying drawings and technical solutions.

摄像机标定通常采用经典的小孔成像模型，该模型的表达式如下：Camera calibration usually adopts the classic pinhole imaging model, the expression of which is as follows:

其中，(X_w,Y_w,Z_w,1)^T为空间点在世界坐标系中的齐次坐标，(u,v,1)^T为对应的图像像点像素坐标系o₀uv中的齐次坐标，α_x＝f/dx为o₀uv坐标系内u轴上的尺度因子，α_y＝f/dy为o₀uv坐标系内v轴上的尺度因子，f为摄像机镜头焦距，dx与dy分别为像元的横、纵物理尺寸，(u₀,v₀)为主点坐标，ρ_c为比例系数，K为摄像机内部参数矩阵，[R|t]为摄像机的外部参数矩阵，其中，R为旋转矩阵，t为平移向量。Among them, (X_w ,Y_w ,Z_w ,1)^T is the homogeneous coordinate of the space point in the world coordinate system, and (u,v,1)^T is the corresponding image pixel pixel coordinate system o₀ uv Homogeneous coordinates, α_x =f/dx is the scale factor on the u axis in the o₀ uv coordinate system, α_y =f/dy is the scale factor on the v axis in the o₀ uv coordinate system, f is the focal length of the camera lens, dx and dy are the horizontal and vertical physical dimensions of the pixel respectively, (u₀ , v₀ ) are the principal point coordinates, ρ_c is the proportional coefficient, K is the internal parameter matrix of the camera, [R|t] is the external parameter matrix of the camera , where R is the rotation matrix and t is the translation vector.

摄像机内部参数包括主点坐标(u₀,v₀)、尺度因子α_x、α_y，径向畸变系数k₁、k₂与切向畸变系数p₁、p₂。摄像机外部参数为摄像机坐标系相对于世界坐标系的方位，包括旋转矩阵R与平移向量t。The internal parameters of the camera include principal point coordinates (u₀ , v₀ ), scale factors α_x , α_y , radial distortion coefficients k₁ , k₂ and tangential distortion coefficients p₁ , p₂ . The external parameters of the camera are the orientation of the camera coordinate system relative to the world coordinate system, including the rotation matrix R and the translation vector t.

步骤1：搭建摄像机标定系统。将左侧四维电控平台2a、右侧四维电控平台2b及投影仪3安装在平台1的台面上，将左侧摄像机4a固定在左侧四维电控平台2a上，将右侧摄像机4b固定在右侧四维电控平台2b上，如图1所示。Step 1: Build a camera calibration system. Install the left four-dimensional electric control platform 2a, the right four-dimensional electric control platform 2b and the projector 3 on the platform 1, fix the left camera 4a on the left four-dimensional electric control platform 2a, and fix the right camera 4b On the right four-dimensional electric control platform 2b, as shown in FIG. 1 .

步骤2：投影高斯网格图案、拍摄并获取交点坐标。通过投影仪3向厂房内光整的平板或墙面5上投影高斯网格图案6，高斯网格图案6由多条平行的横光条与多条平行的纵光条组成，各光条的灰度在宽度方向上均呈高斯分布，其中横、纵光条交点A_i,j为标定特征点，i为横光条的编号，按照从上至下的次序，j为纵光条的编号，按照从左至右的次序。由左侧摄像机4a或右侧摄像机4b拍摄得到的投射高斯网格图案的图像如图2所示。由于横、纵光条交点处光强叠加，对左侧摄像机4a与右侧摄像机4b拍摄得到的图像进行二值化处理后，获得的图像中只剩下网格交点处的亮斑，即一个个孤立的连通区域，如图3所示。利用形心法可以获取连通区域的质心坐标(u0_i,j,v0_i,j)，作为特征点A_i,j的粗略位置。将以该粗略位置为圆心，以Δ个像素为半径的圆形区域作为搜索范围，然后在[u0_i,j-Δ,u0_i,j+Δ]范围内每隔Δ/n沿宽度方向搜索一次横光条，按照高斯分布特性进行拟合，将高斯分布峰值点作为横光条中心线上的点，因此可以得到2n+1个中心线上的点P_i,j,s，下标s为1,2,3,…,2n+1，进而拟合出直线l_h,i,j。同样地，在[v0_i,j-Δ,v0_i,j+Δ]范围内每隔Δ/n沿宽度方向搜索一次纵光条，按照高斯分布特性进行拟合，将高斯分布峰值点作为纵光条中心线上的点，可以得到2n+1个中心线上的点Q_i,j,t，下标t为1,2,3,…,2n+1，进而拟合出直线l_v,i,j。最终，通过求取同一搜索范围内两条相交直线的交点作为标定特征点A_i,j，其坐标为(u_i,j,v_i,j)。Step 2: Project the Gaussian grid pattern, photograph and obtain the intersection coordinates. Through the projector 3, the Gaussian grid pattern 6 is projected onto the flat panel or wall 5 in the factory building. The Gaussian grid pattern 6 is composed of a plurality of parallel horizontal light bars and a plurality of parallel vertical light bars. The degree is Gaussian distributed in the width direction, where the horizontal and vertical light bars intersection point A_{i, j} is the calibration feature point, i is the number of the horizontal light bar, in order from top to bottom, and j is the number of the vertical light bar, according to order from left to right. The image of the projected Gaussian grid pattern captured by the left camera 4 a or the right camera 4 b is shown in FIG. 2 . Due to the superposition of the light intensity at the intersection of the horizontal and vertical light strips, after binarizing the images captured by the left camera 4a and the right camera 4b, only the bright spot at the intersection of the grid remains in the obtained image, that is, a An isolated connected region, as shown in Figure 3. Using the centroid method, the centroid coordinates (u0_i,j ,v0_i,j ) of the connected area can be obtained as the rough position of the feature point A_i,j . Use the rough position as the center and a circular area with a radius of Δ pixels as the search range, and then search along the width direction every Δ/n within the range of [u0_i,j -Δ,u0_i,j +Δ] A horizontal light bar is fitted according to the Gaussian distribution characteristics, and the peak point of the Gaussian distribution is used as a point on the center line of the horizontal light bar, so 2n+1 points P_i,j,s on the center line can be obtained, and the subscript s is 1 ,2,3,…,2n+1, and then fit the straight line l_h,i,j . Similarly, within the range of [v0_i,j -Δ,v0_i,j +Δ], the vertical light strip is searched every Δ/n along the width direction, fitted according to the Gaussian distribution characteristics, and the peak point of the Gaussian distribution is used as the longitudinal Points on the center line of the light bar can get 2n+1 points Q_i,j,t on the center line, and the subscript t is 1,2,3,...,2n+1, and then fit the straight line l_{v, i,j} . Finally, the intersection point of two intersecting straight lines within the same search range is obtained as the calibration feature point A_i,j , and its coordinates are (u_i,j ,v_i,j ).

步骤3：获取主点的粗略坐标。利用变焦距方法求取主点，左侧摄像机4a或右侧摄像机4b在两种不同焦距下拍摄同一个投影高斯网格图案6，特征点A_i,j的图像坐标分别为(u1_i,j,v1_i,j)和(u2_i,j,v2_i,j)，主点坐标为(u₀,v₀)，则有：Step 3: Get the rough coordinates of the principal point. Using the zoom method to find the principal point, the left camera 4a or the right camera 4b shoots the same projected Gaussian grid pattern 6 under two different focal lengths, and the image coordinates of the feature points A_{i, j} are (u1_{i, j} ,v1_i,j ) and (u2_i,j ,v2_i,j ), the principal point coordinates are (u₀ ,v₀ ), then:

暂且将镜头缩放中心视为主点，可以利用上式求出主点的粗略位置的坐标(u₀,v₀)。Considering the zoom center of the lens as the main point for now, the coordinates (u₀ , v₀ ) of the rough position of the main point can be obtained by using the above formula.

步骤5：求取摄像机其余的内部参数。利用修正后的理想交点作为特征点，采用主动视觉方法，利用左侧四维电控平台2a带动左侧摄像机4a作两组正交运动，在每组正交运动的三个始末位置上分别拍摄一张投影高斯网格图案6的图像，最终左侧摄像机4a拍摄得到6张图像。具体流程如下：(1)调节左侧四维电控平台2a至适当的位置，开始第一组正交运动，在正交运动的三个始末位置上分别拍摄一张投影高斯网格图案6的图像；(2)利用左侧四维电控平台2a使左侧摄像机4a俯视一定角度，开始第二组正交运动，在正交运动的三个始末位置上分别拍摄一张投影高斯网格图案6的图像。Step 5: Obtain the remaining internal parameters of the camera. Using the corrected ideal intersection point as the feature point, using the active vision method, using the left four-dimensional electric control platform 2a to drive the left camera 4a to perform two sets of orthogonal movements, and take a picture at the three start and end positions of each set of orthogonal movements An image of the projected Gaussian grid pattern 6, and finally the left camera 4a captures 6 images. The specific process is as follows: (1) Adjust the left four-dimensional electronic control platform 2a to an appropriate position, start the first set of orthogonal movements, and take an image of the projected Gaussian grid pattern 6 at the three starting and ending positions of the orthogonal movement (2) Utilize the left four-dimensional electric control platform 2a to make the left camera 4a look down at a certain angle, start the second group of orthogonal movements, and take a picture of the projected Gaussian grid pattern 6 respectively on the three start and end positions of the orthogonal movement image.

直线上无穷远点的图像称为该直线的隐消点。由于平行直线与无穷远平面相交于同一个无穷远点，即隐消点。一组正交运动含有两次平移，一次平移运动的始末位置上拍摄的两幅图像上对应交点的连线为一组空间平行线，而且两次平移是相互垂直的，因此我们可以获得一组正交的隐消点对e_i1(u_i1,v_i1)和e_i2(u_i2,v_i2)，下标i＝1,2，代表正交运动的次序，并有Oe_i1·Oe_i2＝0，其中O为摄像机的主点。利用两组正交隐消点对，可以通过求解下述方程组分别求解左侧摄像机4a的内参数矩阵K中的尺度因子α_x与α_y：The image of a point at infinity on a line is called the vanishing point of that line. Since the parallel straight line intersects the infinity plane at the same infinity point, that is, the vanishing point. A set of orthogonal motions contains two translations. The lines corresponding to the intersection points on the two images captured at the start and end positions of a translation motion are a set of spatial parallel lines, and the two translations are perpendicular to each other, so we can obtain a set of Orthogonal blanking point pair e_i1 (u_i1 ,v_i1 ) and e_i2 (u_i2 ,v_i2 ), subscript i=1,2, represents the order of orthogonal movement, and Oe_i1 ·Oe_i2 = 0, where O is the principal point of the camera. Using two sets of orthogonal blanking point pairs, the scale factors α_x and α_y in the internal parameter matrix K of the left camera 4a can be solved respectively by solving the following equations:

步骤6：获取外部参数。左侧摄像机4a与右侧摄像机4b拍摄同一投影高斯网格图案6，将世界坐标系建立在左侧摄像机4a的像机坐标系上，利用左、右摄像机4a、4b拍摄图像修正后的匹配点计算出基本矩阵F。利用内参数和基本矩阵可在相差一个比例因子s的情况下计算出本质矩阵E。分解本质矩阵E后可在相差一个比例因子的情况下确定外部参数(旋转矩阵R'与平移向量t')。Step 6: Get external parameters. The left camera 4a and the right camera 4b shoot the same projected Gaussian grid pattern 6, establish the world coordinate system on the camera coordinate system of the left camera 4a, and use the left and right cameras 4a and 4b to shoot the matching points after image correction Calculate the fundamental matrix F. The intrinsic matrix E can be calculated with a difference of a scale factor s by using the internal parameters and the fundamental matrix. After decomposing the essential matrix E, the external parameters (rotation matrix R' and translation vector t') can be determined with a difference of a scale factor.

如图4所示，利用投影仪3投影平行的高斯光条至实际长度L₀已精确测量的量块上，利用光条的高斯特性拟合出亚像素光条中心线，利用灰度骤变的点作为量块的边界点，根据上述求取的内、外参数重建量块的长度L'₀，可以获得比例因子为：s＝L₀/L'₀。因此，可以得到摄像机实际的外部参数(旋转矩阵R＝R'与平移向量t＝s*t')。至此，完成了摄像机的标定过程。As shown in Figure 4, use the projector 3 to project the parallel Gaussian light strips onto the gauge block whose actual length_L0 has been accurately measured, use the Gaussian properties of the light strips to fit the sub-pixel light strip centerline, and use the grayscale sudden change The point of the gauge block is used as the boundary point of the gauge block, and the length L'₀ of the gauge block is reconstructed according to the internal and external parameters obtained above, and the scaling factor can be obtained as: s=L₀ /L'₀ . Therefore, the actual external parameters of the camera (rotation matrix R=R' and translation vector t=s*t') can be obtained. So far, the calibration process of the camera is completed.

本发明提出的摄像机标定方法具有良好的实时性、鲁棒性及较高的标定精度，能够用于锻造现场等复杂环境下对大视场摄像机进行在线标定。The camera calibration method proposed by the present invention has good real-time performance, robustness and high calibration accuracy, and can be used for online calibration of cameras with a large field of view in complex environments such as forging sites.