CN108257187B - A camera-projector system calibration method - Google Patents

Abstract

Translated fromChinese

本发明公开了一种相机‑投影仪系统标定方法，充分利用所有的约束条件，通过事先标定好的相机内参和畸变，以棋盘格角点的世界坐标误差最小为目标，对投影仪内参和畸变、投影仪与相机的外参进行联合优化。避免了现有标定方法中，内参、外参独自标定，且多个标定位姿相互独立，得到的外参不一致的问题，可以得到一致的高精度的标定结果。The invention discloses a camera-projector system calibration method, which makes full use of all the constraints, through the camera internal parameters and distortion calibrated in advance, with the goal of minimizing the world coordinate error of the corner points of the checkerboard, and adjusts the projector internal parameters and distortion. , The external parameters of the projector and the camera are jointly optimized. In the existing calibration method, the internal and external parameters are independently calibrated, and multiple calibration orientations are independent of each other, and the obtained external parameters are inconsistent, and consistent and high-precision calibration results can be obtained.

Description

Translated fromChinese

一种相机-投影仪系统标定方法A camera-projector system calibration method

技术领域technical field

本发明涉及计算机视觉、检测测量技术领域，尤其涉及一种相机-投影仪系统标定方法。The invention relates to the technical fields of computer vision, detection and measurement, and in particular to a camera-projector system calibration method.

背景技术Background technique

相机-投影仪系统是常见的高精度三维成像方法，在三维测量、机器人等领域有广泛应用，系统的标定精度直接影响到最终的性能，是该系统的重点和难点，一直未能得到理想的效果。目前主流的标定方法是：在同样的位姿下，相机拍摄打印棋盘格和投影棋盘格图像，通过打印棋盘格尺寸参数计算相机与棋盘平面的单应矩阵，再根据单应矩阵得到投影棋盘格的世界坐标，结合投影棋盘格的设置参数进行逆向标定，得到投影仪的内参和投影仪与一系列棋盘格平面的相对位姿。这种方法对所有的位姿独立处理，忽略了一个很重要的约束条件：投影仪和相机的相对位姿是固定的。所产生的结果是，标定会产生一系列投影仪与棋盘平面的位姿参数，根据这一系列参数得到的投影仪与相机的相对位姿并不完全一致，可能会存在较大误差，而只能选择一组参数或者这一系列参数的均值作为最终结果，虽然有可能得到比较小的重投影误差，但这个结果是不可靠的。The camera-projector system is a common high-precision 3D imaging method, which is widely used in 3D measurement, robotics and other fields. The calibration accuracy of the system directly affects the final performance, which is the focus and difficulty of the system. Effect. The current mainstream calibration method is: in the same pose, the camera shoots the printed checkerboard and the projected checkerboard image, calculates the homography matrix between the camera and the checkerboard plane by printing the checkerboard size parameters, and then obtains the projected checkerboard according to the homography matrix. The world coordinates of , combined with the setting parameters of the projection checkerboard, are reversely calibrated, and the internal parameters of the projector and the relative poses of the projector and a series of checkerboard planes are obtained. This approach treats all poses independently, ignoring an important constraint: the relative poses of the projector and camera are fixed. The result is that the calibration will generate a series of pose parameters of the projector and the chessboard plane. The relative poses of the projector and the camera obtained according to this series of parameters are not completely consistent, and there may be large errors. A set of parameters or the mean of this series of parameters can be selected as the final result. Although it is possible to obtain a relatively small reprojection error, this result is unreliable.

标定的另一个难点在于如何在同样位姿下，得到打印棋盘格和投影棋盘格的图像，通过单应矩阵的传递得到投影棋盘格的世界坐标。为便于操作，一般是拍摄打印棋盘格和投影棋盘格的叠加图像，采用图像处理算法将两种棋盘格分离，再提取棋盘格角点。分离棋盘格的算法，一般是设定阈值，将投影棋盘格进行二值化，再提取角点。二值化棋盘格提取的角点会存在比较大的误差。Another difficulty in calibration is how to obtain the images of the printed checkerboard and the projected checkerboard under the same pose, and obtain the world coordinates of the projected checkerboard through the transfer of the homography matrix. In order to facilitate the operation, the superimposed images of the printed checkerboard and the projected checkerboard are generally taken, the two checkerboards are separated by an image processing algorithm, and then the corner points of the checkerboard are extracted. The algorithm for separating the checkerboard is generally to set a threshold, binarize the projected checkerboard, and then extract the corner points. The corner points extracted from the binarized checkerboard will have a relatively large error.

ZL201410164584.6对分离打印棋盘格和投影棋盘格以及提起角点的方法进行了优化，通过投影多幅反相的、旋转的棋盘格，经差分、反相、滤波操作后提取角点并取平均值参与标定。一定程度上提高了角点的提取精度，但是对叠加棋盘格的分离依然采用设定阈值的方法，并不能很好的将打印棋盘格从背景中去除。ZL201410164584.6 optimizes the method of separating and printing the checkerboard and projecting the checkerboard and lifting the corner points. By projecting multiple inverted and rotated checkerboards, the corner points are extracted and averaged after the operations of difference, inversion and filtering. The value participates in the calibration. To a certain extent, the extraction accuracy of corner points is improved, but the method of setting a threshold is still used for the separation of superimposed checkerboards, which cannot remove the printed checkerboards from the background very well.

发明内容SUMMARY OF THE INVENTION

为解决现有标定方法存在的问题，本发明提供一种相机-投影仪系统标定方法，充分利用所有的约束条件，通过事先标定好的相机内参和畸变，以棋盘格角点的世界坐标误差最小为目标，对投影仪内参和畸变、投影仪与相机的外参进行联合优化。避免了现有标定方法中，内参、外参独自标定，且多个标定位姿相互独立，得到的外参不一致的问题，可以得到一致的高精度的标定结果。In order to solve the problems existing in the existing calibration methods, the present invention provides a camera-projector system calibration method, which fully utilizes all the constraints, and uses the camera internal parameters and distortions calibrated in advance to minimize the world coordinate error of the checkerboard corner points. As the goal, the internal parameters and distortion of the projector and the external parameters of the projector and the camera are jointly optimized. In the existing calibration method, the internal and external parameters are independently calibrated, and multiple calibration orientations are independent of each other, and the obtained external parameters are inconsistent, and consistent and high-precision calibration results can be obtained.

本发明解决技术问题所采用的技术方案如下：一种相机-投影仪系统标定方法，该方法包括如下步骤：The technical solution adopted by the present invention to solve the technical problem is as follows: a camera-projector system calibration method, the method comprises the following steps:

步骤1：采用现有的相机标定技术对相机内参I_c和畸变d_c进行标定；Step 1: use the existing camera calibration technology to calibrate the camera internal parameter I_c and distortion d_c ;

步骤2：在打印棋盘格背景上，投影仪依次投影全亮、全暗、投影棋盘格图案，相机以同样的曝光参数拍摄相应图像I_bright、I_dark和I_comb；Step 2: On the printed checkerboard background, the projector projects all bright, all dark and projected checkerboard patterns in sequence, and the camera shoots corresponding images I_bright , I_dark and I_comb with the same exposure parameters;

步骤3：对图像I_bright提取打印棋盘格角点，根据相机内参I_c与畸变d_c计算相机与棋盘格平面相对位姿R_i,T_i；Step 3: extract and print the corner points of the checkerboard for the image I_bright , and calculate the relative_{poses Ri, T iof} the camera and the checkerboard plane according to the camera internal parameter I_c and the distortion d_c ;

步骤4：从三幅图像I_bright、I_dark和I_comb中分离出投影棋盘格图像，并提取投影棋盘格角点，根据R_i,T_i计算角点世界坐标；Step 4: separate the projected checkerboard image from the three images I_bright , I_dark and I_comb , and extract the projected checkerboard corner points, and calculate the world coordinates of the corner points according to R_i , T_i ;

步骤5：初始化投影仪与相机的外参R和T，根据R、T以及R_i,T_i计算投影仪与棋盘格平面的相对位姿[R’_i,T’_i]＝f(R,T,R_i,T_i)；Step 5: Initialize the external parameters R and T of the projector and the camera, and calculate the relative pose of the projector and the checkerboard plane according to R, T and R_i , T_i [R'_i ,T'_i ]=f(R, T,R_i ,T_i );

步骤6：初始化投影仪内参I_p和畸变d_p，根据I_p、d_p以及R’_i,T’_i计算投影棋盘角点世界坐标；Step 6: Initialize the projector internal parameters I_p and distortion d_p , and calculate the world coordinates of the corners of the projected chessboard according to I_p , d_p and R'_i , T'_i;

步骤7：更换棋盘格平面的位姿，重复步骤2-6，获取足够标定数据，标定数据组数可事先设定，一般不少于5组，优选10组；Step 7: Change the pose of the checkerboard plane, and repeat steps 2-6 to obtain enough calibration data. The number of calibration data sets can be set in advance, generally not less than 5 sets, preferably 10 sets;

步骤8：以步骤4和步骤6得到的世界坐标误差最小为目标，对未知参数I_p、d_p、R和T进行联合优化，得到标定结果。Step 8: With the goal of minimizing the world coordinate error obtained in Step 4 and Step 6, the unknown parameters I_p , d_p , R and T are jointly optimized to obtain the calibration result.

进一步的，所述步骤4中，分离出投影棋盘格图像的方法为：Further, in the step 4, the method for separating the projected checkerboard image is:

对第n个像素，计算反射系数ρⁿ＝(Iⁿ_comb-Iⁿ_dark)/(Iⁿ_bright-Iⁿ_dark)，则投影棋盘格第n个像素的值Iⁿ为：For the nth pixel, calculate the reflection coefficient ρⁿ =(Iⁿ_comb -Iⁿ_dark )/(Iⁿ_bright -Iⁿ_dark ), then the value Iⁿ of the nth pixel of the projected checkerboard is:

第一个条件和第三个条件保证投影棋盘格内部的对比度，第二个条件保证投影棋盘格原始边界，ρ_t1、ρ_t2分别表示反射系数阈值上限和下限，0.75≤ρ_t1≤1,0≤ρ_t2≤0.25；I_t1、I_t2分别表示强度阈值上限和下限，根据图像的曝光度以I_comb的直方图设置；分离投影棋盘格后进行高斯滤波减小个别像素计算错误带来的误差。The first condition and the third condition guarantee the contrast inside the projected_checkerboard , and the second condition guarantees the original boundary of the_{projectedcheckerboard} . ≤ρ_t2 ≤0.25; I_t1 and I_t2 respectively represent the upper and lower limits of the intensity threshold, which are set according to the histogram of I_comb according to the exposure of the image; Gaussian filtering is performed after separating the projected checkerboard to reduce the error caused by individual pixel calculation errors .

进一步的，所述相机标定技术优选张正友法。Further, the camera calibration technology is preferably Zhang Zhengyou's method.

进一步的，为更准确的从叠加图案中分离打印棋盘格和投影棋盘格，提高棋盘格角点的提取精度，打印棋盘格为单色，优选为绿色，因为三通道彩色图像中绿色通道信息最多，蓝色和红色通道为周围像素共享。特别的，投影棋盘格与打印棋盘格颜色相同，可以进一步降低背景棋盘格的影响。单色打印棋盘格，使用叠加图案的单通道彩色信息计算各个像素的反射系数，将背景打印棋盘格分离，得到准确的投影棋盘格图像，避免了现有的采用阈值方法提取投影棋盘格不准的问题，提高了获取棋盘格角点的精度。Further, in order to more accurately separate the printed checkerboard and the projected checkerboard from the superimposed pattern, and improve the extraction accuracy of the corner points of the checkerboard, the printed checkerboard is monochrome, preferably green, because the green channel has the most information in the three-channel color image. , the blue and red channels are shared for surrounding pixels. In particular, the projected checkerboard is the same color as the printed checkerboard, which can further reduce the influence of the background checkerboard. Monochrome print checkerboard, use the single-channel color information of the superimposed pattern to calculate the reflection coefficient of each pixel, separate the background print checkerboard, and obtain an accurate projected checkerboard image, which avoids the inaccuracy of the existing threshold method to extract the projected checkerboard. , which improves the accuracy of obtaining the corner points of the checkerboard.

进一步的，所述步骤4中，三幅图像I_bright、I_dark和I_comb只使用与打印棋盘格颜色相对应颜色的单通道数据，即彩色图像有三个通道，用的是绿色棋盘，就用绿色通道，其他两个是红、蓝。Further, in the step 4, the three images I_bright , I_dark and I_comb only use the single-channel data of the color corresponding to the color of the printed checkerboard, that is, the color image has three channels, and the green checkerboard is used. The green channel, the other two are red and blue.

进一步的，所述步骤5中，计算投影仪与棋盘格平面的相对位姿为：Further, in the step 5, the relative pose of the calculated projector and the checkerboard plane is:

进一步的，所述步骤8中的联合优化采用非线性优化方法。Further, the joint optimization in the step 8 adopts a nonlinear optimization method.

本发明的有益效果如下：1.预先以成熟的技术标定相机内参，可以保证已知数据的准确性，同时降低其他误差的影响，提高整体标定精度；2.对投影仪的未知参数进行联合标定可以充分利用所有的约束条件，得到一致的标定参数，提高标定精度和可靠性；3.单色棋盘格在对应的单通道图像下，近似为白色，可以很方便的分离，而不影响打印棋盘格和投影棋盘格的角点，绿色的效果最好。投影棋盘格采用一致的颜色可以进一步提升效果；4.采用全亮、全暗、叠加图像三幅图，按照反射系数的方法可以高保真的分离出投影棋盘格，提升角点提取精度。The beneficial effects of the present invention are as follows: 1. The camera internal parameters are calibrated in advance with mature technology, which can ensure the accuracy of known data, reduce the influence of other errors, and improve the overall calibration accuracy; 2. Jointly calibrate the unknown parameters of the projector It can make full use of all the constraints to obtain consistent calibration parameters and improve the calibration accuracy and reliability; 3. The monochrome checkerboard is approximately white under the corresponding single-channel image, which can be easily separated without affecting the printing checkerboard The corners of the grid and projected checkerboard, green works best. The consistent color of the projection checkerboard can further improve the effect; 4. Using three images of full brightness, full darkness, and superimposed images, the projection checkerboard can be separated with high fidelity according to the method of reflection coefficient, and the accuracy of corner extraction can be improved.

附图说明Description of drawings

图1是本发明标定方法的流程图；Fig. 1 is the flow chart of the calibration method of the present invention;

图2是叠加棋盘格的单通道图像示意图；Fig. 2 is the single-channel image schematic diagram of superimposed checkerboard;

图3是分离背景提取的投影棋盘格示意图。FIG. 3 is a schematic diagram of a projected checkerboard extracted from the separated background.

具体实施方式Detailed ways

下面结合附图和实施例对本发明做进一步的说明。The present invention will be further described below with reference to the accompanying drawings and embodiments.

如图1所示，本发明提供一种相机-投影仪系统标定方法，具体包括：As shown in FIG. 1 , the present invention provides a camera-projector system calibration method, which specifically includes:

1、采用张正友标定法标定相机内参I_c和畸变d_c；1. Use Zhang Zhengyou's calibration method to calibrate the camera's internal parameters I_c and distortion d_c ;

2、打印绿色和白色间隔的棋盘格图案，棋盘格数量为9×6，每格尺寸40×40mm，贴在白色平面上，需要表面平整，没有镜面反光；2. Print green and white checkerboard patterns, the number of checkerboards is 9×6, and the size of each grid is 40×40mm. It is attached to a white plane, and the surface needs to be flat without mirror reflection;

3、在贴有打印棋盘格的平面上，投影仪依次投射全亮、全暗、投影棋盘格的图案，投影棋盘格数量为24×19，每格像素38×30；3. On the plane with the printed checkerboard, the projector projects the pattern of full bright, full dark, and projected checkerboard in sequence. The number of projected checkerboards is 24×19, and the pixels per grid are 38×30;

4、相机使用相同的曝光参数依次采集三幅图像I_bright，I_dark，I_comb；4. The camera uses the same exposure parameters to sequentially acquire three images I_bright , I_dark and I_comb ;

5、使用opencv内置函数提取图像I_bright的角点，并根据相机内参I_c和畸变d_c以及打印棋盘格参数计算相机与棋盘格平面位姿R_i,T_i；5. Use the built-in function of opencv to extract the corner points of the image I_bright , and calculate the camera and the checkerboard plane pose Ri,T_i according to the camera's internal parameters_Ic and distortion d_c and the print checkerboard parameters;

6、取三幅图像I_bright，I_dark，I_comb的绿色通道，对每个像素做以下计算：6. Take the green channel of three images I_bright , I_dark and I_comb , and do the following calculation for each pixel:

其中，n为图像第n个像素，

ρ_t1＝0.9，ρ_t2＝0.1，按照相机的曝光度设置，I_t1＝120，I_t2＝10。得到的图像即为去除打印棋盘格背景的投影棋盘格图像，再进行高斯滤波。包含打印棋盘格背景的投影棋盘格图像如图2所示，去除背景的投影棋盘格图像如图3所示。Among them, n is the nth pixel of the image,

ρ_t1 =0.9, ρ_t2 =0.1, according to the exposure setting of the camera, I_t1 =120, I_t2 =10. The obtained image is the projected checkerboard image with the printed checkerboard background removed, and then Gaussian filtering is performed. The projected checkerboard image with the printed checkerboard background is shown in Figure 2, and the projected checkerboard image with the background removed is shown in Figure 3.

7、使用opencv内置函数对步骤6得到的图像提取角点，并使用步骤5得到的I_c、d_c、R_i、T_i计算角点的世界坐标；7. Use the opencv built-in function to extract the corner points from the image obtained in step 6, and use the I_c , d_c , R_i , and T_i obtained in step 5 to calculate the world coordinates of the corner points;

8、初始化外参R,T，计算投影仪与棋盘格平面位姿：8. Initialize the external parameters R, T, and calculate the plane pose of the projector and the checkerboard:

9、初始化投影仪内参I_p和畸变d_p，结合R′_iT′_i和步骤3中的投影棋盘格参数计算角点的世界坐标；9. Initialize the projector internal parameters I_p and distortion d_p , and calculate the world coordinates of the corner points in combination with R′_i T′_i and the projected checkerboard parameters in step 3;

10、变换棋盘格平面的位姿，重复步骤3-9，采集10幅图像的数据；10. Transform the pose of the checkerboard plane, repeat steps 3-9, and collect data of 10 images;

11、使用非线性优化方法，对未知参数I_p、d_p、R、T进行优化，使步骤7和步骤9得到的世界坐标误差最小，完成标定。11. Use the nonlinear optimization method to optimize the unknown parameters I_p , d_p , R and T to minimize the error of the world coordinates obtained in steps 7 and 9, and complete the calibration.

Claims (9)

1. A method for calibrating a camera-projector system, the method comprising the steps of:

step 1: adopt the existing camera calibration technique to calibrate the camera internal parameter I_cAnd distortion d_cCalibrating;

step 2: on the background of the printed checkerboard, the projector projects full bright, full dark and projected checkerboard patterns in sequence, and the camera shoots corresponding images I according to the same exposure parameters_bright、I_darkAnd I_comb；

And step 3: for image I_brightExtracting corner points of the checkerboard, and calculating the angular points according to the internal parameters I of the camera_cAnd distortion d_cCalculating relative pose R of camera and checkerboard plane_i,T_i；

And 4, step 4: from three images I_bright、I_darkAnd I_combSeparating out projection checkerboard image, extracting angular point of projection checkerboard, and obtaining projection checkerboard image from angular point of projection checkerboard image according to R_i,T_iCalculating world coordinates of the corner points;

and 5: initializing external parameters R and T of projector and camera according to R, T and R_i,T_iCalculating relative pose [ R ] of projector and checkerboard plane^’_i,T^’_i]=f(R,T,R_i,T_i)；

Step 6: initializing projector internal parameters I_pAnd distortion d_pAccording to I_p、d_pAnd R^’_i,T^’_iCalculating world coordinates of corner points of the projection chessboard;

and 7: changing the pose of the checkerboard plane, repeating the steps 2-6, and obtaining no less than 5 groups of calibration data;

and 8: aiming at the minimum world coordinate error obtained in the steps 4 and 6,for unknown parameters I_p、d_pAnd performing combined optimization on R and T to obtain a calibration result.

2. The calibration method of a camera-projector system as claimed in claim 1, wherein in the step 4, the method for separating the projected checkerboard image is:

for the nth pixel, the reflection coefficient rho is calculatedⁿ=（Iⁿ_comb-Iⁿ_dark）/(Iⁿ_bright-Iⁿ_dark) Then the value of the nth pixel of the checkerboard is projected

Comprises the following steps:

wherein the first condition and the third condition guarantee the contrast inside the projection checkerboard, the second condition guarantees the original boundary of the projection checkerboard,

、

respectively representing an upper limit and a lower limit of a reflection coefficient threshold;

、

respectively representing the upper and lower limits of the image intensity threshold, in terms of exposure of the image, in terms of I_combSetting a histogram of (1); and after the projection checkerboard is separated, Gaussian filtering is carried out to reduce errors caused by calculation errors of individual pixels.

3. A camera-projector system calibration method according to claim 1, wherein the camera calibration technique is preferably a gnomon method.

4. The calibration method of a camera-projector system as claimed in claim 1, wherein in step 2, the printing checkerboard is a single color.

5. A camera-projector system calibration method according to claim 4, wherein in step 2, the printing checkerboard is preferably green.

6. A camera-projector system calibration method according to claim 1, wherein the projection checkerboard is the same color as the printing checkerboard.

7. A camera-projector system calibration method as claimed in claim 1, wherein in step 4, three images I_bright、I_darkAnd I_combOnly single channel data of the color corresponding to the color of the printing checkerboard is used.

8. The calibration method of a camera-projector system as claimed in claim 1, wherein in the step 5, the relative pose of the projector and the checkerboard plane is calculated as follows:

。

9. a camera-projector system calibration method as claimed in claim 1, wherein the joint optimization in step 8 adopts a nonlinear optimization method.

Images