WO2023040095A1 - Camera calibration method and apparatus, electronic device, and storage medium - Google Patents

Abstract

The present invention provides a camera calibration method and apparatus, an electronic device, and a readable storage medium. The camera calibration method comprises: obtaining measured position coordinates of a plurality of position points in a camera coordinate system; determining an initial pose of a camera in a robot coordinate system; and further determining a compensation matrix for the camera according to the measured position coordinates of the plurality of position points in the camera coordinate system and the initial pose of the camera in the robot coordinate system. The camera calibration method can simultaneously determine external parameters of the camera and the compensation matrix for internal parameters of the camera, such that the position coordinates of the position points in the camera coordinate system can be compensated according to the compensation matrix to obtain more accurate position coordinates, thereby effectively improving the accuracy of a 3D point cloud image captured by a 3D camera.

Description

Translated fromChinese

相机的标定方法、装置、电子设备及存储介质Camera calibration method, device, electronic equipment and storage medium

优先权声明priority statement

本申请要求2021年9月16日递交的、申请号为CN202111083399.0、名称为“相机的标定方法、装置、电子设备及存储介质”的中国发明专利的优先权，上述专利的所有内容在此全部引入。This application claims the priority of the Chinese invention patent with the application number CN202111083399.0 and titled "camera calibration method, device, electronic equipment and storage medium" submitted on September 16, 2021. All the contents of the above patent are here Import all.

技术领域technical field

本发明涉及工业相机技术领域，特别是涉及一种相机的标定方法、装置、电子设备及存储介质。The invention relates to the technical field of industrial cameras, in particular to a camera calibration method, device, electronic equipment and storage medium.

背景技术Background technique

随着机器人技术的发展，越来越多的场景中需要使用到机器视觉技术。例如，可以使用机器视觉技术来识别待检测物体，在识别待检测物体之前，可以通过3D相机来采集待检测物体表面多个位置点在相机坐标系下的位置坐标，进一步可以根据物体表面多个位置点在相机坐标系下的位置坐标生成物体的3D点云图像。物体的3D点云图像的准确度将直接影响后续机器人对物体识别的准确度。例如，针对抓取类的机器人，便需要根据待抓取物体的3D点云图像来确定物体的待抓取位置。在具体应用场景中，如采集物体表面的多个位置点在相机坐标系下的位置坐标的过程中，相机的内参和外参(图像测量过程以及机器视觉应用中，为确定空间物体表面某点的三维几何位置与其在图像中对应点之间的相互关系，必须建立摄像机成像的几何模型，这些几何模型参数就是摄像机参数)将直接影响多个位置点在相机坐标系下的位置坐标的准确度。With the development of robot technology, more and more scenes need to use machine vision technology. For example, machine vision technology can be used to identify the object to be detected. Before identifying the object to be detected, a 3D camera can be used to collect the position coordinates of multiple points on the surface of the object to be detected in the camera coordinate system. The position coordinates of the position point in the camera coordinate system generate a 3D point cloud image of the object. The accuracy of the 3D point cloud image of the object will directly affect the accuracy of subsequent robot recognition of the object. For example, for a grasping robot, it is necessary to determine the position of the object to be grasped according to the 3D point cloud image of the object to be grasped. In specific application scenarios, such as in the process of collecting the position coordinates of multiple position points on the surface of an object in the camera coordinate system, the internal and external parameters of the camera (in the process of image measurement and machine vision applications, in order to determine a certain point on the surface of a space object The relationship between the three-dimensional geometric position and its corresponding point in the image, the geometric model of the camera imaging must be established, and these geometric model parameters are the camera parameters) will directly affect the accuracy of the position coordinates of multiple position points in the camera coordinate system .

现有技术中，3D点云图像的多个位置点是通过相同的相机内参获得的，但是，在对相机进行标定(标定，是摄像头图像在平面上的2D点与摄像头所拍摄的现实场景中的3D点之间转换所需参数的过程)后，由于物体表面距离相机不同深度的位置点的普适性不同，即内参通常不适用整个空间区域内深度不同的位置点，如果按照相 同的相机内参得到的3D点云图像，其在预设区间内和区间外的准确度会有不同。因此，如何能够确定相机采集的点云点对应的准确位置坐标，为本申请要解决的技术问题。In the prior art, multiple position points of the 3D point cloud image are obtained through the same camera internal reference, but when the camera is calibrated (calibration, it is the 2D point of the camera image on the plane and the real scene captured by the camera) After the process of converting the required parameters between 3D points), the universality of the position points at different depths from the object surface to the camera is different, that is, the internal reference is usually not applicable to the position points at different depths in the entire space area. If the same camera The accuracy of the 3D point cloud image obtained by the internal reference will be different within the preset interval and outside the interval. Therefore, how to determine the exact position coordinates corresponding to the point cloud points collected by the camera is a technical problem to be solved in this application.

发明内容Contents of the invention

本发明提供了一种相机的标定方法，用以精准确定相机采集的点云点对应的准确位置坐标，所述相机的标定方法包括：The present invention provides a camera calibration method, which is used to accurately determine the exact position coordinates corresponding to the point cloud points collected by the camera. The camera calibration method includes:

获取多个位置点在相机坐标系下的测量位置坐标；Obtain the measured position coordinates of multiple position points in the camera coordinate system;

确定相机在机器人坐标系下的初始位姿；Determine the initial pose of the camera in the robot coordinate system;

根据多个位置点在相机坐标系下的测量位置坐标及相机在机器人坐标系下的初始位姿，确定针对相机的补偿矩阵。According to the measured position coordinates of multiple position points in the camera coordinate system and the initial pose of the camera in the robot coordinate system, the compensation matrix for the camera is determined.

具体实施中，所述获取多个位置点在相机坐标系下的测量位置坐标，进一步包括：In a specific implementation, the acquisition of the measured position coordinates of multiple position points in the camera coordinate system further includes:

获取标定板移动至多个空间位置时，相机采集的多个标定板图像；Obtain multiple calibration plate images collected by the camera when the calibration plate moves to multiple spatial positions;

根据相机采集的多个标定板图像，获取多个位置点在相机坐标系下的测量位置坐标。According to the multiple calibration plate images collected by the camera, the measurement position coordinates of the multiple position points in the camera coordinate system are obtained.

具体实施中，所述确定相机在机器人坐标系下的初始位姿，进一步包括：In a specific implementation, the determination of the initial pose of the camera in the robot coordinate system further includes:

在机器人通过法兰带动标定板移动的过程中，获取多个法兰在机器人坐标系下的位姿以及多个标定板在相机坐标系下的位姿；When the robot moves the calibration plate through the flange, the poses of multiple flanges in the robot coordinate system and the poses of multiple calibration plates in the camera coordinate system are obtained;

根据多个法兰在机器人坐标系下的位姿及多个标定板在相机坐标系下的位姿，确定相机在机器人坐标系下的初始位姿。According to the poses of multiple flanges in the robot coordinate system and the poses of multiple calibration plates in the camera coordinate system, the initial pose of the camera in the robot coordinate system is determined.

具体实施中，所述根据多个位置点在相机坐标系下的测量位置坐标及相机在机器人坐标系下的初始位姿，确定针对相机的补偿矩阵，进一步包括：In specific implementation, the described compensation matrix for the camera is determined according to the measured position coordinates of the plurality of position points in the camera coordinate system and the initial pose of the camera in the robot coordinate system, further comprising:

在相机坐标系下根据空间区域的不同，将多个位置点划分为多组位置点；In the camera coordinate system, according to the different spatial regions, divide multiple location points into multiple groups of location points;

根据各组位置点在相机坐标系下的测量位置坐标及相机在机器人坐标系下的初始位姿，确定针对相机的补偿矩阵。According to the measured position coordinates of each group of position points in the camera coordinate system and the initial pose of the camera in the robot coordinate system, the compensation matrix for the camera is determined.

具体实施中，所述在相机坐标系下根据空间区域的不同，将多个位置点划分为多组位置点，进一步包括：In the specific implementation, according to different spatial regions under the camera coordinate system, dividing a plurality of position points into multiple groups of position points further includes:

在相机坐标系下将空间根据高度的不同划分为多个分层，各分层内划分有多个分区；In the camera coordinate system, the space is divided into multiple layers according to different heights, and each layer is divided into multiple partitions;

根据多个位置点在相机坐标系下的位置坐标，将处于同一分层内同一分区的位置点划分为一组位置点。According to the position coordinates of multiple position points in the camera coordinate system, the position points in the same partition in the same layer are divided into a group of position points.

具体实施中，在确定针对相机的补偿矩阵后，进一步包括：In the specific implementation, after determining the compensation matrix for the camera, it further includes:

获取相机采集的物体表面位置点在相机坐标系下的位置坐标；Obtain the position coordinates of the object surface position points collected by the camera in the camera coordinate system;

根据补偿矩阵对物体表面位置点在相机坐标系下的位置坐标进行校准，得到补偿后的准确位置坐标。According to the compensation matrix, the position coordinates of the object surface position points in the camera coordinate system are calibrated to obtain the accurate position coordinates after compensation.

具体实施中，所述根据补偿矩阵对物体表面位置点在相机坐标系下的位置坐标进行校准，得到补偿后的准确位置坐标，进一步包括：In the specific implementation, the described position coordinates of the object surface position points in the camera coordinate system are calibrated according to the compensation matrix, and the accurate position coordinates after compensation are obtained, further comprising:

根据物体表面位置点在相机坐标系下的位置坐标和相机的内参，确定物体表面位置点所处的分层和分区；According to the position coordinates of the object surface position point in the camera coordinate system and the internal reference of the camera, determine the layer and partition of the object surface position point;

根据与物体表面位置点所处的分层和分区对应的补偿矩阵，对物体表面位置点在相机坐标系下的位置坐标进行补偿，得到补偿后的准确位置坐标。According to the compensation matrix corresponding to the layer and partition of the object surface position point, the position coordinates of the object surface position point in the camera coordinate system are compensated, and the accurate position coordinates after compensation are obtained.

具体实施中，所述根据物体表面位置点在相机坐标系下的位置坐标和相机的内参，确定物体表面位置点所处的分层和分区，进一步包括：In specific implementation, the said according to the position coordinates of the position point on the surface of the object in the camera coordinate system and the internal reference of the camera, determine the stratification and partition where the position point on the surface of the object is located, further comprising:

根据物体表面位置点在相机坐标系下的z轴坐标，确定物体表面位置点所处的分层；According to the z-axis coordinates of the object surface position point in the camera coordinate system, determine the layer where the object surface position point is located;

根据物体表面位置点在相机坐标系下的x轴坐标及y轴坐标及相机的内参，确定物体表面位置点对应的像素坐标；According to the x-axis coordinates and y-axis coordinates of the object surface position points in the camera coordinate system and the internal parameters of the camera, determine the pixel coordinates corresponding to the object surface position points;

根据物体表面位置点对应的像素坐标确定物体表面位置点所处的分区。The partition where the object surface location point is located is determined according to the pixel coordinates corresponding to the object surface location point.

具体实施中，所述根据与物体表面位置点所处的分层和分区对应的补偿矩阵，对物体表面位置点在相机坐标系下的位置坐标进行校准，进一步包括：In a specific implementation, the calibration of the position coordinates of the object surface position point in the camera coordinate system according to the compensation matrix corresponding to the layer and partition where the object surface position point is located further includes:

在物体表面位置点处于跨分层和/或跨分区时，根据物体表面位置点与所跨分层和/或分区之间的距离确定各所跨分层和/或分区的权重；When the object surface position point is in a cross-layer and/or cross-region, determine the weight of each cross-layer and/or partition according to the distance between the object surface position point and the cross-layer and/or partition;

根据各所跨分层和/或分区的补偿矩阵分别对物体表面位置点在相机坐标系下的位置坐标进行补偿，得到物体表面位置点的多个补偿位置坐标；Compensating the position coordinates of the object surface position points in the camera coordinate system according to the compensation matrix of each crossing layer and/or partition, and obtaining multiple compensated position coordinates of the object surface position points;

根据各所跨分层和/或分区的权重对物体表面位置点的多个补偿位置坐标进行加权求和，获得加权求和后的补偿位置坐标。A weighted summation is performed on the plurality of compensated position coordinates of the object surface position points according to the weights of each spanned layer and/or partition to obtain the weighted summed compensated position coordinates.

具体实施中，所述根据各组位置点在相机坐标系下的测量位置坐标及相机在机器人坐标系下的初始位姿，确定针对相机的补偿矩阵，进一步包括：In specific implementation, the described compensation matrix for the camera is determined according to the measurement position coordinates of each group of position points in the camera coordinate system and the initial pose of the camera in the robot coordinate system, further comprising:

根据相机在机器人坐标系下的初始位姿确定各组位置点在相机坐标系下的初始理论位置坐标；Determine the initial theoretical position coordinates of each group of position points in the camera coordinate system according to the initial pose of the camera in the robot coordinate system;

拟合各组位置点在相机坐标系下的测量位置坐标和初始理论位置坐标，确定针对相机的初始补偿矩阵；Fit the measured position coordinates and initial theoretical position coordinates of each group of position points in the camera coordinate system, and determine the initial compensation matrix for the camera;

获取多个相机在机器人坐标系下调整后的位姿并确定多个各组位置点在相机调整后的理论位置坐标；Obtain the adjusted poses of multiple cameras in the robot coordinate system and determine the theoretical position coordinates of multiple groups of position points after camera adjustment;

根据多个相机在机器人坐标系下调整后返回的当前位姿、多个各组位置点在相机坐标系下的测量位置坐及在相机调整后的当前位置坐标调整初始补偿矩阵，直至各组位置点在相机坐标系下的测量位置坐标和当前理论位置坐标之间误差的欧式距离小于预设阈值和/或调整达到预设次数，将当前补偿矩阵作为针对相机的补偿矩阵。Adjust the initial compensation matrix according to the current pose returned by multiple cameras adjusted in the robot coordinate system, the measurement positions of multiple groups of position points in the camera coordinate system, and the current position coordinates after camera adjustment until each group position The Euclidean distance of the error between the measured position coordinates of the point in the camera coordinate system and the current theoretical position coordinates is less than the preset threshold and/or the adjustment reaches the preset number of times, and the current compensation matrix is used as the compensation matrix for the camera.

具体实施中，所述根据相机在机器人坐标系下的当前位姿，确定各组位置点在相机坐标系下的当前理论位置坐标，进一步包括：In the specific implementation, according to the current pose of the camera in the robot coordinate system, determining the current theoretical position coordinates of each group of position points in the camera coordinate system further includes:

获取与各组位置点相对应的法兰在机器人坐标系下的位姿及标定板在相机坐标系下的位姿；Obtain the pose of the flange corresponding to each group of position points in the robot coordinate system and the pose of the calibration plate in the camera coordinate system;

根据相机在机器人坐标系下的初始位姿、法兰在机器人坐标系下的位姿以及标定板在相机坐标系下的位姿，确定标定板相对于法兰的位姿；According to the initial pose of the camera in the robot coordinate system, the pose of the flange in the robot coordinate system, and the pose of the calibration plate in the camera coordinate system, determine the pose of the calibration plate relative to the flange;

根据各组位置点在标定板坐标系下的位置坐标、标定板相对于法兰的位姿及法兰在机器人坐标系下的位姿，确定各组位置点在机器人坐标系下的的位置坐标；Determine the position coordinates of each group of position points in the robot coordinate system according to the position coordinates of each group of position points in the calibration plate coordinate system, the pose of the calibration plate relative to the flange, and the pose of the flange in the robot coordinate system ;

根据各组位置点在机器人坐标系下的的位置坐标及相机在机器人坐标系下的初始位姿，确定各组位置点在相机坐标系下的初始理论位置坐标。According to the position coordinates of each group of position points in the robot coordinate system and the initial pose of the camera in the robot coordinate system, determine the initial theoretical position coordinates of each group of position points in the camera coordinate system.

本发明还提供了一种相机的标定方法装置，其特征在于，所述相机的标定装置包括：The present invention also provides a camera calibration method and device, characterized in that the camera calibration device includes:

坐标获取模块，获取多个位置点在相机坐标系下的测量位置坐标；The coordinate acquisition module acquires the measurement position coordinates of multiple position points in the camera coordinate system;

外参确定模块，用于确定相机在机器人坐标系下的初始位姿；The external parameter determination module is used to determine the initial pose of the camera in the robot coordinate system;

补偿矩阵确定模块，用于根据多个位置点在相机坐标系下的测量位置坐标及相机在机器人坐标系下的初始位姿，确定针对相机的补偿矩阵。The compensation matrix determination module is used to determine the compensation matrix for the camera according to the measured position coordinates of multiple position points in the camera coordinate system and the initial pose of the camera in the robot coordinate system.

具体实施中，所述坐标获取模块进一步包括：In specific implementation, the coordinate acquisition module further includes:

图像采集子模块，用于获取标定板移动至多个空间位置时，相机采集的多个标定板图像；The image acquisition sub-module is used to acquire multiple calibration plate images collected by the camera when the calibration plate moves to multiple spatial positions;

坐标计算子模块，用于根据相机采集的多个标定板图像，获取多个位置点在相机坐标系下的测量位置坐标。The coordinate calculation sub-module is used to obtain the measurement position coordinates of multiple position points in the camera coordinate system according to the multiple calibration plate images collected by the camera.

具体实施中，所述外参确定模块进一步包括：In a specific implementation, the external reference determination module further includes:

法兰坐标采集子模块，用于在机器人通过法兰带动标定板移动的过程中，获取多个法兰在机器人坐标系下的位姿以及多个标定板在相机坐标系下的位姿；The flange coordinate acquisition sub-module is used to obtain the poses of multiple flanges in the robot coordinate system and the poses of multiple calibration plates in the camera coordinate system when the robot drives the calibration plate to move through the flange;

外参计算子模块，用于根据多个法兰在机器人坐标系下的位姿及多个标定板在相机坐标系下的位姿，确定相机在机器人坐标系下的初始位姿。The external parameter calculation sub-module is used to determine the initial pose of the camera in the robot coordinate system according to the poses of multiple flanges in the robot coordinate system and the poses of multiple calibration plates in the camera coordinate system.

具体实施中，所述补偿矩阵确定模块进一步包括：In specific implementation, the compensation matrix determination module further includes:

坐标分组子模块，用于在相机坐标系下根据空间区域的不同，将多个位置点划分为多组位置点；The coordinate grouping sub-module is used to divide multiple location points into multiple groups of location points according to different spatial regions in the camera coordinate system;

补偿矩阵求取子模块，用于根据各组位置点在相机坐标系下的测量位置坐标及相机在机器人坐标系下的初始位姿，确定针对相机的补偿矩阵。The compensation matrix calculation sub-module is used to determine the compensation matrix for the camera according to the measured position coordinates of each group of position points in the camera coordinate system and the initial pose of the camera in the robot coordinate system.

具体实施中，所述坐标分组子模块进一步包括：In specific implementation, the coordinate grouping submodule further includes:

空间分割子模块，用于在相机坐标系下将空间根据高度的不同划分为多个分层，各分层内划分有多个分区；The space segmentation sub-module is used to divide the space into multiple layers according to different heights in the camera coordinate system, and each layer is divided into multiple partitions;

分组子模块，用于根据多个位置点在相机坐标系下的位置坐标，将处于同一分层内同一分区的位置点划分为一组位置点。The grouping sub-module is used to divide the position points in the same partition in the same layer into a group of position points according to the position coordinates of the multiple position points in the camera coordinate system.

具体实施中，所述相机的标定装置还包括：In specific implementation, the calibration device of the camera also includes:

位置点坐标采集模块，用于在确定针对相机的补偿矩阵后，获取相机采集的物体表面位置点在相机坐标系下的位置坐标；The position point coordinate acquisition module is used to obtain the position coordinates of the object surface position points collected by the camera in the camera coordinate system after determining the compensation matrix for the camera;

准确位置坐标求取模块，用于根据补偿矩阵对物体表面位置点在相机坐标系下的位置坐标进行校准，得到补偿后的准确位置坐标。The accurate position coordinate calculation module is used to calibrate the position coordinates of the object surface position points in the camera coordinate system according to the compensation matrix, and obtain the accurate position coordinates after compensation.

具体实施中，所述准确位置坐标求取模块，进一步包括：In specific implementation, the module for obtaining exact position coordinates further includes:

空间区域确定子模块，用于根据物体表面位置点在相机坐标系下的位置坐标和相机的内参，确定物体表面位置点所处的分层和分区；The spatial area determination sub-module is used to determine the layer and partition of the object surface position point according to the position coordinates of the object surface position point in the camera coordinate system and the internal reference of the camera;

准确位置坐标获取子模块，用于根据与物体表面位置点所处的分层和分区对应的补偿矩阵，对物体表面位置点在相机坐标系下的位置坐标进行补偿，得到补偿后的准确位置坐标。The accurate position coordinate acquisition sub-module is used to compensate the position coordinates of the object surface position points in the camera coordinate system according to the compensation matrix corresponding to the layer and partition where the object surface position points are located, and obtain the correct position coordinates after compensation .

具体实施中，所述空间区域确定子模块进一步包括：In specific implementation, the submodule for determining the spatial area further includes:

分层确定子模块，用于根据物体表面位置点在相机坐标系下的z轴坐标，确定物体表面位置点所处的分层；The stratification determination submodule is used to determine the stratification of the object surface position point according to the z-axis coordinate of the object surface position point in the camera coordinate system;

像素坐标确定子模块，用于根据物体表面位置点在相机坐标系下的x轴坐标及y轴坐标及相机的内参，确定物体表面位置点对应的像素坐标；The pixel coordinate determination sub-module is used to determine the pixel coordinates corresponding to the object surface position point according to the x-axis coordinate and y-axis coordinate of the object surface position point in the camera coordinate system and the internal reference of the camera;

分区确定子模块，用于根据物体表面位置点对应的像素坐标确定物体表面位置点所处的分区。The partition determining submodule is configured to determine the partition where the object surface location point is located according to the pixel coordinates corresponding to the object surface location point.

具体实施中，所述准确位置坐标获取子模块进一步包括：In specific implementation, the sub-module for obtaining accurate position coordinates further includes:

权重确定子模块，用于在物体表面位置点处于跨分层和/或跨分区时，根据物体表面位置点与所跨分层和/或分区之间的距离确定各所跨分层和/或分区的权重；The weight determination submodule is used to determine each spanned layer and/or partition according to the distance between the object surface position point and the spanned layer and/or partition when the object surface position point is in a cross-layer and/or cross-region the weight of;

补偿子模块，用于根据各所跨分层和/或分区的补偿矩阵分别对物体表面位置点在相机坐标系下的位置坐标进行补偿，得到物体表面位置点的多个补偿位置坐标；The compensation sub-module is used to compensate the position coordinates of the object surface position points in the camera coordinate system according to the compensation matrix of each crossing layer and/or partition, and obtain multiple compensated position coordinates of the object surface position points;

加权求和子模块，用于根据各所跨分层和/或分区的权重对物体表面位置点的多个补偿位置坐标进行加权求和，获得加权求和后的补偿位置坐标。The weighted summation sub-module is used to perform weighted summation of multiple compensated position coordinates of the object surface position points according to the weights of each crossed layer and/or partition, and obtain the weighted summed compensated position coordinates.

具体实施中，所述补偿矩阵求取子模块进一步包括：In specific implementation, the submodule for calculating the compensation matrix further includes:

初始理论位置坐标获取子模块，用于根据相机在机器人坐标系下的初始位姿确定各组位置点在相机坐标系下的初始理论位置坐标；The initial theoretical position coordinate acquisition sub-module is used to determine the initial theoretical position coordinates of each group of position points in the camera coordinate system according to the initial pose of the camera in the robot coordinate system;

初始补偿矩阵获取子模块，用于拟合各组位置点在相机坐标系下的测量位置坐标和初始理论位置坐标，确定针对相机的初始补偿矩阵；The initial compensation matrix acquisition sub-module is used to fit the measured position coordinates and initial theoretical position coordinates of each group of position points in the camera coordinate system, and determine the initial compensation matrix for the camera;

调整数据获取子模块，用于获取多个相机在机器人坐标系下调整后的位姿并确定多个各组位置点在相机调整后的理论位置坐标；The adjustment data acquisition sub-module is used to obtain the adjusted poses of multiple cameras in the robot coordinate system and determine the theoretical position coordinates of multiple sets of position points after camera adjustment;

矩阵确定子模块，用于根据多个相机在机器人坐标系下调整后返回的当前位姿、多个各组位置点在相机坐标系下的测量位置坐及在相机调整后的当前位置坐标调整初始补偿矩阵，直至各组位置点在相机坐标系下的测量位置坐标和当前理论位置坐标之间误差的欧式距离小于预设阈值和/或调整达到预设次数，将当前补偿矩阵作为针对相机的补偿矩阵。The matrix determination sub-module is used to adjust the initial pose according to the current pose returned by multiple cameras adjusted in the robot coordinate system, the measurement positions of multiple groups of position points in the camera coordinate system, and the current position coordinates after camera adjustment. Compensation matrix, until the Euclidean distance of the error between the measured position coordinates of each group of position points in the camera coordinate system and the current theoretical position coordinates is less than the preset threshold and/or the adjustment reaches the preset number of times, the current compensation matrix is used as compensation for the camera matrix.

具体实施中，所述初始理论位置坐标获取子模块进一步包括：In specific implementation, the submodule of obtaining the initial theoretical position coordinates further includes:

法兰及标定板位姿获取子模块，用于获取与各组位置点相对应的法兰在机器人坐标系下的位姿及标定板在相机坐标系下的位姿；The flange and calibration board pose acquisition sub-module is used to obtain the pose of the flange corresponding to each group of position points in the robot coordinate system and the pose of the calibration board in the camera coordinate system;

标定板相对法兰位姿获取子模块，用于根据相机在机器人坐标系下的初始位姿、 法兰在机器人坐标系下的位姿以及标定板在相机坐标系下的位姿，确定标定板相对于法兰的位姿；The sub-module for obtaining the pose of the calibration board relative to the flange is used to determine the calibration board according to the initial pose of the camera in the robot coordinate system, the pose of the flange in the robot coordinate system, and the pose of the calibration board in the camera coordinate system pose relative to the flange;

位置点相对机器人坐标获取子模块，用于根据各组位置点在标定板坐标系下的位置坐标、标定板相对于法兰的位姿及法兰在机器人坐标系下的位姿，确定各组位置点在机器人坐标系下的的位置坐标；The position point relative robot coordinate acquisition sub-module is used to determine the position of each group of position points according to the position coordinates of each group of position points in the coordinate system of the calibration plate, the pose of the calibration plate relative to the flange, and the pose of the flange in the robot coordinate system. The position coordinates of the position point in the robot coordinate system;

初始理论位置坐标确定子模块，用于根据各组位置点在机器人坐标系下的的位置坐标及相机在机器人坐标系下的初始位姿，确定各组位置点在相机坐标系下的初始理论位置坐标。The initial theoretical position coordinate determination sub-module is used to determine the initial theoretical position of each group of position points in the camera coordinate system according to the position coordinates of each group of position points in the robot coordinate system and the initial pose of the camera in the robot coordinate system coordinate.

本发明还提供了一种电子设备，包括存储器、处理器及存储在存储器上并可在处理器上运行的计算机程序，所述处理器执行所述计算机程序时实现所述相机的标定方法。The present invention also provides an electronic device, which includes a memory, a processor, and a computer program stored on the memory and operable on the processor. When the processor executes the computer program, the calibration method of the camera is implemented.

本发明还提供了一种计算机可读存储介质，所述计算机可读存储介质存储有执行所述相机的标定方法的计算机程序。The present invention also provides a computer-readable storage medium, the computer-readable storage medium storing a computer program for executing the camera calibration method.

本发明提供的相机的标定方法、装置、电子设备及可读存储介质，其中，该方法包括：获取多个位置点在相机坐标系下的测量位置坐标；确定相机在机器人坐标系下的初始位姿；进而根据多个位置点在相机坐标系下的测量位置坐标及相机在机器人坐标系下的初始位姿，确定针对相机的补偿矩阵。该相机的标定方法可以同时确定相机的外参以及针对相机内参的补偿矩阵，从而可以根据补偿矩阵对相机坐标系下的位置点的位置坐标进行补偿，得到更为准确的位置坐标，进而有效提升3D相机拍摄的3D点云图像的精度。The camera calibration method, device, electronic equipment and readable storage medium provided by the present invention, wherein the method includes: acquiring the measurement position coordinates of multiple position points in the camera coordinate system; determining the initial position of the camera in the robot coordinate system Then, according to the measured position coordinates of multiple position points in the camera coordinate system and the initial pose of the camera in the robot coordinate system, the compensation matrix for the camera is determined. The camera calibration method can simultaneously determine the external parameters of the camera and the compensation matrix for the internal parameters of the camera, so that the position coordinates of the position points in the camera coordinate system can be compensated according to the compensation matrix, and more accurate position coordinates can be obtained, thereby effectively improving The accuracy of the 3D point cloud image captured by the 3D camera.

附图说明Description of drawings

为了更清楚地说明本发明具体实施方式或现有技术中的技术方案，下面将对具体实施方式或现有技术描述中所需要使用的附图作简单地介绍，显而易见地，下面描述中的附图仅仅是本发明的一些具体实施方式，对于本领域普通技术人员来讲，在不付出创造性劳动的前提下，还可以根据这些附图获得其他的附图。在附图中：In order to more clearly illustrate the specific implementation of the present invention or the technical solutions in the prior art, the following will briefly introduce the accompanying drawings that need to be used in the specific implementation or description of the prior art. Obviously, the accompanying drawings in the following description The drawings are only some specific implementations of the present invention, and those skilled in the art can also obtain other drawings based on these drawings without creative work. In the attached picture:

图1是根据本发明一个具体实施方式中相机的标定方法的流程示意图；Fig. 1 is a schematic flow chart of a camera calibration method according to a specific embodiment of the present invention;

图2是根据本发明一个具体实施方式中获取多个位置点在相机坐标系下的位置坐标的流程示意图；FIG. 2 is a schematic flow diagram of obtaining the position coordinates of multiple position points in the camera coordinate system according to a specific embodiment of the present invention;

图3是根据本发明一个具体实施方式中确定相机在机器人坐标系下的初始位置坐标的流程示意图；3 is a schematic flow diagram of determining the initial position coordinates of the camera in the robot coordinate system according to a specific embodiment of the present invention;

图4是根据本发明一个具体实施方式中确定针对相机的补偿矩阵的流程示意图；FIG. 4 is a schematic flow diagram of determining a compensation matrix for a camera according to a specific embodiment of the present invention;

图5是根据本发明一个具体实施方式中位置点分组的流程示意图；Fig. 5 is a schematic flow diagram of location point grouping according to a specific embodiment of the present invention;

图6是根据本发明一个具体实施方式中得到补偿后的准确位置坐标的流程示意图；Fig. 6 is a schematic flow chart of accurate position coordinates after compensation according to a specific embodiment of the present invention;

图7是根据本发明一个具体实施方式中确定物体表面位置点所处的分层和分区的具体流程示意图；Fig. 7 is a specific flow diagram of determining the layering and partitioning of the object surface position point according to a specific embodiment of the present invention;

图8是根据本发明一个具体实施方式中补偿跨分层和/或跨分区位置点的流程示意图；Fig. 8 is a schematic flowchart of compensating cross-layer and/or cross-partition location points according to a specific embodiment of the present invention;

图9是根据本发明一个具体实施方式中确定针对相机的补偿矩阵的具体流程示意图；Fig. 9 is a schematic flowchart of determining a compensation matrix for a camera according to a specific embodiment of the present invention;

图10是根据本发明一个具体实施方式中确定各组位置点在相机坐标系下的初始理论位置坐标的具体流程示意图；Fig. 10 is a specific flow diagram of determining the initial theoretical position coordinates of each group of position points in the camera coordinate system according to a specific embodiment of the present invention;

图11是根据本发明一个具体实施方式中相机的标定装置的结构示意图；Fig. 11 is a schematic structural diagram of a camera calibration device according to a specific embodiment of the present invention;

图12是根据本发明一个具体实施方式中坐标获取模块的结构示意图；Fig. 12 is a schematic structural diagram of a coordinate acquisition module according to a specific embodiment of the present invention;

图13是根据本发明一个具体实施方式中外参确定模块的结构示意图；Fig. 13 is a schematic structural diagram of an external parameter determination module according to a specific embodiment of the present invention;

图14是根据本发明一个具体实施方式中补偿矩阵确定模块的结构示意图；Fig. 14 is a schematic structural diagram of a compensation matrix determination module according to a specific embodiment of the present invention;

图15是根据本发明一个具体实施方式中坐标分组子模块的结构示意图；Fig. 15 is a schematic structural diagram of a coordinate grouping submodule according to a specific embodiment of the present invention;

图16是根据本发明一个具体实施方式中准确位置坐标求取模块的结构示意图；Fig. 16 is a schematic structural diagram of an accurate position coordinate obtaining module according to a specific embodiment of the present invention;

图17是根据本发明一个具体实施方式中空间区域确定子模块的结构示意图；Fig. 17 is a schematic structural diagram of a spatial region determination submodule according to a specific embodiment of the present invention;

图18是根据本发明一个具体实施方式中准确位置坐标获取子模块的结构示意图；Fig. 18 is a schematic structural diagram of an accurate position coordinate acquisition submodule according to a specific embodiment of the present invention;

图19是根据本发明一个具体实施方式中补偿矩阵求取子模块的结构示意图；Fig. 19 is a schematic structural diagram of a compensation matrix calculation sub-module according to a specific embodiment of the present invention;

图20是根据本发明一个具体实施方式中初始理论位置坐标获取子模块的结构示意图；Fig. 20 is a schematic structural diagram of an initial theoretical position coordinate acquisition submodule according to a specific embodiment of the present invention;

图21是根据本发明一个具体实施方式中相机标定系统的结构示意图；Fig. 21 is a schematic structural diagram of a camera calibration system according to a specific embodiment of the present invention;

图22是根据本发明一个具体实施方式中空间区域切分的原理示意图。Fig. 22 is a schematic diagram of the principle of spatial region segmentation according to a specific embodiment of the present invention.

具体实施方式Detailed ways

为使本发明具体实施方式的目的、技术方案和优点更加清楚明白，下面结合附图对本发明具体实施方式做进一步详细说明。在此，本发明的示意性具体实施方式及其说明用于解释本发明，但并不作为对本发明的限定。In order to make the purpose, technical solutions and advantages of the specific embodiments of the present invention clearer, the specific embodiments of the present invention will be further described in detail below in conjunction with the accompanying drawings. Here, the exemplary embodiments and descriptions thereof of the present invention are used to explain the present invention, but not to limit the present invention.

在标定工作开展之前，可以搭建一用于相机标定的系统，该系统可以包括相机、具有机械臂的机器人及标定板(标定板，Calibration Target，在机器视觉、图像测量、摄影测量、三维重建等应用中，可以用于校正镜头畸变等)，其中，该标定板连接于机械臂前端的法兰，进而可以在机械臂的带动下随机械臂移动以改变位置，如图21所示，机器人的机械臂前端可以通过法兰固定标定板，相机则可以固定在预设位置，此外，相机在机器人坐标系下的初始位置坐标为未知量。进一步的，相机位置的设置在实施中可以有多种方案。例如，相机可以固定在机器人外部的支架上，也可以固定在机器人上，进而跟随机器臂进行位置变化，本申请中以相机固定在机器人外部的支架上为例进行说明。Before the calibration work is carried out, a system for camera calibration can be built, which can include a camera, a robot with a robotic arm, and a calibration board (calibration board, Calibration Target, in machine vision, image measurement, photogrammetry, 3D reconstruction, etc. In applications, it can be used to correct lens distortion, etc.), wherein the calibration plate is connected to the flange at the front end of the mechanical arm, and then can be driven by the mechanical arm to move with the mechanical arm to change its position. As shown in Figure 21, the robot’s The front end of the robotic arm can fix the calibration plate through the flange, and the camera can be fixed at a preset position. In addition, the initial position coordinates of the camera in the robot coordinate system are unknown. Further, there may be various schemes for setting the camera position in implementation. For example, the camera can be fixed on a bracket outside the robot, or can be fixed on the robot, and then follow the robot arm to change its position. In this application, the camera is fixed on the bracket outside the robot as an example for illustration.

如图1所示，本发明提供了一种相机的标定方法，用以精准确定相机采集的点云点对应的准确位置坐标，所述相机的标定方法包括：As shown in Figure 1, the present invention provides a camera calibration method for accurately determining the exact position coordinates corresponding to the point cloud points collected by the camera. The camera calibration method includes:

步骤101：获取多个位置点在相机坐标系下的测量位置坐标；Step 101: Obtain the measured position coordinates of multiple position points in the camera coordinate system;

步骤102：确定相机在机器人坐标系下的初始位姿；Step 102: Determine the initial pose of the camera in the robot coordinate system;

步骤103：根据多个位置点在相机坐标系下的测量位置坐标及相机在机器人坐标系下的初始位姿，确定针对相机的补偿矩阵。Step 103: Determine a compensation matrix for the camera according to the measured position coordinates of multiple position points in the camera coordinate system and the initial pose of the camera in the robot coordinate system.

补偿矩阵中包含对相机内参的补偿以及对外参的补偿，相机的内参和外参都会影响相机采集的位置点的位置坐标的准确度，这里可以理解为补偿矩阵会补偿内参和外参偏差引发的误差。The compensation matrix includes the compensation for the internal parameters of the camera and the compensation for the external parameters. The internal parameters and external parameters of the camera will affect the accuracy of the position coordinates of the position points collected by the camera. It can be understood here that the compensation matrix will compensate for the deviation caused by the internal parameters and external parameters. error.

具体实施中，所述步骤101：所述获取多个位置点在相机坐标系下的测量位置坐标，可以有多种实施方案。例如，如图2所示，所述步骤101：获取多个位置点在相机坐标系下的测量位置坐标可以进一步包括：In a specific implementation, the step 101: acquiring the measured position coordinates of multiple position points in the camera coordinate system may have various implementations. For example, as shown in FIG. 2, the step 101: obtaining the measured position coordinates of multiple position points in the camera coordinate system may further include:

步骤201：获取标定板移动至多个空间位置时，相机采集的多个标定板图像；Step 201: Obtain multiple calibration plate images captured by the camera when the calibration plate moves to multiple spatial positions;

步骤202：根据相机采集的多个标定板图像，获取多个位置点在相机坐标系下的测量位置坐标。Step 202: Obtain the measurement position coordinates of multiple position points in the camera coordinate system according to the multiple calibration plate images collected by the camera.

具体实施中，可以通过调整机器人机械臂来改变标定板的位置，实施中，可以获 取每次调整机械臂时，机械臂前端的法兰在机器人坐标系下的位姿，该位姿为已知量，其可以通过机器人上的传感器来测量确定；同时，在机器人的机械臂每次调整标定板的位姿后，可以通过相机采集标定板上的位置点在相机坐标系下的测量位置坐标。In the specific implementation, the position of the calibration plate can be changed by adjusting the robotic arm of the robot. In the implementation, the pose of the flange at the front end of the robotic arm in the robot coordinate system can be obtained each time the robotic arm is adjusted. This pose is known It can be measured and determined by the sensor on the robot; at the same time, after the robot’s mechanical arm adjusts the pose of the calibration plate each time, the measured position coordinates of the position points on the calibration plate in the camera coordinate system can be collected through the camera.

具体实施中，确定相机在机器人坐标系下的初始位姿在实施中可以有多种方案，例如，如图3所示，可以多次采集法兰在机器人坐标系下的位姿以及标定板在相机坐标系下的位姿，进而根据，所述步骤102：确定相机在机器人坐标系下的初始位姿，可以进一步包括：In the specific implementation, there are many options for determining the initial pose of the camera in the robot coordinate system. For example, as shown in Figure 3, the pose of the flange in the robot coordinate system can be collected multiple times and the The pose under the camera coordinate system, and then according to, the step 102: determining the initial pose of the camera under the robot coordinate system may further include:

步骤301：在机器人通过法兰带动标定板移动的过程中，获取多个法兰在机器人坐标系下的位姿以及多个标定板在相机坐标系下的位姿；Step 301: Obtain the poses of multiple flanges in the robot coordinate system and the poses of multiple calibration plates in the camera coordinate system when the robot drives the calibration plate to move through the flange;

步骤302：根据多个法兰在机器人坐标系下的位姿及多个标定板在相机坐标系下的位姿，确定相机在机器人坐标系下的初始位姿。Step 302: Determine the initial pose of the camera in the robot coordinate system according to the poses of the multiple flanges in the robot coordinate system and the poses of the multiple calibration plates in the camera coordinate system.

进一步的，所述步骤302：根据多个法兰在机器人坐标系下的位姿及多个标定板在相机坐标系下的位姿，确定相机在机器人坐标系下的初始位姿，可以根据如下公式进行计算：Further, the step 302: according to the poses of multiple flanges in the robot coordinate system and the poses of multiple calibration plates in the camera coordinate system, determine the initial pose of the camera in the robot coordinate system, which can be based on the following Formula to calculate:

其中，

表示相机在机器人坐标系下的初始位姿；

表示法兰在机器人坐标系下的位姿；

表示标定板相对于法兰的位姿；

表示位置点在相机坐标系下的测量位置坐标。可以理解的是，由于在机械臂通过法兰带动标定板移动的过程中，标定板相对于法兰的位姿不会改变，即

不变，因此可以根据上述公式来求取相机在机器人坐标系下的初始位姿。in,

Indicates the initial pose of the camera in the robot coordinate system;

Indicates the pose of the flange in the robot coordinate system;

Indicates the pose of the calibration plate relative to the flange;

Indicates the measured position coordinates of the position point in the camera coordinate system. It can be understood that the pose of the calibration plate relative to the flange will not change when the mechanical arm drives the calibration plate to move through the flange, that is,

Therefore, the initial pose of the camera in the robot coordinate system can be calculated according to the above formula.

具体实施中，所述步骤103：所述根据多个位置点在相机坐标系下的测量位置坐标及相机在机器人坐标系下的初始位姿，确定针对相机的补偿矩阵，有多种实施方案，例如，如图4所示，可以进一步包括：In the specific implementation, the step 103: according to the measurement position coordinates of multiple position points in the camera coordinate system and the initial pose of the camera in the robot coordinate system, determine the compensation matrix for the camera, there are various implementations, For example, as shown in Figure 4, it may further include:

步骤401：在相机坐标系下根据空间区域的不同，将多个位置点划分为多组位置点；Step 401: Divide multiple location points into multiple groups of location points according to different spatial regions in the camera coordinate system;

步骤402：根据各组位置点在相机坐标系下的测量位置坐标及相机在机器人坐标系下的初始位姿，确定针对相机的补偿矩阵。Step 402: Determine a compensation matrix for the camera according to the measured position coordinates of each group of position points in the camera coordinate system and the initial pose of the camera in the robot coordinate system.

在相机坐标系下将位置点根据空间区域进行划分，可以使得不同空间深度及区域 均具有对应的补偿矩阵，从而提升相机坐标系下位置点坐标的准确度。In the camera coordinate system, the location points are divided according to the spatial regions, so that different spatial depths and regions have corresponding compensation matrices, thereby improving the accuracy of the location point coordinates in the camera coordinate system.

具体实施中，可以根据相机在机器人坐标系下的初始位姿、位置点在相机坐标系下的测量位置坐标及法兰在机器人坐标系下的位姿确定标定板相对于机器人法兰的位姿，进一步的，再根据标定板中各位置点在标定板坐标系下的位置坐标以及法兰在机器人坐标系下的位姿，可以确定多个位置点在相机坐标系中的理论位置坐标

其中，

表示标定板上的多个点在相机坐标系下的理论位置坐标构成的矩阵，即n个位置点对应的理论坐标

可以表示为：

In specific implementation, the pose of the calibration plate relative to the robot flange can be determined according to the initial pose of the camera in the robot coordinate system, the measured position coordinates of the position point in the camera coordinate system, and the pose of the flange in the robot coordinate system , further, according to the position coordinates of each position point in the calibration plate in the calibration plate coordinate system and the pose of the flange in the robot coordinate system, the theoretical position coordinates of multiple position points in the camera coordinate system can be determined

in,

Indicates the matrix formed by the theoretical position coordinates of multiple points on the calibration board in the camera coordinate system, that is, the theoretical coordinates corresponding to n position points

It can be expressed as:

具体实施中，将多个位置点划分为多组位置点，可以有多种实施方案。例如，如图22所示，在相机坐标系下按照空间中不同切分区域对多个位置点进行划分，具体可以按照高度分成多个分层，然后对每个分层按照设定的区域划分为多个分区，得到多组位置点。具体的，如图5所示，所述步骤401：在相机坐标系下根据空间区域的不同，将多个位置点划分为多组位置点，可以进一步包括：In specific implementation, there may be various implementation schemes for dividing multiple location points into multiple groups of location points. For example, as shown in Figure 22, in the camera coordinate system, multiple location points are divided according to different segmentation areas in space. Specifically, they can be divided into multiple layers according to height, and then each layer is divided according to the set area For multiple partitions, multiple sets of location points are obtained. Specifically, as shown in FIG. 5 , the step 401: dividing multiple location points into multiple groups of location points according to different spatial regions in the camera coordinate system may further include:

步骤501：在相机坐标系下将空间根据高度的不同划分为多个分层，各分层内划分有多个分区；Step 501: In the camera coordinate system, the space is divided into multiple layers according to different heights, and each layer is divided into multiple partitions;

步骤502：根据多个位置点在相机坐标系下的位置坐标，将处于同一分层内同一分区的位置点划分为一组位置点。Step 502: According to the position coordinates of the multiple position points in the camera coordinate system, divide the position points in the same partition in the same layer into a group of position points.

具体实施中，在获得补偿矩阵之后，还可以根据补偿矩阵对相机坐标系下的位置点的位置坐标进行补偿，从而得到准确的位置坐标。如图1所示，在步骤103：根据多个位置点在相机坐标系下的位置坐标及相机在机器人坐标系下的初始位姿，确定针对相机的补偿矩阵后，可以进一步包括：In a specific implementation, after the compensation matrix is obtained, the position coordinates of the position points in the camera coordinate system may also be compensated according to the compensation matrix, so as to obtain accurate position coordinates. As shown in Figure 1, in step 103: after determining the compensation matrix for the camera according to the position coordinates of multiple position points in the camera coordinate system and the initial pose of the camera in the robot coordinate system, it may further include:

步骤104：获取相机采集的物体表面位置点在相机坐标系下的位置坐标；Step 104: Obtain the position coordinates of the object surface position points collected by the camera in the camera coordinate system;

步骤105：根据补偿矩阵对物体表面位置点在相机坐标系下的位置坐标进行校准，得到补偿后的准确位置坐标。Step 105: Calibrate the position coordinates of the object surface position points in the camera coordinate system according to the compensation matrix, and obtain the correct position coordinates after compensation.

具体实施中，在获得补偿矩阵之后，还可以通过补偿矩阵对相机的位置坐标进行补偿，因而可以降低对相机拍摄的位置点的初始位置坐标的准确度的要求，进而降低对相机的精度要求。In a specific implementation, after the compensation matrix is obtained, the position coordinates of the camera can also be compensated by the compensation matrix, thus reducing the requirement for the accuracy of the initial position coordinates of the position points captured by the camera, and further reducing the accuracy requirements for the camera.

具体实施中，所述步骤105：根据补偿矩阵对物体表面位置点在相机坐标系下的位置坐标进行校准，得到补偿后的准确位置坐标，可以有多种实施方案，例如，如图6所示，可以进一步包括：In the specific implementation, the step 105: according to the compensation matrix, the position coordinates of the object surface position points in the camera coordinate system are calibrated to obtain the correct position coordinates after compensation. There can be various implementations, for example, as shown in Figure 6 , which can further include:

步骤601：根据物体表面位置点在相机坐标系下的位置坐标和相机的内参，确定物体表面位置点所处的分层和分区；Step 601: According to the position coordinates of the object surface position point in the camera coordinate system and the internal reference of the camera, determine the layer and partition where the object surface position point is located;

步骤602：根据与物体表面位置点所处的分层和分区对应的补偿矩阵，对物体表面位置点在相机坐标系下的位置坐标进行补偿，得到补偿后的准确位置坐标。Step 602: Compensate the position coordinates of the object surface position points in the camera coordinate system according to the compensation matrix corresponding to the layer and partition where the object surface position points are located, and obtain the correct position coordinates after compensation.

具体实施中，所述步骤601：根据物体表面位置点在相机坐标系下的位置坐标和相机的内参，确定物体表面位置点所处的分层和分区，可以有多种实施方案，例如，可以分别提取物体表面位置点在相机坐标系下位置坐标的z轴坐标、x轴坐标及y轴坐标，进而确定位置点的分区和分层，具体的，如图7所示，可以按如下步骤确定物体表面位置点所处的分层和分区：In specific implementation, the step 601: according to the position coordinates of the object surface position point in the camera coordinate system and the internal reference of the camera, determine the layer and partition where the object surface position point is located. There can be multiple implementations, for example, Extract the z-axis coordinates, x-axis coordinates, and y-axis coordinates of the position coordinates of the object surface position points in the camera coordinate system, and then determine the partition and layering of the position points. Specifically, as shown in Figure 7, it can be determined according to the following steps The layering and partitioning of the object surface position points:

步骤701：根据物体表面位置点在相机坐标系下的z轴坐标，确定物体表面位置点所处的分层；Step 701: According to the z-axis coordinates of the object surface position point in the camera coordinate system, determine the layer where the object surface position point is located;

步骤702：根据物体表面位置点在相机坐标系下的x轴坐标及y轴坐标及相机的内参，确定物体表面位置点对应的像素坐标；Step 702: According to the x-axis coordinates and y-axis coordinates of the object surface position points in the camera coordinate system and the internal parameters of the camera, determine the pixel coordinates corresponding to the object surface position points;

步骤703：根据物体表面位置点对应的像素坐标确定物体表面位置点所处的分区。Step 703: Determine the partition where the object surface location point is located according to the pixel coordinates corresponding to the object surface location point.

具体实施中，所述步骤602：根据与物体表面位置点所处的分层和分区对应的补偿矩阵，对物体表面位置点在相机坐标系下的位置坐标进行补偿，得到补偿后的准确位置坐标，可以有多种实施方案。例如，由于位置点在实际当中有可能在各分层或是分区之间，不完全属于任意分层或是分区，因此，为了提升这种跨分层或是分区的位置点的补偿精度，对物体表面位置点在相机坐标系下的位置坐标进行补偿时，如图8所示，可以进一步包括：In specific implementation, the step 602: Compensate the position coordinates of the object surface position points in the camera coordinate system according to the compensation matrix corresponding to the layer and partition where the object surface position points are located, and obtain the correct position coordinates after compensation , can have various implementations. For example, since the location point may be between layers or partitions in practice, and does not completely belong to any layer or partition, in order to improve the compensation accuracy of the location point across layers or partitions, the When the position coordinates of the object surface position point in the camera coordinate system are compensated, as shown in Figure 8, it can further include:

步骤801：在物体表面位置点处于跨分层和/或跨分区时，根据物体表面位置点与所跨分层和/或分区之间的距离确定各所跨分层和/或分区的权重；Step 801: When the object surface position point is across layers and/or across partitions, determine the weight of each spanned layer and/or partition according to the distance between the object surface position point and the spanned layer and/or partition;

步骤802：根据各所跨分层和/或分区的补偿矩阵分别对物体表面位置点在相机坐标系下的位置坐标进行补偿，得到物体表面位置点的多个补偿位置坐标；Step 802: Compensate the position coordinates of the object surface position points in the camera coordinate system according to the compensation matrix of each crossing layer and/or partition, and obtain multiple compensated position coordinates of the object surface position points;

步骤803：根据各所跨分层和/或分区的权重对物体表面位置点的多个补偿位置 坐标进行加权求和，获得加权求和后的补偿位置坐标。Step 803: Perform a weighted summation of multiple compensated position coordinates of the object surface position points according to the weights of each crossed layer and/or partition, and obtain the weighted summed compensated position coordinates.

具体的，在位置点属于同分层的不同分区的情况下，根据位置点与不同分区之间的距离，按照位置点与不同分区之间的距离对应的权重，对位置点的补偿结果进行加权求和，得到位置点对应的加权求和后的补偿位置坐标。在位置点属于不同分层的情况下，根据位置点与不同分层之间的距离，按照位置点与不同分层之间的距离对应的权重，对位置点的补偿结果进行加权求和，得到位置点对应的加权求和后的补偿位置坐标。在位置点既属于不同分层，且属于不同分区的情况下，根据位置点与不同分层之间的距离，以及该位置点与不同分区之间的距离，对位置点的补偿结果进行加权求和，得到位置点对应的加权求和后的补偿位置坐标。因此，通过对位于边界区域的位置点的补偿结果进行加权求和，能够提高得到的边界位置点的坐标位置的准确度。Specifically, when the location point belongs to different partitions of the same layer, the compensation results of the location point are weighted according to the distance between the location point and different partitions, and according to the weight corresponding to the distance between the location point and different partitions The sum is obtained to obtain the weighted and summed compensated position coordinates corresponding to the position points. In the case that the location points belong to different layers, according to the distance between the location point and different layers, and according to the weight corresponding to the distance between the location point and different layers, the compensation results of the location points are weighted and summed to obtain The weighted and summed compensated position coordinates corresponding to the position point. In the case that the location point belongs to different layers and different partitions, according to the distance between the location point and different layers, and the distance between the location point and different partitions, the compensation results of the location point are weighted. and to obtain the weighted and summed compensated position coordinates corresponding to the position points. Therefore, by weighting and summing the compensation results of the position points located in the boundary area, the accuracy of the obtained coordinate positions of the boundary position points can be improved.

具体实施中，所述步骤402：根据各组位置点在相机坐标系下的测量位置坐标及相机在机器人坐标系下的初始位姿，确定针对相机的补偿矩阵，如图9所示，可以进一步包括：In the specific implementation, the step 402: according to the measurement position coordinates of each group of position points in the camera coordinate system and the initial pose of the camera in the robot coordinate system, determine the compensation matrix for the camera, as shown in Figure 9, which can be further include:

901：根据相机在机器人坐标系下的初始位姿确定各组位置点在相机坐标系下的初始理论位置坐标；901: Determine the initial theoretical position coordinates of each group of position points in the camera coordinate system according to the initial pose of the camera in the robot coordinate system;

902：拟合各组位置点在相机坐标系下的测量位置坐标和初始理论位置坐标，确定针对相机的初始补偿矩阵；902: Fit the measured position coordinates and initial theoretical position coordinates of each group of position points in the camera coordinate system, and determine the initial compensation matrix for the camera;

903：获取多个相机在机器人坐标系下调整后的位姿并确定多个各组位置点在相机调整后的理论位置坐标；903: Acquiring the adjusted poses of multiple cameras in the robot coordinate system and determining the theoretical position coordinates of multiple groups of position points after camera adjustment;

904：根据多个相机在机器人坐标系下调整后返回的当前位姿、多个各组位置点在相机坐标系下的测量位置坐及在相机调整后的当前位置坐标调整初始补偿矩阵，直至各组位置点在相机坐标系下的测量位置坐标和当前理论位置坐标之间误差的欧式距离小于预设阈值和/或调整达到预设次数，将当前补偿矩阵作为针对相机的补偿矩阵。904: Adjust the initial compensation matrix according to the current pose returned by multiple cameras adjusted in the robot coordinate system, the measurement positions of multiple groups of position points in the camera coordinate system, and the current position coordinates after camera adjustment, until each The Euclidean distance of the error between the measured position coordinates of the group position points in the camera coordinate system and the current theoretical position coordinates is less than the preset threshold and/or the adjustment reaches the preset number of times, and the current compensation matrix is used as the compensation matrix for the camera.

进一步的，步骤902：拟合各组位置点在相机坐标系下的测量位置坐标和初始理论位置坐标，确定针对相机的初始补偿矩阵时，可以有多种实施方案，例如，可以通过最小二乘法进行拟合。Further, step 902: fitting the measured position coordinates and the initial theoretical position coordinates of each group of position points in the camera coordinate system, when determining the initial compensation matrix for the camera, there can be various implementations, for example, the least square method can be used to fit.

具体实施中，所述步骤402：根据各组位置点在相机坐标系下的测量位置坐标及相机在机器人坐标系下的初始位姿，确定针对相机的补偿矩阵，可以根据如下公式进 行计算：In the specific implementation, the step 402: according to the measurement position coordinates of each group of position points in the camera coordinate system and the initial pose of the camera in the robot coordinate system, determine the compensation matrix for the camera, which can be calculated according to the following formula:

其中，

表示相机在机器人坐标系下的位置坐标的误差；

表示理论位置坐标；

表示测量位置坐标；M表示补偿矩阵。in,

Indicates the error of the position coordinates of the camera in the robot coordinate system;

Indicates the theoretical position coordinates;

Indicates the measurement position coordinates; M indicates the compensation matrix.

具体实施中，根据多组位置点分别对应的测量位置坐标和理论位置坐标，以及误差公式

多次调整相机在机器人坐标系下的位姿，即

使得

(欧式距离)最小，即可以得到针对相机的补偿矩阵M，其中补偿矩阵包含针对每层中的每个位置区域所对应位置的补偿量；其中，

可以通过相机直接拍摄得到。In the specific implementation, according to the measured position coordinates and theoretical position coordinates corresponding to multiple sets of position points, and the error formula

Adjust the pose of the camera in the robot coordinate system multiple times, namely

make

(Euclidean distance) is the smallest, that is, the compensation matrix M for the camera can be obtained, wherein the compensation matrix contains the compensation amount for the corresponding position of each position area in each layer; where,

It can be directly captured by the camera.

具体实施中，所述步骤901：根据相机在机器人坐标系下的初始位姿确定各组位置点在相机坐标系下的初始理论位置坐标，可以有多种实施方案。例如，可以通过寻求各组位置点与相机坐标系之间的中间媒介，从而将标定板中的位置点导入相机坐标系之中，进而确定各组位置点在相机坐标系下的当前理论位置坐标，即如图10所示，所述步骤901：根据相机在机器人坐标系下的初始位姿确定各组位置点在相机坐标系下的初始理论位置坐标，可以进一步包括：In specific implementation, the step 901: determining the initial theoretical position coordinates of each group of position points in the camera coordinate system according to the initial pose of the camera in the robot coordinate system, there may be various implementations. For example, by seeking the intermediary between each group of position points and the camera coordinate system, the position points in the calibration plate can be imported into the camera coordinate system, and then the current theoretical position coordinates of each group of position points in the camera coordinate system can be determined , as shown in Figure 10, the step 901: determine the initial theoretical position coordinates of each group of position points in the camera coordinate system according to the initial pose of the camera in the robot coordinate system, which may further include:

1001：获取与各组位置点相对应的法兰在机器人坐标系下的位姿及标定板在相机坐标系下的位姿；1001: Obtain the pose of the flange corresponding to each group of position points in the robot coordinate system and the pose of the calibration plate in the camera coordinate system;

1002：根据相机在机器人坐标系下的初始位姿、法兰在机器人坐标系下的位姿以及标定板在相机坐标系下的位姿，确定标定板相对于法兰的位姿；1002: Determine the pose of the calibration plate relative to the flange according to the initial pose of the camera in the robot coordinate system, the pose of the flange in the robot coordinate system, and the pose of the calibration plate in the camera coordinate system;

1003：根据各组位置点在标定板坐标系下的位置坐标、标定板相对于法兰的位姿及法兰在机器人坐标系下的位姿，确定各组位置点在机器人坐标系下的的位置坐标；1003: According to the position coordinates of each group of position points in the calibration plate coordinate system, the pose of the calibration plate relative to the flange, and the pose of the flange in the robot coordinate system, determine the position of each group of position points in the robot coordinate system Position coordinates;

1004：根据各组位置点在机器人坐标系下的的位置坐标及相机在机器人坐标系下的初始位姿，确定各组位置点在相机坐标系下的初始理论位置坐标。1004: According to the position coordinates of each group of position points in the robot coordinate system and the initial pose of the camera in the robot coordinate system, determine the initial theoretical position coordinates of each group of position points in the camera coordinate system.

具体实施中，所述步骤105：根据补偿矩阵对物体表面位置点在相机坐标系下的位置坐标进行校准，得到补偿后的准确位置坐标，可以有多种实施方案，例如，可以根据如下公式进行计算：In the specific implementation, the step 105: according to the compensation matrix, the position coordinates of the object surface position points in the camera coordinate system are calibrated to obtain the correct position coordinates after compensation. There can be various implementations, for example, it can be performed according to the following formula calculate:

M·A＝A’M·A=A'

其中，M表示补偿矩阵；A表示物体表面位置点在相机坐标系下的位置坐标；A’表示补偿后的物体表面位置点在相机坐标系下的位置坐标；进一步的，将A’进行归一化处理后，便可以获得的准确位置点坐标I。Among them, M represents the compensation matrix; A represents the position coordinates of the object surface position points in the camera coordinate system; A' represents the position coordinates of the compensated object surface position points in the camera coordinate system; further, A' is normalized After processing, the exact location point coordinate I can be obtained.

如图11所示，本发明还提供了一种相机的标定方法装置，其特征在于，所述相机的标定装置包括：As shown in Fig. 11, the present invention also provides a camera calibration method device, characterized in that the camera calibration device includes:

坐标获取模块1101，获取多个位置点在相机坐标系下的测量位置坐标；Coordinate acquiringmodule 1101, acquires the measurement position coordinates of multiple position points in the camera coordinate system;

外参确定模块1102，用于确定相机在机器人坐标系下的初始位姿；The externalparameter determination module 1102 is used to determine the initial pose of the camera in the robot coordinate system;

补偿矩阵确定模块1103，用于根据多个位置点在相机坐标系下的测量位置坐标及相机在机器人坐标系下的初始位姿，确定针对相机的补偿矩阵。The compensationmatrix determination module 1103 is configured to determine a compensation matrix for the camera according to the measured position coordinates of multiple position points in the camera coordinate system and the initial pose of the camera in the robot coordinate system.

具体实施中，如图12所示，所述坐标获取模块1101进一步包括：In specific implementation, as shown in Figure 12, the coordinateacquisition module 1101 further includes:

图像采集子模块1201，用于获取标定板移动至多个空间位置时，相机采集的多个标定板图像；The image acquisition sub-module 1201 is used to acquire multiple calibration board images collected by the camera when the calibration board moves to multiple spatial positions;

坐标计算子模块1202，用于根据相机采集的多个标定板图像，获取多个位置点在相机坐标系下的测量位置坐标。The coordinatecalculation sub-module 1202 is configured to obtain the measurement position coordinates of multiple position points in the camera coordinate system according to the multiple calibration plate images collected by the camera.

具体实施中，如图13所示，所述外参确定模块1102进一步包括：In specific implementation, as shown in Figure 13, the externalparameter determination module 1102 further includes:

法兰坐标采集子模块1301，用于在机器人通过法兰带动标定板移动的过程中，获取多个法兰在机器人坐标系下的位姿以及多个标定板在相机坐标系下的位姿；The flange coordinate acquisition sub-module 1301 is used to obtain the poses of multiple flanges in the robot coordinate system and the poses of multiple calibration plates in the camera coordinate system when the robot drives the calibration plate to move through the flange;

外参计算子模块1301，用于根据多个法兰在机器人坐标系下的位姿及多个标定板在相机坐标系下的位姿，确定相机在机器人坐标系下的初始位姿。The externalparameter calculation sub-module 1301 is used to determine the initial pose of the camera in the robot coordinate system according to the poses of multiple flanges in the robot coordinate system and the poses of multiple calibration plates in the camera coordinate system.

具体实施中，如图14所示，所述补偿矩阵确定模块1103进一步包括：In specific implementation, as shown in Figure 14, the compensationmatrix determination module 1103 further includes:

坐标分组子模块1401，用于在相机坐标系下根据空间区域的不同，将多个位置点划分为多组位置点；The coordinategrouping sub-module 1401 is used to divide multiple location points into multiple groups of location points according to different spatial regions in the camera coordinate system;

补偿矩阵求取子模块1402，用于根据各组位置点在相机坐标系下的测量位置坐标及相机在机器人坐标系下的初始位姿，确定针对相机的补偿矩阵。The compensationmatrix calculation sub-module 1402 is used to determine the compensation matrix for the camera according to the measured position coordinates of each group of position points in the camera coordinate system and the initial pose of the camera in the robot coordinate system.

具体实施中，如图15所示，所述坐标分组子模块1401进一步包括：In specific implementation, as shown in Figure 15, the coordinategrouping submodule 1401 further includes:

空间分割子模块1501，用于在相机坐标系下将空间根据高度的不同划分为多个分层，各分层内划分有多个分区；Thespace segmentation sub-module 1501 is used to divide the space into multiple layers according to different heights in the camera coordinate system, and each layer is divided into multiple partitions;

分组子模块1502，用于根据多个位置点在相机坐标系下的位置坐标，将处于同一分层内同一分区的位置点划分为一组位置点。The grouping sub-module 1502 is configured to divide the position points in the same partition in the same layer into a group of position points according to the position coordinates of the plurality of position points in the camera coordinate system.

具体实施中，所述相机的标定装置还包括：In specific implementation, the calibration device of the camera also includes:

位置点坐标采集模块1104，用于在确定针对相机的补偿矩阵后，获取相机采集的物体表面位置点在相机坐标系下的位置坐标；The position point coordinateacquisition module 1104 is used to obtain the position coordinates of the object surface position points collected by the camera in the camera coordinate system after determining the compensation matrix for the camera;

准确位置坐标求取模块1105，用于根据补偿矩阵对物体表面位置点在相机坐标系下的位置坐标进行校准，得到补偿后的准确位置坐标。The exact position coordinatecalculation module 1105 is used to calibrate the position coordinates of the object surface position points in the camera coordinate system according to the compensation matrix, and obtain the correct position coordinates after compensation.

具体实施中，如图16所示，所述准确位置坐标求取模块1105进一步包括：In specific implementation, as shown in FIG. 16, the accurate position coordinate obtainingmodule 1105 further includes:

空间区域确定子模块1601，用于根据物体表面位置点在相机坐标系下的位置坐标和相机的内参，确定物体表面位置点所处的分层和分区；The spatialarea determination sub-module 1601 is used to determine the layer and partition of the object surface position point according to the position coordinates of the object surface position point in the camera coordinate system and the internal reference of the camera;

准确位置坐标获取子模块1602，用于根据与物体表面位置点所处的分层和分区对应的补偿矩阵，对物体表面位置点在相机坐标系下的位置坐标进行补偿，得到补偿后的准确位置坐标。The accurate position coordinate acquisition sub-module 1602 is used to compensate the position coordinates of the object surface position point in the camera coordinate system according to the compensation matrix corresponding to the layer and partition where the object surface position point is located, and obtain the correct position after compensation coordinate.

具体实施中，如图17所示，所述空间区域确定子模块1601进一步包括：In specific implementation, as shown in Figure 17, the spatialarea determination submodule 1601 further includes:

分层确定子模块1701，用于根据物体表面位置点在相机坐标系下的z轴坐标，确定物体表面位置点所处的分层；Thelayer determination sub-module 1701 is used to determine the layer of the object surface position point according to the z-axis coordinate of the object surface position point in the camera coordinate system;

像素坐标确定子模块1702，用于根据物体表面位置点在相机坐标系下的x轴坐标及y轴坐标及相机的内参，确定物体表面位置点对应的像素坐标；The pixel coordinate determination sub-module 1702 is used to determine the pixel coordinates corresponding to the object surface position point according to the x-axis coordinate and y-axis coordinate of the object surface position point in the camera coordinate system and the internal reference of the camera;

分区确定子模块1703，用于根据物体表面位置点对应的像素坐标确定物体表面位置点所处的分区。The partition determination sub-module 1703 is configured to determine the partition where the object surface location point is located according to the pixel coordinates corresponding to the object surface location point.

具体实施中，如图18所示，所述准确位置坐标获取子模块1602进一步包括：In a specific implementation, as shown in FIG. 18, thesubmodule 1602 of acquiring accurate position coordinates further includes:

权重确定子模块1801，用于在物体表面位置点处于跨分层和/或跨分区时，根据物体表面位置点与所跨分层和/或分区之间的距离确定各所跨分层和/或分区的权重；Theweight determination sub-module 1801 is used to determine the distance between the object surface position point and the spanned layer and/or partition when the object surface position point is in the cross-layer and/or cross-region. the weight of the partition;

补偿子模块1802，用于根据各所跨分层和/或分区的补偿矩阵分别对物体表面位置点在相机坐标系下的位置坐标进行补偿，得到物体表面位置点的多个补偿位置坐标；The compensation sub-module 1802 is used to compensate the position coordinates of the object surface position points in the camera coordinate system according to the compensation matrix of each crossing layer and/or partition, and obtain multiple compensated position coordinates of the object surface position points;

加权求和子模块1803，用于根据各所跨分层和/或分区的权重对物体表面位置点的多个补偿位置坐标进行加权求和，获得加权求和后的补偿位置坐标。The weighted summation sub-module 1803 is configured to perform weighted summation of multiple compensated position coordinates of object surface position points according to the weights of each crossed layer and/or partition, to obtain weighted summed compensated position coordinates.

具体实施中，如图19所示，所述补偿矩阵求取子模块1402进一步包括：In specific implementation, as shown in Figure 19, the compensationmatrix calculation sub-module 1402 further includes:

初始理论位置坐标获取子模块1901，用于根据相机在机器人坐标系下的初始位姿确定各组位置点在相机坐标系下的初始理论位置坐标；The initial theoretical position coordinate acquisition sub-module 1901 is used to determine the initial theoretical position coordinates of each group of position points in the camera coordinate system according to the initial pose of the camera in the robot coordinate system;

初始补偿矩阵获取子模块1902，用于拟合各组位置点在相机坐标系下的测量位置坐标和初始理论位置坐标，确定针对相机的初始补偿矩阵；The initial compensationmatrix acquisition sub-module 1902 is used to fit the measured position coordinates and initial theoretical position coordinates of each group of position points in the camera coordinate system, and determine the initial compensation matrix for the camera;

调整数据获取子模块1903，用于获取多个相机在机器人坐标系下调整后的位姿并确定多个各组位置点在相机调整后的理论位置坐标；The adjustment data acquisition sub-module 1903 is used to acquire the adjusted poses of multiple cameras in the robot coordinate system and determine the theoretical position coordinates of multiple groups of position points after camera adjustment;

矩阵确定子模块1904，用于根据多个相机在机器人坐标系下调整后返回的当前位姿、多个各组位置点在相机坐标系下的测量位置坐及在相机调整后的当前位置坐标调整初始补偿矩阵，直至各组位置点在相机坐标系下的测量位置坐标和当前理论位置坐标之间误差的欧式距离小于预设阈值和/或调整达到预设次数，将当前补偿矩阵作为针对相机的补偿矩阵。Thematrix determination sub-module 1904 is used to adjust the current position and orientation of multiple cameras in the robot coordinate system after adjustment, the measurement position of each group of position points in the camera coordinate system, and the current position coordinates after camera adjustment The initial compensation matrix, until the Euclidean distance of the error between the measured position coordinates of each group of position points in the camera coordinate system and the current theoretical position coordinates is less than the preset threshold and/or the adjustment reaches the preset number of times, the current compensation matrix is used as the camera. compensation matrix.

具体实施中，如图20所示，所述初始理论位置坐标获取子模块1901进一步包括：In specific implementation, as shown in Figure 20, the initial theoretical position coordinate acquisition sub-module 1901 further includes:

法兰及标定板位姿获取子模块2001，用于获取与各组位置点相对应的法兰在机器人坐标系下的位姿及标定板在相机坐标系下的位姿；The flange and calibration board pose acquisition sub-module 2001 is used to obtain the pose of the flange corresponding to each group of position points in the robot coordinate system and the pose of the calibration board in the camera coordinate system;

标定板相对法兰位姿获取子模块2002，用于根据相机在机器人坐标系下的初始位姿、法兰在机器人坐标系下的位姿以及标定板在相机坐标系下的位姿，确定标定板相对于法兰的位姿；The pose acquisition sub-module 2002 of the calibration board relative to the flange is used to determine the calibration according to the initial pose of the camera in the robot coordinate system, the pose of the flange in the robot coordinate system, and the pose of the calibration board in the camera coordinate system. The pose of the plate relative to the flange;

位置点相对机器人坐标获取子模块2003，用于根据各组位置点在标定板坐标系下的位置坐标、标定板相对于法兰的位姿及法兰在机器人坐标系下的位姿，确定各组位置点在机器人坐标系下的的位置坐标；Position point relative robot coordinate acquisition sub-module 2003 is used to determine each The position coordinates of the group position point in the robot coordinate system;

初始理论位置坐标确定子模块2004，用于根据各组位置点在机器人坐标系下的的位置坐标及相机在机器人坐标系下的初始位姿，确定各组位置点在相机坐标系下的初始理论位置坐标。The initial theoretical position coordinate determination sub-module 2004 is used to determine the initial theory of each group of position points in the camera coordinate system according to the position coordinates of each group of position points in the robot coordinate system and the initial pose of the camera in the robot coordinate system. Position coordinates.

本发明还提供了一种电子设备，包括存储器、处理器及存储在存储器上并可在处理器上运行的计算机程序，所述处理器执行所述计算机程序时实现所述相机的标定方法。The present invention also provides an electronic device, which includes a memory, a processor, and a computer program stored on the memory and operable on the processor. When the processor executes the computer program, the calibration method of the camera is implemented.

本发明还提供了一种计算机可读存储介质，所述计算机可读存储介质存储有执行所述相机的标定方法的计算机程序。The present invention also provides a computer-readable storage medium, the computer-readable storage medium storing a computer program for executing the camera calibration method.

综上所述，本发明提供的相机的标定方法、装置、电子设备及可读存储介质，其中，该方法包括：获取多个位置点在相机坐标系下的测量位置坐标；确定相机在机器人坐标系下的初始位姿；进而根据多个位置点在相机坐标系下的测量位置坐标及相机 在机器人坐标系下的初始位姿，确定针对相机的补偿矩阵。该相机的标定方法可以同时确定相机的外参以及针对相机内参的补偿矩阵，从而可以根据补偿矩阵对相机坐标系下的位置点的位置坐标进行补偿，得到更为准确的位置坐标，进而有效提升3D相机拍摄的3D点云图像的精度。In summary, the camera calibration method, device, electronic equipment and readable storage medium provided by the present invention, wherein the method includes: obtaining the measurement position coordinates of multiple position points in the camera coordinate system; Then, according to the measured position coordinates of multiple position points in the camera coordinate system and the initial pose of the camera in the robot coordinate system, the compensation matrix for the camera is determined. The camera calibration method can simultaneously determine the external parameters of the camera and the compensation matrix for the internal parameters of the camera, so that the position coordinates of the position points in the camera coordinate system can be compensated according to the compensation matrix, and more accurate position coordinates can be obtained, thereby effectively improving The accuracy of the 3D point cloud image captured by the 3D camera.

本领域内的技术人员应明白，本发明的实施例可提供为方法、系统、或计算机程序产品。因此，本发明可采用完全硬件实施例、完全软件实施例、或结合软件和硬件方面的实施例的形式。而且，本发明可采用在一个或多个其中包含有计算机可用程序代码的计算机可用存储介质(包括但不限于磁盘存储器、CD-ROM、光学存储器等)上实施的计算机程序产品的形式。Those skilled in the art should understand that the embodiments of the present invention may be provided as methods, systems, or computer program products. Accordingly, the present invention can take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention may take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) having computer-usable program code embodied therein.

本发明是参照根据本发明实施例的方法、设备(系统)、和计算机程序产品的流程图和/或方框图来描述的。应理解可由计算机程序指令实现流程图和/或方框图中的每一流程和/或方框、以及流程图和/或方框图中的流程和/或方框的结合。可提供这些计算机程序指令到通用计算机、专用计算机、嵌入式处理机或其他可编程数据处理设备的处理器以产生一个机器，使得通过计算机或其他可编程数据处理设备的处理器执行的指令产生用于实现在流程图一个流程或多个流程和/或方框图一个方框或多个方框中指定的功能的装置。The present invention is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It should be understood that each procedure and/or block in the flowchart and/or block diagram, and a combination of procedures and/or blocks in the flowchart and/or block diagram can be realized by computer program instructions. These computer program instructions may be provided to a general purpose computer, special purpose computer, embedded processor, or processor of other programmable data processing equipment to produce a machine such that the instructions executed by the processor of the computer or other programmable data processing equipment produce a An apparatus for realizing the functions specified in one or more procedures of the flowchart and/or one or more blocks of the block diagram.

这些计算机程序指令也可存储在能引导计算机或其他可编程数据处理设备以特定方式工作的计算机可读存储器中，使得存储在该计算机可读存储器中的指令产生包括指令装置的制造品，该指令装置实现在流程图一个流程或多个流程和/或方框图一个方框或多个方框中指定的功能。These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing apparatus to operate in a specific manner, such that the instructions stored in the computer-readable memory produce an article of manufacture comprising instruction means, the instructions The device realizes the function specified in one or more procedures of the flowchart and/or one or more blocks of the block diagram.

这些计算机程序指令也可装载到计算机或其他可编程数据处理设备上，使得在计算机或其他可编程设备上执行一系列操作步骤以产生计算机实现的处理，从而在计算机或其他可编程设备上执行的指令提供用于实现在流程图一个流程或多个流程和/或方框图一个方框或多个方框中指定的功能的步骤。These computer program instructions can also be loaded onto a computer or other programmable data processing device, causing a series of operational steps to be performed on the computer or other programmable device to produce a computer-implemented process, thereby The instructions provide steps for implementing the functions specified in the flow chart or blocks of the flowchart and/or the block or blocks of the block diagrams.

以上所述的具体实施例，对本发明的目的、技术方案和有益效果进行了进一步详细说明，所应理解的是，以上所述仅为本发明的具体实施例而已，并不用于限定本发明的保护范围，凡在本发明的精神和原则之内，所做的任何修改、等同替换、改进等，均应包含在本发明的保护范围之内。The specific embodiments described above have further described the purpose, technical solutions and beneficial effects of the present invention in detail. It should be understood that the above descriptions are only specific embodiments of the present invention and are not intended to limit the scope of the present invention. Protection scope, within the spirit and principles of the present invention, any modification, equivalent replacement, improvement, etc., shall be included in the protection scope of the present invention.

Claims (14)

Translated fromChinese

一种相机的标定方法，其特征在于，所述标定方法包括：A calibration method for a camera, characterized in that the calibration method comprises:获取多个位置点在相机坐标系下的测量位置坐标；Obtain the measured position coordinates of multiple position points in the camera coordinate system;确定相机在机器人坐标系下的初始位姿；Determine the initial pose of the camera in the robot coordinate system;根据多个位置点在相机坐标系下的测量位置坐标及相机在机器人坐标系下的初始位姿，确定针对相机的补偿矩阵。According to the measured position coordinates of multiple position points in the camera coordinate system and the initial pose of the camera in the robot coordinate system, the compensation matrix for the camera is determined.如权利要求1所述的标定方法，其特征在于，所述获取多个位置点在相机坐标系下的测量位置坐标，进一步包括：The calibration method according to claim 1, wherein said obtaining the measured position coordinates of a plurality of position points in the camera coordinate system further comprises:获取标定板移动至多个空间位置时，相机采集的多个标定板图像；Obtain multiple calibration plate images collected by the camera when the calibration plate moves to multiple spatial positions;根据相机采集的多个标定板图像，获取多个位置点在相机坐标系下的测量位置坐标。According to the multiple calibration plate images collected by the camera, the measurement position coordinates of the multiple position points in the camera coordinate system are obtained.如权利要求1所述的标定方法，其特征在于，所述确定相机在机器人坐标系下的初始位姿，进一步包括：The calibration method according to claim 1, wherein said determining the initial pose of the camera in the robot coordinate system further comprises:在机器人通过法兰带动标定板移动的过程中，获取多个法兰在机器人坐标系下的位姿以及多个标定板在相机坐标系下的位姿；When the robot moves the calibration plate through the flange, the poses of multiple flanges in the robot coordinate system and the poses of multiple calibration plates in the camera coordinate system are obtained;根据多个法兰在机器人坐标系下的位姿及多个标定板在相机坐标系下的位姿，确定相机在机器人坐标系下的初始位姿。According to the poses of multiple flanges in the robot coordinate system and the poses of multiple calibration plates in the camera coordinate system, the initial pose of the camera in the robot coordinate system is determined.如权利要求1所述的标定方法，其特征在于，所述根据多个位置点在相机坐标系下的测量位置坐标及相机在机器人坐标系下的初始位姿，确定针对相机的补偿矩阵，进一步包括：The calibration method according to claim 1, wherein the compensation matrix for the camera is determined according to the measured position coordinates of the plurality of position points in the camera coordinate system and the initial pose of the camera in the robot coordinate system, and further include:在相机坐标系下根据空间区域的不同，将多个位置点划分为多组位置点；In the camera coordinate system, according to the different spatial regions, divide multiple location points into multiple groups of location points;根据各组位置点在相机坐标系下的测量位置坐标及相机在机器人坐标系下的初始位姿，确定针对相机的补偿矩阵。According to the measured position coordinates of each group of position points in the camera coordinate system and the initial pose of the camera in the robot coordinate system, the compensation matrix for the camera is determined.如权利要求4所述的标定方法，其特征在于，所述在相机坐标系下根据空间区域的不同，将多个位置点划分为多组位置点，进一步包括：The calibration method according to claim 4, characterized in that, dividing a plurality of location points into multiple groups of location points according to different spatial regions under the camera coordinate system, further comprising:在相机坐标系下将空间根据高度的不同划分为多个分层，各分层内划分有多个分区；In the camera coordinate system, the space is divided into multiple layers according to different heights, and each layer is divided into multiple partitions;根据多个位置点在相机坐标系下的位置坐标，将处于同一分层内同一分区的位置点划分为一组位置点。According to the position coordinates of multiple position points in the camera coordinate system, the position points in the same partition in the same layer are divided into a group of position points.如权利要求1所述的标定方法，其特征在于，在确定针对相机的补偿矩阵后，进一步包括：The calibration method according to claim 1, wherein after determining the compensation matrix for the camera, further comprising:获取相机采集的物体表面位置点在相机坐标系下的位置坐标；Obtain the position coordinates of the object surface position points collected by the camera in the camera coordinate system;根据补偿矩阵对物体表面位置点在相机坐标系下的位置坐标进行校准，得到补偿后的准确位置坐标。According to the compensation matrix, the position coordinates of the object surface position points in the camera coordinate system are calibrated to obtain the accurate position coordinates after compensation.如权利要求6所述的标定方法，其特征在于，所述根据补偿矩阵对物体表面位置点在相机坐标系下的位置坐标进行校准，得到补偿后的准确位置坐标，进一步包括：The calibration method according to claim 6, wherein said calibration of the position coordinates of the object surface position points in the camera coordinate system according to the compensation matrix to obtain the correct position coordinates after compensation further comprises:根据物体表面位置点在相机坐标系下的位置坐标和相机的内参，确定物体表面位置点所处的分层和分区；According to the position coordinates of the object surface position point in the camera coordinate system and the internal reference of the camera, determine the layer and partition of the object surface position point;根据与物体表面位置点所处的分层和分区对应的补偿矩阵，对物体表面位置点在相机坐标系下的位置坐标进行补偿，得到补偿后的准确位置坐标。According to the compensation matrix corresponding to the layer and partition of the object surface position point, the position coordinates of the object surface position point in the camera coordinate system are compensated, and the accurate position coordinates after compensation are obtained.如权利要求7所述的标定方法，其特征在于，所述根据物体表面位置点在相机坐标系下的位置坐标和相机的内参，确定物体表面位置点所处的分层和分区，进一步包括：The calibration method according to claim 7, wherein, according to the position coordinates of the object surface position point in the camera coordinate system and the internal reference of the camera, determining the layer and partition where the object surface position point is located further comprises:根据物体表面位置点在相机坐标系下的z轴坐标，确定物体表面位置点所处的分层；According to the z-axis coordinates of the object surface position point in the camera coordinate system, determine the layer where the object surface position point is located;根据物体表面位置点在相机坐标系下的x轴坐标及y轴坐标及相机的内参，确定物体表面位置点对应的像素坐标；According to the x-axis coordinates and y-axis coordinates of the object surface position points in the camera coordinate system and the internal parameters of the camera, determine the pixel coordinates corresponding to the object surface position points;根据物体表面位置点对应的像素坐标确定物体表面位置点所处的分区。The partition where the object surface location point is located is determined according to the pixel coordinates corresponding to the object surface location point.如权利要求7所述的标定方法，其特征在于，根据与物体表面位置点所处的分层和分区对应的补偿矩阵，对物体表面位置点在相机坐标系下的位置坐标进行补偿，得到补偿后的准确位置坐标，进一步包括：The calibration method according to claim 7, wherein, according to the compensation matrix corresponding to the layer and partition where the object surface position point is located, the position coordinates of the object surface position point in the camera coordinate system are compensated to obtain compensation After the exact location coordinates, further include:在物体表面位置点处于跨分层和/或跨分区时，根据物体表面位置点与所跨分层和/或分区之间的距离确定各所跨分层和/或分区的权重；When the object surface position point is in a cross-layer and/or cross-region, determine the weight of each cross-layer and/or partition according to the distance between the object surface position point and the cross-layer and/or partition;根据各所跨分层和/或分区的补偿矩阵分别对物体表面位置点在相机坐标系下的位置坐标进行补偿，得到物体表面位置点的多个补偿位置坐标；Compensating the position coordinates of the object surface position points in the camera coordinate system according to the compensation matrix of each crossing layer and/or partition, and obtaining multiple compensated position coordinates of the object surface position points;根据各所跨分层和/或分区的权重对物体表面位置点的多个补偿位置坐标进行加权求和，获得加权求和后的补偿位置坐标。A weighted summation is performed on the plurality of compensated position coordinates of the object surface position points according to the weights of each spanned layer and/or partition to obtain the weighted summed compensated position coordinates.如权利要求4所述的标定方法，其特征在于，所述根据各组位置点在相机坐标系下的测量位置坐标及相机在机器人坐标系下的初始位姿，确定针对相机的补偿矩阵，进一步包括：The calibration method according to claim 4, wherein the compensation matrix for the camera is determined according to the measured position coordinates of each group of position points in the camera coordinate system and the initial pose of the camera in the robot coordinate system, and further include:根据相机在机器人坐标系下的初始位姿确定各组位置点在相机坐标系下的初始理论位置坐标；Determine the initial theoretical position coordinates of each group of position points in the camera coordinate system according to the initial pose of the camera in the robot coordinate system;拟合各组位置点在相机坐标系下的测量位置坐标和初始理论位置坐标，确定针对相机的初始补偿矩阵；Fit the measured position coordinates and initial theoretical position coordinates of each group of position points in the camera coordinate system, and determine the initial compensation matrix for the camera;获取多个相机在机器人坐标系下调整后的位姿并确定多个各组位置点在相机调整后的理论位置坐标；Obtain the adjusted poses of multiple cameras in the robot coordinate system and determine the theoretical position coordinates of multiple groups of position points after camera adjustment;根据多个相机在机器人坐标系下调整后返回的当前位姿、多个各组位置点在相机坐标系下的测量位置坐及在相机调整后的当前位置坐标调整初始补偿矩阵，直至各组位置点在相机坐标系下的测量位置坐标和当前理论位置坐标之间误差的欧式距离小于预设阈值和/或调整达到预设次数，将当前补偿矩阵作为针对相机的补偿矩阵。Adjust the initial compensation matrix according to the current pose returned by multiple cameras adjusted in the robot coordinate system, the measurement positions of multiple groups of position points in the camera coordinate system, and the current position coordinates after camera adjustment until each group position The Euclidean distance of the error between the measured position coordinates of the point in the camera coordinate system and the current theoretical position coordinates is less than the preset threshold and/or the adjustment reaches the preset number of times, and the current compensation matrix is used as the compensation matrix for the camera.如权利要求10所述的标定方法，其特征在于，根据相机在机器人坐标系下的初始位姿确定各组位置点在相机坐标系下的初始理论位置坐标，进一步包括：The calibration method according to claim 10, wherein the initial theoretical position coordinates of each group of position points in the camera coordinate system are determined according to the initial pose of the camera in the robot coordinate system, further comprising:获取与各组位置点相对应的法兰在机器人坐标系下的位姿及标定板在相机坐标系下的位姿；Obtain the pose of the flange corresponding to each group of position points in the robot coordinate system and the pose of the calibration plate in the camera coordinate system;根据相机在机器人坐标系下的初始位姿、法兰在机器人坐标系下的位姿以及标定板在相机坐标系下的位姿，确定标定板相对于法兰的位姿；According to the initial pose of the camera in the robot coordinate system, the pose of the flange in the robot coordinate system, and the pose of the calibration plate in the camera coordinate system, determine the pose of the calibration plate relative to the flange;根据各组位置点在标定板坐标系下的位置坐标、标定板相对于法兰的位姿及法兰在机器人坐标系下的位姿，确定各组位置点在机器人坐标系下的的位置坐标；Determine the position coordinates of each group of position points in the robot coordinate system according to the position coordinates of each group of position points in the calibration plate coordinate system, the pose of the calibration plate relative to the flange, and the pose of the flange in the robot coordinate system ;根据各组位置点在机器人坐标系下的的位置坐标及相机在机器人坐标系下的初始位姿，确定各组位置点在相机坐标系下的初始理论位置坐标。According to the position coordinates of each group of position points in the robot coordinate system and the initial pose of the camera in the robot coordinate system, determine the initial theoretical position coordinates of each group of position points in the camera coordinate system.一种相机的标定方法装置，其特征在于，所述相机的标定装置包括：A camera calibration method device, characterized in that the camera calibration device includes:坐标获取模块，获取多个位置点在相机坐标系下的测量位置坐标；The coordinate acquisition module acquires the measurement position coordinates of multiple position points in the camera coordinate system;外参确定模块，用于确定相机在机器人坐标系下的初始位姿；The external parameter determination module is used to determine the initial pose of the camera in the robot coordinate system;补偿矩阵确定模块，用于根据多个位置点在相机坐标系下的测量位置坐标及相机 在机器人坐标系下的初始位姿，确定针对相机的补偿矩阵。The compensation matrix determination module is used to determine the compensation matrix for the camera according to the measured position coordinates of multiple position points in the camera coordinate system and the initial pose of the camera in the robot coordinate system.一种电子设备，包括存储器、处理器及存储在存储器上并可在处理器上运行的计算机程序，其特征在于，所述处理器执行所述计算机程序时实现权利要求1至11任一所述相机的标定方法。An electronic device, comprising a memory, a processor, and a computer program stored on the memory and operable on the processor, characterized in that, when the processor executes the computer program, it implements any one of claims 1 to 11 Camera calibration method.一种计算机可读存储介质，其特征在于，所述计算机可读存储介质存储有执行权利要求1至11任一所述相机的标定方法的计算机程序。A computer-readable storage medium, characterized in that the computer-readable storage medium stores a computer program for executing the camera calibration method according to any one of claims 1 to 11.

Images