WO2019232793A1 - Two-camera calibration method, electronic device and computer-readable storage medium - Google Patents

Abstract

A two-camera calibration method, comprising: capturing images of a calibration board at different angles to obtain calibration images, performing segmentation and surface function fitting according to intrinsic parameters and extrinsic parameters of a single camera to obtain a distortion coefficient of the single camera, and determining initial extrinsic parameters of a two-camera module according to the extrinsic parameters of the single camera of the two-camera module; and performing processing according to the initial extrinsic parameters of the two-camera module, and the distortion coefficient and the intrinsic parameters of the single camera to obtain target extrinsic parameters of the two-camera module.

Description

Translated fromChinese

双摄像头标定方法、电子设备、计算机可读存储介质Calibration method for dual cameras, electronic equipment, and computer-readable storage medium技术领域Technical field

本申请涉及影像技术领域，特别是涉及一种双摄像头标定方法、电子设备、计算机可读存储介质。The present application relates to the field of imaging technology, and in particular, to a method for calibrating a dual camera, an electronic device, and a computer-readable storage medium.

背景技术Background technique

随着电子设备和影像技术的发展，越来越多的用户使用电子设备的摄像头拍摄图像。为了拍摄图像，摄像头出厂前需要进行标定。传统双摄标定方法的精度低。With the development of electronic devices and imaging technology, more and more users use the cameras of electronic devices to capture images. In order to take an image, the camera needs to be calibrated before leaving the factory. The accuracy of the traditional dual camera calibration method is low.

发明内容Summary of the Invention

本申请实施例提供一种双摄像头标定方法、电子设备、计算机可读存储介质，可以提高双摄像头标定的精度。The embodiments of the present application provide a method for calibrating a dual camera, an electronic device, and a computer-readable storage medium, which can improve the accuracy of calibrating the dual camera.

一种双摄像头标定方法，包括：A dual camera calibration method includes:

通过双摄像头模组拍摄不同角度的标定板，得到不同角度的标定图像；The calibration plates of different angles are taken by the dual camera module to obtain the calibration images of different angles;

检测所述标定图像中的特征点；Detecting feature points in the calibration image;

根据所述双摄像头模组中单摄像头的内参和外参及对应的标定图像中的特征点采用分块曲面函数拟合得到所述单摄像头的畸变系数，所述分块曲面函数的拟合结果连续；According to the internal and external parameters of a single camera in the dual camera module and the feature points in the corresponding calibration image, a block surface function is used to obtain the distortion coefficient of the single camera, and the fitting result of the block surface function is obtained. continuous;

根据所述双摄像头模组中单摄像头的外参确定所述双摄像头模组的初始外参；及Determining the initial external parameters of the dual camera module according to the external parameters of the single camera in the dual camera module; and

根据所述双摄像头模组的初始外参、单摄像头的畸变系数和单摄像头的内参进行处理得到所述双摄像头模组的目标外参。The target external parameters of the dual camera module are obtained by processing according to the initial external parameters of the dual camera module, the distortion coefficient of the single camera, and the internal parameters of the single camera.

一种电子设备，包括存储器及处理器，所述存储器中储存有计算机程序，所述计算机程序被所述处理器执行时，使得所述处理器执行以下步骤：An electronic device includes a memory and a processor. The memory stores a computer program. When the computer program is executed by the processor, the processor causes the processor to perform the following steps:

通过双摄像头模组拍摄不同角度的标定板，得到不同角度的标定图像；The calibration plates of different angles are taken by the dual camera module to obtain the calibration images of different angles;

检测所述标定图像中的同名特征点；Detecting feature points of the same name in the calibration image;

根据所述双摄像头模组中单摄像头的内参和外参及对应的标定图像中的特征点采用分块曲面函数拟合得到所述单摄像头的畸变系数，所述分块曲面函数的拟合结果连续；According to the internal and external parameters of a single camera in the dual camera module and the feature points in the corresponding calibration image, a block surface function is used to obtain the distortion coefficient of the single camera, and the fitting result of the block surface function is obtained. continuous;

根据所述双摄像头模组中单摄像头的外参确定所述双摄像头模组的初始外参；及Determining the initial external parameters of the dual camera module according to the external parameters of the single camera in the dual camera module; and

根据所述双摄像头模组的初始外参、单摄像头的畸变系数和单摄像头的内参进行处理得到所述双摄像头模组的目标外参。The target external parameters of the dual camera module are obtained by processing according to the initial external parameters of the dual camera module, the distortion coefficient of the single camera, and the internal parameters of the single camera.

一种非易失性计算机可读存储介质，其上存储有计算机程序，所述计算机程序被处理器执行时实现以下操作：A non-transitory computer-readable storage medium having stored thereon a computer program that, when executed by a processor, implements the following operations:

通过双摄像头模组拍摄不同角度的标定板，得到不同角度的标定图像；The calibration plates of different angles are taken by the dual camera module to obtain the calibration images of different angles;

检测所述标定图像中的特征点；Detecting feature points in the calibration image;

根据所述双摄像头模组中单摄像头的内参和外参及对应的标定图像中的特征点采用分块曲面函数拟合得到所述单摄像头的畸变系数，所述分块曲面函数的拟合结果连续；According to the internal and external parameters of a single camera in the dual camera module and the feature points in the corresponding calibration image, a block surface function is used to obtain the distortion coefficient of the single camera, and the fitting result of the block surface function is obtained. continuous;

根据所述双摄像头模组中单摄像头的外参确定所述双摄像头模组的初始外参；及Determining the initial external parameters of the dual camera module according to the external parameters of the single camera in the dual camera module; and

根据所述双摄像头模组的初始外参、单摄像头的畸变系数和单摄像头的内参进行处理得到所述双摄像头模组的目标外参。The target external parameters of the dual camera module are obtained by processing according to the initial external parameters of the dual camera module, the distortion coefficient of the single camera, and the internal parameters of the single camera.

本申请实施例中的双摄像头标定方法、电子设备、计算机可读存储介质，根据第一摄像头和第二摄像头的内参、外参和标定图像中的特征点进行分块曲面函数拟合得到畸变系数，根据单摄像头内参、畸变系数及双摄像头模组的初始外参进行优化，提高了标定精度。The dual camera calibration method, the electronic device, and the computer-readable storage medium in the embodiments of the present application perform block surface function fitting to obtain distortion coefficients according to feature points in the internal parameters, external parameters, and calibration images of the first and second cameras. According to the optimization of the internal parameters of the single camera, the distortion coefficient and the initial external parameters of the dual camera module, the calibration accuracy is improved.

附图说明BRIEF DESCRIPTION OF THE DRAWINGS

为了更清楚地说明本申请实施例或现有技术中的技术方案，下面将对实施例或现有技术描述中所需要使用的附图作简单地介绍，显而易见地，下面描述中的附图仅仅是本申请的一些实施例，对于本领域普通技术人员来讲，在不付出创造性劳动的前提下，还可以根据这些附图获得其他的附图。In order to explain the technical solutions in the embodiments of the present application or the prior art more clearly, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings in the following description are merely These are some embodiments of the present application. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without paying creative work.

图1为一个实施例中双摄像头标定方法的应用环境示意图。FIG. 1 is a schematic diagram of an application environment of a dual camera calibration method according to an embodiment.

图2为一个实施例中双摄像头标定方法的流程图。FIG. 2 is a flowchart of a dual camera calibration method according to an embodiment.

图3为一个实施例中根据该双摄像头模组中单摄像头的内参和外参、标定图像中的特征点得到单摄像头不同角度的目标点点集和畸变点点集的流程图。3 is a flowchart of obtaining a target point set and a distortion point set of different angles of a single camera according to an internal parameter and an external parameter of a single camera in the dual camera module according to an embodiment, and characteristic points in a calibration image.

图4为一个实施例中目标点点集划分块示意图。FIG. 4 is a schematic diagram of dividing a target point set into blocks according to an embodiment.

图5为一个实施例中x方向畸变变形曲面拟合结果的示意图。FIG. 5 is a schematic diagram of fitting results of a distortion surface in the x direction in one embodiment.

图6为一个实施例中y方向畸变变形曲面拟合的示意图。FIG. 6 is a schematic diagram of y-direction distortion and deformation surface fitting in an embodiment.

图7为又一个实施例中双摄像头标定方法的流程图。FIG. 7 is a flowchart of a dual camera calibration method according to another embodiment.

图8为一个实施例中双摄像头标定装置的结构框图。FIG. 8 is a structural block diagram of a dual camera calibration device according to an embodiment.

图9为一个实施例中电子设备的内部结构示意图。FIG. 9 is a schematic diagram of an internal structure of an electronic device in an embodiment.

图10为一个实施例中图像处理电路的示意图。FIG. 10 is a schematic diagram of an image processing circuit in an embodiment.

具体实施方式Detailed ways

为了使本申请的目的、技术方案及优点更加清楚明白，以下结合附图及实施例，对本申请进行进一步详细说明。应当理解，此处所描述的具体实施例仅仅用以解释本申请，并不用于限定本申请。In order to make the purpose, technical solution, and advantages of the present application clearer, the present application is described in further detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the application, and are not used to limit the application.

可以理解，本申请所使用的术语“第一”、“第二”等可在本文中用于描述各种元件，但这些元件不受这些术语限制。这些术语仅用于将第一个元件与另一个元件区分。举例来说，在不脱离本申请的范围的情况下，可以将第一标定图像称为第二标定图像，且类似地，可将第二标定图像称为第一标定图像。第一标定图像和第二标定图像两者都是标定图像，但其不是同一标定图像。It can be understood that the terms “first”, “second”, and the like used in this application can be used herein to describe various elements, but these elements are not limited by these terms. These terms are only used to distinguish the first element from another element. For example, without departing from the scope of the present application, the first calibration image may be referred to as a second calibration image, and similarly, the second calibration image may be referred to as a first calibration image. Both the first calibration image and the second calibration image are calibration images, but they are not the same calibration image.

图1为一个实施例中双摄像头标定方法的应用环境示意图。如图1所示，该应用环境包括双摄治具110和标定板120。双摄治具110用于放置带有双摄像头模组或者带有双摄像头模组的电子设备。标定板120(chart)上带有chart图案。标定板120可进行旋转，保持不同角度的位姿。双摄治具110上的双摄像头模组或带有双摄像头模组的电子设备在不同距离、不同角度拍摄标定板120上chart图案，通常拍摄图像至少3个角度，如图1中双摄像头模组光轴垂直于标定板旋转轴，标定板120绕Y轴旋转三个角度，其中一个角度为0度，另外两个旋转角度为±θ度，θ大于15，以保证姿态间解耦。通过双摄像头模组拍摄不同角度的标定板得到不同角度的标定图像，根据双摄像头模组中单摄像头的内参和外参及标定图像中的特征点采用分块曲面函数拟合得到单摄像头的畸变系数，以该第一摄像头和第二摄像头所拍摄的标定图像中的同名特征点的偏差为目标函数，将该单摄像头的内参和畸变系数作为该目标函数的输入，将双摄像头模组的外参作为该目标函数的初始参数，进行处理得到该双摄像头模组的目标外参。可以提高双摄像头模组的标定精度。FIG. 1 is a schematic diagram of an application environment of a dual camera calibration method according to an embodiment. As shown in FIG. 1, the application environment includes adual camera 110 and acalibration plate 120. Thedual camera fixture 110 is used to place an electronic device with a dual camera module or a dual camera module. The calibration plate 120 (chart) has a chart pattern. Thecalibration plate 120 can be rotated to maintain different postures. The dual camera module on thedual camera 110 or the electronic device with the dual camera module takes the chart pattern on thecalibration board 120 at different distances and different angles, usually at least 3 angles, as shown in the dual camera module in Figure 1. The optical axis of the group is perpendicular to the rotation axis of the calibration plate. Thecalibration plate 120 rotates three angles around the Y axis, one of which is 0 degrees, and the other two rotation angles are ± θ degrees, θ is greater than 15 to ensure decoupling between attitudes. The calibration images of different angles are taken by the dual camera module to obtain calibration images of different angles. According to the internal and external parameters of the single camera in the dual camera module and the feature points in the calibration image, the block camera is used to fit the distortion of the single camera. Coefficient, taking the deviation of feature points of the same name in the calibration images captured by the first camera and the second camera as the objective function, using the internal parameters and distortion coefficients of the single camera as the input of the objective function, and taking the external of the dual camera module The parameter is used as an initial parameter of the objective function, and is processed to obtain a target external parameter of the dual camera module. Can improve the calibration accuracy of the dual camera module.

图2为一个实施例中双摄像头标定方法的流程图。如图2所示，一种双摄像头标定方法，从操作202开始。FIG. 2 is a flowchart of a dual camera calibration method according to an embodiment. As shown in FIG. 2, a dual camera calibration method starts atoperation 202.

操作202，通过双摄像头模组拍摄不同角度的标定板，得到不同角度的标定图像。Inoperation 202, the calibration plates of different angles are photographed through the dual camera module to obtain calibration images of different angles.

摄像头是用来采集图像的，摄像头出厂前需要进行标定。单摄像头标定是指确定单摄像头内参和外参的值。单摄像头的内参可包括f_x、f_y、c_x、c_y，其中，f_x表示焦距在图 像坐标系x轴方向上单位像元大小，f_y表示焦距在图像坐标系y轴方向上单位像元大小，c_x、c_y表示图像平面的主点坐标，主点是光轴与图像平面的交点。f_x＝f/d_x，f_y＝f/d_y，其中，f为单摄像头的焦距，d_x表示图像坐标系x轴方向上一个像素的宽度，d_y表示图像坐标系y轴方向上一个像素的宽度。图像坐标系是以摄像头拍摄的二维图像为基准建立的坐标系，用于指定物体在拍摄图像中的位置。图像坐标系中的(x,y)坐标系的原点位于摄像头光轴与成像平面的焦点(c_x，c_y)上，单位为长度单位，即米，像素坐标系中的(u，v)坐标系的原点在图像的左上角，单位为数量单位，即个。(x,y)用于表征物体从摄像头坐标系向图像坐标系的透视投影关系，(u，v)用于表征像素坐标。(x,y)与(u，v)之间的转换关系如公式(1)：The camera is used to collect images, and the camera needs to be calibrated before leaving the factory. Single camera calibration refers to determining the values of the internal and external parameters of a single camera. The internal parameters of a single camera can include f_x , f_y , c_x , and c_y , where f_x represents the unit size of the focal length in the image coordinate system x-axis direction, and f_y represents the unit of the focal length in the image coordinate system y-axis direction. Pixel size, c_x , c_y represent the coordinates of the principal point of the image plane, and the principal point is the intersection of the optical axis and the image plane. f_x = f / d_x , f_y = f / d_y , where f is the focal length of a single camera, d_x is the width of a pixel in the x-axis direction of the image coordinate system, and d_y is the y-axis direction in the image coordinate system The width of one pixel. The image coordinate system is a coordinate system established on the basis of a two-dimensional image captured by a camera, and is used to specify the position of an object in the captured image. The origin of the (x, y) coordinate system in the image coordinate system is located on the focal point (c_x , c_y ) of the optical axis of the camera and the imaging plane. The unit is the length unit, that is, meters, (u, v) in the pixel coordinate system. The origin of the coordinate system is in the upper left corner of the image, and the unit is a quantity unit, that is, a unit. (x, y) is used to characterize the perspective projection relationship of the object from the camera coordinate system to the image coordinate system, and (u, v) is used to characterize the pixel coordinates. The conversion relationship between (x, y) and (u, v) is as shown in formula (1):

透视投影是指用中心投影法将形体投射到投影面上，从而获得的一种较为接近视觉效果的单面投影图。Perspective projection refers to a single-sided projection image that is closer to visual effects by projecting a shape onto a projection surface using the center projection method.

单摄像头的外参包括世界坐标系下的坐标转换到摄像头坐标系下的坐标的旋转矩阵和平移矩阵。世界坐标系通过刚体变换到达摄像头坐标系，摄像头坐标系通过透视投影变换到达图像坐标系。刚体变换是指三维空间中，当物体不发生形变时，对一个几何物体做旋转、平移的运动，即为刚体变换。刚体变换如公式(2)The external parameters of a single camera include a rotation matrix and a translation matrix that transform the coordinates in the world coordinate system to the coordinates in the camera coordinate system. The world coordinate system reaches the camera coordinate system through rigid body transformation, and the camera coordinate system reaches the image coordinate system through perspective projection transformation. Rigid body transformation refers to the movement of rotation and translation of a geometric object when the object is not deformed in three-dimensional space, which is the rigid body transformation. The rigid body transformation is as shown in formula (2)

X_c＝RX+T，

X_c = RX + T,

其中，X_c代表摄像头坐标系，X代表世界坐标系，R代表世界坐标系到摄像头坐标系的旋转矩阵，T代表世界坐标系到摄像头坐标系的平移矩阵。世界坐标系原点和摄像头坐标系原点之间的距离受x、y、z三个轴方向上的分量共同控制，具有三个自由度，R为分别绕X、Y、Z轴旋转的效果之和。t_x表示x轴方向的平移量，t_y表示y轴方向的平移量，t_z表示z轴方向的平移量。Among them, X_c represents the camera coordinate system, X represents the world coordinate system, R represents the rotation matrix from the world coordinate system to the camera coordinate system, and T represents the translation matrix from the world coordinate system to the camera coordinate system. The distance between the origin of the world coordinate system and the origin of the camera coordinate system is controlled by the components in the three axis directions x, y, and z. It has three degrees of freedom, and R is the sum of the effects of rotation around the x, y, and z axes, respectively. . t_x represents the translation amount in the x-axis direction, t_y represents the translation amount in the y-axis direction, and t_z represents the translation amount in the z-axis direction.

世界坐标系是客观三维空间的绝对坐标系，可以建立在任意位置。例如对于每张标定图像，世界坐标系可以建立在以标定板的左上角角点为原点，以标定板平面为XY平面，Z轴垂直标定板平面向上。摄像头坐标系是以摄像头光心为坐标系的原点，以摄像头的光轴作为Z轴，X轴、Y轴分别平行于图像坐标系的X轴Y轴。图像坐标系的主点是光轴与图像平面的交点。图像坐标系以主点为原点。像素坐标系是指原点定义在图像平面的左上角 位置。The world coordinate system is an absolute coordinate system in objective three-dimensional space and can be established at any position. For example, for each calibration image, the world coordinate system can be established with the upper left corner point of the calibration plate as the origin, the calibration plate plane as the XY plane, and the Z axis perpendicular to the calibration plate plane upwards. The camera coordinate system is based on the optical center of the camera as the origin, the optical axis of the camera is used as the Z axis, and the X and Y axes are respectively parallel to the X and Y axes of the image coordinate system. The principal point of the image coordinate system is the intersection of the optical axis and the image plane. The image coordinate system uses the principal point as the origin. The pixel coordinate system means that the origin is defined in the upper left corner of the image plane.

双摄像头标定是指确定双摄像头模组的外参值。双摄像头模组的外参包括两个摄像头之间的旋转矩阵和平移矩阵。The dual camera calibration refers to determining the external parameter value of the dual camera module. The external parameters of the dual camera module include a rotation matrix and a translation matrix between the two cameras.

双摄像头模组包括第一摄像头和第二摄像头。第一摄像头和第二摄像头可均为彩色摄像头，或者一个为黑白摄像头，一个为彩色摄像头，或者两个黑白摄像头。The dual camera module includes a first camera and a second camera. The first camera and the second camera may both be color cameras, or one is a black and white camera, one is a color camera, or two black and white cameras.

标定板是用来进行摄像头标定的参照板。标定板上带有图案。标定板可为平面标定板或立体标定板。平面标定板上的图案可为棋盘格图案、圆阵列图案或圆环阵列图案或其他二维码图案等。The calibration board is a reference board used for camera calibration. Patterns on calibration plates. The calibration plate can be a flat calibration plate or a stereo calibration plate. The pattern on the flat calibration plate can be a checkerboard pattern, a circular array pattern, a circular array pattern, or other two-dimensional code patterns.

双摄像头模组的第一摄像头和第二摄像头拍摄不同角度的标定板，得到不同角度的标定图像。不同角度的标定板需要覆盖满整个摄像头的视场角(field of view，简称FOV)。若标定板为平面标定板，则旋转平面标定板在不同角度，如一个角度为0度，另外两个旋转角度为±θ度，θ大于15。若标定板为立体标定板，则可以直接拍摄立体标定板的三个垂直面得到不同角度的标定图像。The first camera and the second camera of the dual camera module shoot the calibration boards of different angles to obtain calibration images of different angles. The calibration plates of different angles need to cover the field of view (FOV) of the entire camera. If the calibration plate is a flat calibration plate, rotate the plane calibration plate at different angles, such as one angle is 0 degrees, the other two rotation angles are ± θ degrees, and θ is greater than 15. If the calibration plate is a stereo calibration plate, three vertical planes of the stereo calibration plate can be directly taken to obtain calibration images of different angles.

操作204，检测标定图像中的特征点。Operation 204: Detect feature points in the calibration image.

首先检测标定板中特征点在第一摄像头所拍摄的标定图像中对应的特征点，再第二摄像头所拍摄的标定图像中对应的特征点，再以第一摄像头所拍摄的标定图像中的特征点在第二摄像头所拍摄的标定图像中搜索对应的同名特征点。同名特征点是指对于标定板中的同一特征点，被第一摄像头拍摄所得到的标定图像中与标定板中该特征点对应的第一特征点，被第二摄像头拍摄的标定图像中与标定板中同一特征点对应的第二特征点，该标定板中的该特征点、第一特征点和第二特征点为同名特征点。检测第一摄像头和第二摄像头所拍摄的标定图像中的同名特征点。First, the feature points in the calibration board corresponding to the feature points in the calibration image captured by the first camera, the corresponding feature points in the calibration image captured by the second camera, and the features in the calibration image captured by the first camera Search for corresponding feature points with the same name in the calibration image captured by the second camera. Feature points with the same name refer to the first feature point in the calibration image obtained by the first camera corresponding to the feature point in the calibration plate for the same feature point in the calibration plate, and in the calibration image captured by the second camera. The second feature point corresponding to the same feature point in the board, the feature point, the first feature point, and the second feature point in the calibration board are feature points with the same name. Detecting feature points with the same name in the calibration images captured by the first camera and the second camera.

在一个实施例中，若标定板上的图案为棋盘格图案，则检测标定图像中的特征点可包括：采用Harris角点检测算子得到图像中角点的初始值；检测标定图像中的边缘信息，对得到的角点进行分组，得到边缘点集合；对选取的边缘点进行曲线拟合，包括全局和局部拟合曲线，求取全局曲线和局部曲线的交点即可得到所求的角点，得到的角点即为标定图像中的特征点。In one embodiment, if the pattern on the calibration plate is a checkerboard pattern, detecting feature points in the calibration image may include: using Harris corner detection operators to obtain initial values of corner points in the image; detecting edges in the calibration image Information, group the obtained corner points to obtain the edge point set; perform curve fitting on the selected edge points, including global and local fitting curves, and find the intersection points of the global and local curves to obtain the required corner points The obtained corner points are the characteristic points in the calibration image.

在一个实施例中，若标定板的图案为椭圆或圆，则检测该标定图像中的特征点包括：采用canny边缘提取椭圆边缘信息，通过椭圆的一般方程和最小二乘法进行拟合得到椭圆的中心点；通过椭圆的中心点坐标表示每个椭圆在图像中的位置，通过椭圆中心点坐标，可以对椭圆中心点进行排序。In one embodiment, if the pattern of the calibration plate is ellipse or circle, detecting feature points in the calibration image includes: using canny edges to extract the edge information of the ellipse, and fitting the ellipse by fitting the general equation of the ellipse and the least square method. Center point; the position of each ellipse in the image is represented by the coordinates of the center point of the ellipse, and the center points of the ellipse can be sorted by the coordinates of the center point of the ellipse.

分别对第一摄像头和第二摄像头拍摄的标定图像进行特征点提取得到各自标定图像中的特征点。对于第一摄像头的某一角度标定图像中的任一特征点，可在第二摄像头对应该某一角度的标定图像中利用极线约束进行搜索匹配。例如，对于三维空间中的一点p，投影到两个不同的平面L1和L2，投影点分别为p1和p2，p、p1和p2在三维空间内构成一个平面S。S与面L1的交线n1过p1点，称之为对应于p2的极线。极线约束是指同一个点在两幅图像上的映射，已知映射点p1，则映射点p2在相对于p1的极线上。Feature points are extracted from the calibration images captured by the first camera and the second camera to obtain feature points in the respective calibration images. For any feature point in the calibration image of a certain angle of the first camera, the second camera can be used for searching and matching by using epipolar constraints in the calibration image of a certain angle. For example, a point p in three-dimensional space is projected onto two different planes L1 and L2, and the projection points are p1 and p2, respectively. P, p1, and p2 form a plane S in the three-dimensional space. The line of intersection n1 of S and the surface L1 passes through the point p1, which is called the epipolar line corresponding to p2. The epipolar constraint refers to the mapping of the same point on two images. If the mapping point p1 is known, the mapping point p2 is on the epipolar line relative to p1.

操作206，根据该双摄像头模组中单摄像头的内参和外参及对应的标定图像中的特征点采用分块曲面函数拟合得到该单摄像头的畸变系数，该分块曲面函数的拟合结果连续。Inoperation 206, according to the internal and external parameters of the single camera in the dual camera module and the feature points in the corresponding calibration image, a block surface function is fitted to obtain the distortion coefficient of the single camera, and the fitting result of the block surface function is fitted. continuous.

其中，分块曲面函数可为B样条函数、自由曲面函数、zernike多项式函数中一种。分块曲面函数可以实现在块与块之间连续可导，拟合结果是连续的。Among them, the block surface function may be one of a B-spline function, a free-form surface function, and a zernike polynomial function. The block surface function can be continuously differentiable between blocks, and the fitting results are continuous.

操作208，根据该双摄像头模组中单摄像头的外参确定该双摄像头模组的初始外参。Inoperation 208, the initial external parameters of the dual camera module are determined according to the external parameters of the single camera in the dual camera module.

双摄像头模组的外参包括双摄像头间的旋转矩阵和双摄像头间的平移矩阵。双摄像头之间的旋转矩阵和平移矩阵可以有公式(3)求取。The external parameters of the dual camera module include a rotation matrix between the dual cameras and a translation matrix between the dual cameras. The rotation matrix and translation matrix between the two cameras can be obtained by formula (3).

其中，R'为双摄像头间的旋转矩阵，T'为双摄像头间的平移矩阵，R_r为第一摄像头经过标定得到的相对标定物的旋转矩阵(即标定物在世界坐标系的坐标转换到第一摄像头的摄像头坐标系的坐标的旋转矩阵)，T_r为第一摄像头经过标定得到的相对标定物的平移矩阵(即标定物在世界坐标系的坐标转换到第一摄像头的摄像头坐标系的坐标的平移矩阵)。R_l为第二摄像头经过标定得到的相对标定物的旋转矩阵(即标定物在世界坐标系的坐标转换到第二摄像头的摄像头坐标系的坐标的旋转矩阵)，T_l为第二摄像头经过标定得到的相对标定物的平移矩阵(即标定物在世界坐标系的坐标转换到第二摄像头的摄像头坐标系的坐标的平移矩阵)。Among them, R ′ is the rotation matrix between the two cameras, T ′ is the translation matrix between the two cameras, and R_r is the rotation matrix of the first calibration camera relative to the calibration object (that is, the coordinates of the calibration object in the world coordinate system are converted to The rotation matrix of the coordinates of the camera coordinate system of the first camera), T_r is the translation matrix of the relative calibration object obtained by the calibration of the first camera (that is, the coordinates of the calibration object in the world coordinate system are converted to the camera coordinate system of the first camera Coordinate translation matrix). R_l is the rotation matrix of the second camera relative to the calibration object (that is, the coordinates of the calibration object in the world coordinate system are converted to the coordinates of the camera camera coordinate system of the second camera), T_l is the second camera calibration The obtained translation matrix of the relative calibration object (that is, the translation matrix of the coordinates of the calibration object in the world coordinate system and the coordinates of the camera coordinate system of the second camera).

操作210，根据该双摄像头模组的初始外参、单摄像头的畸变系数和单摄像头的内参进行处理得到该双摄像头模组的目标外参。Inoperation 210, the target external parameters of the dual camera module are obtained by processing according to the initial external parameters of the dual camera module, the distortion coefficient of the single camera, and the internal parameters of the single camera.

本申请实施例中的双摄像头标定方法，根据单摄像头的内参、外参和标定图像中的特征点进行分块曲面函数拟合得到畸变系数，得到的畸变系数精度更高，再根据单摄像头的内参和畸变系数、双摄像头模组的初始外参进行处理可以得到双摄像头模组的目标外参，提高了标定精度。The dual-camera calibration method in the embodiment of the present application uses a single-camera internal parameter, external parameter, and feature points in the calibration image to perform a block surface function fitting to obtain a distortion coefficient, and the obtained distortion coefficient has higher accuracy. The internal parameters and distortion coefficients, and the initial external parameters of the dual camera module can be processed to obtain the target external parameters of the dual camera module, which improves the calibration accuracy.

在一个实施例中，操作210包括：以该第一摄像头和第二摄像头所拍摄的标定图像中的同名特征点的偏差为目标函数，将该单摄像头的内参和畸变系数作为该目标函数的输入，将该双摄像头模组的初始外参作为该目标函数的初始参数，进行处理得到该双摄像头模组的目标外参。In one embodiment,operation 210 includes: taking the deviation of feature points of the same name in the calibration images captured by the first camera and the second camera as an objective function, and using the internal parameters and distortion coefficients of the single camera as the input of the objective function Using the initial external parameters of the dual camera module as the initial parameters of the objective function, and processing to obtain the target external parameters of the dual camera module.

以第一摄像头拍摄的标定图像的特征点映射到第二摄像头拍摄的标定图像的同名特征点的偏差作为双摄像头的外参优化目标函数，将第一摄像头所拍摄的标定图像中的特征点和第二摄像头所拍摄图像的标定图像中的同名特征点、将第一摄像头的内参、外参和畸变系数、第二摄像头的参、外参和畸变系数作为双摄像头的外参优化目标函数的输入，将该双摄像头模组的初始外参作为该目标函数的初始参数，进行处理得到该双摄像头模组的目标外参。The deviation of the feature points of the calibration image captured by the first camera to the feature points of the same name of the calibration image captured by the second camera is used as the external parameter optimization objective function of the dual camera, and the feature points in the calibration image captured by the first camera and the Feature points with the same name in the calibration image of the image captured by the second camera, using the internal parameters, external parameters, and distortion coefficients of the first camera, the parameters of the second camera, external parameters, and distortion coefficients as the input of the external camera optimization objective function of the dual camera Using the initial external parameters of the dual camera module as the initial parameters of the objective function, and processing to obtain the target external parameters of the dual camera module.

本申请实施例中的双摄像头标定方法，根据第一摄像头和第二摄像头的内参、外参和标定图像中的特征点进行分块曲面函数拟合得到畸变系数，获取到第一摄像头和第二摄像头拍摄的标定图像中的同名特征点，将第一摄像头和第二摄像头所拍摄标定图像中的同名特征点的偏差作为目标函数，将标定图像的特征点、第一摄像头和第二摄像头的内参作为目标函数的输入，双摄像头模组的初始外参作为目标函数的初始参数进行优化，得到了双摄像头模组的目标外参，利用分块曲线函数拟合得到畸变系数，以标定图像中的同名特征点的偏差作为目标函数对双摄像头模组外参进行优化，提高了标定精度。In the dual camera calibration method in the embodiment of the present application, the distortion coefficient is obtained by fitting the block surface function fitting according to the feature points in the internal and external parameters of the first camera and the second camera and the calibration image to obtain the first camera and the second camera. Feature points with the same name in the calibration image captured by the camera, the deviation of the feature points with the same name in the calibration image captured by the first camera and the second camera is used as the objective function, and the feature points of the calibration image, the internal parameters of the first camera and the second camera As the input of the objective function, the initial external parameters of the dual camera module were optimized as the initial parameters of the objective function, and the target external parameters of the dual camera module were obtained. The distortion coefficient was obtained by fitting the block curve function to calibrate the The deviation of feature points with the same name is used as an objective function to optimize the external parameters of the dual camera module, which improves the calibration accuracy.

在一个实施例中，根据该双摄像头模组中单摄像头的内参和外参及对应的标定图像中的特征点采用分块曲面函数拟合得到该单摄像头的畸变系数，包括：根据该双摄像头模组中单摄像头的内参和外参、标定图像中的特征点得到单摄像头不同角度的目标点点集和畸变点点集，采用分块曲面函数对该目标点点集和畸变点点集进行拟合得到拟合系数，将该拟合系数作为对应摄像头的畸变系数。In one embodiment, the distortion coefficient of the single camera is obtained by fitting a block surface function according to the internal and external parameters of a single camera in the dual camera module and the feature points in the corresponding calibration image, including: according to the dual camera The internal and external parameters of the single camera in the module and the feature points in the calibration image are used to obtain the target point set and the distortion point set of different angles of the single camera. The target surface point set and the distortion point point set are fitted by using the block surface function to obtain a pseudo Combining coefficients, and using this fitting coefficient as the distortion coefficient of the corresponding camera.

目标点点集中的目标点是指各个角度标定板上的特征点投影到摄像头坐标系归一化处理后的点坐标。畸变点点集中的畸变点是指各个角度的标定图像上的特征点转换到摄像 头坐标系归一化处理后的点坐标。The target point in the target point set refers to the point coordinates after the feature points on the calibration plate of each angle are projected onto the camera coordinate system normalization process. Distortion points in the distortion point set refer to the point coordinates after the feature points on the calibration image at various angles are converted to the camera coordinate system normalization process.

在一个实施例中，如图3所示，根据该双摄像头模组中单摄像头的内参和外参、标定图像中的特征点得到单摄像头不同角度的目标点点集和畸变点点集，从操作302开始。In one embodiment, as shown in FIG. 3, according to the internal and external parameters of a single camera in the dual camera module and the feature points in the calibration image, a target point set and a distortion point set of different angles of the single camera are obtained. Start.

操作302，根据该双摄像头模组中单摄像头的外参计算不同角度标定板上的特征点投影到摄像头坐标系并归一化，得到目标点坐标。Inoperation 302, the feature points on the calibration boards at different angles are calculated and projected onto the camera coordinate system according to the external parameters of the single camera in the dual camera module, and the coordinates of the target points are obtained.

利用单摄像头的外参计算各个不同角度的标定板上特征点投影到摄像头坐标系并归一化处理，得到目标点坐标。The external parameters of a single camera are used to calculate the feature points on the calibration boards at different angles and projected to the camera coordinate system and normalized to obtain the target point coordinates.

首先摄像头完整几何模型采用公式(4)First, the complete geometric model of the camera uses formula (4)

其中，公式(4)是将世界坐标系构造在Z为0平面上得到的几何模型，X,Y为平面标定板上特征点的世界坐标，x,y,z为标定板上特征点在摄像头坐标系的物理坐标。Among them, formula (4) is a geometric model obtained by constructing the world coordinate system on the plane Z is 0, X, Y are the world coordinates of the feature points on the calibration plate, and x, y, and z are the feature points on the calibration plate on the camera The physical coordinates of the coordinate system.

R为标定板的世界坐标系到摄像头坐标系的旋转矩阵，T为标定板的世界坐标系到摄像头坐标系的平移矩阵。R is the rotation matrix from the world coordinate system of the calibration plate to the camera coordinate system, and T is the translation matrix from the world coordinate system of the calibration plate to the camera coordinate system.

中Z＝0,则得到

为世界坐标系的坐标点的齐次坐标。

In Z = 0, we get

Homogeneous coordinates of the coordinate points of the world coordinate system.

对标定板上特征点在摄像头坐标系的物理坐标[x,y,z]进行归一化处理，得到目标坐标点(x',y')。The physical coordinates [x, y, z] of the feature points on the calibration board in the camera coordinate system are normalized to obtain the target coordinate points (x ', y').

操作304，根据该双摄像头模组中单摄像头的内参将检测到的不同角度标定图像上的特征点转换到摄像头坐标并归一化，得到畸变点坐标。Inoperation 304, according to internal parameters of a single camera in the dual camera module, the detected feature points on the calibration images at different angles are converted to camera coordinates and normalized to obtain the coordinates of the distortion points.

利用单个摄像头的内参对该单个摄像头检测到的不同角度标定图像上的特征点转换到摄像头坐标并归一化处理得到畸变坐标(x”,y”)。The internal parameters of a single camera are used to convert the feature points on the calibration images detected by the single camera to the camera coordinates and normalize to obtain the distortion coordinates (x ", y").

操作306，根据不同角度的目标点坐标得到单摄像头的目标点点集，根据不同角度的畸变点坐标得到单摄像头的畸变点点集。Operation 306: Obtain a set of target points for a single camera according to the coordinates of the target points at different angles, and obtain a set of distortion points for a single camera according to the coordinates of the distortion points at different angles.

综合各不同角度的目标点坐标得到单摄像头的目标点点集(x'_T,y'_T)，综合各不同角度的畸变点坐标得到单摄像头的畸变点点集(x″_T,y″_T)。The target point set (x '_T , y'_T ) of a single camera is obtained by integrating the target point coordinates at different angles, and the distortion point set (x ″_T , y ″_T ) of a single camera is obtained by integrating the coordinate points of the distortion points of different cameras.

公式(7)中，1,2,3，……，n为不同角度，n为角度数目。In formula (7), 1,2,3, ..., n are different angles, and n is the number of angles.

在一个实施例中，采用分块曲面函数对该目标点点集和畸变点点集进行拟合得到拟合系数，包括：将目标点点集按照块区间进行划分，对划分后的各块的目标点点集与对应的畸变点点集采用分块曲面函数拟合得到各块的拟合系数，根据各块的拟合系数得到对应摄像头的畸变系数。In one embodiment, fitting the target point point set and the distortion point point set to obtain a fitting coefficient by using a block surface function, including: dividing the target point point set according to the block interval, and dividing the divided target point point set for each block. The fitting coefficient of each block is obtained by fitting the corresponding distortion point point set to the block surface function, and the distortion coefficient of the corresponding camera is obtained according to the fitting coefficient of each block.

对目标点点集和畸变点点集利用分块曲面函数分别对x、y方向变形做曲面拟合，得到拟合系数D_x，D_y。For the target point set and the distortion point point set, the surface fitting is performed on the deformation in the x and y directions by using the block surface function, respectively, and the fitting coefficients D_x and D_{y are obtained} .

x″_T＝D_x(x'_T,y'_T)，y″_T＝D_y(x'_T,y'_T)   公式(8)x ″_T = D_x (x '_T , y'_T ), y ″_T = D_y (x '_T , y'_T ) Formula (8)

分块曲面函数可为B样条函数、自由曲面函数、zernike多项式函数中一种。zernike多项式函数是由无穷数量的多项式完全集组成的，它有两个变量，ρ和θ，它在单位圆内部是连续正交的。以B样条函数为例，根据镜头径向畸变曲线特性将目标点点集进行块区间划分，块区间的大小可相同，也可以不同。块数量根据镜头畸变复杂度设定，对于畸变曲线较为光滑的镜头，一般图像长边方向划分为6个块，短边方向划分为4个块能够描述镜头畸变，如图4所示，将目标点点集划分为6*4个块，长边方向有7个点，短边方向有5个点。对于镜头畸变特性较为复杂的则需要更多划分。每个区块用二元N次方程描述，N大于2，根据实际畸变变形情况设置。由于B样条的特性，块之间曲面是连续的，拟合的镜头畸变变形也是连续的。The block surface function can be one of B-spline function, free-form surface function, and zernike polynomial function. The zernike polynomial function is composed of an infinite number of complete sets of polynomials. It has two variables, ρ and θ, which are continuously orthogonal inside the unit circle. Taking the B-spline function as an example, the target point point set is divided into block intervals according to the characteristics of the lens radial distortion curve. The size of the block intervals may be the same or different. The number of blocks is set according to the complexity of the lens distortion. For a lens with a smooth distortion curve, the general image is divided into 6 blocks on the long side and 4 blocks on the short side to describe the lens distortion. As shown in Figure 4, the target is The point set is divided into 6 * 4 blocks, with 7 points on the long side and 5 points on the short side. For more complicated lens distortion characteristics, more division is needed. Each block is described by a binary N-degree equation, where N is greater than 2, and is set according to the actual distortion. Due to the characteristics of the B-spline, the curved surfaces between the blocks are continuous, and the distortion of the fitted lens is also continuous.

B样条曲线是在Bezier曲线基础上发展起来的一类曲线，通常采用的是二次。二次B样条曲线的参数方程，已知三个平面离散点P₀、P₁、P₂，由这三点可以定义二次抛物线段，参数矢量方程形式为：B-spline curve is a type of curve developed on the basis of Bezier curve, usually using quadratic. The parametric equation of a quadratic B-spline curve. Three plane discrete points P₀ , P₁ , and P_{2 are known} . From these three points, a quadratic parabola segment can be defined. The form of the parameter vector equation is:

P(t)＝A₀+A₁t+A₂t²  (0≤t≤1)  公式(9)P (t) = A₀ + A₁ t + A₂ t² (0≤t≤1) Formula (9)

二次B样条曲线参数方程的矩形形式为：The rectangular form of the quadratic B-spline curve parameter equation is:

二次B样条曲线的特点是起点为P₀、P₁点的中点，并与线段P₀P₁相切，终点为P₁、P₂的中点，并与线段P₁P₂相切，除起点、终点外，中间点将曲线拉向自己。若多于3个离散点，则采用二次B样条曲线进行分段拟合。Features quadratic B-spline curves is the starting point P_0, the midpoint of the point P_1, and P₀ P₁ and tangent to the line segment end point is P_1, P₂ of the midpoint, and the line segment P₁ P₂ phase Cut, except for the start and end points, the middle point pulls the curve towards itself. If there are more than 3 discrete points, a quadratic B-spline curve is used for piecewise fitting.

图5为一个实施例中x方向畸变变形曲面拟合结果的示意图。图6为一个实施例中y方向畸变变形曲面拟合的示意图。根据图5可知，B样条拟合块之间的曲面是连续的，根据图6可知，B样条拟合块之间的曲面是连续的。FIG. 5 is a schematic diagram of fitting results of a distortion surface in the x direction in one embodiment. FIG. 6 is a schematic diagram of y-direction distortion and deformation surface fitting in an embodiment. According to FIG. 5, it can be known that the surfaces between the B-spline fitting blocks are continuous, and according to FIG. 6, it can be known that the surfaces between the B-spline fitting blocks are continuous.

在一个实施例中，该标定图像包括不同角度的第一标定图像和第二标定图像；In one embodiment, the calibration image includes a first calibration image and a second calibration image at different angles;

根据该双摄像头模组中单摄像头的内参和外参、标定图像中的特征点得到单摄像头不同角度的目标点点集和畸变点点集，采用分块曲面函数对该目标点点集和畸变点点集进行拟合得到拟合系数，将该拟合系数作为对应摄像头的畸变系数，包括：According to the internal and external parameters of the single camera in the dual camera module, and the feature points in the calibration image, the target point set and the distortion point set of different angles of the single camera are obtained, and the target point set and the distortion point set are obtained by using a block surface function. The fitting coefficient is obtained by fitting, and the fitting coefficient is used as the distortion coefficient of the corresponding camera, including:

根据该第一摄像头的内参和外参、第一标定图像的特征点得到第一摄像头所对应不同角度的第一目标点点集和第一畸变点点集，采用分块曲面函数对该第一目标点点集和第一畸变点点集进行拟合得到第一拟合系数，将该第一拟合系数作为第一摄像头的畸变系数；及The first target point set and the first distortion point set of different angles corresponding to the first camera are obtained according to the internal and external parameters of the first camera, and the feature points of the first calibration image. Fitting the first set of distortion points to the first set of fitting coefficients, and using the first fitting coefficient as the distortion coefficient of the first camera; and

根据该第二摄像头的内参和外参、第二标定图像的特征点得到第二摄像头所对应不同角度的第二目标点点集和第二畸变点点集，采用分块曲面函数对该第二目标点点集和第二畸变点点集进行拟合得到第二拟合系数，将该第二拟合系数作为第二摄像头的畸变系数。According to the internal and external parameters of the second camera and the feature points of the second calibration image, a second target point set and a second distortion point set of different angles corresponding to the second camera are obtained, and the second target point point is obtained by using a block surface function. A second fitting coefficient is obtained by fitting the set and the second set of distortion points, and the second fitting coefficient is used as the distortion coefficient of the second camera.

在一个实施例中，该标定图像为标定板的平面垂直于双摄像头模组光轴时所拍摄的图像，该标定板覆盖该双摄像头模组的视场角。In one embodiment, the calibration image is an image captured when the plane of the calibration plate is perpendicular to the optical axis of the dual camera module, and the calibration plate covers the field of view of the dual camera module.

在一个实施例中，该双摄像头标定方法还包括：对该第一摄像头和第二摄像头进行单摄像头标定，得到该第一摄像头的内参和外参、该第二摄像头的内参和外参。In one embodiment, the dual camera calibration method further includes: performing a single camera calibration on the first camera and the second camera to obtain internal and external parameters of the first camera and internal and external parameters of the second camera.

具体地，首先获取标定板，通过单个摄像头拍摄不同角度的标定板得到标定图像，从标定图像中提取特征点，计算无畸变情况下，单个摄像头的5个内参和2个外参，应用最小二乘法计算得到畸变系数，再通过极大似然法进行优化，得到单个摄像头最终的内参和外参。Specifically, a calibration plate is first obtained, and calibration images of different angles are taken by a single camera to obtain a calibration image. Feature points are extracted from the calibration image to calculate the 5 internal and 2 external parameters of a single camera without distortion. The distortion coefficient is calculated by multiplication, and then optimized by the maximum likelihood method to obtain the final internal and external parameters of a single camera.

首先建立摄像头模型，得到公式(11)。First, a camera model is established, and formula (11) is obtained.

其中，

的齐次坐标表示图像平面的像素坐标(u,v,1)，

的齐次坐标表示世界坐标系的坐标点(X,Y,Z,1)，A表示内参矩阵，R表示世界坐标系转换到摄像头坐标系的旋转矩阵，T表示世界坐标系转换到摄像头坐标系的平移矩阵。among them,

Homogeneous coordinates represent the pixel coordinates (u, v, 1) of the image plane,

Homogeneous coordinates represent the coordinate points of the world coordinate system (X, Y, Z, 1), A represents the internal parameter matrix, R represents the rotation matrix converted from the world coordinate system to the camera coordinate system, and T represents the world coordinate system converted to the camera coordinate system Translation matrix.

其中，α＝f/d_x，β＝f/d_y，f为单摄像头的焦距，d_x表示图像坐标系x轴方向上一个像素的宽度，d_y表示图像坐标系y轴方向上一个像素的宽度。γ代表像素点在x,y方向上尺度的偏差。u₀、v₀表示图像平面的主点坐标，主点是光轴与图像平面的交点。Among them, α = f / d_x , β = f / d_y , f is the focal length of a single camera, d_x represents the width of one pixel in the x-axis direction of the image coordinate system, and d_y represents one pixel in the y-axis direction of the image coordinate system. The width. γ represents the deviation of the pixel point in the x, y direction. u₀ and v₀ represent the coordinates of the principal point of the image plane, and the principal point is the intersection of the optical axis and the image plane.

将世界坐标系构造在Z＝0的平面上，再进行单应性计算，令Z＝0则将上述转换为公式(13)。The world coordinate system is constructed on the plane of Z = 0, and then the homography calculation is performed. When Z = 0, the above is converted into formula (13).

单应性是指在计算机视觉中被定义为一个平面到另一个平面的投影映射。令H＝A[r₁ r₂ t]，H为单应性矩阵。H是一个3*3的矩阵，并且有一个元素作为齐次坐标，因此，H有8个未知量待解。将单应性矩阵写成三个列向量的形式，即H＝[h₁ h₂ h₃]，从而得到公式(14)。Homogeneity is defined in computer vision as a projection mapping from one plane to another. Let H = A [r₁ r₂ t], where H is a homography matrix. H is a 3 * 3 matrix and has one element as a homogeneous coordinate. Therefore, H has 8 unknowns to be solved. The homography matrix is written in the form of three column vectors, that is, H = [h₁ h₂ h₃ ], thereby obtaining formula (14).

[h₁ h₂ h₃]＝λA[r₁ r₂ t]        公式(14)[h₁ h₂ h₃ ] = λA [r₁ r₂ t] Formula (14)

对于公式(14)，采用两个约束条件，第一，r₁,r₂正交，得r₁r₂＝0，r₁,r₂分别绕x,y轴旋转。第二，旋转向量的模为1，即|r₁|＝|r₂|＝1。通过两个约束条件，将r₁,r₂代换为h₁,h₂与A的组合进行表达。即r₁＝h₁A^-1，r₂＝h₂A^-1。根据两个约束条件，可以得到公式(15)：For formula (14), two constraints are used. First, r₁ , r_{2 are} orthogonal, and r₁ r₂ = 0, and r₁ , r₂ are rotated around the x and y axes, respectively. Second, the modulus of the rotation vector is 1, that is, | r₁ | = | r₂ | = 1. Through two constraints, r₁ , r_{2 is} replaced by h₁ , h₂ and A. That is, r₁ = h₁ A^-1 and r₂ = h₂ A^-1 . According to two constraints, formula (15) can be obtained:

令make

B为一个对称阵，故B的有效元素为6个，6个元素构成向量b。B is a symmetric matrix, so the effective elements of B are 6, and the 6 elements constitute the vector b.

b＝[B₁₁,B₁₂,B₂₂,B₁₃,B₂₃,B₃₃]^Tb = [B₁₁ , B₁₂ , B₂₂ , B₁₃ , B₂₃ , B₃₃ ]^T

可以计算得到V_ij＝[h_i1h_j1,h_i1h_j2+h_i2h_j1,h_i2h_j2,h_i3h_j1+h_i1h_j3,h_i3h_j2+h_i2h_j3,h_i3h_j3]^TIt can be calculated that V_ij = [h_i1 h_j1 , h_i1 h_j2 + h_i2 h_j1 , h_i2 h_j2 , h_i3 h_j1 + h_i1 h_j3 , h_i3 h_j2 + h_i2 h_j3 , h_i3 h_j3 ]^T

利用约束条件得到方程组：Use constraints to get the equations:

通过至少三幅图像，应用公式(16)估算出B，对B进行分解得到摄像头的内参矩阵A的初始值。Using at least three images, apply formula (16) to estimate B, and decompose B to obtain the initial value of the internal parameter matrix A of the camera.

基于内参矩阵计算外参矩阵，得到外参矩阵的初始值。The external parameter matrix is calculated based on the internal parameter matrix to obtain the initial value of the external parameter matrix.

其中，λ＝1/||A^-1h₁||＝1/||A^-1h₂||。Where λ = 1 / || A^-1 h₁ || = 1 / || A^-1 h₂ ||.

摄像头完整几何模型采用公式(4)The complete geometric model of the camera uses the formula (4)

其中，公式(4)是将世界坐标系构造在Z为0平面上得到的几何模型，X,Y为平面标定板上特征点的世界坐标，x,y,z为标定板上特征点在摄像头坐标系的物理坐标。Among them, formula (4) is a geometric model obtained by constructing the world coordinate system on the plane Z is 0, X, Y are the world coordinates of the feature points on the calibration plate, and x, y, and z are the feature points on the calibration plate on the camera The physical coordinates of the coordinate system.

R为标定板的世界坐标系到摄像头坐标系的旋转矩阵，T为标定板的世界坐标系到摄像头坐标系的平移矩阵。R is the rotation matrix from the world coordinate system of the calibration plate to the camera coordinate system, and T is the translation matrix from the world coordinate system of the calibration plate to the camera coordinate system.

对标定板上特征点在摄像头坐标系的物理坐标[x,y,z]进行归一化处理，得到目标坐标点(x',y')。The physical coordinates [x, y, z] of the feature points on the calibration board in the camera coordinate system are normalized to obtain the target coordinate points (x ', y').

利用畸变模型对摄像头坐标系像点进行畸变变形处理。Distortion processing is performed on the camera coordinate system image points using the distortion model.

利用内参将物理坐标转换为图像坐标。Use internal parameters to convert physical coordinates to image coordinates.

将内参矩阵的初始值和外参矩阵的初始值导入到极大似然公式得到最终的内参矩阵和外参矩阵。极大似然公式为

求取最小值。The initial values of the internal parameter matrix and the external parameter matrix are imported into the maximum likelihood formula to obtain the final internal parameter matrix and external parameter matrix. The maximum likelihood formula is

Find the minimum.

在一个实施例中，对该第一摄像头和第二摄像头进行单摄像头标定，得到该第一摄像头的内参和外参、该第二摄像头的内参和外参，包括：采用分块曲面函数对该第一摄像头和第二摄像头进行单摄像头标定，得到该第一摄像头的内参和外参、该第二摄像头的内参和外参。In one embodiment, performing a single-camera calibration on the first camera and the second camera to obtain internal and external parameters of the first camera and internal and external parameters of the second camera include: The first camera and the second camera perform single-camera calibration to obtain internal and external parameters of the first camera and internal and external parameters of the second camera.

首先获取标定板，通过单个摄像头拍摄不同角度的标定板得到标定图像，从标定图像 中提取特征点，计算无畸变情况下，单个摄像头的内参初始值和外参的初始值，应用分块曲面函数拟合得到畸变系数，再将内参初始值、外参初始值和畸变系数作为输入通过极大似然法进行优化，得到单个摄像头最终的内参和外参。First obtain a calibration plate, and use a single camera to capture calibration images at different angles to obtain a calibration image. Extract feature points from the calibration image and calculate the initial values of the internal and external parameters of a single camera without distortion. Apply the block surface function. The distortion coefficient is obtained by fitting, and the initial value of the internal parameter, the initial value of the external parameter, and the distortion coefficient are used as inputs to optimize by the maximum likelihood method to obtain the final internal and external parameters of a single camera.

图7为又一个实施例中双摄像头标定的方法的流程图。如图7所示，一种双摄像头标定方法，从操作702开始。7 is a flowchart of a method for calibrating a dual camera according to another embodiment. As shown in FIG. 7, a dual camera calibration method starts fromoperation 702.

操作702，第一摄像头拍摄多个角度的标定板图案得到多个角度的第一标定图像。Operation 702: The first camera captures calibration plate patterns at multiple angles to obtain first calibration images at multiple angles.

操作704，检测第一标定图像中的同名特征点。Operation 704: Detect feature points with the same name in the first calibration image.

操作706，对第一摄像头进行单摄像头标定，得到第一摄像头的内参和外参。Operation 706: Perform single-camera calibration on the first camera to obtain internal and external parameters of the first camera.

操作708，根据第一摄像头的内参和外参及对应的第一标定图像中的特征点采用分块曲面函数拟合得到第一摄像头的畸变系数。Inoperation 708, the distortion coefficient of the first camera is obtained by fitting a block surface function according to the internal parameters and external parameters of the first camera and the feature points in the corresponding first calibration image.

操作712，第二摄像头拍摄多个角度的标定板图案得到多个角度的第二标定图像。Inoperation 712, the second camera captures calibration plate patterns of multiple angles to obtain second calibration images of multiple angles.

操作714，检测第二标定图像中的同名特征点。Operation 714: Detect feature points with the same name in the second calibration image.

操作716，对第二摄像头进行单摄像头标定，得到第二摄像头的内参和外参。Inoperation 716, perform single-camera calibration on the second camera to obtain internal and external parameters of the second camera.

操作718，根据第二摄像头的内参和外参及对应的第二标定图像中的特征点采用分块曲面函数拟合得到第二摄像头的畸变系数。Inoperation 718, the distortion coefficient of the second camera is obtained by fitting a block surface function according to the internal parameters and external parameters of the second camera and the feature points in the corresponding second calibration image.

操作720，根据第一摄像头外参和第二摄像头外参计算双摄像头模组的初始外参。Inoperation 720, an initial external parameter of the dual camera module is calculated according to the first camera external parameter and the second camera external parameter.

操作722，以该第一摄像头和第二摄像头所拍摄的标定图像中的同名特征点的偏差为目标函数，将该第一摄像头和第二摄像头所拍摄的标定图像中的同名特征点、第一摄像头的内参和畸变系数、第二摄像头的内参和畸变系数作为该目标函数的输入，将该双摄像头模组的初始外参作为该目标函数的初始参数，进行处理得到该双摄像头模组的目标外参。Inoperation 722, the deviation of feature points with the same name in the calibration images captured by the first camera and the second camera is used as an objective function, and the feature points with the same name in the calibration image captured by the first camera and the second camera, the first The internal parameters and distortion coefficients of the camera and the internal parameters and distortion coefficients of the second camera are used as the input of the objective function. The initial external parameters of the dual camera module are used as the initial parameters of the objective function, and processed to obtain the target of the dual camera module. External reference.

本申请实施例中的双摄像头标定方法，根据第一摄像头和第二摄像头的内参、外参和标定图像中的特征点进行分块曲面函数拟合得到畸变系数，获取到第一摄像头和第二摄像头拍摄的标定图像中的同名特征点，将第一摄像头和第二摄像头所拍摄标定图像中的同名特征点的偏差作为目标函数，将标定图像的特征点、第一摄像头和第二摄像头的内参作为目标函数的输入，双摄像头模组的初始外参作为目标函数的初始参数进行优化，得到了双摄像头模组的目标外参，利用分块曲线函数拟合得到畸变系数，以标定图像中的同名特征点的偏差作为目标函数对双摄像头模组外参进行优化，提高了标定精度。In the dual camera calibration method in the embodiment of the present application, the distortion coefficient is obtained by fitting the block surface function fitting according to the feature points in the internal and external parameters of the first camera and the second camera and the calibration image to obtain the first camera and the second camera. Feature points with the same name in the calibration image captured by the camera, the deviation of the feature points with the same name in the calibration image captured by the first camera and the second camera is used as the objective function, and the feature points of the calibration image, the internal parameters of the first camera and the second camera As the input of the objective function, the initial external parameters of the dual camera module were optimized as the initial parameters of the objective function, and the target external parameters of the dual camera module were obtained. The distortion coefficient was obtained by fitting the block curve function to calibrate the The deviation of feature points with the same name is used as an objective function to optimize the external parameters of the dual camera module, which improves the calibration accuracy.

应该理解的是，虽然图2、图4和图7中的流程图中的各个步骤按照箭头的指示依次显示，但是这些步骤并不是必然按照箭头指示的顺序依次执行。除非本文中有明确的说明，这些步骤的执行并没有严格的顺序限制，这些步骤可以以其它的顺序执行。而且，图2、图4和图7中的至少一部分步骤可以包括多个子步骤或者多个阶段，这些子步骤或者阶段并不必然是在同一时刻执行完成，而是可以在不同的时刻执行，这些子步骤或者阶段的执行顺序也不必然是依次进行，而是可以与其它步骤或者其它步骤的子步骤或者阶段的至少一部分轮流或者交替地执行。It should be understood that although the steps in the flowcharts in FIG. 2, FIG. 4, and FIG. 7 are sequentially displayed according to the directions of the arrows, these steps are not necessarily performed sequentially in the order indicated by the arrows. Unless explicitly stated in this document, the execution of these steps is not strictly limited, and these steps can be performed in other orders. Moreover, at least a part of the steps in FIG. 2, FIG. 4, and FIG. 7 may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily performed at the same time, but may be performed at different times. These The execution order of the sub-steps or stages is not necessarily performed sequentially, but may be performed in turn or alternately with at least a part of the other steps or the sub-steps or stages of the other steps.

图8为一个实施例中双摄像头标定装置的结构框图。如图8所示，一种双摄像头标定装置，包括采集单元810、检测单元820、拟合单元830、确定单元840和调整单元850。FIG. 8 is a structural block diagram of a dual camera calibration device according to an embodiment. As shown in FIG. 8, a dual-camera calibration device includes anacquisition unit 810, adetection unit 820, afitting unit 830, adetermination unit 840, and anadjustment unit 850.

采集单元810用于通过双摄像头模组拍摄不同角度的标定板，得到不同角度的标定图像。该双摄像头模组包括第一摄像头和第二摄像头。Theacquisition unit 810 is used for shooting calibration plates of different angles through a dual camera module to obtain calibration images of different angles. The dual camera module includes a first camera and a second camera.

检测单元820用于检测该标定图像中的特征点。The detectingunit 820 is configured to detect feature points in the calibration image.

拟合单元830用于根据该双摄像头模组中单摄像头的内参和外参及对应的标定图像中的特征点采用分块曲面函数拟合得到该单摄像头的畸变系数，该分块曲面函数的拟合结果连续。该分块曲面函数在块与块之间连续可导，且拟合结果连续。Thefitting unit 830 is configured to obtain a distortion coefficient of the single camera according to the internal parameters and external parameters of the single camera in the dual camera module and feature points in the corresponding calibration image by using a piecewise curved surface function. The fitting results are continuous. The block surface function is continuously differentiable between blocks, and the fitting results are continuous.

确定单元840用于根据该双摄像头模组中单摄像头的外参确定该双摄像头模组的初始外参。The determiningunit 840 is configured to determine an initial external parameter of the dual camera module according to an external parameter of a single camera in the dual camera module.

调整单元850用于根据该双摄像头模组的初始外参、单摄像头的畸变系数和单摄像头 的内参进行处理得到该双摄像头模组的目标外参。The adjustingunit 850 is configured to obtain a target external parameter of the dual camera module by processing based on an initial external parameter of the dual camera module, a distortion coefficient of the single camera, and an internal parameter of the single camera.

调整单元850还用于以双摄像头模组中的第一摄像头和第二摄像头所拍摄的标定图像中的同名特征点的偏差为目标函数，将该第一摄像头和第二摄像头所拍摄的标定图像中的同名特征点、该单摄像头的内参和畸变系数作为该目标函数的输入，将该双摄像头模组的初始外参作为该目标函数的初始参数，进行处理得到该双摄像头模组的目标外参。The adjustingunit 850 is further configured to use the deviation of feature points of the same name in the calibration images captured by the first camera and the second camera in the dual camera module as an objective function, and use the calibration images captured by the first camera and the second camera Feature points of the same name, internal parameters and distortion coefficients of the single camera are used as inputs to the objective function, and initial external parameters of the dual camera module are used as initial parameters of the objective function, and processed to obtain the target external parameters of the dual camera module. See.

在一个实施例中，拟合单元830还用于根据该双摄像头模组中单摄像头的内参和外参、标定图像中的特征点得到单摄像头不同角度的目标点点集和畸变点点集，采用分块曲面函数对该目标点点集和畸变点点集进行拟合得到拟合系数，将该拟合系数作为对应摄像头的畸变系数。In one embodiment, thefitting unit 830 is further configured to obtain the target point set and the distortion point set of different angles of the single camera according to the internal and external parameters of the single camera in the dual camera module, and the feature points in the calibration image. The block surface function fits the target point set and the distortion point point set to obtain a fitting coefficient, and the fitting coefficient is used as the distortion coefficient of the corresponding camera.

在一个实施例中，拟合单元830还用于根据该双摄像头模组中单摄像头的外参计算不同角度标定板上的特征点投影到摄像头坐标系并归一化，得到目标点坐标；根据该双摄像头模组中单摄像头的内参将检测到的不同角度标定图像上的特征点转换到摄像头坐标并归一化，得到畸变点坐标；及根据不同角度的目标点坐标得到单摄像头的目标点点集，根据不同角度的畸变点坐标得到单摄像头的畸变点点集。In one embodiment, thefitting unit 830 is further configured to calculate, based on the external parameters of a single camera in the dual camera module, the feature points on the calibration boards at different angles and project them onto the camera coordinate system and normalize to obtain the target point coordinates; The internal parameters of the single camera in the dual camera module convert the detected feature points on the calibration images at different angles to the camera coordinates and normalize to obtain the distortion point coordinates; and obtain the target point points of the single camera according to the target point coordinates of different angles Set, the distortion point set of a single camera is obtained according to the distortion point coordinates of different angles.

在一个实施例中，该标定图像包括不同角度的第一标定图像和第二标定图像；In one embodiment, the calibration image includes a first calibration image and a second calibration image at different angles;

拟合单元830还用于根据该第一摄像头的内参和外参、第一标定图像的特征点得到第一摄像头所对应不同角度的第一目标点点集和第一畸变点点集，采用分块曲面函数对该第一目标点点集和第一畸变点点集进行拟合得到第一拟合系数，将该第一拟合系数作为第一摄像头的畸变系数；及根据该第二摄像头的内参和外参、第二标定图像的特征点得到第二摄像头所对应不同角度的第二目标点点集和第二畸变点点集，采用分块曲面函数对该第二目标点点集和第二畸变点点集进行拟合得到第二拟合系数，将该第二拟合系数作为第二摄像头的畸变系数。Thefitting unit 830 is further configured to obtain the first target point point set and the first distortion point point set of different angles corresponding to the first camera according to the internal and external parameters of the first camera, and the feature points of the first calibration image. The function fits the first target point set and the first distortion point point set to obtain a first fitting coefficient, and the first fitting coefficient is used as a distortion coefficient of the first camera; and according to an internal parameter and an external parameter of the second camera The characteristic points of the second calibration image are obtained from the second target point set and the second distortion point set at different angles corresponding to the second camera, and the second target point set and the second distortion point set are fitted by using a block surface function. A second fitting coefficient is obtained, and the second fitting coefficient is used as a distortion coefficient of the second camera.

在一个实施例中，拟合单元830还用于将目标点点集按照块区间进行划分，对划分后的各块的目标点点集与对应的畸变点点集采用分块曲面函数拟合得到各块的拟合系数，根据各块的拟合系数得到对应摄像头的畸变系数。In one embodiment, thefitting unit 830 is further configured to divide the target point point set according to the block interval, and use the block surface function to fit the target point point set of each divided block and the corresponding distortion point point set to obtain the Fitting coefficient, according to the fitting coefficient of each block, the distortion coefficient of the corresponding camera is obtained.

在一个实施例中，该块区间的大小相同或不同。In one embodiment, the sizes of the block intervals are the same or different.

在一个实施例中，该标定图像为标定板的平面垂直于双摄像头模组光轴时所拍摄的图像，该标定板覆盖该双摄像头模组的视场角。In one embodiment, the calibration image is an image captured when the plane of the calibration plate is perpendicular to the optical axis of the dual camera module, and the calibration plate covers the field of view of the dual camera module.

在一个实施例中，双摄像头标定装置还包括标定单元。标定单元用于对该第一摄像头和第二摄像头进行单摄像头标定，得到该第一摄像头的内参和外参、该第二摄像头的内参和外参。In one embodiment, the dual camera calibration device further includes a calibration unit. The calibration unit is configured to perform single-camera calibration on the first camera and the second camera to obtain internal and external parameters of the first camera and internal and external parameters of the second camera.

在一个实施例中，标定单元还用于采用分块曲面函数对该第一摄像头和第二摄像头进行单摄像头标定，得到该第一摄像头的内参和外参、该第二摄像头的内参和外参。In one embodiment, the calibration unit is further configured to perform single-camera calibration on the first camera and the second camera using a block surface function to obtain internal and external parameters of the first camera and internal and external parameters of the second camera. .

在一个实施例中，该分块曲面函数为B样条函数、自由曲面函数、zernike多项式函数中一种。In one embodiment, the block surface function is one of a B-spline function, a free-form surface function, and a zernike polynomial function.

在一个实施例中，该标定板为平面标定板或立体标定板，该平面标定板的图案为棋盘格、圆阵列或圆环阵列。In one embodiment, the calibration plate is a planar calibration plate or a three-dimensional calibration plate, and the pattern of the planar calibration plate is a checkerboard, a circular array, or a circular array.

本申请实施例还提供了一种电子设备。该电子设备，包括存储器及处理器，该存储器中储存有计算机程序，该计算机程序被该处理器执行时，使得该处理器执行双摄像头标定方法中的操作。An embodiment of the present application further provides an electronic device. The electronic device includes a memory and a processor. A computer program is stored in the memory. When the computer program is executed by the processor, the processor causes the processor to perform operations in a dual camera calibration method.

本申请实施例提供了一种非易失性计算机可读存储介质。一种非易失性计算机可读存储介质，其上存储有计算机程序，该计算机程序被处理器执行时实现以下双摄像头标定方法中的操作。An embodiment of the present application provides a non-volatile computer-readable storage medium. A non-volatile computer-readable storage medium stores a computer program thereon. When the computer program is executed by a processor, the operations in the following dual camera calibration method are implemented.

图9为一个实施例中电子设备的内部结构示意图。如图9所示，该电子设备包括通过系统总线连接的处理器、存储器和网络接口。其中，该处理器用于提供计算和控制能力， 支撑整个电子设备的运行。存储器用于存储数据、程序等，存储器上存储至少一个计算机程序，该计算机程序可被处理器执行，以实现本申请实施例中提供的适用于电子设备的无线网络通信方法。存储器可包括非易失性存储介质及内存储器。非易失性存储介质存储有操作系统和计算机程序。该计算机程序可被处理器所执行，以用于实现以下各个实施例所提供的一种双摄像头标定方法。内存储器为非易失性存储介质中的操作系统计算机程序提供高速缓存的运行环境。网络接口可以是以太网卡或无线网卡等，用于与外部的电子设备进行通信。该电子设备可以是手机、平板电脑或者个人数字助理或穿戴式设备等。FIG. 9 is a schematic diagram of an internal structure of an electronic device in an embodiment. As shown in FIG. 9, the electronic device includes a processor, a memory, and a network interface connected through a system bus. The processor is used to provide computing and control capabilities to support the operation of the entire electronic device. The memory is used to store data, programs, and the like. At least one computer program is stored on the memory, and the computer program can be executed by a processor to implement the wireless network communication method applicable to the electronic device provided in the embodiments of the present application. The memory may include a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The computer program can be executed by a processor to implement a dual camera calibration method provided by each of the following embodiments. The internal memory provides a cached operating environment for operating system computer programs in a non-volatile storage medium. The network interface may be an Ethernet card or a wireless network card, and is used to communicate with external electronic devices. The electronic device may be a mobile phone, a tablet computer, or a personal digital assistant or a wearable device.

本申请实施例中提供的双摄像头标定装置中的各个模块的实现可为计算机程序的形式。该计算机程序可在终端或服务器上运行。该计算机程序构成的程序模块可存储在终端或服务器的存储器上。该计算机程序被处理器执行时，实现本申请实施例中所描述方法的步骤。The implementation of each module in the dual camera calibration device provided in the embodiment of the present application may be in the form of a computer program. The computer program can be run on a terminal or a server. The program module constituted by the computer program can be stored in the memory of the terminal or server. When the computer program is executed by a processor, the steps of the method described in the embodiments of the present application are implemented.

一种包含指令的计算机程序产品，当其在计算机上运行时，使得计算机执行双摄像头标定方法。A computer program product containing instructions that, when run on a computer, causes the computer to perform a dual camera calibration method.

本申请实施例还提供一种电子设备。上述电子设备中包括图像处理电路，图像处理电路可以利用硬件和/或软件组件实现，可包括定义ISP(Image Signal Processing，图像信号处理)管线的各种处理单元。图10为一个实施例中图像处理电路的示意图。如图10所示，为便于说明，仅示出与本申请实施例相关的图像处理技术的各个方面。An embodiment of the present application further provides an electronic device. The above electronic device includes an image processing circuit. The image processing circuit may be implemented by hardware and / or software components, and may include various processing units that define an ISP (Image Signal Processing) pipeline. FIG. 10 is a schematic diagram of an image processing circuit in an embodiment. As shown in FIG. 10, for convenience of explanation, only aspects of the image processing technology related to the embodiments of the present application are shown.

如图10所示，图像处理电路包括第一ISP处理器1030、第二ISP处理器1040和控制逻辑器1050。第一摄像头1010包括一个或多个第一透镜1012和第一图像传感器1014。第一图像传感器1014可包括色彩滤镜阵列(如Bayer滤镜)，第一图像传感器1014可获取用第一图像传感器1014的每个成像像素捕捉的光强度和波长信息，并提供可由第一ISP处理器1030处理的一组图像数据。第二摄像头1020包括一个或多个第二透镜1022和第二图像传感器1024。第二图像传感器1024可包括色彩滤镜阵列(如Bayer滤镜)，第二图像传感器1024可获取用第二图像传感器1024的每个成像像素捕捉的光强度和波长信息，并提供可由第二ISP处理器1040处理的一组图像数据。As shown in FIG. 10, the image processing circuit includes afirst ISP processor 1030, asecond ISP processor 1040, and acontrol logic 1050. Thefirst camera 1010 includes one or morefirst lenses 1012 and afirst image sensor 1014. Thefirst image sensor 1014 may include a color filter array (such as a Bayer filter). Thefirst image sensor 1014 may obtain light intensity and wavelength information captured by each imaging pixel of thefirst image sensor 1014, and provide information that may be captured by the first ISP. A set of image data processed by theprocessor 1030. Thesecond camera 1020 includes one or moresecond lenses 1022 and asecond image sensor 1024. Thesecond image sensor 1024 may include a color filter array (such as a Bayer filter). Thesecond image sensor 1024 may obtain light intensity and wavelength information captured by each imaging pixel of thesecond image sensor 1024, and provide information that may be captured by a second ISP. A set of image data processed by theprocessor 1040.

第一摄像头1010采集的第一图像传输给第一ISP处理器1030进行处理，第一ISP处理器1030处理第一图像后，可将第一图像的统计数据(如图像的亮度、图像的反差值、图像的颜色等)发送给控制逻辑器1050，控制逻辑器1050可根据统计数据确定第一摄像头1010的控制参数，从而第一摄像头1010可根据控制参数进行自动对焦、自动曝光等操作。第一图像经过第一ISP处理器1030进行处理后可存储至图像存储器1060中，第一ISP处理器1030也可以读取图像存储器1060中存储的图像以对进行处理。另外，第一图像经过ISP处理器1030进行处理后可直接发送至显示器1070进行显示，显示器1070也可以读取图像存储器1060中的图像以进行显示。The first image collected by thefirst camera 1010 is transmitted to thefirst ISP processor 1030 for processing. After thefirst ISP processor 1030 processes the first image, the first image statistical data (such as the brightness of the image and the contrast value of the image) can be processed. , Image color, etc.) to thecontrol logic 1050. Thecontrol logic 1050 can determine the control parameters of thefirst camera 1010 according to the statistical data, so that thefirst camera 1010 can perform operations such as autofocus and automatic exposure according to the control parameters. The first image may be stored in theimage memory 1060 after being processed by thefirst ISP processor 1030, and thefirst ISP processor 1030 may also read the image stored in theimage memory 1060 for processing. In addition, the first image can be directly sent to thedisplay 1070 for display after being processed by theISP processor 1030, and thedisplay 1070 can also read the image in theimage memory 1060 for display.

其中，第一ISP处理器1030按多种格式逐个像素地处理图像数据。例如，每个图像像素可具有8、10、12或14比特的位深度，第一ISP处理器1030可对图像数据进行一个或多个图像处理操作、收集关于图像数据的统计信息。其中，图像处理操作可按相同或不同的位深度精度进行。Thefirst ISP processor 1030 processes the image data pixel by pixel in a variety of formats. For example, each image pixel may have a bit depth of 8, 10, 12, or 14 bits, and thefirst ISP processor 1030 may perform one or more image processing operations on the image data and collect statistical information about the image data. The image processing operations may be performed with the same or different bit depth accuracy.

图像存储器1060可为存储器装置的一部分、存储设备、或电子设备内的独立的专用存储器，并可包括DMA(Direct Memory Access，直接直接存储器存取)特征。Theimage memory 1060 may be a part of a memory device, a storage device, or a separate dedicated memory in an electronic device, and may include a DMA (Direct Memory Access) feature.

当接收到来自第一图像传感器1014接口时，第一ISP处理器1030可进行一个或多个图像处理操作，如时域滤波。处理后的图像数据可发送给图像存储器1060，以便在被显示之前进行另外的处理。第一ISP处理器1030从图像存储器1060接收处理数据，并对所述处理数据进行RGB和YCbCr颜色空间中的图像数据处理。第一ISP处理器1030处理后的图像数据可输出给显示器1070，以供用户观看和/或由图形引擎或GPU(Graphics  Processing Unit，图形处理器)进一步处理。此外，第一ISP处理器1030的输出还可发送给图像存储器1060，且显示器1070可从图像存储器1060读取图像数据。在一个实施例中，图像存储器1060可被配置为实现一个或多个帧缓冲器。When receiving the interface from thefirst image sensor 1014, thefirst ISP processor 1030 may perform one or more image processing operations, such as time-domain filtering. The processed image data may be sent to theimage memory 1060 for further processing before being displayed. Thefirst ISP processor 1030 receives processed data from theimage memory 1060, and performs image data processing in the RGB and YCbCr color spaces on the processed data. The image data processed by thefirst ISP processor 1030 may be output to adisplay 1070 for viewing by a user and / or further processed by a graphics engine or a GPU (Graphics Processing Unit). In addition, the output of thefirst ISP processor 1030 can also be sent to theimage memory 1060, and thedisplay 1070 can read image data from theimage memory 1060. In one embodiment, theimage memory 1060 may be configured to implement one or more frame buffers.

第一ISP处理器1030确定的统计数据可发送给控制逻辑器1050。例如，统计数据可包括自动曝光、自动白平衡、自动聚焦、闪烁检测、黑电平补偿、第一透镜1012阴影校正等第一图像传感器1014统计信息。控制逻辑器1050可包括执行一个或多个例程(如固件)的处理器和/或微控制器，一个或多个例程可根据接收的统计数据，确定第一摄像头1010的控制参数及第一ISP处理器1030的控制参数。例如，第一摄像头1010的控制参数可包括增益、曝光控制的积分时间、防抖参数、闪光控制参数、第一透镜1012控制参数(例如聚焦或变焦用焦距)、或这些参数的组合等。ISP控制参数可包括用于自动白平衡和颜色调整(例如，在RGB处理期间)的增益水平和色彩校正矩阵，以及第一透镜1012阴影校正参数。The statistical data determined by thefirst ISP processor 1030 may be sent to thecontrol logic 1050. For example, the statistical data may include thefirst image sensor 1014 statistical information such as auto exposure, auto white balance, auto focus, flicker detection, black level compensation, andfirst lens 1012 shading correction. Thecontrol logic 1050 may include a processor and / or a microcontroller that executes one or more routines (such as firmware). The one or more routines may determine the control parameters and thefirst camera 1010 according to the received statistical data. Control parameters of anISP processor 1030. For example, the control parameters of thefirst camera 1010 may include gain, integration time of exposure control, image stabilization parameters, flash control parameters, control parameters of the first lens 1012 (for example, focal length for focusing or zooming), or a combination of these parameters. The ISP control parameters may include a gain level and a color correction matrix for automatic white balance and color adjustment (eg, during RGB processing), and afirst lens 1012 shading correction parameter.

同样地，第二摄像头1020采集的第二图像传输给第二ISP处理器1040进行处理，第二ISP处理器1040处理第一图像后，可将第二图像的统计数据(如图像的亮度、图像的反差值、图像的颜色等)发送给控制逻辑器1050，控制逻辑器1050可根据统计数据确定第二摄像头1020的控制参数，从而第二摄像头1020可根据控制参数进行自动对焦、自动曝光等操作。第二图像经过第二ISP处理器1040进行处理后可存储至图像存储器1060中，第二ISP处理器1040也可以读取图像存储器1060中存储的图像以对进行处理。另外，第二图像经过ISP处理器1040进行处理后可直接发送至显示器1070进行显示，显示器1070也可以读取图像存储器1060中的图像以进行显示。第二摄像头1020和第二ISP处理器1040也可以实现如第一摄像头1010和第一ISP处理器1030所描述的处理过程。Similarly, the second image collected by thesecond camera 1020 is transmitted to thesecond ISP processor 1040 for processing. After thesecond ISP processor 1040 processes the first image, statistical data of the second image (such as image brightness, image (Contrast value, image color, etc.) are sent to thecontrol logic 1050. Thecontrol logic 1050 can determine the control parameters of thesecond camera 1020 according to the statistical data, so that thesecond camera 1020 can perform operations such as autofocus and automatic exposure according to the control parameters . The second image may be stored in theimage memory 1060 after being processed by thesecond ISP processor 1040, and thesecond ISP processor 1040 may also read the image stored in theimage memory 1060 for processing. In addition, the second image may be directly sent to thedisplay 1070 for display after being processed by theISP processor 1040, and thedisplay 1070 may also read the image in theimage memory 1060 for display. Thesecond camera 1020 and thesecond ISP processor 1040 can also implement the processing procedures described by thefirst camera 1010 and thefirst ISP processor 1030.

以下为运用图10中图像处理技术实现双摄像头标定方法的步骤。The following are the steps for implementing the dual camera calibration method using the image processing technology in FIG. 10.

本申请所使用的对存储器、存储、数据库或其它介质的任何引用可包括非易失性和/或易失性存储器。合适的非易失性存储器可包括只读存储器(ROM)、可编程ROM(PROM)、电可编程ROM(EPROM)、电可擦除可编程ROM(EEPROM)或闪存。易失性存储器可包括随机存取存储器(RAM)，它用作外部高速缓冲存储器。作为说明而非局限，RAM以多种形式可得，诸如静态RAM(SRAM)、动态RAM(DRAM)、同步DRAM(SDRAM)、双数据率SDRAM(DDR SDRAM)、增强型SDRAM(ESDRAM)、同步链路(Synchlink)DRAM(SLDRAM)、存储器总线(Rambus)直接RAM(RDRAM)、直接存储器总线动态RAM(DRDRAM)、以及存储器总线动态RAM(RDRAM)。Any reference to memory, storage, database, or other media used in this application may include non-volatile and / or volatile memory. Suitable non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory can include random access memory (RAM), which is used as external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDR, SDRAM), enhanced SDRAM (ESDRAM), synchronous Link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

以上所述实施例仅表达了本申请的几种实施方式，其描述较为具体和详细，但并不能因此而理解为对本申请专利范围的限制。应当指出的是，对于本领域的普通技术人员来说，在不脱离本申请构思的前提下，还可以做出若干变形和改进，这些都属于本申请的保护范围。因此，本申请专利的保护范围应以所附权利要求为准。The above-mentioned embodiments only express several implementation manners of the present application, and their descriptions are more specific and detailed, but they should not be construed as limiting the patent scope of the present application. It should be noted that, for those of ordinary skill in the art, without departing from the concept of the present application, several modifications and improvements can be made, and these all belong to the protection scope of the present application. Therefore, the protection scope of this application patent shall be subject to the appended claims.

Claims (34)

Translated fromChinese

一种双摄像头标定方法，其特征在于，包括：A dual camera calibration method is characterized in that it includes:通过双摄像头模组拍摄不同角度的标定板，得到不同角度的标定图像；The calibration plates of different angles are taken by the dual camera module to obtain the calibration images of different angles;检测所述标定图像中的特征点；Detecting feature points in the calibration image;根据所述双摄像头模组中单摄像头的内参和外参及对应的标定图像中的特征点采用分块曲面函数拟合得到所述单摄像头的畸变系数，所述分块曲面函数的拟合结果连续；According to the internal and external parameters of a single camera in the dual camera module and the feature points in the corresponding calibration image, a block surface function is used to obtain the distortion coefficient of the single camera, and the fitting result of the block surface function is obtained. continuous;根据所述双摄像头模组中单摄像头的外参确定所述双摄像头模组的初始外参；及Determining the initial external parameters of the dual camera module according to the external parameters of the single camera in the dual camera module; and根据所述双摄像头模组的初始外参、单摄像头的畸变系数和单摄像头的内参进行处理得到所述双摄像头模组的目标外参。The target external parameters of the dual camera module are obtained by processing according to the initial external parameters of the dual camera module, the distortion coefficient of the single camera, and the internal parameters of the single camera.根据权利要求1所述的方法，其特征在于，所述根据所述双摄像头模组中单摄像头的内参和外参及对应的标定图像中的特征点采用分块曲面函数拟合得到所述单摄像头的畸变系数，包括：The method according to claim 1, wherein the single-camera internal and external parameters in the dual-camera module and the feature points in the corresponding calibration image are fitted using a block surface function to obtain the single Camera distortion coefficients, including:根据所述双摄像头模组中单摄像头的内参和外参、标定图像中的特征点得到单摄像头不同角度的目标点点集和畸变点点集，采用分块曲面函数对所述目标点点集和畸变点点集进行拟合得到拟合系数，将所述拟合系数作为对应摄像头的畸变系数。According to the internal and external parameters of the single camera in the dual camera module, and the feature points in the calibration image, a target point set and a distortion point set of different angles of the single camera are obtained, and a block surface function is adopted to the target point set and the distortion point. A set is fitted to obtain a fitting coefficient, and the fitting coefficient is used as a distortion coefficient of a corresponding camera.根据权利要求2所述的方法，其特征在于，所述根据所述双摄像头模组中单摄像头的内参和外参、标定图像中的特征点得到单摄像头不同角度的目标点点集和畸变点点集，包括：The method according to claim 2, characterized in that the target point set and the distortion point point set of different angles of the single camera are obtained according to the internal and external parameters of a single camera in the dual camera module, and feature points in the calibration image. ,include:根据所述双摄像头模组中单摄像头的外参计算不同角度标定板上的特征点投影到摄像头坐标系并归一化，得到目标点坐标；Calculating the feature points on the calibration plate at different angles according to the external parameters of the single camera in the dual camera module and projecting them onto the camera coordinate system and normalizing to obtain the target point coordinates;根据所述双摄像头模组中单摄像头的内参将检测到的不同角度标定图像上的特征点转换到摄像头坐标并归一化，得到畸变点坐标；及Converting the detected feature points on the calibration images at different angles to the camera coordinates and normalizing according to the internal parameters of the single camera in the dual camera module to obtain the coordinates of the distortion points; and根据不同角度的目标点坐标得到单摄像头的目标点点集，根据不同角度的畸变点坐标得到单摄像头的畸变点点集。The target point set of a single camera is obtained according to the target point coordinates of different angles, and the distortion point set of a single camera is obtained according to the distortion point coordinates of different angles.根据权利要求2所述的方法，其特征在于，所述双摄像头模组包括第一摄像头和第二摄像头，所述标定图像包括不同角度的第一标定图像和第二标定图像；The method according to claim 2, wherein the dual camera module includes a first camera and a second camera, and the calibration image includes a first calibration image and a second calibration image at different angles;根据所述双摄像头模组中单摄像头的内参和外参、标定图像中的特征点得到单摄像头不同角度的目标点点集和畸变点点集，采用分块曲面函数对所述目标点点集和畸变点点集进行拟合得到拟合系数，将所述拟合系数作为对应摄像头的畸变系数，包括：According to the internal and external parameters of the single camera in the dual camera module, and the feature points in the calibration image, a target point set and a distortion point set of different angles of the single camera are obtained, and a block surface function is adopted to the target point set and the distortion point A set is fitted to obtain a fitting coefficient, and the fitting coefficient is used as a distortion coefficient of the corresponding camera, including:根据所述第一摄像头的内参和外参、第一标定图像的特征点得到第一摄像头所对应不同角度的第一目标点点集和第一畸变点点集，采用分块曲面函数对所述第一目标点点集和第一畸变点点集进行拟合得到第一拟合系数，将所述第一拟合系数作为第一摄像头的畸变系数；及The first target point set and the first distortion point set of different angles corresponding to the first camera are obtained according to the internal parameters and external parameters of the first camera and the feature points of the first calibration image, and the first curved point function is used for the first camera. Fitting the target point set and the first distortion point set to obtain a first fitting coefficient, and using the first fitting coefficient as the distortion coefficient of the first camera; and根据所述第二摄像头的内参和外参、第二标定图像的特征点得到第二摄像头所对应不同角度的第二目标点点集和第二畸变点点集，采用分块曲面函数对所述第二目标点点集和第二畸变点点集进行拟合得到第二拟合系数，将所述第二拟合系数作为第二摄像头的畸变系数。According to the internal and external parameters of the second camera, and the feature points of the second calibration image, a second target point set and a second distortion point set of different angles corresponding to the second camera are obtained, and a block surface function is used for the second The target point set and the second distortion point set are fitted to obtain a second fitting coefficient, and the second fitting coefficient is used as a distortion coefficient of the second camera.根据权利要求2所述的方法，其特征在于，所述采用分块曲面函数对所述目标点点集和畸变点点集进行拟合得到拟合系数，包括：The method according to claim 2, wherein the fitting of the target point point set and the distortion point point set by using a piecewise curved surface function to obtain a fitting coefficient comprises:将目标点点集按照块区间进行划分，对划分后的各块的目标点点集与对应的畸变点点集采用分块曲面函数拟合得到各块的拟合系数，根据各块的拟合系数得到对应摄像头的畸变系数。The target point set is divided according to the block interval. The target point point set of each divided block and the corresponding distortion point point set are fitted by a block surface function to obtain the fitting coefficients of each block, and the corresponding coefficients are obtained according to the fitting coefficients of each block. The distortion coefficient of the camera.根据权利要求5所述的方法，其特征在于，所述块区间的大小相同或不同。The method according to claim 5, wherein the sizes of the block intervals are the same or different.根据权利要求1所述的方法，其特征在于，根据所述双摄像头模组的初始外参、 单摄像头的畸变系数、单摄像头的内参进行处理得到所述双摄像头模组的目标外参，包括：The method according to claim 1, characterized in that processing the initial external parameters of the dual camera module, the distortion coefficient of the single camera, and the internal parameters of the single camera to obtain the target external parameters of the dual camera module includes: :以双摄像头模组中的第一摄像头和第二摄像头所拍摄的标定图像中的特征点的偏差为目标函数，将所述第一摄像头和第二摄像头所拍摄的标定图像中的特征点、所述单摄像头的内参和畸变系数作为所述目标函数的输入，将所述双摄像头模组的初始外参作为所述目标函数的初始参数，进行处理得到所述双摄像头模组的目标外参。The deviation of the feature points in the calibration images captured by the first camera and the second camera in the dual camera module is used as an objective function, and the feature points, locations in the calibration images captured by the first camera and the second camera are The internal parameters and distortion coefficients of the single camera are used as the input of the objective function, and the initial external parameters of the dual camera module are used as the initial parameters of the objective function, and processed to obtain the target external parameters of the dual camera module.根据权利要求1所述的方法，其特征在于，所述标定图像为标定板的平面垂直于双摄像头模组光轴时所拍摄的图像，所述标定板覆盖所述双摄像头模组的视场角。The method according to claim 1, wherein the calibration image is an image captured when a plane of the calibration plate is perpendicular to an optical axis of the dual camera module, and the calibration plate covers a field of view of the dual camera module angle.根据权利要求1所述的方法，其特征在于，所述方法还包括：The method according to claim 1, further comprising:对所述第一摄像头和第二摄像头进行单摄像头标定，得到所述第一摄像头的内参和外参、所述第二摄像头的内参和外参。Single camera calibration is performed on the first camera and the second camera to obtain internal and external parameters of the first camera and internal and external parameters of the second camera.根据权利要求9所述的方法，其特征在于，所述对所述第一摄像头和第二摄像头进行单摄像头标定，得到所述第一摄像头的内参和外参、所述第二摄像头的内参和外参，包括：The method according to claim 9, wherein the single camera calibration is performed on the first camera and the second camera to obtain internal and external parameters of the first camera and internal and external parameters of the second camera. External parameters, including:采用分块曲面函数对所述第一摄像头和第二摄像头进行单摄像头标定，得到所述第一摄像头的内参和外参、所述第二摄像头的内参和外参。A single-camera calibration is performed on the first camera and the second camera using a segmented curved surface function to obtain internal and external parameters of the first camera and internal and external parameters of the second camera.根据权利要求1所述的方法，其特征在于，所述分块曲面函数为B样条函数、自由曲面函数、zernike多项式函数中一种。The method according to claim 1, wherein the piecewise curved surface function is one of a B-spline function, a free-form surface function, and a zernike polynomial function.根据权利要求1所述的方法，其特征在于，所述标定板为平面标定板或立体标定板，所述平面标定板的图案为棋盘格、圆阵列或圆环阵列。The method according to claim 1, wherein the calibration plate is a planar calibration plate or a three-dimensional calibration plate, and the pattern of the planar calibration plate is a checkerboard, a circular array, or a circular array.一种电子设备，包括存储器及处理器，所述存储器中储存有计算机程序，所述计算机程序被所述处理器执行时，使得所述处理器执行以下步骤：An electronic device includes a memory and a processor. The memory stores a computer program. When the computer program is executed by the processor, the processor causes the processor to perform the following steps:通过双摄像头模组拍摄不同角度的标定板，得到不同角度的标定图像；The calibration plates of different angles are taken by the dual camera module to obtain the calibration images of different angles;检测所述标定图像中的特征点；Detecting feature points in the calibration image;根据所述双摄像头模组中单摄像头的内参和外参及对应的标定图像中的特征点采用分块曲面函数拟合得到所述单摄像头的畸变系数，所述分块曲面函数的拟合结果连续；According to the internal and external parameters of a single camera in the dual camera module and the feature points in the corresponding calibration image, a block surface function is used to obtain the distortion coefficient of the single camera, and the fitting result of the block surface function is obtained. continuous;根据所述双摄像头模组中单摄像头的外参确定所述双摄像头模组的初始外参；及Determining the initial external parameters of the dual camera module according to the external parameters of the single camera in the dual camera module; and根据所述双摄像头模组的初始外参、单摄像头的畸变系数和单摄像头的内参进行处理得到所述双摄像头模组的目标外参。The target external parameters of the dual camera module are obtained by processing according to the initial external parameters of the dual camera module, the distortion coefficient of the single camera, and the internal parameters of the single camera.根据权利要求13所述的电子设备，其特征在于，所述处理器还配置为执行：根据所述双摄像头模组中单摄像头的内参和外参、标定图像中的特征点得到单摄像头不同角度的目标点点集和畸变点点集，采用分块曲面函数对所述目标点点集和畸变点点集进行拟合得到拟合系数，将所述拟合系数作为对应摄像头的畸变系数。The electronic device according to claim 13, wherein the processor is further configured to execute: obtaining different angles of a single camera according to internal and external parameters of a single camera in the dual camera module and feature points in a calibration image The target point point set and the distortion point point set are obtained by fitting the target point point set and the distortion point point set using a block surface function to obtain a fitting coefficient, and the fitting coefficient is used as the distortion coefficient of the corresponding camera.根据权利要求14所述的电子设备，其特征在于，所述处理器还配置为执行：根据所述双摄像头模组中单摄像头的外参计算不同角度标定板上的特征点投影到摄像头坐标系并归一化，得到目标点坐标；The electronic device according to claim 14, wherein the processor is further configured to execute: calculating a feature point on a calibration plate at different angles according to external parameters of a single camera in the dual camera module and projecting it onto a camera coordinate system And normalize to get the coordinates of the target point;根据所述双摄像头模组中单摄像头的内参将检测到的不同角度标定图像上的特征点转换到摄像头坐标并归一化，得到畸变点坐标；及Converting the detected feature points on the calibration images at different angles to the camera coordinates and normalizing according to the internal parameters of the single camera in the dual camera module to obtain the coordinates of the distortion points; and根据不同角度的目标点坐标得到单摄像头的目标点点集，根据不同角度的畸变点坐标得到单摄像头的畸变点点集。The target point set of a single camera is obtained according to the target point coordinates of different angles, and the distortion point set of a single camera is obtained according to the distortion point coordinates of different angles.根据权利要求14所述的电子设备，其特征在于，所述双摄像头模组包括第一摄像头和第二摄像头，所述标定图像包括不同角度的第一标定图像和第二标定图像；所述处理器还配置为执行：根据所述第一摄像头的内参和外参、第一标定图像的特征点得到第一摄像头所对应不同角度的第一目标点点集和第一畸变点点集，采用分块曲面函数对所述第一目标点点集和第一畸变点点集进行拟合得到第一拟合系数，将所述第一拟合系数作为第一摄像头的畸变系数；及The electronic device according to claim 14, wherein the dual camera module includes a first camera and a second camera, and the calibration image includes a first calibration image and a second calibration image at different angles; the processing The device is further configured to execute: obtaining a first target point point set and a first distortion point point set of different angles corresponding to the first camera according to the internal and external parameters of the first camera and feature points of the first calibration image, and adopting a block surface The function fits the first target point point set and the first distortion point point set to obtain a first fitting coefficient, and the first fitting coefficient is used as a distortion coefficient of the first camera; and根据所述第二摄像头的内参和外参、第二标定图像的特征点得到第二摄像头所对应不同角度的第二目标点点集和第二畸变点点集，采用分块曲面函数对所述第二目标点点集和第二畸变点点集进行拟合得到第二拟合系数，将所述第二拟合系数作为第二摄像头的畸变系数。According to the internal and external parameters of the second camera, and the feature points of the second calibration image, a second target point set and a second distortion point set of different angles corresponding to the second camera are obtained, and a block surface function is used for The target point set and the second distortion point set are fitted to obtain a second fitting coefficient, and the second fitting coefficient is used as a distortion coefficient of the second camera.根据权利要求14所述的电子设备，其特征在于，所述处理器还配置为执行：将目标点点集按照块区间进行划分，对划分后的各块的目标点点集与对应的畸变点点集采用分块曲面函数拟合得到各块的拟合系数，根据各块的拟合系数得到对应摄像头的畸变系数。The electronic device according to claim 14, wherein the processor is further configured to perform: dividing the target point point set according to the block interval, and adopting the target point point set and the corresponding distortion point point set of each divided block The fitting function of the block surface function is used to obtain the fitting coefficient of each block, and the distortion coefficient of the corresponding camera is obtained according to the fitting coefficient of each block.根据权利要求17所述的电子设备，其特征在于，所述块区间的大小相同或不同。The electronic device according to claim 17, wherein the sizes of the block intervals are the same or different.根据权利要求13所述的电子设备，其特征在于，所述处理器还用于以所述双摄像头模组中的第一摄像头和第二摄像头所拍摄的标定图像中的同名特征点的偏差为目标函数，将所述第一摄像头和第二摄像头所拍摄的标定图像中的同名特征点、所述单摄像头的内参和畸变系数作为所述目标函数的输入，将所述双摄像头模组的初始外参作为所述目标函数的初始参数，进行处理得到所述双摄像头模组的目标外参。The electronic device according to claim 13, wherein the processor is further configured to use a deviation of feature points of the same name in a calibration image captured by the first camera and the second camera in the dual camera module as The objective function uses the feature points of the same name in the calibration images captured by the first camera and the second camera, the internal parameters of the single camera, and the distortion coefficient as the input of the objective function, and uses the initial value of the dual camera module as an input. An external parameter is used as an initial parameter of the objective function, and is processed to obtain a target external parameter of the dual camera module.根据权利要求13所述的电子设备，其特征在于，所述标定图像为标定板的平面垂直于双摄像头模组光轴时所拍摄的图像，所述标定板覆盖所述双摄像头模组的视场角。The electronic device according to claim 13, wherein the calibration image is an image captured when a plane of the calibration plate is perpendicular to an optical axis of the dual camera module, and the calibration plate covers a video of the dual camera module. Field angle.根据权利要求13所述的电子设备，其特征在于，所述处理器还被配置为执行：对所述第一摄像头和第二摄像头进行单摄像头标定，得到所述第一摄像头的内参和外参、所述第二摄像头的内参和外参。The electronic device according to claim 13, wherein the processor is further configured to perform single-camera calibration on the first camera and the second camera to obtain internal and external parameters of the first camera Internal and external parameters of the second camera.根据权利要求21所述的电子设备，其特征在于，所述处理器还被配置为执行：采用分块曲面函数对所述第一摄像头和第二摄像头进行单摄像头标定，得到所述第一摄像头的内参和外参、所述第二摄像头的内参和外参。The electronic device according to claim 21, wherein the processor is further configured to perform a single-camera calibration on the first camera and the second camera using a block surface function to obtain the first camera Internal and external parameters of the second camera, and internal and external parameters of the second camera.根据权利要求3所述的电子设备，其特征在于，所述分块曲面函数为B样条函数、自由曲面函数、zernike多项式函数中一种。The electronic device according to claim 3, wherein the piecewise curved surface function is one of a B-spline function, a free-form surface function, and a zernike polynomial function.一种非易失性计算机可读存储介质，其上存储有计算机程序，其特征在于，所述计算机程序被处理器执行时实现以下操作：A non-transitory computer-readable storage medium having a computer program stored thereon is characterized in that, when the computer program is executed by a processor, the following operations are performed:通过双摄像头模组拍摄不同角度的标定板，得到不同角度的标定图像；The calibration plates of different angles are taken by the dual camera module to obtain the calibration images of different angles;检测所述标定图像中的特征点；Detecting feature points in the calibration image;根据所述双摄像头模组中单摄像头的内参和外参及对应的标定图像中的特征点采用分块曲面函数拟合得到所述单摄像头的畸变系数，所述分块曲面函数的拟合结果连续；According to the internal and external parameters of a single camera in the dual camera module and the feature points in the corresponding calibration image, a block surface function is used to obtain the distortion coefficient of the single camera, and the fitting result of the block surface function is obtained. continuous;根据所述双摄像头模组中单摄像头的外参确定所述双摄像头模组的初始外参；及Determining the initial external parameters of the dual camera module according to the external parameters of the single camera in the dual camera module; and根据所述双摄像头模组的初始外参、单摄像头的畸变系数和单摄像头的内参进行处理得到所述双摄像头模组的目标外参。The target external parameters of the dual camera module are obtained by processing according to the initial external parameters of the dual camera module, the distortion coefficient of the single camera, and the internal parameters of the single camera.根据权利要求24所述的非易失性计算机可读存储介质，其特征在于，所述计算机程序被处理器执行时还实现以下操作：The non-transitory computer-readable storage medium according to claim 24, wherein the computer program further implements the following operations when executed by a processor:根据所述双摄像头模组中单摄像头的内参和外参、标定图像中的特征点得到单摄像头不同角度的目标点点集和畸变点点集，采用分块曲面函数对所述目标点点集和畸变点点集进行拟合得到拟合系数，将所述拟合系数作为对应摄像头的畸变系数。According to the internal and external parameters of the single camera in the dual camera module, and the feature points in the calibration image, a target point set and a distortion point set of different angles of the single camera are obtained, and a block surface function is adopted to the target point set and the distortion point. A set is fitted to obtain a fitting coefficient, and the fitting coefficient is used as a distortion coefficient of a corresponding camera.根据权利要求25所述的非易失性计算机可读存储介质，其特征在于，所述计算机程序被处理器执行时还实现以下操作：The non-volatile computer-readable storage medium according to claim 25, wherein the computer program further implements the following operations when executed by a processor:根据所述双摄像头模组中单摄像头的外参计算不同角度标定板上的特征点投影到摄像头坐标系并归一化，得到目标点坐标；Calculating the feature points on the calibration plate at different angles according to the external parameters of the single camera in the dual camera module and projecting them onto the camera coordinate system and normalizing to obtain the target point coordinates;根据所述双摄像头模组中单摄像头的内参将检测到的不同角度标定图像上的特征点转换到摄像头坐标并归一化，得到畸变点坐标；及Converting the detected feature points on the calibration images at different angles to the camera coordinates and normalizing according to the internal parameters of the single camera in the dual camera module to obtain the coordinates of the distortion points; and根据不同角度的目标点坐标得到单摄像头的目标点点集，根据不同角度的畸变点坐标得到单摄像头的畸变点点集。The target point set of a single camera is obtained according to the target point coordinates of different angles, and the distortion point set of a single camera is obtained according to the distortion point coordinates of different angles.根据权利要求25所述的非易失性计算机可读存储介质，其特征在于，所述双摄像头模组包括第一摄像头和第二摄像头，所述标定图像包括不同角度的第一标定图像和第二标定图像；The non-transitory computer-readable storage medium according to claim 25, wherein the dual camera module includes a first camera and a second camera, and the calibration image includes a first calibration image and a first calibration image at different angles. Two calibration images;所述计算机程序被处理器执行时还实现以下操作：根据所述第一摄像头的内参和外参、第一标定图像的特征点得到第一摄像头所对应不同角度的第一目标点点集和第一畸变点点集，采用分块曲面函数对所述第一目标点点集和第一畸变点点集进行拟合得到第一拟合系数，将所述第一拟合系数作为第一摄像头的畸变系数；及When the computer program is executed by the processor, the following operations are further implemented: according to the internal and external parameters of the first camera, and the feature points of the first calibration image, the first target point set and the first set of different angles corresponding to the first camera are obtained. A distortion point point set, using a piecewise curved surface function to fit the first target point point set and the first distortion point point set to obtain a first fitting coefficient, and using the first fitting coefficient as a distortion coefficient of the first camera; and根据所述第二摄像头的内参和外参、第二标定图像的特征点得到第二摄像头所对应不同角度的第二目标点点集和第二畸变点点集，采用分块曲面函数对所述第二目标点点集和第二畸变点点集进行拟合得到第二拟合系数，将所述第二拟合系数作为第二摄像头的畸变系数。According to the internal and external parameters of the second camera, and the feature points of the second calibration image, a second target point set and a second distortion point set of different angles corresponding to the second camera are obtained, and a block surface function is used for the second The target point set and the second distortion point set are fitted to obtain a second fitting coefficient, and the second fitting coefficient is used as a distortion coefficient of the second camera.根据权利要求25所述的非易失性计算机可读存储介质，其特征在于，所述计算机程序被处理器执行时还实现以下操作：将目标点点集按照块区间进行划分，对划分后的各块的目标点点集与对应的畸变点点集采用分块曲面函数拟合得到各块的拟合系数，根据各块的拟合系数得到对应摄像头的畸变系数。The non-transitory computer-readable storage medium according to claim 25, wherein when the computer program is executed by a processor, the following operations are further implemented: the target point set is divided according to the block interval, and the divided each The target point point set of the block and the corresponding distortion point point set are fitted with a piecewise curved surface function to obtain the fitting coefficient of each block, and the distortion coefficient of the corresponding camera is obtained according to the fitting coefficient of each block.根据权利要求28所述的非易失性计算机可读存储介质，其特征在于，所述块区间的大小相同或不同。The non-transitory computer-readable storage medium according to claim 28, wherein the sizes of the block intervals are the same or different.根据权利要求24所述的非易失性计算机可读存储介质，其特征在于，所述计算机程序被处理器执行时还实现以下操作：The non-transitory computer-readable storage medium according to claim 24, wherein the computer program further implements the following operations when executed by a processor:以所述双摄像头模组中的第一摄像头和第二摄像头所拍摄的标定图像中的同名特征点的偏差为目标函数，将所述第一摄像头和第二摄像头所拍摄的标定图像中的同名特征点、所述单摄像头的内参和畸变系数作为所述目标函数的输入，将所述双摄像头模组的初始外参作为所述目标函数的初始参数，进行处理得到所述双摄像头模组的目标外参。The deviation of feature points of the same name in the calibration images captured by the first camera and the second camera in the dual camera module is used as an objective function, and the same names in the calibration images captured by the first camera and the second camera are used Feature points, internal parameters and distortion coefficients of the single camera are used as inputs to the objective function, and initial external parameters of the dual camera module are used as initial parameters of the objective function, and processed to obtain the Target external reference.根据权利要求24所述的非易失性计算机可读存储介质，其特征在于，所述标定图像为标定板的平面垂直于双摄像头模组光轴时所拍摄的图像，所述标定板覆盖所述双摄像头模组的视场角。The non-transitory computer-readable storage medium according to claim 24, wherein the calibration image is an image captured when a plane of the calibration plate is perpendicular to the optical axis of the dual camera module, and the calibration plate covers the calibration plate. The field of view of the dual camera module is described.根据权利要求24所述的非易失性计算机可读存储介质，其特征在于，所述计算机程序被处理器执行时还实现以下操作：The non-transitory computer-readable storage medium according to claim 24, wherein the computer program further implements the following operations when executed by a processor:对所述第一摄像头和第二摄像头进行单摄像头标定，得到所述第一摄像头的内参和外参、所述第二摄像头的内参和外参。Single camera calibration is performed on the first camera and the second camera to obtain internal and external parameters of the first camera and internal and external parameters of the second camera.根据权利要求32所述的非易失性计算机可读存储介质，其特征在于，所述计算机程序被处理器执行时还实现以下操作：采用分块曲面函数对所述第一摄像头和第二摄像头进行单摄像头标定，得到所述第一摄像头的内参和外参、所述第二摄像头的内参和外参。The non-transitory computer-readable storage medium of claim 32, wherein when the computer program is executed by a processor, the following operation is further implemented: using a piecewise curved surface function for the first camera and the second camera Perform single-camera calibration to obtain internal and external parameters of the first camera and internal and external parameters of the second camera.根据权利要求24所述的非易失性计算机可读存储介质，其特征在于，所述分块曲面函数为B样条函数、自由曲面函数、zernike多项式函数中一种。The non-transitory computer-readable storage medium according to claim 24, wherein the piecewise curved surface function is one of a B-spline function, a free-form surface function, and a zernike polynomial function.

Images