技术领域Technical Field
本发明实施例涉及激光雷达标定领域，尤其涉及一种激光雷达标定方法、装置、电子设备及存储介质。The embodiments of the present invention relate to the field of laser radar calibration, and in particular to a laser radar calibration method, device, electronic device and storage medium.
背景技术Background technique
随着人工智能技术的快速发展，能够替代人类工作的自动化设备受到越来越多的关注，如无人车、无人机、机械手等等。这些自动化设备往往配置多种传感器以感知周围环境信息进行作业，在此基础上，由于激光雷达和摄像头的感知信息可互相补充，它们是自动化设备上常见的传感器。With the rapid development of artificial intelligence technology, automated equipment that can replace human work has received more and more attention, such as unmanned vehicles, drones, manipulators, etc. These automated equipment are often equipped with multiple sensors to sense the surrounding environment information for operation. On this basis, since the perception information of laser radar and camera can complement each other, they are common sensors on automated equipment.
对于同时配置有激光雷达和摄像头的自动化设备而言，二者的标定是两个传感器数据进行融合的前提。其中，激光雷达到摄像头的标定工作是确定一个三维旋转矩阵和平移向量，它们可以表示激光雷达坐标系到相机坐标系的相互转换关系，根据该相互转换关系可以将激光雷达坐标系上的点云数据投影到相机坐标系上，由此实现了点云数据和图像数据的融合。For automated equipment equipped with both a laser radar and a camera, calibration of the two is a prerequisite for fusing the data from the two sensors. The calibration of the laser radar to the camera is to determine a three-dimensional rotation matrix and a translation vector, which can represent the mutual conversion relationship from the laser radar coordinate system to the camera coordinate system. Based on this mutual conversion relationship, the point cloud data on the laser radar coordinate system can be projected onto the camera coordinate system, thereby achieving the fusion of point cloud data and image data.
目前常用的激光雷达到摄像头的标定工作主要通过具有明显特征的标定物完成，其从点云数据和图像数据中分别提取与该明显特征相应的特征点以组成特征点对，进而根据各特征点对进行雷达坐标系和相机坐标系的标定。Currently, the commonly used laser radar to camera calibration work is mainly completed through calibration objects with obvious features. The feature points corresponding to the obvious features are extracted from the point cloud data and image data to form feature point pairs, and then the radar coordinate system and camera coordinate system are calibrated according to each feature point pair.
在实现本发明的过程中，发明人发现现有技术中存在以下技术问题：现有的从点云数据中提取特征点的提取方案对标定物的尺寸和摆放位置有较大限制，这给标定工作的便利性带来了较大干扰。In the process of realizing the present invention, the inventors found that the following technical problems exist in the prior art: the existing scheme for extracting feature points from point cloud data has great restrictions on the size and placement of the calibration object, which greatly interferes with the convenience of the calibration work.
发明内容Summary of the invention
本发明实施例提供了一种激光雷达标定方法、装置、电子设备及存储介质，解决了从点云数据中提取特征点时对标定物的尺寸和摆放位置限制较大的问题。The embodiments of the present invention provide a laser radar calibration method, device, electronic device and storage medium, which solve the problem of large restrictions on the size and placement of the calibration object when extracting feature points from point cloud data.
第一方面，本发明实施例提供了一种激光雷达标定方法，可以包括：In a first aspect, an embodiment of the present invention provides a laser radar calibration method, which may include:
分别获取在同一时间采集到的包含有标定物的点云数据和图像数据，其中，标定物是多边形，且多边形中的每条边与至少两条点云线相交，点云线包括多个点云数据；从各点云数据中筛选出属于标定物上的标定数据，并根据各标定数据分别确定每条边在雷达坐标系下的边坐标；根据各边坐标分别确定标定物中每个顶点在雷达坐标系下的顶点坐标，并根据各顶点坐标、以及标定物中的标定点在标定物中的相对位置，确定标定点在雷达坐标系下的雷达坐标；将标定点在相机坐标系下的相机坐标、及雷达坐标作为一组特征点对，基于多组特征点对进行相机坐标系和雷达坐标系的标定。The point cloud data and image data containing the calibration object collected at the same time are respectively acquired, wherein the calibration object is a polygon, and each edge in the polygon intersects with at least two point cloud lines, and the point cloud line includes multiple point cloud data; the calibration data belonging to the calibration object is filtered out from each point cloud data, and the edge coordinates of each edge in the radar coordinate system are respectively determined according to each calibration data; the vertex coordinates of each vertex in the calibration object in the radar coordinate system are respectively determined according to each edge coordinate, and the radar coordinates of the calibration point in the radar coordinate system are determined according to each vertex coordinate and the relative position of the calibration point in the calibration object; the camera coordinates and the radar coordinates of the calibration point in the camera coordinate system are taken as a group of feature point pairs, and the camera coordinate system and the radar coordinate system are calibrated based on multiple groups of feature point pairs.
可选的，从各点云数据中筛选出属于标定物上的标定数据，可以包括：Optionally, the calibration data belonging to the calibration object is filtered out from each point cloud data, which may include:
从各点云数据中筛选出属于同一平面上的平面数据，并从各平面数据构成的平面集合中筛选出端点数据；将各端点数据构成的对角线的第一长度、与标定物相应的对角线的第二长度进行对比，根据对比结果从各点云数据中筛选出属于标定物上的标定数据。Filter out plane data belonging to the same plane from each point cloud data, and filter out endpoint data from the plane set formed by each plane data; compare the first length of the diagonal formed by each endpoint data with the second length of the diagonal corresponding to the calibration object, and filter out the calibration data belonging to the calibration object from each point cloud data based on the comparison result.
可选的，根据对比结果从各点云数据中筛选出属于标定物上的标定数据，可以包括：Optionally, the calibration data belonging to the calibration object is filtered out from each point cloud data according to the comparison result, which may include:
若根据对比结果确定出同一平面不是标定物所在的平面，则从各点云数据中剔除平面数据，并根据剔除结果更新点云数据；重复执行从各点云数据中筛选出属于同一平面上的平面数据的步骤，直至同一平面是标定物所在的平面；将平面数据作为属于标定物上的标定数据。If it is determined based on the comparison result that the same plane is not the plane where the calibration object is located, the plane data is eliminated from each point cloud data, and the point cloud data is updated according to the elimination result; the steps of filtering out the plane data belonging to the same plane from each point cloud data are repeated until the same plane is the plane where the calibration object is located; and the plane data is used as the calibration data belonging to the calibration object.
可选的，根据各标定数据分别确定每条边在雷达坐标系下的边坐标，包括：Optionally, the edge coordinates of each edge in the radar coordinate system are determined according to each calibration data, including:
获取各标定数据构成的多条标定线，确定每条标定线在扫描方向上的边缘数据，并根据各边缘数据分别确定每条边在雷达坐标系下的边坐标。A plurality of calibration lines formed by each calibration data is obtained, edge data of each calibration line in the scanning direction is determined, and edge coordinates of each edge in the radar coordinate system are respectively determined according to each edge data.
可选的，标定物包括四边形标定物，四边形标定物内设置有与至少两条边相切的曲边形，标定点是曲边形的中心点。Optionally, the calibration object includes a quadrilateral calibration object, a curved shape tangent to at least two sides is arranged in the quadrilateral calibration object, and the calibration point is the center point of the curved shape.
可选，多边形包括方形，曲边形包括圆形，圆形与顶点相连的两条边相切；Optionally, the polygon includes a square, the curved-edge polygon includes a circle, and the two sides connecting the vertices of the circle are tangent;
相应的，根据各顶点坐标、以及标定物中的标定点在标定物中的相对位置，确定标定点在雷达坐标系下的雷达坐标，可以包括：Accordingly, determining the radar coordinates of the calibration point in the radar coordinate system according to the coordinates of each vertex and the relative position of the calibration point in the calibration object may include:
各顶点包括上顶点、左顶点和右顶点，通过如下公式确定出与上顶点相邻的标定点在雷达坐标系下的雷达坐标Ptop：Each vertex includes an upper vertex, a left vertex, and a right vertex. The radar coordinates Ptop of the calibration point adjacent to the upper vertex in the radar coordinate system are determined by the following formula:
其中，Ctop是上顶点在雷达坐标系下的上顶点坐标，Cleft是左顶点在雷达坐标系下的左顶点坐标，Cright是右顶点在雷达坐标系下的右顶点坐标，w是各边中第一边的长度，h是各边中与第一边相垂直的第二边的长度，r是圆形的半径。Among them, Ctop is the coordinate of the upper vertex in the radar coordinate system, Cleft is the coordinate of the left vertex in the radar coordinate system, Cright is the coordinate of the right vertex in the radar coordinate system, w is the length of the first side of each side, h is the length of the second side perpendicular to the first side of each side, and r is the radius of the circle.
可选的，基于多组特征点对进行相机坐标系和雷达坐标系的标定，可包括：Optionally, calibrating the camera coordinate system and the radar coordinate system based on multiple sets of feature point pairs may include:
基于多组特征点对确定相机坐标系到雷达坐标系的坐标变化关系，根据坐标变化关系对相机坐标系和雷达坐标系进行标定。The coordinate change relationship from the camera coordinate system to the radar coordinate system is determined based on multiple sets of feature point pairs, and the camera coordinate system and the radar coordinate system are calibrated according to the coordinate change relationship.
第二方面，本发明实施例还提供了一种激光雷达标定装置，可以包括：In a second aspect, an embodiment of the present invention further provides a laser radar calibration device, which may include:
数据获取模块，用于分别获取在同一时间采集到的包含有标定物的点云数据和图像数据，其中，标定物是多边形，且多边形中的每条边与至少两条点云线相交，点云线包括多个点云数据；A data acquisition module, used to respectively acquire point cloud data and image data containing a calibration object collected at the same time, wherein the calibration object is a polygon, and each edge of the polygon intersects with at least two point cloud lines, and the point cloud line includes a plurality of point cloud data;
边坐标确定模块，用于从各点云数据中筛选出属于标定物上的标定数据，并根据各标定数据分别确定每条边在雷达坐标系下的边坐标；The edge coordinate determination module is used to filter out the calibration data belonging to the calibration object from each point cloud data, and determine the edge coordinates of each edge in the radar coordinate system according to each calibration data;
雷达坐标确定模块，用于根据各边坐标分别确定标定物中每个顶点在雷达坐标系下的顶点坐标，并根据各顶点坐标、以及标定物中的标定点在标定物中的相对位置，确定标定点在雷达坐标系下的雷达坐标；A radar coordinate determination module is used to determine the vertex coordinates of each vertex in the calibration object in the radar coordinate system according to the coordinates of each edge, and determine the radar coordinates of the calibration point in the radar coordinate system according to the coordinates of each vertex and the relative position of the calibration point in the calibration object;
激光雷达标定模块，用于将标定点在相机坐标系下的相机坐标、及雷达坐标作为一组特征点对，基于多组特征点对进行相机坐标系和雷达坐标系的标定。The laser radar calibration module is used to use the camera coordinates and radar coordinates of the calibration point in the camera coordinate system as a set of feature point pairs, and calibrate the camera coordinate system and the radar coordinate system based on multiple sets of feature point pairs.
第三方面，本发明实施例还提供了一种电子设备，该电子设备可以包括：In a third aspect, an embodiment of the present invention further provides an electronic device, which may include:
一个或多个处理器；one or more processors;
存储器，用于存储一个或多个程序；A memory for storing one or more programs;
当一个或多个程序被一个或多个处理器执行，使得一个或多个处理器实现本发明任意实施例所提供的激光雷达标定方法。When one or more programs are executed by one or more processors, the one or more processors implement the laser radar calibration method provided by any embodiment of the present invention.
第四方面，本发明实施例还提供了一种计算机可读存储介质，其上存储有计算机程序，该计算机程序被处理器执行时实现本发明任意实施例所提供的激光雷达标定方法。In a fourth aspect, an embodiment of the present invention further provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the laser radar calibration method provided by any embodiment of the present invention.
本发明实施例的技术方案，通过分别获取在同一时间采集到的包含有多边形状的标定物的点云数据和图像数据，从各点云数据中筛选出属于标定物上的标定数据，且由于多边形中的每条边与至少两条点云线相交，这意味着根据各标定数据可以分别确定每条边在雷达坐标系下的边坐标；进而，根据各边坐标可以确定标定物中各顶点在该雷达坐标系下的顶点坐标，并根据各顶点坐标、及标定物中的标定点在标定物中的相对位置，确定标定点在该雷达坐标系下的雷达坐标；由此，可以将标定点在相机坐标系下的相机坐标、及雷达坐标作为一组特征点对，并基于多组特征点对进行相机坐标系和雷达坐标系间的标定。上述技术方案，通过多边形中各条边的边坐标可以确定出特征点的雷达坐标，这意味着多边形中的每条边与至少两条点云线相交时，即可从各点云数据中提取出特征点，这对标定物的尺寸和摆放位置的限制较小，由此提高了标定物在制作和使用方面的便利性，进而提高了标定工作的便利性。The technical solution of the embodiment of the present invention is to respectively obtain point cloud data and image data of a polygonal calibration object collected at the same time, and filter out calibration data belonging to the calibration object from each point cloud data. Since each edge in the polygon intersects with at least two point cloud lines, this means that the edge coordinates of each edge in the radar coordinate system can be determined respectively according to each calibration data; further, the vertex coordinates of each vertex in the calibration object in the radar coordinate system can be determined according to each edge coordinate, and the radar coordinates of the calibration point in the radar coordinate system can be determined according to each vertex coordinate and the relative position of the calibration point in the calibration object; thus, the camera coordinates and radar coordinates of the calibration point in the camera coordinate system can be taken as a set of feature point pairs, and calibration between the camera coordinate system and the radar coordinate system can be performed based on multiple sets of feature point pairs. The above technical solution can determine the radar coordinates of the feature points through the edge coordinates of each edge in the polygon, which means that when each edge in the polygon intersects with at least two point cloud lines, the feature points can be extracted from each point cloud data. This places less restrictions on the size and placement of the calibration object, thereby improving the convenience of the calibration object in production and use, and further improving the convenience of the calibration work.
附图说明BRIEF DESCRIPTION OF THE DRAWINGS
图1是现有技术中的标定物的示意图；FIG1 is a schematic diagram of a calibration object in the prior art;
图2是本发明各实施例中的一种激光雷达标定方法中标定物的示意图；FIG2 is a schematic diagram of a calibration object in a laser radar calibration method in various embodiments of the present invention;
图3是本发明实施例一中的一种激光雷达标定方法的流程图；FIG3 is a flow chart of a laser radar calibration method in Embodiment 1 of the present invention;
图4是本发明实施例一中的一种激光雷达标定方法中标定物的摆放示意图；FIG4 is a schematic diagram of the placement of calibration objects in a laser radar calibration method in Embodiment 1 of the present invention;
图5是本发明实施例一中的一种激光雷达标定方法中点云数据的示意图；5 is a schematic diagram of point cloud data in a laser radar calibration method in Embodiment 1 of the present invention;
图6是本发明实施例二中的一种激光雷达标定方法的流程图；FIG6 is a flow chart of a laser radar calibration method in Embodiment 2 of the present invention;
图7是本发明实施例三中的一种激光雷达标定方法的流程图；FIG7 is a flow chart of a laser radar calibration method in Embodiment 3 of the present invention;
图8是本发明实施例四中的一种激光雷达标定装置的结构框图；8 is a structural block diagram of a laser radar calibration device in Embodiment 4 of the present invention;
图9是本发明实施例五中的一种电子设备的结构示意图。FIG9 is a schematic diagram of the structure of an electronic device in Embodiment 5 of the present invention.
具体实施方式Detailed ways
下面结合附图和实施例对本发明作进一步详细说明。可以理解的是，此处所描述的具体实施例仅仅用于解释本发明，而非对本发明的限定。另外还需要说明的是，为了便于描述，附图中仅示出了与本发明相关的部分而非全部结构。The present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It is to be understood that the specific embodiments described herein are only used to explain the present invention, rather than to limit the present invention. It is also necessary to explain that, for ease of description, only parts related to the present invention, rather than all structures, are shown in the accompanying drawings.
在介绍本发明实施例之前，先对本发明实施例的应用场景进行示例性说明：目前常用的激光雷达到摄像头的标定工作中涉及到的标定物可以是如图1所示的带有圆洞的平板，该标定工作是根据各点云数据中深度显著变化的点云数据拟合出4个圆形，再根据拟合结果得到4个圆心的坐标，该坐标可以作为特征点对中的一个特征点。换言之，该标定工作需要保证每个圆与至少两条点云线相交，这是因为至少三个不共线的点云数据才可以拟合出一个圆形。Before introducing the embodiment of the present invention, the application scenario of the embodiment of the present invention is first exemplified: the calibration object involved in the currently commonly used laser radar to camera calibration work can be a flat plate with a circular hole as shown in Figure 1. The calibration work is to fit 4 circles based on the point cloud data with significant depth changes in each point cloud data, and then obtain the coordinates of the 4 circle centers based on the fitting results. The coordinates can be used as a feature point in the feature point pair. In other words, the calibration work needs to ensure that each circle intersects with at least two point cloud lines, because at least three non-collinear point cloud data can fit a circle.
需要说明的是，对于多线激光雷达而言，例如16线激光雷达，其对标定物进行扫描是以锥形束的方式向外发射多条高低不同的扫描线，这意味着当各条扫描线与激光雷达的距离越远时，彼此间的稀疏程度越大。此时，为保证每个圆与至少两条点云线相交，一种可选方案是，标定物与激光雷达间的摆放距离足够近，此时对标定物的尺寸没有具体限制，但对标定物的摆放位置存在限制；另一可选方案是，标定物的尺寸足够大，此时标定物可摆放在任意位置，而且考虑到摄像头的焦点通常在远处，摄像头和激光雷达摆放在同一处，这种可选方案有利于摄像头拍摄到标定物的清晰的图像数据，这是后续从图像数据中提取特征点的重要前提。由上可知，现有的从点云数据中提取特征点的提取方案对标定物的尺寸和摆放位置具有较大限制，这样的标定物需要特殊订制，这给标定工作的便利性带来了较大干扰。It should be noted that for a multi-line laser radar, such as a 16-line laser radar, it scans the calibration object by emitting multiple scanning lines of different heights in a cone beam manner, which means that the farther the distance between each scanning line and the laser radar, the greater the sparseness between them. At this time, in order to ensure that each circle intersects with at least two point cloud lines, one option is that the distance between the calibration object and the laser radar is close enough. At this time, there is no specific restriction on the size of the calibration object, but there is a restriction on the placement of the calibration object; another option is that the size of the calibration object is large enough. At this time, the calibration object can be placed at any position, and considering that the focus of the camera is usually far away, the camera and the laser radar are placed in the same place. This option is conducive to the camera capturing clear image data of the calibration object, which is an important prerequisite for the subsequent extraction of feature points from the image data. As can be seen from the above, the existing extraction scheme for extracting feature points from point cloud data has great restrictions on the size and placement of the calibration object. Such calibration objects need to be specially customized, which brings great interference to the convenience of calibration work.
为了解决上述问题，本发明各实施例提出的激光雷达标定方法只需标定物是多边形，且多边形中的每条边与至少两条点云线相交即可完成标定工作，这意味着其对标定物的摆放位置和尺寸均没有具体限制，例如，可以将标定物摆放在摄像头能够清晰对焦的坐标处，且标定物的尺寸无需足够大，也不需要对标定物进行挖洞，这样的标定物无需特殊订制，制作过程和使用过程的便利性较高，可以更好地应用于16线激光雷达中。示例性的，上述标定物的外围形状可以是三角形、方形(如正方形、长方形)、五边形、六边形等；在此基础上，可选的，该多边形内可以设置有曲边形，如圆形、椭圆形等，这样设置的好处在于，可以将曲边形中的某个点(如中心点、角点)作为标定物中的标定点，在此基础上，还可以将该标定点的坐标作为标定工作中的特征点。示例性的，以图2所示的标定物为例，该标定物的外围形状是长方形，其内设置有4个圆形，每个圆形与两条边相切，该两条边是与同一顶点相连的边，每个圆形的圆心均可以作为该标定物的标定点。上述标定物的制作过程较为简单，例如可以先打印出4个圆形，并将该4个圆形直接粘在一个方形木板的四个角落上；再如可直接在某方形木板上画4个圆形；等等，在此未做具体限定。In order to solve the above problems, the laser radar calibration method proposed in each embodiment of the present invention only requires that the calibration object is a polygon, and each edge in the polygon intersects with at least two point cloud lines to complete the calibration work, which means that there is no specific restriction on the placement and size of the calibration object. For example, the calibration object can be placed at the coordinates where the camera can focus clearly, and the size of the calibration object does not need to be large enough, and there is no need to dig a hole in the calibration object. Such a calibration object does not need to be specially customized, and the convenience of the production process and the use process is high, and it can be better applied to 16-line laser radars. Exemplarily, the outer shape of the above-mentioned calibration object can be a triangle, a square (such as a square, a rectangle), a pentagon, a hexagon, etc.; on this basis, optionally, a curved shape such as a circle, an ellipse, etc. can be set in the polygon. The advantage of such a setting is that a certain point in the curved shape (such as a center point, a corner point) can be used as a calibration point in the calibration object, and on this basis, the coordinates of the calibration point can also be used as feature points in the calibration work. For example, taking the calibration object shown in FIG2 as an example, the outer shape of the calibration object is a rectangle, and 4 circles are arranged inside it, each circle is tangent to two sides, and the two sides are sides connected to the same vertex, and the center of each circle can be used as the calibration point of the calibration object. The production process of the above calibration object is relatively simple. For example, 4 circles can be printed out first, and the 4 circles can be directly glued to the four corners of a square wooden board; for example, 4 circles can be directly drawn on a square wooden board; and so on, which are not specifically limited here.
实施例一Embodiment 1
图3是本发明实施例一中提供的一种激光雷达标定方法的流程图。本实施例可以适用于激光雷达标定的情况，尤其适用于在对标定物的摆放位置和尺寸没有限制的前提下，进行激光雷达标定的情况。该方法可以由本发明实施例提供的激光雷达标定装置来执行，该装置可以由软件和/或硬件的方式实现，该装置可以集成在电子设备上，该电子设备可以是各种用户终端或是服务器。FIG3 is a flow chart of a laser radar calibration method provided in Embodiment 1 of the present invention. This embodiment can be applicable to the case of laser radar calibration, and is particularly applicable to the case of laser radar calibration without restrictions on the placement and size of the calibration object. The method can be executed by the laser radar calibration device provided in the embodiment of the present invention, which can be implemented by software and/or hardware, and the device can be integrated in an electronic device, which can be various user terminals or servers.
参见图3，本发明实施例的方法具体包括如下步骤：Referring to FIG. 3 , the method of the embodiment of the present invention specifically includes the following steps:
S110、分别获取在同一时间采集到的包含有标定物的点云数据和图像数据，其中，标定物是多边形，且该多边形中的每条边与至少两条点云线相交，该点云线包括多个点云数据。S110, respectively acquiring point cloud data and image data containing a calibration object collected at the same time, wherein the calibration object is a polygon, and each edge of the polygon intersects with at least two point cloud lines, and the point cloud line includes a plurality of point cloud data.
其中，将标定物摆放在激光雷达传感器的扫描方向，控制激光雷达传感器对标定物进行扫描后得到包含有该标定物的点云数据，该激光雷达传感器可以是能够扫描到点云数据的传感器，如多线激光雷达；该激光雷达传感器的扫描结果可以是多条点云线，每条点云线包括多个点云数据，该多个点云数据可能包括隶属于该标定物上的点云数据，也可能包括隶属于该标定物以外的物体上的点云数据，且由于两点构成一条直线，在多边形中的每条边与至少两条点云线相交时，根据该边上的点云数据可以确定出该边在该激光雷达传感器对应的雷达坐标系下的坐标，这样的坐标可以称为边坐标。Among them, the calibration object is placed in the scanning direction of the laser radar sensor, and the laser radar sensor is controlled to scan the calibration object to obtain point cloud data containing the calibration object. The laser radar sensor can be a sensor that can scan point cloud data, such as a multi-line laser radar; the scanning result of the laser radar sensor can be multiple point cloud lines, each point cloud line includes multiple point cloud data, and the multiple point cloud data may include point cloud data belonging to the calibration object, and may also include point cloud data belonging to objects other than the calibration object. Since two points form a straight line, when each edge in the polygon intersects with at least two point cloud lines, the coordinates of the edge in the radar coordinate system corresponding to the laser radar sensor can be determined according to the point cloud data on the edge, and such coordinates can be called edge coordinates.
相应的，图像数据的获取过程类似，可选的，标定物的摆放位置也是相机传感器的拍摄方向，控制该相机传感器对标定物进行拍摄后得到包含有该标定物的图像数据，该相机传感器可以是能够拍摄到图像数据的传感器，如摄像头、数码相机等，该相机传感器对应的坐标系可以是相机坐标系。需要说明的是，每帧图像数据和每帧点云数据在时间上具有一一对应关系，即，每帧图像数据均对应有一帧与其在同一时间采集到的点云数据，反之亦然成立。Correspondingly, the process of acquiring image data is similar. Optionally, the placement position of the calibration object is also the shooting direction of the camera sensor. After controlling the camera sensor to shoot the calibration object, image data containing the calibration object is obtained. The camera sensor can be a sensor that can capture image data, such as a camera, a digital camera, etc. The coordinate system corresponding to the camera sensor can be a camera coordinate system. It should be noted that each frame of image data and each frame of point cloud data have a one-to-one correspondence in time, that is, each frame of image data corresponds to a frame of point cloud data collected at the same time, and vice versa.
考虑到本发明实施例可能涉及到的应用场景，可选的，以图4为例，上述各点云数据的获取过程可以是：将标定物倾斜一定角度固定起来，如将标定物倾斜45°后用细脚支架支撑、或者用细线吊起来，倾斜后的标定物可以避免因其中的某条边与点云线平行而无法相交的情况出现。标定物的周围尽可能不要出现其余物体，这些其余物体可能会对从点云数据中提取出特征点造成干扰。小幅度移动多线激光雷达，并在不同方位采集点云数据。在移动过程中，可以尽可能保证标定物中的每条边与多线激光雷达发射的至少两条扫描线相交，这是后续实现每条边与至少两条点云线相交的重要前提。在此基础上，可选的，为保证后续的边坐标的确定精度，可以将标定物的尺寸制作的大一些，以便每条边可以与更多的点云线相交，由此可以避免出现因某个点云数据是噪声点而对边坐标的确定精度造成影响的情况。除此之外，还可以尽可能保证标定物中的各个顶点出现在每帧点云数据中，且不同帧的点云数据中标定物的坐标具有差异性。例如，多线激光雷达在某一时间扫描得到的一帧点云数据如图5所示，该帧点云数据包括多条点云线，属于同一颜色的每条点云线上的各点云数据是基于同一扫描线扫描到的数据，每个点云数据可同时记录其坐标和隶属于哪条扫描线。图5中的各点云线构成的方形区域(即，虚线区域)是标定物所在的区域，其余的点云线可以是扫描线打在标定物之外的墙上后反射回来的线条。需要说明的是，在实际应用中，图5所示的点云线可以是彩色线条，此处已将其转化为灰度线条。当然，图像数据的采集过程与点云数据类似，例如小幅度移动相机传感器，并在移动过程中从不同方位对标定物进行拍摄。Considering the application scenarios that may be involved in the embodiments of the present invention, optionally, taking FIG. 4 as an example, the acquisition process of each point cloud data mentioned above may be: the calibration object is tilted at a certain angle and fixed, such as tilting the calibration object at 45° and then supporting it with a thin leg bracket, or hanging it with a thin wire. The tilted calibration object can avoid the situation where one of its edges is parallel to the point cloud line and cannot intersect. There should be no other objects around the calibration object as much as possible, as these other objects may interfere with the extraction of feature points from the point cloud data. Move the multi-line laser radar in a small amplitude and collect point cloud data in different directions. During the movement, it can be ensured as much as possible that each edge in the calibration object intersects with at least two scanning lines emitted by the multi-line laser radar, which is an important prerequisite for the subsequent realization of each edge intersecting with at least two point cloud lines. On this basis, optionally, in order to ensure the accuracy of the subsequent determination of the edge coordinates, the size of the calibration object can be made larger so that each edge can intersect with more point cloud lines, thereby avoiding the situation where the accuracy of the determination of the edge coordinates is affected by a certain point cloud data being a noise point. In addition, it is also possible to ensure that each vertex in the calibration object appears in each frame of point cloud data as much as possible, and the coordinates of the calibration object in different frames of point cloud data are different. For example, a frame of point cloud data scanned by a multi-line laser radar at a certain time is shown in Figure 5. The frame of point cloud data includes multiple point cloud lines. Each point cloud data on each point cloud line of the same color is based on the data scanned by the same scan line. Each point cloud data can simultaneously record its coordinates and which scan line it belongs to. The square area (i.e., the dotted area) formed by each point cloud line in Figure 5 is the area where the calibration object is located, and the remaining point cloud lines can be the lines reflected back after the scan line hits the wall outside the calibration object. It should be noted that in actual applications, the point cloud lines shown in Figure 5 can be colored lines, which have been converted into grayscale lines here. Of course, the acquisition process of image data is similar to that of point cloud data, such as moving the camera sensor slightly and shooting the calibration object from different directions during the movement.
S120、从各点云数据中筛选出属于标定物上的标定数据，并根据各标定数据分别确定每条边在雷达坐标系下的边坐标。S120, filtering out calibration data belonging to the calibration object from each point cloud data, and determining the edge coordinates of each edge in the radar coordinate system according to each calibration data.
其中，有些点云数据是扫描线打在标定物后反射回来的数据，而有些点云数据是扫描线打在标定物之外的其余物体上反射回来的数据，该其余物体可以是设置于标定物一侧的墙体。因此，为了提高后续根据各点云数据确定特征点的精度，可以先从各点云数据中筛选出属于标定物上的标定数据，该标定数据是那些打在标定物上反射回来的数据。当然，为了提高各标定数据的确定精度，可选的，先根据标定物和激光雷达传感器间的相对位置确定标定物在每帧点云数据中的目标范围，再从隶属于该目标范围中的各点云数据中筛选出标定数据。Among them, some point cloud data are data reflected after the scan line hits the calibration object, and some point cloud data are data reflected after the scan line hits the remaining objects other than the calibration object. The remaining objects can be walls set on one side of the calibration object. Therefore, in order to improve the accuracy of subsequent determination of feature points based on each point cloud data, the calibration data belonging to the calibration object can be first filtered out from each point cloud data. The calibration data is the data reflected from the calibration object. Of course, in order to improve the determination accuracy of each calibration data, it is optional to first determine the target range of the calibration object in each frame of point cloud data based on the relative position between the calibration object and the lidar sensor, and then filter out the calibration data from each point cloud data belonging to the target range.
进一步，由于各标定数据均是标定物上的点云数据，那么根据各标定数据可以分别确定每条边在雷达坐标系下的边坐标，如获取各标定数据构成的多条标定线，该标定线是上文所述的点云线，且该标定线上的每个点云数据均是标定数据；确定每条标定线在扫描方向上的边缘数据，扫描方向是激光雷达传感器进行扫描时的方向，如水平方向、垂直方向等，该边缘数据是标定线在扫描方向上的最边缘的标定数据，例如扫描方向是水平方向时，该边缘数据可以是标定线上的最左边的标定数据和最右边的标定数据，再如扫描方向是垂直方向时，该边缘数据可以是标定线上的最上边的标定数据和最下边的标定数据，Furthermore, since each calibration data is point cloud data on the calibration object, the edge coordinates of each edge in the radar coordinate system can be determined separately according to each calibration data, such as obtaining multiple calibration lines formed by each calibration data, the calibration line is the point cloud line mentioned above, and each point cloud data on the calibration line is calibration data; the edge data of each calibration line in the scanning direction is determined, the scanning direction is the direction when the laser radar sensor scans, such as the horizontal direction, the vertical direction, etc., the edge data is the calibration data of the edge of the calibration line in the scanning direction, for example, when the scanning direction is the horizontal direction, the edge data can be the leftmost calibration data and the rightmost calibration data on the calibration line, for example, when the scanning direction is the vertical direction, the edge data can be the uppermost calibration data and the lowermost calibration data on the calibration line,
再进一步，根据各个边缘数据分别确定每条边在雷达坐标系下的边坐标，例如根据隶属于一侧的各边缘数据确定在该侧的边在雷达坐标系下的边坐标，示例性的，以图5中标定物所在的虚线区域为例，各边缘数据包括每条标定线上的最左边的标定数据Pleft和最右边的标定数据Pright，在各Pleft中，基于随机抽样一致算法(Random Sample Consensus，RANSAC)拟合左侧长边(即，位于左上角的边)的直线方程，并根据各Pleft中除该左侧长边上的边缘数据以外的各边缘数据拟合左侧短边(即，位于左下角的边)的直线方程，Pright的处理过程类似，依次得到右侧长边和右侧短边的直线方程，这些直线方程可以表示出每条边在雷达坐标系下的边坐标。Furthermore, the edge coordinates of each edge in the radar coordinate system are determined according to each edge data. For example, the edge coordinates of the edge on one side in the radar coordinate system are determined according to each edge data belonging to the side. Exemplarily, taking the dotted area where the calibration object is located in Figure 5 as an example, each edge data includes the leftmost calibration data Pleft and the rightmost calibration data Pright on each calibration line. In each Pleft , the straight line equation of the left long side (i.e., the side located at the upper left corner) is fitted based on the Random Sample Consensus (RANSAC) algorithm, and the straight line equation of the left short side (i.e., the side located at the lower left corner) is fitted according to each edge data in each Pleft except the edge data on the left long side. The processing process of Pright is similar, and the straight line equations of the right long side and the right short side are obtained in turn. These straight line equations can represent the edge coordinates of each edge in the radar coordinate system.
S130、根据各边坐标分别确定标定物中每个顶点在雷达坐标系下的顶点坐标，并根据各顶点坐标、以及标定物中的标定点在标定物中的相对位置，确定标定点在雷达坐标系下的雷达坐标。S130, determining the vertex coordinates of each vertex in the calibration object in the radar coordinate system according to the coordinates of each edge, and determining the radar coordinates of the calibration point in the radar coordinate system according to the coordinates of each vertex and the relative position of the calibration point in the calibration object.
其中，由于各条扫描线较为稀疏，在基于各扫描线对标定物进行扫描时，可能出现并未扫描到该标定物的顶点的情况，该顶点是多边形的顶点，即上文所述的边缘数据不一定是对顶点进行扫描后得到的点云数据，这意味着根据各点云数据可能无法直接得到顶点在雷达坐标系下的顶点坐标。此时，可以先拟合出每条边坐标，再根据各边坐标分别确定标定物中的各顶点坐标，如将根据各条边的边坐标确定出的各条边的交点所在的坐标作为相应的顶点坐标。Among them, since each scanning line is relatively sparse, when scanning the calibration object based on each scanning line, it may happen that the vertex of the calibration object is not scanned. The vertex is the vertex of the polygon, that is, the edge data mentioned above is not necessarily the point cloud data obtained after scanning the vertex, which means that the vertex coordinates of the vertex in the radar coordinate system may not be directly obtained according to each point cloud data. At this time, each edge coordinate can be fitted first, and then the coordinates of each vertex in the calibration object can be determined according to each edge coordinate, such as using the coordinates of the intersection of each edge determined according to the edge coordinates of each edge as the corresponding vertex coordinates.
进一步，标定物中设置有标定点，其是预先设定的用于进行标定的特征点，如标定物的中心点、设置于标定物内的曲边形的中心点等等，因此该标定点在标定物中的相对位置是可以预先得到的信息，此时根据各顶点坐标、及标定物中标定点在标定物中的相对位置，可确定该标定点在雷达坐标系下的雷达坐标，其可以作为标定工作中涉及到的特征点对中的一个特征点。Furthermore, calibration points are provided in the calibration object, which are pre-set feature points for calibration, such as the center point of the calibration object, the center point of a curved shape provided in the calibration object, and the like. Therefore, the relative position of the calibration point in the calibration object is information that can be obtained in advance. At this time, according to the coordinates of each vertex and the relative position of the calibration point in the calibration object, the radar coordinates of the calibration point in the radar coordinate system can be determined, and it can be used as one of the feature point pairs involved in the calibration work.
S140、将标定点在相机坐标系下的相机坐标、及雷达坐标作为一组特征点对，基于多组特征点对进行相机坐标系和雷达坐标系的标定。S140: taking the camera coordinates and the radar coordinates of the calibration points in the camera coordinate system as a set of feature point pairs, and calibrating the camera coordinate system and the radar coordinate system based on the multiple sets of feature point pairs.
其中，相机坐标是标定点在相机坐标系下的坐标，其可以是在标定工作中涉及到的特征点对中的另一个特征点。根据图像数据确定标定点在相机坐标系下的相机坐标有多种实现方案，如根据相机传感器的畸变参数对图像数据去畸变；利于canny、sobe等边缘提取算子从图像数据中提取边缘特征图；最后利用hough变换对边缘特征图中的线条进行检测，并根据检测结果确定标定点的相机坐标，示例性，如标定点是设置于标定物内圆形的圆心，则可利用hough变换对边缘特征图中的圆形进行检测，并根据检测出的圆形确定圆心的坐标。实际应用中，可选的，在制作标定物时，可以将圆形和多边形设置出差异显著的颜色，这可以提高hough变换对圆形的检测精度。Among them, the camera coordinates are the coordinates of the calibration point in the camera coordinate system, which can be another feature point in the feature point pair involved in the calibration work. There are many implementation schemes for determining the camera coordinates of the calibration point in the camera coordinate system according to the image data, such as dedistorting the image data according to the distortion parameters of the camera sensor; facilitating edge extraction operators such as Canny and Sobe to extract edge feature maps from the image data; finally, using Hough transform to detect the lines in the edge feature map, and determining the camera coordinates of the calibration point according to the detection results. For example, if the calibration point is set at the center of a circle in the calibration object, the circle in the edge feature map can be detected using Hough transform, and the coordinates of the center of the circle can be determined based on the detected circle. In practical applications, optionally, when making the calibration object, the circle and the polygon can be set to have significantly different colors, which can improve the detection accuracy of the circle by Hough transform.
需要说明的是，在实际应用中，可选的，在同一时间采集到的图像数据和点云数据可以作为一组数据，以每组数据为单位执行上述相机坐标和雷达坐标的确定步骤。而且，二者的确定过程没有严格的先后顺序，二者可以同时确定或是先后确定，在此未做具体限定。另外，若某组数据中的雷达坐标和/或相机坐标提取失败，则可以跳过该组数据，并从下一组数据中继续提取。It should be noted that, in practical applications, the image data and point cloud data collected at the same time can be used as a group of data, and the above-mentioned camera coordinate and radar coordinate determination steps are performed in units of each group of data. Moreover, there is no strict order in the determination process of the two. The two can be determined simultaneously or successively, and no specific limitation is made here. In addition, if the extraction of radar coordinates and/or camera coordinates in a certain group of data fails, the group of data can be skipped and the extraction can be continued from the next group of data.
由此，将雷达坐标和相机坐标作为一组特征点对，即，每个标定点在雷达坐标系下的三维特征点(即，雷达坐标qi)和在相机坐标系下的二维特征点(即，相机坐标pi)是一组特征点对(pi，qi)，在实际应用中，标定物上的标定点的数量可以是至少三个，即每组数据中特征点对的数量可以是至少三个。在获取到多组特征点对后，可以根据它们对相机坐标系和雷达坐标系进行标定。Therefore, the radar coordinates and the camera coordinates are taken as a set of feature point pairs, that is, the three-dimensional feature point of each calibration point in the radar coordinate system (that is, the radar coordinates qi ) and the two-dimensional feature point in the camera coordinate system (that is, the camera coordinatespi ) are a set of feature point pairs (pi , qi ). In practical applications, the number of calibration points on the calibration object can be at least three, that is, the number of feature point pairs in each set of data can be at least three. After obtaining multiple sets of feature point pairs, the camera coordinate system and the radar coordinate system can be calibrated according to them.
本发明实施例的技术方案，通过分别获取在同一时间采集到的包含有多边形状的标定物的点云数据和图像数据，从各点云数据中筛选出属于标定物上的标定数据，且由于多边形中的每条边与至少两条点云线相交，这意味着根据各标定数据可以分别确定每条边在雷达坐标系下的边坐标；进而，根据各边坐标可以确定标定物中各顶点在该雷达坐标系下的的顶点坐标，并根据各顶点坐标、及标定物中的标定点在标定物中的相对位置，确定标定点在该雷达坐标系下的雷达坐标；由此，可以将标定点在相机坐标系下的相机坐标、及雷达坐标作为一组特征点对，并基于多组特征点对进行相机坐标系和雷达坐标系间的标定。上述技术方案，通过多边形中各条边的边坐标可以确定出特征点的雷达坐标，这意味着多边形中的每条边与至少两条点云线相交时，即可从各点云数据中提取出特征点，这对标定物的尺寸和摆放坐标的限制较小，由此提高了标定物在制作和使用方面的便利性，进而提高了标定工作的便利性。The technical solution of the embodiment of the present invention is to respectively obtain point cloud data and image data of a polygonal calibration object collected at the same time, and filter out calibration data belonging to the calibration object from each point cloud data. Since each edge in the polygon intersects with at least two point cloud lines, this means that the edge coordinates of each edge in the radar coordinate system can be determined respectively according to each calibration data; further, the vertex coordinates of each vertex in the calibration object in the radar coordinate system can be determined according to each edge coordinate, and the radar coordinates of the calibration point in the radar coordinate system can be determined according to each vertex coordinate and the relative position of the calibration point in the calibration object; thus, the camera coordinates and radar coordinates of the calibration point in the camera coordinate system can be taken as a set of feature point pairs, and calibration between the camera coordinate system and the radar coordinate system can be performed based on multiple sets of feature point pairs. The above technical solution can determine the radar coordinates of the feature points through the edge coordinates of each edge in the polygon, which means that when each edge in the polygon intersects with at least two point cloud lines, the feature points can be extracted from each point cloud data. This places less restrictions on the size and placement coordinates of the calibration object, thereby improving the convenience of the calibration object in production and use, and further improving the convenience of the calibration work.
一种可选的技术方案，标定物可以包括四边形标定物，如正方形、长方形、平行四边形状的标定物；该四边形标定物内设置有与至少两条边相切的曲边形，如在四边形标定物的四个顶点的附近分别设置有与该顶点相连的两条边相切的曲边形，再如在长方形标定物内设置有与三条边相切的曲边形，再如在正方形标定物内设置有与四条边均相切的曲边形等，该曲边形可以是圆形、椭圆形等；该标定点可以是曲边形的中心点，如圆形的圆心等。An optional technical solution, the calibration object may include a quadrilateral calibration object, such as a square, rectangle, or parallelogram-shaped calibration object; a curved shape tangent to at least two sides is arranged in the quadrilateral calibration object, such as curved shapes tangent to two sides connected to the vertices are arranged near the four vertices of the quadrilateral calibration object, for example, a curved shape tangent to three sides is arranged in the rectangular calibration object, and for example, a curved shape tangent to all four sides is arranged in the square calibration object, etc. The curved shape may be a circle, an ellipse, etc.; the calibration point may be the center point of the curved shape, such as the center of a circle, etc.
在此基础上，可选的，若多边形是方形，曲边形是圆形，圆形与顶点相连的两条边相切，示例性的，如图2所示，此时，根据各顶点坐标、以及标定物中的标定点在标定物中的相对位置，确定标定点在雷达坐标系下的雷达坐标，具体可包括：各顶点包括上顶点、左顶点和右顶点，通过如下公式确定出与上顶点相邻的标定点在雷达坐标系下的雷达坐标Ptop：On this basis, optionally, if the polygon is a square, the curved shape is a circle, and the two sides connected to the vertices of the circle are tangent, exemplarily, as shown in FIG. 2 , at this time, according to the coordinates of each vertex and the relative position of the calibration point in the calibration object, the radar coordinates of the calibration point in the radar coordinate system are determined, which may specifically include: each vertex includes an upper vertex, a left vertex, and a right vertex, and the radar coordinate Ptop of the calibration point adjacent to the upper vertex in the radar coordinate system is determined by the following formula:
其中，Ctop是上顶点在雷达坐标系下的上顶点坐标，Cleft是左顶点在雷达坐标系下的左顶点坐标，Cright是右顶点在雷达坐标系下的右顶点坐标，w是各边中第一边的长度，h是各边中与第一边相垂直的第二边的长度，r是圆形的半径。为了更好地理解上述公式，以图4为例，cleft-ctop表示位于左上角的边所在的方向，(cleft-ctop)/w表示该方向上的单元向量，(cright-ctop)/h的含义类似；然后，在ctop的基础上沿着这两个单元向量去除掉该半径，即可得到圆心的坐标Ptop。当然，与其余顶点相邻的圆心在雷达坐标系下的雷达坐标的确定过程相类似，在此不再赘述。Among them, Ctop is the coordinate of the upper vertex of the upper vertex in the radar coordinate system, Cleft is the coordinate of the left vertex of the left vertex in the radar coordinate system, Cright is the coordinate of the right vertex of the right vertex in the radar coordinate system, w is the length of the first side of each side, h is the length of the second side perpendicular to the first side of each side, and r is the radius of the circle. In order to better understand the above formula, taking Figure 4 as an example, cleft -ctop represents the direction of the side located in the upper left corner, (cleft -ctop )/w represents the unit vector in this direction, and (cright -ctop )/h has a similar meaning; then, based on ctop , the radius is removed along these two unit vectors to obtain the coordinate of the center of the circle Ptop . Of course, the process of determining the radar coordinates of the center of the circle adjacent to the other vertices in the radar coordinate system is similar and will not be repeated here.
需要说明的是，如图2所示的标定物的设置方式仅是一个可选而非唯一的方式，在从对这样设置出的标定物进行扫描后得到的点云数据中提取特征点、或是进行拍摄后得到的图像数据中提取特征点的提取过程较为简单且提取精度较佳，且根据多个特征点对进行标定时可以提高标定工作的标定精度。其中，通常情况下，设置于四边形中的圆形的数量可以是三个及其以上，这是因为基于至少三个特征点能够完成标定工作。It should be noted that the setting method of the calibration object shown in FIG2 is only an optional method and not the only method. The extraction process of extracting feature points from the point cloud data obtained by scanning the calibration object set in this way or extracting feature points from the image data obtained by shooting is relatively simple and has better extraction accuracy. When calibrating according to multiple feature points, the calibration accuracy of the calibration work can be improved. In general, the number of circles set in the quadrilateral can be three or more, because the calibration work can be completed based on at least three feature points.
实施例二Embodiment 2
图6是本发明实施例二中提供的一种激光雷达标定方法的流程图。本实施例以上述各技术方案为基础进行优化。在本实施例中，可选的，从各点云数据中筛选出属于标定物上的标定数据，具体可以包括：从各点云数据中筛选出属于同一平面上的平面数据，并从各平面数据构成的平面集合中筛选出端点数据；将各端点数据构成的对角线的第一长度、与标定物相应的对角线的第二长度进行对比，根据对比结果从各点云数据中筛选出属于标定物上的标定数据。其中，与上述各实施例相同或相应的术语的解释在此不再赘述。FIG6 is a flow chart of a laser radar calibration method provided in Embodiment 2 of the present invention. This embodiment is optimized based on the above-mentioned technical solutions. In this embodiment, optionally, the calibration data belonging to the calibration object is filtered out from each point cloud data, which may specifically include: filtering out plane data belonging to the same plane from each point cloud data, and filtering out endpoint data from the plane set composed of each plane data; comparing the first length of the diagonal formed by each endpoint data and the second length of the diagonal corresponding to the calibration object, and filtering out the calibration data belonging to the calibration object from each point cloud data according to the comparison result. Among them, the explanations of the terms that are the same or corresponding to the above-mentioned embodiments are not repeated here.
参见图6，本实施例的方法具体可以包括如下步骤：Referring to FIG. 6 , the method of this embodiment may specifically include the following steps:
S210、分别获取在同一时间采集到的包含有标定物的点云数据和图像数据，其中，标定物是多边形，且多边形中的每条边与至少两条点云线相交，点云线包括多个点云数据。S210, respectively acquiring point cloud data and image data containing a calibration object collected at the same time, wherein the calibration object is a polygon, and each edge of the polygon intersects with at least two point cloud lines, and the point cloud line includes a plurality of point cloud data.
S220、从各点云数据中筛选出属于同一平面上的平面数据，并从各平面数据构成的平面集合中筛选出端点数据。S220 , filtering out plane data belonging to the same plane from each point cloud data, and filtering out endpoint data from a plane set formed by each plane data.
其中，各点云数据并非是隶属于同一平面上的数据，有些点云数据可能是隶属于标定物所在的平面上的数据，有些点云数据可能是隶属于标定物一侧的墙体所在的平面上的数据，等等。为此，可以先从各点云数据中筛选出隶属于同一平面上的平面数据，如采用RANSAC算法提取平面数据，该平面数据可能是标定物所在的平面上的点云数据，也可能是其余平面上的点云数据，将这些平面数据构成的点云集合称为平面集合。在实际应用中，可选的，由于该平面集合中的某些点云数据可能是离群点，该离群点可以认为是噪声点，其可能是该平面集合所在的平面的周围的噪声点，此时可以基于统计滤波等算法去除掉该离群点，由此提高了后续的端点数据的筛选精度。Among them, each point cloud data is not data belonging to the same plane. Some point cloud data may be data belonging to the plane where the calibration object is located, and some point cloud data may be data belonging to the plane where the wall on one side of the calibration object is located, and so on. To this end, the plane data belonging to the same plane can be first screened out from each point cloud data, such as using the RANSAC algorithm to extract plane data. The plane data may be point cloud data on the plane where the calibration object is located, or it may be point cloud data on other planes. The point cloud set composed of these plane data is called a plane set. In practical applications, optionally, since some point cloud data in the plane set may be outliers, the outliers can be considered as noise points, which may be noise points around the plane where the plane set is located. At this time, the outliers can be removed based on algorithms such as statistical filtering, thereby improving the screening accuracy of subsequent endpoint data.
进一步，从各平面数据构成的平面集合中筛选出端点数据，该端点数据是该平面集合中位于极端位置上的点云数据，例如位于最上端ftop、最下端fbottom、最左端fleft和/或最右端fright的点云数据。需要说明的是，由于各条扫描线较为稀疏，在基于各扫描线对标定物进行扫描时，可能出现并未扫描到该标定物的顶点的情况，即这些端点数据不一定是对顶点进行扫描后得到的点云数据。Furthermore, endpoint data is screened out from the plane set formed by each plane data, and the endpoint data is point cloud data located at an extreme position in the plane set, such as point cloud data located at the top ftop , the bottom fbottom , the leftmost fleft , and/or the rightmost fright . It should be noted that, since each scanning line is relatively sparse, when the calibration object is scanned based on each scanning line, it may happen that the vertex of the calibration object is not scanned, that is, these endpoint data are not necessarily point cloud data obtained after scanning the vertex.
S230、将各端点数据构成的对角线的第一长度、与标定物相应的对角线的第二长度进行对比，根据对比结果从各点云数据中筛选出属于标定物上的标定数据，并根据各标定数据分别确定每条边在雷达坐标系下的边坐标。S230, comparing the first length of the diagonal formed by each endpoint data with the second length of the diagonal corresponding to the calibration object, filtering out the calibration data belonging to the calibration object from each point cloud data according to the comparison result, and determining the edge coordinates of each edge in the radar coordinate system according to each calibration data.
其中，由于点云数据包括自身的坐标，根据各端点数据可以确定其构成的对角线的第一长度。在此基础上，将标定物的对角线的长度作为第二长度，则将第一长度和第二长度进行对比可以确定出该平面集合对应的同一平面是否为标定物所在的平面，如在第一长度和第二长度间的差值小于预设长度阈值时，该同一平面为标定物所在的平面，此时可以将平面数据作为隶属于标定物上的标定数据。示例性的，以标定物是四边形标定物为例，令d1＝||fleft-fright||、d2＝||ftop-fbottom||，d＝sqrt(w2+h2)，即d是第二长度，d1和d2均是第一长度，若d-a<d1<d+b且d-a<d2<d+b，a、b为参数，则认为该同一平面为标定物所在的平面。Among them, since the point cloud data includes its own coordinates, the first length of the diagonal formed by it can be determined according to the data of each endpoint. On this basis, the length of the diagonal of the calibration object is taken as the second length, and the first length and the second length are compared to determine whether the same plane corresponding to the plane set is the plane where the calibration object is located. If the difference between the first length and the second length is less than the preset length threshold, the same plane is the plane where the calibration object is located, and the plane data can be used as the calibration data belonging to the calibration object. Exemplarily, taking the calibration object as a quadrilateral calibration object as an example, let d1 =||fleft -fright ||, d2 =||ftop -fbottom ||, d = sqrt(w2 +h2 ), that is, d is the second length, d1 and d2 are both the first length, if da<d1 <d+b and da<d2 <d+b, a and b are parameters, then the same plane is considered to be the plane where the calibration object is located.
当然，在此基础上，可选的，若根据对比结果确定出同一平面不是标定物所在的平面，则可以从各点云数据中剔除平面数据，并根据剔除结果更新点云数据，即仅保留各点云数据中除各平面数据之外的点云数据；重复执行上述从各点云数据中筛选出属于同一平面上的平面数据的步骤，直至同一平面是标定物所在的平面；将平面数据作为属于标定物上的标定数据。在此基础上，可选，若保留下来的点云数据的数量过少，可以直接给出标定数据提取失败的提示。Of course, on this basis, optionally, if it is determined based on the comparison results that the same plane is not the plane where the calibration object is located, the plane data can be removed from each point cloud data, and the point cloud data can be updated based on the removal results, that is, only the point cloud data other than the plane data in each point cloud data is retained; the above steps of filtering out the plane data belonging to the same plane from each point cloud data are repeated until the same plane is the plane where the calibration object is located; the plane data is used as the calibration data belonging to the calibration object. On this basis, optionally, if the amount of retained point cloud data is too small, a prompt that the calibration data extraction failed can be directly given.
S240、根据各边坐标分别确定标定物中每个顶点在雷达坐标系下的顶点坐标，并根据各顶点坐标、以及标定物中的标定点在标定物中的相对位置，确定标定点在雷达坐标系下的雷达坐标。S240, determining the vertex coordinates of each vertex in the calibration object in the radar coordinate system according to the coordinates of each edge, and determining the radar coordinates of the calibration point in the radar coordinate system according to the coordinates of each vertex and the relative position of the calibration point in the calibration object.
S250、将标定点在相机坐标系下的相机坐标、及雷达坐标作为一组特征点对，基于多组特征点对进行相机坐标系和雷达坐标系的标定。S250: taking the camera coordinates and the radar coordinates of the calibration points in the camera coordinate system as a set of feature point pairs, and calibrating the camera coordinate system and the radar coordinate system based on the multiple sets of feature point pairs.
本发明实施例的技术方案，通过从各点云数据中筛选出的属于同一平面上的各平面数据构成的平面集合中筛选出端点数据，进而根据各端点数据构成的对角线的第一长度、与标定物相应的对角线的第二长度进行对比，可以从各点云数据中筛选出属于标定物上的标定数据，该对角线的长度的对比方式既简单又高效，由此提高了标定数据的提取效率和提取精度。The technical solution of the embodiment of the present invention is to filter out endpoint data from a plane set composed of plane data belonging to the same plane filtered out from each point cloud data, and then compare the first length of the diagonal formed by each endpoint data with the second length of the diagonal corresponding to the calibration object. The calibration data belonging to the calibration object can be filtered out from each point cloud data. The comparison method of the diagonal length is simple and efficient, thereby improving the extraction efficiency and extraction accuracy of the calibration data.
实施例三Embodiment 3
图7是本发明实施例三中提供的一种激光雷达标定方法的流程图。本实施例以上述各技术方案为基础进行优化。在本实施例中，可选的，基于多组特征点对进行相机坐标系和雷达坐标系的标定，具体可以包括：基于多组特征点对确定相机坐标系到雷达坐标系的坐标变化关系，根据坐标变化关系对相机坐标系和雷达坐标系进行标定。其中，与上述各实施例相同或相应的术语的解释，在此不再赘述。FIG7 is a flow chart of a laser radar calibration method provided in Embodiment 3 of the present invention. This embodiment is optimized based on the above-mentioned technical solutions. In this embodiment, optionally, the camera coordinate system and the radar coordinate system are calibrated based on multiple sets of feature point pairs, which may specifically include: determining the coordinate change relationship from the camera coordinate system to the radar coordinate system based on the multiple sets of feature point pairs, and calibrating the camera coordinate system and the radar coordinate system according to the coordinate change relationship. The explanations of the terms that are the same as or corresponding to the above-mentioned embodiments are not repeated here.
参见图7，本实施例的方法具体可以包括如下步骤：Referring to FIG. 7 , the method of this embodiment may specifically include the following steps:
S310、分别获取在同一时间采集到的包含有标定物的点云数据和图像数据，其中，标定物是多边形，且多边形中的每条边与至少两条点云线相交，点云线包括多个点云数据。S310, respectively acquiring point cloud data and image data containing a calibration object collected at the same time, wherein the calibration object is a polygon, and each edge of the polygon intersects with at least two point cloud lines, and the point cloud line includes a plurality of point cloud data.
S320、从各点云数据中筛选出属于标定物上的标定数据，并根据各标定数据分别确定每条边在雷达坐标系下的边坐标。S320, filtering out calibration data belonging to the calibration object from each point cloud data, and determining the edge coordinates of each edge in the radar coordinate system according to each calibration data.
S330、根据各边坐标分别确定标定物中每个顶点在雷达坐标系下的顶点坐标，并根据各顶点坐标、以及标定物中的标定点在标定物中的相对位置，确定标定点在雷达坐标系下的雷达坐标。S330, determining the vertex coordinates of each vertex in the calibration object in the radar coordinate system according to the coordinates of each edge, and determining the radar coordinates of the calibration point in the radar coordinate system according to the coordinates of each vertex and the relative position of the calibration point in the calibration object.
S340、将标定点在相机坐标系下的相机坐标和雷达坐标作为一组特征点对，基于多组特征点对确定相机坐标系到雷达坐标系的坐标变化关系，根据坐标变化关系对相机坐标系和雷达坐标系进行标定。S340, taking the camera coordinates and the radar coordinates of the calibration point in the camera coordinate system as a set of feature point pairs, determining the coordinate change relationship from the camera coordinate system to the radar coordinate system based on multiple sets of feature point pairs, and calibrating the camera coordinate system and the radar coordinate system according to the coordinate change relationship.
其中，每个标定点在雷达坐标系下的三维特征点(即，雷达坐标qi)和在相机坐标系下的二维特征点(即，相机坐标pi)是一组特征点对(pi，qi)，实际应用中，标定物上的标定点的数量可以是至少三个，即每组数据中特征点对的数量可以是至少三个。汇总各组数据的特征点对，根据这些特征点对确定雷达坐标系到相机坐标系间的坐标变化关系，进而根据该坐标变化关系可以对相机坐标系和雷达坐标系进行标定。Among them, the three-dimensional feature point of each calibration point in the radar coordinate system (i.e., radar coordinates qi ) and the two-dimensional feature point in the camera coordinate system (i.e., camera coordinatespi ) are a set of feature point pairs (pi , qi ). In practical applications, the number of calibration points on the calibration object can be at least three, that is, the number of feature point pairs in each set of data can be at least three. The feature point pairs of each set of data are summarized, and the coordinate change relationship between the radar coordinate system and the camera coordinate system is determined based on these feature point pairs, and then the camera coordinate system and the radar coordinate system can be calibrated based on the coordinate change relationship.
需要说明的是，这一坐标变化关系的确定过程可以是典型的PnP(Perspective NPoints)问题，其可以通过多种方式进行运算，例如P3P、直接线性变换(DLT)、EPnP(Efficient PnP)、UPnP、Bundle Adjustment等等，通过运算结果可以得到该坐标变化关系，该坐标变化关系可以认为是标定结果。It should be noted that the process of determining this coordinate change relationship can be a typical PnP (Perspective NPoints) problem, which can be calculated in a variety of ways, such as P3P, direct linear transformation (DLT), EPnP (Efficient PnP), UPnP, Bundle Adjustment, etc. The coordinate change relationship can be obtained through the calculation results, and the coordinate change relationship can be considered as the calibration result.
这里以DLT为例进行说明：Here we take DLT as an example:
对于特征点对(pi，qi)，记录其对应的齐次坐标分别为pi＝[u,v,1]T以及qi＝[x,y,z,1]T。令相机传感器的内参矩阵为For a pair of feature points (pi ,qi ), record their corresponding homogeneous coordinates aspi = [u,v,1]T andqi = [x,y,z,1]T. Let the intrinsic parameter matrix of the camera sensor be
那么，从雷达坐标系到相机坐标系的投影关系为Then, the projection relationship from the radar coordinate system to the camera coordinate system is
其中，R为三维旋转矩阵，t为平移向量，它们描述了雷达坐标系到相机坐标系的坐标变化关系。对R和t进行展开，结果是：Among them, R is the three-dimensional rotation matrix and t is the translation vector, which describe the coordinate change relationship from the radar coordinate system to the camera coordinate system. Expanding R and t, the result is:
消去，得到Eliminate, get
也就是说，对于一组特征点对(pi，qi)，我们可以得到方程Bia＝0。其中Bi为由(pi，qi)及K组成的矩阵，a为由三维旋转矩阵和平移向量的系数组成的向量。That is to say, for a set of feature point pairs (pi ,qi ), we can get the equationBia = 0. WhereBi is a matrix composed of (pi ,qi ) and K, and a is a vector composed of the coefficients of the three-dimensional rotation matrix and the translation vector.
汇总全部的特征点对，可以得到约束方程Ba＝0。这个方程无法求出精确解，但是可以获得一个|a|＝1约束下的最小二乘解arg min||Ba||2。Summarizing all the feature point pairs, we can get the constraint equation Ba = 0. This equation cannot be solved exactly, but we can get a least squares solution arg min||Ba||2 under the constraint |a|=1.
1.根据全部特征点对及相机内参，得到矩阵B；1. Obtain matrix B based on all feature point pairs and camera internal parameters;
2.对B执行SVD分解，有[U Σ V]＝SVD(B)。2. Perform SVD decomposition on B, and [U Σ V] = SVD(B).
3.取V矩阵的最后一列，记为取/>的9个分量，组成三维旋转矩阵3. Take the last column of the V matrix and record it as Take/> The 9 components of form a three-dimensional rotation matrix
4.对R执行SVD分解，有4. Perform SVD decomposition on R, and we have
5.计算缩放因子β。β满足且5. Calculate the scaling factor β. β satisfies and
6.最优旋转矩阵R为平移向量t为/>6. The optimal rotation matrix R is The translation vector t is/>
本发明实施例的技术方案，通过基于多组特征点对确定相机坐标系到雷达坐标系的坐标变化关系，并根据坐标变化关系对相机坐标系和雷达坐标系进行标定，由此实现了相机坐标系和雷达坐标系间的准确标定的效果。The technical solution of the embodiment of the present invention determines the coordinate change relationship from the camera coordinate system to the radar coordinate system based on multiple sets of feature point pairs, and calibrates the camera coordinate system and the radar coordinate system according to the coordinate change relationship, thereby achieving the effect of accurate calibration between the camera coordinate system and the radar coordinate system.
实施例四Embodiment 4
图8为本发明实施例四提供的激光雷达标定装置的结构框图，该装置用于执行上述任意实施例所提供的激光雷达标定方法。该装置与上述各实施例的激光雷达标定方法属于同一个发明构思，在激光雷达标定装置的实施例中未详尽描述的细节内容，可以参考上述激光雷达标定方法的实施例。参见图8，该装置具体可以包括：数据获取模块410、边坐标确定模块420、雷达坐标确定模块430和激光雷达标定模块440。FIG8 is a block diagram of a laser radar calibration device provided in Embodiment 4 of the present invention, and the device is used to execute the laser radar calibration method provided in any of the above embodiments. The device and the laser radar calibration method of the above embodiments belong to the same inventive concept, and the details not described in detail in the embodiment of the laser radar calibration device can refer to the embodiment of the above laser radar calibration method. Referring to FIG8 , the device may specifically include: a data acquisition module 410, an edge coordinate determination module 420, a radar coordinate determination module 430, and a laser radar calibration module 440.
其中，数据获取模块410，用于分别获取在同一时间采集到的包含有标定物的点云数据和图像数据，其中，标定物是多边形，且多边形中的每条边与至少两条点云线相交，点云线包括多个点云数据；The data acquisition module 410 is used to respectively acquire point cloud data and image data containing a calibration object collected at the same time, wherein the calibration object is a polygon, and each edge of the polygon intersects with at least two point cloud lines, and the point cloud line includes a plurality of point cloud data;
边坐标确定模块420，用于从各点云数据中筛选出属于标定物上的标定数据，并根据各标定数据分别确定每条边在雷达坐标系下的边坐标；The edge coordinate determination module 420 is used to filter out the calibration data belonging to the calibration object from each point cloud data, and determine the edge coordinates of each edge in the radar coordinate system according to each calibration data;
雷达坐标确定模块430，用于根据各边坐标分别确定标定物中每个顶点在雷达坐标系下的顶点坐标，并根据各顶点坐标、以及标定物中的标定点在标定物中的相对位置，确定标定点在雷达坐标系下的雷达坐标；The radar coordinate determination module 430 is used to determine the vertex coordinates of each vertex in the calibration object in the radar coordinate system according to the coordinates of each edge, and determine the radar coordinates of the calibration point in the radar coordinate system according to the coordinates of each vertex and the relative position of the calibration point in the calibration object;
激光雷达标定模块440，用于将标定点在相机坐标系下的相机坐标、以及雷达坐标作为一组特征点对，基于多组特征点对进行相机坐标系和雷达坐标系的标定。The laser radar calibration module 440 is used to use the camera coordinates of the calibration point in the camera coordinate system and the radar coordinates as a set of feature point pairs, and calibrate the camera coordinate system and the radar coordinate system based on multiple sets of feature point pairs.
可选的，边坐标确定模块420，具体可以包括：Optionally, the edge coordinate determination module 420 may specifically include:
端点数据筛选单元，用于从各点云数据中筛选出属于同一平面上的平面数据，并从各平面数据构成的平面集合中筛选出端点数据；标定数据确定单元，用于将各端点数据构成的对角线的第一长度、与标定物相应的对角线的第二长度进行对比，根据对比结果从各点云数据中筛选出属于标定物上的标定数据。The endpoint data screening unit is used to screen out plane data belonging to the same plane from each point cloud data, and to screen out endpoint data from the plane set formed by each plane data; the calibration data determination unit is used to compare the first length of the diagonal formed by each endpoint data with the second length of the diagonal corresponding to the calibration object, and to screen out the calibration data belonging to the calibration object from each point cloud data according to the comparison result.
可选的，标定数据确定单元，具体可以包括：Optionally, the calibration data determination unit may specifically include:
点云数据更新子单元，用于若根据对比结果确定出同一平面不是标定物所在的平面，则从各点云数据中剔除平面数据，并根据剔除结果更新点云数据；重复执行子单元，用于重复执行从各点云数据中筛选出属于同一平面上的平面数据的步骤，直至同一平面是标定物所在的平面；标定数据确定子单元，用于将平面数据作为属于标定物上的标定数据。The point cloud data updating subunit is used to eliminate the plane data from each point cloud data if it is determined according to the comparison result that the same plane is not the plane where the calibration object is located, and update the point cloud data according to the elimination result; the repeated execution subunit is used to repeatedly execute the steps of filtering out the plane data belonging to the same plane from each point cloud data until the same plane is the plane where the calibration object is located; the calibration data determination subunit is used to use the plane data as the calibration data belonging to the calibration object.
可选的，边坐标确定模块420，具体可包括：边坐标确定单元，用于获取各标定数据构成的多条标定线，确定每条标定线在扫描方向上的边缘数据，并根据各边缘数据分别确定每条边在雷达坐标系下的边坐标。Optionally, the edge coordinate determination module 420 may specifically include: an edge coordinate determination unit, used to obtain multiple calibration lines composed of various calibration data, determine the edge data of each calibration line in the scanning direction, and determine the edge coordinates of each edge in the radar coordinate system according to each edge data.
可选的，标定物包括四边形标定物，四边形标定物内设置有与至少两条边相切的曲边形，标定点是曲边形的中心点。Optionally, the calibration object includes a quadrilateral calibration object, a curved shape tangent to at least two sides is arranged in the quadrilateral calibration object, and the calibration point is the center point of the curved shape.
可选，多边形包括方形，曲边形包括圆形，圆形与顶点相连的两条边相切；相应的，雷达坐标确定模块430，具体可以用于：Optionally, the polygon includes a square, the curved edge includes a circle, and two sides of the circle connected to the vertex are tangent; accordingly, the radar coordinate determination module 430 can be specifically used for:
各顶点包括上顶点、左顶点和右顶点，通过如下公式确定出与上顶点相邻的标定点在雷达坐标系下的雷达坐标Ptop：Each vertex includes an upper vertex, a left vertex, and a right vertex. The radar coordinate Ptop of the calibration point adjacent to the upper vertex in the radar coordinate system is determined by the following formula:
其中，Ctop是上顶点在雷达坐标系下的上顶点坐标，Cleft是左顶点在雷达坐标系下的左顶点坐标，Cright是右顶点在雷达坐标系下的右顶点坐标，w是各边中第一边的长度，h是各边中与第一边相垂直的第二边的长度，r是圆形的半径。Among them, Ctop is the coordinate of the upper vertex in the radar coordinate system, Cleft is the coordinate of the left vertex in the radar coordinate system, Cright is the coordinate of the right vertex in the radar coordinate system, w is the length of the first side of each side, h is the length of the second side perpendicular to the first side of each side, and r is the radius of the circle.
可选的，激光雷达标定模块440，具体可以包括：Optionally, the laser radar calibration module 440 may specifically include:
激光雷达标定单元，用于基于多组特征点对确定相机坐标系到雷达坐标系的坐标变化关系，根据坐标变化关系对相机坐标系和雷达坐标系进行标定。The laser radar calibration unit is used to determine the coordinate change relationship from the camera coordinate system to the radar coordinate system based on multiple sets of feature point pairs, and calibrate the camera coordinate system and the radar coordinate system according to the coordinate change relationship.
本发明实施例四提供的激光雷达标定装置，通过数据获取模块以及边坐标确定模块相互配合，分别获取在同一时间采集到的包含有多边形状的标定物的点云数据和图像数据，从各点云数据中筛选出属于标定物上的标定数据，而且由于多边形中的每条边与至少两条点云线相交，这意味着根据各标定数据可以分别确定每条边在雷达坐标系下的边坐标；进而，雷达坐标确定模块根据各边坐标可以确定标定物中各顶点在该雷达坐标系下的顶点坐标，并根据各顶点坐标、及标定物中的标定点在标定物中的相对位置，确定标定点在该雷达坐标系下的雷达坐标；由此，激光雷达标定模块可将标定点在相机坐标系下的相机坐标、以及雷达坐标作为一组特征点对，并基于多组特征点对进行相机坐标系和雷达坐标系间的标定。上述装置，通过多边形中各条边的边坐标可以确定出特征点的雷达坐标，这意味着多边形中的每条边与至少两条点云线相交时，即可从各点云数据中提取出特征点，这对标定物的尺寸和摆放位置的限制较小，由此提高了标定物在制作和使用方面的便利性，进而提高了标定工作的便利性。The laser radar calibration device provided in the fourth embodiment of the present invention, through the cooperation of the data acquisition module and the edge coordinate determination module, respectively acquires the point cloud data and image data of the polygonal calibration object collected at the same time, and filters out the calibration data belonging to the calibration object from each point cloud data. Moreover, since each edge in the polygon intersects with at least two point cloud lines, this means that the edge coordinates of each edge in the radar coordinate system can be determined respectively according to each calibration data; furthermore, the radar coordinate determination module can determine the vertex coordinates of each vertex in the calibration object in the radar coordinate system according to each edge coordinate, and determine the radar coordinates of the calibration point in the radar coordinate system according to each vertex coordinate and the relative position of the calibration point in the calibration object; thus, the laser radar calibration module can use the camera coordinates of the calibration point in the camera coordinate system and the radar coordinates as a set of feature point pairs, and perform calibration between the camera coordinate system and the radar coordinate system based on multiple sets of feature point pairs. The above device can determine the radar coordinates of the feature points through the edge coordinates of each edge in the polygon, which means that when each edge in the polygon intersects with at least two point cloud lines, the feature points can be extracted from each point cloud data. This places less restrictions on the size and placement of the calibration object, thereby improving the convenience of the calibration object in production and use, and further improving the convenience of the calibration work.
本发明实施例所提供的激光雷达标定装置可执行本发明任意实施例所提供的激光雷达标定方法，具备执行方法相应的功能模块和有益效果。The laser radar calibration device provided in the embodiment of the present invention can execute the laser radar calibration method provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the execution method.
值得注意的是，上述激光雷达标定装置的实施例中，所包括的各个单元和模块只是按照功能逻辑进行划分的，但并不局限于上述的划分，只要能够实现相应的功能即可；另外，各功能单元的具体名称也只是为了便于相互区分，并不用于限制本发明的保护范围。It is worth noting that in the embodiment of the above-mentioned laser radar calibration device, the various units and modules included are only divided according to functional logic, but are not limited to the above-mentioned division, as long as the corresponding functions can be achieved; in addition, the specific names of the functional units are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of the present invention.
实施例五Embodiment 5
图9为本发明实施例五提供的一种电子设备的结构示意图，如图9所示，该电子设备包括存储器510、处理器520、输入装置530和输出装置540。电子设备中的处理器520的数量可以是一个或多个，图9中以一个处理器520为例；电子设备中的存储器510、处理器520、输入装置530和输出装置540可以通过总线或其它方式连接，图9中以通过总线550连接为例。FIG9 is a schematic diagram of the structure of an electronic device provided in Embodiment 5 of the present invention. As shown in FIG9 , the electronic device includes a memory 510, a processor 520, an input device 530, and an output device 540. The number of processors 520 in the electronic device may be one or more, and FIG9 takes one processor 520 as an example; the memory 510, the processor 520, the input device 530, and the output device 540 in the electronic device may be connected via a bus or other means, and FIG9 takes the connection via a bus 550 as an example.
存储器510作为一种计算机可读存储介质，可用于存储软件程序、计算机可执行程序以及模块，如本发明实施例中的激光雷达标定方法对应的程序指令/模块(例如，激光雷达标定装置中的数据获取模块410、边坐标确定模块420、雷达坐标确定模块430和激光雷达标定模块440)。处理器520通过运行存储在存储器510中的软件程序、指令以及模块，从而执行电子设备的各种功能应用以及激光雷达标定，即实现上述的激光雷达标定方法。The memory 510, as a computer-readable storage medium, can be used to store software programs, computer executable programs and modules, such as program instructions/modules corresponding to the laser radar calibration method in the embodiment of the present invention (for example, the data acquisition module 410, the edge coordinate determination module 420, the radar coordinate determination module 430 and the laser radar calibration module 440 in the laser radar calibration device). The processor 520 executes various functional applications of the electronic device and the laser radar calibration by running the software programs, instructions and modules stored in the memory 510, that is, implements the above-mentioned laser radar calibration method.
存储器510可主要包括存储程序区和存储数据区，其中，存储程序区可存储操作系统、至少一个功能所需的应用程序；存储数据区可存储根据电子设备的使用所创建的数据等。此外，存储器510可以包括高速随机存取存储器，还可以包括非易失性存储器，例如至少一个磁盘存储器件、闪存器件、或其他非易失性固态存储器件。在一些实例中，存储器510可进一步包括相对于处理器520远程设置的存储器，这些远程存储器可以通过网络连接至设备。上述网络的实例包括但不限于互联网、企业内部网、局域网、移动通信网及其组合。The memory 510 may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system and at least one application required for a function; the data storage area may store data created according to the use of the electronic device, etc. In addition, the memory 510 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other non-volatile solid-state storage device. In some instances, the memory 510 may further include a memory remotely arranged relative to the processor 520, and these remote memories may be connected to the device via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.
输入装置530可用于接收输入的数字或字符信息，以及产生与装置的用户设置以及功能控制有关的键信号输入。输出装置540可包括显示屏等显示设备。The input device 530 may be used to receive input digital or character information and generate key signal input related to user settings and function control of the device. The output device 540 may include a display device such as a display screen.
实施例六Embodiment 6
本发明实施例六提供一种包含计算机可执行指令的存储介质，所述计算机可执行指令在由计算机处理器执行时用于执行一种激光雷达标定方法，该方法包括：Embodiment 6 of the present invention provides a storage medium including computer executable instructions, wherein the computer executable instructions are used to execute a laser radar calibration method when executed by a computer processor, the method comprising:
分别获取在同一时间采集到的包含有标定物的点云数据和图像数据，其中，标定物是多边形，且多边形中的每条边与至少两条点云线相交，点云线包括多个点云数据；从各点云数据中筛选出属于标定物上的标定数据，并根据各标定数据分别确定每条边在雷达坐标系下的边坐标；根据各边坐标分别确定标定物中每个顶点在雷达坐标系下的顶点坐标，并根据各顶点坐标、以及标定物中的标定点在标定物中的相对位置，确定标定点在雷达坐标系下的雷达坐标；将标定点在相机坐标系下的相机坐标、以及雷达坐标作为一组特征点对，基于多组特征点对进行相机坐标系和雷达坐标系的标定。The point cloud data and image data containing the calibration object collected at the same time are respectively acquired, wherein the calibration object is a polygon, and each edge in the polygon intersects with at least two point cloud lines, and the point cloud lines include multiple point cloud data; the calibration data belonging to the calibration object are screened out from each point cloud data, and the edge coordinates of each edge in the radar coordinate system are respectively determined according to each calibration data; the vertex coordinates of each vertex in the calibration object in the radar coordinate system are respectively determined according to each edge coordinate, and the radar coordinates of the calibration point in the radar coordinate system are determined according to each vertex coordinate and the relative position of the calibration point in the calibration object; the camera coordinates of the calibration point in the camera coordinate system and the radar coordinates are taken as a group of feature point pairs, and the camera coordinate system and the radar coordinate system are calibrated based on the multiple groups of feature point pairs.
当然，本发明实施例所提供的一种包含计算机可执行指令的存储介质，其计算机可执行指令不限于如上所述的方法操作，还可以执行本发明任意实施例所提供的激光雷达标定方法中的相关操作。Of course, the storage medium containing computer executable instructions provided in an embodiment of the present invention is not limited to the method operations described above, and the computer executable instructions can also execute related operations in the laser radar calibration method provided in any embodiment of the present invention.
通过以上关于实施方式的描述，所属领域的技术人员可以清楚地了解到，本发明可借助软件及必需的通用硬件来实现，当然也可以通过硬件实现，但很多情况下前者是更佳的实施方式。依据这样的理解，本发明的技术方案本质上或者说对现有技术做出贡献的部分可以以软件产品的形式体现出来，该计算机软件产品可以存储在计算机可读存储介质中，如计算机的软盘、只读存储器(Read-Only Memory,ROM)、随机存取存储器(RandomAccess Memory,RAM)、闪存(FLASH)、硬盘或光盘等，包括若干指令用以使得一台计算机设备(可以是个人计算机，服务器，或者网络设备等)执行本发明各个实施例所述的方法。Through the above description of the implementation method, the technicians in the relevant field can clearly understand that the present invention can be implemented by means of software and necessary general hardware, and of course it can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as a computer floppy disk, read-only memory (ROM), random access memory (RAM), flash memory (FLASH), hard disk or optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, server, or network device, etc.) to execute the methods described in each embodiment of the present invention.
注意，上述仅为本发明的较佳实施例及所运用技术原理。本领域技术人员会理解，本发明不限于这里所述的特定实施例，对本领域技术人员来说能够进行各种明显的变化、重新调整和替代而不会脱离本发明的保护范围。因此，虽然通过以上实施例对本发明进行了较为详细的说明，但是本发明不仅仅限于以上实施例，在不脱离本发明构思的情况下，还可以包括更多其他等效实施例，而本发明的范围由所附的权利要求范围决定。Note that the above are only preferred embodiments of the present invention and the technical principles used. Those skilled in the art will understand that the present invention is not limited to the specific embodiments described herein, and that various obvious changes, readjustments and substitutions can be made by those skilled in the art without departing from the scope of protection of the present invention. Therefore, although the present invention has been described in more detail through the above embodiments, the present invention is not limited to the above embodiments, and may include more other equivalent embodiments without departing from the concept of the present invention, and the scope of the present invention is determined by the scope of the appended claims.
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011077526.1ACN113759346B (en) | 2020-10-10 | 2020-10-10 | Laser radar calibration method and device, electronic equipment and storage medium |
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011077526.1ACN113759346B (en) | 2020-10-10 | 2020-10-10 | Laser radar calibration method and device, electronic equipment and storage medium |
| Publication Number | Publication Date |
|---|---|
| CN113759346A CN113759346A (en) | 2021-12-07 |
| CN113759346Btrue CN113759346B (en) | 2024-06-18 |
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202011077526.1AActiveCN113759346B (en) | 2020-10-10 | 2020-10-10 | Laser radar calibration method and device, electronic equipment and storage medium |
| Country | Link |
|---|---|
| CN (1) | CN113759346B (en) |
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116485917B (en)* | 2023-06-19 | 2023-09-22 | 擎翌(上海)智能科技有限公司 | Combined calibration method, system, equipment and medium for shooting device and radar device |
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111754578A (en)* | 2019-03-26 | 2020-10-09 | 舜宇光学（浙江）研究院有限公司 | Combined calibration method and system for laser radar and camera and electronic equipment |
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104142157B (en)* | 2013-05-06 | 2017-08-25 | 北京四维图新科技股份有限公司 | A kind of scaling method, device and equipment |
| CN104574406B (en)* | 2015-01-16 | 2017-06-23 | 大连理工大学 | A kind of combined calibrating method between 360 degree of panorama laser and multiple vision systems |
| JP6878045B2 (en)* | 2017-02-28 | 2021-05-26 | 国立研究開発法人理化学研究所 | Point cloud data extraction method and point cloud data extraction device |
| CN109492656B (en)* | 2017-09-11 | 2022-04-29 | 阿波罗智能技术(北京)有限公司 | Method and apparatus for outputting information |
| CN109839624A (en)* | 2017-11-27 | 2019-06-04 | 北京万集科技股份有限公司 | A kind of multilasered optical radar position calibration method and device |
| CN108509918B (en)* | 2018-04-03 | 2021-01-08 | 中国人民解放军国防科技大学 | Target detection and tracking method fusing laser point cloud and image |
| CN109509256B (en)* | 2018-06-21 | 2023-07-18 | 华南理工大学 | Automatic measurement and 3D model generation method of building structure based on lidar |
| CN109343061B (en)* | 2018-09-19 | 2021-04-02 | 百度在线网络技术（北京）有限公司 | Sensor calibration method and device, computer equipment, medium and vehicle |
| CN111123242B (en)* | 2018-10-31 | 2022-03-15 | 北京亚兴智数科技有限公司 | Combined calibration method based on laser radar and camera and computer readable storage medium |
| CN111753858B (en)* | 2019-03-26 | 2024-07-12 | 理光软件研究所（北京）有限公司 | Point cloud matching method, device and repositioning system |
| CN110349221A (en)* | 2019-07-16 | 2019-10-18 | 北京航空航天大学 | A kind of three-dimensional laser radar merges scaling method with binocular visible light sensor |
| CN110780285B (en)* | 2019-10-24 | 2022-10-18 | 深圳市镭神智能系统有限公司 | Pose calibration method, system and medium for laser radar and combined inertial navigation |
| CN110823252B (en)* | 2019-11-06 | 2022-11-18 | 大连理工大学 | An automatic calibration method for multi-line lidar and monocular vision |
| CN111077506B (en)* | 2019-12-12 | 2022-04-19 | 苏州智加科技有限公司 | Method, device and system for calibrating millimeter wave radar |
| CN111127563A (en)* | 2019-12-18 | 2020-05-08 | 北京万集科技股份有限公司 | Joint calibration method, device, electronic device and storage medium |
| CN111220993B (en)* | 2020-01-14 | 2020-07-28 | 长沙智能驾驶研究院有限公司 | Target scene positioning method and device, computer equipment and storage medium |
| CN111369630A (en)* | 2020-02-27 | 2020-07-03 | 河海大学常州校区 | A method of multi-line lidar and camera calibration |
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111754578A (en)* | 2019-03-26 | 2020-10-09 | 舜宇光学（浙江）研究院有限公司 | Combined calibration method and system for laser radar and camera and electronic equipment |
| Publication number | Publication date |
|---|---|
| CN113759346A (en) | 2021-12-07 |
| Publication | Publication Date | Title |
|---|---|---|
| CN112581629A (en) | Augmented reality display method and device, electronic equipment and storage medium | |
| WO2020124988A1 (en) | Vision-based parking space detection method and device | |
| TW202022809A (en) | Depth camera calibration device and method thereof | |
| CN110568447A (en) | Visual positioning method, device and computer readable medium | |
| CN110223226A (en) | Panorama Mosaic method and system | |
| CN113077523B (en) | Calibration method, calibration device, computer equipment and storage medium | |
| CN110163025A (en) | Two dimensional code localization method and device | |
| CN113177988A (en) | Calibration method, device, equipment and storage medium for dome camera and laser | |
| US20240202975A1 (en) | Data processing | |
| CN113759346B (en) | Laser radar calibration method and device, electronic equipment and storage medium | |
| CN117274563A (en) | A photovoltaic panel position and orientation recognition method, device and installation robot | |
| CN117315046A (en) | Method and device for calibrating looking-around camera, electronic equipment and storage medium | |
| CN114660577B (en) | Obtaining calibration plate feature points and corresponding joint calibration method through laser radar | |
| CN114155291A (en) | Box body pose identification method and device, terminal and storage medium | |
| CN114219871A (en) | Obstacle detection method and device based on depth image and mobile robot | |
| CN108269288B (en) | Intelligent special-shaped projection non-contact interaction system and method | |
| CN115100658B (en) | A paper correction method, system and storage medium in an image | |
| CN114445415A (en) | Method for dividing drivable area and related device | |
| CN114043111B (en) | Method and system for automatic assembly of intersecting line structures based on line laser scanning | |
| CN115375741A (en) | A registration method and related device for panoramic image and laser point cloud | |
| CN115439552A (en) | Multi-phase machine external parameter calibration method and device and electronic equipment | |
| CN112419383B (en) | Method, device and storage medium for generating a depth map | |
| CN118279295A (en) | Power distribution cabinet state detection method, device, equipment and storage medium | |
| CN117252931A (en) | Camera combined external parameter calibration method and system using laser radar and storage medium | |
| CN112308933B (en) | Method and device for calibrating camera internal reference and computer storage medium |
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |