CN108416257A - Merge the underground railway track obstacle detection method of vision and laser radar data feature - Google Patents

Abstract

Translated fromChinese

本发明公开了一种融合视觉与激光雷达数据特征的地铁轨道障碍物检测方法，包括以下步骤：S1、为地铁装配激光雷达和摄像头；S2、利用装配有激光雷达和摄像头的地铁沿着行驶线路往返至少一次，采集沿途的摄像头图像数据和激光雷达点云数据；S3、利用激光雷达点云数据进行地图的构建；对于交叉轨道处和弯道位置，利用摄像头图像数据进行逐一标定；S4、对于摄像头图像数据和激光雷达点云数据都无法确定的路况，利用人工进行手动标注；形成S2中的行驶线路的地铁地图数据。本发明融合视觉与激光雷达数据特征实现地铁轨道的障碍物检测；以激光雷达数据为主，以视觉特征为辅，对两者数据特征进行挖掘，检测障碍物。

The invention discloses a method for detecting obstacles on a subway track by combining vision and laser radar data features, comprising the following steps: S1, assembling a laser radar and a camera for the subway; S2, using the subway equipped with a laser radar and a camera to travel along the line Go back and forth at least once, collect the camera image data and lidar point cloud data along the way; S3, use the lidar point cloud data to construct the map; for the cross track and curve position, use the camera image data to calibrate one by one; S4, for The road conditions that cannot be determined by the camera image data and the lidar point cloud data are manually marked; the subway map data of the driving route in S2 is formed. The invention integrates vision and laser radar data features to realize obstacle detection on subway tracks; mainly uses laser radar data, supplemented by visual features, and excavates the two data features to detect obstacles.

Description

Translated fromChinese

融合视觉与激光雷达数据特征的地铁轨道障碍物检测方法Subway track obstacle detection method based on fusion of vision and lidar data features

技术领域technical field

本发明涉及自动驾驶技术领域，具体涉及一种融合视觉与激光雷达数据特征的地铁轨道障碍物检测方法。The invention relates to the technical field of automatic driving, in particular to a method for detecting obstacles on a subway track by integrating vision and laser radar data features.

背景技术Background technique

近年来汽车自动驾驶技术如火如荼的发展，却受限于城市路况的诸多随机性而屡屡碰壁，虽然可以通过配备激光雷达、毫米波雷达等传感器增强自动驾驶的可行性，确又受到传感器高价的困扰而无法实施，比如一个64线激光雷达造价达到8000多美元，接近一辆汽车的价格。而城市中主要的公共交通工具——地铁，因其具有固定的行驶路线及其昂贵的价格，以及地铁司机训练的长周期性，将最迫切以及最容易实现自动驾驶。而地铁自动驾驶中，最关注的一个主要问题就是障碍物的检测。In recent years, the development of autopilot technology has been in full swing, but it has repeatedly hit the wall due to the randomness of urban road conditions. Although the feasibility of autopilot can be enhanced by equipped with sensors such as laser radar and millimeter-wave radar, it is indeed troubled by the high price of sensors. But it cannot be implemented. For example, the cost of a 64-line lidar reaches more than 8,000 US dollars, which is close to the price of a car. The main public transportation in the city, the subway, will be the most urgent and easiest to realize automatic driving because of its fixed driving route and its expensive price, as well as the long period of subway driver training. In the automatic driving of the subway, one of the most concerned issues is the detection of obstacles.

发明内容Contents of the invention

基于以上背景技术，本发明提供一种融合视觉与激光雷达数据特征的地铁轨道障碍物检测方法。以激光雷达数据为主，以视觉特征为辅，对两者数据特征进行挖掘，检测障碍物。Based on the above background technology, the present invention provides a method for detecting obstacles on subway tracks that integrates features of vision and lidar data. Based on lidar data and supplemented by visual features, the data features of the two are mined to detect obstacles.

为了实现以上目的，本发明采用以下技术方案：In order to achieve the above object, the present invention adopts the following technical solutions:

一种融合视觉与激光雷达数据特征的地铁轨道障碍物检测方法，包括以下步骤：A method for detecting obstacles on subway tracks that combines vision and lidar data features, comprising the following steps:

S1、为地铁装配激光雷达和摄像头；S1. Assembling lidar and cameras for the subway;

S2、利用装配有激光雷达和摄像头的地铁沿着行驶线路往返至少一次，采集沿途的摄像头图像数据和激光雷达点云数据；S2. Use the subway equipped with laser radar and camera to go back and forth at least once along the driving line, and collect camera image data and laser radar point cloud data along the way;

S3、利用激光雷达点云数据进行地图的构建；对于交叉轨道处和弯道位置，利用摄像头图像数据进行逐一标定；S3. Use the lidar point cloud data to construct the map; for the cross track and curve position, use the camera image data to calibrate one by one;

S4、对于摄像头图像数据和激光雷达点云数据都无法确定的路况，利用人工进行手动标注；形成S2中的行驶线路的地铁地图数据。S4. For the road conditions that cannot be determined by both the camera image data and the lidar point cloud data, manually mark the road conditions; form the subway map data of the driving route in S2.

优选地，所述地铁地图数据中的摄像头图像数据的集合构成视觉特征数据库，所述检测方法还包括视觉特征数据库的不断完善。Preferably, the collection of camera image data in the subway map data constitutes a visual feature database, and the detection method also includes continuous improvement of the visual feature database.

更优选地，所述视觉特征数据库的不断完善具体过程为：当根据激光雷达判断轨道上方有障碍物出现时，结合地铁地图数据中的摄像头图像数据和当前摄像头所拍摄的障碍物图像数据，判断最终障碍物是否对地铁轨道行驶造成危险，并将障碍物图像数据加入视觉特征数据库中。More preferably, the specific process of continuous improvement of the visual feature database is: when it is judged that there is an obstacle above the track according to the laser radar, combined with the camera image data in the subway map data and the obstacle image data captured by the current camera, judge Finally, whether the obstacle poses a danger to the subway track, and the image data of the obstacle is added to the visual feature database.

本发明另一方面还提供利用以上检测方法进行地铁轨道障碍物的检测的地铁自动驾驶权的方法。On the other hand, the present invention also provides a method for subway automatic driving right for detecting subway track obstacles using the above detection method.

本发明的有益效果Beneficial effects of the present invention

本发明提供的融合视觉与激光雷达数据特征的地铁轨道障碍物检测方法，融合视觉与激光雷达数据特征实现地铁轨道的障碍物检测；以激光雷达数据为主，以视觉特征为辅，对两者数据特征进行挖掘，检测障碍物。The subway track obstacle detection method that combines vision and laser radar data features provided by the present invention can realize the obstacle detection of subway track by combining vision and laser radar data features; mainly based on laser radar data and supplemented by visual features, both Data features are mined to detect obstacles.

附图说明Description of drawings

图1为本发明联合标定的理论实验标定板图片。Fig. 1 is a picture of the theoretical experiment calibration plate of the joint calibration of the present invention.

图2为本发明联合标定的理论实验流程图。Fig. 2 is a theoretical experiment flow chart of the joint calibration of the present invention.

具体实施方式Detailed ways

下面通过实施例对本发明进行具体描述，有必要在此指出的是本实施例只用于对本发明进行进一步说明，不能理解为对本发明保护范围的限制，该领域的技术熟练人员可以根据以上发明的内容做出一些非本质的改进和调整。在不冲突的情况下，本发明中的实施例及实施例中的特征可以相互组合。The present invention is described in detail by the following examples. It is necessary to point out that this example is only used to further illustrate the present invention, and can not be interpreted as limiting the protection scope of the present invention. Those skilled in the art can according to the above invention Some non-essential improvements and adjustments have been made to the content. In the case of no conflict, the embodiments and the features in the embodiments of the present invention can be combined with each other.

本发明首先在实验室条件下验证了摄像机和激光雷达联合标定的实验理论：The present invention first verifies the experimental theory of camera and laser radar joint calibration under laboratory conditions:

在室内摆好棋盘格，利用激光雷达和摄像头同时拍摄该棋盘格，分别获得点云数据和图像数据。Set up the checkerboard indoors, use the lidar and camera to shoot the checkerboard at the same time, and obtain point cloud data and image data respectively.

利用如图1所示的标定板进行摄像头和激光雷达的联合标定，具体标定流程如图2所示。The calibration board shown in Figure 1 is used for joint calibration of the camera and lidar, and the specific calibration process is shown in Figure 2.

其中黑色圆形为圆形空洞区域，其它为实心区域，这样激光雷达在碰到扫描圆形区域时，就会出现深度点云数据的跳跃，便于进行坐标的确定。而摄像头数据可以通过检测各个圆形的中心确定世界坐标和图像坐标间的关系，进而确定相机内外参数，最终实现两者的联合标定。Among them, the black circle is a circular hollow area, and the others are solid areas. In this way, when the lidar encounters a scanning circular area, there will be a jump in the depth point cloud data, which is convenient for determining the coordinates. The camera data can determine the relationship between the world coordinates and the image coordinates by detecting the center of each circle, and then determine the internal and external parameters of the camera, and finally realize the joint calibration of the two.

本发明进一步地根据激光雷达和摄像头进行地铁地图数据的构建，具体如下：The present invention further carries out the construction of subway map data according to lidar and camera, specifically as follows:

为地铁装配激光雷达和摄像头；利用装配有激光雷达和摄像头的地铁沿着行驶线路往返至少一次，采集沿途的摄像头图像数据和激光雷达点云数据。Install lidar and cameras for the subway; use the subway equipped with lidar and cameras to go back and forth along the driving line at least once, and collect camera image data and lidar point cloud data along the way.

由于激光雷达比较准确可靠，将利用激光雷达进行地图的构建，对于交叉轨道处和弯道等位置，将利用摄像头的图像数据进行逐一标定，以确定该位置行驶的正确性。摄像头图像数据和激光雷达点云数据都无法确定的路况，将利用人工进行手动标注。这样就形成了特定线路的地铁地图数据，确保地铁可以在无任何突发和异常情况下的行驶。Since lidar is relatively accurate and reliable, lidar will be used to construct maps. For locations such as cross tracks and curves, image data from cameras will be used to calibrate one by one to determine the correctness of driving at this location. Road conditions that cannot be determined by camera image data and lidar point cloud data will be manually marked. In this way, the subway map data of a specific line is formed to ensure that the subway can run without any sudden and abnormal conditions.

进一步的，将该方法应用于地铁自动驾驶中时，在行驶过程中，轨道上方会出现障碍物，此时会进行异常情况的检测，具体如下：Further, when this method is applied to the automatic driving of the subway, obstacles will appear above the track during the driving process, and abnormal conditions will be detected at this time, as follows:

通常对于可以造成轨道行驶安全的大型物体，如倒落的电线杆、穿梭的行人和牲畜，仅利用构建的地铁地图数据和行驶中实时采集的雷达点云数据是可以做出正确的判断的。对于弯道数据，如接近直角的左右转弯，激光雷达点云数据可能会误判前方有障碍物，但是由于已经构建好了地铁地图数据，可以避免这样的情况发生。Usually, for large objects that can cause rail safety, such as fallen utility poles, passing pedestrians and livestock, correct judgments can be made only by using the constructed subway map data and the real-time radar point cloud data collected during driving. For curve data, such as left and right turns close to right angles, lidar point cloud data may misjudge that there are obstacles ahead, but since the subway map data has been constructed, this situation can be avoided.

主要的异常来源于几点：下雪天大片的雪花出现，或者是悬浮于轨道之上的朔料袋、气球和风筝等物体的出现。这些物体不会对轨道行驶造成危险，但是激光雷达却可能产生误判。因此，此时需要结合结合地铁地图数据中的摄像头图像数据和当前摄像头所拍摄的障碍物图像数据，视觉特征进行辅助判断。视觉特征数据库还可以不断的进行更新，如每次出现的上述障碍物时，将不断的加入视觉特征数据库中，这样在下次出现，可以直接进行比对以便做出正确判断。其中的视觉特征数据库为地铁地图数据中的摄像头图像数据的集合。The main anomalies come from several points: the appearance of large snowflakes on snowy days, or the appearance of objects such as plastic bags, balloons and kites suspended above the orbit. These objects do not pose a danger to the track, but lidar may produce false positives. Therefore, at this time, it is necessary to combine the camera image data in the subway map data with the obstacle image data captured by the current camera, and visual features for auxiliary judgment. The visual feature database can also be continuously updated. For example, each time the above-mentioned obstacles appear, they will be continuously added to the visual feature database, so that they can be directly compared to make a correct judgment when they appear next time. The visual feature database is a collection of camera image data in the subway map data.

例如，轨道上方悬浮的气球，可以利用激光雷达检测出该物体的位置，包括该物体距离轨道的高度。由于激光雷达属于离散点扫描，可能无法判断该物体是否与轨道有连接，此时可以利用摄像头图像数据进行辅助判断，通常已经可以判断出该物体为悬浮物体。为了安全起见，将利用视觉数据库中存储的不断加入的轨道中出现的物体的视觉特征进行该物体的分类与识别，判断出该物体为无危害的悬浮物体，随着数据库的完善，可判断出该物体为气球。For example, a balloon suspended above the orbit can use lidar to detect the position of the object, including the height of the object from the orbit. Since the lidar is a discrete-point scan, it may not be possible to determine whether the object is connected to the orbit. At this time, the camera image data can be used for auxiliary judgment. Usually, the object can be judged as a suspended object. For the sake of safety, the visual features of objects appearing in the continuously added tracks stored in the visual database will be used to classify and identify the object, and it will be judged that the object is a harmless suspended object. With the improvement of the database, it can be judged The object is a balloon.

地铁自动驾驶中，最关注的一个主要问题就是障碍物检测，本发明以此为目的，融合视觉与激光雷达数据特征实现地铁的障碍物检测。本发明以激光雷达点云数据为主，以视觉特征数据为辅，对两者数据特征进行挖掘，检测障碍物。In the automatic driving of the subway, one of the most concerned issues is the obstacle detection. The present invention aims at this, and realizes the obstacle detection of the subway by integrating the features of vision and laser radar data. The present invention mainly uses laser radar point cloud data and visual feature data as a supplement, and mines the data features of the two data to detect obstacles.

显然，所描述的实施例仅仅是本发明一部分实施例，而不是全部的实施例。基于本发明中的实施例，本领域普通技术人员在没有做出创造性劳动前提下所获得的所有其他实施例，都属于本发明保护范围。Apparently, the described embodiments are only some of the embodiments of the present invention, but not all of them. Based on the embodiments of the present invention, all other embodiments obtained by persons of ordinary skill in the art without making creative efforts belong to the protection scope of the present invention.

Claims (4)

1. a kind of underground railway track obstacle detection method of fusion vision and laser radar data feature, which is characterized in that includingFollowing steps：

S1, laser radar and camera are assembled for subway；

S2, using the subway of laser radar and camera is equipped with along vehicle line back and forth at least once, acquire taking the photograph on the wayAs head image data and laser radar point cloud data；

S3, the structure that map is carried out using laser radar point cloud data；For at cross track and bend position, camera is utilizedImage data is demarcated one by one；

S4, the road conditions that can not be all determined for camera image data and laser radar point cloud data are carried out manually using artificialMark；Form the subway maps data of the vehicle line in S2.

2. detection method according to claim 1, which is characterized in that the camera image number in the subway maps dataAccording to set constitute visual signature database, the detection method further includes the constantly improve of visual signature database.

3. detection method according to claim 2, which is characterized in that the constantly improve of the visual signature database is specificProcess is：When judging to there is barrier to occur above track according to laser radar, in conjunction with the camera figure in subway map datumBarrier image data as data and captured by the preceding camera, judges whether final barrier causes underground railway track travelingDanger, and barrier image data is added in visual signature database.

4. a kind of method of subway automatic Pilot power, which is characterized in that using any detection methods of claim 1-3 intoThe detection of row underground railway track barrier.