CN111986209A - System and method for predicting train drivable area based on rail instance segmentation - Google Patents
System and method for predicting train drivable area based on rail instance segmentationDownload PDFInfo
- Publication number
- CN111986209A CN111986209ACN202010588137.9ACN202010588137ACN111986209ACN 111986209 ACN111986209 ACN 111986209ACN 202010588137 ACN202010588137 ACN 202010588137ACN 111986209 ACN111986209 ACN 111986209A
- Authority
- CN
- China
- Prior art keywords
- track
- image
- instance
- rail
- module
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
- G06T7/11—Region-based segmentation
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/22—Matching criteria, e.g. proximity measures
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/23—Clustering techniques
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20081—Training; Learning
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30204—Marker
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30248—Vehicle exterior or interior
- G06T2207/30252—Vehicle exterior; Vicinity of vehicle
- G06T2207/30256—Lane; Road marking
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02T—CLIMATE CHANGE MITIGATION TECHNOLOGIES RELATED TO TRANSPORTATION
- Y02T10/00—Road transport of goods or passengers
- Y02T10/10—Internal combustion engine [ICE] based vehicles
- Y02T10/40—Engine management systems
Landscapes
- Engineering & Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Computer Vision & Pattern Recognition (AREA)
- General Physics & Mathematics (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Bioinformatics & Computational Biology (AREA)
- Evolutionary Biology (AREA)
- Evolutionary Computation (AREA)
- Artificial Intelligence (AREA)
- General Engineering & Computer Science (AREA)
- Life Sciences & Earth Sciences (AREA)
- Train Traffic Observation, Control, And Security (AREA)
- Image Analysis (AREA)
Abstract
Translated fromChinese
Description
Translated fromChinese技术领域technical field
本发明涉及轨道预测技术领域，具体涉及基于铁轨实例分割预测列车可行驶区域系统和方法。The present invention relates to the technical field of track prediction, in particular to a system and method for predicting train drivable areas based on rail instance segmentation.
背景技术Background technique
轨道交通是指运营车辆需要在特定轨道上行驶的一类交通工具或运输系统，最典型的轨道交通就是由传统火车和标准铁路所组成的铁路系统，随着火车和铁路技术的多元化发展，轨道交通呈现出越来越多的类型，不仅遍布于长距离的陆地运输，也广泛运用于中短距离的城市公共交通中，轨道交通中，铁轨分布复杂多变，列车行驶至有道岔路段需要准确预测列车将要转入哪一条铁轨运行，即预测可行驶区域，现有技术方案分为两个阶段，首先分割轨道，再次基础上检测道岔，根据道岔信息预测列车可行驶区域，道岔检测高度依赖轨道分割的结果，由于轨道难以精确分割，道岔处铁轨分割粘连问题难以解决，对分析道岔信息带来一定难度，因此给预测可行驶区域带来一定困难。Rail transit refers to a type of vehicle or transportation system that requires running vehicles to run on specific tracks. The most typical rail transit is a railway system composed of traditional trains and standard railways. With the diversified development of train and railway technology, There are more and more types of rail transit, not only in long-distance land transportation, but also widely used in urban public transportation in medium and short distances. Accurately predict which rail the train will transfer to, that is, predict the drivable area. The existing technical solution is divided into two stages. First, the track is divided, and then the turnout is detected on the basis of the turnout, and the train drivable area is predicted according to the turnout information. As a result of track segmentation, due to the difficulty of accurate segmentation of the track, the problem of rail segmentation and adhesion at the turnout is difficult to solve, which brings certain difficulties to the analysis of the turnout information, and thus brings certain difficulties to the prediction of the drivable area.
发明内容SUMMARY OF THE INVENTION
本发明实施例提供了基于铁轨实例分割预测列车可行驶区域系统和方法，本发明基于深度学习，数据获取模块标注当前轨道为同一实例，标注阶段隐含了道岔信息，使用大量标注图片，铁轨实例分割模型能过通过端到端的方式学习到隐含的道岔信息，既减少计算量，又能提高性能，解决了目前列车运行轨道预测技术存在的道岔检测高度依赖轨道分割的结果，道岔处铁轨分割粘连问题难以解决，对分析道岔信息带来一定难度，预测可行驶区域准确率低的问题。The embodiment of the present invention provides a system and method for predicting train drivable area based on rail instance segmentation. The present invention is based on deep learning. The data acquisition module marks the current track as the same instance, and the switch information is implied in the labeling stage. The segmentation model can learn the implicit turnout information in an end-to-end manner, which not only reduces the amount of computation, but also improves performance, and solves the problem that the current train running track prediction technology is highly dependent on the track segmentation results, and the track segmentation at the turnout The problem of sticking is difficult to solve, which brings certain difficulties to the analysis of turnout information, and the problem of low accuracy in predicting the drivable area.
基于铁轨实例分割预测列车可行驶区域系统，包括：数据获取模块和铁轨实例分割模型；A system for predicting train drivable areas based on rail instance segmentation, including: a data acquisition module and a rail instance segmentation model;
数据获取模块，用于获取轨道图像信息并对图像进行标注并将数据传输到铁轨实例分割模型；The data acquisition module is used to acquire the track image information, label the image and transmit the data to the track instance segmentation model;
铁轨实例分割模型，用于接收数据获取模块传输的数据并对数据进行分析预测轨道的可行驶区域；The rail instance segmentation model is used to receive the data transmitted by the data acquisition module and analyze the data to predict the drivable area of the track;
进一步的，所述数据获取模块包括采集单元和标注单元，所述采集单元用于采集轨道的图像，所述标注单元用于在所述采集单元采集轨道的图像中标注轨道边缘的关键点并将这些关键点连接，形成完轨道的轮廓实例，形成一个封闭的多边形，对不同的实例给标识不同的ID；Further, the data acquisition module includes a collection unit and a labeling unit, the collection unit is used for collecting the image of the track, and the labeling unit is used for labeling the key points of the track edge in the image of the track collected by the collection unit. These key points are connected to form the contour instance of the track, forming a closed polygon, and identify different IDs for different instances;
进一步的，获取轨道的图像信息包括轨道的图像和轨道的道岔信息。Further, the acquired image information of the track includes the image of the track and the switch information of the track.
进一步的，铁轨实例分割模型包括图像二值化分割模块、轨道像素嵌入模块、聚类模块和检测预测模块，所述图像二值化分割模块用于将背景与车道线分离，得到车道线像素点，所述轨道像素嵌入模块用于得到图像像素点间的距离向量，所述聚类模块用于将得到所有图像像素点与实例ID相匹配，所述检测预测模块用于检测车道线实例，预测可行驶区域。Further, the railway track instance segmentation model includes an image binarization segmentation module, a track pixel embedding module, a clustering module and a detection and prediction module, and the image binarization segmentation module is used to separate the background from the lane lines to obtain lane line pixels. , the track pixel embedding module is used to obtain the distance vector between image pixels, the clustering module is used to match all obtained image pixels with instance IDs, the detection prediction module is used to detect lane line instances, predict drivable area.
进一步的，所述轨道像素嵌入模块用于基于距离度量学习的方法，对每一个像素点训练出不同的像素向量；Further, the track pixel embedding module is used for a method based on distance metric learning to train different pixel vectors for each pixel point;
其中，式中Lvar为方差损失，Ldist为距离损失；C为集群，即轨道的数量；c为聚类簇中心的个数；cA，cB表示不同的轨道；Nc为轨道像素的数量；xi为原始图像中轨道的像素坐标；μc为簇中心坐标的平均值，μcA，μcB对应不同轨道的簇中心的像素点坐标；δv，δd是设定的方差和距离阈值；方差损失使得xi向轨道的均值μc靠近，距离损失会使得各轨道的簇中心远离彼此。where Lvar is the variance loss, Ldist is the distance loss; C is the number of clusters, that is, the number of tracks; c is the number of cluster centers; cA, cB represent different tracks; Nc is the number of track pixels; xi is the pixel coordinates of the track in the original image; μc is the average of the cluster center coordinates, μcA , μcB correspond to the pixel coordinates of the cluster centers of different tracks; δv , δd are the set variance and distance thresholds ; the variance loss makesxi move closer to the mean μc of the orbits, and the distance loss makes the cluster centers of the orbits move away from each other.
第二方面，本发明实施例提供基于铁轨实例分割预测列车可行驶区域方法，包括以下步骤：In a second aspect, an embodiment of the present invention provides a method for predicting a train's drivable area based on rail instance segmentation, including the following steps:
S1，采集标注，采集单元采集轨道的图像，标注单元在采集单元采集轨道的图像中标注轨道边缘的关键点并将这些关键点连接，形成完轨道的轮廓实例，形成一个封闭的多边形，对不同的实例给标识不同的ID并将实例数据输入到铁轨实例分割模型；S1, collect and label, the collection unit collects the image of the track, the labeling unit labels the key points of the track edge in the image of the track collected by the collection unit and connects these key points to form an outline instance of the track and form a closed polygon. Instances are identified with different IDs and the instance data is input into the rail instance segmentation model;
S2，分析预测，将轨道图像输入二值化分割模块，将背景与车道线分离，得到车道线像素点的二值图；将得到的二值图输入到像素嵌入模块，得到图像像素点间的距离向量，聚类模块将得到的像素点间的距离向量采用DBSCAN算法进行聚类，输出结果为图像中每一个像素对应的实例ID，形成实例分割图；检测预测模块检测到不同的车道线实例后，采用多项式拟合像素点，参数化每一条轨道，因为铁轨的采集图像比较特殊，当前轨道在图像的最下方基本处于同一位置，可以确定当前的左右轨道，根据实例分割结果确定可行驶区域。S2, analyze and predict, input the track image into the binarization segmentation module, separate the background from the lane line, and obtain a binary image of the pixel points of the lane line; input the obtained binary image into the pixel embedding module to obtain the image pixel points. Distance vector, the clustering module uses the DBSCAN algorithm to cluster the obtained distance vector between pixels, and the output result is the instance ID corresponding to each pixel in the image, forming an instance segmentation map; the detection and prediction module detects different lane line instances After that, use polynomial to fit the pixel points and parameterize each track. Because the collected image of the railway track is special, the current track is basically at the same position at the bottom of the image, and the current left and right tracks can be determined, and the drivable area can be determined according to the instance segmentation result. .
本发明实施例提供的上述技术方案的有益效果至少包括：本发明基于深度学习，通过数据获取模块采集和标注轨道的图像数据，标注当前轨道为同一实例，标注阶段隐含了道岔信息，使用大量标注图片，将得到的实例输入到铁轨实例分割模型通过端到端的方式学习到隐含的道岔信息，既减少计算量，又能提高性能，解除了预测结果依赖于铁轨分割结果，特别是道岔路段结果的限制，基于端到端的方式直接预测可行驶区域，通过大量的标注图片从而提高准确率的效果。The beneficial effects of the above technical solutions provided by the embodiments of the present invention at least include: the present invention is based on deep learning, the image data of the track is collected and marked by the data acquisition module, the current track is marked as the same instance, the switch information is implied in the marking stage, and a large number of Annotate the picture, input the obtained instance into the rail instance segmentation model, and learn the implicit switch information in an end-to-end manner, which not only reduces the amount of calculation, but also improves the performance, and relieves the prediction result from the rail segmentation result, especially the switch section. Due to the limitation of the results, the drivable area is directly predicted based on the end-to-end method, and the accuracy is improved by a large number of labeled pictures.
本发明通过物联网技术，将多种传感器和视频监控等设备应用于认养农产品，构成认养农产品监控系统，通过数据采集传输系统，。The invention uses the Internet of Things technology to apply various sensors, video monitoring and other equipment to the adoption of agricultural products to form a monitoring system for the adoption of agricultural products, through a data collection and transmission system.
本发明的其它特征和优点将在随后的说明书中阐述，并且，部分地从说明书中变得显而易见，或者通过实施本发明而了解。本发明的目的和其他优点可通过在所写的说明书、权利要求书、以及附图中所特别指出的结构来实现和获得。Other features and advantages of the present invention will be set forth in the description which follows, and in part will be apparent from the description, or may be learned by practice of the invention. The objectives and other advantages of the invention may be realized and attained by the structure particularly pointed out in the written description, claims, and drawings.
下面通过附图和实施例，对本发明的技术方案做进一步的详细描述。The technical solutions of the present invention will be further described in detail below through the accompanying drawings and embodiments.
附图说明Description of drawings
附图用来提供对本发明的进一步理解，并且构成说明书的一部分，与本发明的实施例一起用于解释本发明，并不构成对本发明的限制。在附图中：The accompanying drawings are used to provide a further understanding of the present invention, and constitute a part of the specification, and are used to explain the present invention together with the embodiments of the present invention, and do not constitute a limitation to the present invention. In the attached image:
图1为本发明实施例公开的轨道结构示意图；1 is a schematic diagram of a track structure disclosed in an embodiment of the present invention;
图2为本发明实施例公开的实例分割结构示意图；2 is a schematic diagram of an instance segmentation structure disclosed in an embodiment of the present invention;
图3为本发明实施例公开的不同轨道对应的mask图像；3 is a mask image corresponding to different tracks disclosed in an embodiment of the present invention;
图4为本发明实施例公开的实例分割效果图；4 is an example segmentation effect diagram disclosed by an embodiment of the present invention;
图5为本发明实施例公开的基于铁轨实例分割预测列车可行驶区域系统结构示意图；5 is a schematic structural diagram of a system for predicting train drivable areas based on rail instance segmentation disclosed in an embodiment of the present invention;
图6为本发明实施例公开的基于铁轨实例分割预测列车可行驶区域方法流程示意图。FIG. 6 is a schematic flowchart of a method for segmenting and predicting a train's drivable area based on a rail instance disclosed in an embodiment of the present invention.
附图标记：Reference number:
1-数据获取模块；101-采集单元；102-标注单元；2-铁轨实例分割模型；201-图像二值化分割模块；202-轨道像素嵌入模块；203-聚类模块；204-检测模块。1-data acquisition module; 101-collection unit; 102-marking unit; 2-railway instance segmentation model; 201-image binarization segmentation module; 202-track pixel embedding module; 203-clustering module; 204-detection module.
具体实施例specific embodiment
下面将参照附图更详细地描述本公开的示例性实施例。虽然附图中显示了本公开的示例性实施例，然而应当理解，可以以各种形式实现本公开而不应被这里阐述的实施例所限制。相反，提供这些实施例是为了能够更透彻地理解本公开，并且能够将本公开的范围完整的传达给本领域的技术人员。Exemplary embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. While exemplary embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure may be embodied in various forms and should not be limited by the embodiments set forth herein. Rather, these embodiments are provided so that the present disclosure will be more thoroughly understood, and will fully convey the scope of the present disclosure to those skilled in the art.
实施例一Example 1
如图1-5所示，本发明实施例提供基于铁轨实例分割预测列车可行驶区域系统，包括：数据获取模块1和铁轨实例分割模型2；As shown in Figures 1-5, an embodiment of the present invention provides a system for predicting train drivable areas based on rail instance segmentation, including: a data acquisition module 1 and a rail instance segmentation model 2;
数据获取模块1，用于获取轨道图像信息并对图像进行标注并将数据传输到铁轨实例分割模型2，所述数据获取模块1包括采集单元101和标注单元102，所述采集单元101用于采集轨道的图像，获取轨道的图像信息包括轨道的图像和轨道的道岔信息，所述标注单元102用于在所述采集单元101采集轨道的图像中标注轨道边缘的关键点并将这些关键点连接，形成完轨道的轮廓实例，形成一个封闭的多边形，对不同的实例给标识不同的ID；The data acquisition module 1 is used for acquiring track image information, marking the images and transmitting the data to the rail instance segmentation model 2. The data acquisition module 1 includes a collection unit 101 and a labeling unit 102, and the collection unit 101 is used for collecting The image of the track, the image information of the track obtained includes the image of the track and the switch information of the track, and the labeling unit 102 is used to label the key points of the track edge in the image of the track collected by the collecting unit 101 and connect these key points, After the contour instance of the track is formed, a closed polygon is formed, and different IDs are identified for different instances;
如图1-4所示，具体的：As shown in Figure 1-4, the specific:
(1)沿着轨道边缘选取关键点，并将这些点连接，直到勾勒完轨道的轮廓，形成一个封闭的多边形，并且不同的轨道即不同的实例给定不同的ID，如图1所示，轨道从左至右依次为第一实例，第二实例，第三实例和第四实例；(1) Select key points along the edge of the track and connect these points until the outline of the track is drawn to form a closed polygon, and different tracks, i.e. different instances, are given different IDs, as shown in Figure 1, The tracks are the first instance, the second instance, the third instance and the fourth instance from left to right;
(2)将关键点的坐标按照ID，分类保存在json文件中并导出；(2) The coordinates of the key points are classified and saved in the json file according to the ID and exported;
(3)根据json文件中的点坐标，按照ID为每一个轨道实例生成一个mask图像，例如，图1中有四条轨道，即四个实例，则需要将图像与图2中四个mask图像一起输入模型进行训练，在mask图像中，轨道的区域像素值取255，其他背景区域像素值均为0，按照实例的ID给mask图像命名。(3) According to the point coordinates in the json file, a mask image is generated for each track instance according to the ID. For example, if there are four tracks in Figure 1, that is, four instances, the image needs to be combined with the four mask images in Figure 2. Input the model for training. In the mask image, the pixel value of the track area is 255, and the pixel value of other background areas is 0. Name the mask image according to the ID of the instance.
铁轨实例分割模型2，用于接收数据获取模块1传输的数据并对数据进行分析预测轨道的可行驶区域铁轨实例分割模型2包括图像二值化分割模块201、轨道像素嵌入模块202、聚类模块203和检测预测模块204，所述图像二值化分割模块201用于将背景与车道线分离，得到车道线像素点，所述轨道像素嵌入模块202用于得到图像像素点间的距离向量，所述聚类模块203用于将得到所有图像像素点与实例ID相匹配，所述检测预测模块204用于检测车道线实例，预测可行驶区域；The rail instance segmentation model 2 is used to receive the data transmitted by the data acquisition module 1 and analyze the data to predict the drivable area of the track. The rail instance segmentation model 2 includes an image binarization segmentation module 201, a track pixel embedding module 202, and a clustering module. 203 and the detection and prediction module 204, the image binarization segmentation module 201 is used to separate the background from the lane line to obtain the lane line pixels, and the track pixel embedding module 202 is used to obtain the distance vector between the image pixels, so The clustering module 203 is used to match all the image pixels obtained with the instance ID, and the detection and prediction module 204 is used to detect the lane line instance and predict the drivable area;
具体的：specific:
(1)将原始图像经过缩放后载入到训练好的二值化分割模块，输出的结果是轨道像素与背景像素分隔开的二值图，在模型训练阶段，因为图像中轨道和背景的像素数据分布不均匀，采用标准交叉熵损失函数对二值化分割模块进行训练，可以较快更新网络参数数值；(1) The original image is scaled and loaded into the trained binarization segmentation module. The output result is a binary image separated by track pixels and background pixels. In the model training stage, because the track and background in the image are separated The distribution of pixel data is uneven, and the standard cross-entropy loss function is used to train the binarized segmentation module, which can update the network parameter values quickly;
(2)将步骤(1)得到的二值图输入像素嵌入模块，输出是图片中每一个像素点间的距离向量，在模型训练阶段，为了区分得到的像素点属于哪条轨道，使用基于距离度量学习的方法，对每一个像素点训练出不同的像素向量；(2) Input the binary image obtained in step (1) into the pixel embedding module, and the output is the distance vector between each pixel in the image. In the model training stage, in order to distinguish which track the obtained pixel belongs to, the distance-based The metric learning method trains a different pixel vector for each pixel point;
其中，式中Lvar为方差损失，Ldist为距离损失；C为集群，即轨道的数量；c为聚类簇中心的个数；cA，cB表示不同的轨道；Nc为轨道像素的数量；xi为原始图像中轨道的像素坐标；μc为簇中心坐标的平均值，μcA，μcB对应不同轨道的簇中心的像素点坐标；δv，δd是设定的方差和距离阈值；方差损失使得xi向轨道的均值μc靠近，距离损失会使得各轨道的簇中心远离彼此；where Lvar is the variance loss, Ldist is the distance loss; C is the number of clusters, that is, the number of tracks; c is the number of cluster centers; cA, cB represent different tracks; Nc is the number of track pixels; xi is the pixel coordinates of the track in the original image; μc is the average of the cluster center coordinates, μcA , μcB correspond to the pixel coordinates of the cluster centers of different tracks; δv , δd are the set variance and distance thresholds ; the variance loss makes xi approach the mean μc of the orbits, and the distance loss makes the cluster centers of the orbits move away from each other;
方差损失使得xi向轨道的均值μc靠近，距离损失会使得各轨道的簇中心远离彼此；The variance loss makes xi approach the mean μc of the orbits, and the distance loss makes the cluster centers of the orbits move away from each other;
(3)基于步骤(2)得到的损失函数，把得到的像素点间的距离向量采用DBSCAN算法进行聚类。输出结果为图像中每一个像素对应的实例id，即实例分割图：(3) Based on the loss function obtained in step (2), the obtained distance vector between pixel points is clustered using the DBSCAN algorithm. The output result is the instance id corresponding to each pixel in the image, that is, the instance segmentation map:
a.给定一个距离阈值和一个最小样本个数；a. Given a distance threshold and a minimum number of samples;
b.若某个像素点周围满足距离阈值的像素点个数大于最小样本个数，则该点称为核心对象。遍历所有像素点，找到核心对象的集合；b. If the number of pixels around a pixel that meets the distance threshold is greater than the minimum number of samples, the point is called a core object. Traverse all pixels to find the set of core objects;
c.选取一个核心对象做为簇中心，找到所有密度可达的像素点并生成聚类簇；c. Select a core object as the cluster center, find all density-reachable pixels and generate clusters;
d.从剩余的核心对象中移除找到的密度可达的样本；d. Remove the found density-reachable samples from the remaining core objects;
e.从更新后的核心对象中重复步骤c和步骤d，直到核心对象都被遍历；即所有的像素都分配给了对应的轨道。e. Repeat steps c and d from the updated core objects until the core objects are all traversed; that is, all pixels are assigned to the corresponding tracks.
(4)当检测预测模块204检测到不同的车道线实例后，采用多项式拟合像素点，参数化每一条轨道，因为铁轨的采集图像比较特殊，当前轨道在图像的最下方基本处于同一位置，因此可以确定当前的左右轨道，再根据实例分割结果可以确定可行驶区域。(4) When the detection and prediction module 204 detects different lane line instances, it adopts polynomial fitting pixel points to parameterize each track. Because the collected image of the railway track is relatively special, the current track is basically at the same position at the bottom of the image, Therefore, the current left and right tracks can be determined, and then the drivable area can be determined according to the instance segmentation result.
本发明克服了现有技术高度依赖轨道分割结果，导致由于轨道难以精确分割，道岔处铁轨分割粘连问题难以解决，本发明基于深度学习，通过数据获取模块1采集和标注轨道的图像数据，标注当前轨道为同一实例，标注阶段隐含了道岔信息，使用大量标注图片，将得到的实例输入到铁轨实例分割模型2通过端到端的方式学习到隐含的道岔信息，既减少计算量，又能提高性能，解除了预测结果依赖于铁轨分割结果，特别是道岔路段结果的限制，基于端到端的方式直接预测可行驶区域，通过大量的标注图片从而提高准确率的效果。The present invention overcomes the fact that the prior art is highly dependent on the track segmentation result, which leads to the difficulty of accurate segmentation of the track and the difficulty of solving the problem of rail segmentation and adhesion at the switch. The track is the same instance, and the turnout information is implied in the labeling stage. A large number of labeled pictures are used, and the obtained instance is input into the rail instance segmentation model. The implicit turnout information is learned in an end-to-end manner, which not only reduces the amount of computation, but also improves the The performance is relieved of the limitation that the prediction results depend on the results of rail segmentation, especially the results of turnouts, and the drivable area is directly predicted based on an end-to-end method, and the accuracy is improved by a large number of labeled pictures.
实施例二Embodiment 2
本发明实施例还公开了基于铁轨实例分割预测列车可行驶区域方法，如图1-6，包括以下步骤：The embodiment of the present invention also discloses a method for predicting train drivable area based on rail instance segmentation, as shown in Figures 1-6, including the following steps:
S1，采集标注，采集单元101采集轨道的图像，标注单元102在采集单元101采集轨道的图像中标注轨道边缘的关键点并将这些关键点连接，形成完轨道的轮廓实例，形成一个封闭的多边形，对不同的实例给标识不同的ID并将实例数据输入到铁轨实例分割模型2；S1, collect and mark, the collecting unit 101 collects the image of the track, the marking unit 102 marks the key points of the edge of the track in the image of the track collected by the collecting unit 101 and connects these key points to form a contour instance of the track and form a closed polygon , identify different instances with different IDs and input the instance data into the rail instance segmentation model 2;
具体的，采集单元101采集轨道的图像数据，标注单元102对采集的图像进行标注，沿着轨道边缘选取关键点，并将这些点连接，直到勾勒完轨道的轮廓，形成一个封闭的多边形，并且不同的轨道即不同的实例给定不同的ID，按照ID为每一个轨道实例生成一个mask图像；Specifically, the collecting unit 101 collects image data of the track, the labeling unit 102 annotates the collected image, selects key points along the edge of the track, and connects these points until the outline of the track is drawn to form a closed polygon, and Different tracks, that is, different instances are given different IDs, and a mask image is generated for each track instance according to the ID;
S2，分析预测，将轨道图像输入二值化分割模块，将背景与车道线分离，得到车道线像素点的二值图；将得到的二值图输入到像素嵌入模块，得到图像像素点间的距离向量，聚类模块203将得到的像素点间的距离向量采用DBSCAN算法进行聚类，输出结果为图像中每一个像素对应的实例ID，形成实例分割图；检测预测模块204检测到不同的车道线实例后，采用多项式拟合像素点，参数化每一条轨道，因为铁轨的采集图像比较特殊，当前轨道在图像的最下方基本处于同一位置，可以确定当前的左右轨道，根据实例分割结果确定可行驶区域；S2, analyze and predict, input the track image into the binarization segmentation module, separate the background from the lane line, and obtain a binary image of the pixel points of the lane line; input the obtained binary image into the pixel embedding module to obtain the image pixel points. The distance vector, the clustering module 203 uses the DBSCAN algorithm to cluster the obtained distance vector between the pixels, and the output result is the instance ID corresponding to each pixel in the image, forming an instance segmentation map; the detection and prediction module 204 detects different lanes After the line instance, a polynomial is used to fit the pixel points, and each track is parameterized. Because the collected image of the railway track is special, the current track is basically at the same position at the bottom of the image, and the current left and right tracks can be determined. driving area;
具体的，二值化分割模块在模型训练阶段，因为图像中轨道和背景的像素数据分布不均匀，采用标准交叉熵损失函数对二值化分割模块进行训练，将原始图像经过缩放后载入到训练好的二值化分割模块，输出的结果是轨道像素与背景像素分隔开的二值图；将二值图输入像素嵌入模块，输出是图片中每一个像素点间的距离向量；聚类模块203把得到的像素点间的距离向量采用DBSCAN算法进行聚类。输出结果为图像中每一个像素对应的实例id，即实例分割图，当检测预测模块204检测到不同的车道线实例后，采用多项式拟合像素点，参数化每一条轨道，因为铁轨的采集图像比较特殊，当前轨道在图像的最下方基本处于同一位置，因此可以确定当前的左右轨道，再根据实例分割结果可以确定可行驶区域。Specifically, in the model training stage, the binarization segmentation module uses the standard cross entropy loss function to train the binarization segmentation module because the pixel data of the track and the background in the image are unevenly distributed, and the original image is scaled and loaded into the The trained binarization segmentation module outputs a binary image separated by track pixels and background pixels; the binary image is input into the pixel embedding module, and the output is the distance vector between each pixel in the image; clustering Module 203 uses the DBSCAN algorithm to cluster the obtained distance vectors between pixels. The output result is the instance id corresponding to each pixel in the image, that is, the instance segmentation map. When the detection and prediction module 204 detects different lane line instances, it uses a polynomial to fit the pixel points and parameterize each track, because the collected images of the rails It is special, the current track is basically at the same position at the bottom of the image, so the current left and right tracks can be determined, and then the drivable area can be determined according to the instance segmentation result.
本实施例公开的基于铁轨实例分割预测列车可行驶区域方法，基于深度学习，通过数据获取模块1采集和标注轨道的图像数据，标注当前轨道为同一实例，标注阶段中包含了道岔信息，使用大量标注图片，将得到的实例输入到铁轨实例分割模型2通过端到端的方式学习到隐含的道岔信息，既减少计算量，又能提高性能，解除了预测结果依赖于铁轨分割结果，特别是道岔路段结果的限制，基于端到端的方式直接预测可行驶区域，通过大量的标注图片从而提高准确率的效果。The method disclosed in this embodiment based on rail instance segmentation and prediction of train drivable area is based on deep learning, and the image data of the track is collected and marked by the data acquisition module 1, and the current track is marked as the same instance, and the switch information is included in the marking stage. Annotate the picture, and input the obtained instance into the rail instance segmentation model 2 to learn the implicit switch information in an end-to-end manner, which not only reduces the amount of calculation, but also improves performance, and relieves the prediction result from being dependent on the rail segmentation results, especially the switch. Due to the limitation of road section results, the drivable area is directly predicted based on an end-to-end method, and the accuracy is improved by a large number of labeled pictures.
应该明白，公开的过程中的步骤的特定顺序或层次是示例性方法的实例。基于设计偏好，应该理解，过程中的步骤的特定顺序或层次可以在不脱离本公开的保护范围的情况下得到重新安排。所附的方法权利要求以示例性的顺序给出了各种步骤的要素，并且不是要限于所述的特定顺序或层次。It is understood that the specific order or hierarchy of steps in the disclosed processes is an example of a sample approach. Based upon design preferences, it is understood that the specific order or hierarchy of steps in the processes may be rearranged without departing from the scope of the present disclosure. The accompanying method claims present elements of the various steps in a sample order, and are not meant to be limited to the specific order or hierarchy presented.
在上述的详细描述中，各种特征一起组合在单个的实施方案中，以简化本公开。不应该将这种公开方法解释为反映了这样的意图，即，所要求保护的主题的实施方案需要清楚地在每个权利要求中所陈述的特征更多的特征。相反，如所附的权利要求书所反映的那样，本发明处于比所公开的单个实施方案的全部特征少的状态。因此，所附的权利要求书特此清楚地被并入详细描述中，其中每项权利要求独自作为本发明单独的优选实施方案。In the foregoing Detailed Description, various features are grouped together in a single embodiment for the purpose of simplifying the disclosure. This method of disclosure should not be construed as reflecting an intention that embodiments of the claimed subject matter require more features than are expressly recited in each claim. Rather, as the following claims reflect, present invention lies in less than all features of a single disclosed embodiment. Thus, the following claims are hereby expressly incorporated into the Detailed Description, with each claim standing on its own as a separate preferred embodiment of this invention.
本领域技术人员还应当理解，结合本文的实施例描述的各种说明性的逻辑框、模块、电路和算法步骤均可以实现成电子硬件、计算机软件或其组合。为了清楚地说明硬件和软件之间的可交换性，上面对各种说明性的部件、框、模块、电路和步骤均围绕其功能进行了一般地描述。至于这种功能是实现成硬件还是实现成软件，取决于特定的应用和对整个系统所施加的设计约束条件。熟练的技术人员可以针对每个特定应用，以变通的方式实现所描述的功能，但是，这种实现决策不应解释为背离本公开的保护范围。Those skilled in the art will also appreciate that the various illustrative logical blocks, modules, circuits, and algorithm steps described in connection with the embodiments herein may be implemented as electronic hardware, computer software, or combinations thereof. To clearly illustrate this interchangeability of hardware and software, various illustrative components, blocks, modules, circuits, and steps have been described above generally in terms of their functionality. Whether this functionality is implemented as hardware or software depends on the specific application and design constraints imposed on the overall system. Skilled artisans may implement the described functionality in varying ways for each particular application, however, such implementation decisions should not be interpreted as a departure from the scope of the present disclosure.
结合本文的实施例所描述的方法或者算法的步骤可直接体现为硬件、由处理器执行的软件模块或其组合。软件模块可以位于RAM存储器、闪存、ROM存储器、EPROM存储器、EEPROM存储器、寄存器、硬盘、移动磁盘、CD-ROM或者本领域熟知的任何其它形式的存储介质中。一种示例性的存储介质连接至处理器，从而使处理器能够从该存储介质读取信息，且可向该存储介质写入信息。当然，存储介质也可以是处理器的组成部分。处理器和存储介质可以位于ASIC中。该ASIC可以位于用户终端中。当然，处理器和存储介质也可以作为分立组件存在于用户终端中。The steps of a method or algorithm described in connection with the embodiments herein may be directly embodied in hardware, a software module executed by a processor, or a combination thereof. A software module may reside in RAM memory, flash memory, ROM memory, EPROM memory, EEPROM memory, registers, hard disk, removable disk, CD-ROM, or any other form of storage medium known in the art. An exemplary storage medium is coupled to the processor such that the processor can read information from, and write information to, the storage medium. Of course, the storage medium can also be an integral part of the processor. The processor and storage medium may reside in an ASIC. The ASIC may be located in the user terminal. Of course, the processor and the storage medium may also exist in the user terminal as discrete components.
对于软件实现，本申请中描述的技术可用执行本申请所述功能的模块(例如，过程、函数等)来实现。这些软件代码可以存储在存储器单元并由处理器执行。存储器单元可以实现在处理器内，也可以实现在处理器外，在后一种情况下，它经由各种手段以通信方式耦合到处理器，这些都是本领域中所公知的。For a software implementation, the techniques described in this application may be implemented in modules (eg, procedures, functions, etc.) that perform the functions described in this application. These software codes may be stored in a memory unit and executed by a processor. The memory unit may be implemented within the processor or external to the processor, in which case it is communicatively coupled to the processor via various means, as is known in the art.
上文的描述包括一个或多个实施例的举例。当然，为了描述上述实施例而描述部件或方法的所有可能的结合是不可能的，但是本领域普通技术人员应该认识到，各个实施例可以做进一步的组合和排列。因此，本文中描述的实施例旨在涵盖落入所附权利要求书的保护范围内的所有这样的改变、修改和变型。此外，就说明书或权利要求书中使用的术语“包含”，该词的涵盖方式类似于术语“包括”，就如同“包括，”在权利要求中用作衔接词所解释的那样。此外，使用在权利要求书的说明书中的任何一个术语“或者”是要表示“非排它性的或者”。The above description includes examples of one or more embodiments. Of course, it is not possible to describe all possible combinations of components or methods in order to describe the above embodiments, but one of ordinary skill in the art will recognize that further combinations and permutations of the various embodiments are possible. Accordingly, the embodiments described herein are intended to cover all such changes, modifications and variations that fall within the scope of the appended claims. Furthermore, with respect to the term "comprising," as used in the specification or claims, the word is encompassed in a manner similar to the term "comprising," as if "comprising," were interpreted as a conjunction in the claims. Furthermore, any use of the term "or" in the specification of the claims is intended to mean a "non-exclusive or."
Claims (6)
Translated fromChinese
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010588137.9ACN111986209B (en) | 2020-06-24 | 2020-06-24 | System and method for predicting train drivable area based on rail instance segmentation |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010588137.9ACN111986209B (en) | 2020-06-24 | 2020-06-24 | System and method for predicting train drivable area based on rail instance segmentation |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111986209Atrue CN111986209A (en) | 2020-11-24 |
| CN111986209B CN111986209B (en) | 2025-06-17 |
Family
ID=73441828
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010588137.9AActiveCN111986209B (en) | 2020-06-24 | 2020-06-24 | System and method for predicting train drivable area based on rail instance segmentation |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111986209B (en) |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109740465A (en)* | 2018-12-24 | 2019-05-10 | 南京理工大学 | A Lane Line Detection Algorithm Based on Instance Segmentation Neural Network Framework |
| CN109948690A (en)* | 2019-03-14 | 2019-06-28 | 西南交通大学 | A high-speed rail scene perception method based on deep learning and structural information |
| US20190384304A1 (en)* | 2018-06-13 | 2019-12-19 | Nvidia Corporation | Path detection for autonomous machines using deep neural networks |
| CN111178245A (en)* | 2019-12-27 | 2020-05-19 | 深圳佑驾创新科技有限公司 | Lane line detection method, lane line detection device, computer device, and storage medium |
- 2020
- 2020-06-24CNCN202010588137.9Apatent/CN111986209B/enactiveActive
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20190384304A1 (en)* | 2018-06-13 | 2019-12-19 | Nvidia Corporation | Path detection for autonomous machines using deep neural networks |
| CN109740465A (en)* | 2018-12-24 | 2019-05-10 | 南京理工大学 | A Lane Line Detection Algorithm Based on Instance Segmentation Neural Network Framework |
| CN109948690A (en)* | 2019-03-14 | 2019-06-28 | 西南交通大学 | A high-speed rail scene perception method based on deep learning and structural information |
| CN111178245A (en)* | 2019-12-27 | 2020-05-19 | 深圳佑驾创新科技有限公司 | Lane line detection method, lane line detection device, computer device, and storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111986209B (en) | 2025-06-17 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111310583B (en) | Vehicle abnormal behavior identification method based on improved long-term and short-term memory network | |
| CN109840523B (en) | A recognition method of urban rail train numbers based on image processing | |
| WO2019001044A1 (en) | Mobile phone data-based vehicle model identification method and device | |
| CN113160575A (en) | Traffic violation detection method and system for non-motor vehicles and drivers | |
| CN103150903B (en) | Video vehicle detection method for adaptive learning | |
| CN105513354A (en) | Video-based urban road traffic jam detecting system | |
| CN110910353B (en) | A method and system for detecting industrial false defects | |
| CN112309126B (en) | License plate detection method and device, electronic equipment and computer readable storage medium | |
| CN113610770B (en) | License plate recognition method, device and equipment | |
| CN107909601A (en) | A kind of shipping anti-collision early warning video detection system and detection method suitable for navigation mark | |
| CN102426785A (en) | Traffic flow information perception method and system based on contour and local feature points | |
| CN110516633A (en) | A kind of method for detecting lane lines and system based on deep learning | |
| CN113538585B (en) | High-precision multi-target intelligent identification, positioning and tracking method and system based on unmanned aerial vehicle | |
| CN117671972B (en) | Vehicle speed detection method and device for slow traffic system | |
| CN111414878B (en) | Social attribute analysis and image processing method and device for land parcels | |
| CN115050028A (en) | Sample vehicle license plate detection method in severe weather | |
| CN105740828A (en) | A Stop Line Detection Method Based on Fast Marker Connectivity | |
| CN113674329A (en) | Vehicle driving behavior detection method and system | |
| CN111986209A (en) | System and method for predicting train drivable area based on rail instance segmentation | |
| CN114005097A (en) | Train operation environment real-time detection method and system based on image semantic segmentation | |
| CN117782070A (en) | Railway train visual inertial positioning method and system based on kilometer sign information assistance | |
| CN117456482A (en) | Abnormal event identification method and system for traffic monitoring scene | |
| CN106157388A (en) | Local high-efficiency encoding driving record method and system | |
| CN116452943A (en) | A Water Level Recognition Method Based on Convolutional Neural Network for Image Recognition | |
| CN114429621A (en) | UFSA algorithm-based improved lane line intelligent detection method |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |