技术领域technical field
本发明涉及路面检测技术领域，尤其涉及一种基于精密三维的路面磨耗检测方法。The invention relates to the technical field of road surface detection, in particular to a precision three-dimensional based road wear detection method.
背景技术Background technique
路面磨耗是路面技术状况评价的一个重要指标。现有技术中，主要使用的路面磨耗评价方法为：首先通过构造深度仪获得轮迹中央的纵断面曲线，再通过理论模型推算路面构造深度，最后通过同时测量的左轮迹带、右轮迹带、车道中线三条测线的构造深度计算路面磨耗。构造深度仪通常是由一个加速度传感器和一个点激光测距传感器组成。点激光测距传感器测量车辆与路面的距离，加速度传感器通过两次积分计算该车辆上下振动的距离，从理论上获得路面的纵向高程变化，即纵断面曲线。Pavement wear is an important index for evaluating the technical condition of pavement. In the prior art, the road wear evaluation method mainly used is as follows: firstly, obtain the longitudinal section curve in the center of the wheel track through the structure depth gauge, then calculate the road surface structure depth through the theoretical model, and finally use the simultaneously measured left wheel track and right wheel track The road wear is calculated based on the structural depth of the three measuring lines of the center line of the lane. A construction depth gauge usually consists of an acceleration sensor and a point laser ranging sensor. The point laser ranging sensor measures the distance between the vehicle and the road surface, and the acceleration sensor calculates the distance of the vehicle's up and down vibration through two integrations, and theoretically obtains the longitudinal elevation change of the road surface, that is, the profile curve.
目前路面磨耗测量结果受驾驶人员的行车轨迹影响，特别是对路况较差的路段，为了保障自身行车的舒适性和安全性，行车过程中将会故意避开路况较差的轮迹带位置，进而导致路面磨耗测量结果不准确。At present, the measurement results of road surface wear are affected by the driving trajectory of the driver, especially for road sections with poor road conditions. This leads to inaccurate measurement results of pavement wear.
发明内容Contents of the invention
本发明提供一种基于精密三维的路面磨耗检测方法，用以解决现有技术中路面磨耗测量结果不准确的问题。The invention provides a precise three-dimensional road wear detection method to solve the problem of inaccurate road wear measurement results in the prior art.
本发明提供一种基于精密三维的路面磨耗检测方法，包括：The present invention provides a precise three-dimensional road wear detection method, including:
接收线扫描三维测量传感器获取的道路上各测点的原始路面高程数据和原始路面灰度数据，所述线扫描三维测量传感器的测量范围覆盖整个车道宽度；Receiving the original road surface elevation data and the original road surface grayscale data of each measuring point on the road obtained by the line-scanning three-dimensional measurement sensor, the measurement range of the line-scan three-dimensional measurement sensor covers the entire lane width;
从原始路面高程数据和原始路面灰度数据中确定车道线位置，并提取车道范围内的目标路面高程数据和目标路面灰度数据；Determine the position of the lane line from the original road surface elevation data and the original road surface grayscale data, and extract the target road surface elevation data and target road surface grayscale data within the lane range;
基于所述目标路面高程数据和目标路面灰度数据，重建路面三维点云数据；Reconstructing three-dimensional point cloud data of the road surface based on the elevation data of the target road surface and the gray scale data of the target road surface;
基于重建的路面三维点云数据，结合预设的路面构造深度计算模型，确定路面全幅构造深度；Based on the reconstructed 3D point cloud data of the pavement, combined with the preset pavement structure depth calculation model, determine the full pavement structure depth;
基于所述车道线位置和所述路面全幅构造深度，确定左轮迹带、右轮迹带和车道中线的位置；Based on the position of the lane line and the depth of the overall structure of the road surface, determine the positions of the left wheel mark, the right wheel mark and the center line of the lane;
基于所述路面全幅构造深度，以及左轮迹带、右轮迹带和车道中线各自位置，计算路面磨耗。The pavement wear is calculated based on the overall construction depth of the pavement, and the respective positions of the left wheel trail, the right wheel trail, and the center line of the lane.
根据本发明提供的一种基于精密三维的路面磨耗检测方法，从原始路面高程数据和原始路面灰度数据中确定车道线位置，并提取车道范围内的目标路面高程数据和目标路面灰度数据，包括：According to a precise three-dimensional road surface wear detection method provided by the present invention, the position of the lane line is determined from the original road surface elevation data and the original road surface grayscale data, and the target road surface elevation data and target road surface grayscale data within the range of the lane are extracted, include:
基于所述原始路面高程数据，利用路面车道线的高程特征和几何尺寸特征，标记潜在的车道线第一区域；Based on the original road surface elevation data, using the elevation feature and geometric dimension feature of the road surface lane line to mark the potential first area of the lane line;
基于所述原始路面灰度数据，利用路面车道线的反光特性和几何尺寸特征，标记潜在的车道线第二区域；Based on the original road surface grayscale data, using the reflective characteristics and geometric dimension features of the road surface lane lines to mark the potential second area of the lane line;
结合所述潜在的车道线第一区域和潜在的车道线第二区域，确定当前路面的车道线位置；Combining the potential first area of the lane line and the second area of the potential lane line, determine the position of the lane line on the current road surface;
基于当前路面的车道线位置，提取车道范围内的目标路面高程数据和目标路面灰度数据。Based on the lane line position of the current road surface, the target road surface elevation data and target road surface grayscale data within the lane range are extracted.
根据本发明提供的一种基于精密三维的路面磨耗检测方法，基于所述目标路面高程数据和目标路面灰度数据，重建路面三维点云数据，包括：According to a precise three-dimensional road surface wear detection method provided by the present invention, based on the target road surface elevation data and the target road surface grayscale data, the three-dimensional point cloud data of the road surface is reconstructed, including:
基于所述目标路面高程数据和目标路面灰度数据，确定车道范围内的异常高程测点；Based on the target road surface elevation data and the target road surface grayscale data, determine abnormal elevation measuring points within the range of the lane;
基于非异常高程测点的目标路面高程数据，估计异常高程测点的高程估计数据，以生成有效路面高程数据，所述有效路面高程数据包括：非异常高程测点的目标路面高程数据和所述高程估计数据；Based on the target road surface elevation data of the non-abnormal elevation measuring points, estimate the elevation estimation data of the abnormal elevation measuring points to generate effective road surface elevation data, and the effective road surface elevation data include: the target road surface elevation data of the non-abnormal elevation measuring points and the Elevation estimate data;
基于所述有效路面高程数据，重建路面三维点云数据。Based on the effective road surface elevation data, the three-dimensional point cloud data of the road surface is reconstructed.
根据本发明提供的一种基于精密三维的路面磨耗检测方法，基于所述目标路面高程数据和目标路面灰度数据，确定车道范围内的异常高程测点，包括：According to a precise three-dimensional road surface wear detection method provided by the present invention, based on the target road surface elevation data and the target road surface grayscale data, the abnormal elevation measurement points within the range of the lane are determined, including:
基于所述目标路面高程数据，确定车道范围内的初步异常高程测点；Based on the target road surface elevation data, determine preliminary abnormal elevation measurement points within the range of the lane;
基于所述目标路面灰度数据和所述初步异常高程测点，确定所述异常高程测点。The abnormal elevation measuring point is determined based on the grayscale data of the target road surface and the preliminary abnormal elevation measuring point.
根据本发明提供的一种基于精密三维的路面磨耗检测方法，基于所述目标路面高程数据，确定车道范围内的初步异常高程测点，包括：According to a precise three-dimensional road surface wear detection method provided by the present invention, based on the target road surface elevation data, the preliminary abnormal elevation measurement points within the range of the lane are determined, including:
获取所述目标路面高程数据中的高频路面高程信号；Obtaining the high-frequency road surface elevation signal in the target road surface elevation data;
对任一测点，计算任一测点周围第一预设范围内所有测点的高频路面高程信号的高程均值和高程方差；For any measuring point, calculate the height mean value and height variance of the high-frequency road surface elevation signals of all measuring points within the first preset range around any measuring point;
对任一测点，基于对应的高程均值和高程方差计算第一异常高程分割阈值和第二异常高程分割阈值，使第一异常高程分割阈值大于第二异常高程分割阈值；For any measuring point, calculate the first abnormal elevation segmentation threshold and the second abnormal elevation segmentation threshold based on the corresponding elevation mean and elevation variance, so that the first abnormal elevation segmentation threshold is greater than the second abnormal elevation segmentation threshold;
对任一测点，其对应的高频路面高程信号在大于第一异常高程分割阈值，或小于第二异常高程分割阈值的情况下，确定所述任一测点为所述初步异常高程测点。For any measuring point, if its corresponding high-frequency road surface elevation signal is greater than the first abnormal elevation segmentation threshold, or less than the second abnormal elevation segmentation threshold, determine that any measuring point is the preliminary abnormal elevation measuring point .
根据本发明提供的一种基于精密三维的路面磨耗检测方法，基于所述目标路面灰度数据和所述初步异常高程测点，确定所述异常高程测点，包括：According to a precise three-dimensional road surface wear detection method provided by the present invention, the abnormal elevation measurement point is determined based on the grayscale data of the target road surface and the preliminary abnormal elevation measurement point, including:
对任一初步异常高程测点，计算任一初步高程异常测点周围第二预设范围内所有测点的目标路面灰度数据的灰度均值和灰度方差；For any preliminary abnormal height measuring point, calculate the gray mean value and gray value variance of the target road surface gray data of all measuring points within the second preset range around any preliminary height abnormal measuring point;
对任一初步异常高程测点，基于对应的灰度均值和灰度方差计算第一异常灰度分割阈值和第二异常灰度分割阈值，使第一异常灰度分割阈值大于第二异常灰度分割阈值；For any preliminary abnormal elevation measuring point, calculate the first abnormal gray-scale segmentation threshold and the second abnormal gray-scale segmentation threshold based on the corresponding gray-scale mean and gray-scale variance, so that the first abnormal gray-scale segmentation threshold is greater than the second abnormal gray-scale segmentation threshold;
对任一初步异常高程测点，其对应的目标路面灰度数据在大于第一异常高程分割阈值，或小于第二异常高程分割阈值的情况下，确定所述任一初步异常高程测点为所述异常高程测点。For any preliminary abnormal elevation measuring point, if the corresponding target road surface grayscale data is greater than the first abnormal elevation segmentation threshold, or less than the second abnormal elevation segmentation threshold, it is determined that any preliminary abnormal elevation measuring point is the The above-mentioned abnormal elevation measuring points.
根据本发明提供的一种基于精密三维的路面磨耗检测方法，基于非异常高程测点的目标路面高程数据，估计异常高程测点的高程估计数据，以生成有效路面高程数据，包括：According to a precise three-dimensional road surface wear detection method provided by the present invention, based on the target road surface elevation data of non-abnormal elevation measurement points, the elevation estimation data of abnormal elevation measurement points are estimated to generate effective road surface elevation data, including:
对任一异常高程测点，基于所述任一异常高程测点周围预定区域内的非异常高程测点的目标路面高程数据，估计所述任一异常高程测点的高程估计数据；For any abnormal elevation measuring point, based on the target road surface elevation data of non-abnormal elevation measuring points in the predetermined area around the any abnormal elevation measuring point, estimate the elevation estimation data of any abnormal elevation measuring point;
将所述高程估计数据和非异常高程测点的目标路面高程数据确定为所述有效路面高程数据。The elevation estimation data and target road surface elevation data of non-abnormal elevation measuring points are determined as the effective road surface elevation data.
根据本发明提供的一种基于精密三维的路面磨耗检测方法，基于重建的路面三维点云数据，结合预设的路面构造深度计算模型，确定路面全幅构造深度，包括：According to a precise three-dimensional road surface wear detection method provided by the present invention, based on the reconstructed three-dimensional point cloud data of the road surface, combined with the preset calculation model of the road surface structure depth, the overall structure depth of the road surface is determined, including:
沿行车方向将重建的路面三维点云数据划分为多个一级点云单元；Divide the reconstructed road surface 3D point cloud data into multiple first-level point cloud units along the driving direction;
将任一所述一级点云单元沿道路宽度方向划分为多个二级点云单元；Divide any one of the first-level point cloud units into a plurality of second-level point cloud units along the road width direction;
对任一所述一级点云单元中的所有二级点云单元，基于预设的路面构造深度计算模型计算得到所述所有二级点云单元的构造深度，进而得到每个所述一级点云单元的构造深度集合；For all secondary point cloud units in any of the first-level point cloud units, the construction depth of all the second-level point cloud units is calculated based on the preset road surface structure depth calculation model, and then each of the first-level point cloud units is obtained. A collection of constructed depths of point cloud cells;
基于所述构造深度集合确定所述路面全幅构造深度。Determining the overall pavement construction depth based on the construction depth set.
根据本发明提供的一种基于精密三维的路面磨耗检测方法，基于所述车道线位置和所述路面全幅构造深度，确定左轮迹带、右轮迹带和车道中线的位置，包括：According to a precise three-dimensional road surface wear detection method provided by the present invention, based on the position of the lane line and the full-width structure depth of the road surface, the positions of the left wheel mark, the right wheel mark and the center line of the road are determined, including:
利用左轮迹带和右轮迹带之间距离为一固定范围，以及左轮迹带和右轮迹带的中心位于车道中心的特征，结合车道线位置，确定准左轮迹带、准右轮迹带和准车道中心线各自对应的二级点云单元的ID，分别记为：L′、R′和M′，以各自对应的二级点云单元的ID作为各自的位置；Using the distance between the left wheel mark and the right wheel mark as a fixed range, and the characteristics that the centers of the left wheel mark and the right wheel mark are located in the center of the lane, combined with the position of the lane line, determine the quasi-left wheel mark and the quasi-right wheel mark The IDs of the secondary point cloud units corresponding to the centerline of the quasi-lane are respectively recorded as: L', R' and M', and the IDs of the corresponding secondary point cloud units are used as their respective positions;
依据二级点云单元中相邻单元沿道路宽度方向的间距，确定预设搜寻范围D；Determine the default search range D according to the distance between adjacent units in the second-level point cloud unit along the width direction of the road;
在所述预设搜寻范围D内，按以下最大化目标Z搜索满足约束条件s.t.的位置偏差d，基于所述位置偏差d以及准左轮迹带、准右轮迹带和准车道中心线，分别得到所述左轮迹带、右轮迹带和车道中心线各自对应的二级点云单元的ID，分别记为：L、R和M，Within the preset search range D, search for the position deviation d satisfying the constraint condition s.t. according to the following maximization target Z. Obtain the IDs of the respective secondary point cloud units corresponding to the left wheel track, the right wheel track and the lane centerline, which are respectively denoted as: L, R and M,
max Z＝SMTDM-(SMTDL+SMTDR)/2max Z＝SMTDM -(SMTDL +SMTDR )/2
其中，SMTDL、SMTDR和SMTDM分别表示左轮迹带、右轮迹带和车道中线对应的二级点云单元的构造深度。Among them, SMTDL , SMTDR and SMTDM represent the construction depths of the secondary point cloud units corresponding to the left wheel track, the right wheel track and the center line of the lane, respectively.
根据本发明提供的一种基于精密三维的路面磨耗检测方法，基于所述路面全幅构造深度，以及左轮迹带、右轮迹带和车道中线各自位置，计算路面磨耗，包括：计算轮迹带位置的路面磨耗和非轮迹带位置的路面磨耗至少之一，According to a precise three-dimensional road surface wear detection method provided by the present invention, based on the overall structure depth of the road surface, and the respective positions of the left wheel track, the right wheel track, and the center line of the lane, the road wear is calculated, including: calculating the position of the wheel track At least one of the road wear and the road wear at non-wheel-track locations,
其中，计算轮迹带位置的路面磨耗包括：Among them, the calculation of road surface wear at the position of the wheel track includes:
基于左轮迹带、右轮迹带和车道中线的位置对应的构造深度，按如下公式计算轮迹带位置的路面磨耗率，得到轮迹带位置的路面磨耗WR1：Based on the structural depths corresponding to the positions of the left wheel mark, the right wheel mark and the center line of the lane, the road wear rate at the wheel mark position is calculated according to the following formula, and the road wear WR1 at the wheel mark position is obtained:
计算非轮迹带位置的路面磨耗包括：Calculation of road wear at non-track locations involves:
对任一一级点云单元中的任一二级点云单元，以车道中线位置对应的构造深度作为无磨耗的构造深度基准值，按如下公式计算路面磨耗率，得到非轮迹带位置的路面磨耗WR2：For any second-level point cloud unit in any first-level point cloud unit, the structural depth corresponding to the lane centerline position is used as the reference value of the non-wearing structural depth, and the road surface wear rate is calculated according to the following formula to obtain the non-wheel-track position Road wear WR2 :
其中，n为各一级点云单元中二级点云单元的个数。Among them, n is the number of second-level point cloud units in each first-level point cloud unit.
本发明提供的基于精密三维的路面磨耗检测方法，通过接收线扫描三维测量传感器获取的道路上各测点的原始路面高程数据和原始路面灰度数据，所述线扫描三维测量传感器的测量范围覆盖整个车道宽度；从原始路面高程数据和原始路面灰度数据中确定车道线位置，并提取车道范围内的目标路面高程数据和目标路面灰度数据；基于所述目标路面高程数据和目标路面灰度数据，重建路面三维点云数据；基于重建的路面三维点云数据，结合预设的路面构造深度计算模型，确定路面全幅构造深度；基于所述车道线位置和所述路面全幅构造深度，确定左轮迹带、右轮迹带和车道中线的位置；基于所述路面全幅构造深度，以及左轮迹带、右轮迹带和车道中线各自位置，计算路面磨耗。由于线扫描三维测量传感器的测量范围覆盖整个车道宽度，原始路面高程数据和原始路面灰度数据即为覆盖整个车道宽度的数据，不会避开路况较差的轮迹带位置，而且采用车道范围内的目标路面高程数据和目标路面灰度数据重建路面三维点云数据，去除了车道线及车道边缘其它非车道区域的数据干扰，能够准确地测量路面全幅构造深度，基于准确的路面全幅构造深度和车道线位置可以准确定位路面轮迹带和车道中线位置，从而能够准确地测量路面磨耗。同时，由于是基于覆盖整个路面宽度的路面全幅构造深度来检测路面磨耗，因此，还可评估车道内任一位置的路面磨损情况。The precise three-dimensional road wear detection method provided by the present invention receives the original road surface elevation data and original road surface grayscale data of each measuring point on the road obtained by the line-scan three-dimensional measurement sensor, and the measurement range of the line-scan three-dimensional measurement sensor covers The entire lane width; determine the position of the lane line from the original road surface elevation data and the original road surface grayscale data, and extract the target road surface elevation data and target road surface grayscale data within the range of the lane; based on the target road surface elevation data and the target road surface grayscale data to reconstruct the three-dimensional point cloud data of the road surface; based on the reconstructed three-dimensional point cloud data of the road surface, combined with the preset calculation model of the road surface structure depth, determine the full-width structure depth of the road surface; based on the position of the lane line and the overall structure depth of the road surface, determine the left wheel The position of the track, the right wheel track and the center line of the lane; the pavement wear is calculated based on the overall construction depth of the road surface, and the respective positions of the left wheel track, the right wheel track and the center line of the lane. Since the measurement range of the line-scan 3D measurement sensor covers the entire lane width, the original road surface elevation data and original road surface grayscale data are the data covering the entire lane width, which will not avoid the position of the wheel tracks with poor road conditions, and use the lane range The 3D point cloud data of the road surface is reconstructed from the target road surface elevation data and the target road surface grayscale data, which removes the data interference of the lane line and other non-lane areas on the edge of the lane, and can accurately measure the full-scale structure depth of the road surface. Based on the accurate full-scale structure depth of the road surface and the position of the lane line can accurately locate the position of the road wheel mark and the center line of the lane, so that the road wear can be accurately measured. At the same time, since pavement wear is detected based on the full pavement build-up depth covering the entire pavement width, it is also possible to evaluate pavement wear at any location within the lane.
附图说明Description of drawings
为了更清楚地说明本发明或现有技术中的技术方案，下面将对实施例或现有技术描述中所需要使用的附图作一简单地介绍，显而易见地，下面描述中的附图是本发明的一些实施例，对于本领域普通技术人员来讲，在不付出创造性劳动的前提下，还可以根据这些附图获得其他的附图。In order to more clearly illustrate the present invention or the technical solutions in the prior art, the accompanying drawings that need to be used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the accompanying drawings in the following description are the present invention. For some embodiments of the invention, those skilled in the art can also obtain other drawings based on these drawings without creative effort.
图1是本发明提供的基于精密三维的路面磨耗检测方法的流程示意图之一；Fig. 1 is one of the flow diagrams of the precision three-dimensional based road wear detection method provided by the present invention;
图2是本发明提供的基于精密三维的路面磨耗检测方法的流程示意图之二；Fig. 2 is the second schematic flow diagram of the precise three-dimensional based road wear detection method provided by the present invention;
图3是本发明提供的基于精密三维的路面磨耗检测方法的流程示意图之三；Fig. 3 is the third schematic flow diagram of the precision three-dimensional based road wear detection method provided by the present invention;
图4是本发明提供的基于精密三维的路面磨耗检测方法的流程示意图之四；Fig. 4 is the fourth schematic flow diagram of the precise three-dimensional based road wear detection method provided by the present invention;
图5是本发明提供的基于精密三维的路面磨耗检测方法的流程示意图之五；Fig. 5 is the fifth schematic flow diagram of the precise three-dimensional based road wear detection method provided by the present invention;
图6是本发明提供的基于精密三维的路面磨耗检测方法的流程示意图之六；Fig. 6 is the sixth schematic flow diagram of the precise three-dimensional based road wear detection method provided by the present invention;
图7是本发明提供的基于精密三维的路面磨耗检测方法的流程示意图之七Fig. 7 is the seventh schematic flow chart of the precision three-dimensional based road wear detection method provided by the present invention
图8是本发明提供的基于精密三维的路面磨耗检测方法的流程示意图之八Fig. 8 is the eighth schematic flow chart of the precision three-dimensional based road wear detection method provided by the present invention
图9是本发明提供的基于精密三维的全幅构造深度检测装置的结构示意图；Fig. 9 is a schematic structural diagram of a precision three-dimensional full-frame depth detection device provided by the present invention;
图10是本发明提供的电子设备的结构示意图。Fig. 10 is a schematic structural diagram of an electronic device provided by the present invention.
具体实施方式Detailed ways
为使本发明的目的、技术方案和优点更加清楚，下面将结合本发明中的附图，对本发明中的技术方案进行清楚、完整地描述，显然，所描述的实施例是本发明一部分实施例，而不是全部的实施例。基于本发明中的实施例，本领域普通技术人员在没有作出创造性劳动前提下所获得的所有其他实施例，都属于本发明保护的范围。In order to make the purpose, technical solutions and advantages of the present invention clearer, the technical solutions in the present invention will be clearly and completely described below in conjunction with the accompanying drawings in the present invention. Obviously, the described embodiments are part of the embodiments of the present invention , but not all examples. Based on the embodiments of the present invention, all other embodiments obtained by persons of ordinary skill in the art without creative efforts fall within the protection scope of the present invention.
本发明实施例提供的基于精密三维的路面磨耗检测方法，如图1所示，包括：The precise three-dimensional road surface wear detection method provided by the embodiment of the present invention, as shown in Figure 1, includes:
步骤S110：接收线扫描三维测量传感器获取的道路上各测点的原始路面高程数据和原始路面灰度数据，所述线扫描三维测量传感器的测量范围覆盖整个车道宽度。具体地，线扫描三维测量传感器包括激光器和高速三维相机，将该线扫描三维测量传感器安装在车载平台上，测量过程中，线扫描三维测量传感器沿道路方向连续采集路面的高程信息和灰度信息，即原始路面高程数据和原始路面灰度数据，通过接收线扫描三维测量传感器回传的数据，即可同时获取原始路面高程数据和原始路面灰度数据。为了反映路面全幅构造深度情况，要求线扫描三维测量传感器的测量范围在道路宽度方向全车道覆盖，且原始路面高程数据和原始路面灰度数据在道路宽度方向的采集间距小于等于5mm，行车方向的采集间距小于等于5mm。可以根据道路宽度在车上设置一套或多套线扫描三维测量传感器，使其总的覆盖宽度能够达到全车道覆盖。Step S110: Receive the original road surface elevation data and original road surface grayscale data of each measuring point on the road acquired by a line-scanning three-dimensional measurement sensor whose measurement range covers the entire lane width. Specifically, the line-scanning 3D measurement sensor includes a laser and a high-speed 3D camera, and the line-scanning 3D measurement sensor is installed on a vehicle-mounted platform. During the measurement process, the line-scanning 3D measurement sensor continuously collects elevation information and grayscale information of the road surface along the road direction , that is, the original road surface elevation data and the original road surface grayscale data, by receiving the data returned by the line scan 3D measurement sensor, the original road surface elevation data and the original road surface grayscale data can be obtained at the same time. In order to reflect the overall structure depth of the road surface, it is required that the measurement range of the line-scanning 3D measurement sensor covers the entire lane in the width direction of the road, and the collection distance between the original road surface elevation data and the original road surface grayscale data in the road width direction is less than or equal to 5mm. The collection distance is less than or equal to 5mm. One or more sets of line-scanning three-dimensional measurement sensors can be installed on the vehicle according to the width of the road so that the total coverage width can reach full lane coverage.
步骤S120：从原始路面高程数据和原始路面灰度数据中确定车道线位置，并提取车道范围内的目标路面高程数据和目标路面灰度数据。具体地，由于车道线的高程和灰度与常规路面区域(不含标线区域)的高程和灰度有较大差异，因此，可通过这种差异化分析可从原始路面高程数据和原始路面灰度数据中确定车道线位置，进而去除道路边缘车道线甚至车道线外侧的原始路面高程数据和原始路面灰度数据，使得用于三维点云重建的数据只包含车道内的数据，从而最终的测量结果能够准确地反映路面全幅构造深度，基于准确的路面全幅构造深度也能够准确地检测路面磨耗。Step S120: Determine the position of the lane line from the original road surface elevation data and the original road surface grayscale data, and extract the target road surface elevation data and target road surface grayscale data within the range of the lane. Specifically, since the elevation and grayscale of the lane lines are quite different from those of the conventional road surface area (excluding the line marking area), the difference analysis can be performed from the original road surface elevation data and the original road surface Determine the position of the lane line in the grayscale data, and then remove the original road surface elevation data and original road surface grayscale data of the lane line at the edge of the road or even outside the lane line, so that the data used for 3D point cloud reconstruction only includes the data in the lane, so that the final The measurement results can accurately reflect the overall structure depth of the pavement, and the pavement wear can also be accurately detected based on the accurate overall structure depth of the pavement.
步骤S130：基于所述目标路面高程数据和目标路面灰度数据，重建路面三维点云数据。Step S130: Based on the target road surface elevation data and the target road surface grayscale data, reconstruct road surface three-dimensional point cloud data.
步骤S140：基于重建的路面三维点云数据，结合预设的路面构造深度计算模型，确定路面全幅构造深度。Step S140: Based on the reconstructed 3D point cloud data of the road surface, combined with the preset calculation model of the road surface structure depth, determine the overall structure depth of the road surface.
步骤S150：基于所述车道线位置和所述路面全幅构造深度，确定左轮迹带、右轮迹带和车道中线的位置。由于后续计算路面磨耗需要用到左轮迹带、右轮迹带和车道中线的构造深度，因此，本步骤中，通过车道线位置和所述路面全幅构造深度，确定左轮迹带、右轮迹带和车道中线的位置。Step S150: Based on the position of the lane line and the depth of the overall structure of the road surface, determine the positions of the left wheel mark, the right wheel mark and the center line of the road. Since the follow-up calculation of road surface wear needs to use the structural depth of the left wheel track, the right wheel track and the center line of the lane, in this step, the left wheel track and the right wheel track are determined by the position of the lane line and the overall structural depth of the road surface and the position of the centerline of the lane.
步骤S160：基于路面全幅构造深度，以及左轮迹带、右轮迹带和车道中线各自位置，计算路面磨耗。通过左轮迹带、右轮迹带和车道中线各自位置和路面全幅构造深度可以确定左轮迹带、右轮迹带和车道中线各自位置对应的构造深度，从而计算轮迹带位置的路面磨耗，同理，以车道中线位置对应的构造深度作为无磨耗的构造深度基准值，也可计算非轮迹带位置的路面磨耗。Step S160: Calculate road surface wear based on the overall structure depth of the road surface, and the respective positions of the left wheel track, the right wheel track, and the center line of the lane. According to the respective positions of the left wheel track, the right wheel track and the center line of the lane and the overall structure depth of the road surface, the corresponding structural depths of the respective positions of the left wheel track, the right wheel track and the center line of the lane can be determined, so as to calculate the road surface wear at the position of the wheel track, and at the same time According to the principle, taking the construction depth corresponding to the centerline position of the lane as the reference value of the construction depth without wear, the road surface wear at the non-wheel-mark position can also be calculated.
本实施例的基于精密三维的全幅构造深度检测方法中，由于线扫描三维测量传感器的测量范围覆盖整个车道宽度，原始路面高程数据和原始路面灰度数据即为覆盖整个车道宽度的数据，不会避开路况较差的轮迹带位置，而且采用车道范围内的目标路面高程数据和目标路面灰度数据重建路面三维点云数据，去除了车道线及车道边缘其它非车道区域的数据干扰，能够准确地测量路面全幅构造深度，基于准确的路面全幅构造深度和车道线位置可以准确定位路面轮迹带和车道中线位置，从而能够准确地测量路面磨耗。同时，由于是基于覆盖整个路面宽度的路面全幅构造深度来检测路面磨耗，因此，还可评估车道内任一位置的路面磨损情况。In the precise three-dimensional-based full-frame structure depth detection method of this embodiment, since the measurement range of the line-scanning three-dimensional measurement sensor covers the entire lane width, the original road surface elevation data and original road surface grayscale data are the data covering the entire lane width, and will not Avoiding the position of the wheel tracks with poor road conditions, and using the target road surface elevation data within the lane range and the target road surface grayscale data to reconstruct the 3D point cloud data of the road surface, removing the data interference of the lane line and other non-lane areas on the edge of the lane, and being able to Accurately measure the full-width structure depth of the road surface. Based on the accurate overall structure depth of the road surface and the position of the lane line, the position of the road wheel track and the center line of the lane can be accurately located, so that the road wear can be accurately measured. At the same time, since pavement wear is detected based on the full pavement build-up depth covering the entire pavement width, it is also possible to evaluate pavement wear at any location within the lane.
如图2所示，在一些实施例中，步骤S120具体包括：As shown in Figure 2, in some embodiments, step S120 specifically includes:
步骤S210：基于所述原始路面高程数据，利用路面车道线的高程特征和几何尺寸特征，标记潜在的车道线第一区域。通常道路上的一条车道有其两边的车道线界定，车道线的几何尺寸(即宽度和高度)都符合道路的相关标准，车道线的高程要高于正常路面，因此，可以通过原始路面高程数据、车道线的高程特征和几何尺寸特征标记出车道线第一区域(即两条车道线的区域)。Step S210: Based on the original road surface elevation data, use the elevation feature and geometric dimension feature of the road surface lane line to mark a potential first area of the lane line. Usually, a lane on the road is bounded by lane lines on both sides. The geometric dimensions (ie width and height) of the lane line conform to the relevant standards of the road. The elevation of the lane line is higher than the normal road surface. Therefore, the original road surface elevation data can , the elevation feature and the geometric dimension feature of the lane line mark the first area of the lane line (ie, the area of two lane lines).
步骤S220：基于所述原始路面灰度数据，利用路面车道线的反光特性和几何尺寸特征，标记潜在的车道线第二区域。通常车道线都是带有白色或黄色，与灰色路面的反光特性不同，因此，可以通过原始路面灰度数据、车道线的反光特征和几何尺寸特征标记出车道线第二区域。Step S220: Based on the original grayscale data of the road surface, the potential second area of the lane line is marked by using the reflection characteristic and the geometric dimension feature of the road surface lane line. Usually, the lane lines are white or yellow, which is different from the reflective characteristics of the gray road surface. Therefore, the second area of the lane line can be marked by the original road surface grayscale data, the reflective features of the lane line, and the geometric dimension features.
步骤S230：结合所述潜在的车道线第一区域和潜在的车道线第二区域，确定当前路面的车道线位置。上述步骤S210和S220分别用不同的方式标记出了两个不同的车道线区域，不同方法得到的结果可以相互验证，例如：两个不同的车道线区域完全重叠或重叠率高达80％以上，则可认为通过两种方式标记出的车道线区域都比较准确。确定当前路面的车道线位置时，可以将两个车道线区域覆盖的最大区域(即两个车道线区域取并集)作为车道线的区域，即车道线所在位置。采用两个车道线区域覆盖的最大区域作为车道线区域，可以尽可能地排除车道边缘的异常点，使得最终检测的路面磨耗结果更准确。Step S230: Combining the first potential lane line area and the potential second lane line area, determine the position of the lane line on the current road surface. The above steps S210 and S220 mark two different lane line areas in different ways, and the results obtained by different methods can be mutually verified. For example, if two different lane line areas completely overlap or the overlap rate is as high as 80%, then It can be considered that the lane line areas marked by the two methods are relatively accurate. When determining the position of the lane line on the current road surface, the largest area covered by the two lane line areas (that is, the union of the two lane line areas) can be taken as the area of the lane line, that is, the location of the lane line. Using the largest area covered by the two lane line areas as the lane line area can eliminate the abnormal points on the edge of the lane as much as possible, so that the final detection of road wear results is more accurate.
步骤S240：基于当前路面的车道线位置，提取车道范围内的目标路面高程数据和目标路面灰度数据。具体地，由于当前路面的车道线位置确定，即车道线的区域确定，可根据左车道线区域和右车道线区域各自的内侧边缘之间的区域确定车道范围，从而提取车道范围内的目标路面高程数据和目标路面灰度数据。Step S240: Based on the lane line position of the current road surface, extract the target road surface elevation data and the target road surface grayscale data within the lane range. Specifically, since the position of the lane line on the current road surface is determined, that is, the area of the lane line is determined, the lane range can be determined according to the area between the respective inner edges of the left lane line area and the right lane line area, thereby extracting the target road surface within the lane range Elevation data and target road surface grayscale data.
上述步骤S130中，重建路面三维点云数据可以直接采用目标路面高程数据和目标路面灰度数据进行重建，但是车道范围内的目标路面高程数据和目标路面灰度数据也会存在一些异常数据，异常数据对应的检测点为异常测点。对道路路面(例如：沥青路面)而言，路面集料较为密集，集料间形态各异。对集料间轮廓变化剧烈，且缝隙较深的位置，或部分路面裂缝区域，可能出现部分测点无效情况(3D相机无法观测到对应测点返回的光信号)，这些测点的测量值明显异常，且此类测点通常表现为高频、局部突变等特征。若不剔除此类测点，将明显影响路面磨耗结果的准确性。In the above step S130, the reconstructed 3D point cloud data of the road surface can be directly reconstructed using the target road surface elevation data and the target road surface grayscale data, but the target road surface elevation data and the target road surface grayscale data within the range of the lane will also have some abnormal data. The detection points corresponding to the data are abnormal detection points. For road pavement (for example: asphalt pavement), pavement aggregates are relatively dense, and the aggregates have different shapes. For locations where the contours of the aggregates change drastically and the gaps are deep, or in some pavement crack areas, some measuring points may be invalid (3D cameras cannot observe the light signals returned by the corresponding measuring points), and the measured values of these measuring points are obviously Abnormal, and such measuring points are usually characterized by high frequency, local mutation and so on. If such measuring points are not eliminated, the accuracy of pavement wear results will be significantly affected.
因此，为了使最终检测的路面磨耗结果更加准确，在一些实施例中，可以如图3的方式对异常测点的数据进行处理，形成有效数据，再通过有效数据重建路面三维点云数据。具体地，步骤S130包括：Therefore, in order to make the final detection result of road surface wear more accurate, in some embodiments, the data of abnormal measurement points can be processed in the manner shown in Figure 3 to form effective data, and then the three-dimensional point cloud data of the road surface can be reconstructed from the effective data. Specifically, step S130 includes:
步骤S310：基于所述目标路面高程数据和目标路面灰度数据，确定车道范围内的异常高程测点。Step S310: Based on the target road surface elevation data and the target road surface grayscale data, determine abnormal elevation measuring points within the range of the lane.
步骤S320：基于非异常高程测点的目标路面高程数据，估计异常高程测点的高程估计数据，以生成有效路面高程数据，所述有效路面高程数据包括：非异常高程测点的目标路面高程数据和所述高程估计数据。Step S320: Based on the target road surface elevation data of non-abnormal elevation measuring points, estimate the elevation estimation data of abnormal elevation measuring points to generate effective road surface elevation data, the effective road surface elevation data including: target road surface elevation data of non-abnormal elevation measuring points and the elevation estimate data.
步骤S330：基于所述有效路面高程数据，重建路面三维点云数据。Step S330: Based on the effective road surface elevation data, reconstruct the three-dimensional point cloud data of the road surface.
如图4所示，步骤S310具体包括：As shown in Figure 4, step S310 specifically includes:
步骤S410：基于所述目标路面高程数据，确定车道范围内的初步异常高程测点。Step S410: Based on the target road surface elevation data, determine preliminary abnormal elevation measurement points within the range of the lane.
步骤S420：基于所述目标路面灰度数据和所述初步异常高程测点，确定所述异常高程测点。Step S420: Determine the abnormal elevation measuring point based on the grayscale data of the target road surface and the preliminary abnormal elevation measuring point.
在一些实施例中，如图5所示，步骤S410具体包括：In some embodiments, as shown in FIG. 5, step S410 specifically includes:
步骤S510：获取所述目标路面高程数据中的高频路面高程信号。其中，对于每一个测点，线扫描三维测量传感器采集的目标路面高程数据中包括低频信号和高频信号两部分，本实施例中提取高频信号部分，高频信号可通过滤波(如：高通滤波)、或频域变换(如：傅里叶变换、小波变换等)方法获取。Step S510: Obtain a high-frequency road surface elevation signal in the target road surface elevation data. Wherein, for each measuring point, the target road surface elevation data collected by the line scanning three-dimensional measurement sensor includes two parts, a low-frequency signal and a high-frequency signal. In this embodiment, the high-frequency signal part is extracted, and the high-frequency signal can be filtered (such as: high pass Filtering), or frequency domain transformation (such as: Fourier transform, wavelet transform, etc.) method to obtain.
步骤S520：对任一测点，计算任一测点周围第一预设范围内所有测点的高频路面高程信号的高程均值和高程方差，其中，第一预设范围的测点可以是该任一测点周围5～20行，以及5～20列内的测点。Step S520: For any measuring point, calculate the elevation mean value and elevation variance of the high-frequency road surface elevation signals of all measuring points within the first preset range around any measuring point, wherein the measuring points in the first preset range can be the 5 to 20 rows around any measuring point, and measuring points within 5 to 20 columns.
步骤S530：对任一测点，基于对应的高程均值和高程方差计算第一异常高程分割阈值和第二异常高程分割阈值，使第一异常高程分割阈值大于第二异常高程分割阈值。具体地，第一异常高程分割阈值T1和第二异常高程分割阈值T2的计算方式如下：Step S530: For any measuring point, calculate the first abnormal elevation segmentation threshold and the second abnormal elevation segmentation threshold based on the corresponding elevation mean and elevation variance, so that the first abnormal elevation segmentation threshold is greater than the second abnormal elevation segmentation threshold. Specifically, the calculation method of the first abnormal elevation segmentation thresholdT1 and the second abnormal elevation segmentation thresholdT2 is as follows:
T1＝Ah+k1*ShT1 =Ah +k1 *Sh
T2＝Ah-k2*ShT2 ＝Ah -k2 *Sh
其中，Ah为高程均值，Sh为高程方差，k1和k2分别为第一系数和第二系数，两者均取大于0的值。可以通过调节k1和k2的值来调节T1和的T2值，从而确定初步异常高程测点的筛选范围。Among them, Ah is the elevation mean value,Sh is the elevation variance, k1 and k2 are the first coefficient and the second coefficient respectively, and both of them take values greater than 0. The values ofT1 andT2 can be adjusted by adjusting the values ofk1 andk2 , so as to determine the screening range of preliminary abnormal elevation measuring points.
步骤S540：对任一测点，其对应的高频路面高程信号在大于第一异常高程分割阈值，或小于第二异常高程分割阈值的情况下，确定所述任一测点为所述初步异常高程测点，即将[T2，T1]区间以外的对应的任一测点确定为初步异常高程测点。Step S540: For any measuring point, if its corresponding high-frequency road surface elevation signal is greater than the first abnormal elevation segmentation threshold or less than the second abnormal elevation segmentation threshold, determine that any measuring point is the preliminary abnormality Elevation measuring point, that is, to determine any corresponding measuring point outside the [T2 , T1 ] interval as the preliminary abnormal elevation measuring point.
在一些实施例中，如图6所示，步骤S420具体包括：In some embodiments, as shown in FIG. 6, step S420 specifically includes:
步骤S610：对任一初步异常高程测点，计算任一初步高程异常测点周围第二预设范围内所有测点的目标路面灰度数据的灰度均值和灰度方差。其中，第二预设范围的测点可以是该任一测点周围5～100行，以及5～100列内的测点。Step S610: For any preliminary abnormal elevation measurement point, calculate the gray level mean and gray level variance of the target road surface gray level data of all measurement points within the second preset range around any preliminary abnormal elevation measurement point. Wherein, the measuring points in the second preset range may be the measuring points within 5-100 rows and 5-100 columns around any measuring point.
步骤S620：对任一初步异常高程测点，基于对应的灰度均值和灰度方差计算第一异常灰度分割阈值和第二异常灰度分割阈值，使第一异常灰度分割阈值大于第二异常灰度分割阈值。具体地，第一异常灰度分割阈值T3和第二异常灰度分割阈值T4的计算方式如下：Step S620: For any preliminary abnormal elevation measuring point, calculate the first abnormal gray-scale segmentation threshold and the second abnormal gray-scale segmentation threshold based on the corresponding gray-scale mean and gray-scale variance, so that the first abnormal gray-scale segmentation threshold is greater than the second Abnormal grayscale segmentation threshold. Specifically, the calculation method of the first abnormal gray-scale segmentation thresholdT3 and the second abnormal gray-scale segmentation thresholdT4 is as follows:
T3＝kg1*Ag+k3*SgT3 ＝kg1 *Ag +k3 *Sg
T4＝kg2*Ag-k4*SgT4 ＝kg2 *Ag -k4 *Sg
其中，Ag为灰度均值，Sg为灰度方差，k3、k4、kg1和kg2分别为第三系数、第四系数、第五系数和第六系数，k3、k4、kg1和kg2均取大于0的值。可以通过调节k3、k4、kg1和kg2的值来调节T3和的T4值，从而确定异常高程测点的筛选范围。Among them, Ag is the mean value of the gray scale, Sg is the variance of the gray scale, k3 , k4 , kg1 and kg2 are the third coefficient, the fourth coefficient, the fifth coefficient and the sixth coefficient respectively, and k3 , k4 , kg1 and kg2 all take values greater than 0. The values of T3 and T4 can be adjusted by adjusting the values of k3 , k4 , kg1 and kg2 , so as to determine the screening range of abnormal elevation measuring points.
步骤S630：对任一初步异常高程测点，其对应的目标路面灰度数据在大于第一异常高程分割阈值，或小于第二异常高程分割阈值的情况下，确定所述任一初步异常高程测点为所述异常高程测点，即将[T4，T3]区间以外的对应的任一初步异常高程测点确定为异常高程测点。Step S630: For any preliminary abnormal elevation measurement point, if the corresponding target road surface grayscale data is greater than the first abnormal elevation segmentation threshold, or less than the second abnormal elevation segmentation threshold, determine the value of any preliminary abnormal elevation measurement point. point is the abnormal elevation measuring point, that is, any corresponding preliminary abnormal elevation measuring point outside the [T4 , T3 ] interval is determined as the abnormal elevation measuring point.
在一些实施例中，步骤S320具体包括：In some embodiments, step S320 specifically includes:
对任一异常高程测点，基于所述任一异常高程测点周围预定区域内(如：该测点周围5～10行和5～10列范围内)的非异常高程测点的目标路面高程数据，估计所述任一异常高程测点的高程估计数据。具体地，可采用三角建网或插值的方式估算异常高程测点对应位置的高程估计数据。For any abnormal elevation measuring point, based on the target road surface elevation of non-abnormal elevation measuring points within the predetermined area around any abnormal elevation measuring point (for example: within the range of 5-10 rows and 5-10 columns around the measuring point) Data, to estimate the elevation estimation data of any abnormal elevation measuring point. Specifically, triangulation or interpolation methods can be used to estimate the elevation estimation data corresponding to the abnormal elevation measurement points.
将高程估计数据和非异常高程测点的目标路面高程数据确定为所述有效路面高程数据。The elevation estimation data and the target road surface elevation data of non-abnormal elevation measuring points are determined as the effective road surface elevation data.
本实施例中，采用异常高程测点周围预定区域内的正常测点的目标路面高程数据对异常高程测点的目标路面高程数据进行估计，使得异常高程测点的目标路面高程数据回归该预定区域范围内的正常值，从而得到有效路面高程数据，以有效路面高程数据重建的路面三维点云数据更准确，进而使得最终检测的路面磨耗也更准确。In this embodiment, the target road surface elevation data of the normal measuring points in the predetermined area around the abnormal elevation measuring point are used to estimate the target road surface elevation data of the abnormal elevation measuring point, so that the target road surface elevation data of the abnormal elevation measuring point returns to the predetermined area The normal value within the range, so as to obtain the effective road surface elevation data, and the road surface three-dimensional point cloud data reconstructed with the effective road surface elevation data is more accurate, which in turn makes the final detection of road surface wear more accurate.
在一些实施例中，如图7所示，步骤S140具体包括：In some embodiments, as shown in FIG. 7, step S140 specifically includes:
步骤S710：沿行车方向将重建的路面三维点云数据划分为多个一级点云单元，即将重建的路面三维点云数据按道路长度方向划分为若干个一级点云单元，一级点云单元覆盖了整个路面宽度。具体地，可以依据构造深度计算的长度要求(如：0.3米、1米、10米、20米、100米或1000米等)，将重建的路面三维点云数据沿行车方向划分为多个一级点云单元。Step S710: Divide the reconstructed road surface 3D point cloud data into multiple first-level point cloud units along the driving direction, that is, divide the reconstructed road surface 3D point cloud data into several first-level point cloud units according to the road length direction, and the first-level point cloud The unit covers the entire width of the pavement. Specifically, the reconstructed road surface 3D point cloud data can be divided into a plurality of one-dimensional point cloud data along the driving direction according to the length requirements of the structural depth calculation (such as: 0.3 meters, 1 meter, 10 meters, 20 meters, 100 meters or 1000 meters, etc.). level point cloud unit.
步骤S720：将任一所述一级点云单元沿道路宽度方向划分为多个二级点云单元，即将每一个一级点云单元按道路宽度方向划分为若干个二级点云单元。Step S720: Divide any one of the first-level point cloud units into multiple second-level point cloud units along the road width direction, that is, divide each first-level point cloud unit into several second-level point cloud units according to the road width direction.
步骤S730：对任一所述一级点云单元中的所有二级点云单元，基于预设的路面构造深度计算模型计算得到所述所有二级点云单元的构造深度，进而得到每个所述一级点云单元的构造深度集合SMTD，记为：{SMTD1,SMTD2,…,SMTDn}，对于所述一级点云单元的构造深度SMTDi，为所述一级点云单元中第i个二级点云单元的构造深度，i＝1,2,…,n。Step S730: For all the second-level point cloud units in any one of the first-level point cloud units, calculate the construction depth of all the second-level point cloud units based on the preset road surface structure depth calculation model, and then obtain each of the second-level point cloud units. The construction depth set SMTD of the first-level point cloud unit is denoted as: {SMTD1 , SMTD2 ,..., SMTDn }, for the construction depth SMTDi of the first-level point cloud unit, it is the first-level point cloud unit The construction depth of the i-th secondary point cloud unit in , i=1,2,...,n.
步骤S740：基于所述构造深度集合确定所述路面全幅构造深度。Step S740: Determine the overall structure depth of the road surface based on the set of structure depths.
具体地，该步骤包括：Specifically, this step includes:
基于所述构造深度集合确定：左轮迹构造深度、右轮迹构造深度、车道中心线构造深度以及所述构造深度集合。Based on the set of build depths, a left wheel track build depth, a right wheel track build depth, a lane centerline build depth and the set of build depths are determined.
在路面磨耗检测中，需要用到左轮迹构造深度、右轮迹构造深度和车道中心线构造深度。因此，在一些实施例中，如图8所示，步骤S150具体包括：In pavement wear detection, it is necessary to use the structural depth of the left wheel mark, the structural depth of the right wheel mark and the structural depth of the centerline of the lane. Therefore, in some embodiments, as shown in FIG. 8, step S150 specifically includes:
步骤S810：利用左轮迹带和右轮迹带之间距离为一固定范围，以及左轮迹带和右轮迹带的中心位于车道中心的特征，结合车道线位置，确定准左轮迹带、准右轮迹带和准车道中心线各自对应的二级点云单元的ID，分别记为：L′、R′和M′，以各自对应的二级点云单元的ID作为各自的位置。Step S810: Using the feature that the distance between the left wheel track and the right wheel track is a fixed range, and the center of the left wheel track and the right wheel track is located in the center of the lane, combined with the position of the lane line, determine the quasi-left wheel track, quasi-right The IDs of the secondary point cloud units corresponding to the wheel tracks and the quasi-lane centerline are respectively recorded as: L', R' and M', and the IDs of the corresponding secondary point cloud units are taken as their respective positions.
步骤S820：依据二级点云单元中相邻单元沿道路宽度方向的间距，确定预设搜寻范围D。其中，由于车道线检测结果可能存在一定偏差，或者在车道并道、弯道等特殊区域，行车轨迹通常会偏离车道中线，因此，根据步骤S810中得到的准左轮迹带、准右轮迹带和准车道中心线各自对应的二级点云单元的ID可能不准确，需要基于预设搜寻范围D及下述公式进行调整。预设搜寻范围D可以是相邻二级点云单元在道路宽度方向间距±80cm的范围。Step S820: Determine the default search range D according to the distance between adjacent units in the secondary point cloud unit along the width direction of the road. Wherein, since there may be a certain deviation in the detection result of the lane line, or in special areas such as lane merges and curves, the driving trajectory usually deviates from the center line of the lane, therefore, according to the quasi-left wheelmark and quasi-right wheelmark obtained in step S810 The IDs of the secondary point cloud units corresponding to the centerline of the quasi-lane may be inaccurate, and need to be adjusted based on the preset search range D and the following formula. The preset search range D may be a range of ±80 cm between adjacent secondary point cloud units in the road width direction.
步骤S830：在所述预设搜寻范围D内，按以下最大化目标Z搜索满足约束条件s.t.的位置偏差d，基于所述位置偏差d以及准左轮迹带、准右轮迹带和准车道中心线，分别得到所述左轮迹带、右轮迹带和车道中心线各自对应的二级点云单元的ID，分别记为：L、R和M。Step S830: Within the preset search range D, search for a position deviation d that satisfies the constraint condition s.t. according to the following maximization target Z, based on the position deviation d and the quasi-left wheel track, quasi-right wheel track and quasi-lane center line, respectively obtain the IDs of the secondary point cloud units corresponding to the left wheel track, the right wheel track and the centerline of the lane, respectively denoted as: L, R and M.
max Z＝SMTDM-(SMTDL+SMTDR)/2max Z＝SMTDM -(SMTDL +SMTDR )/2
其中，SMTDL、SMTDR和SMTDM分别表示左轮迹带、右轮迹带和车道中线对应的二级点云单元的构造深度。Among them, SMTDL , SMTDR and SMTDM represent the construction depths of the secondary point cloud units corresponding to the left wheel track, the right wheel track and the center line of the lane, respectively.
由于基于覆盖整个路面宽度的路面全幅构造深度来检测路面磨耗，因此，可以检测车道内任一位置的路面磨损情况。在一些实施例中，步骤S160包括：计算轮迹带位置的路面磨耗和非轮迹带位置的路面磨耗至少之一。Since pavement wear is detected based on the overall pavement build-up depth covering the entire pavement width, it is possible to detect pavement wear at any location within the lane. In some embodiments, step S160 includes: calculating at least one of the road surface wear at the wheel track location and the road surface wear at the non-wheel track location.
具体地，计算轮迹带位置的路面磨耗包括：Specifically, the calculation of road wear at the position of the wheel tracks includes:
基于左轮迹带、右轮迹带和车道中线的位置对应的构造深度，按如下公式计算轮迹带位置的路面磨耗率，得到轮迹带位置的路面磨耗WR1：Based on the structural depths corresponding to the positions of the left wheel mark, the right wheel mark and the center line of the lane, the road wear rate at the wheel mark position is calculated according to the following formula, and the road wear WR1 at the wheel mark position is obtained:
具体地，计算非轮迹带位置的路面磨耗包括：Specifically, the calculation of road wear at non-wheel-track locations includes:
对任一一级点云单元中的任一二级点云单元，以车道中线位置对应的构造深度作为无磨耗的构造深度基准值，按如下公式计算路面磨耗率，得到非轮迹带位置的路面磨耗WR2：For any second-level point cloud unit in any first-level point cloud unit, the structural depth corresponding to the lane centerline position is used as the reference value of the non-wearing structural depth, and the road surface wear rate is calculated according to the following formula to obtain the non-wheel-track position Road wear WR2 :
其中，n为各一级点云单元中二级点云单元的个数。Among them, n is the number of second-level point cloud units in each first-level point cloud unit.
下面对本发明提供的基于精密三维的路面磨耗检测装置进行描述，下文描述的基于精密三维的路面磨耗检测装置与上文描述的基于精密三维的路面磨耗检测方法可相互对应参照。The precise 3D-based road wear detection device provided by the present invention is described below, and the precise 3D-based road wear detection device described below and the precise 3D-based road wear detection method described above can be referred to in correspondence.
本发明提供的基于精密三维的路面磨耗检测装置，如图9所示，包括：The precision three-dimensional based road wear detection device provided by the present invention, as shown in Figure 9, includes:
数据接收模块910，用于接收线扫描三维测量传感器获取的道路上各测点的原始路面高程数据和原始路面灰度数据，所述线扫描三维测量传感器的测量范围覆盖整个车道宽度。The data receiving module 910 is configured to receive the original road surface elevation data and original road surface grayscale data of each measuring point on the road acquired by a line-scanning three-dimensional measurement sensor whose measurement range covers the entire lane width.
数据提取模块920，从原始路面高程数据和原始路面灰度数据中确定车道线位置，并提取车道范围内的目标路面高程数据和目标路面灰度数据。The data extraction module 920 determines the position of the lane line from the original road surface elevation data and the original road surface grayscale data, and extracts the target road surface elevation data and the target road surface grayscale data within the range of the lane.
数据重建模块930，基于所述目标路面高程数据和目标路面灰度数据，重建路面三维点云数据。The data reconstruction module 930 reconstructs the three-dimensional point cloud data of the road surface based on the target road surface elevation data and the target road surface grayscale data.
构造深度确定模块940，用于基于重建的路面三维点云数据，结合预设的路面构造深度计算模型，确定路面全幅构造深度。The structure depth determination module 940 is configured to determine the overall structure depth of the road surface based on the reconstructed three-dimensional point cloud data of the road surface and in combination with a preset calculation model of the road surface structure depth.
位置确定模块950，用于基于所述车道线位置和所述路面全幅构造深度，确定左轮迹带、右轮迹带和车道中线的位置。The position determining module 950 is configured to determine the positions of the left wheel mark, the right wheel mark and the center line of the lane based on the position of the lane line and the overall depth of the road surface.
路面磨耗计算模块960，用于基于所述路面全幅构造深度，以及左轮迹带、右轮迹带和车道中线各自位置，计算路面磨耗。The road surface wear calculation module 960 is configured to calculate the road surface wear based on the overall construction depth of the road surface, and the respective positions of the left wheel track, the right wheel track, and the center line of the lane.
图10示例了一种电子设备的实体结构示意图，如图10所示，该电子设备可以包括：处理器(processor)101、通信接口(Communications Interface)102、存储器(memory)103和通信总线104，其中，处理器101，通信接口102，存储器103通过通信总线104完成相互间的通信。处理器101可以调用存储器103中的逻辑指令，以执行基于精密三维的路面磨耗检测方法，该方法包括：FIG. 10 illustrates a schematic diagram of the physical structure of an electronic device. As shown in FIG. 10 , the electronic device may include: a processor (processor) 101, a communication interface (Communications Interface) 102, a memory (memory) 103 and a communication bus 104, Wherein, the processor 101 , the communication interface 102 , and the memory 103 communicate with each other through the communication bus 104 . The processor 101 can call the logic instructions in the memory 103 to execute a precision three-dimensional based road wear detection method, the method comprising:
接收线扫描三维测量传感器获取的道路上各测点的原始路面高程数据和原始路面灰度数据，所述线扫描三维测量传感器的测量范围覆盖整个车道宽度。The original road surface elevation data and the original road surface grayscale data of each measuring point on the road acquired by the line-scanning three-dimensional measurement sensor are received, and the measurement range of the line-scan three-dimensional measurement sensor covers the entire width of the lane.
从原始路面高程数据和原始路面灰度数据中确定车道线位置，并提取车道范围内的目标路面高程数据和目标路面灰度数据。The position of the lane line is determined from the original road surface elevation data and the original road surface grayscale data, and the target road surface elevation data and target road surface grayscale data within the lane range are extracted.
基于所述目标路面高程数据和目标路面灰度数据，重建路面三维点云数据。Based on the target road surface elevation data and the target road surface grayscale data, the three-dimensional point cloud data of the road surface is reconstructed.
基于重建的路面三维点云数据，结合预设的路面构造深度计算模型，确定路面全幅构造深度。Based on the reconstructed 3D point cloud data of the pavement, combined with the preset pavement structure depth calculation model, the full pavement structure depth is determined.
基于所述车道线位置和所述路面全幅构造深度，确定左轮迹带、右轮迹带和车道中线的位置。Based on the position of the lane line and the depth of the overall structure of the road surface, the positions of the left wheel mark, the right wheel mark and the center line of the road are determined.
基于所述路面全幅构造深度，以及左轮迹带、右轮迹带和车道中线各自位置，计算路面磨耗。The pavement wear is calculated based on the overall construction depth of the pavement, and the respective positions of the left wheel trail, the right wheel trail, and the center line of the lane.
此外，上述的存储器103中的逻辑指令可以通过软件功能单元的形式实现并作为独立的产品销售或使用时，可以存储在一个计算机可读取存储介质中。基于这样的理解，本发明的技术方案本质上或者说对现有技术做出贡献的部分或者该技术方案的部分可以以软件产品的形式体现出来，该计算机软件产品存储在一个存储介质中，包括若干指令用以使得一台计算机设备(可以是个人计算机，服务器，或者网络设备等)执行本发明各个实施例所述方法的全部或部分步骤。而前述的存储介质包括：U盘、移动硬盘、只读存储器(ROM，Read-Only Memory)、随机存取存储器(RAM，Random Access Memory)、磁碟或者光盘等各种可以存储程序代码的介质。In addition, the above-mentioned logic instructions in the memory 103 may be implemented in the form of software functional units and may be stored in a computer-readable storage medium when sold or used as an independent product. Based on this understanding, the essence of the technical solution of the present invention or the part that contributes to the prior art or the part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including Several instructions are used to make a computer device (which may be a personal computer, a server, or a network device, etc.) execute all or part of the steps of the methods described in various embodiments of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), magnetic disk or optical disk and other media that can store program codes. .
另一方面，本发明还提供一种计算机程序产品，所述计算机程序产品包括计算机程序，计算机程序可存储在非暂态计算机可读存储介质上，所述计算机程序被处理器执行时，计算机能够执行上述各方法所提供的基于精密三维的路面磨耗检测方法，该方法包括：On the other hand, the present invention also provides a computer program product. The computer program product includes a computer program that can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can Implementing the precise three-dimensional pavement wear detection method provided by the above methods, the method includes:
接收线扫描三维测量传感器获取的道路上各测点的原始路面高程数据和原始路面灰度数据，所述线扫描三维测量传感器的测量范围覆盖整个车道宽度。The original road surface elevation data and the original road surface grayscale data of each measuring point on the road acquired by the line-scanning three-dimensional measurement sensor are received, and the measurement range of the line-scan three-dimensional measurement sensor covers the entire width of the lane.
从原始路面高程数据和原始路面灰度数据中确定车道线位置，并提取车道范围内的目标路面高程数据和目标路面灰度数据。The position of the lane line is determined from the original road surface elevation data and the original road surface grayscale data, and the target road surface elevation data and target road surface grayscale data within the lane range are extracted.
基于所述目标路面高程数据和目标路面灰度数据，重建路面三维点云数据。Based on the target road surface elevation data and the target road surface grayscale data, the three-dimensional point cloud data of the road surface is reconstructed.
基于重建的路面三维点云数据，结合预设的路面构造深度计算模型，确定路面全幅构造深度。Based on the reconstructed 3D point cloud data of the pavement, combined with the preset pavement structure depth calculation model, the full pavement structure depth is determined.
基于所述车道线位置和所述路面全幅构造深度，确定左轮迹带、右轮迹带和车道中线的位置。Based on the position of the lane line and the depth of the overall structure of the road surface, the positions of the left wheel mark, the right wheel mark and the center line of the road are determined.
基于所述路面全幅构造深度，以及左轮迹带、右轮迹带和车道中线各自位置，计算路面磨耗。The pavement wear is calculated based on the overall construction depth of the pavement, and the respective positions of the left wheel trail, the right wheel trail, and the center line of the lane.
又一方面，本发明还提供一种非暂态计算机可读存储介质，其上存储有计算机程序，该计算机程序被处理器执行时实现以执行上述各方法提供的基于精密三维的路面磨耗检测方法，该方法包括：In yet another aspect, the present invention also provides a non-transitory computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, it is implemented to perform the precise three-dimensional-based road surface wear detection method provided by the above-mentioned methods , the method includes:
接收线扫描三维测量传感器获取的道路上各测点的原始路面高程数据和原始路面灰度数据，所述线扫描三维测量传感器的测量范围覆盖整个车道宽度。The original road surface elevation data and the original road surface grayscale data of each measuring point on the road acquired by the line-scanning three-dimensional measurement sensor are received, and the measurement range of the line-scan three-dimensional measurement sensor covers the entire width of the lane.
从原始路面高程数据和原始路面灰度数据中确定车道线位置，并提取车道范围内的目标路面高程数据和目标路面灰度数据。The position of the lane line is determined from the original road surface elevation data and the original road surface grayscale data, and the target road surface elevation data and target road surface grayscale data within the lane range are extracted.
基于所述目标路面高程数据和目标路面灰度数据，重建路面三维点云数据。Based on the target road surface elevation data and the target road surface grayscale data, the three-dimensional point cloud data of the road surface is reconstructed.
基于重建的路面三维点云数据，结合预设的路面构造深度计算模型，确定路面全幅构造深度。Based on the reconstructed 3D point cloud data of the pavement, combined with the preset pavement structure depth calculation model, the full pavement structure depth is determined.
基于所述车道线位置和所述路面全幅构造深度，确定左轮迹带、右轮迹带和车道中线的位置。Based on the position of the lane line and the depth of the overall structure of the road surface, the positions of the left wheel mark, the right wheel mark and the center line of the road are determined.
基于所述路面全幅构造深度，以及左轮迹带、右轮迹带和车道中线各自位置，计算路面磨耗。The pavement wear is calculated based on the overall construction depth of the pavement, and the respective positions of the left wheel trail, the right wheel trail, and the center line of the lane.
以上所描述的装置实施例仅仅是示意性的，其中所述作为分离部件说明的单元可以是或者也可以不是物理上分开的，作为单元显示的部件可以是或者也可以不是物理单元，即可以位于一个地方，或者也可以分布到多个网络单元上。可以根据实际的需要选择其中的部分或者全部模块来实现本实施例方案的目的。本领域普通技术人员在不付出创造性的劳动的情况下，即可以理解并实施。The device embodiments described above are only illustrative, and the units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in One place, or it can be distributed to multiple network elements. Part or all of the modules can be selected according to actual needs to achieve the purpose of the solution of this embodiment. It can be understood and implemented by those skilled in the art without any creative effort.
通过以上的实施方式的描述，本领域的技术人员可以清楚地了解到各实施方式可借助软件加必需的通用硬件平台的方式来实现，当然也可以通过硬件。基于这样的理解，上述技术方案本质上或者说对现有技术做出贡献的部分可以以软件产品的形式体现出来，该计算机软件产品可以存储在计算机可读存储介质中，如ROM/RAM、磁碟、光盘等，包括若干指令用以使得一台计算机设备(可以是个人计算机，服务器，或者网络设备等)执行各个实施例或者实施例的某些部分所述的方法。Through the above description of the implementations, those skilled in the art can clearly understand that each implementation can be implemented by means of software plus a necessary general-purpose hardware platform, and of course also by hardware. Based on this understanding, the essence of the above technical solution or the part that contributes to the prior art can be embodied in the form of software products, and the computer software products can be stored in computer-readable storage media, such as ROM/RAM, magnetic discs, optical discs, etc., including several instructions to make a computer device (which may be a personal computer, server, or network device, etc.) execute the methods described in various embodiments or some parts of the embodiments.
最后应说明的是：以上实施例仅用以说明本发明的技术方案，而非对其限制；尽管参照前述实施例对本发明进行了详细的说明，本领域的普通技术人员应当理解：其依然可以对前述各实施例所记载的技术方案进行修改，或者对其中部分技术特征进行等同替换；而这些修改或者替换，并不使相应技术方案的本质脱离本发明各实施例技术方案的精神和范围。Finally, it should be noted that: the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that: it can still be Modifications are made to the technical solutions described in the foregoing embodiments, or equivalent replacements are made to some of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the various embodiments of the present invention.
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310726458.4ACN116591006A (en) | 2023-06-16 | 2023-06-16 | Pavement abrasion detection method based on precise three-dimension |
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310726458.4ACN116591006A (en) | 2023-06-16 | 2023-06-16 | Pavement abrasion detection method based on precise three-dimension |
| Publication Number | Publication Date |
|---|---|
| CN116591006Atrue CN116591006A (en) | 2023-08-15 |
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310726458.4APendingCN116591006A (en) | 2023-06-16 | 2023-06-16 | Pavement abrasion detection method based on precise three-dimension |
| Country | Link |
|---|---|
| CN (1) | CN116591006A (en) |
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN119510731A (en)* | 2025-01-22 | 2025-02-25 | 广州市市维检测有限公司 | Road marking performance rapid detection method and device |
| CN119509448A (en)* | 2025-01-17 | 2025-02-25 | 交通运输部公路科学研究所 | A method for detecting error of road wear detection equipment |
| US20250196863A1 (en)* | 2023-12-13 | 2025-06-19 | Qualcomm Incorporated | Generating a road-height profile |
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| AU1653299A (en)* | 1997-12-22 | 1999-07-12 | Roads And Maritime Services | Road pavement deterioration inspection system |
| JP2008045273A (en)* | 2006-08-10 | 2008-02-28 | Bridgestone Corp | Elastic pavement repairing method |
| CN203113206U (en)* | 2013-01-31 | 2013-08-07 | 长安大学 | Height-adjustable concrete pavement grooving detection device |
| US20150285712A1 (en)* | 2014-04-03 | 2015-10-08 | The Goodyear Tire & Rubber Company | Model-based longitudinal stiffness estimation system and method |
| CN105839505A (en)* | 2016-03-18 | 2016-08-10 | 交通运输部科学研究院 | Three-dimensional visual detection method and detection device for road surface damage information |
| US20200307692A1 (en)* | 2019-03-28 | 2020-10-01 | Volkswagen Aktiengesellschaft | On-road localization methodologies and equipment utilizing road surface characteristics |
| CN112710250A (en)* | 2020-11-23 | 2021-04-27 | 武汉光谷卓越科技股份有限公司 | Three-dimensional measurement method based on line structured light and sensor |
| CN114387576A (en)* | 2021-12-09 | 2022-04-22 | 杭州电子科技大学信息工程学院 | A lane line identification method, system, medium, equipment and information processing terminal |
| CN114739312A (en)* | 2022-04-26 | 2022-07-12 | 黄晓明 | Hand-held type road surface structure degree of depth laser survey device |
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| AU1653299A (en)* | 1997-12-22 | 1999-07-12 | Roads And Maritime Services | Road pavement deterioration inspection system |
| JP2008045273A (en)* | 2006-08-10 | 2008-02-28 | Bridgestone Corp | Elastic pavement repairing method |
| CN203113206U (en)* | 2013-01-31 | 2013-08-07 | 长安大学 | Height-adjustable concrete pavement grooving detection device |
| US20150285712A1 (en)* | 2014-04-03 | 2015-10-08 | The Goodyear Tire & Rubber Company | Model-based longitudinal stiffness estimation system and method |
| CN105839505A (en)* | 2016-03-18 | 2016-08-10 | 交通运输部科学研究院 | Three-dimensional visual detection method and detection device for road surface damage information |
| US20200307692A1 (en)* | 2019-03-28 | 2020-10-01 | Volkswagen Aktiengesellschaft | On-road localization methodologies and equipment utilizing road surface characteristics |
| CN112710250A (en)* | 2020-11-23 | 2021-04-27 | 武汉光谷卓越科技股份有限公司 | Three-dimensional measurement method based on line structured light and sensor |
| CN114387576A (en)* | 2021-12-09 | 2022-04-22 | 杭州电子科技大学信息工程学院 | A lane line identification method, system, medium, equipment and information processing terminal |
| CN114739312A (en)* | 2022-04-26 | 2022-07-12 | 黄晓明 | Hand-held type road surface structure degree of depth laser survey device |
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20250196863A1 (en)* | 2023-12-13 | 2025-06-19 | Qualcomm Incorporated | Generating a road-height profile |
| CN119509448A (en)* | 2025-01-17 | 2025-02-25 | 交通运输部公路科学研究所 | A method for detecting error of road wear detection equipment |
| CN119509448B (en)* | 2025-01-17 | 2025-04-29 | 交通运输部公路科学研究所 | Error detection method of pavement abrasion detection equipment |
| CN119510731A (en)* | 2025-01-22 | 2025-02-25 | 广州市市维检测有限公司 | Road marking performance rapid detection method and device |
| CN119510731B (en)* | 2025-01-22 | 2025-05-06 | 广州市市维检测有限公司 | Rapid detection method and device for pavement marking performance |
| Publication | Publication Date | Title |
|---|---|---|
| CN116591006A (en) | Pavement abrasion detection method based on precise three-dimension | |
| CN109653045B (en) | Track gauge measuring method and device | |
| CN113638295B (en) | Full-width flatness detection method based on three-dimensional pavement | |
| CN112881042B (en) | Wheel set detection system, method and medium based on laser profile | |
| CN105006150B (en) | Method and device for detecting number of vehicle axles | |
| CN107705294B (en) | Cross laser image type roadbed surface settlement monitoring method and monitoring system | |
| CN110736999A (en) | Railway turnout detection method based on laser radar | |
| CN111207687B (en) | Boundary detection method, device and system for railway vehicle | |
| CN113256990B (en) | Method and system for collecting road vehicle information by radar based on clustering algorithm | |
| CN114842166B (en) | Negative obstacle detection method, system, medium and equipment for structured roads | |
| CN119556303B (en) | Unmanned vehicle road obstacle sensing system based on laser radar | |
| CN115356732A (en) | Potential Landslide Risk Area Identification Method Based on InSAR Deformation Results | |
| CN108089024A (en) | A kind of vehicle speed detector and method | |
| CN115825920B (en) | ICESat-2 photon denoising method considering glacier morphology | |
| US20160187467A1 (en) | Axle detection apparatus | |
| CN118422551A (en) | Road flatness detection method and equipment based on three-dimensional structure light vibration fusion | |
| CN113291207A (en) | Dynamic measurement method of rigid subway overhead line system | |
| CN114620091B (en) | A method for detecting out-of-roundness of train wheels based on three-dimensional information | |
| CN217543871U (en) | Wheel set tread polygon detection system | |
| CN119374525A (en) | Line laser road surface flatness measurement method and device | |
| CN120220428A (en) | Berth detection system and method | |
| CN116732852B (en) | Full-width construction depth detection method based on precise three-dimension | |
| JP5241556B2 (en) | Road surface condition estimation device | |
| CN116883340A (en) | Pavement well lid detection method based on precise three-dimension | |
| CN116732852A (en) | Full-width structural depth detection method based on precise three-dimensional |
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |