技术领域Technical Field
本发明涉及港口装船机自动化领域，具体地，涉及基于三维点云进行船舱检测的装船机防碰撞系统与方法。The present invention relates to the field of port ship loader automation, and in particular to a ship loader anti-collision system and method for performing cabin detection based on three-dimensional point clouds.
背景技术Background Art
装船机是一种在散煤、装卸作业中使用的大型机械，由于装船机的工作区域与驾驶室相距较远，司机室视角有限，看不到作业的全部过程，在使用集装箱及逆行装货时容易发生碰撞，影响工作效率。目前，解决这一问题的主要方法是采用视频监控系统，即在合适的地方安装摄像头，将装船部位的视频信号传至司机室，进而做出相应操作。Ship loader is a large machine used in bulk coal loading and unloading operations. Since the working area of the ship loader is far away from the cab, the driver's cab has a limited view and cannot see the entire process of the operation. It is easy to collide when using containers and reverse loading, affecting work efficiency. At present, the main way to solve this problem is to use a video monitoring system, that is, to install cameras in appropriate places, transmit the video signal of the loading area to the driver's cab, and then make corresponding operations.
发明内容Summary of the invention
本发明的目的是提供一种基于三维点云进行船舱检测的装船机防碰撞系统与方法，其可以防止装船机在使用集装箱进行装货时与船舱舱口发生碰撞。The object of the present invention is to provide a ship loader anti-collision system and method for cabin detection based on three-dimensional point cloud, which can prevent the ship loader from colliding with the cabin hatch when using containers for loading.
为了实现上述目的，本发明提供一种基于三维点云进行船舱检测的装船机防碰撞方法，该系统包括三维激光扫描仪，用于生成实时的三维船舱点云图；船舱检测与定位装置，通过对激光雷达获取的装船机三维点云数据进行一系列处理，识别船舱的位置和形状；臂架投影长度测量装置，用于确定所述装船机的臂架投影与船舱坐标的位置关系；以及控制装置，根据上述位置关系执行报警或控制装船机停止作业的操作。In order to achieve the above-mentioned purpose, the present invention provides a ship loader anti-collision method for cabin detection based on three-dimensional point cloud, the system comprising a three-dimensional laser scanner for generating a real-time three-dimensional cabin point cloud map; a cabin detection and positioning device for identifying the position and shape of the cabin by performing a series of processing on the three-dimensional point cloud data of the ship loader acquired by the laser radar; a boom projection length measuring device for determining the positional relationship between the boom projection of the ship loader and the cabin coordinates; and a control device for executing an alarm or controlling the ship loader to stop operating according to the above-mentioned positional relationship.
包含以下步骤：The following steps are involved:
步骤1、装船机上的角度传感器用于检测船体是否进入检测范围，当角度传感器判断到船体进入检测范围时，开始检测；所述三维激光扫描仪实时采集船舱点云数据。其中，根据激光扫描仪建立直角坐标系，所述激光扫描仪直角坐标系以扫描仪光点为原点，激光扫描仪在0°时，其方向向量在水平面上投影为x轴，z轴与重力方向相反，y轴方向由右手定则决定，xoy为水平面；Step 1: The angle sensor on the ship loader is used to detect whether the hull has entered the detection range. When the angle sensor determines that the hull has entered the detection range, the detection begins; the three-dimensional laser scanner collects the point cloud data of the cabin in real time. Among them, a rectangular coordinate system is established according to the laser scanner. The rectangular coordinate system of the laser scanner takes the scanner light spot as the origin. When the laser scanner is at 0°, its direction vector is projected on the horizontal plane as the x-axis, the z-axis is opposite to the direction of gravity, and the y-axis direction is determined by the right-hand rule. xoy is the horizontal plane.
步骤2、所述船舱检测与定位装置将三维点云数据转换为二维图，并通过灰度图存储三维图像的信息。二维图像中每个点的x轴和y轴坐标等同于点云中对应位置的x轴和y轴坐标，灰度图像中每个点的灰度值对于点云图中的z轴坐标；Step 2: The cabin detection and positioning device converts the three-dimensional point cloud data into a two-dimensional image and stores the information of the three-dimensional image through a grayscale image. The x-axis and y-axis coordinates of each point in the two-dimensional image are equivalent to the x-axis and y-axis coordinates of the corresponding position in the point cloud, and the grayscale value of each point in the grayscale image corresponds to the z-axis coordinate in the point cloud image;
步骤3、所述激光扫描仪位置固定且只能扫描扇形区域，通过已知条件，建立大小可变且可移动搜索的矩形框作为搜索种子，确定船舱角点的坐标；Step 3: The laser scanner is fixed in position and can only scan a sector-shaped area. Based on known conditions, a rectangular frame with a variable size and movable search is established as a search seed to determine the coordinates of the corner points of the cabin;
步骤4、利用标定的激光雷达参数，结合步骤3中的得到的角点位置与对应的灰度值，重新映射回点云数据中的空间位置坐标，进而标定船舱边框；Step 4: Use the calibrated LiDAR parameters, combined with the corner point positions and corresponding grayscale values obtained in step 3, to remap the spatial position coordinates in the point cloud data, and then calibrate the cabin border;
步骤5、所述臂架投影长度测量装置测量装船机臂架与船舱边框之间的位置关系，并将位置关系送入相应的控制装置；Step 5, the arm projection length measuring device measures the positional relationship between the arm of the ship loader and the cabin frame, and sends the positional relationship to the corresponding control device;
步骤6、所述控制装置在检测到位置关系低于预设阈值的情况下，发送急停信号，避免碰撞的发生。Step 6: When the control device detects that the position relationship is lower than a preset threshold, it sends an emergency stop signal to avoid a collision.
优选地，所述步骤2中包括：Preferably, the step 2 includes:
步骤2-1、首先对三维点云进行滤波增强与下采样，去除数据中的噪声与离散点；Step 2-1: First, filter, enhance and downsample the 3D point cloud to remove noise and discrete points in the data;
步骤2-2、将步骤2-1得到的三维点云图转换到二维灰度图，换算公式为：Step 2-2: Convert the three-dimensional point cloud image obtained in step 2-1 into a two-dimensional grayscale image. The conversion formula is:
方程中的“Linear scale”是指z轴坐标与灰度之间的映射系数，考虑港口的实际情况，选取Linear scale为0.25m-1； The "Linear scale" in the equation refers to the mapping coefficient between the z-axis coordinate and the grayscale. Considering the actual situation of the port, the Linear scale is selected as 0.25m-1 ;
步骤2-3、将步骤2-2得到的灰度图进行二值化处理，进一步降低维数，考虑港口的实际情况，选取港口的海岸线T作为阈值，换算公式为：Step 2-3: Binarize the grayscale image obtained in step 2-2 to further reduce the dimension. Considering the actual situation of the port, the coastline T of the port is selected as the threshold. The conversion formula is:
将二维图像存储在0-1矩阵中。 Store a 2D image in a 0-1 matrix.
优选地，所述步骤3中包括：Preferably, the step 3 includes:
步骤3-1、将步骤2-3得到的二维图像采用开闭运算，填充小的孔洞，平滑物体边缘，去除噪声，改善图像质量；Step 3-1, using opening and closing operations on the two-dimensional image obtained in step 2-3 to fill small holes, smooth the edges of objects, remove noise, and improve image quality;
步骤3-2、定义搜索种子矩阵，坐标位置为(m,n)，尺寸为(l,h)，从图像左下角开始移动。由于0-1矩阵中仅存在0与1，当搜索种子矩阵检测到能与矩阵顶点近似拟合的0、1交界点时，则确定为船舱角点的坐标，其中近似拟合的角度区间设置在85°-95°之间。Step 3-2, define the search seed matrix, with coordinate position (m,n), size (l,h), and move from the lower left corner of the image. Since there are only 0 and 1 in the 0-1 matrix, when the search seed matrix detects the intersection point of 0 and 1 that can be approximately fitted with the matrix vertex, it is determined as the coordinate of the cabin corner point, where the angle interval of the approximate fitting is set between 85° and 95°.
优选地，所述步骤4中包括：Preferably, the step 4 includes:
步骤4-1、通过坐标灰度值反推该点的z轴坐标，映射公式为Step 4-1: Use the grayscale value to reverse the z-axis coordinate of the point. The mapping formula is:
步骤4-2、将步骤3-2中得到的角点坐标映射到点云数据中的换算公式为：Step 4-2: The conversion formula for mapping the corner point coordinates obtained in step 3-2 to the point cloud data is:
其中为激光扫描仪的内参矩阵。 in is the internal parameter matrix of the laser scanner.
还提供一种运行任一所述的基于三维点云进行船舱检测的装船机防碰撞方法的系统，包括三维激光扫描仪，用于生成实时的三维船舱点云图；船舱检测与定位装置，通过对激光雷达获取的装船机三维点云数据进行一系列处理，识别船舱的位置和形状；臂架投影长度测量装置，用于确定所述装船机的臂架投影与船舱坐标的位置关系；以及控制装置，根据上述位置关系执行报警或控制装船机停止作业的操作。A system is also provided for running any of the above-mentioned ship loader anti-collision methods for performing cabin detection based on three-dimensional point clouds, comprising a three-dimensional laser scanner for generating a real-time three-dimensional cabin point cloud map; a cabin detection and positioning device for identifying the position and shape of the cabin by performing a series of processing on the three-dimensional point cloud data of the ship loader acquired by a laser radar; a boom projection length measuring device for determining the positional relationship between the boom projection of the ship loader and the cabin coordinates; and a control device for executing an alarm or controlling the ship loader to stop operating according to the above-mentioned positional relationship.
本发明专利的工作原理主要依赖于二维图像处理后的船舱检测，进而通过臂架投影长度测量装置确定位置关系，避免碰撞。The working principle of the patent of this invention mainly relies on the cabin detection after two-dimensional image processing, and then determines the position relationship through the boom projection length measurement device to avoid collision.
与现有的技术相比，本发明的优点是：Compared with the prior art, the advantages of the present invention are:
1.通过三维点云进行装船机舱口检测，避免司机室由于视觉受限造成的集装箱于舱口碰撞。1. Use 3D point cloud to detect the hatch of the loading machine to avoid collision between the container and the hatch due to limited vision in the driver's cab.
2.通过对三维点云进行二维图像处理，避免三维数据量过大影响检测的实时性。2. By performing two-dimensional image processing on the three-dimensional point cloud, the real-time performance of the detection can be prevented from being affected by excessive three-dimensional data.
附图说明BRIEF DESCRIPTION OF THE DRAWINGS
图1是本发明的基于三维点云进行船舱检测的装船机防碰撞流程图；FIG1 is a flow chart of ship loader anti-collision for cabin detection based on three-dimensional point cloud according to the present invention;
图2是本发明中实时船舱点云数据采集示意图；FIG2 is a schematic diagram of real-time cabin point cloud data acquisition in the present invention;
图3是本发明中船体点云数据采集示意图；FIG3 is a schematic diagram of hull point cloud data acquisition in the present invention;
图4是本发明中实时船舱投影示意图；FIG4 is a schematic diagram of real-time cabin projection in the present invention;
图5是本发明中实时船舱二值化示意图；FIG5 is a schematic diagram of real-time cabin binarization in the present invention;
图6是本发明的经过开闭运算的二维图像；FIG6 is a two-dimensional image after opening and closing operations of the present invention;
图7是本发明的船舱区域检测效果示意图；FIG7 is a schematic diagram of the cabin area detection effect of the present invention;
具体实施方式DETAILED DESCRIPTION
以下结合具体实施例，对本发明进行详细说明。The present invention is described in detail below in conjunction with specific embodiments.
本实施例的一种基于三维点云进行船舱检测的装船机防碰撞系统与方法，包括三维激光扫描仪，船舱检测与定位装置，臂架投影长度测量装置，以及控制装置。A ship loader anti-collision system and method for cabin detection based on three-dimensional point cloud in this embodiment includes a three-dimensional laser scanner, a cabin detection and positioning device, a boom projection length measuring device, and a control device.
如图1所示，本实施例的一种基于三维点云进行船舱检测的装船机防碰撞系统与方法如下：As shown in FIG1 , a ship loader anti-collision system and method for cabin detection based on three-dimensional point cloud in this embodiment is as follows:
步骤一、在每次角度传感器检测到船体到达指定区域后，激光扫描仪每隔0.25°发送和接受激光，利用激光信息构成一个弧形光学检测区域，以10s为一个单位，生成三维点云图，具体结果如图2所示；Step 1: After each angle sensor detects that the hull has reached the designated area, the laser scanner sends and receives lasers every 0.25°, and uses the laser information to form an arc-shaped optical detection area. A three-dimensional point cloud image is generated in units of 10 seconds. The specific results are shown in Figure 2.
步骤二、为简化运算，将步骤一中生成的三维点云图投影为二维灰度图，灰度图中的x、y坐标对应于点云中的x、y坐标，每个点的灰度值近似表示z轴的坐标。投影的换算公式为：Step 2: To simplify the calculation, project the 3D point cloud image generated in step 1 into a 2D grayscale image. The x and y coordinates in the grayscale image correspond to the x and y coordinates in the point cloud, and the grayscale value of each point approximately represents the coordinate of the z-axis. The projection conversion formula is:
其中xc、yc等价于x、y，只有z变为相应的灰度值，方程中的Linear scale是z轴坐标与灰度值之间的映射系数，由三维点云的整体坐标决定，选择为0.25，具体结果如图4所示。 Among them, xc and yc are equivalent to x and y, and only z becomes the corresponding grayscale value. The linear scale in the equation is the mapping coefficient between the z-axis coordinate and the grayscale value, which is determined by the overall coordinates of the 3D point cloud and is selected as 0.25. The specific results are shown in Figure 4.
步骤三、由于要通过二维图像实时识别船舱区域，灰度图中包含的信息依旧较为繁琐，因此我们选用港口海岸线的高度作为阈值，将步骤二得到的灰度图转为二值图，转换公式为Step 3: Since the cabin area needs to be identified in real time through a two-dimensional image, the information contained in the grayscale image is still relatively cumbersome. Therefore, we use the height of the port coastline as the threshold and convert the grayscale image obtained in step 2 into a binary image. The conversion formula is:
上述方程将船舱区域置0，船体置1，针对0、1交界处，进行舱口检测，具体结果如图5所示。 In the above equation, the cabin area is set to 0 and the hull is set to 1. The hatch detection is performed at the junction of 0 and 1. The specific results are shown in Figure 5.
步骤四、为实现实时船舱检测，步骤一中的激光扫描仪未能完整扫描到指定区域的所有信息，存在噪声与空洞。针对图像的形状分析，采用开闭运算以填充小空洞，删除不必要的噪声，具体结果如图6所示。Step 4: To achieve real-time cabin detection, the laser scanner in step 1 failed to completely scan all the information in the specified area, and there were noise and holes. Based on the shape analysis of the image, the opening and closing operations were used to fill the small holes and delete unnecessary noise. The specific results are shown in Figure 6.
步骤五，由于不确定要检测的船舱数量与位置，我们采用种子生长算法。首先，定义搜索种子矩阵，坐标位置为(m,n)，尺寸为(l,h)，从图像左下角开始移动。由于步骤四得到的0-1矩阵中仅存在0与1，当搜索种子矩阵检测到能与矩阵顶点近似拟合的0、1交接点，则确定为船舱角点的坐标，其中近似拟合的角度区间设置在85°-95°之间。如果该检测矩阵右侧与图像右侧重合，则停止检测，输出角点坐标，如果该检测矩阵右侧未达到图像右侧边缘，则重新定义搜索种子矩阵，继续进行拟合，直至与图像右侧边缘重合为止，检测结果如图7所示；Step five, since the number and position of the cabins to be detected are uncertain, we use a seed growth algorithm. First, define the search seed matrix with coordinate position (m,n) and size (l,h), and move from the lower left corner of the image. Since there are only 0 and 1 in the 0-1 matrix obtained in step four, when the search seed matrix detects the intersection points of 0 and 1 that can be approximately fitted with the matrix vertices, they are determined as the coordinates of the cabin corner points, where the approximate fitting angle interval is set between 85° and 95°. If the right side of the detection matrix coincides with the right side of the image, stop the detection and output the corner point coordinates. If the right side of the detection matrix does not reach the right edge of the image, redefine the search seed matrix and continue fitting until it coincides with the right edge of the image. The detection results are shown in Figure 7;
步骤六、利用步骤五得到的二值化图像坐标，映射到灰度图相应位置计算z轴坐标，映射关系为Step 6: Use the binary image coordinates obtained in step 5 to map to the corresponding position of the grayscale image to calculate the z-axis coordinates. The mapping relationship is:
步骤七、利用步骤六得到的角点坐标，通过激光扫描仪的内参矩阵，确定该点在实际空间中相对于扫描仪的具体位置，换算公式为：Step 7: Using the corner point coordinates obtained in step 6, determine the specific position of the point in the actual space relative to the scanner through the internal parameter matrix of the laser scanner. The conversion formula is:
步骤八、臂架投影长度测量装置通过对装船机臂架和船舱边框的位置关系进行精确计算，确定它们之间的相对距离和相对位置。通过将这一位置关系数据传输至相应的控制装置，系统能够实时获取关键的空间几何信息，以便更有效地进行装船操作。Step 8: The boom projection length measuring device accurately calculates the positional relationship between the loader boom and the cabin frame to determine the relative distance and relative position between them. By transmitting this positional relationship data to the corresponding control device, the system can obtain key spatial geometric information in real time to perform loading operations more efficiently.
步骤九、控制装置在监测到臂架与船舱边框位置关系低于预设阈值的情况下，立即采取紧急措施。具体而言，控制装置将发出急停信号，以迅速中止装船机的运动，防止发生潜在的碰撞事件。Step 9: When the control device detects that the position relationship between the boom and the cabin frame is lower than the preset threshold, the control device immediately takes emergency measures. Specifically, the control device will send an emergency stop signal to quickly stop the movement of the ship loader to prevent potential collision events.
应当理解的是，对本领域普通技术人员来说，可以根据上述说明加以改进或变换，而所有这些改进和变换都应属于本发明所附权利要求的保护范围。It should be understood that those skilled in the art can make improvements or changes based on the above description, and all these improvements and changes should fall within the scope of protection of the appended claims of the present invention.
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202410986582.9ACN118753744A (en) | 2024-07-23 | 2024-07-23 | Ship loader anti-collision system and method for cabin detection based on three-dimensional point cloud |
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202410986582.9ACN118753744A (en) | 2024-07-23 | 2024-07-23 | Ship loader anti-collision system and method for cabin detection based on three-dimensional point cloud |
| Publication Number | Publication Date |
|---|---|
| CN118753744Atrue CN118753744A (en) | 2024-10-11 |
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202410986582.9APendingCN118753744A (en) | 2024-07-23 | 2024-07-23 | Ship loader anti-collision system and method for cabin detection based on three-dimensional point cloud |
| Country | Link |
|---|---|
| CN (1) | CN118753744A (en) |
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN120589494A (en)* | 2025-08-08 | 2025-09-05 | 浙江天新智能研究院有限公司 | Spiral ship unloading cabin material collaborative sensing and autonomous operation method based on multi-source laser scanning array |
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| KR101494323B1 (en)* | 2013-08-14 | 2015-02-23 | (주)세아에스에이 | Method of anti-collision control in continuous ship unloader and apparatus thereof |
| CN112150388A (en)* | 2020-09-30 | 2020-12-29 | 大连华锐重工集团股份有限公司 | Continuous ship unloader ship and material identification sensing method |
| CN115909216A (en)* | 2022-12-13 | 2023-04-04 | 浙江大学 | A cargo ship hatch detection method and system based on laser radar and monocular camera |
| CN117974773A (en)* | 2023-09-25 | 2024-05-03 | 长江三峡通航管理局 | Method for calibrating bow direction based on geographic azimuth under ship static condition in ship lock |
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| KR101494323B1 (en)* | 2013-08-14 | 2015-02-23 | (주)세아에스에이 | Method of anti-collision control in continuous ship unloader and apparatus thereof |
| CN112150388A (en)* | 2020-09-30 | 2020-12-29 | 大连华锐重工集团股份有限公司 | Continuous ship unloader ship and material identification sensing method |
| CN115909216A (en)* | 2022-12-13 | 2023-04-04 | 浙江大学 | A cargo ship hatch detection method and system based on laser radar and monocular camera |
| CN117974773A (en)* | 2023-09-25 | 2024-05-03 | 长江三峡通航管理局 | Method for calibrating bow direction based on geographic azimuth under ship static condition in ship lock |
| Title |
|---|
| 欧阳兴东: "基于激光雷达的散货装船机智能防撞系统", 《港口装卸》, no. 5, 20 October 2023 (2023-10-20), pages 53 - 56* |
| 董席亮;宓超;沈阳;凤宇飞;: "基于激光三维视觉的散货船舱检测与定位算法研究", 中国工程机械学报, no. 06, 15 December 2014 (2014-12-15), pages 555 - 559* |
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN120589494A (en)* | 2025-08-08 | 2025-09-05 | 浙江天新智能研究院有限公司 | Spiral ship unloading cabin material collaborative sensing and autonomous operation method based on multi-source laser scanning array |
| Publication | Publication Date | Title |
|---|---|---|
| CN113376654B (en) | Method and device for detecting anti-smashing of integrated card based on three-dimensional laser and computer equipment | |
| CN113748357B (en) | Attitude correction method, device and system of laser radar | |
| US10776651B2 (en) | Material handling method, apparatus, and system for identification of a region-of-interest | |
| CN105894499B (en) | A kind of space object three-dimensional information rapid detection method based on binocular vision | |
| CN113110451B (en) | Mobile robot obstacle avoidance method based on fusion of depth camera and single-line laser radar | |
| JP5602392B2 (en) | Information processing apparatus, information processing method, and program | |
| US20060115113A1 (en) | Method for the recognition and tracking of objects | |
| CN112106111A (en) | Calibration method, calibration equipment, movable platform and storage medium | |
| WO2020215172A1 (en) | Obstacle detection method and device, mobile platform, and storage medium | |
| CN118753744A (en) | Ship loader anti-collision system and method for cabin detection based on three-dimensional point cloud | |
| TWI808434B (en) | Obstacle detection device and obstacle detection method | |
| KR102484298B1 (en) | An inspection robot of pipe and operating method of the same | |
| EP3846127A1 (en) | Dark parcel dimensioning | |
| Buck et al. | Generic 3D obstacle detection for AGVs using time-of-flight cameras | |
| CN116559887A (en) | Ship-shore co-positioning method and device for bulk cargo ship cabin cleaning operation and electronic equipment | |
| CN116922775A (en) | Printing abnormality detection method, 3D printer, and computer-readable storage medium | |
| WO2022054497A1 (en) | Filling rate measurement method, information processing device, and program | |
| CN114995387A (en) | Control method and device for intelligent guided transport vehicle | |
| US20230267593A1 (en) | Workpiece measurement method, workpiece measurement system, and program | |
| CN117741688A (en) | Open wagon empty box detection device and method | |
| CN116342832A (en) | A Container Feature Extraction Method Based on Dense Point Cloud | |
| CN119704163A (en) | Robot self-adaptive material grabbing method and device based on machine vision | |
| CN114545925A (en) | Compound robot control method and compound robot | |
| CN119600068B (en) | Point cloud registration method for positioning bulk cargo wharf engineering equipment and application | |
| Sansoni et al. | Combination of 2D and 3D vision systems into robotic cells for improved flexibility and performance |
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |