技术领域Technical Field
本发明涉及自动化技术领域，特别涉及基于三维重建的钢筋笼智能绑扎方法、装置、设备及介质。The present invention relates to the field of automation technology, and in particular to a method, device, equipment and medium for intelligently binding a steel cage based on three-dimensional reconstruction.
背景技术Background Art
建筑行业在工业化和自动化方面已经取得了巨大的进步。然而，钢筋绑扎工序作为建筑过程中不可或缺的一环，手工绑扎是一种常见的方法，通过使用铁丝或钢丝将钢筋捆扎在一起来完成。在这个过程中，工人通常使用钳子或专用的绑扎工具，通过扭转来固定钢丝，确保钢筋之间的连接紧密而稳固。但是依赖于人工作业的方式，不仅费时费力，还容易出现绑扎不牢、漏绑等现象，导致建筑结构的不稳定性和安全问题。而机器自动化作业场景中，一般通过双目相机拍摄钢筋图像，然后通过计算拍摄的钢筋图像的视差图获取深度信息，但是在实际应用场景中当钢筋表面纹理较弱或缺乏明显的纹理特征时，视差图可能对于这些区域的深度估计不够准确、且图像提供的集合信息有限，无法捕捉钢筋的详细三维形状，当在双层钢筋笼场景中，上下层钢筋存在交叉情况，此类钢筋交叉“假点”与需要绑扎的真实绑扎点在图像中难以区分，导致基于图像的钢筋绑扎点识别算法存在精度和鲁棒性的限制，因此现有自动绑扎技术无法应用到双层钢筋笼中。The construction industry has made great progress in industrialization and automation. However, as an indispensable part of the construction process, manual tying is a common method, which is completed by using iron wire or steel wire to tie the steel bars together. In this process, workers usually use pliers or special tying tools to fix the steel wire by twisting it to ensure that the connection between the steel bars is tight and stable. However, relying on manual work is not only time-consuming and labor-intensive, but also prone to loose tying, missing tying, etc., leading to instability and safety problems of the building structure. In machine automation operation scenarios, the steel bar image is generally captured by a binocular camera, and then the depth information is obtained by calculating the disparity map of the captured steel bar image. However, in actual application scenarios, when the surface texture of the steel bar is weak or lacks obvious texture features, the disparity map may not be accurate enough for the depth estimation of these areas, and the collective information provided by the image is limited, and it is impossible to capture the detailed three-dimensional shape of the steel bar. In the double-layer steel cage scenario, there is an intersection between the upper and lower layers of steel bars. Such steel bar intersection "false points" are difficult to distinguish from the real binding points that need to be tied in the image, resulting in the image-based steel bar binding point recognition algorithm having limitations in accuracy and robustness. Therefore, the existing automatic binding technology cannot be applied to double-layer steel cages.
综上，针对双层和单层钢筋笼的绑扎场景，如何实现机器人有序、准确、执行钢筋笼绑扎任务，优化绑扎任务的执行顺序，提高整体绑扎效率是本领域有待解决的技术问题。In summary, for the binding scenarios of double-layer and single-layer steel cages, how to enable the robot to execute the steel cage binding tasks in an orderly and accurate manner, optimize the execution order of the binding tasks, and improve the overall binding efficiency are technical problems to be solved in this field.
发明内容Summary of the invention
有鉴于此，本发明的目的在于提供基于三维重建的钢筋笼智能绑扎方法、装置、设备及介质，能够针对双层和单层钢筋笼的绑扎场景，实现机器人有序、准确、执行钢筋笼绑扎任务，优化绑扎任务的执行顺序，提高整体绑扎效率。其具体方案如下：In view of this, the purpose of the present invention is to provide a method, device, equipment and medium for intelligent tying of steel cages based on three-dimensional reconstruction, which can realize the orderly and accurate execution of steel cage tying tasks by robots for the tying scenarios of double-layer and single-layer steel cages, optimize the execution order of tying tasks, and improve the overall tying efficiency. The specific scheme is as follows:
第一方面，本申请公开了一种基于三维重建的钢筋笼智能绑扎方法，包括：In a first aspect, the present application discloses a steel cage intelligent binding method based on three-dimensional reconstruction, comprising:
根据预设手眼标定方式确定世界坐标系到机械臂基座坐标系下的目标变换矩阵；通过结构光相机与机械臂获取不同相机拍摄位姿对应的目标钢筋笼的点云数据及相机位姿信息；基于所述目标变换矩阵、所述相机位姿信息和所述点云数据重建双层钢筋笼点云模型；Determine the target transformation matrix from the world coordinate system to the robot base coordinate system according to a preset hand-eye calibration method; obtain the point cloud data and camera pose information of the target steel cage corresponding to different camera shooting poses through a structured light camera and a robot arm; reconstruct a double-layer steel cage point cloud model based on the target transformation matrix, the camera pose information and the point cloud data;
根据拟合地面得到的平面方程将所述双层钢筋笼点云模型的点云数据垂直于地面方向进行投影，并基于投影后的点云密度差将投影后的所述点云数据划分为竖向筋点云和水平筋点云；Projecting the point cloud data of the double-layer steel cage point cloud model perpendicular to the ground direction according to the plane equation obtained by fitting the ground, and dividing the projected point cloud data into vertical reinforcement point cloud and horizontal reinforcement point cloud based on the point cloud density difference after projection;
利用预设聚类算法分别对所述竖向筋点云和所述水平筋点云进行单根钢筋的识别和分离，以获取单根竖向筋和单根水平筋各自对应的单根钢筋点云数据；Using a preset clustering algorithm to identify and separate the single steel bar on the vertical steel bar point cloud and the horizontal steel bar point cloud, respectively, to obtain the single steel bar point cloud data corresponding to the single vertical steel bar and the single horizontal steel bar;
对所述单根钢筋点云数据进行圆柱体拟合，以得到各个所述单根钢筋的拟合后的圆柱体表达方程，基于所述圆柱体表达方程确定出钢筋绑扎点坐标信息，通过预设路径规划算法并根据所有所述钢筋绑扎点坐标信息进行路径规划，以得到包含绑扎顺序和绑扎点坐标信息的钢筋笼绑扎方案。A cylinder fitting is performed on the point cloud data of the single steel bar to obtain the fitted cylindrical expression equation of each of the single steel bars, and the coordinate information of the steel bar binding points is determined based on the cylindrical expression equation. A path planning algorithm is preset and path planning is performed based on the coordinate information of all the steel bar binding points to obtain a steel cage binding scheme including the binding sequence and the binding point coordinate information.
可选的，所述根据预设手眼标定方式确定世界坐标系到机械臂基座坐标系下的目标变换矩阵；通过结构光相机与机械臂获取不同相机拍摄位姿对应的目标钢筋笼的点云数据及相机位姿信息；基于所述目标变换矩阵、所述相机位姿信息和所述点云数据重建双层钢筋笼点云模型，包括：Optionally, the method includes determining a target transformation matrix from a world coordinate system to a robot base coordinate system according to a preset hand-eye calibration method; obtaining point cloud data and camera pose information of a target steel cage corresponding to different camera shooting poses through a structured light camera and a robot arm; and reconstructing a double-layer steel cage point cloud model based on the target transformation matrix, the camera pose information and the point cloud data, including:
将机械臂末端执行器作为手，将结构光相机作为眼，采用眼在手的手眼标定方式确定相机坐标系与机械臂末端坐标系之间的第一变换矩阵；The end effector of the robot arm is used as the hand, the structured light camera is used as the eye, and the first transformation matrix between the camera coordinate system and the end coordinate system of the robot arm is determined by using the eye-in-hand hand-eye calibration method;
基于所述机械臂末端执行器的末端位姿信息与机械臂基座坐标系的基座位姿信息计算并确定机械臂末端坐标系到机械臂基座坐标系之间的第二变换矩阵；Calculate and determine a second transformation matrix from the robot arm end coordinate system to the robot arm base coordinate system based on the end pose information of the robot arm end effector and the base pose information of the robot arm base coordinate system;
根据所述结构光相机的相机内参、畸变系数计算并确定世界坐标系到相机坐标系之间的第三变换矩阵；Calculate and determine a third transformation matrix from a world coordinate system to a camera coordinate system according to the camera intrinsic parameters and distortion coefficients of the structured light camera;
通过所述第一变换矩阵、所述第二变换矩阵、所述第三变换矩阵计算并确定世界坐标系到机械臂基座坐标系下的目标变换矩阵；Calculate and determine the target transformation matrix from the world coordinate system to the robot base coordinate system through the first transformation matrix, the second transformation matrix, and the third transformation matrix;
通过所述结构光相机分别在不同相机拍摄位姿下拍摄目标钢筋笼，以捕捉所述目标钢筋笼反射的结构化光计算对应的深度信息；The target steel cage is photographed by the structured light camera at different camera shooting positions to capture the structured light reflected by the target steel cage and calculate the corresponding depth information;
利用所述深度信息生成每一相机拍摄位姿下的所述目标钢筋笼的点云数据，并利用机械臂获取每一相机拍摄位姿下的相机位姿信息；Generate point cloud data of the target steel cage in each camera shooting posture using the depth information, and obtain camera posture information in each camera shooting posture using a robotic arm;
基于所述目标变换矩阵、所述相机位姿信息和所述点云数据确定所述目标钢筋笼的所述机械臂基座坐标系中的目标点云数据，以根据所述目标点云数据生成双层钢筋笼点云模型。The target point cloud data of the target steel cage in the robotic arm base coordinate system is determined based on the target transformation matrix, the camera pose information and the point cloud data, so as to generate a double-layer steel cage point cloud model according to the target point cloud data.
可选的，所述根据所述目标点云数据生成双层钢筋笼点云模型，包括：Optionally, generating a double-layer steel cage point cloud model according to the target point cloud data includes:
选择一个或多个坐标轴，并确定各所述坐标轴的最小值和最大值，以基于所述最小值和所述最大值构建目标保留范围；Selecting one or more coordinate axes, and determining a minimum value and a maximum value of each of the coordinate axes, so as to construct a target retention range based on the minimum value and the maximum value;
对目标点云数据中的各个目标点是否位于所述目标保留范围进行检查，并将位于所述目标保留范围外的目标点作为删除点，并将所述删除点从所述目标点云数据中删除，以得到去除环境点云的第一点云数据；Checking whether each target point in the target point cloud data is located in the target reserved range, and taking the target points outside the target reserved range as deletion points, and deleting the deletion points from the target point cloud data to obtain first point cloud data with the environment point cloud removed;
获取拟合地面得到的平面方程，并对所述第一点云数据中的所有目标点到所述平面方程的平面距离与预设距离阈值的大小进行比较，然后将所述平面距离小于所述预设距离阈值的目标点进行删除，以得到去除地面点云的第二点云数据，并基于所述第二点云数据生成双层钢筋笼点云模型。The plane equation obtained by fitting the ground is obtained, and the plane distances from all target points in the first point cloud data to the plane equation are compared with the size of a preset distance threshold, and then the target points whose plane distances are less than the preset distance threshold are deleted to obtain the second point cloud data with the ground point cloud removed, and a double-layer steel cage point cloud model is generated based on the second point cloud data.
可选的，所述根据拟合地面得到的平面方程将所述双层钢筋笼点云模型的点云数据垂直于地面方向进行投影，并基于投影后的点云密度差将投影后的所述点云数据划分为竖向筋点云和水平筋点云，包括：Optionally, projecting the point cloud data of the double-layer steel cage point cloud model perpendicular to the ground direction according to the plane equation obtained by fitting the ground, and dividing the projected point cloud data into vertical reinforcement point cloud and horizontal reinforcement point cloud based on the point cloud density difference after projection, including:
将所述双层钢筋笼点云模型的所述第二点云数据按照垂直于地面方向进行投影，以得到地面方向上的目标第二点云数据；Projecting the second point cloud data of the double-layer steel cage point cloud model in a direction perpendicular to the ground to obtain target second point cloud data in the ground direction;
统计在预设外包矩形网格内目标第二点云数量，通过计算各个预设外包矩形网格中统计的所述目标第二点云数量的离群因子对所述目标第二点云数据按照垂直方向和水平方向进行划分，以得到竖向筋点云和水平筋点云。The number of target second point clouds in a preset outer rectangular grid is counted, and the target second point cloud data is divided in a vertical direction and a horizontal direction by calculating the outlier factor of the number of target second point clouds counted in each preset outer rectangular grid to obtain a vertical rib point cloud and a horizontal rib point cloud.
可选的，所述统计在预设外包矩形网格内目标第二点云数量，通过计算各个预设外包矩形网格中统计的所述目标第二点云数量的离群因子对所述目标第二点云数据按照垂直方向和水平方向进行划分，以得到竖向筋点云和水平筋点云，包括：Optionally, the counting of the number of target second point clouds in a preset outer rectangular grid and the dividing of the target second point cloud data in a vertical direction and a horizontal direction by calculating the outlier factor of the number of target second point clouds counted in each preset outer rectangular grid to obtain a vertical rib point cloud and a horizontal rib point cloud include:
获取所述目标第二点云数据的最小外包矩形，对所述最小外包矩形按照长宽尺寸均为3毫米尺寸标准对所述最小外包矩形进行等距网格划分，以得到各预设外包矩形网格；Obtaining a minimum outer bounding rectangle of the target second point cloud data, and dividing the minimum outer bounding rectangle into equidistant grids according to a size standard in which both the length and width dimensions are 3 mm, so as to obtain preset outer bounding rectangle grids;
统计各所述预设外包矩形网格内目标第二点云数量，并通过局部离群因子检测算法计算各所述目标第二点云数量对应的离群因子；Counting the number of target second point clouds in each of the preset outer rectangular grids, and calculating the outlier factor corresponding to each of the number of target second point clouds by a local outlier factor detection algorithm;
利用所述离群因子作为点云密度差对所述目标第二点云数据按照垂直方向和水平方向进行划分，以得到平面上的平面竖向筋点云和平面水平筋点云；Using the outlier factor as the point cloud density difference to divide the target second point cloud data in the vertical direction and the horizontal direction to obtain a plane vertical rib point cloud and a plane horizontal rib point cloud on the plane;
对所述平面竖向筋点云和所述平面水平筋点云进行逆操作映射至原始双层钢筋笼点云模型，以便获取竖向筋点云和水平筋点云。The plane vertical reinforcement point cloud and the plane horizontal reinforcement point cloud are inversely mapped to the original double-layer reinforcement cage point cloud model to obtain the vertical reinforcement point cloud and the horizontal reinforcement point cloud.
可选的，所述对所述单根钢筋点云数据进行圆柱体拟合，以得到各个所述单根钢筋的拟合后的圆柱体表达方程，包括：Optionally, performing cylinder fitting on the single steel bar point cloud data to obtain a fitted cylinder expression equation of each single steel bar includes:
对所述单根钢筋点云数据的各个所述单根竖向筋进行圆柱体拟合，以便基于拟合后的所述竖向筋圆柱体的中心信息和方向向量信息确定所述单根竖向筋的半径参数、轴向量参数及拟合后的第一圆柱体表达方程；Performing cylinder fitting on each of the single vertical bars of the single steel bar point cloud data, so as to determine the radius parameter, the axis vector parameter and the fitted first cylinder expression equation of the single vertical bar based on the center information and the direction vector information of the fitted vertical bar cylinder;
通过直线拟合对所述单根钢筋点云数据的各个所述单根水平筋进行分段划分，以得到各所述单根水平筋的分段水平筋，对所述分段水平筋进行圆柱体拟合，以便基于拟合后的分段水平筋圆柱体的中心信息和方向向量信息确定所述单根水平筋的半径参数、轴向量参数及拟合后的第二圆柱体表达方程。The single horizontal reinforcement of the single steel bar point cloud data is segmented by straight line fitting to obtain the segmented horizontal reinforcement of each single horizontal reinforcement, and the segmented horizontal reinforcement is fitted with a cylinder so as to determine the radius parameter, axis vector parameter and the fitted second cylinder expression equation of the single horizontal reinforcement based on the center information and direction vector information of the fitted segmented horizontal reinforcement cylinder.
可选的，所述基于所述圆柱体表达方程确定出钢筋绑扎点坐标信息，通过预设路径规划算法并根据所有所述钢筋绑扎点坐标信息进行路径规划，以得到包含绑扎顺序和绑扎点坐标信息的钢筋笼绑扎方案，包括：Optionally, the coordinate information of the steel bar binding points is determined based on the cylinder expression equation, and path planning is performed according to the coordinate information of all the steel bar binding points through a preset path planning algorithm to obtain a steel cage binding scheme including the binding sequence and the binding point coordinate information, including:
基于所述钢筋空间形状中的半径参数、轴向量参数、第一圆柱体表达方程、第二圆柱体表达方程确定钢筋绑扎点及对应的钢筋绑扎点坐标信息；Determine the steel bar binding points and the corresponding steel bar binding point coordinate information based on the radius parameter, the axis vector parameter, the first cylinder expression equation, and the second cylinder expression equation in the steel bar spatial shape;
根据预设路径规划算法生成的随机采样点以及碰撞检测所述随机采样点与构建的树结构中的所述钢筋绑扎点坐标信息的最短距离，确定连接所有所述钢筋绑扎点的目标路径，以基于所述目标路径得到包含绑扎顺序和绑扎点坐标信息的钢筋笼绑扎方案。According to the random sampling points generated by the preset path planning algorithm and the shortest distance between the random sampling points and the coordinate information of the steel bar binding points in the constructed tree structure through collision detection, the target path connecting all the steel bar binding points is determined to obtain the steel cage binding scheme including the binding sequence and the binding point coordinate information based on the target path.
第二方面，本申请公开了一种基于三维重建的钢筋笼智能绑扎装置，包括：In a second aspect, the present application discloses a steel cage intelligent binding device based on three-dimensional reconstruction, comprising:
模型构建模块，用于根据预设手眼标定方式确定世界坐标系到机械臂基座坐标系下的目标变换矩阵；通过结构光相机与机械臂获取不同相机拍摄位姿对应的目标钢筋笼的点云数据及相机位姿信息；基于所述目标变换矩阵、所述相机位姿信息和所述点云数据重建双层钢筋笼点云模型；A model building module is used to determine the target transformation matrix from the world coordinate system to the robot base coordinate system according to a preset hand-eye calibration method; obtain the point cloud data and camera pose information of the target steel cage corresponding to different camera shooting poses through a structured light camera and a robot arm; and reconstruct a double-layer steel cage point cloud model based on the target transformation matrix, the camera pose information and the point cloud data;
点云分割模块，用于根据拟合地面得到的平面方程将所述双层钢筋笼点云模型的点云数据垂直于地面方向进行投影，并基于投影后的点云密度差将投影后的所述点云数据划分为竖向筋点云和水平筋点云；A point cloud segmentation module, used for projecting the point cloud data of the double-layer steel cage point cloud model perpendicular to the ground direction according to the plane equation obtained by fitting the ground, and dividing the projected point cloud data into vertical reinforcement point cloud and horizontal reinforcement point cloud based on the density difference of the projected point cloud;
点云聚类模块，用于利用预设聚类算法分别对所述竖向筋点云和所述水平筋点云进行单根钢筋的识别和分离，以获取单根竖向筋和单根水平筋各自对应的单根钢筋点云数据；A point cloud clustering module, used to identify and separate the single steel bar on the vertical bar point cloud and the horizontal bar point cloud respectively by using a preset clustering algorithm, so as to obtain the single steel bar point cloud data corresponding to the single vertical bar and the single horizontal bar respectively;
坐标结算模块，用于对所述单根钢筋点云数据进行圆柱体拟合，以得到各个所述单根钢筋的拟合后的圆柱体表达方程，基于所述圆柱体表达方程确定出钢筋绑扎点坐标信息；A coordinate settlement module, used for performing cylinder fitting on the single steel bar point cloud data to obtain the fitted cylinder expression equation of each single steel bar, and determining the coordinate information of the steel bar binding point based on the cylinder expression equation;
路径规划模块，用于通过预设路径规划算法并根据所有所述钢筋绑扎点坐标信息进行路径规划，以得到包含绑扎顺序和绑扎点坐标信息的钢筋笼绑扎方案。The path planning module is used to perform path planning based on the preset path planning algorithm and the coordinate information of all the steel bar binding points to obtain a steel cage binding plan including the binding sequence and the binding point coordinate information.
第三方面，本申请公开了一种电子设备，包括：In a third aspect, the present application discloses an electronic device, including:
存储器，用于保存计算机程序；Memory, used to store computer programs;
处理器，用于执行所述计算机程序，以实现前述公开的基于三维重建的钢筋笼智能绑扎方法的步骤。A processor is used to execute the computer program to implement the steps of the aforementioned three-dimensional reconstruction-based intelligent binding method for steel cages.
第四方面，本申请公开了一种计算机可读存储介质，用于存储计算机程序；其中，所述计算机程序被处理器执行时实现前述公开的基于三维重建的钢筋笼智能绑扎方法的步骤。In a fourth aspect, the present application discloses a computer-readable storage medium for storing a computer program; wherein, when the computer program is executed by a processor, the steps of the aforementioned disclosed method for intelligent binding of steel cages based on three-dimensional reconstruction are implemented.
可见，本申请公开了一种基于三维重建的钢筋笼智能绑扎方法，包括：根据预设手眼标定方式确定世界坐标系到机械臂基座坐标系下的目标变换矩阵；通过结构光相机与机械臂获取不同相机拍摄位姿对应的目标钢筋笼的点云数据及相机位姿信息；基于所述目标变换矩阵、所述相机位姿信息和所述点云数据重建双层钢筋笼点云模型；根据拟合地面得到的平面方程将所述双层钢筋笼点云模型的点云数据垂直于地面方向进行投影，并基于投影后的点云密度差将投影后的所述点云数据划分为竖向筋点云和水平筋点云；利用预设聚类算法分别对所述竖向筋点云和所述水平筋点云进行单根钢筋的识别和分离，以获取单根竖向筋和单根水平筋各自对应的单根钢筋点云数据；对所述单根钢筋点云数据进行圆柱体拟合，以得到各个所述单根钢筋的拟合后的圆柱体表达方程，基于所述圆柱体表达方程确定出钢筋绑扎点坐标信息，通过预设路径规划算法并根据所有所述钢筋绑扎点坐标信息进行路径规划，以得到包含绑扎顺序和绑扎点坐标信息的钢筋笼绑扎方案。由此可见，通过利用结构光相机获取钢筋信息，结构光相机在捕捉物体的形状和结构方面表现优越，具有更高的深度测量精确度和分辨率，因此，捕获的目标钢筋笼信息能够建立精确钢筋三维模型，根据获取的点云数据能够直接表示钢筋在三维空间中的位置和形状，提供更直观、更全面的钢筋空间结构形式，有利于获取更加准确的钢筋交叉点坐标信息，然后将钢筋模型投影至平面，并利用竖向筋和水平筋的点云数量差异，将钢筋点云模型分为竖向筋和水平筋两个类别，之后通过聚类以及竖向筋点云和水平筋点云不同的数据点密度分布进行点云合理分割，实现竖向和水平向各自的聚类，克服了双层钢筋笼点云模型由于其纵横交错的特点无法直接聚类识别单根钢筋点云簇的问题、最后拟合出钢筋空间形状，进一步计算并提取该钢筋空间形状中所有绑扎点的空间坐标，以便基于空间坐标进行路径规划，引导机器人进行钢筋绑扎操作。It can be seen that the present application discloses a method for intelligent binding of steel cages based on three-dimensional reconstruction, including: determining the target transformation matrix from the world coordinate system to the robot arm base coordinate system according to a preset hand-eye calibration method; obtaining point cloud data and camera pose information of the target steel cage corresponding to different camera shooting poses through a structured light camera and a robot arm; reconstructing a double-layer steel cage point cloud model based on the target transformation matrix, the camera pose information and the point cloud data; projecting the point cloud data of the double-layer steel cage point cloud model perpendicular to the ground direction according to the plane equation obtained by fitting the ground, and projecting the point cloud data after projection based on the density difference of the projected point cloud. The data is divided into vertical reinforcement point cloud and horizontal reinforcement point cloud; a preset clustering algorithm is used to identify and separate the single steel bar on the vertical reinforcement point cloud and the horizontal reinforcement point cloud respectively, so as to obtain the single steel bar point cloud data corresponding to the single vertical reinforcement and the single horizontal reinforcement respectively; a cylinder fitting is performed on the single steel bar point cloud data to obtain the fitted cylinder expression equation of each of the single steel bars, and the coordinate information of the steel bar binding point is determined based on the cylinder expression equation. A preset path planning algorithm is used and path planning is performed according to the coordinate information of all the steel bar binding points to obtain a steel cage binding scheme including the binding sequence and the binding point coordinate information. It can be seen that by using the structured light camera to obtain the steel bar information, the structured light camera performs well in capturing the shape and structure of the object, and has higher depth measurement accuracy and resolution. Therefore, the captured target steel cage information can establish an accurate steel bar three-dimensional model. According to the acquired point cloud data, the position and shape of the steel bar in the three-dimensional space can be directly represented, providing a more intuitive and comprehensive steel bar spatial structure form, which is conducive to obtaining more accurate steel bar intersection coordinate information. Then the steel bar model is projected onto the plane, and the steel bar point cloud model is divided into two categories of vertical bars and horizontal bars by using the difference in the number of point clouds of vertical bars and horizontal bars. After that, the point cloud is reasonably segmented through clustering and different data point density distributions of vertical bar point clouds and horizontal bar point clouds, and vertical and horizontal clustering is achieved. The problem that the double-layer steel cage point cloud model cannot directly cluster and identify a single steel bar point cloud cluster due to its criss-crossing characteristics is overcome. Finally, the steel bar spatial shape is fitted, and the spatial coordinates of all binding points in the steel bar spatial shape are further calculated and extracted, so as to perform path planning based on the spatial coordinates and guide the robot to perform steel bar binding operations.
附图说明BRIEF DESCRIPTION OF THE DRAWINGS
为了更清楚地说明本发明实施例或现有技术中的技术方案，下面将对实施例或现有技术描述中所需要使用的附图作简单地介绍，显而易见地，下面描述中的附图仅仅是本发明的实施例，对于本领域普通技术人员来讲，在不付出创造性劳动的前提下，还可以根据提供的附图获得其他的附图。In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying creative work.
图1为本申请公开的一种基于三维重建的钢筋笼智能绑扎方法流程图；FIG1 is a flow chart of a steel cage intelligent binding method based on three-dimensional reconstruction disclosed in the present application;
图2为本申请公开的一种不同坐标系的坐标系示意图；FIG2 is a schematic diagram of a coordinate system of a different coordinate system disclosed in the present application;
图3为本申请公开的一种基于直通滤波方法去除周围环境点云原理图；FIG3 is a schematic diagram of a method for removing surrounding environment point clouds based on a straight-through filtering method disclosed in the present application;
图4为本申请公开的一种去除地面点云方法流程图；FIG4 is a flow chart of a method for removing ground point clouds disclosed in the present application;
图5为本申请公开的一种去除环境点云及地面点云后的双层钢筋笼点云模型图；FIG5 is a point cloud model diagram of a double-layer steel cage after removing the environmental point cloud and the ground point cloud disclosed in the present application;
图6为本申请公开的一种点云聚类算法原理示意图；FIG6 is a schematic diagram of a point cloud clustering algorithm disclosed in the present application;
图7为本申请公开的一种竖向筋与水平筋点云聚类效果图，其中，（a）为竖向筋聚类效果图，（b）为水平筋聚类效果图；FIG. 7 is a point cloud clustering effect diagram of vertical ribs and horizontal ribs disclosed in the present application, wherein (a) is a vertical rib clustering effect diagram, and (b) is a horizontal rib clustering effect diagram;
图8为本申请公开的一种钢筋空间形状拟合算法流程图；FIG8 is a flow chart of a reinforcement spatial shape fitting algorithm disclosed in the present application;
图9为本申请公开的一种水平筋钢筋圆柱体直线拟合算法流程图；FIG9 is a flow chart of a linear fitting algorithm for a horizontal steel bar cylinder disclosed in the present application;
图10为本申请公开的一种钢筋绑扎点空间坐标定位推导示意图；FIG10 is a schematic diagram of a derivation of spatial coordinate positioning of a reinforcement binding point disclosed in the present application;
图11为本申请公开的一种竖向筋与水平筋拟合方法流程图；FIG11 is a flow chart of a vertical rib and horizontal rib fitting method disclosed in the present application;
图12为本申请公开的一种竖向筋与水平筋点云模型图，其中，（a）为竖向筋点云模型图，（b）表示水平筋点云模型图；FIG12 is a point cloud model diagram of a vertical rib and a horizontal rib disclosed in the present application, wherein (a) is a point cloud model diagram of a vertical rib, and (b) is a point cloud model diagram of a horizontal rib;
图13为本申请公开的一种具体的双层钢筋笼绑扎方法流程图；FIG13 is a flow chart of a specific double-layer steel cage binding method disclosed in the present application;
图14为本申请公开的一种基于三维重建的钢筋笼绑扎智能装置结构示意图；FIG14 is a schematic structural diagram of an intelligent device for tying steel cages based on three-dimensional reconstruction disclosed in the present application;
图15为本申请公开的一种电子设备结构图。FIG. 15 is a structural diagram of an electronic device disclosed in the present application.
具体实施方式DETAILED DESCRIPTION
下面将结合本申请实施例中的附图，对本申请实施例中的技术方案进行清楚、完整地描述，显然，所描述的实施例仅仅是本发明一部分实施例，而不是全部的实施例。基于本发明中的实施例，本领域普通技术人员在没有做出创造性劳动前提下所获得的所有其他实施例，都属于本发明保护的范围。The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.
本发明提供了一种基于三维重建的钢筋笼智能绑扎方案，能够针对双层和单层钢筋笼的绑扎场景，实现机器人有序、准确、执行钢筋笼绑扎任务，优化绑扎任务的执行顺序，提高整体绑扎效率。The present invention provides an intelligent steel cage binding scheme based on three-dimensional reconstruction, which can realize the robot's orderly and accurate execution of steel cage binding tasks for double-layer and single-layer steel cage binding scenarios, optimize the execution order of binding tasks, and improve the overall binding efficiency.
参照图1所示，本发明实施例公开了一种基于三维重建的钢筋笼智能绑扎方法，包括：1 , an embodiment of the present invention discloses a steel cage intelligent binding method based on three-dimensional reconstruction, comprising:
步骤S11：根据预设手眼标定方式确定世界坐标系到机械臂基座坐标系下的目标变换矩阵；通过结构光相机与机械臂获取不同相机拍摄位姿对应的目标钢筋笼的点云数据及相机位姿信息；基于所述目标变换矩阵、所述相机位姿信息和所述点云数据重建双层钢筋笼点云模型。Step S11: Determine the target transformation matrix from the world coordinate system to the robot arm base coordinate system according to a preset hand-eye calibration method; obtain the point cloud data and camera pose information of the target steel cage corresponding to different camera shooting postures through a structured light camera and a robot arm; reconstruct a double-layer steel cage point cloud model based on the target transformation matrix, the camera pose information and the point cloud data.
本实施例中，在获取双层钢筋笼点云模型的过程中，首先根据预设手眼标定方式确定世界坐标系到机械臂基座坐标系下的目标变换矩阵，这样一来，能够将世界坐标系下的钢筋笼坐标信息转为机械臂基座坐标系下的坐标，能够在后续规划绑扎路线后，通过机器人及对应的各个绑扎点在机械臂基座坐标系下的坐标信息按照规划绑扎路线自动绑扎。具体的，将机械臂末端执行器作为手，将结构光相机作为眼，采用眼在手的手眼标定方式确定相机坐标系与机械臂末端坐标系之间的第一变换矩阵；基于所述机械臂末端执行器的末端位姿信息与机械臂基座坐标系的基座位姿信息计算并确定机械臂末端坐标系到机械臂基座坐标系之间的第二变换矩阵；根据所述结构光相机的相机内参、畸变系数计算并确定世界坐标系到相机坐标系之间的第三变换矩阵；通过所述第一变换矩阵、所述第二变换矩阵、所述第三变换矩阵计算并确定世界坐标系到机械臂基座坐标系下的目标变换矩阵。可以理解的是，由于在使用结构光相机获取钢筋在世界坐标系中的位姿信息的过程中，钢筋的位姿是相对于相机坐标系的，机械臂的末端绑扎执行器需要知道钢筋绑扎点在机械臂坐标系（通常是机械臂基座）下的位姿，以便精确地执行绑扎操作。手眼标定（Hand-Eye Calibration）是一种用于确定机器人末端执行器（手）和传感器（通常是相机，称为眼）之间坐标变换关系的技术，其主要目的是获得相机坐标系与机械臂末端坐标系之间的变换矩阵，实现在机械臂控制和相机数据处理中的协同工作，各坐标系示意如图2所示。因此，采用眼在手上的标定方式获取，也即相机坐标系到机械臂末端坐标系之间的变换矩阵（第一变换矩阵），具体过程如下：In this embodiment, in the process of obtaining the double-layer steel cage point cloud model, the target transformation matrix from the world coordinate system to the robot base coordinate system is first determined according to the preset hand-eye calibration method, so that the steel cage coordinate information in the world coordinate system can be converted into the coordinates in the robot base coordinate system, and after the subsequent planning of the binding route, the robot and the corresponding coordinate information of each binding point in the robot base coordinate system can be automatically bound according to the planned binding route. Specifically, the robot end effector is used as the hand and the structured light camera is used as the eye, and the eye-in-hand hand-eye calibration method is used to determine the first transformation matrix between the camera coordinate system and the robot end coordinate system; the second transformation matrix between the robot end coordinate system and the robot base coordinate system is calculated and determined based on the end posture information of the robot end effector and the base posture information of the robot base coordinate system; the third transformation matrix between the world coordinate system and the camera coordinate system is calculated and determined according to the camera intrinsic parameters and distortion coefficients of the structured light camera; the target transformation matrix from the world coordinate system to the robot base coordinate system is calculated and determined through the first transformation matrix, the second transformation matrix, and the third transformation matrix. It is understandable that, since the posture of the steel bar is relative to the camera coordinate system when using a structured light camera to obtain the posture information of the steel bar in the world coordinate system, the end-binding actuator of the robot arm needs to know the posture of the steel bar binding point in the robot arm coordinate system (usually the robot arm base) in order to accurately perform the binding operation. Hand-Eye Calibration is a technology used to determine the coordinate transformation relationship between the robot end effector (hand) and the sensor (usually a camera, called the eye). Its main purpose is to obtain the transformation matrix between the camera coordinate system and the end-coordinate system of the robot arm, and to achieve collaborative work in robot arm control and camera data processing. The schematic diagram of each coordinate system is shown in Figure 2. Therefore, the eye-on-hand calibration method is used to obtain , that is, the transformation matrix between the camera coordinate system and the robot end coordinate system (the first transformation matrix). The specific process is as follows:
将结构光相机固定在机械臂上，当机械臂移动时，标定板和机器臂的基座保持不动，标定板上一点在机械臂基座坐标系和世界坐标系下的坐标值和不变，则有：Fix the structured light camera on the robotic arm. When the robotic arm moves, the calibration plate and the base of the robotic arm remain stationary. The coordinate value of a point on the calibration plate in the robotic arm base coordinate system and the world coordinate system and If unchanged, then:
； ;
其中，为机械臂末端坐标系到基座坐标系之间的变换矩阵，为机械臂处于初始位置时，机械臂末端坐标系到基座坐标系之间的变换矩阵，为世界坐标系到相机坐标系之间的变换矩阵，为机械臂处于初始位置时，世界坐标系到相机坐标系之间的变换矩阵。移动机械臂，对于同一点：in, is the transformation matrix between the robot end coordinate system and the base coordinate system, is the transformation matrix between the end coordinate system of the robot arm and the base coordinate system when the robot arm is in the initial position, is the transformation matrix from the world coordinate system to the camera coordinate system, is the transformation matrix between the world coordinate system and the camera coordinate system when the robot is in the initial position. Move the robot, for the same point:
； ;
因此，可以得到：Therefore, we can get:
； ;
其中，可由机械臂示教器得到的机械臂末端执行器位姿与基座坐标系中的位姿计算得到，也即第二变换矩阵、可由相机内参与畸变系数计算得到，也即第三变换矩阵，为在移动机械臂后，此时机械臂末端坐标系到基座坐标系之间的变换矩阵，为在移动机械臂后，世界坐标系到相机坐标系之间的变换矩阵，联立求解以上方程即可得到第一变换矩阵。in, It can be calculated from the pose of the end effector of the robot arm obtained by the robot arm teaching device and the pose in the base coordinate system, that is, the second transformation matrix, It can be calculated by the distortion coefficients in the camera, that is, the third transformation matrix, is the transformation matrix between the end coordinate system of the robot arm and the base coordinate system after the robot arm is moved. is the transformation matrix from the world coordinate system to the camera coordinate system after the robot arm is moved. Solving the above equations together can get the first transformation matrix .
因此，获得世界坐标系到基座坐标系的目标变换矩阵，例如：世界坐标系中任意钢筋绑扎点，基座坐标系下表示为：Therefore, we obtain the target transformation matrix from the world coordinate system to the base coordinate system , for example: any reinforcement binding point in the world coordinate system , expressed in the base coordinate system as:
。 .
本实施例中，当确定目标变换矩阵后，进一步通过结构光相机与机械臂获取不同相机拍摄位姿对应的目标钢筋笼的点云数据及相机位姿信息。具体的，通过所述结构光相机分别在不同相机拍摄位姿下拍摄目标钢筋笼，以捕捉所述目标钢筋笼反射的结构化光计算对应的深度信息；利用所述深度信息生成每一相机拍摄位姿下的所述目标钢筋笼的点云数据，并利用机械臂获取每一相机拍摄位姿下的相机位姿信息。可以理解的是，使用结构光相机获取目标钢筋笼在世界坐标系中的位姿信息，具体过程如下：确定钢筋的形状和尺寸、相机拍摄的起始位置后利用三维路径规划算法自动规划相机拍摄路径，以覆盖整个钢筋的区域。通过结构光相机与机械臂拍摄可以直接生成双层钢筋的点云数据，其工作原理是通过投射结构化光或光栅模式到场景上，然后通过结构光相机捕捉被反射的结构化光来计算深度信息，这个深度信息可以用来生成点云数据，其中每个点的位置由其在三维空间中的坐标表示。通过手眼标定的结果（目标变换矩阵），可以计算出结构光相机相对于机械臂基座的初始位姿，每次移动拍摄后，将机械臂的末端位姿和结构光相机的初始位姿组合，可以计算出当前相机相对于机械臂底座位姿，用变换矩阵来表示:，其中，、分别代表第次拍摄时结构光相机的旋转与平移，在三维向量的末尾添加1，使其变成齐次坐标，方便计算。储存每一次拍摄所获取的点云数据以及相应的相机拍摄位姿，为下一步钢筋的三维重建提供数据基础。In this embodiment, after determining the target transformation matrix, the point cloud data and camera pose information of the target steel cage corresponding to different camera shooting postures are further obtained by the structured light camera and the mechanical arm. Specifically, the target steel cage is photographed by the structured light camera at different camera shooting postures to capture the structured light reflected by the target steel cage to calculate the corresponding depth information; the point cloud data of the target steel cage under each camera shooting posture is generated by using the depth information, and the camera pose information under each camera shooting posture is obtained by using the mechanical arm. It can be understood that the posture information of the target steel cage in the world coordinate system is obtained by using the structured light camera. The specific process is as follows: after determining the shape and size of the steel bar and the starting position of the camera shooting, the camera shooting path is automatically planned using the three-dimensional path planning algorithm to cover the entire steel bar area. The point cloud data of the double-layer steel bar can be directly generated by shooting with a structured light camera and a mechanical arm. The working principle is to project structured light or a grating pattern onto the scene, and then capture the reflected structured light by the structured light camera to calculate the depth information. This depth information can be used to generate point cloud data, in which the position of each point is represented by its coordinates in three-dimensional space. The initial position of the structured light camera relative to the base of the robotic arm can be calculated through the result of hand-eye calibration (target transformation matrix). After each mobile shooting, the end position of the robotic arm and the initial position of the structured light camera are combined to calculate the current position of the camera relative to the base of the robotic arm. , expressed as a transformation matrix: ,in, , Respectively represent The rotation and translation of the structured light camera during each shot, add 1 to the end of the 3D vector to make it a homogeneous coordinate for easy calculation. Store the point cloud data obtained for each shot And the corresponding camera shooting pose , providing a data basis for the next step of three-dimensional reconstruction of steel bars.
本实施例中，基于所述目标变换矩阵、所述相机位姿信息和所述点云数据确定所述目标钢筋笼的所述机械臂基座坐标系中的目标点云数据，以根据所述目标点云数据生成双层钢筋笼点云模型。具体的，获取的双层钢筋点云数据集是相对于相机坐标系中的，本发明将根据相机位姿进行坐标变换，把采集到的双层钢筋的局部点云转化为基座坐标系中的点云，具体计算为：对于局部点云模型，相机位姿为，可得中任意一点在机械臂基座坐标系下表示为，将以上计算应用至所有局部点云模型中，可以得到一个基于基座坐标系下的点云集。将点云集中的点云数据进行合并，以形成一个完整的双层钢筋笼点云模型，此时的完整双层钢筋笼点云模型为原始双层钢筋笼点云模型，也即包含了周围环境信息和地面环境信息的钢筋笼点云模型，此外还需要去除重复点云数据，以确保点云数据的一致性和准确性。In this embodiment, the target point cloud data of the target steel cage in the robot arm base coordinate system is determined based on the target transformation matrix, the camera pose information and the point cloud data, so as to generate a double-layer steel cage point cloud model according to the target point cloud data. Specifically, the obtained double-layer steel cage point cloud data set is relative to the camera coordinate system. The present invention will be based on the camera pose Perform coordinate transformation to transform the collected local point cloud of the double-layer steel bar into the point cloud in the base coordinate system. The specific calculation is as follows: For the local point cloud model , the camera pose is , can be obtained Any point in In the robot base coordinate system, it is expressed as , applying the above calculation to all local point cloud models, we can get a point cloud set based on the base coordinate system . Gather the point cloud The point cloud data in are merged to form a complete double-layer steel cage point cloud model. At this time, the complete double-layer steel cage point cloud model is the original double-layer steel cage point cloud model, that is, the steel cage point cloud model that contains the surrounding environment information and the ground environment information. In addition, it is necessary to remove duplicate point cloud data to ensure the consistency and accuracy of the point cloud data.
本实施例中，在生成双层钢筋笼点云模型的过程中，需要将原始双层钢筋笼点云模型的周围环境信息和地面环境信息的点云数据进行去除，以获取去噪之后仅包含钢筋点云数据的双层笼钢筋点云模型，其中，去除周围环境点云数据和地面点云数据，具体的：选择一个或多个坐标轴，并确定各所述坐标轴的最小值和最大值，以基于所述最小值和所述最大值构建目标保留范围；对目标点云数据中的各个目标点是否位于所述目标保留范围进行检查，并将位于所述目标保留范围外的目标点作为删除点，并将所述删除点从所述目标点云数据中删除，以得到去除环境点云的第一点云数据；获取拟合地面得到的平面方程，并对所述第一点云数据中的所有目标点到所述平面方程的平面距离与预设距离阈值的大小进行比较，然后将所述平面距离小于所述预设距离阈值的目标点进行删除，以得到去除地面点云的第二点云数据，并基于所述第二点云数据生成双层钢筋笼点云模型。可以理解的是，从前一步获得的三维模型中，不仅包含了双层钢筋点云模型，还包括了周围环境或其他物体的点云数据。为了进一步分析和处理双层钢筋，需要执行一个附加步骤来提取和保留仅与钢筋相关的点云数据，而忽略掉其他部分的点云数据。本发明提出利用点云直通滤波来去除不与双层钢筋直接接触的周围环境或其他物体。如图3所示，指定一个或多个坐标轴（例如，x、y、z），以及每个坐标轴上的最小值和最大值，以得到一个在Min（x，y，z）至Max（x，y，z）的坐标轴指定范围，对于每个点，检查它在各个坐标轴上的数值是否在指定范围内。如果点的坐标在指定范围内，作为保留点将其保留；否则，作为删除点将其删除，至此得到第一点云数据。由于地面与双层钢筋直接接触，难以确定一个合适的阈值来准确区分地面点云和钢筋点云。因此，为了有效去除地面点云，本发明采用平面拟合方法来更精确地检测和去除与钢筋直接接触的地面点云，而不需要依赖事先定义的阈值范围。在平面拟合过程中，平面方程由法向量和一个点来表示，假设平面法向量为，平面上存在一点为，则平面方程可以表示为，其中，计算双层钢筋模型中每个点到其拟合的平面距离，如果小于预定的阈值，则被标记为内点，通过计算内点数量，可以评估方程的拟合质量。如图4所示，利用RANSAC算法迭代执行这些步骤，直到找到最佳拟合平面的法向量和一个点，平面方程为：，同时去除平面的内点，仅保留钢筋点云数据。至此，双层钢筋模型中仅保留与钢筋相关的点云数据，周围环境与物体点云数据已被去除，如图5所示。In this embodiment, in the process of generating a double-layer steel cage point cloud model, it is necessary to remove the point cloud data of the surrounding environment information and the ground environment information of the original double-layer steel cage point cloud model to obtain a double-layer cage steel point cloud model that only contains steel point cloud data after denoising, wherein the surrounding environment point cloud data and the ground point cloud data are removed, specifically: one or more coordinate axes are selected, and the minimum and maximum values of each of the coordinate axes are determined to construct a target retention range based on the minimum and maximum values; whether each target point in the target point cloud data is located in the target retention range is checked, and the target points located outside the target retention range are used as deletion points, and the deleted points are deleted from the target point cloud data to obtain the first point cloud data with the environmental point cloud removed; the plane equation obtained by fitting the ground is obtained, and the plane distances of all target points in the first point cloud data to the plane equation are compared with the size of a preset distance threshold, and then the target points whose plane distances are less than the preset distance threshold are deleted to obtain the second point cloud data with the ground point cloud removed, and the double-layer steel cage point cloud model is generated based on the second point cloud data. It can be understood that the three-dimensional model obtained from the previous step not only includes the double-layer steel bar point cloud model, but also includes the point cloud data of the surrounding environment or other objects. In order to further analyze and process the double-layer steel bars, an additional step needs to be performed to extract and retain the point cloud data related only to the steel bars, while ignoring the point cloud data of other parts. The present invention proposes to use point cloud pass-through filtering to remove the surrounding environment or other objects that are not in direct contact with the double-layer steel bars. As shown in Figure 3, one or more coordinate axes (for example, x, y, z), as well as the minimum and maximum values on each coordinate axis, are specified to obtain a coordinate axis specified range from Min (x, y, z) to Max (x, y, z). For each point, check whether its value on each coordinate axis is within the specified range. If the coordinates of the point are within the specified range, it is retained as a retained point; otherwise, it is deleted as a deleted point, and the first point cloud data is obtained. Since the ground is in direct contact with the double-layer steel bars, it is difficult to determine a suitable threshold to accurately distinguish between the ground point cloud and the steel bar point cloud. Therefore, in order to effectively remove the ground point cloud, the present invention adopts a plane fitting method to more accurately detect and remove the ground point cloud that is in direct contact with the steel bar without relying on a predefined threshold range. In the plane fitting process, the plane equation is represented by a normal vector and a point. Assume that the plane normal vector is , there is a point on the plane , then the plane equation can be expressed as ,in , calculate each point in the double-layer steel bar model The distance to the plane it fits to ,if If the value is less than a predetermined threshold, it is marked as an inlier. By counting the number of inliers, the fitting quality of the equation can be evaluated. As shown in Figure 4, the RANSAC algorithm is used to iteratively perform these steps until the normal vector and a point of the best fitting plane are found. The plane equation is: , and remove the inner points of the plane at the same time, leaving only the point cloud data of the steel bar. So far, only the point cloud data related to the steel bar is retained in the double-layer steel bar model, and the point cloud data of the surrounding environment and objects have been removed, as shown in Figure 5.
相较于TOF相机来说，TOF相机对时间测量设备要求高，其深度分辨率和精度可能无法满足高精度深度测量的要求，而双目相机的深度信息是通过计算视差图来获得的，当钢筋表面纹理较弱或缺乏明显的纹理特征时，视差图可能对于这些区域的深度估计不够准确。因此，利用结构光相机获取钢筋信息能够使捕捉物体的形状和结构的表现优越，具有更高的深度测量精确度和分辨率，尤其是对于近距离和高精度要求的场景，适用于需要建立精确钢筋三维模型的应用。并且，现有技术中，图像提供的几何信息相对有限，无法捕捉目标的详细三维形状，双层钢筋笼是一个复杂的几何结构，上下层钢筋存在交叉情况，此类钢筋交叉“假点”与需要绑扎的真实绑扎点在图像中难以区分，导致基于图像的钢筋绑扎点识别算法存在精度和鲁棒性的限制。而本发明将点云作为数据输入，点云数据能够直接表示物体在三维空间中的位置和形状，提供更直观、更全面的钢筋空间结构形式，有利于后期算法处理得到准确的钢筋交叉点坐标。通过利用结构光相机获取钢筋信息，结构光相机在捕捉物体的形状和结构方面表现优越，具有更高的深度测量精确度和分辨率，因此，捕获的目标钢筋笼信息能够建立精确钢筋三维模型。Compared with TOF cameras, TOF cameras have high requirements for time measurement equipment, and their depth resolution and accuracy may not meet the requirements of high-precision depth measurement. The depth information of binocular cameras is obtained by calculating disparity maps. When the surface texture of the steel bar is weak or lacks obvious texture features, the disparity map may not be accurate enough for the depth estimation of these areas. Therefore, using a structured light camera to obtain steel bar information can make the performance of capturing the shape and structure of the object superior, with higher depth measurement accuracy and resolution, especially for scenes with close distances and high precision requirements, and is suitable for applications that require the establishment of accurate steel bar three-dimensional models. In addition, in the prior art, the geometric information provided by the image is relatively limited, and the detailed three-dimensional shape of the target cannot be captured. The double-layer steel cage is a complex geometric structure, and there is an intersection between the upper and lower layers of steel bars. Such steel bar intersection "false points" are difficult to distinguish from the real binding points that need to be tied in the image, resulting in the image-based steel bar binding point recognition algorithm being limited in accuracy and robustness. The present invention uses point cloud as data input, and point cloud data can directly represent the position and shape of the object in three-dimensional space, providing a more intuitive and comprehensive steel bar spatial structure form, which is conducive to the later algorithm processing to obtain accurate steel bar intersection coordinates. By using a structured light camera to obtain steel bar information, the structured light camera is superior in capturing the shape and structure of objects and has higher depth measurement accuracy and resolution. Therefore, the captured target steel cage information can establish an accurate three-dimensional steel bar model.
步骤S12：根据拟合地面得到的平面方程将所述双层钢筋笼点云模型的点云数据垂直于地面方向进行投影，并基于投影后的点云密度差将投影后的所述点云数据划分为竖向筋点云和水平筋点云。Step S12: projecting the point cloud data of the double-layer steel cage point cloud model perpendicular to the ground direction according to the plane equation obtained by fitting the ground, and dividing the projected point cloud data into vertical reinforcement point cloud and horizontal reinforcement point cloud based on the density difference of the projected point cloud.
本实施例中，将去除环境点云数据和地面点云数据的点云形成的双层钢筋笼点云模型按照垂直地面方向对其进行投影，能够得到投影后的包含竖向筋点云和水平筋点云，并基于投影后的点云密度差对其进行分割操作；然后将分割后的竖向筋点云数据和水平筋点云数据映射到原始三维模型中，得到以得到竖向筋点云和水平筋点云。In this embodiment, the double-layer steel cage point cloud model formed by removing the point cloud of the environmental point cloud data and the ground point cloud data is projected in a direction perpendicular to the ground to obtain the projected point cloud containing vertical reinforcement point cloud and horizontal reinforcement point cloud, and the point cloud is segmented based on the density difference of the projected point cloud; the segmented vertical reinforcement point cloud data and horizontal reinforcement point cloud data are then mapped to the original three-dimensional model to obtain the vertical reinforcement point cloud and horizontal reinforcement point cloud.
步骤S13：利用预设聚类算法分别对所述竖向筋点云和所述水平筋点云进行单根钢筋的识别和分离，以获取单根竖向筋和单根水平筋各自对应的单根钢筋点云数据。Step S13: using a preset clustering algorithm to identify and separate the single steel bar on the vertical bar point cloud and the horizontal bar point cloud, respectively, to obtain the single steel bar point cloud data corresponding to the single vertical bar and the single horizontal bar respectively.
本实施例中，由于每一根竖向筋点云和每一根水平筋点云在空间上彼此是明确分开的。本发明利用DBSCAN（Density-Based Spatial Clustering of Applications withNoise）聚类算法，高精度地将各个竖向筋对应的单根钢筋点云数据和各个水平筋对应的单根钢筋点云数据精确识别和分离，便于后续对于钢筋空间形状的拟合。DBSCAN是一种基于密度的聚类分析算法，其核心原理在于将数据点聚合成紧密相邻的群集，这些群集在密度上显著区别于周围的数据点，如图6所示。具体利用DBSCAN聚类算法对目标竖向筋点云和目标水平筋点云聚类操作的具体流程如下：首先，从双层钢筋模型选择一个起始点，计算其邻域内的数据点密度，如果该点的邻域内包含足够多的数据点，它被标记为核心点。然后，从核心点出发，通过密度可达性链接，将相邻核心点以及它们的密度可达点聚合成一个聚类。重复这个过程，直到没有更多的核心点可访问，从而形成多个钢筋聚类。竖向筋与水平筋聚类效果如图7所示，不同形态特征表示不同点云族。In this embodiment, since each vertical reinforcement point cloud and each horizontal reinforcement point cloud are clearly separated from each other in space. The present invention uses the DBSCAN (Density-Based Spatial Clustering of Applications with Noise) clustering algorithm to accurately identify and separate the single steel bar point cloud data corresponding to each vertical reinforcement and the single steel bar point cloud data corresponding to each horizontal reinforcement with high precision, so as to facilitate the subsequent fitting of the reinforcement space shape. DBSCAN is a density-based clustering analysis algorithm, and its core principle is to aggregate data points into closely adjacent clusters, which are significantly different from the surrounding data points in density, as shown in Figure 6. The specific process of clustering the target vertical reinforcement point cloud and the target horizontal reinforcement point cloud using the DBSCAN clustering algorithm is as follows: First, select a starting point from the double-layer reinforcement model, calculate the density of data points in its neighborhood, and if the neighborhood of the point contains enough data points, it is marked as a core point. Then, starting from the core point, the adjacent core points and their density-reachable points are aggregated into a cluster through density accessibility links. This process is repeated until no more core points are accessible, thus forming multiple reinforcement clusters. The clustering effect of vertical and horizontal reinforcements is shown in Figure 7. Different morphological features represent different point cloud families.
步骤S14：对所述单根钢筋点云数据进行圆柱体拟合，以得到各个所述单根钢筋的拟合后的圆柱体表达方程，基于所述圆柱体表达方程确定出钢筋绑扎点坐标信息，通过预设路径规划算法并根据所有所述钢筋绑扎点坐标信息进行路径规划，以得到包含绑扎顺序和绑扎点坐标信息的钢筋笼绑扎方案。Step S14: Perform cylinder fitting on the single steel bar point cloud data to obtain the fitted cylindrical expression equation of each of the single steel bars, determine the coordinate information of the steel bar binding points based on the cylindrical expression equation, and perform path planning through a preset path planning algorithm and based on the coordinate information of all the steel bar binding points to obtain a steel cage binding scheme including the binding sequence and binding point coordinate information.
本实施例中，所述对所述单根钢筋点云数据进行圆柱体拟合，以得到各个所述单根钢筋的拟合后的圆柱体表达方程，包括：对所述单根钢筋点云数据的各个所述单根竖向筋进行圆柱体拟合，以便基于拟合后的竖向筋圆柱体的中心信息和方向向量信息确定所述单根竖向筋的半径参数、轴向量参数及拟合后的第一圆柱体表达方程；通过直线拟合对所述单根钢筋点云数据的各个所述单根水平筋进行分段划分，以得到各所述单根水平筋的分段水平筋，对所述分段水平筋进行圆柱体拟合，以便基于拟合后的分段水平筋圆柱体的中心信息和方向向量信息确定所述单根水平筋的半径参数、轴向量参数及拟合后的第二圆柱体表达方程。可以理解的是，钢筋直径、轴向量等参数对于确定两根钢筋的交叉绑扎点至关重要，由于钢筋的形状与圆柱体相似，通过圆柱体拟合可以获得钢筋的这些关键参数。具体实现方法如图8所示：首先，通过主成分分析（Principal Component Analysis，PCA）估计初始的圆柱体方向，然后，通过运行BFGS算法迭代地最小化点到圆柱体轴线的距离标准差，当迭代次数达到预设次数阈值时，停止迭代，计算所有点到轴线的距离平均值，获取拟合后的竖向钢筋的半径信息，以实现拟合圆柱体的中心和方向向量。由于水平筋环绕在竖向筋的四周，无法直接利用圆柱体拟合算法估算钢筋直径、圆心坐标等参数，因此，本发明先使用直线拟合算法将一根水平筋划分为四根钢筋，通过对每根钢筋进行圆柱体拟合，即可得到水平筋参数信息。其中所采用的圆柱体拟合算法与竖向筋的圆柱体拟合算法相同，不再赘述，通过RANSAC拟合直线，计算方法如下：三维直线表示为：，其中，是直线的方向向量，t表示直线上的点，可以在整个实数范围内变化，对于点到直线的距离。RANSAC算法通过迭代随机选择一些样本，拟合模型，然后通过内点的数量评估模型的好坏，最终得到一组最佳的内点和直线拟合结果，计算流程如图9所示。In this embodiment, the cylinder fitting of the single steel bar point cloud data to obtain the fitted cylinder expression equation of each single steel bar includes: cylinder fitting of each single vertical bar of the single steel bar point cloud data to determine the radius parameter, axis vector parameter and the fitted first cylinder expression equation of the single vertical bar based on the center information and direction vector information of the fitted vertical bar cylinder; segmenting each single horizontal bar of the single steel bar point cloud data by linear fitting to obtain the segmented horizontal bar of each single horizontal bar, and cylinder fitting of the segmented horizontal bar to determine the radius parameter, axis vector parameter and the fitted second cylinder expression equation of the single horizontal bar based on the center information and direction vector information of the fitted segmented horizontal bar cylinder. It can be understood that parameters such as steel bar diameter and axis vector are crucial to determining the cross-binding point of two steel bars. Since the shape of the steel bar is similar to that of a cylinder, these key parameters of the steel bar can be obtained by cylinder fitting. The specific implementation method is shown in Figure 8: First, estimate the initial cylinder direction through principal component analysis (PCA), then iteratively minimize the standard deviation of the distance from the point to the cylinder axis by running the BFGS algorithm. When the number of iterations reaches the preset threshold, stop the iteration, calculate the average distance from all points to the axis, and obtain the radius information of the fitted vertical steel bars to achieve the center and direction vector of the fitted cylinder. Since the horizontal bars surround the vertical bars, it is impossible to directly use the cylinder fitting algorithm to estimate the parameters such as the steel bar diameter and the center coordinates. Therefore, the present invention first uses a straight line fitting algorithm to divide a horizontal bar into four steel bars. By fitting each steel bar with a cylinder, the horizontal bar parameter information can be obtained. The cylinder fitting algorithm used is the same as the cylinder fitting algorithm for the vertical bars, so it will not be repeated. The straight line is fitted by RANSAC, and the calculation method is as follows: The three-dimensional straight line is expressed as: , in, is the direction vector of the line, t represents the point on the line, and can vary over the entire real number range. To the straight line Distance The RANSAC algorithm randomly selects some samples through iteration, fits the model, and then evaluates the quality of the model by the number of inliers, and finally obtains a set of optimal inliers and straight line fitting results. The calculation process is shown in Figure 9.
本实施例中，基于半径参数、轴向量参数的参数信息、第一圆柱体表达方程、第二圆柱体表达方程确定钢筋绑扎点及对应的钢筋绑扎点坐标信息；根据预设路径规划算法生成的随机采样点以及碰撞检测所述随机采样点与构建的树结构中的所述钢筋绑扎点坐标信息的最短距离，确定连接所有所述钢筋绑扎点的目标路径，以基于所述目标路径得到包含绑扎顺序和绑扎点坐标信息的钢筋笼绑扎方案。可以理解的是，当获取了竖向筋聚类和水平筋聚类后各自进行圆柱体拟合后的第一圆柱体表达方程和第二圆柱体表达方及对应的参数信息后，建立圆柱体表达方程与绑扎点空间坐标之间的数学模型，根据上述目标竖向钢筋的半径参数、轴向量参数以及目标水平钢筋的参数信息进行钢筋空间形状的拟合，以得到各个拟合后的钢筋空间形状；其中，目标竖向钢筋的轴向量参数为中轴线信息，目标水平钢筋的参数信息包括：中轴线信息和半径信息。具体的，已知水平筋圆柱体的中轴线，半径为，竖向筋圆柱体的中轴线，半径为，具体绑扎点空间坐标推导过程如下，各点、线、面如图10所示。可见，通过建立钢筋的圆柱体方程与钢筋绑扎点之间的数学模型，实现了对钢筋交叉点的精确描述和定位。In this embodiment, the steel bar binding points and the corresponding steel bar binding point coordinate information are determined based on the parameter information of the radius parameter, the axis vector parameter, the first cylinder expression equation, and the second cylinder expression equation; the target path connecting all the steel bar binding points is determined according to the random sampling points generated by the preset path planning algorithm and the shortest distance between the random sampling points and the coordinate information of the steel bar binding points in the constructed tree structure by collision detection, so as to obtain the steel cage binding scheme containing the binding sequence and the binding point coordinate information based on the target path. It can be understood that after obtaining the first cylinder expression equation and the second cylinder expression equation and the corresponding parameter information after the vertical reinforcement clustering and the horizontal reinforcement clustering are respectively fitted with the cylinder, a mathematical model between the cylinder expression equation and the binding point spatial coordinates is established, and the steel bar spatial shape is fitted according to the radius parameter, the axis vector parameter and the parameter information of the target horizontal reinforcement of the above-mentioned target vertical reinforcement, so as to obtain each fitted reinforcement spatial shape; wherein, the axis vector parameter of the target vertical reinforcement is the central axis information, and the parameter information of the target horizontal reinforcement includes: central axis information and radius information. Specifically, the central axis of the horizontal reinforcement cylinder is known. , the radius is , the center axis of the vertical rib cylinder , the radius is The specific derivation process of the spatial coordinates of the binding points is as follows, and the points, lines, and surfaces are shown in Figure 10. It can be seen that by establishing the mathematical model between the cylindrical equation of the steel bar and the steel bar binding point, the accurate description and positioning of the steel bar intersection point is achieved.
首先，过，垂直于，作平面，然后过，垂直于平面，作平面，求解平面与平面的交线，最后求解与水平钢筋圆柱面之间的交点，将距离中轴线较远的点则为钢筋绑扎点，并获对应的该钢筋绑扎点坐标信息。当确定双层钢筋笼绑扎点位的空间坐标值，利用快速遍历随机树算法（Rapidly-Exploring Random Tree，RRT）进行绑扎点位路径规划，其主要原理为通过迭代式的随机采样和节点扩展，逐步建立一颗树状结构，其中起始节点为已知的绑扎点。在每次迭代中，算法随机生成新的采样点，然后连接已有树中距离最近的节点，并在连接路径上进行碰撞检测以确保路径的可行性。通过不断迭代，RRT算法能够快速探索搜索空间，找到连接所有绑扎点的最优路径，并最终形成一颗树，该树的末端节点即为最终的路径。这种基于随机采样和节点连接的方式使得算法能够在高维空间中高效地寻找复杂结构下的绑扎路径，为机械臂等设备提供了有效的导航引导。First, , perpendicular to , make a plane , then pass , perpendicular to the plane , make a plane , solve the plane With plane The intersection of , and finally solve The intersection point with the horizontal steel bar cylinder is The farther point is the steel bar binding point, and the corresponding coordinate information of the steel bar binding point is obtained. When the spatial coordinate value of the binding point of the double-layer steel cage is determined, the Rapidly-Exploring Random Tree (RRT) algorithm is used to plan the binding point path. Its main principle is to gradually establish a tree structure through iterative random sampling and node expansion, in which the starting node is a known binding point. In each iteration, the algorithm randomly generates new sampling points, then connects the nearest nodes in the existing tree, and performs collision detection on the connection path to ensure the feasibility of the path. Through continuous iteration, the RRT algorithm can quickly explore the search space, find the optimal path connecting all binding points, and finally form a tree, the end node of which is the final path. This method based on random sampling and node connection enables the algorithm to efficiently find binding paths under complex structures in high-dimensional space, providing effective navigation guidance for equipment such as robotic arms.
可见，本申请公开了一种基于三维重建的钢筋笼智能绑扎方法，包括：根据预设手眼标定方式确定世界坐标系到机械臂基座坐标系下的目标变换矩阵；通过结构光相机与机械臂获取不同相机拍摄位姿对应的目标钢筋笼的点云数据及相机位姿信息；基于所述目标变换矩阵、所述相机位姿信息和所述点云数据重建双层钢筋笼点云模型；根据拟合地面得到的平面方程将所述双层钢筋笼点云模型的点云数据垂直于地面方向进行投影，并基于投影后的点云密度差将投影后的所述点云数据划分为竖向筋点云和水平筋点云；利用预设聚类算法分别对所述竖向筋点云和所述水平筋点云进行单根钢筋的识别和分离，以获取单根竖向筋和单根水平筋各自对应的单根钢筋点云数据；对所述单根钢筋点云数据进行圆柱体拟合，以得到各个所述单根钢筋的拟合后的圆柱体表达方程，基于所述圆柱体表达方程确定出钢筋绑扎点坐标信息，通过预设路径规划算法并根据所有所述钢筋绑扎点坐标信息进行路径规划，以得到包含绑扎顺序和绑扎点坐标信息的钢筋笼绑扎方案。由此可见，通过利用结构光相机获取钢筋信息，结构光相机在捕捉物体的形状和结构方面表现优越，具有更高的深度测量精确度和分辨率，因此，捕获的目标钢筋笼信息能够建立精确钢筋三维模型，根据获取的点云数据能够直接表示钢筋在三维空间中的位置和形状，提供更直观、更全面的钢筋空间结构形式，有利于获取更加准确的钢筋交叉点坐标信息，然后将钢筋模型投影至平面，并利用竖向筋和水平筋的点云数量差异，将钢筋点云模型分为竖向筋和水平筋两个类别，之后通过聚类以及竖向筋点云和水平筋点云不同的数据点密度分布进行点云合理分割，实现竖向和水平向各自的聚类，克服了双层钢筋笼点云模型由于其纵横交错的特点无法直接聚类识别单根钢筋点云簇的问题、最后拟合出钢筋空间形状，进一步计算并提取该钢筋空间形状中所有绑扎点的空间坐标，以便基于空间坐标进行路径规划，引导机器人进行钢筋绑扎操作。It can be seen that the present application discloses a method for intelligent binding of steel cages based on three-dimensional reconstruction, including: determining the target transformation matrix from the world coordinate system to the robot arm base coordinate system according to a preset hand-eye calibration method; obtaining point cloud data and camera pose information of the target steel cage corresponding to different camera shooting poses through a structured light camera and a robot arm; reconstructing a double-layer steel cage point cloud model based on the target transformation matrix, the camera pose information and the point cloud data; projecting the point cloud data of the double-layer steel cage point cloud model perpendicular to the ground direction according to the plane equation obtained by fitting the ground, and projecting the point cloud data after projection based on the density difference of the projected point cloud. The data is divided into vertical reinforcement point cloud and horizontal reinforcement point cloud; a preset clustering algorithm is used to identify and separate the single steel bar on the vertical reinforcement point cloud and the horizontal reinforcement point cloud respectively, so as to obtain the single steel bar point cloud data corresponding to the single vertical reinforcement and the single horizontal reinforcement respectively; a cylinder fitting is performed on the single steel bar point cloud data to obtain the fitted cylinder expression equation of each of the single steel bars, and the coordinate information of the steel bar binding point is determined based on the cylinder expression equation, and a preset path planning algorithm is used and path planning is performed according to the coordinate information of all the steel bar binding points to obtain a steel cage binding scheme including the binding sequence and the binding point coordinate information. It can be seen that by using the structured light camera to obtain the steel bar information, the structured light camera performs well in capturing the shape and structure of the object, and has higher depth measurement accuracy and resolution. Therefore, the captured target steel cage information can establish an accurate steel bar three-dimensional model. According to the acquired point cloud data, the position and shape of the steel bar in the three-dimensional space can be directly represented, providing a more intuitive and comprehensive steel bar spatial structure form, which is conducive to obtaining more accurate steel bar intersection coordinate information. Then the steel bar model is projected onto the plane, and the steel bar point cloud model is divided into two categories of vertical bars and horizontal bars by using the difference in the number of point clouds of vertical bars and horizontal bars. After that, the point cloud is reasonably segmented through clustering and different data point density distributions of vertical bar point clouds and horizontal bar point clouds, and vertical and horizontal clustering is achieved. The problem that the double-layer steel cage point cloud model cannot directly cluster and identify a single steel bar point cloud cluster due to its criss-crossing characteristics is overcome. Finally, the steel bar spatial shape is fitted, and the spatial coordinates of all binding points in the steel bar spatial shape are further calculated and extracted, so as to perform path planning based on the spatial coordinates and guide the robot to perform steel bar binding operations.
参照图11所示，本发明实施例公开了步骤S12：根据拟合地面得到的平面方程将所述双层钢筋笼点云模型的点云数据垂直于地面方向进行投影，并基于投影后的点云密度差将投影后的所述点云数据进行点云分割，以得到竖向筋点云和水平筋点云，相对于上一实施例，本实施例对技术方案作了进一步的说明和优化。具体的：As shown in FIG11 , the embodiment of the present invention discloses step S12: projecting the point cloud data of the double-layer steel cage point cloud model perpendicular to the ground direction according to the plane equation obtained by fitting the ground, and performing point cloud segmentation on the projected point cloud data based on the point cloud density difference after projection to obtain vertical reinforcement point cloud and horizontal reinforcement point cloud. Compared with the previous embodiment, this embodiment further explains and optimizes the technical solution. Specifically:
步骤S1211：将所述双层钢筋笼点云模型的所述第二点云数据按照垂直于地面方向进行投影，以得到地面方向上的目标第二点云数据。Step S1211: projecting the second point cloud data of the double-layer steel cage point cloud model in a direction perpendicular to the ground to obtain target second point cloud data in the ground direction.
本实施例中，将双层钢筋点云垂直于地平面（）进行投影，以获得一个平面上的钢筋点云模型。同时，设置钢筋点云最小外包矩形，并进一步将钢筋点云最小外包矩形等距划分长宽尺寸均为3毫米的外包矩形网格。In this embodiment, the double-layer steel bar point cloud is perpendicular to the ground plane ( ) to obtain a steel point cloud model on a plane. At the same time, the minimum outer rectangle of the steel point cloud is set, and the minimum outer rectangle of the steel point cloud is further divided into outer rectangle grids with length and width of 3 mm at equal intervals.
步骤S1212：统计在预设外包矩形网格内目标第二点云数量，通过计算各个预设外包矩形网格中统计的所述目标第二点云数量的离群因子对所述目标第二点云数据按照垂直方向和水平方向进行划分，以得到竖向筋点云和水平筋点云。Step S1212: Count the number of target second point clouds in the preset outer rectangular grid, and divide the target second point cloud data in the vertical direction and the horizontal direction by calculating the outlier factor of the number of target second point clouds counted in each preset outer rectangular grid to obtain vertical rib point cloud and horizontal rib point cloud.
本实施例中，获取所述目标第二点云数据的最小外包矩形，对所述最小外包矩形按照长宽尺寸均为3毫米尺寸标准对所述最小外包矩形进行等距网格划分，以得到各预设外包矩形网格；统计各所述预设外包矩形网格内目标第二点云数量，并通过局部离群因子检测算法计算各所述目标第二点云数量对应的离群因子；利用所述离群因子作为点云密度差对所述目标第二点云数据按照垂直方向和水平方向进行划分，以得到平面上的平面竖向筋点云和平面水平筋点云；对所述平面竖向筋点云和所述平面水平筋点云进行逆操作映射至原始双层钢筋笼点云模型，以便获取竖向筋点云和水平筋点云。可以理解的是，统计分析分每个外包矩形网格中的点云数量，发现竖向筋处的点云数量远多于水平筋的点云数量。为将竖向筋和水平筋分开，本发明将所有外包矩形网格点云数量统计值作为输入，竖向筋处点云数量统计值视为离群值，采用局部离群因子检测算法（Local Outlier Factor，LOF）计算每个点的离群因子，离群因子越小，越有可能是异常值，因此，可以通过将离群因子与阈值（通常为-1）进行比较，小于-1处点云为竖向筋点云，大于-1处点云为水平筋点云，分割效果如图12所示。根据对平面钢筋模型中竖向筋点云和水平筋点云的分类结果进行逆操作，将这些点云数据重新映射到三维模型中，从而定位原始三维钢筋模型中的目标竖向筋点云和目标水平筋点云，如图12所示。In this embodiment, the minimum outer bounding rectangle of the target second point cloud data is obtained, and the minimum outer bounding rectangle is divided into equidistant grids according to the size standard of 3 mm in length and width to obtain each preset outer bounding rectangular grid; the number of target second point clouds in each preset outer bounding rectangular grid is counted, and the outlier factor corresponding to each number of target second point clouds is calculated by the local outlier factor detection algorithm; the target second point cloud data is divided in the vertical direction and the horizontal direction using the outlier factor as the point cloud density difference to obtain the plane vertical reinforcement point cloud and the plane horizontal reinforcement point cloud on the plane; the plane vertical reinforcement point cloud and the plane horizontal reinforcement point cloud are reversely mapped to the original double-layer steel cage point cloud model to obtain the vertical reinforcement point cloud and the horizontal reinforcement point cloud. It can be understood that the statistical analysis of the number of point clouds in each outer bounding rectangular grid finds that the number of point clouds at the vertical reinforcement is much larger than the number of point clouds at the horizontal reinforcement. In order to separate the vertical reinforcement and the horizontal reinforcement, the present invention takes the statistical value of the number of all the outer rectangular grid point clouds as input, and regards the statistical value of the point cloud at the vertical reinforcement as an outlier. The local outlier factor detection algorithm (Local Outlier Factor, LOF) is used to calculate the outlier factor of each point. The smaller the outlier factor, the more likely it is an outlier. Therefore, by comparing the outlier factor with the threshold (usually -1), the point cloud less than -1 is the vertical reinforcement point cloud, and the point cloud greater than -1 is the horizontal reinforcement point cloud. The segmentation effect is shown in Figure 12. According to the classification results of the vertical reinforcement point cloud and the horizontal reinforcement point cloud in the plane reinforcement model, the point cloud data are remapped to the three-dimensional model, so as to locate the target vertical reinforcement point cloud and the target horizontal reinforcement point cloud in the original three-dimensional reinforcement model, as shown in Figure 12.
参照图13所示，本发明具体公开了一种双层钢筋笼绑扎方法，包括：确定钢筋的形状和尺寸、相机拍摄的起始位置，通过结构光相机并利用三维路径规划算法自动规划相机拍摄路径，以拍摄覆盖整个钢筋笼的区域；通过机械臂与结构光相机的手眼标定方法确定目标变换矩阵，然后通过坐标系的转换将世界坐标系下的钢筋笼的各个空间坐标，并获取对应的点云数据和相机位姿，然后利用点云数据和相机位姿进一步完成点云模型的重建。然后对该重建的点云模型进行单根钢筋点云簇的聚类识别，具体的，对重建的点云模型进行周围环境点云和地面点云的去除，然后对去除环境点云和地面点云的目标点云进行竖向筋点云和水平筋点云的分割操作，实现单根钢筋点云的聚类活动；点云聚类完成后，对钢筋空间形状进行拟合，具体包括竖向筋圆柱体拟合和水平筋圆柱体拟合，其中水平筋圆柱体拟合需要将其进行分段然后再进行拟合；基于拟合后的钢筋空间形态进一步确定钢筋绑扎点定位信息以及绑扎点的路径规划信息，具体的，首先解算钢筋绑扎点的空间坐标信息，根据该空间坐标信息以及路径规划算法确定绑扎点位的路径规划，输出钢筋绑扎点空间坐标及绑扎顺序，通过机器人执行绑扎操作。Referring to Figure 13, the present invention specifically discloses a double-layer steel cage binding method, including: determining the shape and size of the steel bars, the starting position of the camera shooting, and automatically planning the camera shooting path through a structured light camera and a three-dimensional path planning algorithm to shoot an area covering the entire steel cage; determining the target transformation matrix through a hand-eye calibration method of a robotic arm and a structured light camera, and then converting the various spatial coordinates of the steel cage in the world coordinate system through the conversion of the coordinate system, and obtaining the corresponding point cloud data and camera pose, and then using the point cloud data and camera pose to further complete the reconstruction of the point cloud model. Then, the reconstructed point cloud model is clustered and identified as a single steel bar point cloud cluster. Specifically, the surrounding environment point cloud and ground point cloud are removed from the reconstructed point cloud model, and then the vertical bar point cloud and horizontal bar point cloud are segmented for the target point cloud after the environment point cloud and ground point cloud are removed, so as to realize the clustering activity of the single steel bar point cloud. After the point cloud clustering is completed, the spatial shape of the steel bars is fitted, including vertical bar cylinder fitting and horizontal bar cylinder fitting, wherein the horizontal bar cylinder fitting needs to be segmented and then fitted. Based on the fitted spatial shape of the steel bars, the positioning information of the steel bar binding points and the path planning information of the binding points are further determined. Specifically, the spatial coordinate information of the steel bar binding points is first solved, and the path planning of the binding points is determined according to the spatial coordinate information and the path planning algorithm, and the spatial coordinates and binding sequence of the steel bar binding points are output, and the binding operation is performed by the robot.
由此可见，根据双层钢筋笼点云模型呈纵横交错的特点，通过将钢筋模型投影至平面，并利用竖向筋和水平筋的点云数量差异，将钢筋点云模型分为竖向筋和水平筋两个类别，接着再利用DBSCAN聚类算法成功识别出每根钢筋点云簇。通过PCA及BFGS算法拟合得到所有钢筋的空间圆柱体方程，对钢筋笼空间结构关系的精细化数学建模，使得在复杂空间结构下的双层钢筋绑扎任务变得更为可行，为自动化施工提供了可靠的解决方案。It can be seen that according to the criss-crossing characteristics of the double-layer steel cage point cloud model, the steel bar point cloud model is projected onto a plane and the difference in the number of vertical and horizontal bars is used to divide the steel bar point cloud model into two categories: vertical bars and horizontal bars. Then, the DBSCAN clustering algorithm is used to successfully identify each steel bar point cloud cluster. The spatial cylinder equation of all steel bars is fitted by the PCA and BFGS algorithms, and the refined mathematical modeling of the spatial structural relationship of the steel cage makes the double-layer steel bar binding task under complex spatial structures more feasible, providing a reliable solution for automated construction.
参照图14所示，本发明还相应公开了一种基于三维重建的钢筋笼智能绑扎装置，包括：As shown in FIG. 14 , the present invention also discloses a steel cage intelligent binding device based on three-dimensional reconstruction, including:
模型构建模块11，用于根据预设手眼标定方式确定世界坐标系到机械臂基座坐标系下的目标变换矩阵；通过结构光相机与机械臂获取不同相机拍摄位姿对应的目标钢筋笼的点云数据及相机位姿信息；基于所述目标变换矩阵、所述相机位姿信息和所述点云数据重建双层钢筋笼点云模型；The model building module 11 is used to determine the target transformation matrix from the world coordinate system to the robot base coordinate system according to a preset hand-eye calibration method; obtain the point cloud data and camera pose information of the target steel cage corresponding to different camera shooting poses through the structured light camera and the robot arm; reconstruct the double-layer steel cage point cloud model based on the target transformation matrix, the camera pose information and the point cloud data;
点云分割模块12，用于根据拟合地面得到的平面方程将所述双层钢筋笼点云模型的点云数据垂直于地面方向进行投影，并基于投影后的点云密度差将投影后的所述点云数据划分为竖向筋点云和水平筋点云；A point cloud segmentation module 12 is used to project the point cloud data of the double-layer steel cage point cloud model perpendicular to the ground direction according to the plane equation obtained by fitting the ground, and divide the projected point cloud data into vertical reinforcement point cloud and horizontal reinforcement point cloud based on the point cloud density difference after projection;
点云聚类模块13，用于利用预设聚类算法分别对所述竖向筋点云和所述水平筋点云进行单根钢筋的识别和分离，以获取单根竖向筋和单根水平筋各自对应的单根钢筋点云数据；The point cloud clustering module 13 is used to identify and separate the single steel bar on the vertical bar point cloud and the horizontal bar point cloud respectively by using a preset clustering algorithm, so as to obtain the single steel bar point cloud data corresponding to the single vertical bar and the single horizontal bar respectively;
坐标结算模块14，用于对所述单根钢筋点云数据进行圆柱体拟合，以得到各个所述单根钢筋的拟合后的圆柱体表达方程，基于所述圆柱体表达方程确定出钢筋绑扎点坐标信息；A coordinate settlement module 14 is used to perform cylinder fitting on the single steel bar point cloud data to obtain a fitted cylinder expression equation of each single steel bar, and determine the coordinate information of the steel bar binding point based on the cylinder expression equation;
路径规划模块15，用于通过预设路径规划算法并根据所有所述钢筋绑扎点坐标信息进行路径规划，以得到包含绑扎顺序和绑扎点坐标信息的钢筋笼绑扎方案。The path planning module 15 is used to perform path planning based on the preset path planning algorithm and the coordinate information of all the steel bar binding points to obtain a steel cage binding scheme including the binding sequence and the binding point coordinate information.
可见，本申请公开了根据预设手眼标定方式确定世界坐标系到机械臂基座坐标系下的目标变换矩阵；通过结构光相机与机械臂获取不同相机拍摄位姿对应的目标钢筋笼的点云数据及相机位姿信息；基于所述目标变换矩阵、所述相机位姿信息和所述点云数据重建双层钢筋笼点云模型；根据拟合地面得到的平面方程将所述双层钢筋笼点云模型的点云数据垂直于地面方向进行投影，并基于投影后的点云密度差将投影后的所述点云数据划分为竖向筋点云和水平筋点云；利用预设聚类算法分别对所述竖向筋点云和所述水平筋点云进行单根钢筋的识别和分离，以获取单根竖向筋和单根水平筋各自对应的单根钢筋点云数据；对所述单根钢筋点云数据进行圆柱体拟合，以得到各个所述单根钢筋的拟合后的圆柱体表达方程，基于所述圆柱体表达方程确定出钢筋绑扎点坐标信息，通过预设路径规划算法并根据所有所述钢筋绑扎点坐标信息进行路径规划，以得到包含绑扎顺序和绑扎点坐标信息的钢筋笼绑扎方案。由此可见，通过利用结构光相机获取钢筋信息，结构光相机在捕捉物体的形状和结构方面表现优越，具有更高的深度测量精确度和分辨率，因此，捕获的目标钢筋笼信息能够建立精确钢筋三维模型，根据获取的点云数据能够直接表示钢筋在三维空间中的位置和形状，提供更直观、更全面的钢筋空间结构形式，有利于获取更加准确的钢筋交叉点坐标信息，然后将钢筋模型投影至平面，并利用竖向筋和水平筋的点云数量差异，将钢筋点云模型分为竖向筋和水平筋两个类别，之后通过聚类以及竖向筋点云和水平筋点云不同的数据点密度分布进行点云合理分割，实现竖向和水平向各自的聚类，克服了双层钢筋笼点云模型由于其纵横交错的特点无法直接聚类识别单根钢筋点云簇的问题、最后能够拟合出钢筋空间形状，进一步计算并提取该钢筋空间形状中所有绑扎点的空间坐标，以便基于空间坐标进行路径规划，引导机器人进行钢筋绑扎操作。It can be seen that the present application discloses determining a target transformation matrix from a world coordinate system to a robot base coordinate system according to a preset hand-eye calibration method; obtaining point cloud data and camera pose information of a target steel cage corresponding to different camera shooting poses through a structured light camera and a robot arm; reconstructing a double-layer steel cage point cloud model based on the target transformation matrix, the camera pose information and the point cloud data; projecting the point cloud data of the double-layer steel cage point cloud model perpendicular to the ground direction according to the plane equation obtained by fitting the ground, and dividing the projected point cloud data into vertical reinforcement point cloud and vertical reinforcement point cloud based on the point cloud density difference after projection. horizontal reinforcement point cloud; using a preset clustering algorithm to respectively identify and separate the vertical reinforcement point cloud and the horizontal reinforcement point cloud as single steel bars, so as to obtain the single steel bar point cloud data corresponding to the single vertical reinforcement and the single horizontal reinforcement respectively; performing cylinder fitting on the single steel bar point cloud data, so as to obtain the fitted cylindrical expression equation of each of the single steel bars, and determining the coordinate information of the steel bar binding points based on the cylindrical expression equation, and performing path planning through a preset path planning algorithm and according to the coordinate information of all the steel bar binding points, so as to obtain a steel cage binding scheme including the binding sequence and the binding point coordinate information. It can be seen that by using the structured light camera to obtain the steel bar information, the structured light camera performs well in capturing the shape and structure of the object, and has higher depth measurement accuracy and resolution. Therefore, the captured target steel cage information can establish an accurate steel bar three-dimensional model. According to the acquired point cloud data, the position and shape of the steel bar in the three-dimensional space can be directly represented, providing a more intuitive and comprehensive steel bar spatial structure form, which is conducive to obtaining more accurate steel bar intersection coordinate information. Then the steel bar model is projected onto the plane, and the steel bar point cloud model is divided into two categories of vertical bars and horizontal bars by using the difference in the number of point clouds of vertical bars and horizontal bars. After that, the point cloud is reasonably segmented through clustering and different data point density distributions of vertical bar point clouds and horizontal bar point clouds, and vertical and horizontal clustering is achieved. The problem that the double-layer steel cage point cloud model cannot directly cluster and identify a single steel bar point cloud cluster due to its criss-crossing characteristics is overcome. Finally, the steel bar spatial shape can be fitted, and the spatial coordinates of all binding points in the steel bar spatial shape can be further calculated and extracted, so as to perform path planning based on the spatial coordinates and guide the robot to perform steel bar binding operations.
进一步的，本申请实施例还公开了一种电子设备，图15是根据一示例性实施例示出的电子设备20结构图，图中的内容不能认为是对本申请的使用范围的任何限制。Furthermore, an embodiment of the present application also discloses an electronic device. FIG. 15 is a structural diagram of an electronic device 20 according to an exemplary embodiment. The content in the diagram cannot be regarded as any limitation on the scope of use of the present application.
图15为本申请实施例提供的一种电子设备20的结构示意图。该电子设备20，具体可以包括：至少一个处理器21、至少一个存储器22、电源23、通信接口24、输入输出接口25和通信总线26。其中，所述存储器22用于存储计算机程序，所述计算机程序由所述处理器21加载并执行，以实现前述任一实施例公开的基于三维重建的钢筋笼智能绑扎方法中的相关步骤。另外，本实施例中的电子设备20具体可以为电子计算机。FIG15 is a schematic diagram of the structure of an electronic device 20 provided in an embodiment of the present application. The electronic device 20 may specifically include: at least one processor 21, at least one memory 22, a power supply 23, a communication interface 24, an input/output interface 25, and a communication bus 26. The memory 22 is used to store a computer program, which is loaded and executed by the processor 21 to implement the relevant steps in the intelligent binding method of steel cage based on three-dimensional reconstruction disclosed in any of the aforementioned embodiments. In addition, the electronic device 20 in this embodiment may specifically be an electronic computer.
本实施例中，电源23用于为电子设备20上的各硬件设备提供工作电压；通信接口24能够为电子设备20创建与外界设备之间的数据传输通道，其所遵循的通信协议是能够适用于本申请技术方案的任意通信协议，在此不对其进行具体限定；输入输出接口25，用于获取外界输入数据或向外界输出数据，其具体的接口类型可以根据具体应用需要进行选取，在此不进行具体限定。In this embodiment, the power supply 23 is used to provide working voltage for each hardware device on the electronic device 20; the communication interface 24 can create a data transmission channel between the electronic device 20 and the external device, and the communication protocol it follows is any communication protocol that can be applied to the technical solution of the present application, and is not specifically limited here; the input and output interface 25 is used to obtain external input data or output data to the outside world, and its specific interface type can be selected according to specific application needs and is not specifically limited here.
其中，处理器21可以包括一个或多个处理核心，比如4核心处理器、8核心处理器等。处理器21可以采用DSP(Digital Signal Processing，数字信号处理)、FPGA(Field－Programmable Gate Array，现场可编程门阵列)、PLA(Programmable Logic Array，可编程逻辑阵列)中的至少一种硬件形式来实现。处理器21也可以包括主处理器和协处理器，主处理器是用于对在唤醒状态下的数据进行处理的处理器，也称CPU(Central ProcessingUnit，中央处理器)；协处理器是用于对在待机状态下的数据进行处理的低功耗处理器。在一些实施例中，处理器21可以在集成有GPU(Graphics Processing Unit，图像处理器)，GPU用于负责显示屏所需要显示的内容的渲染和绘制。一些实施例中，处理器21还可以包括AI(Artificial Intelligence，人工智能)处理器，该AI处理器用于处理有关机器学习的计算操作。Among them, the processor 21 may include one or more processing cores, such as a 4-core processor, an 8-core processor, etc. The processor 21 can be implemented in at least one hardware form of DSP (Digital Signal Processing), FPGA (Field-Programmable Gate Array), and PLA (Programmable Logic Array). The processor 21 may also include a main processor and a coprocessor. The main processor is a processor for processing data in the awake state, also known as a CPU (Central Processing Unit); the coprocessor is a low-power processor for processing data in the standby state. In some embodiments, the processor 21 may be integrated with a GPU (Graphics Processing Unit), which is responsible for rendering and drawing the content to be displayed on the display screen. In some embodiments, the processor 21 may also include an AI (Artificial Intelligence) processor, which is used to process computing operations related to machine learning.
另外，存储器22作为资源存储的载体，可以是只读存储器、随机存储器、磁盘或者光盘等，其上所存储的资源可以包括操作系统221、计算机程序222等，存储方式可以是短暂存储或者永久存储。In addition, the memory 22, as a carrier for storing resources, can be a read-only memory, a random access memory, a disk or an optical disk, etc. The resources stored thereon can include an operating system 221, a computer program 222, etc., and the storage method can be temporary storage or permanent storage.
其中，操作系统221用于管理与控制电子设备20上的各硬件设备以及计算机程序222，以实现处理器21对存储器22中海量数据223的运算与处理，其可以是Windows Server、Netware、Unix、Linux等。计算机程序222除了包括能够用于完成前述任一实施例公开的由电子设备20执行的基于三维重建的钢筋笼智能绑扎方法的计算机程序之外，还可以进一步包括能够用于完成其他特定工作的计算机程序。数据223除了可以包括电子设备接收到的由外部设备传输进来的数据，也可以包括由自身输入输出接口25采集到的数据等。Among them, the operating system 221 is used to manage and control the hardware devices and computer programs 222 on the electronic device 20, so as to realize the operation and processing of the massive data 223 in the memory 22 by the processor 21, which can be Windows Server, Netware, Unix, Linux, etc. In addition to including a computer program that can be used to complete the three-dimensional reconstruction-based intelligent binding method of steel cages performed by the electronic device 20 disclosed in any of the aforementioned embodiments, the computer program 222 can also further include a computer program that can be used to complete other specific tasks. In addition to data transmitted from an external device received by the electronic device, the data 223 can also include data collected by its own input and output interface 25.
进一步的，本申请还公开了一种计算机可读存储介质，用于存储计算机程序；其中，所述计算机程序被处理器执行时实现前述公开的基于三维重建的钢筋笼智能绑扎方法。关于该方法的具体步骤可以参考前述实施例中公开的相应内容，在此不再进行赘述。Furthermore, the present application also discloses a computer-readable storage medium for storing a computer program; wherein, when the computer program is executed by a processor, the above-disclosed intelligent binding method for steel cages based on three-dimensional reconstruction is implemented. The specific steps of the method can be referred to the corresponding contents disclosed in the above-mentioned embodiments, and will not be repeated here.
本说明书中各个实施例采用递进的方式描述，每个实施例重点说明的都是与其它实施例的不同之处，各个实施例之间相同或相似部分互相参见即可。对于实施例公开的装置而言，由于其与实施例公开的方法相对应，所以描述的比较简单，相关之处参见方法部分说明即可。In this specification, each embodiment is described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the embodiments can be referred to each other. For the device disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant parts can be referred to the method part.
专业人员还可以进一步意识到，结合本文中所公开的实施例描述的各示例的单元及算法步骤，能够以电子硬件、计算机软件或者二者的结合来实现，为了清楚地说明硬件和软件的可互换性，在上述说明中已经按照功能一般性地描述了各示例的组成及步骤。这些功能究竟以硬件还是软件方式来执行，取决于技术方案的特定应用和设计约束条件。专业技术人员可以对每个特定的应用来使用不同方法来实现所描述的功能，但是这种实现不应认为超出本申请的范围。结合本文中所公开的实施例描述的方法或算法的步骤可以直接用硬件、处理器执行的软件模块，或者二者的结合来实施。软件模块可以置于随机存储器RAM（Random Access Memory）、内存、只读存储器ROM（Read Only Memory）、电可编程EPROM（Electrically Programmable Read Only Memory）、电可擦除可编程EEPROM（ElectricErasable Programmable Read Only Memory）、寄存器、硬盘、可移动磁盘、CD-ROM（CompactDisc-Read Only Memory，紧凑型光盘只读储存器）、或技术领域内所公知的任意其它形式的存储介质中。Professionals may further appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented with electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in terms of function in the above description. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel may use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application. The steps of the method or algorithm described in conjunction with the embodiments disclosed herein can be implemented directly with hardware, a software module executed by a processor, or a combination of the two. The software module can be placed in a random access memory RAM (Random Access Memory), memory, read-only memory ROM (Read Only Memory), electrically programmable EPROM (Electrically Programmable Read Only Memory), electrically erasable programmable EEPROM (ElectricErasable Programmable Read Only Memory), register, hard disk, removable disk, CD-ROM (CompactDisc-Read Only Memory), or any other form of storage medium known in the technical field.
最后，还需要说明的是，在本文中，诸如第一和第二等之类的关系术语仅仅用来将一个实体或者操作与另一个实体或操作区分开来，而不一定要求或者暗示这些实体或操作之间存在任何这种实际的关系或者顺序。而且，术语“包括”、“包含”或者其任何其他变体意在涵盖非排他性的包含，从而使得包括一系列要素的过程、方法、物品或者设备不仅包括那些要素，而且还包括没有明确列出的其他要素，或者是还包括为这种过程、方法、物品或者设备所固有的要素。在没有更多限制的情况下，由语句“包括一个……”限定的要素，并不排除在包括所述要素的过程、方法、物品或者设备中还存在另外的相同要素。Finally, it should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the presence of other identical elements in the process, method, article or device including the elements.
以上对本发明所提供的基于三维重建的钢筋笼智能绑扎方法、装置、设备及介质进行了详细介绍，本文中应用了具体个例对本发明的原理及实施方式进行了阐述，以上实施例的说明只是用于帮助理解本发明的方法及其核心思想；同时，对于本领域的一般技术人员，依据本发明的思想，在具体实施方式及应用范围上均会有改变之处，综上所述，本说明书内容不应理解为对本发明的限制。The above is a detailed introduction to the intelligent binding method, device, equipment and medium for steel cages based on three-dimensional reconstruction provided by the present invention. Specific examples are used in this article to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only used to help understand the method of the present invention and its core idea; at the same time, for general technical personnel in this field, according to the idea of the present invention, there will be changes in the specific implementation method and application scope. In summary, the content of this specification should not be understood as a limitation on the present invention.
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202410910938.0ACN118456446B (en) | 2024-07-09 | 2024-07-09 | Intelligent reinforcement cage binding method, device, equipment and medium based on three-dimensional reconstruction |
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202410910938.0ACN118456446B (en) | 2024-07-09 | 2024-07-09 | Intelligent reinforcement cage binding method, device, equipment and medium based on three-dimensional reconstruction |
| Publication Number | Publication Date |
|---|---|
| CN118456446A CN118456446A (en) | 2024-08-09 |
| CN118456446Btrue CN118456446B (en) | 2024-09-06 |
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202410910938.0AActiveCN118456446B (en) | 2024-07-09 | 2024-07-09 | Intelligent reinforcement cage binding method, device, equipment and medium based on three-dimensional reconstruction |
| Country | Link |
|---|---|
| CN (1) | CN118456446B (en) |
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN118990520B (en)* | 2024-10-24 | 2025-02-11 | 湖南大学 | Vision-guided welding method, device, medium and robot arm for steel frame |
| CN119180865A (en)* | 2024-11-26 | 2024-12-24 | 湖南大学 | Reinforcing steel bar binding method, reinforcing steel bar binding system, storage medium and electronic equipment |
| CN119221712A (en)* | 2024-11-28 | 2024-12-31 | 杭州宇泛智能科技股份有限公司 | Batch fully automatic lashing control system and method |
| CN119188789B (en)* | 2024-11-28 | 2025-03-28 | 湖南大学 | Reinforcement framework size measurement method and device based on reinforcement binding robot |
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115464652A (en)* | 2022-09-22 | 2022-12-13 | 北京理工华汇智能科技有限公司 | Machine vision-based steel bar binding method and system |
| CN115519547A (en)* | 2022-10-25 | 2022-12-27 | 江苏图知天下科技有限公司 | Hand-eye calibration method and system for steel bar binding |
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US10864638B2 (en)* | 2018-09-27 | 2020-12-15 | Logistics and Supply Chain MultiTech R&D Centre Limited | Reinforcement bar joint recognition using artificial intelligence vision |
| US20220325544A1 (en)* | 2019-11-09 | 2022-10-13 | SkyMul Inc. | Robot for tying rebar on a rebar grid |
| CN115854883B (en)* | 2022-11-09 | 2023-11-03 | 浙江大学 | Automatic detection method and device suitable for long reinforcement cage |
| CN115972174B (en)* | 2022-11-21 | 2024-09-03 | 中国建筑第八工程局有限公司 | Construction robot and visual identification positioning method for construction nodes thereof |
| CN115837673A (en)* | 2022-12-02 | 2023-03-24 | 河北工业大学 | Automatic binding method for reinforcing mesh based on point cloud environment sensing |
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115464652A (en)* | 2022-09-22 | 2022-12-13 | 北京理工华汇智能科技有限公司 | Machine vision-based steel bar binding method and system |
| CN115519547A (en)* | 2022-10-25 | 2022-12-27 | 江苏图知天下科技有限公司 | Hand-eye calibration method and system for steel bar binding |
| Publication number | Publication date |
|---|---|
| CN118456446A (en) | 2024-08-09 |
| Publication | Publication Date | Title |
|---|---|---|
| CN118456446B (en) | Intelligent reinforcement cage binding method, device, equipment and medium based on three-dimensional reconstruction | |
| CN111951397B (en) | A method, device and storage medium for multi-machine cooperative construction of three-dimensional point cloud map | |
| Lee et al. | Skeleton-based 3D reconstruction of as-built pipelines from laser-scan data | |
| CN112184789B (en) | Plant model generation method, device, computer equipment and storage medium | |
| CN112836734A (en) | Heterogeneous data fusion method and device, and storage medium | |
| CN110544298B (en) | Substation modeling method, device, computer equipment and storage medium | |
| KR20170068462A (en) | 3-Dimensional Model Generation Using Edges | |
| CN113400301B (en) | Robot 3D hand-eye calibration method, system, device and medium | |
| CN114758337A (en) | A semantic instance reconstruction method, apparatus, device and medium | |
| CN117193278B (en) | Method, device, computer equipment and storage medium for dynamic edge path generation | |
| CN118781178B (en) | A volume measurement method based on surface reconstruction and triple integral | |
| CN114494648B (en) | Grid adjusting method, equipment and storage medium based on finite element meshing | |
| CN113763529B (en) | Substation modeling method based on three-dimensional scanning | |
| CN114283266B (en) | A three-dimensional model adjustment method, device, storage medium and equipment | |
| CN115797568A (en) | Modeling method and device based on three-dimensional GIS and BIM integration | |
| CN118396875A (en) | A point cloud denoising method, device and application based on Mesh | |
| CN118887263A (en) | Automatic combined point cloud registration system and method | |
| CN113971718B (en) | Method for performing Boolean operation on three-dimensional point cloud model | |
| CN118470209A (en) | Building relic restoration method and system based on three-dimensional modeling technology | |
| CN117739954B (en) | Map partial updating method, device and electronic equipment | |
| CN118628653A (en) | A 3D modeling method for power transmission lines based on airborne Lidar | |
| CN117974887A (en) | Tunnel wall modeling method and system based on three-dimensional laser point cloud | |
| CN117893603A (en) | Visual positioning method and device based on 3D visual data and computer equipment | |
| CN116310753A (en) | A vectorized skeleton extraction method and system for outdoor scene point cloud data | |
| CN115346019A (en) | Method, device and system for measuring geometric parameters of point cloud circular holes |
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| TR01 | Transfer of patent right | ||
| TR01 | Transfer of patent right | Effective date of registration:20250709 Address after:410082 Hunan Province, Changsha City, Yuelu District, Tianjing Street, Lianhu Third Road No. 30, China Construction Meixi Jiayuan Building No. 1, Room 1609 Patentee after:Changsha Jijiang Technology Co.,Ltd. Country or region after:China Address before:Yuelu District City, Hunan province 410082 Changsha Lushan Road No. 1 Patentee before:HUNAN University Country or region before:China |