CN112627261B

Movatterモバイル変換

Info

Publication number: CN112627261B
Application number: CN202011302023.XA
Authority: CN
Inventors: 梁帮修; 侯志强; 刘浩; 卜令臣; 张乐天
Original assignee: Jiangsu Xugong Construction Machinery Research Institute Co ltd; Xuzhou XCMG Road Construction Machinery Co Ltd
Current assignee: Jiangsu Xugong Construction Machinery Research Institute Co ltd; Xuzhou XCMG Road Construction Machinery Co Ltd
Priority date: 2020-11-19
Filing date: 2020-11-19
Publication date: 2022-06-14
Anticipated expiration: 2040-11-19
Also published as: CN112627261A

Abstract

The invention discloses a cutting blade attitude control system and method based on machine vision and a land leveler, which belong to the technical field of land levelers. The method comprises the following steps: the system comprises a plurality of cameras, a control unit and a control unit, wherein the cameras are arranged at set positions of the grader and are used for acquiring real-time position information of a plurality of visual perception tracking points arranged at the set positions of the grader; the vision processing unit is used for acquiring the current posture of the scraper knife according to the received real-time position information of each vision perception tracking point, and generating a target state of the scraper knife posture control mechanism by combining the target posture of the scraper knife sent by the human-computer display, and the vehicle-mounted controller is used for controlling the scraper knife posture control mechanism through the electric proportional control valve according to the target state and the current state of the scraper knife posture control mechanism, so that the scraper knife posture control mechanism drives the scraper knife to reach the target posture.

Description

Translated fromChinese

一种基于机器视觉的铲刀姿态控制系统、方法及平地机A machine vision-based blade attitude control system, method and motor grader

技术领域technical field

本发明属于平地机技术领域，具体涉及一种基于机器视觉的铲刀姿态控制系统、方法及平地机。The invention belongs to the technical field of motor graders, and in particular relates to a machine vision-based blade attitude control system, method and motor grader.

背景技术Background technique

平地机是一种以铲刀为主体，配以其他多种可换作业装置，进行土壤铲掘、平整和整形作业的铲土运输施工机械，主要应用于道路、机场、农田、水利等大面积土壤平整作业及刮坡、挖沟、推土、松土、清除路面冰雪等施工作业，是国防、交通、农业、水利基本建设施工中的重要设备之一，在国民经济建设中发挥着巨大的作用。与其他工程机械相比，平地机核心工作装置铲刀的运动灵活且运动范围较大，铲刀姿态的精确检测与控制难度较大，且由于驾驶员操作失误经常导致铲刀与平地机其他结构（如前后轮胎、登车梯等）发生碰撞，操控难度大、劳动强度高，施工作业质量难以保证。The motor grader is a kind of shovel transport construction machinery with a blade as the main body, with a variety of other interchangeable operating devices for soil digging, leveling and shaping operations, mainly used in roads, airports, farmland, water conservancy and other large areas Soil leveling operations and construction operations such as scraping slopes, digging trenches, bulldozing, loosening soil, and clearing road ice and snow are one of the important equipment in the construction of national defense, transportation, agriculture, and water conservancy infrastructure, and play a huge role in national economic construction. effect. Compared with other construction machinery, the blade of the core working device of the grader has a flexible movement and a large range of motion, and the precise detection and control of the blade attitude is difficult, and the blade and other structures of the grader are often caused by driver errors. (such as front and rear tires, boarding ladders, etc.) collide, the control is difficult, the labor intensity is high, and the construction quality is difficult to guarantee.

发明内容SUMMARY OF THE INVENTION

为解决现有技术中的不足，本发明提供一种基于机器视觉的铲刀姿态控制系统、方法及平地机，能够根据驾驶员的操作指令自动调整铲刀至设定的姿态，降低了驾驶员的操控难度和劳动强度。In order to solve the deficiencies in the prior art, the present invention provides a blade attitude control system, method and grader based on machine vision, which can automatically adjust the blade to the set attitude according to the operation instruction of the driver, thereby reducing the driver's risk. control difficulty and labor intensity.

为达到上述目的，本发明所采用的技术方案是：一种基于机器视觉的铲刀姿态控制系统，包括：若干个安装于平地机的设定位置的摄像头，用于采集安装于平地机的设定位置的多个视觉感知追踪点的实时位置信息并发送至视觉处理单元；所述视觉处理单元，用于根据接收到的各个所述视觉感知追踪点的实时位置信息获取铲刀的当前姿态，并结合人机显示器发送的铲刀的目标姿态，生成铲刀姿态控制机构的目标状态，并把铲刀姿态控制机构的目标状态及当前状态发送至车载控制器；所述车载控制器，用于根据接收到的铲刀姿态控制机构的目标状态及当前状态，并通过电比例控制阀控制铲刀姿态控制机构，使铲刀姿态控制机构驱动铲刀至目标姿态。In order to achieve the above purpose, the technical scheme adopted in the present invention is: a machine vision-based blade attitude control system, comprising: a plurality of cameras installed at the set position of the grader for collecting the settings installed on the grader. The real-time position information of a plurality of visual perception tracking points at fixed positions is sent to the visual processing unit; the visual processing unit is used to obtain the current posture of the blade according to the received real-time position information of each of the visual perception tracking points, And combined with the target attitude of the blade sent by the man-machine display, the target state of the blade attitude control mechanism is generated, and the target state and the current status of the blade attitude control mechanism are sent to the vehicle-mounted controller; the vehicle-mounted controller is used for According to the received target state and current state of the blade attitude control mechanism, the blade attitude control mechanism is controlled by the electric proportional control valve, so that the blade attitude control mechanism drives the blade to the target attitude.

进一步地，所述摄像头安装于整车的固定位置，包括驾驶室和前机架；所述视觉感知追踪点按照设定的尺寸间隔安装于铲刀、回转支撑和铲刀姿态控制机构上。Further, the camera is installed at a fixed position of the whole vehicle, including the cab and the front frame; the visual perception tracking points are installed on the blade, the slewing support and the blade attitude control mechanism according to a set size interval.

进一步地，所述摄像头为红外摄像头。Further, the camera is an infrared camera.

进一步地，所述铲刀姿态控制机构包括左升降油缸、右升降油缸、斜拉油缸、引出油缸、铲土角油缸和液压马达；所述左升降油缸和右升降油缸的缸体分别与左摆架和右摆架铰接，所述左升降油缸和右升降油缸的伸缩杆分别与回转支撑铰接；所述斜拉油缸的缸体与所述回转支撑铰接，所述斜拉油缸的伸缩杆与所述左摆架铰接；所述引出油缸一端与角位器铰接，另一端与铲刀固定连接；所述角位器的一端与铲土角油缸的活塞杆铰接，所述铲土角油缸的缸体与所述回转支撑铰接；所述回转支撑的内齿圈内嵌于牵引架的凹槽中，所述液压马达驱动固定在牵引架上的涡轮箱进而通过内齿-外齿传动驱动铲刀做回转运动。Further, the blade attitude control mechanism includes a left lifting oil cylinder, a right lifting oil cylinder, a cable-pulling oil cylinder, an outgoing oil cylinder, a shovel angle oil cylinder and a hydraulic motor; The frame and the right swing frame are hinged, and the telescopic rods of the left and right lift cylinders are hinged with the slewing support respectively; the cylinder body of the cable-stayed cylinder is hinged with the slewing support, and the telescopic rod of the cable-stayed cylinder is hinged with the slewing support. The left swing frame is hinged; one end of the lead-out oil cylinder is hinged with the angle positioner, and the other end is fixedly connected with the shovel blade; one end of the angle positioner is hinged with the piston rod of the shovel angle oil cylinder, and the The body is hinged with the slewing support; the inner gear ring of the slewing support is embedded in the groove of the draw frame, the hydraulic motor drives the turbine box fixed on the draw frame, and then drives the blade through the internal tooth-external tooth transmission Do a swivel motion.

进一步地，所述车载控制器配置有铲刀碰撞姿态子集，当车载控制器接收到铲刀姿态控制机构的动作信息后，自动选择一条避开所述铲刀碰撞姿态子集的铲刀姿态控制机构的控制序列对铲刀姿态进行实时调整。Further, the vehicle-mounted controller is configured with a subset of blade collision attitudes, and when the vehicle-mounted controller receives the action information of the blade attitude control mechanism, it automatically selects a blade attitude that avoids the subset of blade collision attitudes. The control sequence of the control mechanism adjusts the blade attitude in real time.

进一步地，所述人机显示器，用于设置并向所述视觉处理单元发送铲刀的目标姿态、基于所述视觉处理单元获取的铲刀的当前姿态实时显示整车三维图形、切换手动控制状态和自动控制状态。Further, the man-machine display is used to set and send the target attitude of the blade to the vision processing unit, display the three-dimensional graphics of the whole vehicle in real time based on the current attitude of the blade acquired by the vision processing unit, and switch the manual control state. and automatic control state.

进一步地，在手动控制状态下，当驾驶员手动调整铲刀姿态并达到铲刀碰撞姿态子集的边界时，车载控制器停止铲刀运动并发出报警信息。Further, in the manual control state, when the driver manually adjusts the blade attitude and reaches the boundary of the blade collision attitude subset, the vehicle-mounted controller stops the blade movement and issues an alarm message.

一种平地机，所述平地机配置有前述的基于机器视觉的铲刀姿态控制系统。A motor grader equipped with the aforementioned machine vision-based blade attitude control system.

一种基于机器视觉的铲刀姿态控制方法，包括：采集安装于平地机的设定位置的多个视觉感知追踪点的实时位置信息；根据各个视觉感知追踪点的实时位置信息获取铲刀的当前姿态，并结合铲刀的目标姿态，生成铲刀姿态控制机构的目标状态；根据铲刀姿态控制机构的目标状态和当前状态，通过电比例控制阀控制铲刀姿态控制机构，使铲刀姿态控制机构驱动铲刀至目标姿态。A blade attitude control method based on machine vision, comprising: collecting real-time position information of a plurality of visual perception tracking points installed at a set position of a grader; According to the target state and current state of the blade attitude control mechanism, the electric proportional control valve is used to control the blade attitude control mechanism to make the blade attitude control The mechanism drives the blade to the target attitude.

进一步地，所述根据铲刀姿态控制机构的目标状态和当前状态，通过电比例控制阀控制铲刀姿态控制机构，使铲刀姿态控制机构驱动铲刀至目标姿态，包括：c1、根据铲刀姿态控制机构的目标状态和当前状态，生成铲刀姿态控制机构的初始控制序列；c2、将生成的铲刀姿态控制机构的初始控制序列与铲刀碰撞姿态子集进行比较并选择一条避开所述铲刀碰撞姿态子集的铲刀姿态控制机构的控制序列作为执行控制序列；c3、基于步骤c2中选择的执行控制序列，通过电比例控制阀控制铲刀姿态控制机构，使铲刀姿态控制机构驱动铲刀至目标姿态。Further, according to the target state and the current state of the blade attitude control mechanism, the blade attitude control mechanism is controlled by the electric proportional control valve, so that the blade attitude control mechanism drives the blade to the target attitude, including: c1, according to the blade attitude control mechanism. The target state and current state of the attitude control mechanism, generate the initial control sequence of the blade attitude control mechanism; c2, compare the generated initial control sequence of the blade attitude control mechanism with the blade collision attitude subset, and select a method that avoids the collision. Describe the control sequence of the blade attitude control mechanism of the blade collision attitude subset as the execution control sequence; c3, based on the execution control sequence selected in step c2, control the blade attitude control mechanism through the electric proportional control valve, so that the blade attitude control The mechanism drives the blade to the target attitude.

与现有技术相比，本发明所达到的有益效果：Compared with the prior art, the beneficial effects achieved by the present invention:

（1）本发明基于机器视觉技术，通过摄像头采集安装于平地机上的视觉感知追踪点的实时位置信息，进而获取铲刀的当前姿态，并与要求的铲刀目标姿态进行比较，获取铲刀姿态控制机构的目标状态，车载控制器基于铲刀姿态控制机构的目标状态实现对铲刀姿态的自动控制，降低了驾驶员的操控难度和劳动强度；(1) The present invention is based on machine vision technology. The camera collects the real-time position information of the visual perception tracking point installed on the grader, and then obtains the current attitude of the blade, and compares it with the required target attitude of the blade to obtain the blade attitude The target state of the control mechanism, the on-board controller realizes the automatic control of the blade attitude based on the target state of the blade attitude control mechanism, which reduces the driver's control difficulty and labor intensity;

（2）本发明通过在车载控制器内配置铲刀碰撞姿态子集，当车载控制器接收到铲刀姿态控制机构的动作信息后，自动选择一条避开所述铲刀碰撞姿态子集的铲刀姿态控制机构的控制序列对铲刀姿态进行实时调整，解决了铲刀自动和手动调整过程中可能发生的铲刀与平地机其他机构之间的碰撞问题，提高了作业安全性。(2) The present invention configures the blade collision attitude subset in the vehicle-mounted controller. After the vehicle-mounted controller receives the action information of the blade attitude control mechanism, it automatically selects a blade that avoids the blade collision attitude subset. The control sequence of the blade attitude control mechanism adjusts the blade attitude in real time, solves the collision problem between the blade and other mechanisms of the grader that may occur during the automatic and manual adjustment of the blade, and improves the safety of the operation.

附图说明Description of drawings

图1是本发明实施例提供的一种基于机器视觉的铲刀姿态控制系统的铲刀姿态控制机构的立体结构示意图；Fig. 1 is a three-dimensional structural schematic diagram of a blade attitude control mechanism of a machine vision-based blade attitude control system provided by an embodiment of the present invention;

图2是本发明实施例提供的一种基于机器视觉的铲刀姿态控制系统的铲刀姿态控制机构的平面结构示意图；2 is a schematic plan view of a blade attitude control mechanism of a machine vision-based blade attitude control system provided by an embodiment of the present invention;

图3是铲刀入土深度、铲土角度定义示意图；Figure 3 is a schematic diagram of the definition of the depth of the blade into the soil and the angle of the shovel;

图4是铲刀回转角度、引出位移定义示意图；Figure 4 is a schematic diagram of the definition of blade rotation angle and lead-out displacement;

图5是铲刀横坡角度定义示意图；Fig. 5 is a schematic diagram of the definition of the cross slope angle of the blade;

图6是本发明实施例提供的一种基于机器视觉的铲刀姿态控制系统的系统框图；6 is a system block diagram of a machine vision-based blade attitude control system provided by an embodiment of the present invention;

图7是本发明实施例提供的一种基于机器视觉的铲刀姿态控制系统中摄像头和视觉感知追踪点的分布状态示意图一；7 is a schematic diagram 1 of the distribution state of cameras and visual perception tracking points in a machine vision-based blade attitude control system provided by an embodiment of the present invention;

图8是本发明实施例提供的一种基于机器视觉的铲刀姿态控制系统中摄像头和视觉感知追踪点的分布状态示意图二；8 is a schematic diagram 2 of the distribution state of cameras and visual perception tracking points in a machine vision-based blade attitude control system provided by an embodiment of the present invention;

图9是本发明实施例提供的一种基于机器视觉的铲刀姿态控制系统中摄像头和视觉感知追踪点的分布状态示意图三；9 is a schematic diagram 3 of the distribution state of cameras and visual perception tracking points in a machine vision-based blade attitude control system provided by an embodiment of the present invention;

图10是图9中视觉感知追踪点的识别及铲刀姿态换算示意图。FIG. 10 is a schematic diagram of the recognition of the visual perception tracking point and the conversion of the blade attitude in FIG. 9 .

具体实施方式Detailed ways

下面结合附图对本发明作进一步描述。以下实施例仅用于更加清楚地说明本发明的技术方案，而不能以此来限制本发明的保护范围。The present invention will be further described below in conjunction with the accompanying drawings. The following examples are only used to illustrate the technical solutions of the present invention more clearly, and cannot be used to limit the protection scope of the present invention.

实施例一：Example 1:

如图1~图10所示，一种基于机器视觉的铲刀姿态控制系统，包括：若干个安装于平地机的设定位置的摄像头15，用于采集安装于平地机的设定位置的多个视觉感知追踪点16的实时位置信息并发送至视觉处理单元17；视觉处理单元17用于根据接收到的各个所述视觉感知追踪点16的实时位置信息获取铲刀10的当前姿态，并结合人机显示器19发送的铲刀10的目标姿态，生成铲刀姿态控制机构的目标状态，并把铲刀姿态控制机构的目标状态和当前状态发送至车载控制器18；车载控制器18用于根据接收到的铲刀姿态控制机构的目标状态和当前状态，并通过电比例控制阀20控制铲刀姿态控制机构，使铲刀姿态控制机构驱动铲刀10至目标姿态。As shown in FIGS. 1 to 10 , a blade attitude control system based on machine vision includes: a plurality ofcameras 15 installed at the set position of the grader for collecting multiple cameras installed at the set position of the grader. The real-time position information of each visualperception tracking point 16 is sent to the visual processing unit 17; the visual processing unit 17 is used to obtain the current posture of theblade 10 according to the received real-time position information of each of the visualperception tracking points 16, and combine The target posture of theblade 10 sent by the man-machine display 19 generates the target state of the blade posture control mechanism, and sends the target state and the current state of the blade posture control mechanism to the on-board controller 18; The target state and current state of the blade attitude control mechanism are received, and the electric proportional control valve 20 controls the blade attitude control mechanism, so that the blade attitude control mechanism drives theblade 10 to the target attitude.

如图1、2、7、8所示，本实施例中，铲刀姿态控制机构包括左升降油缸4、右升降油缸5、斜拉油缸6、引出油缸12、铲土角油缸11和液压马达21；左升降油缸4和右升降油缸5的缸体分别与左摆架2和右摆架3铰接，左升降油缸4和右升降油缸5的伸缩杆分别与回转支撑8铰接；斜拉油缸6的缸体与回转支撑8铰接，斜拉油缸6的伸缩杆与左摆架2铰接；引出油缸12一端与角位器9铰接，另一端与铲刀10固定连接；角位器9的一端（图1中角位器9的上端）与铲土角油缸11的活塞杆端部铰接，角位器9的另一端（图1中角位器9的下端）铰接于回转支撑8下方销轴O处，铲土角油缸11的缸体与回转支撑8铰接；回转支撑8的内齿圈内嵌于牵引架7的凹槽中，液压马达21驱动固定在牵引架7上的涡轮箱13进而通过内齿-外齿传动驱动铲刀10做回转运动；铲刀10通过其后方的导轨内嵌于角位器9的导槽中，并通过引出油缸12实现铲刀10在角位器9的凹槽中滑动；牵引架7一端通过球铰铰接接于前机架1的前端，牵引架7的另一端与左升降油缸4、右升降油缸5、斜拉油缸6的活塞杆部分通过球铰进行连接；左升降油缸4、右升降油缸5、斜拉油缸6的缸体部分与摆架进行铰接。As shown in Figures 1, 2, 7, and 8, in this embodiment, the blade attitude control mechanism includes aleft lift cylinder 4, aright lift cylinder 5, an inclined-pull cylinder 6, a lead-outcylinder 12, ashovel angle cylinder 11 and ahydraulic motor 21; the cylinder bodies of theleft lift cylinder 4 and theright lift cylinder 5 are hinged with theleft swing frame 2 and theright swing frame 3 respectively, and the telescopic rods of theleft lift cylinder 4 and theright lift cylinder 5 are respectively hinged with theslewing support 8; The cylinder body is hinged with theslewing support 8, the telescopic rod of the inclined-pull cylinder 6 is hinged with theleft swing frame 2; one end of the lead-outcylinder 12 is hinged with theangle positioner 9, and the other end is fixedly connected with theblade 10; one end of the angle positioner 9 ( The upper end of theangle positioner 9 in FIG. 1 is hinged with the end of the piston rod of the shovelangle oil cylinder 11 , and the other end of the angle positioner 9 (the lower end of theangle positioner 9 in FIG. 1 ) is hinged to the pin shaftO below theslewing support 8 The cylinder body of the shovelangle oil cylinder 11 is hinged with theslewing support 8; the ring gear of theslewing support 8 is embedded in the groove of thedraw frame 7, and thehydraulic motor 21 drives theturbine box 13 fixed on thedraw frame 7 to pass through The internal tooth-external tooth transmission drives theblade 10 to rotate; theblade 10 is embedded in the guide groove of theangle positioner 9 through the guide rail behind it, and the concave position of theblade 10 in theangle positioner 9 is realized by drawing out theoil cylinder 12 Sliding in the groove; one end of thetraction frame 7 is hinged to the front end of thefront frame 1 through a ball hinge, and the other end of thetraction frame 7 is connected to the piston rod of the left liftingcylinder 4, theright lifting cylinder 5, and the inclined-pull cylinder 6 through the ball hinge. Connection; the cylinder parts of theleft lift cylinder 4, theright lift cylinder 5, and the inclined-pull cylinder 6 are hinged with the swing frame.

对铲刀10的姿态的定义，如图3、4、5所示，平地机铲刀10运动灵活，其作业姿态有入土深度H、铲土角度θ、回转角度ψ、横坡角度Φ、引出位移L等5个。铲刀10的入土深度H是指铲刀10侵入土壤内部的深度，其数值大小为铲刀刀尖与土壤表面的距离（见图3）；铲刀10的铲土角度θ是指铲刀10最下端水平切入土壤的角度，其数值大小为铲刀10最下端圆弧切线与水平面间的夹角（见图3）；铲刀回转角度ψ是指铲刀10随回转支撑8的旋转角度，其数值大小为铲刀10的长度方向与平地机行驶方向间的水平夹角（见图4）；铲刀10的横坡角度Φ是指铲刀10下铲刃在平地机行驶方向垂直面上的投影与水平面间的角度，其数值大小为平地机铲刀10所形成的土壤表面的横坡角度（见图5）；铲刀10的引出位移L是指铲刀10相对于回转支撑8的侧向移出距离，其数值大小为铲刀10在长度方向上的对称面与过回转支撑8圆心且与铲刀的长度方向垂直的对称面间的距离（见图4，图中，m表示铲刀长度方向对称面，n表示过回转支撑圆心且与铲刀的长度方向垂直的对称面）。For the definition of the attitude of theblade 10, as shown in Figures 3, 4, and 5, theblade 10 of the grader moves flexibly, and its working attitude includes the depth of entryH , the angle of shovelingθ , the angle of rotationψ , the angle of the transverse slopeΦ , the lead-out DisplacementL and other 5. The penetration depthH of theblade 10 refers to the depth of theblade 10 into the soil, and its value is the distance between the blade tip and the soil surface (see Figure 3); the shovel angleθ of theblade 10 refers to theblade 10 The angle at which the lowermost end cuts into the soil horizontally, and its value is the angle between the tangent line of the lowermost arc of theblade 10 and the horizontal plane (see Figure 3); the blade rotation angleψ refers to the rotation angle of theblade 10 with theslewing support 8, The numerical value is the horizontal angle between the length direction of theblade 10 and the driving direction of the grader (see Figure 4); the transverse slope angleΦ of theblade 10 refers to the lower cutting edge of theblade 10 on the vertical plane of the driving direction of the grader. The angle between the projection of , and the horizontal plane, the value of which is thetransverse slope angle of the soil surface formed by theblade 10 of the grader (see FIG. 5 ); The lateral movement distance is the distance between the symmetrical plane of theblade 10 in the length direction and the symmetrical plane passing through the center of theslewing support 8 and perpendicular to the length of the blade (see Figure 4, in the figure, m represents the shovel Symmetric plane in the length direction of the blade, n represents the plane of symmetry passing through the center of the slewing support circle and perpendicular to the length direction of the blade).

如图6~图10所示，各个摄像头15分别将采集到的各个视觉感知追踪点16的实时位置信息发送至视觉处理单元17；视觉处理单元17为具有很强计算功能的GPU，它需要进行视觉识别的计算，视觉处理单元17根据接收到的各个视觉感知追踪点16的实时位置信息获取铲刀10的当前姿态，并结合人机显示器19发送的铲刀10的目标姿态，生成铲刀姿态控制机构的目标状态，并把铲刀姿态控制机构的目标状态和当前状态等控制决策机制通过CAN总线发送至车载控制器18；车载控制器18根据接收到的铲刀姿态控制机构的目标状态和当前状态，通过电比例控制阀20控制铲刀姿态控制机构，使铲刀姿态控制机构驱动铲刀10至目标姿态；电比例控制阀20与车载控制器18的PWM输出相连并控制铲刀姿态控制机构，包括左升降液压油缸4、右升降液压油缸5、斜拉油缸6、铲刀引出油缸12、铲土角控制油缸11、驱动铲刀回转角度的液压马达21等；通过这些执行机构的动作，可以改变和调整铲刀10的全部空间作业姿态。人机显示器19与视觉处理单元17和车载控制器18分别通过CAN总线相互通讯，人机显示器19在系统上电时，会对视觉识别系统的自检结果进行显示，提供整车三维图形，并用红色异常点标注出需要清洗或者更换的摄像头15或者视觉感知追踪点16，并且向用户提示后续维护的具体操作步骤；人机显示器19同时用于设置并向视觉处理单元17发送铲刀10的目标姿态、基于视觉处理单元17获取的铲刀10的当前姿态实时显示整车三维图形、切换手动控制状态和自动控制状态等。As shown in FIGS. 6 to 10 , eachcamera 15 sends the collected real-time position information of each visualperception tracking point 16 to the visual processing unit 17; the visual processing unit 17 is a GPU with a strong computing function, which needs to perform For the calculation of visual recognition, the visual processing unit 17 obtains the current posture of theblade 10 according to the received real-time position information of each visualperception tracking point 16, and generates the blade posture in combination with the target posture of theblade 10 sent by the man-machine display 19 The target state of the control mechanism, and the control decision mechanisms such as the target state and the current state of the blade attitude control mechanism are sent to the on-board controller 18 through the CAN bus; the on-board controller 18 is based on the received target state of the blade attitude control mechanism and In the current state, the blade attitude control mechanism is controlled by the electric proportional control valve 20, so that the blade attitude control mechanism drives theblade 10 to the target attitude; the electric proportional control valve 20 is connected to the PWM output of the vehicle controller 18 and controls the blade attitude control Mechanisms, including the left liftinghydraulic cylinder 4, the right liftinghydraulic cylinder 5, theoblique pulling cylinder 6, the blade lead-outcylinder 12, the shovelangle control cylinder 11, thehydraulic motor 21 that drives the blade rotation angle, etc.; through the actions of these actuators , the working posture of theblade 10 in all spaces can be changed and adjusted. The human-machine display 19 communicates with the visual processing unit 17 and the vehicle-mounted controller 18 respectively through the CAN bus. When the system is powered on, the human-machine display 19 will display the self-test results of the visual recognition system, provide the three-dimensional graphics of the whole vehicle, and use the The red abnormal point marks thecamera 15 or the visualperception tracking point 16 that needs to be cleaned or replaced, and prompts the user for the specific operation steps of the follow-up maintenance; the human-machine display 19 is also used to set and send the target of theblade 10 to the visual processing unit 17 Attitude, real-time display of three-dimensional graphics of the entire vehicle based on the current attitude of theblade 10 obtained by the visual processing unit 17 , switching between the manual control state and the automatic control state, etc.

本实施例中，摄像头15共设置7个，分别安装于整车的固定位置，包括驾驶室14和前机架1等；视觉感知追踪点16按照设定的尺寸间隔均匀安装于铲刀10、回转支撑8和铲刀姿态控制机构的各个控制油缸及液压马达21等关键部位上，且视觉感知追踪点16的个数符合1用2备的比例（即对于同样尺寸获取的追踪点，有两个备用点）。视觉感知摄像头15为红外摄像头，具有毫米级别的定位精度及毫秒级别的响应时间，且该摄像头具有自动控制摄像头盖，在正常施工过程中可以打开镜头盖，在施工结束后自动关闭镜头盖。工作时，利用视觉感知摄像头15捕捉视觉感知追踪点16的空间坐标，传送至视觉处理单元17，在视觉处理单元17中通过预制的算法，解算出当前铲刀10的全部空间姿态以及铲刀姿态控制机构中各油缸（左升降油缸4、右升降油缸5、斜拉油缸6、铲土角油缸11、引出油缸12）的活塞杆伸长量（即铲刀姿态控制机构的当前状态），全面获得平地机工作装置系统的当前状态，见图9、图10。In this embodiment, a total of 7cameras 15 are installed, which are respectively installed at fixed positions of the vehicle, including thecab 14 and thefront frame 1, etc.; the visualperception tracking points 16 are evenly installed on theblades 10, 10 and 10 according to the set size intervals. On key parts such as theslewing support 8 and each control cylinder andhydraulic motor 21 of the blade attitude control mechanism, and the number of visualperception tracking points 16 conforms to the ratio of 1 for use and 2 for backup (that is, for the tracking points obtained with the same size, there are two tracking points. a spare point). Thevisual perception camera 15 is an infrared camera with millimeter-level positioning accuracy and millisecond-level response time, and the camera has an automatically controlled camera cover, which can be opened during normal construction and automatically closed after construction. When working, thevisual perception camera 15 is used to capture the spatial coordinates of the visualperception tracking point 16, and then sent to the visual processing unit 17, in the visual processing unit 17, through the prefabricated algorithm, the overall spatial posture and the blade posture of thecurrent blade 10 are calculated. The elongation of the piston rod of each oil cylinder in the control mechanism (left lift cylinder 4,right lift cylinder 5, inclined-pull cylinder 6,shovel angle cylinder 11, lead-out cylinder 12) (that is, the current state of the blade attitude control mechanism), comprehensive Get the current status of the grader work gear system, see Figure 9, Figure 10.

铲刀姿态的控制：视觉处理单元17从人机显示器19中获得用户设定的铲刀10目标姿态，基于该目标姿态解算出铲刀姿态控制机构（左升降油缸4、右升降油缸5、斜拉油缸6、铲土角油缸11、引出油缸12）的目标状态（即在目标姿态下，铲刀姿态控制机构中各油缸的活塞杆伸长量），并把铲刀姿态控制机构的当前状态以及目标状态信息传送至车载控制器18，车载控制器17发出铲刀姿态控制机构中各油缸的运动方向和速度（实测姿态与目标姿态相差越大，控制机构的执行速度越快）的控制信号，通过电比例控制阀20控制左升降油缸4、右升降油缸5、斜拉油缸6、引出油缸12、铲土角油缸11、用于铲刀回转角度驱动的液压马达21进行动作，实时调整铲刀10的全部空间姿态向目标姿态逼近，通过视觉感知摄像头15实时获取铲刀10的最新姿态信息，形成闭环控制，直到铲刀10达到目标姿态时停止。由于视觉感知追踪点16直接安装于铲刀10上，并通过闭环控制进行铲刀姿态的调整与定位，铲刀姿态的最终控制精度不受平地机各运动结构的间隙影响，控制精度高。Control of blade attitude: the visual processing unit 17 obtains the target attitude of theblade 10 set by the user from the man-machine display 19, and calculates the blade attitude control mechanism (left liftingcylinder 4,right lifting cylinder 5, inclined blade 10) based on the target attitude. The target state of the pullingoil cylinder 6, the shovelangle oil cylinder 11, and the lead-out oil cylinder 12) (that is, the extension of the piston rod of each oil cylinder in the blade attitude control mechanism under the target attitude), and the current state of the blade attitude control mechanism And the target state information is sent to the on-board controller 18, and the on-board controller 17 sends out the motion direction and speed of each oil cylinder in the blade attitude control mechanism (the greater the difference between the measured attitude and the target attitude, the faster the execution speed of the control mechanism). , through the electric proportional control valve 20 to control theleft lift cylinder 4, theright lift cylinder 5, the inclined-pull cylinder 6, the lead-out cylinder 12, theshovel angle cylinder 11, and thehydraulic motor 21 for driving the blade rotation angle to operate, and adjust the shovel in real time. All the spatial postures of theblade 10 approach the target posture, and the latest posture information of theblade 10 is obtained in real time through thevisual perception camera 15 to form a closed-loop control until theblade 10 reaches the target posture and stops. Since the visualperception tracking point 16 is directly installed on theblade 10, and the blade attitude is adjusted and positioned through closed-loop control, the final control accuracy of the blade attitude is not affected by the gaps of the motor graders, and the control precision is high.

防碰撞控制：基于平地机工作装置系统结构及铲刀姿态控制机构（左升降油缸4、右升降油缸5、斜拉油缸6、铲土角油缸11、引出油缸12、液压马达21）的运动范围，解算出平地机铲刀10在所有可能的姿态全集中与平地机其他结构间的距离小于某一设定值或者发生碰撞、干涉的姿态子集，命名为：铲刀碰撞姿态子集，并将该铲刀碰撞姿态子集的信息存放在车载控制器18中。在铲刀10由当前姿态向目标姿态变换的过程中，车载控制器18自动选择一条避开铲刀碰撞姿态子集的铲刀姿态控制机构的控制序列对铲刀姿态进行实时调整；同时，在手动控制状态下，当驾驶员手动调整铲刀姿态并达到铲刀碰撞姿态子集的边界时，车载控制器18会停止铲刀运动，并发出声、光报警信息，提醒驾驶员进行处理，提高了作业安全性。Anti-collision control: based on the system structure of the grader working device and the blade attitude control mechanism (left lift cylinder 4,right lift cylinder 5,inclined pull cylinder 6,shovel angle cylinder 11, lead-out cylinder 12, hydraulic motor 21) The movement range , and calculate the subset of attitudes in which the distance between thegrader blade 10 and other structures of the grader is less than a certain set value or collides or interferes in the full set of all possible attitudes, which is named: blade collision attitude subset, and The information on the subset of blade collision attitudes is stored in the onboard controller 18 . During the process of changing theblade 10 from the current attitude to the target attitude, the vehicle-mounted controller 18 automatically selects a control sequence of the blade attitude control mechanism that avoids the blade collision attitude subset to adjust the blade attitude in real time; In the manual control state, when the driver manually adjusts the blade attitude and reaches the boundary of the blade collision attitude subset, the on-board controller 18 will stop the blade movement, and send out sound and light alarm information to remind the driver to deal with it and improve the situation. job safety.

本实施例可实现铲刀全姿态（铲刀入土深度、铲刀铲土角度、铲刀回转角度、铲刀横坡角度、铲刀引出位移等）以及液压控制油缸的活塞杆伸长量实时检测；直接追踪铲刀上的视觉感知追踪点，并通过换算得到铲刀的作业姿态，检测精度高，且不受机构间隙的影响；基于平地机工作装置系统结构、铲刀目标姿态和实际姿态的差异，快速解算出铲刀由当前姿态变换到目标姿态时，各个铲刀姿态控制机构的运动幅度和方向，反馈至控制系统实时调整和控制平地机铲刀姿态，精确、快速达到目标姿态，提高施工质量，降低驾驶员劳动强度；基于平地机工作装置系统结构及铲刀姿态控制机构的运动范围，解算出平地机铲刀在所有可能的姿态全集中与平地机其他结构间的距离小于某一设定值或者发生碰撞、干涉的姿态子集，并将该子集的信息存放在车载控制器中；在铲刀由当前姿态向目标姿态变换的过程中，控制器自动选择一条避开铲刀碰撞姿态子集的执行机构控制序列对铲刀姿态进行实时调整；同时，当驾驶员手动调整铲刀姿态并达到铲刀碰撞姿态子集的边界时，系统会停止铲刀运动，并发出声音报警，提醒驾驶员进行处理，提高了作业安全性。This embodiment can realize the real-time detection of the full posture of the blade (the depth of the blade entering the soil, the angle of the blade, the angle of rotation of the blade, the angle of the transverse slope of the blade, the displacement of the blade, etc.) and the elongation of the piston rod of the hydraulic control cylinder. ; Directly track the visual perception tracking points on the blade, and obtain the working attitude of the blade through conversion, with high detection accuracy and is not affected by the gap of the mechanism; Based on the system structure of the grader working device, the target attitude and the actual attitude of the blade Differences, quickly calculate the movement range and direction of each blade attitude control mechanism when the blade changes from the current attitude to the target attitude, and feed it back to the control system to adjust and control the blade attitude of the grader in real time, accurately and quickly reach the target attitude, improve The construction quality reduces the labor intensity of the driver; based on the system structure of the grader working device and the motion range of the blade attitude control mechanism, it is calculated that the distance between the grader blade and other structures of the grader in all possible attitudes is less than a certain Set the value or a subset of attitudes that collide and interfere, and store the information of the subset in the on-board controller; in the process of changing the blade from the current attitude to the target attitude, the controller automatically selects a blade that avoids the blade The actuator control sequence of the collision attitude subset adjusts the blade attitude in real time; at the same time, when the driver manually adjusts the blade attitude and reaches the boundary of the blade collision attitude subset, the system will stop the blade movement and issue an audible alarm , reminding the driver to deal with it, improving the safety of the operation.

实施例二：Embodiment 2:

基于实施例一所述的一种基于机器视觉的铲刀姿态控制系统，本实施例提供一种平地机，所述平地机配置有实施例一所述的基于机器视觉的铲刀姿态控制系统。Based on the machine vision-based blade attitude control system described inEmbodiment 1, this embodiment provides a motor grader equipped with the machine vision-based blade attitude control system described inEmbodiment 1.

实施例三：Embodiment three:

基于实施例一所述的一种基于机器视觉的铲刀姿态控制系统，本实施例提供一种基于机器视觉的铲刀姿态控制方法，包括：Based on the machine vision-based blade attitude control system described inEmbodiment 1, this embodiment provides a machine vision-based blade attitude control method, including:

1）采集安装于平地机的设定位置的多个视觉感知追踪点的实时位置信息；1) Collect real-time position information of multiple visual perception tracking points installed at the set position of the grader;

2）根据各个视觉感知追踪点的实时位置信息获取铲刀的当前姿态，并结合铲刀的目标姿态，生成铲刀姿态控制机构的目标状态；2) Obtain the current attitude of the blade according to the real-time position information of each visual perception tracking point, and combine the target attitude of the blade to generate the target state of the blade attitude control mechanism;

3）根据铲刀姿态控制机构的目标状态和当前状态，通过电比例控制阀控制铲刀姿态控制机构，使铲刀姿态控制机构驱动铲刀至目标姿态，具体为：3) According to the target state and current state of the blade attitude control mechanism, the blade attitude control mechanism is controlled by the electric proportional control valve, so that the blade attitude control mechanism drives the blade to the target attitude, specifically:

31）根据铲刀姿态控制机构的目标状态和当前状态，生成铲刀姿态控制机构的初始控制序列；31) According to the target state and current state of the blade attitude control mechanism, generate the initial control sequence of the blade attitude control mechanism;

32）将生成的铲刀姿态控制机构的初始控制序列与铲刀碰撞姿态子集进行比较并选择一条避开所述铲刀碰撞姿态子集的铲刀姿态控制机构的控制序列作为执行控制序列；32) Compare the generated initial control sequence of the blade attitude control mechanism with the blade collision attitude subset and select a control sequence of the blade attitude control mechanism that avoids the blade collision attitude subset as the execution control sequence;

33）基于步骤32）中选择的执行控制序列，通过电比例控制阀控制铲刀姿态控制机构，使铲刀姿态控制机构驱动铲刀至目标姿态。33) Based on the execution control sequence selected in step 32), control the blade attitude control mechanism through the electric proportional control valve, so that the blade attitude control mechanism drives the blade to the target attitude.

以上所述仅是本发明的优选实施方式，应当指出，对于本技术领域的普通技术人员来说，在不脱离本发明技术原理的前提下，还可以做出若干改进和变形，这些改进和变形也应视为本发明的保护范围。The above are only the preferred embodiments of the present invention. It should be pointed out that for those skilled in the art, without departing from the technical principle of the present invention, several improvements and modifications can also be made. These improvements and modifications It should also be regarded as the protection scope of the present invention.