CN109947108B - Method for predicting road condition in front of mobile robot - Google Patents

Abstract

The method for predicting the road condition in front of the mobile robot comprises a binocular stereo camera arranged at the front part of the mobile robot, wherein the optical axis of the binocular stereo camera is parallel to the chassis of the mobile robot, the method also comprises a processor connected with the binocular stereo camera, and the processor is used for setting the method for predicting the road condition in front and comprises the following steps: obtaining depth information z = f₁(x, y); calculating a projection angle θ = arctan (y/f); the perpendicular distance l = z · tan θ from the ground point to the optical axis; calculating the degree of unevenness f₂(x, y) = h-l, binary f₃(x, y); mapping the binary image f₃(x, y) performing erosion and expansion operations to obtain optimized values, and removing f by using hadamard product of matrix₂Stray data in (x, y); from the direction of travel, the area s of the uneven road surface and the average unevenness are calculated. The method provides front road information for the mobile robot on the basis of binocular stereoscopic vision, and ensures safe work.

Description

Method for predicting road condition in front of mobile robot

Technical Field

The invention relates to a method for predicting a road condition in front of a mobile robot, belonging to the field of image processing of machine vision.

Background

Most of the outdoor mobile robots and the indoor mobile robots are driven by three wheels or four wheels. The exception is that the two-foot or four-foot upright walking robot of boston power company can overcome the difficulty of uneven road surface under laboratory conditions or limited practical environments, and most mobile robots work on the premise of flat road surface, but accidents can be caused when the ground is uneven. The household dust collector is characterized in that an infrared proximity sensor is arranged at the front end of a driving wheel, and the household dust collector must be stopped to avoid falling when the infrared proximity sensor cannot detect the ground. But this approach is not universally significant.

Machine vision and binocular stereo vision developed on the basis have the advantages of wide detection range and rich information, and can be used for detecting the front road surface condition.

Disclosure of Invention

Aiming at the problems, the invention provides a method for predicting the road condition in front of a mobile robot, which helps the mobile robot to detect the leveling condition of the ground in front.

The technical scheme adopted by the invention for solving the technical problems is as follows:

the method for predicting the road condition in front of the mobile robot comprises a binocular stereo camera arranged in front of the mobile robot, wherein the focal length is f, the base line width is b, the height is h, the optical axis of the binocular stereo camera is parallel to the chassis of the mobile robot, the method further comprises a processor connected with the binocular stereo camera, and the processor is used for setting the method for predicting the road condition in front and comprises the following steps:

(1) the processor acquires the image pair f of the binocular stereo camera_LAnd f_RForming depth information z = f₁(x, y) = f b/d, where d is from image pair f_LAnd f_RCalculating the parallax of the obtained position (x, y), wherein x, y are image plane coordinates, and z is the corresponding depth;

(2) for imaging point (x, y), depth z = f₁(x, y), projection angle θ = arctan (y/f); the vertical distance l = z · tan θ = z · y/f from the ground point corresponding to the imaging point to the optical axis of the binocular stereo camera;

(3) calculating the degree of unevenness f₂(x, y) = h-l if | h-l->T, then is denoted as f₃(x, y) =1, otherwise f₃(x, y) =0, wherein the threshold T is the maximum unevenness that the mobile robot can cross;

(4) mapping the binary image f₃(x, y) removing the binary image f by performing an erosion-first and then dilation operation₃Excess impurity sites in (x, y); on the basis, the operation of expansion and corrosion is carried out, and a binary image f is filled₃Fine holes in (x, y) to finally obtain an optimized binary image f₃(x, y) calculating a binary map f₃(x, y) and f₂Hadamard product of (x, y), removing f₂Stray data in (x, y), i.e. f₂(x,y)=f₃(x,y)*f₂(x,y)；

The mobile robot is in f according to the advancing direction₃Scanning the road surface condition on (x, y), and uneven road surface area s = ∑ f₃(x,y)·z·s_pixF, average roughness ∑ f₂(x,y)/∑f₃(x, y), wherein the value range of x, y is determined according to the moving direction of the mobile robot, s_pixIs the sensor pixel size.

The invention has the following beneficial effects: 1. the leveling state of the front ground can be predicted in advance, and environmental information is provided for the motion and navigation control of the mobile robot; 2. the operation speed is high, and finally a detailed ground unevenness description file is formed.

Drawings

FIG. 1 is a schematic exterior view of a mobile robot;

FIG. 2 is a schematic of the out-of-flatness calculation;

FIG. 3 is a schematic illustration of unevenness patterning.

Detailed Description

The invention is further described below with reference to the accompanying drawings:

referring to fig. 1 to 3, the method for predicting the road condition ahead of the mobile robot includes a binocular stereo camera disposed at the front of the mobile robot, the focal length is f, the baseline width is b, and the height is h, and the optical axis of the binocular stereo camera is parallel to the chassis of the mobile robot. The binocular stereo camera is the basic configuration of the mobile robot, can output monocular common images, and can provide depth information for obstacle avoidance navigation.

The binocular stereo camera system further comprises a processor connected with the binocular stereo camera, the processor is provided with a front road condition prediction method, and the binocular stereo camera system comprises the following steps:

(1) the processor acquires the image pair f of the binocular stereo camera_LAnd f_RForming depth information z = f₁(x, y) = f b/d, where d is from image pair f_LAnd f_RThe calculated parallax of the position (x, y),x and y are image plane coordinates, and z is corresponding depth;

the processor calculates depth information according to the parallax principle, and the z = f can be obtained by using the parameters of the binocular stereo camera₁(x, y) = f b/d, where d is from image pair f_LAnd f_RThe parallax of the resulting position (x, y) is calculated.

(2) For imaging point (x, y), depth z = f₁(x, y), projection angle θ = arctan (y/f); the vertical distance l = z · tan θ = z · y/f from the ground point corresponding to the imaging point to the optical axis of the binocular stereo camera;

as shown in fig. 2, the vertical distance l can be calculated according to the tangent formula of the right triangle.

(3) Calculating the degree of unevenness f₂(x, y) = h-l if | h-l->T, then is denoted as f₃(x, y) =1, otherwise f₃(x, y) =0, wherein the threshold T is the maximum unevenness that the mobile robot can cross;

f₂(x, y) is according to the depth map f₁(x, y) and f is calculated by using a threshold value T to filter the influence of the calculation noise₃(x, y) is subjected to binarization to obtain f₂(x,y)。

(4) Mapping the binary image f₃(x, y) removing the binary image f by performing an erosion-first and then dilation operation₃Excess impurity sites in (x, y); on the basis, the operation of expansion and corrosion is carried out, and a binary image f is filled₃Fine holes in (x, y) to finally obtain an optimized binary image f₃(x, y) calculating a binary map f₃(x, y) and f₂Hadamard product of (x, y), removing f₂Stray data in (x, y), i.e. f₂(x,y)=f₃(x,y)*f₂(x,y)；

Mapping the binary image f₃(x, y) carrying out optimization treatment, removing stray points and missed detection points, and then optimizing f by using a matrix operation hadamard product₂(x,y)。

(5) The mobile robot is in f according to the advancing direction₃Scanning the road surface condition on (x, y), and uneven road surface area s = ∑ f₃(x,y)·z·s_pixF, average ofFlatness of Σ f₂(x,y)/∑f₃(x, y), wherein the value range of x, y is determined according to the moving direction of the mobile robot, s_pixIs the sensor pixel size.

Finally, according to the optimized unevenness map f₃(x, y) the area and depth of the protrusion or depression location can be calculated, where z · s_pixAnd/f is the imaging area size corresponding to a single sensor pixel.

Claims (1)

1. The method for predicting the road condition in front of the mobile robot comprises a binocular stereo camera arranged at the front part of the mobile robot, wherein the focal length is f, the base line width is b, and the height is h, and the method is characterized in that: the optical axis of the binocular stereo camera is parallel to the chassis of the mobile robot, the binocular stereo camera system further comprises a processor connected with the binocular stereo camera, and the processor is provided with a front road condition prediction method, and the method comprises the following steps:

(1) the processor acquires the image pair f of the binocular stereo camera_LAnd f_RForming depth information z = f₁(x, y) = f b/d, where d is from image pair f_LAnd f_RCalculating the parallax of the obtained position (x, y), wherein x, y are image plane coordinates, and z is the corresponding depth;

(2) for imaging point (x, y), depth z = f₁(x, y), projection angle θ = arctan (y/f); the vertical distance l = z · tan θ = z · y/f from the ground point corresponding to the imaging point to the optical axis of the binocular stereo camera;

(3) calculating the degree of unevenness f₂(x, y) = h-l if | h-l->T, then is denoted as f₃(x, y) =1, otherwise f₃(x, y) =0, wherein the threshold T is the maximum unevenness that the mobile robot can cross;

(4) mapping the binary image f₃(x, y) removing the binary image f by performing an erosion-first and then dilation operation₃Excess impurity sites in (x, y); on the basis, the operation of expansion and corrosion is carried out, and a binary image f is filled₃Fine holes in (x, y) to finally obtain an optimized binary image f₃(x, y) calculating a binary map f₃(x, y) and f₂Hadamard product of (x, y), removing f₂Stray data in (x, y), i.e. f₂(x,y)=f₃(x,y)*f₂(x,y)；

(5) The mobile robot is in f according to the advancing direction₃Scanning the road surface condition on (x, y), and uneven road surface area s = ∑ f₃(x,y)·z·s_pixF, average roughness ∑ f₂(x,y)/∑f₃(x, y), wherein the value range of x, y is determined according to the moving direction of the mobile robot, s_pixIs the sensor pixel size.

Images