Mobile robot safety protection method and deviceTechnical Field
The invention relates to the field of robot safety, in particular to a mobile robot safety protection method and device.
Background
Scene perception is a very critical link in the operation of mobile robots, which directly determines that mobile robots cannot operate stably, reliably and safely in a scene. The sensors for sensing the robot scene can be divided into a contact type and a non-contact type according to the contact type, wherein the contact type is widely applied to household sweeping robots, and other types of mobile robots generally adopt the non-contact type due to the limitation of application scenes.
Non-contact mainstream sensors include 2D lasers, 3D lasers, depth cameras, and radar.
The 2D laser can only sense the distance information of the object under a certain plane, and the object exists in the whole three-dimensional space, so that the scene sensing capability is weaker.
The 3D laser can sense 3D information of the whole scene, but its cost is higher, and it also can cause missing detection problem due to the laser passing through transparent glass.
The depth camera has good performance in the short-distance three-dimensional space sensing capability, and the cost is far lower than that of 3D laser, but the sensing problems of glass walls and the like are difficult to solve by using TOF, structured light and binocular depth cameras.
The radar can effectively detect the glass wall, but can only detect whether an obstacle exists in a certain area, and cannot give out space position information, so that the radar generally plays a role in safety warning assistance. For example, a robot obstacle avoidance system disclosed in Chinese patent literature, the bulletin number CN108527364A of which comprises an acquisition module, a control module and an execution module, wherein the acquisition module is used for acquiring distance information between a robot and an obstacle, the control module is in communication connection with the acquisition module and acquires the distance information to generate a control signal, and the execution module is in communication connection with the control module and controls the movement of the robot according to the control signal. According to the scheme, the active obstacle avoidance of the robot is realized by arranging two types of sensors, namely the laser radar and the ultrasonic array. The scheme adopts two sensors to realize the obstacle avoidance function for glass, and has high cost.
Disclosure of Invention
The invention mainly solves the problem that the single sensor in the prior art is difficult to solve the perception of glass walls and the like; the method and the device for protecting the mobile robot safely are provided, and by means of the structural information of the projection pattern, the obstacle in the scene is detected, so that the problem of perceived pain points of other sensors such as a glass wall can be effectively solved, the cost is relatively low, and the protection requirement of the mobile robot can be better met.
The technical problems of the invention are mainly solved by the following technical proposal:
a mobile robot safety protection method, comprising the following steps:
S1, acquiring a real-time image of a detection area relative to a projection image by a camera, and comparing the real-time image with a reference image of a preset plane area relative to the projection image to acquire an image difference area;
S2, determining the position of an obstacle according to the image difference region, and calculating the starting point of line segment difference in the image difference region;
S3, determining the position information of the line segment difference starting point in a vehicle body coordinate system through the mapping relation of the coordinate system;
And S4, carrying out robot safety protection in real time according to the position information in the vehicle body coordinate system.
The scheme includes comparing a real-time image of a projected image with a reference image, determining the position of an obstacle starting point through image difference, obtaining the coordinates of the obstacle starting point under a vehicle body coordinate system through a coordinate system mapping relation, and controlling the protection start of the robot according to the threshold comparison of the distance. By means of the structural information of the projection patterns, obstacles in a scene are detected, the problem of perception pain points of other sensors such as a glass wall can be effectively solved, the cost is relatively low, and the protection requirement of a mobile robot can be better met.
Preferably, the presetting process of the reference graph is as follows:
the pattern projector projects the projection image into the scene, and when the projection image is in a plane area, the reference frame image is shot and acquired to be used as a preset reference image.
The reference image is a reference frame image taken when the image and the planar area are projected.
Preferably, the real-time image and the reference image obtain an image difference region through image difference, and whether a non-planar object exists in the scene is judged according to whether the difference region exists after the image difference.
Preferably, the expression of the image difference is:
wherein I (x, y) is the pixel value at coordinates (x, y) in the real-time map;
Iref (x, y) is the pixel value at coordinate (x, y) in the reference map;
thre is a threshold of whether a change occurs.
Preferably, when determining that a non-planar object exists in the scene, determining the starting point of the difference of each line segment, searching for a difference point along the direction from bottom to top of each line segment in the reference graph, wherein the first difference point is the starting point of the difference of each line segment.
The difference points are searched from the bottom to the top, the first difference point is used as the starting point of the difference of each line segment, the lines in the image are farther from the robot body from bottom to top due to the installation position relation of the pattern projector, and one point closest to the robot is selected as the starting point of the obstacle.
Preferably, the process for establishing the mapping relation from the point of the image plane to the vehicle body coordinate system comprises the following steps:
A1, fixedly placing a calibration plate on a ground plane;
A2, mapping points on the image to a ground plane coordinate system;
Establishing a ground plane coordinate system on a calibration plate (so that coordinate points of each angular point of the checkerboard under the ground plane coordinate system can be determined), and simultaneously detecting the angular points of the checkerboard (so that coordinate points of each angular point of the checkerboard under the image plane coordinate system can be determined), and solving a homography matrix based on the corresponding relation between the coordinate points under the ground plane coordinate system and the coordinate points under the image plane coordinate system;
a3, mapping the ground plane coordinate system to a camera coordinate system;
acquiring image data of a covering complete calibration plate, and performing external parameter calibration (assuming that the internal parameter calibration of the camera is finished in advance) to obtain a mapping relation from a ground plane coordinate system to a camera coordinate system;
and A4, mapping the camera coordinate system to the vehicle body coordinate system through hand-eye calibration.
The mapping relation between the image plane points and the vehicle body coordinate system is established in advance, and the position information of the starting point of the obstacle under the vehicle body coordinate system can be determined through the mapping relation.
Preferably, the distance from the current coordinate of the robot to the starting point of the obstacle is calculated, and when the distance is smaller than or equal to a set safety threshold value, protection is started, and the robot is braked or controlled to turn.
And carrying out robot safety protection in real time according to the position information of the starting point of the non-planar object under the vehicle body coordinate system.
A mobile robot safety shield apparatus, comprising:
A camera for acquiring a real-time image including a projection image in a detection area;
a pattern projector that projects a projection image into a detection area in a robot advancing direction;
The calculation and analysis unit compares the real-time image comprising the projection image with the reference image, determines the position of the starting point of the obstacle through image difference, obtains the coordinate of the starting point of the obstacle under the coordinate system of the vehicle body through the mapping relation of the coordinate system, and controls the protection start of the robot according to the threshold comparison of the distance.
By means of the structural information of the projection patterns, obstacles in a scene are detected, the problem of perception pain points of other sensors such as a glass wall can be effectively solved, the cost is relatively low, and the protection requirement of a mobile robot can be better met.
Preferably, the projection image is a plurality of parallel lines, and the line direction extends along the advancing direction of the robot. The arrangement of the scheme enables lines in the image to be farther from the robot body from bottom to top, and a point closest to the robot is selected as a starting point of the obstacle.
Preferably, the line pitches are equal. The line density can be adjusted according to the measurement accuracy requirement.
The beneficial effects of the invention are as follows:
Comparing a real-time image comprising the projected image with a reference image, determining the position of an obstacle starting point through image difference, obtaining the coordinates of the obstacle starting point under a vehicle body coordinate system through a coordinate system mapping relation, and controlling the protection starting of the robot according to the threshold comparison of the distance. By means of the structural information of the projection patterns, obstacles in a scene are detected, the problem of perception pain points of other sensors such as a glass wall can be effectively solved, the cost is relatively low, and the protection requirement of a mobile robot can be better met.
Drawings
Fig. 1 is a schematic view of a mobile robot safety device of the present invention.
Fig. 2 is a flow chart of the mobile robot safety protection method of the present invention.
Fig. 3 is a planar scene image of the present invention.
Fig. 4 is a non-planar scene image of the present invention.
Fig. 5 is a graph of the differential results of a planar scene image of the present invention.
Fig. 6 is a differential result diagram of a non-planar scene image of the present invention.
In the figure, 1, a robot body, 2, a pattern projector, and 3, a camera and 4, projecting an image.
Detailed Description
The technical scheme of the invention is further specifically described below through examples and with reference to the accompanying drawings.
Examples:
A mobile robot safety device of the present embodiment, as shown in fig. 1, includes a robot body 1, a pattern projector 2 and a camera 3 provided on the robot body 1.
In the present embodiment, the pattern projector 2 and the camera 3 are provided on the top of the robot body 1, facing the robot advancing direction.
The pattern projector 2 projects a projection image 4 into a detection area in the robot advancing direction, and the camera 3 acquires a real-time map including the projection image in the detection area. In the present embodiment, the camera 3 employs an RGB camera.
In this embodiment, the projected image 4 is a plurality of parallel lines, and the line direction extends along the robot advancing direction. So that the line in the projected image 4 is further from the robot body 1 from bottom to top, and a point nearest to the robot is selected as the starting point of the obstacle. In this embodiment, the line pitches in the projected image 4 are equal, and the line density can be adaptively adjusted according to the measurement accuracy requirement.
The mobile robot safety protection device further comprises a calculation and analysis unit, the calculation and analysis unit compares a real-time image comprising the projection image 4 with a reference image, determines the starting point position of the obstacle through image difference, obtains the starting point coordinates of the obstacle under the vehicle body coordinate system through the coordinate system mapping relation, and compares and controls the protection starting of the robot according to the threshold value of the distance.
By means of the structural information of the projection pattern 4, obstacles in a scene are detected, the problem of perception pain points of other sensors such as a glass wall can be effectively solved, the cost is relatively low, and the protection requirement of a mobile robot can be better met.
The mobile robot safety protection method of the embodiment, as shown in fig. 2, includes the following steps:
S1, acquiring a real-time image of a detection area relative to a projection image by a camera, and comparing the real-time image with a reference image of a preset plane area relative to the projection image to acquire an image difference area.
The presetting process of the reference picture is as follows:
the pattern projector 2 projects a projection image 4 into a scene, and takes a reference frame image as a preset reference image when the projection image is in a planar area as shown in fig. 3.
The reference image is a reference frame image shot when the projection image 4 is in a plane area, and is used for comparing with a real-time image acquired by the camera 3 to obtain an image difference area.
When the area detected by the robot is a plane, as shown in fig. 3, the straight line of the projected image 4 is still a straight line on the image plane. When there is a non-planar object in the area detected by the robot, the straight line of the projected image 4 will be deformed in the image plane as shown in fig. 4. When the laser line penetrates the glass, the straight line of the projected image 4 will be deformed due to the refraction of the light.
And the real-time image and the reference image obtain an image difference area through image difference, and whether a non-planar object exists in the scene is judged according to whether the difference area exists after the image difference.
The expression of the image difference is:
wherein I (x, y) is the pixel value at coordinates (x, y) in the real-time map;
Iref (x, y) is the pixel value at coordinate (x, y) in the reference map;
thre is a threshold of whether a change occurs.
When the difference area does not exist after the image difference, the image difference result is shown in fig. 5, and the scenes are judged to be plane objects.
When a difference area exists after the image difference, the image difference result is shown in fig. 6, and then a non-planar object or glass exists in the scene.
S2, determining the position of the obstacle according to the image difference area, and calculating the starting point of the line segment difference in the image difference area.
And determining the position of the obstacle according to the image difference area of the image difference, wherein the position with larger difference indicates that a non-planar object (obstacle) appears.
When it is determined that a non-planar object exists in the scene, a starting point for each line segment difference is determined. And searching a difference point along the direction from bottom to top of each line segment in the reference graph, wherein the first difference point is the starting point of the difference of each line segment.
The difference points are searched from the bottom to the top, the first difference point is used as the starting point of the difference of each line segment, the lines in the image are farther from the robot body from bottom to top due to the installation position relation of the pattern projector, and one point closest to the robot is selected as the starting point of the obstacle.
And S3, determining the position information of the line segment difference starting point in the vehicle body coordinate system through the mapping relation of the coordinate system.
The mapping relation between the image plane points and the vehicle body coordinate system is established in advance, and the position information of the starting point of the obstacle under the vehicle body coordinate system can be determined through the mapping relation.
The mapping relation establishment process from the point of the image plane to the vehicle body coordinate system comprises the following steps:
A1, fixedly placing a calibration plate on a ground plane;
A2, mapping points on the image to a ground plane coordinate system;
Establishing a ground plane coordinate system on a calibration plate (so that coordinate points of each angular point of the checkerboard under the ground plane coordinate system can be determined), and simultaneously detecting the angular points of the checkerboard (so that coordinate points of each angular point of the checkerboard under the image plane coordinate system can be determined), and solving a homography matrix based on the corresponding relation between the coordinate points under the ground plane coordinate system and the coordinate points under the image plane coordinate system;
a3, mapping the ground plane coordinate system to a camera coordinate system;
acquiring image data of a covering complete calibration plate, and performing external parameter calibration (assuming that the internal parameter calibration of the camera is finished in advance) to obtain a mapping relation from a ground plane coordinate system to a camera coordinate system;
and A4, mapping the camera coordinate system to the vehicle body coordinate system through hand-eye calibration.
And S4, carrying out robot safety protection in real time according to the position information in the vehicle body coordinate system.
And calculating the distance from the current coordinate of the robot to the starting point of the obstacle, and starting protection and braking or controlling the robot to turn when the distance is smaller than or equal to a set safety threshold value.
And carrying out robot safety protection in real time according to the position information of the starting point of the non-planar object under the vehicle body coordinate system.
The scheme of the embodiment compares a real-time image including a projection image with a reference image, determines the position of an obstacle starting point through image difference, obtains the coordinates of the obstacle starting point under a vehicle body coordinate system through a coordinate system mapping relation, and controls the protection start of the robot according to the threshold comparison of the distance. By means of the structural information of the projection patterns, obstacles in a scene are detected, the problem of perception pain points of other sensors such as a glass wall can be effectively solved, the cost is relatively low, and the protection requirement of a mobile robot can be better met.
It should be understood that the examples are only for illustrating the present application and are not intended to limit the scope of the present application. Furthermore, it should be understood that various changes and modifications can be made by one skilled in the art after reading the teachings of the present application, and such equivalents are intended to fall within the scope of the application as defined in the appended claims.