CN109143167B - Obstacle information acquisition device and method - Google Patents
Obstacle information acquisition device and methodDownload PDFInfo
- Publication number
- CN109143167B CN109143167BCN201710509194.1ACN201710509194ACN109143167BCN 109143167 BCN109143167 BCN 109143167BCN 201710509194 ACN201710509194 ACN 201710509194ACN 109143167 BCN109143167 BCN 109143167B
- Authority
- CN
- China
- Prior art keywords
- laser
- obstacle
- position information
- image
- target image
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034methodMethods0.000titleclaimsabstractdescription56
- 238000003384imaging methodMethods0.000claimsabstractdescription121
- 230000000875corresponding effectEffects0.000claimsdescription21
- 230000003287optical effectEffects0.000claimsdescription10
- 230000002596correlated effectEffects0.000claimsdescription6
- 238000005259measurementMethods0.000claimsdescription5
- 230000033001locomotionEffects0.000claimsdescription4
- 230000004888barrier functionEffects0.000abstractdescription6
- 238000010586diagramMethods0.000description19
- 230000008569processEffects0.000description11
- 238000009434installationMethods0.000description8
- 230000009471actionEffects0.000description3
- 239000011159matrix materialSubstances0.000description3
- 238000004590computer programMethods0.000description2
- 230000001678irradiating effectEffects0.000description2
- 238000013473artificial intelligenceMethods0.000description1
- 230000009286beneficial effectEffects0.000description1
- 238000004891communicationMethods0.000description1
- 238000011161developmentMethods0.000description1
- 238000005516engineering processMethods0.000description1
- 230000007613environmental effectEffects0.000description1
- 238000005286illuminationMethods0.000description1
- 230000006872improvementEffects0.000description1
- 238000002372labellingMethods0.000description1
- 238000012986modificationMethods0.000description1
- 230000004048modificationEffects0.000description1
- 238000012545processingMethods0.000description1
- 238000011160researchMethods0.000description1
- 238000013519translationMethods0.000description1
Images
Classifications
- G—PHYSICS
- G01—MEASURING; TESTING
- G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
- G01S5/00—Position-fixing by co-ordinating two or more direction or position line determinations; Position-fixing by co-ordinating two or more distance determinations
- G01S5/16—Position-fixing by co-ordinating two or more direction or position line determinations; Position-fixing by co-ordinating two or more distance determinations using electromagnetic waves other than radio waves
- B—PERFORMING OPERATIONS; TRANSPORTING
- B25—HAND TOOLS; PORTABLE POWER-DRIVEN TOOLS; MANIPULATORS
- B25J—MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES
- B25J9/00—Programme-controlled manipulators
- G—PHYSICS
- G01—MEASURING; TESTING
- G01C—MEASURING DISTANCES, LEVELS OR BEARINGS; SURVEYING; NAVIGATION; GYROSCOPIC INSTRUMENTS; PHOTOGRAMMETRY OR VIDEOGRAMMETRY
- G01C3/00—Measuring distances in line of sight; Optical rangefinders
Landscapes
- Physics & Mathematics (AREA)
- Engineering & Computer Science (AREA)
- Electromagnetism (AREA)
- General Physics & Mathematics (AREA)
- Radar, Positioning & Navigation (AREA)
- Remote Sensing (AREA)
- Robotics (AREA)
- Mechanical Engineering (AREA)
- Manipulator (AREA)
- Control Of Position, Course, Altitude, Or Attitude Of Moving Bodies (AREA)
- Optical Radar Systems And Details Thereof (AREA)
- Length Measuring Devices By Optical Means (AREA)
Abstract
Description
Technical Field
The invention relates to the technical field of robot control, in particular to a device and a method for acquiring obstacle information.
Background
With the continuous development of artificial intelligence, electronic communication technology and the like, robots have gradually become a new hot research technical field, and are widely applied to multiple industries such as military, civil and the like. During the advancing process of the robot, the robot often encounters some obstacles, such as fixed pillars, obstacles which are difficult to be crossed by the temporarily stored robot, and the like. When the robot moves from one position to another, collision with these obstacles must be avoided, and thus, the robot needs to go around. How to enable the robot to avoid the obstacle without collision is an important precondition for determining the position of the obstacle.
One known solution to the problem of determining the position of an obstacle is to use an infrared sensor to obtain the position of the obstacle. Specifically, the position of the obstacle is determined according to the distance of one point on the obstacle obtained by the emission of the infrared sensor. However, the position accuracy of the obstacle obtained is low only by the distance information at a certain point.
Disclosure of Invention
The embodiment of the invention aims to provide an obstacle information acquisition device and method so as to improve the accuracy of determining the position of an obstacle. The specific technical scheme is as follows:
in a first aspect, an embodiment of the present invention provides an obstacle information acquiring apparatus, including:
a laser for emitting line laser light;
the image acquisition equipment is used for acquiring a target image, the target image comprises an image of an irradiation area of the line laser on the barrier, and the image acquisition equipment and the laser have a fixed relative position relation;
the acquisition module is used for acquiring imaging position information of laser line imaging of a laser line irradiation area of the line laser on the obstacle in the target image from the target image;
and the determining module is used for determining the actual position information of the obstacle according to the imaging position information.
Optionally, the obtaining module is specifically configured to determine an area where the laser line is imaged in the target image, and obtain imaging position information of a plurality of pixel points in the area in the target image;
the determining module is specifically configured to determine actual position information of the obstacle according to the imaging position information of the plurality of pixel points on the laser line in the target image.
Optionally, the determining module is specifically configured to determine, according to imaging position information of a plurality of pixel points on the laser line in the target image and preset parameters of the image acquisition device, a sight line equation corresponding to the plurality of pixel points on the laser line; and determining the actual position information of the obstacle according to the sight line equation corresponding to the multiple pixel points on the laser line and the light plane equation of the laser plane of the laser under the coordinate system corresponding to the image acquisition equipment.
Optionally, the optical axis of the image capturing device and the laser plane of the laser intersect in a region right in front of the image capturing device.
Optionally, the image acquisition device is located vertically below or vertically above the laser;
or the image acquisition equipment and the vertical direction of the laser have preset deviation, wherein the preset deviation is positively correlated with the emission angle of the laser.
Optionally, the laser and the image acquisition device are mounted on a robot.
Optionally, the direction of the line laser emitted by the laser is consistent with the moving direction of the robot; or,
the image acquisition equipment is horizontally arranged, and an acquisition area of the image acquisition equipment is an area right in front of the robot.
In a second aspect, an embodiment of the present invention further provides an obstacle information acquiring method, which is applied to an obstacle information acquiring apparatus, where the obstacle information acquiring apparatus includes: a laser and an image acquisition device; wherein the image acquisition equipment and the laser have a fixed relative position relationship; the method comprises the following steps:
acquiring a target image, wherein the target image comprises an image of a line laser emitted by the laser on an irradiated area of an obstacle;
acquiring imaging position information of laser line imaging of a laser line irradiation area of the line laser on an obstacle in the target image from the target image;
and determining the actual position information of the obstacle according to the imaging position information.
Optionally, the step of obtaining imaging position information of the laser line imaging of the laser line irradiation area on the obstacle in the target image from the target image includes:
determining the area occupied by the laser line imaging in the target image;
acquiring imaging position information of a plurality of pixel points in the region in the target image;
the step of determining the actual position information of the obstacle according to the imaging position information includes:
and determining the actual position information of the obstacle according to the imaging position information of a plurality of pixel points on the laser line in the target image.
Optionally, the step of determining the actual position information of the obstacle according to the imaging position information of the plurality of pixel points on the laser line in the target image includes:
determining a sight line equation corresponding to the multiple pixel points on the laser line according to imaging position information of the multiple pixel points in the target image and preset parameters of the image acquisition equipment;
and determining the actual position information of the obstacle according to the sight line equation and a light plane equation of a laser plane of the laser under a coordinate system corresponding to the image acquisition equipment.
Optionally, the optical axis of the image capturing device and the laser plane of the laser intersect in a region right in front of the image capturing device.
Optionally, the image acquisition device is placed vertically below or vertically above the laser;
or the image acquisition equipment and the laser are vertically arranged in a preset deviation manner, wherein the preset deviation is positively correlated with the emission angle of the laser.
Optionally, the laser and the image acquisition device are mounted on a robot.
Optionally, the direction of the line laser emitted by the laser is consistent with the moving direction of the robot; or,
the image acquisition equipment is horizontally arranged, and an acquisition area of the image acquisition equipment is an area right in front of the robot.
According to the obstacle information acquisition device and method provided by the embodiment of the invention, the target image can be acquired through the image acquisition equipment, and the target image comprises the image of the irradiation area of the line laser on the obstacle; acquiring imaging position information of laser line imaging of a line laser irradiation area on an obstacle in a target image from the target image; and then determining the actual position information of the obstacle according to the imaging position information. Compared with the known obstacle position information determined by the distance of a certain point, the obstacle information acquiring device and the obstacle information acquiring method provided by the embodiment of the invention can finally determine the actual position information of the obstacle according to the imaging position information of the laser line imaged in the target image, and the actual position information of the obstacle determined by the imaging position information of the laser line imaged in the target image is more accurate compared with the position information correspondingly determined by the certain point. Therefore, the obstacle information acquisition device and the obstacle information acquisition method provided by the embodiment of the invention can improve the accuracy of determining the position of the obstacle. And then can realize that the robot keeps away the barrier accurately in the in-process of advancing. Of course, it is not necessary for any product or method of practicing the invention to achieve all of the above-described advantages at the same time.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, it is obvious that the drawings in the following description are only some embodiments of the present invention, and for those skilled in the art, other drawings can be obtained according to the drawings without creative efforts.
Fig. 1 is a schematic structural diagram of an obstacle information acquiring apparatus according to an embodiment of the present invention;
FIG. 2(a) is a schematic diagram of a position of a laser line imaged on a target image according to an embodiment of the present invention;
FIG. 2(b) is a schematic diagram of another position of the laser line imaged on the target image in the embodiment of the present invention;
FIG. 2(c) is a schematic diagram of another position of a laser line imaged on a target image according to an embodiment of the present invention;
FIG. 2(d) is a schematic diagram of another position of a laser line imaged on a target image according to an embodiment of the present invention;
FIG. 3 is a schematic diagram of the positions of multiple laser lines imaged in a target image according to an embodiment of the present invention;
FIG. 4 is a flow chart of a process for extracting connected regions of an image;
fig. 5(a) is a schematic top view of an image capturing device and a laser in an obstacle information acquiring apparatus according to an embodiment of the present invention;
fig. 5(b) is a schematic side view of an image capturing device and a laser in the obstacle information acquiring apparatus according to the embodiment of the present invention;
FIG. 6 is a schematic structural diagram of an image capturing device located at the lower left of a laser according to an embodiment of the present invention;
FIG. 7 is a schematic structural diagram of an image capturing device located vertically directly below a laser according to an embodiment of the present invention;
FIG. 8 is a schematic diagram of an embodiment of the present invention in which the image capturing device is located at the lower right of the laser;
fig. 9 is an installation manner of the obstacle information acquiring apparatus according to the embodiment of the present invention;
fig. 10 is another installation manner of the obstacle information acquiring apparatus according to the embodiment of the present invention;
fig. 11 is another installation manner of the obstacle information acquiring apparatus according to the embodiment of the present invention;
fig. 12 is another installation manner of the obstacle information acquiring apparatus according to the embodiment of the present invention;
fig. 13 is a flowchart of an obstacle information obtaining method according to an embodiment of the present invention.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
The obstacle information acquisition device provided by the embodiment of the invention can be applied to the obstacle avoidance process of the robot. In order to enable the robot to accurately avoid obstacles, it is an important condition to accurately determine the position of an obstacle. The obstacle information acquiring device provided by the embodiment of the invention can accurately determine the position information of the obstacle, and is described in detail below.
Fig. 1 is a schematic structural diagram of an obstacle information acquiring apparatus according to an embodiment of the present invention, and a detailed description is given to the obstacle information acquiring apparatus according to the embodiment of the present invention with reference to fig. 1, where the apparatus includes:
and alaser 101 for emitting line laser light.
And theimage acquisition device 102 is used for acquiring a target image, and the target image comprises imaging of the irradiation area of the line laser on the obstacle. It should be noted that, in the obstacle information acquiring apparatus provided in the embodiment of the present invention, the relative positional relationship between thelaser 101 and theimage capturing device 102 is fixed.
The acquiringmodule 103 is configured to acquire imaging position information of a laser line imaging in the target image, where the line laser irradiates an area on the obstacle, from the target image.
And the determiningmodule 104 is used for determining the actual position information of the obstacle according to the imaging position information.
The actual position information of the obstacle generally refers to a distance of the obstacle with respect to the camera or the laser.
Fig. 2(a), fig. 2(b), fig. 2(c) and fig. 2(d) are schematic diagrams illustrating different positions of imaging of different laser lines on a target image according to an embodiment of the present invention, wherein fig. 2(a) is a schematic diagram illustrating an imaging position of a laser line of a line laser light irradiation area of a line laser light generated by alaser 101 on an obstacle located at a distance of 30cm from a robot in the target image; fig. 2(b) is a schematic diagram showing the imaging position of the laser line imaging of the line laser light generated by thelaser 101 in the target image in the area irradiated by the line laser light on the obstacle at a distance of 90cm from the robot; fig. 2(c) is a schematic diagram showing the position of the line laser generated by thelaser 101 in the target image, wherein the line laser irradiates the area on the obstacle 150cm away from the robot; fig. 2(d) is a schematic diagram showing the position of the line laser generated by thelaser 101 in the target image, where the laser line of the irradiation area on the obstacle at a distance of 210cm from the robot is imaged.
As can be seen from fig. 2(a), 2(b), 2(c) and 2(d), when the line laser generated by thelaser 101 is irradiated on an obstacle at a different position from the robot, theimage acquisition device 102 acquires that the laser line of the irradiation area of the line laser on the obstacle is imaged in different imaging position information in the target image. It should be noted that, in the practical application process, the imaging position information of the laser line imaged in the target image is related to the distance of the obstacle, and also related to the parameters of the obstacle information acquiring device, such as the included angle between theimage acquisition device 102 and thelaser 101. Therefore, according to different imaging position information of the laser line of the line laser irradiated area on the obstacle, which is imaged in the target image, emitted by thelaser 101, the actual position information of the obstacle at different distances from the robot can be determined.
The obstacle information acquiring device provided by the embodiment of the invention utilizes thelaser 101 to emit the line laser, and further prepares for determining the actual position information of different obstacles according to the different imaging position information of the laser line imaging in the target image of the line laser emitted by thelaser 101 in the obstacle irradiation area.
In the embodiment of the invention, line laser emitted by alaser 101 irradiates on an obstacle encountered by a robot in the process of moving; theimage acquisition equipment 102 acquires an imaged target image containing an irradiation area of the line laser on the obstacle, the laser line in the target image acquired by theimage acquisition equipment 102 indicates that the obstacle exists, and the laser lines in the acquired target image indicate that the obstacle exists.
Theimage acquisition device 102 may acquire an imaged target image of an area irradiated by the line laser on the obstacle, and provide conditions for determining actual position information of the obstacle according to imaging position information of the line laser imaged in the target image. The image capturing device may be any device that can capture an image, such as a camera or a video camera.
After the target image is acquired, imaging position information of laser line imaging of a line laser irradiating area on the obstacle in the target image can be determined. For example, the area where the laser line is imaged in the target image may be determined first, then the imaging position information of the plurality of pixel points on the laser line in the target image in the area where the laser line is located is determined, and the imaging position information of the plurality of pixel points on the laser line in the target image is used as the imaging position information of the laser line. And then determining the actual position information of the obstacle according to the imaging position information of the plurality of pixel points of the laser line in the target image.
It should be noted that, after the image acquisition device acquires the target image including the image of the area irradiated by the line laser on the obstacle, the image acquisition device may determine the imaging position information of the laser line image of the area irradiated by the line laser on the obstacle in the target image through its own processing function, and further determine the actual position information of the obstacle. The target image acquired by the image acquisition device can also be processed by a processor independent of the image acquisition device, that is, the imaging position information of the laser line imaged in the target image is determined by theacquisition module 103, and then the actual position information of the obstacle is determined by the imaging position information of the laser line imaged in the target image.
It can be seen that, in the obstacle information acquiring apparatus provided in the embodiment of the present invention, the actual position information of the obstacle finally determined according to the imaging position information of the laser line imaged in the target image is the actual position information of the obstacle composed of the plurality of position information determined according to the imaging position information of the plurality of pixel points on the laser line imaged in the target image, and the position information determined by the plurality of pixel points is more accurate than the position information determined by a certain point. Therefore, the obstacle information acquisition device provided by the embodiment of the invention can improve the accuracy of determining the obstacle position information.
In addition, the actual position information of the obstacle is finally determined by utilizing the imaging position information of the line laser emitted by the laser device in the target image, the cost is greatly reduced compared with a laser radar and the like, and the obstacle avoidance sensing method is not easily influenced by environmental factors such as illumination and the like and is more stable compared with a binocular vision obstacle avoidance sensing method.
Fig. 3 is a schematic diagram of positions of multiple laser lines imaged in a target image in the embodiment of the present invention, and a position of a white strip line in the target image in fig. 3 is a position of the laser line imaged in the target image. The process of theimage capturing device 102 finally determining the actual position information of the obstacle through the imaging position information of the laser line in the target image according to the embodiment of the present invention will be described in detail with reference to fig. 3. Specifically, the method comprises the following steps:
firstly, determining the area occupied by laser line imaging in a target image.
If only the laser line appears in the target image, the obstacle exists, otherwise, the target image is completely black. In addition, during the traveling of the robot, one obstacle or a plurality of obstacles may be encountered. According to the embodiment of the invention, the area of the laser line imaged in the target image is determined by determining the connected area in the target image. And if the target image has several connected regions, several obstacles are determined.
Specifically, a connected region labeling method based on region growing may be adopted to extract a connected region in the target image, as shown in fig. 4. Inputting an image to be marked, namely a target image, initializing a marking matrix with the same size as the input image, a queue and a marking count; then, scanning an image to be marked from left to right and from top to bottom, when scanning an unmarked foreground pixel p, adding 1 to the mark count, marking the p in the mark count (the value of the corresponding point is marked as the mark count), simultaneously scanning eight neighborhood points of the p, if the unmarked foreground pixel exists, marking in the mark matrix, and putting in a queue as a seed for region growth; and when the queue is not empty, taking a growth seed point p1 out of the queue, scanning eight neighborhood points of p1, if the foreground pixels which are not marked exist, marking the image to be marked, putting the image into the queue, and repeating until the queue is empty and a connected region is marked. And according to the same steps, obtaining the number of the mark matrix and the number of the connected areas until the whole image is scanned.
And secondly, determining imaging position information of a plurality of pixel points in the target image in the area, and taking the imaging position information of the plurality of pixel points in the target image as the imaging position information of the laser line imaged in the target image.
It is understood that the determined connected region is composed of a plurality of pixel points, and the imaging position information of the plurality of pixel points in the target image is the coordinate positions of the pixel points in the target image. The imaging position information of the pixel points is combined to be used as the imaging position information of the laser line in the target image.
And thirdly, determining the actual position information of the obstacle according to the imaging position information of a plurality of pixel points on the laser line in the target image.
After the imaging position information of the laser line imaging in the target image is determined, the actual position information of the obstacle can be determined according to the imaging position information of the laser line imaging in the target image, namely the actual position information of the obstacle is determined according to the imaging position information of a plurality of pixel points on the laser line in the target image.
Specifically, a sight line equation corresponding to a plurality of pixel points on the laser line can be determined according to imaging position information of the plurality of pixel points on the laser line in the target image and preset parameters of theimage acquisition device 102; and further, according to a sight line equation corresponding to the multiple pixel points on the laser line and a light plane equation of the laser plane in a coordinate system corresponding to theimage acquisition device 102, determining the actual position information of the obstacle. The following steps are detailed to determine the actual position information of the obstacle according to the imaging position information of a plurality of pixel points on the laser line in the target image. Theimage capturing device 102 here employs a commonly used camera.
Assuming the image coordinates (u, v) of any pixel point on the laser line in the target image, the internal parameters of the camera (the focal length f of the camera in the x and y directions) are knownxAnd fyAnd center of image (u)0,v0) View equation is then:
wherein O is (000)TIs the origin of the camera coordinate system and λ is the direction vector.
The equation for the known laser plane in the camera coordinate system is:
AX+BY+Z+C＝O
a, B, C are equation parameters, respectively, according to actual conditions.
The intersection point of the sight line equation and the light plane equation is obtained as follows:
when the motion direction of the robot is horizontal forward, if the camera is horizontal forward, XCI.e. the position of the obstacle. If the direction of the line laser emitted by thelaser 101 is horizontal forward and the external parameters of the camera, i.e. the rotation R and translation t of the camera coordinate system with respect to the light plane coordinate system corresponding to the laser plane, are known, the intersection point can be converted to the light plane coordinate system as follows:
Xw＝RXC+t，
then the calculated XwI.e. the position of the obstacle.
It should be noted that, according to the description of the above steps, an X may be determined according to the correspondence between each pixel pointCOr XwThe method and the device determine the actual position information of the obstacle according to the imaging position information of a plurality of pixel points on the laser line in the target image, and the determined actual position information of the obstacle is a plurality of XCOr a plurality of XwThe composed location information. That is, the finally determined actual position information of the obstacle may also be understood as position information composed of a "point cloud".
As an implementation manner of the embodiment of the present invention, in order to ensure that the image capturing device can capture the target image including the image of the irradiation area of the line laser on the obstacle, the image capturing device and the laser may have a certain angle. The specific angle can be adjusted according to the actually measured distance range of the obstacle, and the larger the angle is, the smaller the measured distance is; conversely, the smaller the angle, the larger the measurement distance.
Fig. 5(a) is a schematic top view of an image capturing device and a laser in an obstacle information acquiring apparatus according to an embodiment of the present invention; fig. 5(b) is a schematic side view of an image capturing device and a laser in the obstacle information acquiring apparatus according to the embodiment of the present invention. As shown in FIGS. 5(a) and 5(b)In the obstacle information acquiring apparatus according to the embodiment of the present invention, the optical axis of theimage capturing device 102 and the laser plane of thelaser 101 intersect in the area directly in front of theimage capturing device 102. Specifically, the angle between theimage capturing device 102 and thelaser 101 may be determined according to the distance range of the obstacle to be measured. Wherein, by ZCRepresenting the optical axis, Z, of the image-capturingdevice 102CAnd YCForming a coordinate system of the image-capturing device by Xw、YwAnd ZwThe coordinate system formed represents the laser plane generated by thelaser 101.
The optical axis of theimage acquisition device 102 and the laser plane of thelaser 101 intersect in the area right in front of theimage acquisition device 102, so that theimage acquisition device 102 can acquire the imaging target image of the line laser in the irradiation area of the obstacle, and further the actual position information of the obstacle is finally determined through the imaging position information of the line laser emitted by thelaser 101 and the laser line in the irradiation area of the obstacle in the target image.
In addition, the position of theimage acquisition device 102 in the direction perpendicular to thelaser 101 is not limited in the embodiment of the present invention, and theimage acquisition device 102 may be located vertically below or vertically above thelaser 101; alternatively, theimage capturing device 102 may allow a preset deviation from the perpendicular direction of thelaser 101, wherein the preset deviation is positively correlated with the emission angle of thelaser 101.
For example, theimage capture device 102 may be located at the lower left, lower right, vertically below thelaser 101, or the like. FIG. 6 is a schematic structural diagram of an image capturing device located at the lower left of a laser according to an embodiment of the present invention; FIG. 7 is a schematic structural diagram of an image capturing device located vertically directly below a laser according to an embodiment of the present invention; fig. 8 is a schematic structural diagram of the image capturing device located at the lower right of the laser in the embodiment of the present invention, and it should be noted that theimage capturing device 102 and thelaser 101 may be mounted on therobot 104 according to the above positional relationship.
The obstacle information acquisition device provided by the embodiment of the invention can be applied to the robot obstacle avoidance environment, so that the obstacle information acquisition device can be installed on the robot. Specifically, the laser and the image acquisition device may be mounted separately on the robot; or: the laser and the image acquisition equipment can be integrated and then installed on the robot.
The robot has different moving directions in the moving process, and the image acquisition equipment or the laser in the obstacle information acquisition device provided by the embodiment of the invention can have different position relations with the robots in different moving directions.
When the obstacle information acquisition device provided by the embodiment of the invention is installed on a robot, the direction of the laser of the line emitted by the laser can be consistent with the movement direction of the robot; or the image acquisition equipment can be horizontally arranged, and the acquisition area of the image acquisition equipment is the area right in front of the robot. Specifically, the direction of the line laser emitted by the laser may be the same as the moving direction of the robot, and the laser plane of the laser and the optical axis of the image acquisition device intersect in the area directly in front of the image acquisition device, and are located vertically above, above left, above right, vertically below, below left, below right, or the like of the image acquisition device; or the image acquisition device can be horizontally placed, the acquisition area of the image acquisition device is an area right in front of the robot, the optical axis of the image acquisition device and the laser plane of the laser intersect in the area right in front of the image acquisition device, and the image acquisition device is positioned above, above left, above right, below left, below right, and the like of the laser.
For example, fig. 9 is an installation manner of the obstacle information acquiring apparatus according to the embodiment of the present invention; fig. 10 is another installation manner of the obstacle information acquiring apparatus according to the embodiment of the present invention; fig. 11 is another installation manner of the obstacle information acquiring apparatus according to the embodiment of the present invention; fig. 12 is another installation manner of the obstacle information acquiring apparatus according to the embodiment of the present invention. Thelaser 101 and theimage capturing device 102 may be mounted on therobot 104 according to any of the above-mentioned mounting manners, wherein the moving direction of therobot 104 in fig. 9, 10, 11 and 12 is shown by anarrow 105, and the laser plane corresponding to the line laser generated by thelaser 101 is shown by astraight line 106.
The embodiment of the invention also provides an obstacle information acquisition method, which is applied to the obstacle information acquisition device. The obstacle information acquiring apparatus includes: a laser and an image acquisition device; wherein, the image acquisition equipment and the laser have a fixed relative position relationship. Fig. 13 is a flowchart of an obstacle information obtaining method according to an embodiment of the present invention, and details of the obstacle information obtaining method according to the embodiment of the present invention are described with reference to fig. 13, where the method includes:
The obstacle information acquisition method provided by the embodiment of the invention can be applied to an obstacle information acquisition device, image acquisition equipment, a processor and the like.
In the embodiment of the invention, the image acquisition device can acquire the imaged target image containing the irradiation area of the line laser on the obstacle, and provides conditions for determining the actual position information of the obstacle by the imaging position information of the line laser imaged in the target image. The image capturing device may be any device that can capture an image, such as a camera or a video camera.
After the target image is acquired, imaging position information of the laser line imaged in the target image can be determined. Because the laser line imaging is composed of a plurality of pixel points in the target image, the actual position information of the obstacle is determined by determining the imaging position information of the plurality of pixel points on the laser line in the target image.
Specifically, the step of determining imaging position information of the laser line imaging of the line laser irradiation area on the obstacle in the target image may include: determining the area occupied by laser line imaging in the target image; and determining imaging position information of a plurality of pixel points in the target image in the area, and taking the imaging position information of the plurality of pixel points in the target image as position information of laser line imaging in the target image.
And step 1303, determining the actual position information of the obstacle according to the imaging position information of the laser line imaged in the target image.
When line laser generated by the laser irradiates on obstacles at different positions away from the robot, the image acquisition equipment acquires that the imaging positions of the laser line in the target image are different, so that the position information of the obstacles at different distances away from the robot can be determined according to the different imaging position information of the laser line in the target image in the area irradiated by the line laser on the obstacles.
For example, the area where the laser line is imaged in the target image may be determined first, then the imaging position information of the plurality of pixel points on the laser line in the target image in the area where the laser line is located is determined, and the imaging position information of the plurality of pixel points on the laser line in the target image is used as the imaging position information of the laser line. And then determining the actual position information of the obstacle according to the imaging position information of a plurality of pixel points on the laser line in the target image.
According to the obstacle information acquisition method provided by the embodiment of the invention, the actual position information of the obstacle is determined by determining the imaging position information of the laser line imaged in the target image. It can be seen that the obstacle information finally determined by the obstacle information obtaining method provided by the embodiment of the invention is actual position information of the obstacle composed of a plurality of pieces of position information determined according to imaging position information of a plurality of pixel points on the laser line in the target image, and the position information determined by the plurality of pixel points correspondingly is more accurate than the position information determined by a certain point correspondingly. Therefore, the obstacle information acquisition method provided by the embodiment of the invention can improve the accuracy of determining the obstacle position information.
After a target image containing the imaging of the line laser irradiation area on the obstacle is acquired, the imaging position information of the line laser imaging in the target image is determined. In an implementation manner of the embodiment of the invention, the imaging position information of a plurality of pixel points in the target image in the area is determined by determining the area where the laser line is imaged in the target image; and finally, determining the actual position information of the obstacle according to the imaging position information of a plurality of pixel points on the laser line in the target image. The method specifically comprises the following steps:
firstly, determining the area occupied by laser line imaging in a target image.
If only the laser line appears in the target image, the obstacle exists, otherwise, the target image is completely black. In addition, during the traveling of the robot, one obstacle or a plurality of obstacles may be encountered. According to the embodiment of the invention, the area of the laser line imaged in the target image is determined by determining the connected area in the target image. And if the target image has several connected regions, several obstacles are determined. The method for specifically extracting the connected region has been described in detail in the embodiment of the obstacle information acquiring apparatus, and is not described herein again.
And secondly, determining imaging position information of a plurality of pixel points in the target image in the area, and taking the position information of the plurality of pixel points in the target image as the imaging position information of the laser line imaged in the target image.
It is understood that the determined connected region is composed of a plurality of pixel points, and the imaging position information of the plurality of pixel points in the target image is the coordinate positions of the pixel points in the target image. The imaging position information of the pixel points is combined to be used as the imaging position information of the laser line in the target image.
And thirdly, determining the actual position information of the obstacle according to the imaging position information of a plurality of pixel points on the laser line in the target image.
After the imaging position information of the laser line imaging in the target image is determined, the actual position information of the obstacle can be determined according to the imaging position information of the laser line imaging in the target image, namely the actual position information of the obstacle is determined according to the imaging position information of a plurality of pixel points on the laser line in the target image.
Specifically, a sight line equation corresponding to a plurality of pixel points on the laser line can be determined according to imaging position information of the plurality of pixel points on the laser line in the target image and preset parameters of the image acquisition equipment; and further determining the actual position information of the obstacle according to a sight line equation corresponding to the multiple pixel points on the laser line and a light plane equation of the laser plane of the laser under a coordinate system corresponding to the image acquisition equipment. Specifically, the process of determining the actual position information of the obstacle according to the imaging position information of the plurality of pixel points on the laser line in the target image is described in detail in the embodiment of the obstacle information acquiring device, and is not described here again.
Optionally, the optical axis of the image capturing device intersects the laser plane of the laser in a region directly in front of the image capturing device.
Optionally, the image acquisition device in the obstacle information acquisition device is placed vertically below or vertically above the laser; or the image acquisition equipment and the laser are vertically arranged in a preset deviation, wherein the preset deviation is positively correlated with the emission angle of the laser.
Optionally, the laser and the image acquisition device are mounted on the robot.
Optionally, the direction of the laser of the line emitted by the laser is consistent with the movement direction of the robot; or the image acquisition equipment is horizontally arranged, and the acquisition area of the image acquisition equipment is the area right in front of the robot.
It should be noted that, the obstacle information obtaining method provided by the embodiment of the present invention is applied to an obstacle information obtaining apparatus, and all embodiments of the obstacle information obtaining apparatus are applicable to the obstacle information obtaining method and can achieve the same or similar beneficial effects.
In addition, an embodiment of the present invention further provides a computer-readable storage medium, where a computer program is stored in the computer-readable storage medium, and when the computer program is executed by a processor, the steps of the obstacle information acquiring method are implemented.
In the embodiment of the invention, the imaging target image containing the line laser irradiation area on the obstacle can be acquired through the image acquisition equipment, and then the actual position information of the obstacle can be determined by determining the imaging position information of the laser line imaging in the target image. Compared with the known method for determining the actual position information of the obstacle through the distance of a certain point, the method and the device for determining the actual position information of the obstacle can finally determine the actual position information of the obstacle according to the imaging position information of the laser line in the target image, the laser line comprises a plurality of pixel points, and the position information determined by the plurality of pixel points correspondingly is more accurate compared with the position information determined by the certain point correspondingly. The accuracy of determining the position of the obstacle can be improved through the embodiment of the invention. And then can realize that the robot keeps away the barrier accurately in the in-process of advancing.
It is noted that, herein, relational terms such as first and second, and the like may be used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Also, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other identical elements in a process, method, article, or apparatus that comprises the element.
All the embodiments in the present specification are described in a related manner, and the same and similar parts among the embodiments may be referred to each other, and each embodiment focuses on the differences from the other embodiments. In particular, for the system embodiment, since it is substantially similar to the method embodiment, the description is simple, and for the relevant points, reference may be made to the partial description of the method embodiment.
The above description is only for the preferred embodiment of the present invention, and is not intended to limit the scope of the present invention. Any modification, equivalent replacement, or improvement made within the spirit and principle of the present invention shall fall within the protection scope of the present invention.
Claims (10)
1. An obstacle information acquiring apparatus, characterized by comprising:
a laser for emitting line laser light;
the system comprises an image acquisition device and a laser device, wherein the image acquisition device is used for acquiring a target image, the target image comprises the imaging of the irradiation area of the line laser on at least two obstacles, and a fixed relative position relationship is formed between the image acquisition device and the laser device, an included angle exists between the image acquisition direction of the image acquisition device and the laser emission direction of the laser device, and the larger the included angle is, the smaller the measurement range is, and the larger the measurement range is;
the acquisition module is used for determining an area where the laser line is imaged in the target image and acquiring imaging position information of a plurality of pixel points in the area in the target image;
the determining module is used for determining a sight line equation corresponding to a plurality of pixel points on the laser line according to imaging position information of the pixel points on the laser line in the target image and preset parameters of the image acquisition equipment; according to the sight line equation corresponding to the multiple pixel points on the laser line and the light plane equation of the laser plane of the laser under the coordinate system corresponding to the image acquisition equipment, aiming at the sight line equation corresponding to each pixel point, the intersection point of the sight line equation and the light plane equation is calculated to obtain the position information of the intersection point, and the actual position information of the obstacle is determined according to the position information formed by the intersection points.
2. The apparatus of claim 1, wherein an optical axis of the image capture device intersects a laser plane of the laser at a region directly in front of the image capture device.
3. The apparatus of claim 2, wherein the image capture device is located directly vertically below or directly vertically above the laser;
or the image acquisition equipment and the vertical direction of the laser have preset deviation, wherein the preset deviation is positively correlated with the emission angle of the laser.
4. The apparatus of claim 3, wherein the laser and the image capture device are mounted on a robot.
5. The device of claim 4, wherein the direction of the line laser emitted by the laser is consistent with the moving direction of the robot; or,
the image acquisition equipment is horizontally arranged, and an acquisition area of the image acquisition equipment is an area right in front of the robot.
6. An obstacle information acquisition method applied to an obstacle information acquisition apparatus, the obstacle information acquisition apparatus comprising: a laser and an image acquisition device; wherein the image acquisition equipment and the laser have a fixed relative position relationship; the method comprises the following steps:
acquiring a target image, wherein the target image comprises an image of a line laser emitted by the laser device in an irradiation area on at least two obstacles, an included angle exists between an image acquisition direction of the image acquisition device and a laser emission direction of the laser device, and the larger the included angle is, the smaller the measurement range is, the smaller the included angle is, and the larger the measurement range is;
determining the area of the laser line imaging in the target image;
acquiring imaging position information of a plurality of pixel points in the region in the target image;
determining a sight line equation corresponding to the multiple pixel points on the laser line according to imaging position information of the multiple pixel points in the target image and preset parameters of the image acquisition equipment;
according to the sight line equation and a light plane equation of a laser plane of the laser under a coordinate system corresponding to the image acquisition equipment, aiming at the sight line equation corresponding to each pixel point, calculating an intersection point of the sight line equation and the light plane equation to obtain position information of the intersection point, and determining actual position information of the obstacle according to the position information formed by the intersection points.
7. The method of claim 6, wherein an optical axis of the image capture device intersects a laser plane of the laser at a region directly in front of the image capture device.
8. The method of claim 7, wherein the image capture device is placed directly vertically below or directly vertically above the laser;
or the image acquisition equipment and the laser are vertically arranged in a preset deviation manner, wherein the preset deviation is positively correlated with the emission angle of the laser.
9. The method of claim 8, wherein the laser and the image capture device are mounted on a robot.
10. The method of claim 9, wherein the direction of the line laser emitted by the laser coincides with the direction of motion of the robot; or,
the image acquisition equipment is horizontally arranged, and an acquisition area of the image acquisition equipment is an area right in front of the robot.
Priority Applications (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710509194.1ACN109143167B (en) | 2017-06-28 | 2017-06-28 | Obstacle information acquisition device and method |
| PCT/CN2018/078034WO2019001001A1 (en) | 2017-06-28 | 2018-03-05 | Obstacle information acquisition apparatus and method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710509194.1ACN109143167B (en) | 2017-06-28 | 2017-06-28 | Obstacle information acquisition device and method |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN109143167A CN109143167A (en) | 2019-01-04 |
| CN109143167Btrue CN109143167B (en) | 2021-07-23 |
Family
ID=64740334
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201710509194.1AActiveCN109143167B (en) | 2017-06-28 | 2017-06-28 | Obstacle information acquisition device and method |
Country Status (2)
| Country | Link |
|---|---|
| CN (1) | CN109143167B (en) |
| WO (1) | WO2019001001A1 (en) |
Families Citing this family (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109885053B (en) | 2019-02-28 | 2024-09-17 | 深圳市道通智能航空技术股份有限公司 | Obstacle detection method and device and unmanned aerial vehicle |
| CN109940612A (en)* | 2019-03-04 | 2019-06-28 | 东北师范大学 | Intelligent obstacle avoidance robot based on one-line laser and its obstacle avoidance method |
| CN109946703B (en)* | 2019-04-10 | 2021-09-28 | 北京小马智行科技有限公司 | Sensor attitude adjusting method and device |
| US11942546B2 (en) | 2020-12-03 | 2024-03-26 | Taiwan Semiconductor Manufacturing Company, Ltd. | Semiconductor device and method for forming the same |
| CN113247745B (en)* | 2021-07-12 | 2021-09-28 | 深圳市爱深盈通信息技术有限公司 | Elevator door control method based on image and anti-pinch detection module |
| CN114445494B (en)* | 2022-01-11 | 2025-10-03 | 珠海一微半导体股份有限公司 | Image acquisition and processing method, image acquisition device and robot |
Family Cites Families (17)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US6496754B2 (en)* | 2000-11-17 | 2002-12-17 | Samsung Kwangju Electronics Co., Ltd. | Mobile robot and course adjusting method thereof |
| KR20050083341A (en)* | 2004-02-23 | 2005-08-26 | (주)다사테크 | Surrounding recognition sensor device using laser beam line |
| KR100791389B1 (en)* | 2006-12-26 | 2008-01-07 | 삼성전자주식회사 | Distance measuring device and method using structured light |
| CN101968353B (en)* | 2010-09-29 | 2012-02-08 | 清华大学 | Terrain Tracking Method for Unmanned Helicopter Based on Laser Detection and Image Recognition |
| CN102023003A (en)* | 2010-09-29 | 2011-04-20 | 清华大学 | Unmanned helicopter three-dimensional positioning and mapping method based on laser detection and image recognition |
| CN101968354A (en)* | 2010-09-29 | 2011-02-09 | 清华大学 | Laser detection and image identification based unmanned helicopter distance measuring method |
| CN102354400A (en)* | 2011-07-14 | 2012-02-15 | 中国人民解放军国防科学技术大学 | Multi-point-constraint monocular trajectory intersection method of rigid body motion and structure parameter measurement |
| CN102636152B (en)* | 2012-04-19 | 2013-12-18 | 慈溪思达电子科技有限公司 | Active visual ranging system of movable platform |
| KR101949277B1 (en)* | 2012-06-18 | 2019-04-25 | 엘지전자 주식회사 | Autonomous mobile robot |
| ES2610755T3 (en)* | 2012-08-27 | 2017-05-03 | Aktiebolaget Electrolux | Robot positioning system |
| CN102794763B (en)* | 2012-08-31 | 2014-09-24 | 江南大学 | Calibration method of welding robot system based on line structured light vision sensor guidance |
| CN104236521A (en)* | 2013-06-14 | 2014-12-24 | 科沃斯机器人科技(苏州)有限公司 | Line-laser ranging method applied to auto-moving robots |
| CN103914871B (en)* | 2014-03-06 | 2016-06-08 | 河南农业大学 | The method of body surface coordinate point is chosen based on the interactive mode of cloud data |
| CN103942830B (en)* | 2014-04-04 | 2016-08-17 | 浙江大学 | Directly utilize and there is the method that the phase place of nonlinearity erron realizes scene three-dimensional reconstruction |
| CN205581299U (en)* | 2015-05-25 | 2016-09-14 | 北京雷动云合智能技术有限公司 | High accuracy camera chip multiple spot range unit that dual laser markd |
| CN104859562A (en)* | 2015-05-27 | 2015-08-26 | 北京信息科技大学 | Method and device for detecting barriers behind vehicle |
| CN106199618A (en)* | 2016-06-28 | 2016-12-07 | 联想(北京)有限公司 | A kind of electronic equipment and localization method |
- 2017
- 2017-06-28CNCN201710509194.1Apatent/CN109143167B/enactiveActive
- 2018
- 2018-03-05WOPCT/CN2018/078034patent/WO2019001001A1/ennot_activeCeased
Also Published As
| Publication number | Publication date |
|---|---|
| CN109143167A (en) | 2019-01-04 |
| WO2019001001A1 (en) | 2019-01-03 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109143167B (en) | Obstacle information acquisition device and method | |
| CN112907676B (en) | Calibration method, device and system of sensor, vehicle, equipment and storage medium | |
| US11436750B2 (en) | Optical tracking system and optical tracking method | |
| US9185402B2 (en) | Traffic camera calibration update utilizing scene analysis | |
| TWI624170B (en) | Image scanning system and method thereof | |
| CN112106111A (en) | Calibration method, calibration equipment, movable platform and storage medium | |
| JP2013101045A (en) | Recognition device and recognition method of three-dimensional position posture of article | |
| JP2006252473A (en) | Obstacle detection device, calibration device, calibration method, and calibration program | |
| CN112184793B (en) | Depth data processing method and device and readable storage medium | |
| JPWO2017051480A1 (en) | Image processing apparatus and image processing method | |
| KR102102291B1 (en) | Optical tracking system and optical tracking method | |
| JP4344860B2 (en) | Road plan area and obstacle detection method using stereo image | |
| KR20240056516A (en) | Method and system for generating camera model for camera calibration | |
| CN111256651B (en) | A method and device for perimeter vehicle ranging based on monocular vehicle camera | |
| CN113378606A (en) | Method, device and system for determining labeling information | |
| KR20200076628A (en) | Location measuring method of mobile device, location measuring device and electronic device | |
| CN112513931A (en) | System and method for creating a single-view composite image | |
| JP6410231B2 (en) | Alignment apparatus, alignment method, and computer program for alignment | |
| JPH08110807A (en) | Method and device for automatic calibration | |
| JP7675557B2 (en) | Positioning System | |
| CN112686963B (en) | Target positioning method of aerial work robot for shielding | |
| CN114596358A (en) | Object detection method and device and electronic equipment | |
| Salau et al. | 2.3. Development of a multi-Kinect-system for gait analysis and measuring body characteristics in dairy cows | |
| JP7117878B2 (en) | processor and program | |
| JP2021105965A (en) | Camera calibration device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| CP03 | Change of name, title or address | Address after:310051 room 304, B / F, building 2, 399 Danfeng Road, Binjiang District, Hangzhou City, Zhejiang Province Patentee after:Hangzhou Hikvision Robot Co.,Ltd. Address before:No.555, Qianmo Road, Binjiang District, Hangzhou City, Zhejiang Province Patentee before:HANGZHOU HIKROBOT TECHNOLOGY Co.,Ltd. | |
| CP03 | Change of name, title or address |