Disclosure of Invention
The technical problem to be solved by the invention is to overcome the defects of the prior art and provide a monocular SLAM method capable of creating a large-scale map.
In order to solve the technical problems, the invention provides the following technical scheme:
the invention relates to a monocular SLAM method capable of creating a large-scale map, which specifically comprises the following steps:
a. acquiring upper-end image information of a space needing to be mapped through an image acquisition device to obtain a primary environment image;
b. acquiring image information of the non-upper end of a space needing drawing construction through a 3D laser radar to obtain a secondary environment image;
c. performing data processing on the primary environment image, constructing an initial environment image and identifying the pose of the image acquisition device;
d. transforming the pose of the image acquisition device by using the inherent pose transformation relation between the image acquisition device and the 3D laser radar to obtain the pose of the 3D laser radar;
e. transforming the secondary environment image by using the pose of the 3D laser radar to obtain a pose-matched secondary environment image;
f. and (3) carrying out monocular SLAM mapping on the secondary environment image with the matched pose and the initial SLAM map by combining the primary environment image obtained by the image acquisition device to obtain a final environment image.
As a preferred technical scheme of the invention, the image acquisition device comprises a TOF depth camera, a binocular stereo camera, a structured light depth camera and an obstacle avoidance camera.
As a preferred technical solution of the present invention, the primary environment image is a visible light image, and the secondary environment image is a depth image and a visible light image.
As a preferred technical solution of the present invention, the construction of the pose of the initial environment map and the image capturing device mainly includes the following steps:
g. acquiring two initial images respectively shot by any two image acquisition devices, wherein the viewing ranges of the two image acquisition devices are at least partially overlapped;
h. and determining the three-dimensional space positions of the mutually matched characteristic points according to the internal parameters and the external parameters which are calibrated in advance by the two image acquisition devices and the parallaxes of the mutually matched characteristic points in the two initial images so as to obtain map points corresponding to the mutually matched characteristic points, and constructing an initial SLAM map so as to complete initialization.
As a preferred technical solution of the present invention, the acquiring two initial images respectively captured by any two image capturing devices includes: acquiring an image A and an image B which are obtained by respectively shooting by two image acquisition devices; wherein, the focal lengths of the two image acquisition devices are different; the image a and the image B are processed in accordance with the image effect obtained by shooting with the same focal length, with the processed image a and image B as two initial images.
As a preferred embodiment of the present invention, the monocular SLAM is constructed by projecting a point p on the reference frame onto a ray of the current frame, the line is also called epipolar line, the process of determining p on the epipolar line is called epipolar line search, after the epipolar line search is used, the depth of p can be determined by using triangulation, and the depth is updated by using a filter.
As a preferred technical scheme of the invention, the epipolar line search specifically comprises the step of constructing two three-dimensional points p on a depth extension line of the seed p (x, y)1And p2The two three-dimensional points originate from the same pixel, but are determined to be different in depth. In depth filters typically set to p1(x, y, z-n sigma) and p2(x, y, z + n sigma), where z is the initial depth of the seed, sigma is the variance of the depth, and n can be adjusted to different values, such as 1, 2, 3.. etc., typically taking 3;
p is to be1 andp2projecting the pose of the frame to the current frame with the projection point u1And u2Is connected to u1And u2And obtaining the polar line.
As a preferred technical solution of the present invention, the filter is a depth filter, a statistical filter, or a voxel filter.
Compared with the prior art, the invention has the following beneficial effects:
1: according to the invention, wide-angle rotation scanning and fixed-point detection are realized through the 3d laser radar, a blind area with a large distance in the detection process is avoided, meanwhile, detailed detection information of an object can be obtained, the information of a reticular barrier and the side environment of the robot is detected, and the comprehensiveness and high resolution of the detection data are ensured.
2: the invention optimizes the positioning precision of monocular SLAM and creates richer map information. The current pose information and the map are calculated through the acquired image based on the SLAM, a depth sensor which is horizontally placed is newly added to acquire the depth information, real-time operation can be realized, more accurate pose can be provided, richer map information is created, and the advantages of the SLAM in positioning navigation and automatic obstacle avoidance are brought into play.
3: according to the invention, two initial images are obtained by respectively shooting through two image acquisition devices, an initial SLAM map is constructed by using the mutually matched characteristic points in the initial images, and after the initialization is successful, the images are shot by using the image acquisition devices to carry out image construction of the monocular SLAM, so that the success rate of image construction is improved, and the information loss in the map is reduced.
Detailed Description
The following description of the preferred embodiments of the present invention is provided for the purpose of illustration and description, and is in no way intended to limit the invention.
Example 1
The invention provides a monocular SLAM method capable of creating a large-scale map, which specifically comprises the following steps:
a. acquiring upper-end image information of a space needing to be mapped through a TOF depth camera, a structured light depth camera and an obstacle avoidance camera to obtain a first-level environment image;
b. acquiring image information of the non-upper end of a space needing drawing construction through a 3D laser radar to obtain a secondary environment image;
c. performing data processing on the primary environment image, constructing an initial environment image, and identifying the poses of a TOF depth camera, a structured light depth camera and an obstacle avoidance camera;
d. transforming the poses of the TOF depth camera, the structured light depth camera and the obstacle avoidance camera by using the inherent pose transformation relation among the TOF depth camera, the structured light depth camera and the obstacle avoidance camera and the 3D laser radar to obtain the pose of the 3D laser radar;
e. transforming the secondary environment image by using the pose of the 3D laser radar to obtain a pose-matched secondary environment image;
f. and (3) establishing a monocular SLAM by combining a primary environment image obtained by a TOF depth camera, a structured light depth camera and an obstacle avoidance camera on the basis of the pose-matched secondary environment image and the initial SLAM map to obtain a final environment image.
Specifically, the establishment of the initial environment image and the poses of the TOF depth camera, the structured light depth camera and the obstacle avoidance camera mainly comprises the following steps:
g. acquiring any two initial images which are respectively obtained by shooting by a TOF depth camera, a structured light depth camera or an obstacle avoidance camera, and acquiring any two initial images which are respectively obtained by shooting by the TOF depth camera, the structured light depth camera or the obstacle avoidance camera comprises: acquiring an image A and an image B which are obtained by respectively shooting two TOF depth cameras, structured light depth cameras or obstacle avoidance cameras; the two TOF depth cameras, the structured light depth camera or the obstacle avoidance camera have different focal lengths; processing the image A and the image B according to an image effect obtained by shooting with the same focal length, taking the processed image A and the processed image B as two initial images, wherein at least part of the view ranges of the two TOF depth cameras, the structured light depth cameras or the obstacle avoidance cameras are overlapped;
h. and determining the three-dimensional space positions of the mutually matched feature points according to the internal parameters and the external parameters which are calibrated in advance by the two TOF depth cameras, the structured light depth cameras or the obstacle avoidance cameras and the parallax of the mutually matched feature points in the two initial images so as to obtain the map points corresponding to the mutually matched feature points, and constructing an initial SLAM map to complete initialization.
Example 2
The invention provides a monocular SLAM method capable of creating a large-scale map, which specifically comprises the following steps:
a. acquiring upper-end image information of a space needing to be mapped through a TOF depth camera and an obstacle avoidance camera to obtain a first-level environment image;
b. acquiring image information of the non-upper end of a space needing drawing construction through a 3D laser radar to obtain a secondary environment image;
c. performing data processing on the primary environment image, constructing an initial environment image, and identifying the poses of a TOF depth camera and an obstacle avoidance camera;
d. transforming the poses of the TOF depth camera and the obstacle avoidance camera by using the inherent pose transformation relation between the TOF depth camera and the obstacle avoidance camera and the 3D laser radar to obtain the pose of the 3D laser radar;
e. transforming the secondary environment image by using the pose of the 3D laser radar to obtain a pose-matched secondary environment image;
f. and (3) carrying out monocular SLAM mapping on the secondary environment image with the matched pose and the initial SLAM map by combining the primary environment image obtained by the TOF depth camera and the obstacle avoidance camera to obtain a final environment image.
Specifically, the construction of the initial environment image and the poses of the TOF depth camera and the obstacle avoidance camera mainly comprises the following steps:
g. acquiring two initial images obtained by respectively shooting any two TOF depth cameras or obstacle avoidance cameras, wherein the acquiring of two initial images obtained by respectively shooting any two TOF depth cameras or obstacle avoidance cameras comprises: acquiring an image A and an image B which are obtained by respectively shooting two TOF depth cameras or obstacle avoidance cameras; the two TOF depth cameras or the obstacle avoidance cameras have different focal lengths; processing the image A and the image B according to an image effect obtained by shooting with the same focal length, taking the processed image A and the processed image B as two initial images, wherein the viewing ranges of the two TOF depth cameras or the obstacle avoidance cameras are at least partially overlapped;
h. and determining the three-dimensional space positions of the mutually matched feature points according to the internal parameters and the external parameters which are calibrated in advance by the two TOF depth cameras or the obstacle avoidance cameras and the parallaxes of the mutually matched feature points in the two initial images so as to obtain map points corresponding to the mutually matched feature points, and constructing an initial SLAM map so as to complete initialization.
Example 3
The invention provides a monocular SLAM method capable of creating a large-scale map, which specifically comprises the following steps:
a. acquiring upper-end image information of a space needing to be mapped through a binocular stereo camera, a structured light depth camera and an obstacle avoidance camera to obtain a primary environment image;
b. acquiring image information of the non-upper end of a space needing drawing construction through a 3D laser radar to obtain a secondary environment image;
c. performing data processing on the primary environment image, constructing an initial environment image, and identifying the poses of a binocular stereo camera, a structured light depth camera and an obstacle avoidance camera;
d. transforming the poses of the binocular stereo camera, the structured light depth camera and the obstacle avoidance camera by using the inherent pose transformation relation among the binocular stereo camera, the structured light depth camera and the obstacle avoidance camera and the 3D laser radar to obtain the pose of the 3D laser radar;
e. transforming the secondary environment image by using the pose of the 3D laser radar to obtain a pose-matched secondary environment image;
f. and (3) establishing a monocular SLAM by combining a primary environment image obtained by a binocular stereo camera, a structured light depth camera and an obstacle avoidance camera on the basis of the pose-matched secondary environment image and the initial SLAM map to obtain a final environment image.
Specifically, the establishment of the initial environment image and the poses of the binocular stereo camera, the structured light depth camera and the obstacle avoidance camera mainly comprises the following steps:
g. the method comprises the following steps of acquiring two initial images obtained by respectively shooting any two binocular stereo cameras, any two structured light depth cameras and or any two obstacle avoidance cameras, and acquiring two initial images obtained by respectively shooting any two binocular stereo cameras, any two structured light depth cameras and any two obstacle avoidance cameras, wherein the two initial images comprise: acquiring an image A and an image B which are obtained by respectively shooting two binocular stereo cameras, a structured light depth camera and/or an obstacle avoidance camera; the two binocular stereo cameras, the structured light depth camera and/or the obstacle avoidance camera have different focal lengths; processing the image A and the image B according to an image effect obtained by shooting with the same focal length, taking the processed image A and the processed image B as two initial images, wherein the viewing ranges of the two binocular stereo cameras, the structured light depth camera and/or the obstacle avoidance camera are at least partially overlapped;
h. according to internal parameters and external parameters which are calibrated in advance by the two binocular stereo cameras, the structured light depth camera and/or the obstacle avoidance camera and the parallax error of the mutually matched feature points in the two initial images, the three-dimensional space positions of the mutually matched feature points are determined so as to obtain map points corresponding to the mutually matched feature points, and an initial SLAM map is constructed to complete initialization.
Example 4
The invention provides a monocular SLAM method capable of creating a large-scale map, which specifically comprises the following steps:
a. acquiring upper-end image information of a space needing to be mapped through a TOF depth camera, a binocular stereo camera and an obstacle avoidance camera to obtain a primary environment image;
b. acquiring image information of the non-upper end of a space needing drawing construction through a 3D laser radar to obtain a secondary environment image;
c. performing data processing on the primary environment image, constructing an initial environment image, and identifying the poses of a TOF depth camera, a binocular stereo camera and an obstacle avoidance camera;
d. transforming the poses of the TOF depth camera, the binocular stereo camera and the obstacle avoidance camera by using the inherent pose transformation relation among the TOF depth camera, the binocular stereo camera and the obstacle avoidance camera and the 3D laser radar to obtain the pose of the 3D laser radar;
e. transforming the secondary environment image by using the pose of the 3D laser radar to obtain a pose-matched secondary environment image;
f. and (3) establishing a monocular SLAM by combining a primary environment image obtained by a TOF depth camera, a binocular stereo camera and an obstacle avoidance camera on the basis of the pose-matched secondary environment image and the initial SLAM map to obtain a final environment image.
Specifically, the establishment of the initial environment image and the poses of the TOF depth camera, the binocular stereo camera and the obstacle avoidance camera mainly comprises the following steps:
g. the method comprises the following steps of acquiring two initial images obtained by respectively shooting by two TOF depth cameras, two binocular stereo cameras or an obstacle avoidance camera, and acquiring two initial images obtained by respectively shooting by two TOF depth cameras, two binocular stereo cameras or an obstacle avoidance camera: acquiring an image A and an image B which are obtained by respectively shooting two TOF depth cameras, binocular stereo cameras or obstacle avoidance cameras; wherein the focal lengths of the two TOF depth cameras, the binocular stereo camera or the obstacle avoidance camera are different; processing the image A and the image B according to an image effect obtained by shooting with the same focal length, taking the processed image A and the processed image B as two initial images, wherein the viewing ranges of the two TOF depth cameras, the binocular stereo camera or the obstacle avoidance camera are at least partially overlapped;
h. and determining the three-dimensional space positions of the mutually matched feature points according to the internal parameters and the external parameters which are calibrated in advance by the two TOF depth cameras, the binocular stereo cameras or the obstacle avoidance cameras and the parallaxes of the mutually matched feature points in the two initial images so as to obtain the map points corresponding to the mutually matched feature points, and constructing an initial SLAM map to complete initialization.
Example 5
The invention provides a monocular SLAM method capable of creating a large-scale map, which specifically comprises the following steps:
a. acquiring upper-end image information of a space needing to be mapped through a TOF depth camera, a binocular stereo camera, a structured light depth camera and an obstacle avoidance camera to obtain a primary environment image;
b. acquiring image information of the non-upper end of a space needing drawing construction through a 3D laser radar to obtain a secondary environment image;
c. performing data processing on the primary environment image, constructing an initial environment image, and identifying the poses of a TOF depth camera, a binocular stereo camera, a structured light depth camera and an obstacle avoidance camera;
d. transforming the poses of the TOF depth camera, the binocular stereo camera, the structured light depth camera and the obstacle avoidance camera by using the inherent pose transformation relation between the TOF depth camera, the binocular stereo camera, the structured light depth camera and the obstacle avoidance camera and the 3D laser radar to obtain the pose of the 3D laser radar;
e. transforming the secondary environment image by using the pose of the 3D laser radar to obtain a pose-matched secondary environment image;
f. and (3) carrying out monocular SLAM mapping on the secondary environment image with the matched pose and the initial SLAM map by combining the primary environment image obtained by the TOF depth camera, the binocular stereo camera, the structured light depth camera and the obstacle avoidance camera to obtain a final environment image.
Specifically, the establishment of the initial environment image and the poses of the TOF depth camera, the binocular stereo camera, the structured light depth camera and the obstacle avoidance camera mainly comprises the following steps:
g. acquire two arbitrary TOF depth cameras, two mesh stereo cameras, structured light depth camera or keep away the barrier camera and shoot two initial images that obtain respectively, acquire two arbitrary TOF depth cameras, two mesh stereo cameras of structured light depth camera or keep away the barrier camera and shoot two initial images that obtain respectively and include: acquiring an image A and an image B which are obtained by respectively shooting two TOF depth cameras, a binocular stereo camera, a structured light depth camera or an obstacle avoidance camera; the two TOF depth cameras, the binocular stereo camera, the structured light depth camera or the obstacle avoidance camera have different focal lengths; processing the image A and the image B according to an image effect obtained by shooting with the same focal length, taking the processed image A and the processed image B as two initial images, wherein at least partial overlapping of view ranges of the two TOF depth cameras, the binocular stereo camera, the structured light depth camera and the obstacle avoidance camera exists;
h. according to internal parameters and external parameters which are calibrated in advance by the two TOF depth cameras, the binocular stereo camera, the structured light depth camera or the obstacle avoidance camera and the parallax of the mutually matched feature points in the two initial images, the three-dimensional space positions of the mutually matched feature points are determined so as to obtain map points corresponding to the mutually matched feature points, and an initial SLAM map is constructed to complete initialization.
The invention realizes wide-angle rotation scanning and fixed-point detection through the 3d laser radar, avoids a blind area with a larger distance in the detection process, can acquire detailed detection information of an object, detects the information of a reticular obstacle and the side environment of the robot, ensures the comprehensiveness and high resolution of detection data, optimizes the positioning precision of a monocular SLAM, creates richer map information, calculates the current pose information and map based on the SLAM through the acquired images, and newly adds a depth sensor which is horizontally arranged to acquire the depth information, thereby not only realizing real-time operation, but also providing more accurate pose, creating richer map information, further playing the advantages of the SLAM in positioning navigation and automatic obstacle avoidance, respectively shooting through two image acquisition devices to obtain two initial images, constructing the initial SLAM map by utilizing mutually matched characteristic points in the initial images, after the initialization is successful, the image acquisition device is used for shooting the image so as to build the map of the monocular SLAM, the success rate of map building is improved, and the information loss in the map is reduced.
Finally, it should be noted that: although the present invention has been described in detail with reference to the foregoing embodiments, it will be apparent to those skilled in the art that changes may be made in the embodiments and/or equivalents thereof without departing from the spirit and scope of the invention. Any modification, equivalent replacement, or improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.