CN112001955A - Point cloud registration method and system based on two-dimensional projection plane matching constraint - Google Patents

Abstract

The invention discloses a point cloud registration method and a point cloud registration system based on two-dimensional projection plane matching constraint, wherein the point cloud registration method comprises the following steps: s100, loading two point clouds, and projecting the point clouds to a screen coordinate system according to the initial postures of the point clouds to generate two pieces of two-dimensional image data; s200, respectively extracting Affine-SIFT feature points and descriptors of two pieces of two-dimensional image data, and calculating a ratio of nearest neighbor to next nearest neighbor to obtain an initial matching point set; s300, carrying out error matching point elimination on the initial matching point set to obtain ransac _ match _ ASIFT; s400, reading a pixel point set coordinate in the ransac _ match _ ASIFT to obtain a mapping point set three-dimensional coordinate of the pixel point set coordinate in the point cloud; s500, solving a transformation matrix through the three-dimensional coordinates of the mapping point set to complete image pre-registration; and S600, iterating to obtain an accurate registration result. The point cloud registration comprises a display module, an image registration module, a preprocessing module and a point cloud registration module, and the four modules are sequentially in communication connection.

Description

Point cloud registration method and system based on two-dimensional projection plane matching constraint

Technical Field

The invention relates to the technical field of computer vision, in particular to a point cloud registration method and system based on two-dimensional projection plane matching constraint.

Background

Conventional data registration is mostly image registration. Image registration based on image gray scale and image registration based on image features are two types of registration methods which are common at present. The registration method based on the image gray level has the characteristics of improving estimation accuracy and robustness, the registration principle is to convert two images into gray level images, then establish similarity measurement between the gray level images, then seek the maximum or minimum transformation model parameters of the similarity measurement, and the seeking process mainly adopts a searching mode, and the algorithm has the defects of large calculated amount and low calculating speed. The image feature-based registration method greatly compresses image information, accelerates calculation speed, has certain illumination change resistance, mainly extracts and matches the significant features of two images, so the method is sensitive to the errors of feature extraction modes and feature matching. The method is generally divided into four steps, namely feature extraction, feature matching, calculation of transformation models and transformation parameters, coordinate transformation and interpolation. The image registration technology is mature, and has wide application in the fields of remote sensing image processing, target change monitoring, target identification, medical image analysis and the like.

Inspired by the registration of two-dimensional data, in the mid-eighties of the twentieth century, many scholars began to study the spatial registration of three-dimensional data, which benefits from the application of three-dimensional laser scanning systems.

Faugeras and Heber proposed the concept of a quadruple in 1986, applying the quadruple to spatial registration, and proposed a matching method between a point set and a point set. With this inspiring in 1987, Horn and Arun et al propose a method pstps (point to point registration) for point set registration based on a four-tuple method, which was later proven to be a key method for point set registration. Besl and Mckay, the field of computer vision research later, introduced an iterative closest point method, namely an ICP algorithm, which is a high-level method based on a free form surface.

The ICP algorithm well solves the registration problem among complex point sets, but the traditional ICP algorithm has the defects that if the initial corresponding point set is not well selected, the initial corresponding point set is easy to fall into a local extreme value, each iteration needs to calculate the corresponding points of the target point set in the reference point set, the calculation speed is low, and then a plurality of scholars propose some improvement schemes.

In 1991, Chen calculates the tangent plane distance from a point in a first view to a point in a second view by using a least square method, and further obtains coordinate transformation. Besl provides a rapid ICP algorithm in 1992, the algorithm well solves the problem of solving a 6-degree-of-freedom data point transformation model, but the processing speed is still slow due to the fact that the shortest distance between a point and the model needs to be calculated; in 2001, Fan et al calculate the shortest distance from a registration point set to a reference point set by a method of solving a nonlinear equation set by a Newton-Raphson method, and then use a least square method to calculate a rotation matrix R and a translation vector t, but the algorithm only uses a model smaller than a data point set; wang proposes an algorithm for extracting a point cloud boundary by using a mapping principle from a two-dimensional image to a three-dimensional point cloud. In the aspect of point cloud registration, Wang Dong et al use a method of mapping two-dimensional SIFT feature points to three-dimensional corresponding points to complete point cloud registration, but the method is not applicable to point cloud registration with obvious affine transformation, Zhaka proposes a data fusion method based on k-domain information, Rusu uses the fast histogram feature of point cloud to solve the problem of point cloud pre-registration, Horn proposes the concept of Extended Gaussian Image (EGI) as early as 1984 and summarizes relevant properties thereof, Makadia uses a method of converting point cloud into Extended Gaussian Image (EGI) to complete point cloud full-automatic registration by using affine invariance thereof, is inspired by two-dimensional image registration, Cyr, Wang et al propose a 2D-3D mapping method, and uses 2D features as shape features and point features to obtain better pre-registration effect. In the aspect of surface reconstruction, Fang explains the Delaunay triangulation principle, and a Particle Swarm Optimization (PSO) is applied to the region growing method in Chengjinui, so that the triangulation efficiency is improved. Cheng and Huang et al propose some improved algorithms based on the ICP algorithm in terms of surface reconstruction and package them in PCL library.

The point cloud registration algorithm based on the local descriptor can automatically and accurately register point clouds with partial overlapping, however, when the point clouds to be registered have strong similar characteristics, a large number of mismatching points occur, which is not beneficial to accurately estimating an initial transformation matrix, and therefore, a point cloud registration method capable of reducing the mismatching points needs to be researched to solve the problems.

Disclosure of Invention

In order to solve the technical problems, the invention provides a point cloud registration method and a point cloud registration system based on two-dimensional projection plane matching constraint, the method and the system have the characteristic of affine transformation resistance, and the problem of establishing correct corresponding point sets of two point clouds with affine transformation when the two point clouds are shot at different view angles is well solved.

In order to achieve the purpose, the technical scheme of the invention is as follows:

a point cloud registration method based on two-dimensional projection plane matching constraint comprises the following steps:

s100, loading two point clouds, and projecting the point clouds to a screen coordinate system according to the initial postures of the point clouds to generate two pieces of two-dimensional image data;

s200, extracting Affinine-SIFT feature points and descriptors of two pieces of two-dimensional image data respectively by utilizing an Affinine-SIFT algorithm, and calculating a ratio of nearest neighbor to next nearest neighbor by utilizing an Euclidean distance to obtain an initial matching point set;

s300, carrying out error matching point elimination on the initial matching point set by utilizing a RANSAC algorithm to obtain RANSAC _ match _ ASIFT;

s400, reading a pixel point set coordinate in the ransac _ match _ ASIFT in the VTK, and obtaining a mapping point set three-dimensional coordinate of the pixel point set coordinate in the point cloud in a man-machine interaction mode;

s500, solving a transformation matrix by using a least square method through mapping the three-dimensional coordinates of the point set to complete image pre-registration;

and S600, iterating the pre-registered point cloud by utilizing an ICP (inductively coupled plasma) algorithm to obtain an accurate registration result.

Further, the step S100 includes the steps of:

s110, loading and displaying the two point clouds;

and S120, projecting the point cloud initial posture to a screen coordinate system to generate a two-dimensional plane image, and intercepting and storing the two-dimensional plane image as two-dimensional image data in a png format.

Further, the step S500 includes the steps of:

s510, manually selecting six groups of corresponding point sets from the three-dimensional coordinates of the mapping point sets to obtain a transformation matrix by using a least square method;

and S510, multiplying the transformation matrix by the point cloud to be registered to obtain a pre-registration result of the point cloud data.

Further, in step S600, an ICP algorithm is used to iterate the pre-registered point cloud for 4 times, so as to obtain an accurate registration result.

A point cloud registration system based on two-dimensional projection plane matching constraint comprises a display module for displaying point cloud data and two-dimensional image data, an image registration module for extracting feature points of the two-dimensional image data and completing initial matching of the feature points, a preprocessing module for purifying an initial matching point set and finding a corresponding mapping point set in a point cloud for the purified initial matching point set, and a point cloud registration module for solving a transformation matrix to complete pre-registration and performing ICP (inductively coupled plasma) iterative processing on the point cloud data after the pre-registration is completed; the display module, the image registration module, the preprocessing module and the point cloud registration module are sequentially in communication connection.

Compared with the prior art, the invention has the advantages and positive effects that:

according to the method, the Affine transformation resisting characteristic of Affinine-SIFT feature points of the two-dimensional image is utilized, meanwhile, the RANSAC algorithm is utilized to eliminate mismatching points of the initial matching point set, the two-dimensional feature points are mapped to obtain the corresponding point set of the three-dimensional point cloud, then the transformation matrix is obtained to obtain the pre-registration result, and then the ICP algorithm is utilized, so that the iteration times of the ICP algorithm are greatly reduced, the algorithm efficiency is improved, the condition that the ICP cannot converge due to the fact that the initial corresponding point set cannot be found is effectively prevented, the accurate registration result is obtained, and the calculation rate and the accuracy of point cloud registration are effectively improved.

Drawings

In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and for those skilled in the art, other drawings can be obtained according to these drawings without creative efforts.

FIG. 1 is a schematic flow diagram of the present invention;

FIG. 2 is a schematic diagram of the projection of a point cloud onto a two-dimensional plane;

FIG. 3 is an experimental sample diagram in an embodiment, which is a rabbit point cloud and a dragon point cloud at different viewing angles, respectively;

FIG. 4 is a two-dimensional image feature point matching result before and after error point elimination, wherein a is a matching result before rabbit point cloud RANSAC purification, and b is a matching result after rabbit point cloud RANSAC purification;

FIG. 5 is a two-dimensional image feature point matching result before and after error point elimination, wherein a is a matching result before dragon point cloud RANSAC purification, and b is a matching result after dragon point cloud RANSAC purification;

FIG. 6 is two sets of rabbit point clouds before and after registration, wherein a is the rabbit point cloud before registration, b is the rabbit point cloud after pre-registration, and c is the rabbit point cloud after ICP algorithm fine registration;

FIG. 7 is two sets of dragon point clouds before and after registration, wherein, a is the dragon point clouds before registration, b is the dragon point clouds after pre-registration, and c is the dragon point clouds after ICP algorithm fine registration;

fig. 8 is a frame structure diagram of the point cloud registration system.

Detailed Description

The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived from the embodiments of the present invention by a person skilled in the art without any creative effort, should be included in the protection scope of the present invention.

In the embodiment, a corresponding computer program is written by using a C + + development language to automatically execute the invention, that is, the written computer program is used to complete the accurate registration of the point cloud.

In this embodiment, the rabbit point cloud and the dragon point cloud shown in fig. 3, 4, 5, 6, and 7 are taken as examples, and the written computer program is used to register point cloud data at two different viewing angles.

The flow of registration of the point cloud in this embodiment will be described in detail with reference to fig. 1.

And Step1, loading and storing the point cloud data.

Because the point cloud is three-dimensional data, the feature point extraction is directly carried out on the point cloud, the repetition rate is low and the point cloud is unstable, the point cloud data is loaded by using the system, a two-dimensional plane image (as shown in figure 2) is generated by projecting the initial attitude of the point cloud to a screen coordinate system, and the two-dimensional plane image is stored as two-dimensional image data in a PNG format.

Step2, processing of two-dimensional image data.

And extracting the characteristic points of the two-dimensional image data, and acquiring initial corresponding point set coordinates.

One specific embodiment of this step will be provided below.

There are many existing methods for extracting feature points, such as SIFT, ORB, SURF, etc., and when the front of the camera is blank, the direction of the optical axis of the camera may change, which causes distortion. Algorithms such as SIFT (scale invariant feature transform) have complete scale invariance but not complete affine invariance, and have certain limitation on the extraction of image features with large angle space change of shooting angles. In the embodiment, the Affinine-SIFT is used for extracting the feature points. The Affine-SIFT algorithm is high in automation, has Affine invariance and does not need manual intervention, and the method comprises the following specific steps:

firstly, selecting sampling parameters and simulating images with different longitudes and latitudes; then calculating the characteristics of the simulated image; and finally, combining the characteristics of all the simulated images to perform characteristic matching to obtain an initial characteristic point set.

Step3, processing of the initial feature point set.

The initial feature point set may include a plurality of erroneous point sets, if the number of erroneous point sets is too large, the final point cloud registration is affected, and the initial feature point set needs to be subjected to erroneous elimination, that is, point set purification. The RANSAC algorithm can eliminate the error matching of the point set to a large extent.

Step4, reading and mapping the feature point set.

The points calculated by the Affinine-SITF algorithm are two-dimensional image points, the point cloud data are three-dimensional image points, and three-dimensional coordinate points which are mapped one by one are found on the point cloud data for the two-dimensional image points to realize the point cloud registration. On the premise of consistent viewpoints and no scaling, the corresponding points of the two-dimensional pixel points in the three-dimensional point cloud are obtained by using a VTK-based human-computer interaction mode, so that the process of mapping from two dimensions to three dimensions is completed.

Step5, solving the transformation matrix.

Any one of the two point cloud data with different visual angles can be completely overlapped with the other point cloud data through operations of translation, scaling, rotation and the like, namely, each three-dimensional point of the two point clouds with different visual angles has a certain mathematical relationship with a corresponding point in the other point cloud, and the mathematical relationship is a transformation matrix between images. The method comprises the steps of obtaining corresponding points of two-dimensional pixel points in three-dimensional point cloud through a man-machine interaction mode based on VTK, manually selecting six groups of corresponding point sets from the points, obtaining a transformation matrix through a least square method, and multiplying the transformation matrix by point cloud to be registered to obtain a coarse registration result of point cloud data.

The selection of the corresponding point set in this embodiment is specifically as follows:

after the corresponding point set in the three-dimensional point cloud is obtained, six groups of point cloud corresponding points are manually selected by eyes, and the selected corresponding points cannot be excessively concentrated in the point cloud and are distributed in a balanced manner as far as possible.

Step6, coarse registration point cloud optimization processing.

Due to the fact that a corresponding point set is selected by manual visual observation to calculate a transformation matrix, selection errors may exist, a small amount of registration errors still exist in the point cloud subjected to coarse registration, deep registration needs to be conducted on the point cloud subjected to coarse registration, registration errors are further reduced, iteration is conducted by means of an ICP algorithm, and convergence can be completed only by iterating for 4 times. Under the condition that the ICP algorithm can conduct direct registration, the result can be converged only by iterating 30-50 times generally, and after coarse registration, the iteration times are reduced to 4, so that the efficiency of the ICP algorithm is greatly improved.

The invention provides a point cloud registration system for implementing the steps, as shown in fig. 8, the system comprises a display module for displaying point cloud data and two-dimensional image data, an image registration module for extracting feature points of the two-dimensional image data and completing initial matching of the feature points, a preprocessing module for purifying an initial matching point set and finding a corresponding mapping point set in the point cloud for the purified initial matching point set, and a point cloud registration module for solving a transformation matrix to complete pre-registration and performing ICP iterative processing on the point cloud data after the pre-registration is completed; the display module, the image registration module, the preprocessing module and the point cloud registration module are sequentially in communication connection.

The point cloud registration system can be implemented in electronic hardware, computer software, or a combination of both, and the components and steps of the examples have been generally described in the foregoing description in terms of functionality in order to clearly illustrate the interchangeability of hardware and software. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the implementation. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present invention.

The steps of a method or algorithm described in this disclosure may be embodied directly in hardware, in a software module executed by a processor, or in a combination of the two. A software module may reside in random access memory, read only memory, electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art.

According to the method, the Affine transformation resisting characteristic of Affinine-SIFT feature points of the two-dimensional image is utilized, meanwhile, the RANSAC algorithm is utilized to eliminate mismatching points of the initial matching point set, the two-dimensional feature points are mapped to obtain the corresponding point set of the three-dimensional point cloud, then the transformation matrix is obtained to obtain the pre-registration result, and then the ICP algorithm is utilized, so that the iteration times of the ICP algorithm are greatly reduced, the algorithm efficiency is improved, the condition that the ICP cannot converge due to the fact that the initial corresponding point set cannot be found is effectively prevented, the accurate registration result is obtained, and the calculation rate and the accuracy of point cloud registration are effectively improved.

Claims (5)

1. A point cloud registration method based on two-dimensional projection plane matching constraint comprises the following steps:

s100, loading two point clouds, and projecting the point clouds to a screen coordinate system according to the initial postures of the point clouds to generate two pieces of two-dimensional image data;

s200, extracting Affinine-SIFT feature points and descriptors of two pieces of two-dimensional image data respectively by utilizing an Affinine-SIFT algorithm, and calculating a ratio of nearest neighbor to next nearest neighbor by utilizing an Euclidean distance to obtain an initial matching point set;

s300, carrying out error matching point elimination on the initial matching point set by utilizing a RANSAC algorithm to obtain RANSAC _ match _ ASIFT;

s400, reading a pixel point set coordinate in the ransac _ match _ ASIFT in the VTK, and obtaining a mapping point set three-dimensional coordinate of the pixel point set coordinate in the point cloud in a man-machine interaction mode;

s500, solving a transformation matrix by using a least square method through mapping the three-dimensional coordinates of the point set to complete image pre-registration;

and S600, iterating the pre-registered point cloud by utilizing an ICP (inductively coupled plasma) algorithm to obtain an accurate registration result.

2. The point cloud registration method based on two-dimensional projection plane matching constraints of claim 1, wherein: the step S100 includes the steps of:

s110, loading and displaying the two point clouds;

and S120, projecting the point cloud initial posture to a screen coordinate system to generate a two-dimensional plane image, and intercepting and storing the two-dimensional plane image as two-dimensional image data in a png format.

3. The point cloud registration method based on two-dimensional projection plane matching constraints of claim 2, wherein: the step S500 includes the steps of:

s510, manually selecting six groups of corresponding point sets from the three-dimensional coordinates of the mapping point sets to obtain a transformation matrix by using a least square method;

and S510, multiplying the transformation matrix by the point cloud to be registered to obtain a pre-registration result of the point cloud data.

4. The point cloud registration method based on two-dimensional projection plane matching constraints of claim 3, wherein: in the step S600, the ICP algorithm is used to iterate the pre-registered point cloud for 4 times, so as to obtain an accurate registration result.

5. A point cloud registration system based on two-dimensional projection plane matching constraint, the point cloud registration system being configured to implement the point cloud registration method based on two-dimensional projection plane matching constraint of claim 4, wherein: the point cloud registration system comprises a display module for displaying point cloud data and two-dimensional image data, an image registration module for extracting characteristic points of the two-dimensional image data and completing initial matching of the characteristic points, a preprocessing module for purifying an initial matching point set and finding a corresponding mapping point set in a point cloud for the purified initial matching point set, and a point cloud registration module for solving a transformation matrix to complete pre-registration and performing ICP (inductively coupled plasma) iterative processing on the point cloud data after the pre-registration is completed; the display module, the image registration module, the preprocessing module and the point cloud registration module are sequentially in communication connection.

Images