Disclosure of Invention
In view of the above, the present invention is directed to a ToF-based weld joint identification method, which can improve the efficiency and accuracy of weld joint identification.
In order to achieve the purpose, the invention provides the following technical scheme:
a welding seam identification method based on ToF comprises the following steps:
step S1, acquiring an original weld image of the weldment to be processed through a camera based on the ToF technology, wherein the original weld image comprises; an amplitude image and a depth image;
step S2, preprocessing the amplitude image obtained in the step S1 to obtain a preprocessed amplitude image;
step S3, local threshold value binarization processing is carried out on the amplitude image after preprocessing obtained in the step S2 to obtain a corresponding binarized image;
s4, extracting the edge characteristics of the binarized image through a Gabor filter, and acquiring an edge image of a welding seam;
step S5, performing radon transformation on the edge image acquired in the step S4 to obtain a horizontally corrected edge image, and identifying a weld image based on the appearance condition of the weld;
s6, acquiring two-dimensional information of the welding seam through the welding seam image identified in the S5, and solving a three-dimensional coordinate of the welding seam by combining the depth information acquired in the S1;
step S7, constructing a conversion relation among a world coordinate system, a camera coordinate system, an image coordinate system and a pixel coordinate system;
and step S8, converting the three-dimensional coordinates of the welding seam acquired in the step S6 into space coordinates in the world coordinate system according to the conversion relation between the pixel coordinate system and the world coordinate system in the step S7.
Further, the sensor in the camera based on the ToF technology is an array type.
Further, in the step S2, the preprocessing includes: the amplitude image is clipped to obtain an image containing a weld region, and the image is subjected to filtering processing.
Further, the step S4 specifically includes:
s401, passing through the imaginary part of the Gabor filter function, wherein the expression is shown as formula (1), 4 scales are selected, f is 0.15, 0.3, 0.15 and 0.6 respectively, 6 directions are selected, and theta is 0,
Pi and
constructing 24 filter banks;
in the formula (1), x is a Gaussian scale in the main shaft direction; y is a gaussian scale in which the principal axis directions are orthogonal, f is a filter center frequency, θ is a rotation angle of the gaussian principal axis, η and γ are constants, and x ═ xcos θ + ysin θ, y ═ xsin θ + ycos θ.
S402, performing space domain convolution on the 24 filter banks obtained in the S401 and the binary image obtained in the S3 to obtain preliminary edge detection images with 4 scales and 6 directions;
step S403, performing non-maximum value suppression on the preliminary edge detection image obtained in step S402, comparing two points near the corresponding image according to the detection direction, if the two points are the maximum value, reserving the two points, and if the two points are not the maximum value, changing the two points to 0;
and S404, fusing the preliminary edge detection images of 4 scales and 6 directions, and then performing edge connection on the fused images to obtain edge images of the welding seams.
Further, the step S7 specifically includes:
firstly, converting a world coordinate system into a camera coordinate system through rigid body transformation;
then, converting the camera coordinate system into an image coordinate system through perspective projection;
and finally, discretizing the image coordinate system to obtain a pixel coordinate system.
The invention has the beneficial effects that:
compared with a contact-based weld joint identification method, the method provided by the invention has the advantages that the algorithm is simple, the identification speed is higher, the complex weld joint can be accurately identified in a shorter time, and the identification precision is higher. And the ToF camera can directly acquire the depth information of the welding seam while acquiring the welding seam image, so that the target can be quickly and accurately reconstructed in three dimensions compared with a binocular vision method and a structured light method.
Detailed Description
In order to make the objects, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are some, but not all, embodiments of the present invention. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
Example 1
Referring to fig. 1 to 5, the present embodiment provides a ToF-based weld joint identification method, including the following steps:
s1, acquiring an original weld image of the weldment to be processed through a camera based on the ToF technology, wherein the original weld image comprises an image of the original weld; an amplitude image and a depth image;
specifically, the original weld image is a weld image before or after welding, in this embodiment, a camera based on the ToF technology is used for direct shooting and acquisition, and sensors in the camera based on the ToF technology are in an array type, so that target three-dimensional information can be rapidly acquired in the process of acquiring an image of each frame; a sensor in the camera based on the ToF technology can emit modulated infrared light, the light is subjected to diffuse reflection after encountering a weld joint, and a receiving end can obtain corresponding weld joint depth information by analyzing the phase difference or time difference between emitted light and received light, so that the depth information, point cloud information, gray scale information and the like of a target image can be obtained.
Step S2, preprocessing the amplitude image obtained in the step S1 to obtain a preprocessed amplitude image;
specifically, the pretreatment comprises: the amplitude image is cropped to obtain an image containing a weld joint area, and the image is subjected to filtering processing, wherein the purpose of the filtering processing is to weaken the influence of light on the amplitude image in the environment.
More specifically, the weld acquired by the camera based on the ToF technology contains various data information and is easily affected by light in the environment. The acquired image contains a large amount of noise and image information interference irrelevant to the welding seam, and pixels in the point cloud picture contain 3 data, namely XYZ three-dimensional coordinates. The three-dimensional point cloud data of the space points in the collected point cloud image can be converted into a two-dimensional matrix with depth information as an index. All the two-dimensional matrixes are arranged according to the spatial sequence to obtain a central matrix, and the average difference value of the depth information indexed by the central matrix and the depth information indexed by the surrounding matrixes is calculated. The average difference serves as a global threshold for depth information. And removing the three-dimensional point cloud with the depth information in the point cloud image being too far away from the global threshold value as a noise point, thereby improving the subsequent calculation efficiency and accuracy.
Step S3, local threshold value binarization processing is carried out on the amplitude image after preprocessing obtained in the step S2 to obtain a corresponding binarized image;
specifically, the threshold is obtained by calculating a local image Gaussian weighted average, and the preprocessed amplitude image is used for determining a binarization threshold by using a histogram method, so that a binarization image capable of reflecting the overall and local characteristics of the image is obtained.
S4, extracting the edge characteristics of the binary image through a Gabor filter, and acquiring the edge image of the welding seam;
specifically, step S4 specifically includes:
the principle of the Gabor filtering algorithm is as follows:
wherein x 'xcos θ + ysin θ, y' xsin θ + ycos θ, f is the center frequency, and θ is the selected direction;
the imaginary part of the Gabor filter function is expressed as formula (1), 4 scales are selected, f is 0.15, 0.3, 0.15 and 0.6 respectively, 6 directions are selected, theta is 0,
Pi and
constructing 24 filter banks;
wherein x is a Gaussian scale in the direction of the main shaft; y is a Gaussian scale orthogonal to the main shaft direction; f is expressed as the filter center frequency; θ represents the rotation angle of the gaussian main shaft; η and γ are constants, and in this embodiment, η is 1 and γ is 2.
S402, performing space domain convolution on the 24 filter banks obtained in the S401 and the binary image obtained in the S3 to obtain preliminary edge detection images with 4 scales and 6 directions;
specifically, a 3 × 3 convolution kernel is defined
In the originalAnd sliding the convolution kernel on the binary image, and performing summation operation until the convolution kernel slides over pixels of the whole image to obtain values output by all the pixels, so as to obtain primary edge detection images with different scales and different directions, wherein each image represents edge information of welding seams with different scales and different directions.
Step S403, performing non-maximum value suppression on the preliminary edge detection image obtained in step S402, comparing two points near the corresponding image according to the detection direction, if the two points are the maximum value, reserving the two points, and if the two points are not the maximum value, changing the two points to 0;
and S404, fusing the preliminary edge detection images of 4 scales and 6 directions, and then performing edge connection on the fused images to obtain edge images of the welding seams.
Step S5, performing radon transformation on the edge image acquired in the step S4 to obtain a horizontally corrected edge image, and identifying a weld image based on the appearance condition of the weld;
specifically, the image subjected to edge feature extraction is subjected to Radon Transform (RT):
rotating the image by any theta angle (the rotation angle is between 0 and 180 degrees) by taking the center of the image as an origin to obtain a corresponding horizontal projection value r in rho-theta space; different R forms a projection set R, the maximum value of elements in the R is solved, and corresponding values theta and rho are solved, wherein theta is the horizontal rotation angle, and rho is the distance from the corresponding origin to the straight line;
then, converting theta and rho values obtained in the rho-theta space to a certain point Q through which the edge of the welding seam passes in an image plane coordinate system;
solving the position of the edge of the welding seam in the image coordinate plane according to a linear equation solving method;
and finally, integrally identifying the welding seam according to the prior knowledge of the appearance conditions (such as the width and the type) of the welding seam and the like.
S6, acquiring two-dimensional information of the welding seam through the welding seam image identified in the S5, and solving a three-dimensional coordinate of the welding seam by combining the depth image acquired in the S1;
specifically, since the acquisition of the weld image in the amplitude image has been identified in step S5, two-dimensional information of the weld can be directly acquired. Since the amplitude image and the depth image acquired by the ToF camera are directly related, the depth information of the weld joint can be acquired from the depth image. And combining the two-dimensional information and the depth information of the welding seam to obtain the three-dimensional coordinate information of the welding seam.
Step S7, constructing a conversion relation among a world coordinate system, a camera coordinate system, an image coordinate system and a pixel coordinate system;
specifically, a world coordinate system is converted into a camera coordinate system through rigid body transformation; then, converting the camera coordinate system into an image coordinate system through perspective projection; and finally, discretizing the image coordinate system to obtain a pixel coordinate system.
More specifically:
world coordinate system (X)w,Yw,Zw) -a three-dimensional coordinate system in the real world describing the location of the object in the real world;
camera coordinate system (X)c,Yc,Zc) A three-dimensional rectangular coordinate system is established by taking the focusing center of the camera as an origin and taking the optical axis as Z;
image coordinate system (x, y) -to describe how the image in the camera coordinate system is projected onto the camera's negative;
pixel coordinates (u, v) -the image is composed of pixels, so the pixel coordinate system is used to determine the position of the pixel in the image.
As shown in fig. 1, rigid transformation is required to convert the world coordinate system to the camera coordinate system, and the rigid transformation is a transformation that only translates, rotates, and inverts an object without deforming the object.
The related transformation between the world coordinate system and the camera coordinate system can be completed only by performing rotation transformation and translation transformation.
The transformation of the world coordinate system to the camera coordinate system can be represented by a rotation matrix R and a translation matrix t:
expressed in homogeneous coordinate system as:
wherein [ r ]
11,r
12,r
13]
T,[r
21,r
22,r
23]
T,[r
31,r
32,r
33]
TBase vectors respectively representing original coordinate systems
t
x,t
y,t
zIndicating the amount of translation in the x, y, z direction for the transformation to another coordinate system.
From the camera coordinate system to the image coordinate system, belonging to the projection perspective, i.e. from 3D to 2D, the schematic diagram is shown in fig. 2, where P is a point in space corresponding to P in the image coordinate system, whose coordinates are (x.y), according to the triangle-like principle:
expressed in homogeneous coordinates as:
where f denotes the focal length of the camera in fig. 2.
This step completes the conversion of the camera coordinate system to the ideal image coordinate system.
From the image coordinate system to the pixel coordinate system, the image coordinate system and the pixel coordinate system are on the same plane, but the origins of the two are different, so that a transformation is required, and the principle is as shown in fig. 3, where the transformation relationship between the pixel coordinate and the image coordinate is:
and (3) homogenizing to obtain:
the conversion matrix can be obtained by combining the four conversions:
in a world coordinate system, assuming that the position coordinate of one point on a welding seam is (x, y, z), combining a rotation matrix and a translation matrix, obtaining the coordinate of the point under the camera coordinate system based on the ToF technology through rigid body conversion, and then using a similar triangle principle, completing the conversion of the point on the welding seam in the three-dimensional camera image coordinate system based on the ToF technology.
And step S8, converting the three-dimensional coordinates of the welding seam acquired in the step S6 into space coordinates in the world coordinate system according to the conversion relation between the pixel coordinate system and the world coordinate system in the step S7.
In order to perform three-dimensional positioning on the weld joint, coordinate transformation is required. The position of the real-world weld obtained by the camera based on the ToF technology can be established with the corresponding relation with the pixel on the imaging plane of the ToF camera according to the conversion method.
Radon Transform (RT), which is a projection transform of the resulting digital image in various angular directions, is mathematically understood as a linear integral of a two-dimensional function f (x, y), and the resulting integral is projected onto the RT plane.
The integrated value obtained by linear projective transformation is also called Radon curve, which is determined by the distance ρ of the straight line in the image from the origin of the image coordinate system and the inclination angle θ of the straight line.
The digital image in the plane is linearly integrated along a straight line ρ ═ xcos θ + ysin θ, and F (θ, ρ) obtained by the linear integration is Radon transform of the digital image, that is, a certain point (θ, ρ) in the transform plane corresponds to a certain line integral value of the original image F (x, y). The Radon transform formula for a digital image f (x, y) is:
F(θ,ρ)=∫∫f(x,y)δ(ρ-xcosθ-ysinθ)dxdy
wherein:
f (x, y) is the pixel gray value of a certain point (x, y) on the image, delta is the Dirac function, and rho is the distance from the projection line to the origin in the (x, y) plane; theta is the angle between the normal of the projection line and the x-axis.
The characteristic function δ linearly integrates the image along a straight line ρ ═ xcos θ + ysin θ from the definition of RT, which can be seen as a linear projection of the digital image in the ρ - θ coordinate system, with each point in the coordinate system corresponding to each straight line in the image coordinate system; RT can also be seen as a linear projection on the horizontal axis of the image obtained after rotating the digital image clockwise by an angle theta.
So RT can be used for edge line detection in digital images: in the digital image coordinate system, a line with a high gray value will form a point with relatively high brightness in the rho-theta space, while a line with a low gray value will form a point with relatively dark brightness in the rho-theta space.
In this embodiment, fig. 4 is an original weld image obtained by using a camera based on the ToF technology in embodiment 1, and is processed by the method in this embodiment to obtain fig. 5, and fig. 5 is a point cloud image of a weld centerline finally obtained in embodiment 1.
The invention is not described in detail, but is well known to those skilled in the art.
The foregoing detailed description of the preferred embodiments of the invention has been presented. It should be understood that numerous modifications and variations could be devised by those skilled in the art in light of the present teachings without departing from the inventive concepts. Therefore, the technical solutions available to those skilled in the art through logic analysis, reasoning and limited experiments based on the prior art according to the concept of the present invention should be within the scope of protection defined by the claims.