




技术领域technical field
本发明属于焊接质量自动检测和控制技术领域。涉及基于视觉图像纹理分割的焊缝识别与跟踪技术的一种基于纹理分割的焊缝识别方法,可广泛应用与机器人自动化焊接等方面。The invention belongs to the technical field of welding quality automatic detection and control. The invention relates to a welding seam recognition method based on texture segmentation based on visual image texture segmentation welding seam recognition and tracking technology, which can be widely used in robot automatic welding and other aspects.
背景技术Background technique
焊缝的自动识别和焊缝跟踪在焊接机器人智能化发展中具有重要地位。而常用并且比较实用的是通过视觉来实现,主要包括两种方法:主动光视觉和被动光视觉。主动光视觉采用激光扫描、结构光等主动发光装置在焊缝坡口上形成一条包含坡口形状信息的光亮条纹,该方法系统较复杂、成本较高。被动视觉是依靠自然光或弧光条件下,取得包含焊缝的图像,通过图像处理,获得焊缝的边缘,这种方法常常需要焊缝图像具有明显的灰度突变特征。The automatic identification and seam tracking of welds play an important role in the intelligent development of welding robots. The commonly used and more practical is to achieve through vision, mainly including two methods: active light vision and passive light vision. Active light vision uses active light-emitting devices such as laser scanning and structured light to form a bright stripe containing groove shape information on the weld groove. This method is more complicated and costly. Passive vision relies on natural light or arc light to obtain images containing welds, and obtains the edges of welds through image processing. This method often requires weld images to have obvious gray-scale mutation characteristics.
对于厚板焊接,常常采用多道焊、多层焊方法,焊接时首先在坡口中进行打底焊,然后采用填充焊填充坡口,在填充焊时根据需要可采用多道焊,最后往往会进行盖面焊。打底焊后,随着焊接过程的进行,焊缝坡口的特征越来越不明显,即不利于结构光视觉方法和一般的被动视觉方法:坡口的三维结构特征不明显,使得结构光在坡口上不能形成具有明显转折光亮条纹,从而不易确定焊缝中心,并极易受焊缝旁边飞溅、油污等影响;焊缝图像上焊缝边缘没有明显的灰度梯度,不能通过简单的图像处理方法(如边缘提取、灰度阈值分割)来实现确定焊缝边缘。For thick plate welding, multi-pass welding and multi-layer welding methods are often used. When welding, firstly perform root welding in the groove, and then use filling welding to fill the groove. During filling welding, multi-pass welding can be used according to needs, and finally Perform cover welding. After rooting welding, as the welding process proceeds, the characteristics of the weld groove become less and less obvious, which is not conducive to the structured light vision method and the general passive vision method: the three-dimensional structural characteristics of the groove are not obvious, making the structured light Bright stripes with obvious transitions cannot be formed on the groove, so it is difficult to determine the center of the weld, and it is easily affected by splashes, oil stains, etc. beside the weld; there is no obvious gray gradient at the edge of the weld on the weld image, and it cannot be passed through a simple image. Processing methods (such as edge extraction, gray threshold segmentation) to determine the edge of the weld.
发明内容Contents of the invention
本发明的目的在于克服现有技术不足,提出了基于焊缝图像纹理分割的焊缝识别方法,以便实现在多层焊中盖面焊前的焊缝识别问题。The purpose of the present invention is to overcome the deficiencies of the prior art, and propose a weld seam recognition method based on weld image texture segmentation, so as to realize the weld seam recognition problem before cover welding in multi-layer welding.
为了实现这一目的,本发明的技术方案中,首先使用CCD摄像机拍摄包含焊缝区域的图像,然后分析图像中的焊缝区域和母材区域的纹理特征,根据焊缝区域与母材区域纹理特征的差异来区分焊缝区域,最后提取焊缝区域的边缘即为焊缝边缘。In order to achieve this goal, in the technical solution of the present invention, at first use the CCD camera to take the image that contains the weld region, then analyze the texture features of the weld region and the base metal region in the image, according to the texture characteristics of the weld region and the base metal region The difference of the features is used to distinguish the weld area, and finally the edge of the weld area is extracted as the weld edge.
本发明的基于纹理分割的焊缝识别方法主要包括以下几个步骤。The welding seam recognition method based on texture segmentation of the present invention mainly includes the following steps.
1)图像获取,使用CCD摄像机获取包含焊缝在内的图像,使得图像中部为焊缝区域,焊缝区域两侧为母材区域。1) Image acquisition, using a CCD camera to acquire an image including the weld, so that the middle of the image is the weld area, and the two sides of the weld area are the base metal area.
2)纹理特征提取,对步骤1获取的图像进行纹理特征分析,提取的纹理特征可采用统计特征或频谱特征进行分析;所述统计特征常用基于灰度共生矩阵的特征描述符表示;频谱特征则通过Gabor小波变换获得。统计特征的提取是先将所述的步骤1的图像进行子图像划分,子图像的尺寸根据图像纹理特征和焊缝识别精度要求确定一即使得每个子图像体现图像的纹理特征并且子图像尺寸又能保证图像分割的边缘准确性;然后通过式(1)计算各个子图像的共生矩阵M(h,k),再用共生矩阵的纹理特征描述符表征该子图像的纹理特征,常用的纹理特征描述符有表示能量的二阶矩WM、对比度WC、熵WE、逆差矩WH等,如式(2)~(5)所示;具体使用时,可挑选一个或多个合适的使用;频谱特征提取是将原图像通过小波变换分解成多个频道,其中Gabor小波可采用如式(6)的Gabor函数作为基小波,把基小波伸缩、平移的参数和方向离散化后可得如式(7)的离散小波族,相应的小波变换如式(8)所示;再在这些频道上计算纹理能量来表征纹理特征,定义在(2u+1)×(2v+1)窗口中称为“纹理能量”的特征e(i,j)如式(9)所示。2) Texture feature extraction, texture feature analysis is performed on the image obtained in step 1, the extracted texture features can be analyzed using statistical features or spectral features; Obtained by Gabor wavelet transform. The extraction of statistical features is to first divide the image in step 1 into sub-images, and the size of the sub-images is determined according to the image texture features and weld recognition accuracy requirements—that is, each sub-image reflects the texture features of the image and the size of the sub-images Can guarantee the edge accuracy of image segmentation; then calculate the co-occurrence matrix M(h, k) of each sub-image by formula (1), and then use the texture feature descriptor of the co-occurrence matrix to characterize the texture feature of the sub-image, the commonly used texture feature Descriptors include second-order moment WM representing energy, contrast WC , entropy WE , reverse difference moment WH, etc., as shown in formulas (2) to (5); in specific use, one or more suitable Use; Spectrum feature extraction is to decompose the original image into multiple channels through wavelet transform, wherein the Gabor wavelet can use the Gabor function such as formula (6) as the base wavelet, and discretize the expansion and translation parameters and directions of the base wavelet to obtain For the discrete wavelet family of formula (7), the corresponding wavelet transform is shown in formula (8); then the texture energy is calculated on these channels to represent the texture features, defined in the (2u+1)×(2v+1) window The characteristic e(i, j) called "texture energy" is shown in equation (9).
其中,f(x,y)是焊缝子图像,h,k为图像f(x,y)中像素的灰度值,#代表像素的数量;Among them, f(x, y) is the weld sub-image, h, k are the gray value of the pixel in the image f(x, y), and # represents the number of pixels;
gλ(x,y,θ)=exp[-(λ2x′2+y′2)+iπx′], (6)gλ (x, y, θ) = exp[-(λ2 x′2 +y′2 )+iπx′], (6)
x′=xcosθ+ysinθ,y′=-xsinθ+ycosθ,x'=xcosθ+ysinθ, y'=-xsinθ+ycosθ,
其中λ是x和y方向的纵横比,θ是方向参数。where λ is the aspect ratio in the x and y directions, and θ is the orientation parameter.
g(αj(x-x0,y-y0),θk),α∈R,j={0,-1,-2,…}, (7)g(αj (xx0 , yy0 ), θk ), α∈R, j={0, -1, -2,...}, (7)
式中θk=kπ/N,k={0,…,N-1},N表示离散方向的数目。In the formula, θk =kπ/N, k={0,...,N-1}, N represents the number of discrete directions.
Wj(x,y,θ)=∫f(x1,y1)g*(αj(x-x0,y-y0),θ)dx1dy1. (8)Wj (x, y, θ) = ∫f(x1 , y1 ) g* (αj (xx0 , yy0 ), θ)dx1 dy1 . (8)
其中,m,n,i,j为图像坐标,W(m,n)为图像进行小波变换后的结果。Among them, m, n, i, j are image coordinates, and W(m, n) is the result of image wavelet transformation.
3)纹理分割。根据焊缝区域和母材区域纹理特征的差异性,选定阈值,对计算得到的纹理特征进行阈值分割,将图像分成焊缝区域-黑色标识和母材区域-白色标识,获得图像的初步分割结果。3) Texture segmentation. According to the difference of the texture features of the weld area and the base metal area, the threshold is selected, and the calculated texture features are thresholded, and the image is divided into the weld area-black mark and the base metal area-white mark, and the preliminary segmentation of the image is obtained. result.
4)误分割消除及焊缝边缘确定。根据焊缝边缘的连续、小曲率特点,消除母材区域中被误识别为焊缝的区域,并确定焊缝边缘。具体方法如下,在被初步确定为焊缝区域的子图像中,从图像两侧分别向中部搜索焊缝边缘,将不满足以下特点的子图像更改为母材区域,将首次满足以下特点的子图像作为焊缝边缘:a)该子图像的沿焊缝纵向的8-邻域内存是属于焊缝区域的子图像;b)在以该子图像为中心的一个图像区域内,属于焊缝区域的子图像的数量占50%以上的比例,将两侧所有的焊缝边缘各自连接起来,即形成两条黑线为焊缝的边缘。4) Mis-segmentation elimination and weld edge determination. According to the continuous and small curvature characteristics of the weld edge, the area in the base metal area that is misidentified as the weld is eliminated, and the weld edge is determined. The specific method is as follows. In the sub-image initially determined as the weld area, search for the edge of the weld from both sides of the image to the middle, change the sub-image that does not meet the following characteristics into the base metal area, and replace the sub-image that meets the following characteristics for the first time. Image as the edge of the weld: a) the 8-neighborhood memory of the sub-image along the longitudinal direction of the weld is a sub-image belonging to the weld area; b) in an image area centered on the sub-image, it belongs to the weld area The number of sub-images of is more than 50%, and all the weld edges on both sides are connected respectively, that is, two black lines are formed as the edges of the weld.
本发明提出的基于纹理分割的焊缝识别方法,利用焊缝区域和母材区域的纹理特征的差异,能够实现焊缝识别问题,特别是对于多层焊中的填充焊和盖面焊的焊缝识别较结构光方法和一般被动光视觉方法具有明显优势。The weld seam recognition method based on texture segmentation proposed by the present invention can realize the weld seam recognition problem by using the difference in texture features between the weld seam area and the base metal area, especially for the welding of filler welding and cover welding in multilayer welding. Seam recognition has obvious advantages over structured light methods and general passive light vision methods.
附图说明Description of drawings
图1是基于纹理分割的焊缝识别方法流程。Figure 1 is the process flow of the weld seam recognition method based on texture segmentation.
图2是焊缝图像的子图像划分。Figure 2 is the sub-image division of the weld image.
图3是待识别图像的纹理特征分布。Figure 3 is the texture feature distribution of the image to be recognized.
图4是纹理分割结果。Figure 4 is the result of texture segmentation.
图5是焊缝边缘提取结果。Figure 5 is the result of weld edge extraction.
具体实施方式Detailed ways
为了更好地讲解本发明的技术方案,以下结合实施例作进一步的详细描述。In order to better explain the technical solutions of the present invention, further detailed descriptions will be given below in conjunction with examples.
图1所示为本发明的焊缝识别方法流程,包括以下几个步骤:Fig. 1 shows the flow chart of the welding seam identification method of the present invention, including the following steps:
1、图像获取,使用CCD摄像机获取包含焊缝在内的图像,使得图像中部为焊缝区域,焊缝区域两侧为母材区域。1. Image acquisition, using a CCD camera to acquire images including welds, so that the middle of the image is the weld area, and the two sides of the weld area are the base metal areas.
2、纹理特征提取,对步骤1获取的图像进行纹理特征分析,获得图像的纹理特征;提取的纹理特征包括统计特征和频谱特征。本实例采用基于灰度共生矩阵的纹理特征描述符中的能量(二阶矩)WM来表示纹理特征。首先根据图像像素与实物的关系确定子图像大小,并对原图像进行区域划分,划分结果如图2所示,本实例采用24×10(焊缝纵向像素数×焊缝横向像素数)的子图像尺寸;然后计算每个子图像的共生矩阵,共生矩阵的计算参数为灰度级32、灰度步长l、方向0度,即先将子图像f(x,y)变换为灰度级为32的图像,计算M(h,k)时令公式(1)中的像素点(x1,y1)、(x2,y2)满足(x2=x1+1,y2=y1+1);再基于该共生矩阵M(h,k)用公式(2)计算各个子图像的纹理特征值(用二阶矩表示),结果如图3所示。2. Extraction of texture features, analyzing the texture features of the image obtained in step 1 to obtain the texture features of the image; the extracted texture features include statistical features and spectral features. In this example, the energy (second moment) WM in the texture feature descriptor based on the gray level co-occurrence matrix is used to represent the texture feature. First, the size of the sub-image is determined according to the relationship between the image pixels and the real object, and the original image is divided into regions. Image size; then calculate the co-occurrence matrix of each sub-image, the calculation parameters of the co-occurrence matrix are grayscale 32, grayscale step size 1, direction 0 degrees, that is, the subimage f (x, y) is transformed into a grayscale of 32 image, calculate M(h, k) when the pixel points (x1 , y1 ) and (x2 , y2 ) in formula (1) satisfy (x2 =x1 +1, y2 =y1 +1); Then, based on the co-occurrence matrix M (h, k), the texture feature value (expressed by the second-order moment) of each sub-image is calculated with the formula (2), and the result is shown in FIG. 3 .
3、纹理分割。根据焊缝区域和母材区域纹理特征的差异性,选定阈值T0=0.13,对图3中的纹理特征值进行分割,将纹理特征值大于阈值的子图像标识为母材区域,将纹理特征值小于等于阈值的子图像标识为焊缝区域,图像的初步分割结果,如图4所示。3. Texture segmentation. According to the difference in texture features between the weld area and the base metal area, the threshold T0 = 0.13 is selected to segment the texture feature values in Figure 3, and the sub-images with texture feature values greater than the threshold are identified as the base metal area, and the The sub-image whose eigenvalue is less than or equal to the threshold is identified as the weld area, and the preliminary segmentation results of the image are shown in Figure 4.
4、误分割消除及焊缝边缘确定。根据焊缝边缘的连续、小曲率特点,消除母材区域中被误识别为焊缝的区域,并确定焊缝边缘。具体方法如下,在被初步确定为焊缝区域的子图像中,从图像两侧分别向中部搜索焊缝边缘,将不满足以下特点的子图像更改为母材区域,将首次满足以下特点的子图像作为焊缝边缘:4. Mis-segmentation elimination and weld edge determination. According to the continuous and small curvature characteristics of the weld edge, the area in the base metal area that is misidentified as the weld is eliminated, and the weld edge is determined. The specific method is as follows. In the sub-image initially determined as the weld area, search for the edge of the weld from both sides of the image to the middle, change the sub-image that does not meet the following characteristics into the base metal area, and replace the sub-image that meets the following characteristics for the first time. Image as weld edge:
a)该子图像的沿焊缝的两个方向的8领域内存在属于焊缝区域的点;a) There are points belonging to the weld area within the 8 domains along the two directions of the weld in the sub-image;
b)在以该子图像为中心的一个5×3(焊缝纵向×焊缝横向)图像区域内,属于焊缝区域的子图像的数量占有大于50%的比例。将两侧所有的焊缝边缘分别连接起来,即形成焊缝的两条边缘。b) In a 5×3 (welding seam longitudinal×welding seam transverse) image area centered on the sub-image, the number of sub-images belonging to the weld seam area accounts for more than 50%. Connect all the weld edges on both sides respectively to form the two edges of the weld.
根据上述原则,如图4所示,图中最左边的3个被标识为黑色的焊缝区域不满足特点(1)和特点(2),故应改为母材区域;图中最右侧突出的两个子图像区域由于不满足特点(2)故也改为母材区域;之后将余下子图像的最左端作为焊缝的左侧边缘,最右端作为焊缝的右侧边缘,得到如图5所示的焊缝边缘。According to the above principles, as shown in Figure 4, the three weld areas marked as black on the far left in the figure do not meet the characteristics (1) and (2), so they should be changed to the base metal area; the far right in the figure The two prominent sub-image areas are also changed to the base metal area because they do not satisfy the characteristic (2); then the leftmost end of the remaining sub-images is used as the left edge of the weld, and the rightmost end is used as the right edge of the weld, as shown in Fig. 5 shows the edge of the weld.
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN2007101758596ACN101135652B (en) | 2007-10-15 | 2007-10-15 | Weld Seam Recognition Method Based on Texture Segmentation |
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN2007101758596ACN101135652B (en) | 2007-10-15 | 2007-10-15 | Weld Seam Recognition Method Based on Texture Segmentation |
| Publication Number | Publication Date |
|---|---|
| CN101135652A CN101135652A (en) | 2008-03-05 |
| CN101135652Btrue CN101135652B (en) | 2011-03-30 |
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN2007101758596AExpired - Fee RelatedCN101135652B (en) | 2007-10-15 | 2007-10-15 | Weld Seam Recognition Method Based on Texture Segmentation |
| Country | Link |
|---|---|
| CN (1) | CN101135652B (en) |
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102749290B (en)* | 2012-07-02 | 2014-12-03 | 浙江大学 | Method for detecting growth state of branches of crown canopy of cherry tree |
| CN104865277B (en)* | 2015-05-11 | 2017-05-17 | 四川大学 | Method for automatically identifying pipe weld defect based on double-wall double-projection transillumination image |
| CN106482653A (en)* | 2015-08-28 | 2017-03-08 | 华硕电脑股份有限公司 | Optical measuring device and gap measuring method |
| CN107894217B (en)* | 2017-11-14 | 2020-04-10 | 中车长春轨道客车股份有限公司 | Laser stitch welding seam undercut amount detection method based on line structure light sensor |
| CN111300417B (en)* | 2020-03-12 | 2021-12-10 | 福建永越智能科技股份有限公司 | Welding path control method and device for welding robot |
| CN112489010A (en)* | 2020-11-27 | 2021-03-12 | 桂林电子科技大学 | Method and device for quickly identifying welding line and storage medium |
| CN112233116B (en)* | 2020-12-11 | 2021-08-03 | 常州微亿智造科技有限公司 | Concave-convex mark visual detection method based on neighborhood decision and gray level co-occurrence matrix description |
| CN113588692B (en)* | 2021-09-28 | 2021-12-10 | 南通皋亚钢结构有限公司 | Computer vision-based weld defect nondestructive testing method |
| CN114897773B (en)* | 2022-03-31 | 2024-01-05 | 上海途巽通讯科技有限公司 | Method and system for detecting distorted wood based on image processing |
| CN115839956B (en)* | 2023-02-17 | 2023-05-02 | 天津诺瑞信精密电子有限公司 | Product qualification detection method, device, system and readable storage medium |
| CN117444458B (en)* | 2023-12-21 | 2024-03-19 | 深圳杩蚁科技有限公司 | Automatic control method for welding robot |
| CN119131025B (en)* | 2024-11-12 | 2025-02-25 | 广东省特种设备检测研究院茂名检测院 | A pipeline weld identification method and system based on image processing and magnetic flux leakage detection |
| CN119477924B (en)* | 2025-01-16 | 2025-06-13 | 新疆天维无损检测有限公司 | Method, device, electronic equipment and storage medium for repeated X-ray weld film detection |
| CN120451688B (en)* | 2025-07-10 | 2025-09-19 | 陕西省特种设备检验检测研究院 | Construction method of weld defect grade evaluation system based on feature fusion |
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US5614109A (en)* | 1995-07-05 | 1997-03-25 | Cecil; Dimitrios G. | Welding line with quality control storage devices |
| CN1845175A (en)* | 2006-04-27 | 2006-10-11 | 上海交通大学 | Texture surface defect detection method based on wavelet and co-occurrence matrix |
| CN1924896A (en)* | 2006-09-14 | 2007-03-07 | 上海交通大学 | Partial image processing based butt type welding seam recognition method |
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US5614109A (en)* | 1995-07-05 | 1997-03-25 | Cecil; Dimitrios G. | Welding line with quality control storage devices |
| CN1845175A (en)* | 2006-04-27 | 2006-10-11 | 上海交通大学 | Texture surface defect detection method based on wavelet and co-occurrence matrix |
| CN1924896A (en)* | 2006-09-14 | 2007-03-07 | 上海交通大学 | Partial image processing based butt type welding seam recognition method |
| Title |
|---|
| 同上. |
| 同上.;黎明,马聪,杨小芹.机械加工零件表面纹理缺陷检测.中国图象图形学报9 3.2004,9(3),319-321.* |
| 黎明,马聪,杨小芹.机械加工零件表面纹理缺陷检测.中国图象图形学报9 3.2004,9(3),319-321. |
| Publication number | Publication date |
|---|---|
| CN101135652A (en) | 2008-03-05 |
| Publication | Publication Date | Title |
|---|---|---|
| CN101135652B (en) | Weld Seam Recognition Method Based on Texture Segmentation | |
| CN113160192B (en) | Visual sense-based snow pressing vehicle appearance defect detection method and device under complex background | |
| CN100507937C (en) | A Vision Method for Automatic Weld Seam Recognition Based on Local Image Texture Feature Matching | |
| CN109961049B (en) | Cigarette brand identification method under complex scene | |
| CN113506240B (en) | Metallographic grain boundary extraction and communication method based on deep learning | |
| CN113240623B (en) | Pavement disease detection method and device | |
| CN101854467B (en) | Method for adaptively detecting and eliminating shadow in video segmentation | |
| CN111310760B (en) | Oracle Bone Inscription Text Detection Method Combining Local Prior Features and Deep Convolution Features | |
| CN110490892A (en) | A kind of Thyroid ultrasound image tubercle automatic positioning recognition methods based on USFaster R-CNN | |
| CN106203342A (en) | Target identification method based on multi-angle local feature coupling | |
| CN114612456B (en) | A method for automatic semantic segmentation and recognition of steel billets based on deep learning | |
| CN110097524B (en) | SAR image target detection method based on fusion convolutional neural network | |
| CN110288538A (en) | A shadow detection and elimination method for moving objects based on multi-feature fusion | |
| CN103514608B (en) | Moving object detection based on movement attention fusion model and extracting method | |
| CN101770583B (en) | Template matching method based on global features of scene | |
| CN105205821A (en) | A Welding Image Segmentation Method | |
| CN113034454B (en) | Underwater image quality evaluation method based on human visual sense | |
| CN102393902A (en) | Vehicle color detection method based on H_S two-dimensional histogram and regional color matching | |
| CN106447673A (en) | Chip pin extraction method under non-uniform illumination condition | |
| CN110889840A (en) | Validity Detection Method of Gaofen-6 Remote Sensing Satellite Data Oriented to Ground Objects | |
| CN103761736B (en) | A kind of image partition method based on Bayes's harmony degree | |
| CN114863492A (en) | Method and device for repairing low-quality fingerprint image | |
| CN105631456A (en) | Particle swarm optimization ITTI model-based white cell region extraction method | |
| CN118967550B (en) | Lightweight insulator defect detection method based on improved YOLOv network | |
| CN117557542A (en) | A PCB surface defect detection method based on improved Faster RCNN deep learning network |
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| C14 | Grant of patent or utility model | ||
| GR01 | Patent grant | ||
| CF01 | Termination of patent right due to non-payment of annual fee | Granted publication date:20110330 Termination date:20161015 | |
| CF01 | Termination of patent right due to non-payment of annual fee |