技术领域technical field
本申请涉及图像处理技术领域，具体涉及一种面板缺陷检测方法、装置、存储介质、设备及程序产品。The present application relates to the technical field of image processing, in particular to a panel defect detection method, device, storage medium, equipment and program product.
背景技术Background technique
工业制造过程中，产品的缺陷检测准确率往往直接影响经济效益，现有智能技术通常是采用深度学习神经网络算法对产品缺陷进行判别，再通过人工对关键缺陷进行复判，以此减少人工成本。例如对于PCB面板的检测中，通过工业相机在其正上方拍摄图像，再将图像输入检测模型即可完成对缺陷的识别。In the industrial manufacturing process, the accuracy of product defect detection often directly affects economic benefits. Existing intelligent technologies usually use deep learning neural network algorithms to identify product defects, and then manually re-evaluate key defects to reduce labor costs. . For example, in the detection of PCB panels, an industrial camera can be used to capture images directly above them, and then input the images into the detection model to complete the identification of defects.
在PCB面板的生产制造中一般都是流水线式作业，PCB面板通过传送装置送到固定的拍摄区域，拍照完成后即送回传输装置进行下一步处理，而伴随着传输装置的运动、电机等驱动装置的工作，都会引起面板的振动，此外，即便是成品面板，也会进行振动测试，在振动的情况下不可避免地会造成拍摄的图像变模糊，以及图像尺寸相对标准无振动的图像产生变化，对其上缺陷的判别、定位造成了困难。In the production and manufacture of PCB panels, it is generally an assembly line operation. The PCB panel is sent to a fixed shooting area through the transmission device. After the photo is taken, it is sent back to the transmission device for the next step of processing. The operation of the device will cause the panel to vibrate. In addition, even the finished panel will be subjected to a vibration test. In the case of vibration, the captured image will inevitably become blurred, and the image size will change compared to the standard image without vibration. , causing difficulties in identifying and locating defects.
发明内容Contents of the invention
本申请的主要目的在于提供一种面板缺陷检测方法、装置、存储介质、设备及程序产品，旨在解决现有技术中对振动状态下的PCB面板上的缺陷检测较为困难的问题。The main purpose of the present application is to provide a panel defect detection method, device, storage medium, equipment and program product, aiming to solve the problem in the prior art that it is difficult to detect defects on PCB panels under vibration.
为实现上述目的，本申请的实施例采用的技术方案如下：In order to achieve the above object, the technical solutions adopted in the embodiments of the present application are as follows:
第一方面，本申请实施例提供一种面板缺陷检测方法，包括以下步骤：In the first aspect, the embodiment of the present application provides a panel defect detection method, including the following steps:
根据拉伸参数拉伸待检测图像，获得拉伸图像；其中，待检测图像基于面板产品的正投影方向获得，拉伸参数基于面板产品的振动参数获得，振动参数根据面板产品的振动方向信息及振动幅度信息获得；Stretch the image to be detected according to the stretching parameters to obtain the stretched image; wherein, the image to be detected is obtained based on the orthographic projection direction of the panel product, the stretching parameter is obtained based on the vibration parameter of the panel product, and the vibration parameter is obtained according to the vibration direction information of the panel product and Acquisition of vibration amplitude information;
根据拉伸参数拉伸待检测图像，获得拉伸图像之前，面板缺陷检测方法还包括：Stretching the image to be detected according to the stretching parameters, before obtaining the stretched image, the panel defect detection method also includes:
获取标准图像上的沿面板产品的振动方向分布的若干像素点；Obtain several pixel points distributed along the vibration direction of the panel product on the standard image;
根据面板产品的振动幅度信息，分别获得若干像素点的水平偏移量信息；According to the vibration amplitude information of the panel product, obtain the horizontal offset information of several pixels respectively;
根据若干像素点在振动方向上的相对距离以及若干像素点的水平偏移量信息，构建像素点的水平偏移点图；According to the relative distance of several pixels in the vibration direction and the horizontal offset information of several pixels, construct the horizontal offset point map of the pixels;
对水平偏移点图上的点进行拟合，获得水平偏移曲线图；Fit the points on the horizontal offset point diagram to obtain the horizontal offset curve diagram;
根据水平偏移曲线图，获得拉伸参数；Obtain the stretching parameters according to the horizontal offset graph;
根据拉伸参数拉伸待检测图像，获得拉伸图像，包括：Stretch the image to be detected according to the stretching parameters to obtain the stretched image, including:
根据拉伸参数，基于像素点拉伸待检测图像，获得拉伸图像；Stretch the image to be detected based on the pixel points according to the stretching parameters to obtain the stretched image;
对拉伸图像进行去模糊处理，获得目标图像；Deblurring the stretched image to obtain the target image;
将目标图像输入缺陷检测模型，获得缺陷信息；其中，缺陷检测模型包括卷积层，卷积层的卷积核，基于若干并行的分支卷积层的卷积核融合获得。Input the target image into the defect detection model to obtain defect information; wherein, the defect detection model includes a convolution layer, and the convolution kernel of the convolution layer is obtained based on the fusion of convolution kernels of several parallel branch convolution layers.
通过将面板产品的振动参数引入对待检测图像的矫正过程，利用根据振动参数获得的拉伸参数先将待检测图像的尺寸进行矫正，然后再利用去模糊处理将图像变得清晰，以便于缺陷检测模型的识别，由于缺陷检测模型的卷积层的卷积核是通过若干并行的分支卷积层的卷积核融合获得，模型没有增加额外的计算量，还能增强特征提取的能力，从而提升模型的分割精度与泛化能力，由于前述步骤的拉伸消除了振动带来的尺寸偏差，因此检测图像获得的缺陷信息能够与实际面板上的缺陷信息高度匹配，实现了对于振动情况下面板上缺陷的准确提取。By introducing the vibration parameters of the panel product into the correction process of the image to be detected, the stretching parameters obtained according to the vibration parameters are used to first correct the size of the image to be detected, and then the image is cleared by deblurring to facilitate defect detection For model identification, since the convolution kernel of the convolution layer of the defect detection model is obtained through the fusion of convolution kernels of several parallel branch convolution layers, the model does not add additional calculations, and it can also enhance the ability of feature extraction, thereby improving The segmentation accuracy and generalization ability of the model, because the stretching of the previous steps eliminates the dimensional deviation caused by vibration, the defect information obtained from the detection image can be highly matched with the defect information on the actual panel, realizing the control of the panel under the condition of vibration. Accurate extraction of defects.
在第一方面的一种可能实现方式中，根据拉伸参数拉伸待检测图像，获得拉伸图像之前，面板缺陷检测方法还包括：In a possible implementation of the first aspect, the image to be detected is stretched according to the stretching parameters, and before the stretched image is obtained, the panel defect detection method further includes:
根据面板产品的振动方向信息及振动幅度信息，获得振动参数。Vibration parameters are obtained according to the vibration direction information and vibration amplitude information of the panel product.
通过将振动参数引入并与拉伸参数相匹配，再根据拉伸参数去将待检测图像做还原拉伸，实现对其尺寸的矫正，实际情况中，振动源不足以让面板产品振动并发生翻转，从空间坐标系角度来说，面板产品发生的振动，都可以分解到X轴、Y轴方轴的转动，也即拍摄的待检测图像当时的状态，可以通过分别在X轴、Y轴方向翻转得到。而其中的振动参数则可以通过检测样品的振动情况获得，包括了振动方向信息与振动幅度信息，对应产品拍摄时的状态可以理解为其翻转的方向与翻转的角度，振幅越大说明在该振动方向上面板产品具有更大角度的翻转。By introducing the vibration parameters and matching them with the stretching parameters, and then restoring and stretching the image to be detected according to the stretching parameters, the size correction is realized. In actual situations, the vibration source is not enough to make the panel product vibrate and flip , from the perspective of the space coordinate system, the vibration of the panel product can be decomposed into the rotation of the X-axis and the Y-axis, that is, the state of the captured image to be detected at that time, which can be measured in the directions of the X-axis and Y-axis respectively. Flip to get. The vibration parameters can be obtained by detecting the vibration of the sample, including vibration direction information and vibration amplitude information. Orientation upper panel products have a greater angle of flipping.
在第一方面的一种可能实现方式中，将目标图像输入缺陷检测模型，获得缺陷信息之前，面板缺陷检测方法还包括：In a possible implementation manner of the first aspect, the target image is input into the defect detection model, and before defect information is obtained, the panel defect detection method further includes:
获取三组并行的分支卷积层；其中，三组分支卷积层的卷积核尺寸分别为3*3、1*3以及3*1，卷积核尺寸为3*3的分支卷积层采用分布位移卷积，卷积核尺寸为1*3与3*1的分支卷积层采用分组卷积；Obtain three sets of parallel branch convolution layers; among them, the convolution kernel sizes of the three sets of branch convolution layers are 3*3, 1*3, and 3*1, respectively, and the branch convolution layer with a convolution kernel size of 3*3 Distributed displacement convolution is adopted, and the branch convolution layer with the convolution kernel size of 1*3 and 3*1 adopts group convolution;
将三组分支卷积层的卷积核融合为3*3尺寸的卷积核，以获得卷积层的卷积核。The convolution kernels of the three groups of branch convolution layers are fused into a 3*3 size convolution kernel to obtain the convolution kernel of the convolution layer.
以增强网络特征提取能力，使用非对称混洗卷积结构的三层并行分支共同提取图像特征，使用3*3、1*3以及3*1的卷积层代替原有的3*3卷积，其中，3*3的分支卷积层采用分布位移卷积，1*3与3*1的分支卷积层采用分组卷积，3*3的卷积层主要用于增大感受野以获取更丰富的特征信息，而分组卷积层主要用于提升模型对翻转或旋转的泛化能力，对空间信息的依赖性降低，可大幅度减小模型参数。In order to enhance the ability of network feature extraction, the three-layer parallel branch of the asymmetric shuffled convolution structure is used to extract image features, and the convolution layer of 3*3, 1*3 and 3*1 is used to replace the original 3*3 convolution , where the 3*3 branch convolution layer uses distributed displacement convolution, the 1*3 and 3*1 branch convolution layers use group convolution, and the 3*3 convolution layer is mainly used to increase the receptive field to obtain Richer feature information, while the group convolution layer is mainly used to improve the generalization ability of the model for flipping or rotating, and the dependence on spatial information is reduced, which can greatly reduce the model parameters.
在第一方面的一种可能实现方式中，缺陷检测模型还包括通道混洗层，通道混洗层位于采用分组卷积的分支卷积层之后，通道混洗层用于：In a possible implementation manner of the first aspect, the defect detection model further includes a channel shuffling layer, the channel shuffling layer is located after the branch convolution layer using group convolution, and the channel shuffling layer is used for:
对分支卷积层提取的特征图进行分组，获得分组矩阵；Group the feature maps extracted by the branch convolution layer to obtain a grouping matrix;
转置分组矩阵，获得转置矩阵；Transpose the grouping matrix to obtain the transposed matrix;
将转置矩阵平坦化重分组，获得目标特征图。Flatten and regroup the transposed matrix to obtain the target feature map.
分组卷积会阻碍通道间信息流动，因此设置通道混洗层，将其放在分组卷积层之后，能够有效提高分组卷积中不同通道间的关联性。Group convolution will hinder the flow of information between channels, so setting the channel shuffling layer and placing it after the group convolution layer can effectively improve the correlation between different channels in group convolution.
在第一方面的一种可能实现方式中，缺陷检测模型还包括中间层，中间层基于缺陷检测模型的特征金字塔网络的特征层获得。In a possible implementation manner of the first aspect, the defect detection model further includes an intermediate layer, and the intermediate layer is obtained based on a feature layer of a feature pyramid network of the defect detection model.
由于特征金字塔中的映射在X轴、Y轴方向上都被缩放，并且在相邻两层之间的尺度差距很大，导致两个大小相似的物体被预测并分到不同的层，因此通过在缺陷检测模型的特征金字塔网络的特征层中生成中间层来解决，使得不同尺寸特征图之间的过渡更为平滑，从而提升检测的效果。Since the mapping in the feature pyramid is scaled in the X-axis and Y-axis directions, and the scale gap between two adjacent layers is large, two objects of similar size are predicted and assigned to different layers, so by The intermediate layer is generated in the feature layer of the feature pyramid network of the defect detection model to solve the problem, making the transition between feature maps of different sizes smoother, thereby improving the detection effect.
在第一方面的一种可能实现方式中，将目标图像输入缺陷检测模型，获得缺陷信息之前，面板缺陷检测方法还包括：In a possible implementation manner of the first aspect, the target image is input into the defect detection model, and before defect information is obtained, the panel defect detection method further includes:
分别对缺陷检测模型的特征金字塔网络的特征层进行线性缩放，获得若干第一特征层；Respectively linearly scale the feature layers of the feature pyramid network of the defect detection model to obtain several first feature layers;
将第一特征层进行逐像素相加，获得第二特征层；Adding the first feature layer pixel by pixel to obtain the second feature layer;
对第二特征层进行卷积与融合，获得中间层。Convolve and fuse the second feature layer to obtain the middle layer.
将特征金字塔网络中每一层的相邻层按一定系数进行线性缩放，并将所得结果，即第一特征层进行逐像素相加得到第二特征层，第二特征层通过3*3卷积，从原始层中对特征进行融合，生成中间层，由此使得特征金字塔中相邻层之间都有一个中间层进行过渡，减少尺寸截断的影响，提高模型性能，进而提升缺陷提取的准确。The adjacent layers of each layer in the feature pyramid network are linearly scaled by a certain coefficient, and the result, that is, the first feature layer is added pixel by pixel to obtain the second feature layer, and the second feature layer is convolved by 3*3 , the features are fused from the original layer to generate an intermediate layer, so that there is an intermediate layer for transition between adjacent layers in the feature pyramid, reducing the impact of size truncation, improving model performance, and improving the accuracy of defect extraction.
第二方面，本申请实施例提供一种面板缺陷检测装置，包括：In the second aspect, the embodiment of the present application provides a panel defect detection device, including:
拉伸模块，拉伸模块用于根据拉伸参数拉伸待检测图像，获得拉伸图像；其中，待检测图像基于面板产品的正投影方向获得，拉伸参数基于面板产品的振动参数获得，振动参数根据面板产品的振动方向信息及振动幅度信息获得；Stretching module, the stretching module is used to stretch the image to be detected according to the stretching parameters to obtain the stretched image; wherein, the image to be detected is obtained based on the orthographic projection direction of the panel product, the stretching parameters are obtained based on the vibration parameters of the panel product, and the vibration The parameters are obtained according to the vibration direction information and vibration amplitude information of the panel product;
拉伸模块还用于在根据拉伸参数拉伸待检测图像，获得拉伸图像之前，获取标准图像上的沿面板产品的振动方向分布的若干像素点；The stretching module is also used to stretch the image to be detected according to the stretching parameters to obtain a number of pixels distributed along the vibration direction of the panel product on the standard image before obtaining the stretched image;
根据面板产品的振动幅度信息，分别获得若干像素点的水平偏移量信息；According to the vibration amplitude information of the panel product, obtain the horizontal offset information of several pixels respectively;
根据若干像素点在振动方向上的相对距离以及若干像素点的水平偏移量信息，构建像素点的水平偏移点图；According to the relative distance of several pixels in the vibration direction and the horizontal offset information of several pixels, construct the horizontal offset point map of the pixels;
对水平偏移点图上的点进行拟合，获得水平偏移曲线图；Fit the points on the horizontal offset point diagram to obtain the horizontal offset curve diagram;
根据水平偏移曲线图，获得拉伸参数；Obtain the stretching parameters according to the horizontal offset graph;
根据拉伸参数拉伸待检测图像，获得拉伸图像，包括：Stretch the image to be detected according to the stretching parameters to obtain the stretched image, including:
根据拉伸参数，基于像素点拉伸待检测图像，获得拉伸图像；Stretch the image to be detected based on the pixel points according to the stretching parameters to obtain the stretched image;
去模糊模块，去模糊模块用于对拉伸图像进行去模糊处理，获得目标图像；A deblurring module, the deblurring module is used for deblurring the stretched image to obtain the target image;
检测模块，检测模块用于将目标图像输入缺陷检测模型，获得缺陷信息；其中，缺陷检测模型包括卷积层，卷积层的卷积核，基于若干并行的分支卷积层的卷积核融合获得。Detection module, the detection module is used to input the target image into the defect detection model to obtain defect information; wherein, the defect detection model includes a convolution layer, a convolution kernel of the convolution layer, and a convolution kernel fusion based on several parallel branch convolution layers get.
第三方面，本申请实施例提供一种计算机可读存储介质，储存有计算机程序，计算机程序被处理器加载执行时，实现如上述第一方面中任一项提供的面板缺陷检测方法。In a third aspect, an embodiment of the present application provides a computer-readable storage medium storing a computer program, and when the computer program is loaded and executed by a processor, the panel defect detection method provided in any one of the above-mentioned first aspects is implemented.
第四方面，本申请实施例提供一种电子设备，包括处理器及存储器，其中，In a fourth aspect, the embodiment of the present application provides an electronic device, including a processor and a memory, wherein,
存储器用于存储计算机程序；memory for storing computer programs;
处理器用于加载执行计算机程序，以使电子设备执行如上述第一方面中任一项提供的面板缺陷检测方法。The processor is used to load and execute a computer program, so that the electronic device executes the panel defect detection method provided in any one of the first aspects above.
与现有技术相比，本申请的有益效果是：Compared with prior art, the beneficial effect of the present application is:
本申请实施例提出的一种面板缺陷检测方法、装置、存储介质、设备及程序产品，该方法包括：根据拉伸参数拉伸待检测图像，获得拉伸图像；其中，待检测图像基于面板产品的正投影方向获得，拉伸参数基于面板产品的振动参数获得；对拉伸图像进行去模糊处理，获得目标图像；将目标图像输入缺陷检测模型，获得缺陷信息；其中，缺陷检测模型包括卷积层，卷积层的卷积核，基于若干并行的分支卷积层的卷积核融合获得。本申请的方法通过将面板产品的振动参数引入对待检测图像的矫正过程，利用根据振动参数获得的拉伸参数先将待检测图像的尺寸进行矫正，然后再利用去模糊处理将图像变得清晰，以便于缺陷检测模型的识别，由于缺陷检测模型的卷积层的卷积核是通过若干并行的分支卷积层的卷积核融合获得，模型没有增加额外的计算量，还能增强特征提取的能力，从而提升模型的分割精度与泛化能力，由于前述步骤的拉伸消除了振动带来的尺寸偏差，因此检测图像获得的缺陷信息能够与实际面板上的缺陷信息高度匹配，实现了对于振动情况下面板上缺陷的准确提取。A panel defect detection method, device, storage medium, equipment and program product proposed in the embodiment of the present application, the method includes: stretching the image to be detected according to the stretching parameters to obtain the stretched image; wherein the image to be detected is based on the panel product The orthographic projection direction is obtained, and the stretching parameters are obtained based on the vibration parameters of the panel product; the stretched image is deblurred to obtain the target image; the target image is input into the defect detection model to obtain defect information; wherein, the defect detection model includes convolution Layer, the convolution kernel of the convolution layer, is obtained based on the fusion of convolution kernels of several parallel branch convolution layers. The method of the present application introduces the vibration parameters of the panel product into the correction process of the image to be detected, uses the stretching parameters obtained according to the vibration parameters to first correct the size of the image to be detected, and then uses deblurring processing to make the image clear, In order to facilitate the identification of the defect detection model, since the convolution kernel of the convolution layer of the defect detection model is obtained through the fusion of convolution kernels of several parallel branch convolution layers, the model does not add additional calculations, and it can also enhance feature extraction. ability, thereby improving the segmentation accuracy and generalization ability of the model. Since the stretching in the preceding steps eliminates the dimensional deviation caused by vibration, the defect information obtained from the detection image can be highly matched with the defect information on the actual panel, and the vibration Accurate extraction of defects on the panel under certain conditions.
附图说明Description of drawings
图1为本申请实施例涉及的硬件运行环境的电子设备结构示意图；FIG. 1 is a schematic structural diagram of an electronic device in a hardware operating environment involved in an embodiment of the present application;
图2为本申请实施例提供的面板缺陷检测方法的流程示意图；FIG. 2 is a schematic flow diagram of a panel defect detection method provided in an embodiment of the present application;
图3为本申请实施例提供的面板缺陷检测方法中在一种振动情况下面板产品的示意图；3 is a schematic diagram of a panel product under a vibration condition in the panel defect detection method provided by the embodiment of the present application;
图4为本申请实施例提供的面板缺陷检测方法中单个位置点的水平偏移示意图；FIG. 4 is a schematic diagram of the horizontal offset of a single position point in the panel defect detection method provided by the embodiment of the present application;
图5为本申请实施例提供的面板缺陷检测方法中的水平偏移的点图；FIG. 5 is a dot diagram of horizontal offset in the panel defect detection method provided by the embodiment of the present application;
图6为本申请实施例提供的面板缺陷检测方法中的水平偏移曲线图；Fig. 6 is a horizontal offset curve in the panel defect detection method provided by the embodiment of the present application;
图7为本申请实施例提供的面板缺陷检测装置的模块示意图；FIG. 7 is a schematic diagram of modules of a panel defect detection device provided in an embodiment of the present application;
图中标记：101-处理器，102-通信总线，103-网络接口，104-用户接口，105-存储器。Marks in the figure: 101-processor, 102-communication bus, 103-network interface, 104-user interface, 105-memory.
具体实施方式Detailed ways
应当理解，此处所描述的具体实施例仅仅用以解释本申请，并不用于限定本申请。It should be understood that the specific embodiments described here are only used to explain the present application, and are not intended to limit the present application.
本申请实施例的主要解决方案是：提出一种面板缺陷检测方法、装置、存储介质、设备及程序产品，该方法包括：根据拉伸参数拉伸待检测图像，获得拉伸图像；其中，待检测图像基于面板产品的正投影方向获得，拉伸参数基于面板产品的振动参数获得；对拉伸图像进行去模糊处理，获得目标图像；将目标图像输入缺陷检测模型，获得缺陷信息；其中，缺陷检测模型包括卷积层，卷积层的卷积核，基于若干并行的分支卷积层的卷积核融合获得。The main solution of the embodiment of the present application is to propose a panel defect detection method, device, storage medium, equipment and program product, the method includes: stretching the image to be detected according to the stretching parameters to obtain the stretched image; The detection image is obtained based on the orthographic projection direction of the panel product, and the stretching parameters are obtained based on the vibration parameters of the panel product; the stretched image is deblurred to obtain the target image; the target image is input into the defect detection model to obtain defect information; among them, the defect The detection model includes a convolutional layer, a convolutional kernel of the convolutional layer, which is obtained based on the fusion of convolutional kernels of several parallel branch convolutional layers.
工业制造过程中，产品的缺陷检测准确率往往直接影响经济效益，现有智能技术通常是采用深度学习神经网络算法对产品缺陷进行判别，再通过人工对关键缺陷进行复判，以此减少人工成本。例如对于PCB面板的检测中，通过工业相机在其正上方拍摄图像，再将图像输入检测模型即可完成对缺陷的识别。但是，由于实际生产环境中各种不确定的干扰因素较多，产品自身背景也较为复杂，因此深度学习算法对此类工业产品、密集多分类的小目标缺陷检查效果还有待提升。In the industrial manufacturing process, the accuracy of product defect detection often directly affects economic benefits. Existing intelligent technologies usually use deep learning neural network algorithms to identify product defects, and then manually re-evaluate key defects to reduce labor costs. . For example, in the detection of PCB panels, an industrial camera can be used to capture images directly above them, and then input the images into the detection model to complete the identification of defects. However, due to the many uncertain interference factors in the actual production environment and the complex background of the product itself, the effect of deep learning algorithms on such industrial products and dense multi-category small target defects needs to be improved.
在深度神经网络的使用中，将图像进行翻转或旋转后，k×k卷积层提取的图像特征会产生变化，模型对同一目标的识别结果可能会产生偏差，因此导致模型的旋转、翻转目标的泛化能力降低；特征金字塔中的特征图可以在不同尺度上捕捉物体的视觉特征，浅层保留了细节，如纹理边缘、角落等，深层覆盖了更抽象的语义信息，在真实生产环境中，不同尺寸的多个目标通常出现在一起，如何在一张图上对这些不同目标进行识别是一个关键问题，但是常规金字塔网络在X轴和Y轴方向上的映射尺度在相邻层的差距较大，通常成倍缩小，因此尺度相似的目标可能被分类到不同预测层上，出现尺度截断问题，造成预测框定位不准和分类精度降低问题。In the use of deep neural network, after the image is flipped or rotated, the image features extracted by the k×k convolutional layer will change, and the recognition result of the model for the same target may be biased, thus causing the model to rotate and flip the target. The generalization ability of the feature pyramid is reduced; the feature map in the feature pyramid can capture the visual features of the object at different scales, the shallow layer retains details, such as texture edges, corners, etc., and the deep layer covers more abstract semantic information. In the real production environment , Multiple targets of different sizes usually appear together. How to identify these different targets on a single map is a key issue, but the mapping scale of the conventional pyramid network in the X-axis and Y-axis directions is in the gap between adjacent layers Larger, usually doubled, so objects with similar scales may be classified into different prediction layers, resulting in the problem of scale truncation, resulting in inaccurate positioning of the prediction frame and reduced classification accuracy.
在PCB面板的生产制造中一般都是流水线式作业，PCB面板通过传送装置送到固定的拍摄区域，拍照完成后即送回传输装置进行下一步处理，而伴随着传输装置的运动、电机等驱动装置的工作，都会引起面板的振动，此外，即便是成品面板，也会进行振动测试，在振动的情况下不可避免地会造成拍摄的图像变模糊，以及图像尺寸相对标准无振动的图像产生变化，对其上缺陷的判别、定位造成了困难。In the production and manufacture of PCB panels, it is generally an assembly line operation. The PCB panel is sent to a fixed shooting area through the transmission device. After the photo is taken, it is sent back to the transmission device for the next step of processing. The operation of the device will cause the panel to vibrate. In addition, even the finished panel will be subjected to a vibration test. In the case of vibration, the captured image will inevitably become blurred, and the image size will change compared to the standard image without vibration. , causing difficulties in identifying and locating defects.
为此，本申请提供一种解决方案，通过将面板产品的振动参数引入对待检测图像的矫正过程，利用根据振动参数获得的拉伸参数先将待检测图像的尺寸进行矫正，然后再利用去模糊处理将图像变得清晰，以便于缺陷检测模型的识别，由于缺陷检测模型的卷积层的卷积核是通过若干并行的分支卷积层的卷积核融合获得，模型没有增加额外的计算量，还能增强特征提取的能力，从而提升模型的分割精度与泛化能力，由于前述步骤的拉伸消除了振动带来的尺寸偏差，因此检测图像获得的缺陷信息能够与实际面板上的缺陷信息高度匹配，实现了对于振动情况下面板上缺陷的准确提取。To this end, this application provides a solution. By introducing the vibration parameters of the panel product into the correction process of the image to be detected, the stretching parameters obtained according to the vibration parameters are used to first correct the size of the image to be detected, and then use the deblurring The processing makes the image clear to facilitate the recognition of the defect detection model. Since the convolution kernel of the convolution layer of the defect detection model is obtained through the fusion of convolution kernels of several parallel branch convolution layers, the model does not add additional calculations , can also enhance the ability of feature extraction, thereby improving the segmentation accuracy and generalization ability of the model. Since the stretching of the previous steps eliminates the size deviation caused by vibration, the defect information obtained from the detection image can be compared with the defect information on the actual panel. Highly matched, the accurate extraction of defects on the panel under vibration conditions is realized.
参照附图1，附图1为本申请实施例方案涉及的硬件运行环境的电子设备结构示意图，该电子设备可以包括：处理器101，例如中央处理器（Central Processing Unit，CPU），通信总线102、用户接口104，网络接口103，存储器105。其中，通信总线102用于实现这些组件之间的连接通信。用户接口104可以包括显示屏（Display）、输入单元比如键盘（Keyboard），可选用户接口104还可以包括标准的有线接口、无线接口。网络接口103可选的可以包括标准的有线接口、无线接口（如无线保真（WIreless-FIdelity，WI-FI）接口）。存储器105可选的可以是独立于前述处理器101的存储装置，存储器105可能是高速的随机存取存储器（Random Access Memory，RAM）存储器，也可能是稳定的非易失性存储器（Non-Volatile Memory，NVM），例如至少一个磁盘存储器；处理器101可以是通用处理器，包括中央处理器、网络处理器等，还可以是数字信号处理器、专用集成电路、现场可编程门阵列或者其他可编程逻辑器件、分立门或者晶体管逻辑器件、分立硬件组件。Referring to accompanying drawing 1, accompanying drawing 1 is the schematic structural diagram of the electronic device of the hardware operating environment involved in the embodiment of the present application, the electronic device may include: a processor 101, such as a central processing unit (Central Processing Unit, CPU), a communication bus 102 , user interface 104, network interface 103, storage 105. Wherein, the communication bus 102 is used to realize connection and communication between these components. The user interface 104 may include a display screen (Display), an input unit such as a keyboard (Keyboard), and the optional user interface 104 may also include a standard wired interface and a wireless interface. Optionally, the network interface 103 may include a standard wired interface and a wireless interface (such as a wireless fidelity (WIreless-FIdelity, WI-FI) interface). The memory 105 may optionally be a storage device independent of the aforementioned processor 101, and the memory 105 may be a high-speed random access memory (Random Access Memory, RAM) memory, or a stable non-volatile memory (Non-Volatile Memory, NVM), such as at least one disk memory; the processor 101 can be a general-purpose processor, including a central processing unit, a network processor, etc., and can also be a digital signal processor, an application-specific integrated circuit, a field programmable gate array, or other possible Program logic devices, discrete gate or transistor logic devices, discrete hardware components.
本领域技术人员可以理解，附图1中示出的结构并不构成对电子设备的限定，可以包括比图示更多或更少的部件，或者组合某些部件，或者不同的部件布置。Those skilled in the art can understand that the structure shown in FIG. 1 does not constitute a limitation to the electronic device, and may include more or less components than shown in the figure, or combine some components, or arrange different components.
如附图1所示，作为一种存储介质的存储器105中可以包括操作系统、网络通信模块、用户接口模块以及面板缺陷检测装置。As shown in FIG. 1 , the memory 105 as a storage medium may include an operating system, a network communication module, a user interface module, and a panel defect detection device.
在附图1所示的电子设备中，网络接口103主要用于与网络服务器进行数据通信；用户接口104主要用于与用户进行数据交互；本申请中的处理器101、存储器105可以设置在电子设备中，电子设备通过处理器101调用存储器105中存储的面板缺陷检测装置，并执行本申请实施例提供的面板缺陷检测方法。In the electronic device shown in accompanying drawing 1, the network interface 103 is mainly used for data communication with the network server; the user interface 104 is mainly used for data interaction with the user; In the device, the electronic device invokes the panel defect detection device stored in the memory 105 through the processor 101, and executes the panel defect detection method provided by the embodiment of the present application.
参照附图2，基于前述实施例的硬件设备，本申请的实施例提供一种面板缺陷检测方法，包括以下步骤：Referring to accompanying drawing 2, based on the hardware equipment of foregoing embodiment, the embodiment of the present application provides a kind of panel defect detection method, comprises the following steps:
S10：根据拉伸参数拉伸待检测图像，获得拉伸图像；其中，待检测图像基于面板产品的正投影方向获得，拉伸参数基于面板产品的振动参数获得。S10: Stretch the image to be inspected according to the stretching parameters to obtain the stretched image; wherein, the image to be inspected is obtained based on the orthographic projection direction of the panel product, and the stretching parameters are obtained based on the vibration parameters of the panel product.
在具体实施过程中，待检测图像为拍摄需要进行缺陷检测的面板产品获取的图像，比如将工业相机固定设置在某一工位，面板产品即通过传输装置输送到该工位下的正投影方向，获得正视方向的面板产品的图像，理论上来说，在不存在人为干扰、振动的情况，面板位置不会存在偏移，拍摄同种面板产品获取的待检测图像尺寸应当是一致的。但在振动的情况下，面板的各个方向均可存在相对水平的起伏，导致了从正投影方向拍摄获取的待检测图像尺寸发生变化。以一个方向的变化为例，如果面板产品做左右方向的振动，相当于在左右方向上对面板做翻转，而在一个方向的翻转下，待检测图像可以是被认为向这个方向进行了压缩，而其上的缺陷也随之变化，导致难以对缺陷实现准确的提取。In the specific implementation process, the image to be inspected is the image obtained by shooting the panel product that needs to be inspected for defects. For example, the industrial camera is fixed at a certain station, and the panel product is transported to the orthographic projection direction under the station through the transmission device. , to obtain the image of the panel product in the front view direction. In theory, in the absence of human interference and vibration, there will be no offset in the position of the panel, and the size of the image to be inspected obtained by shooting the same panel product should be consistent. However, in the case of vibration, there may be relative horizontal fluctuations in all directions of the panel, resulting in a change in the size of the image to be detected captured from the front projection direction. Taking a change in one direction as an example, if the panel product vibrates in the left and right directions, it is equivalent to flipping the panel in the left and right directions, and under the flip in one direction, the image to be detected can be considered to be compressed in this direction. The defects on it also change accordingly, making it difficult to accurately extract the defects.
因此，通过将振动参数引入并与拉伸参数相匹配，再根据拉伸参数去将待检测图像做还原拉伸，实现对其尺寸的矫正，即：根据拉伸参数拉伸待检测图像，获得拉伸图像之前，面板缺陷检测方法还包括：Therefore, by introducing the vibration parameters and matching them with the stretching parameters, and then stretching the image to be detected according to the stretching parameters, the correction of its size is realized, that is, stretching the image to be detected according to the stretching parameters, and obtaining Before stretching the image, the panel defect detection method also includes:
根据面板产品的振动方向信息及振动幅度信息，获得振动参数。Vibration parameters are obtained according to the vibration direction information and vibration amplitude information of the panel product.
在具体实施过程中，实际情况中，振动源不足以让面板产品振动并发生翻转，从空间坐标系角度来说，面板产品发生的振动，都可以分解到X轴、Y轴方向的转动，也即拍摄的待检测图像当时的状态，可以通过分别在X轴、Y轴方向翻转得到。而其中的振动参数则可以通过检测样品的振动情况获得，包括了振动方向信息与振动幅度信息，对应产品拍摄时的状态可以理解为其翻转的方向与翻转的角度，振幅越大说明在该振动方向上面板产品具有更大角度的翻转。那么拉伸就可以根据振动来决定，即，根据拉伸参数拉伸待检测图像，获得拉伸图像之前，面板缺陷检测方法还包括：In the specific implementation process, in the actual situation, the vibration source is not enough to cause the panel product to vibrate and turn over. From the perspective of the space coordinate system, the vibration of the panel product can be decomposed into the rotation of the X-axis and Y-axis. That is, the current state of the captured image to be detected can be obtained by flipping in the X-axis and Y-axis directions respectively. The vibration parameters can be obtained by detecting the vibration of the sample, including vibration direction information and vibration amplitude information. Orientation upper panel products have a greater angle of flipping. Then the stretching can be determined according to the vibration, that is, the image to be detected is stretched according to the stretching parameters. Before obtaining the stretched image, the panel defect detection method also includes:
分别根据振动方向信息及振动幅度信息，获得拉伸方向信息及拉伸长度信息；Obtain stretching direction information and stretching length information according to vibration direction information and vibration amplitude information respectively;
根据拉伸方向信息及拉伸长度信息，获得拉伸参数。The stretching parameters are obtained according to the stretching direction information and the stretching length information.
在具体实施过程中，拉伸方向、拉伸长度分别与振动方向、振动幅度对应，以面板在水平面上发生向左的振动时拍摄了待检测图像，向左振动，也即面板的右侧抬升，以左侧为转动中心发生翻转，从拍摄的图像上看，图像相对无振动的情况向左压缩，而拉伸方向则是使其恢复，应当是向右拉伸，而振幅越大，翻转的角度越大，图像被压缩的程度就越大，由此就可以决定图像需要拉伸多少才能恢复，当其拉伸至与无振动时拍摄的图像大小一致时，即可认为完成拉伸，实现了尺寸的矫正。即：根据拉伸参数拉伸待检测图像，获得拉伸图像，包括：In the specific implementation process, the stretching direction and stretching length correspond to the vibration direction and vibration amplitude respectively. When the panel vibrates to the left on the horizontal plane, the image to be detected is taken, vibrating to the left, that is, the right side of the panel is lifted , flipped with the left side as the rotation center. From the captured image, the image is compressed to the left when there is no vibration, and the stretching direction is to restore it. It should be stretched to the right, and the larger the amplitude, the flipped The larger the angle of the image is, the greater the degree of compression of the image can be determined. From this, it can be determined how much the image needs to be stretched to recover. When it is stretched to the same size as the image taken without vibration, the stretching can be considered complete. A size correction is achieved. That is: Stretch the image to be detected according to the stretching parameters to obtain the stretched image, including:
根据拉伸方向信息，由边缘拉伸待检测图像，直至满足拉伸长度信息，获得拉伸图像。According to the stretching direction information, the image to be detected is stretched from the edge until the stretching length information is satisfied, and the stretched image is obtained.
在具体实施过程中，前述仅仅是以单一方向为例说明，实际情况中振动可能是多方向的，但是基于前面的论述可知，多方向的也可以是逐步分解为单一方向，因此拉伸可以基于图像的边缘进行，由边缘拉伸待检测图像直至边缘都拉伸至满足拉伸长度信息的位置后，说明图像的尺寸均被矫正。In the specific implementation process, the foregoing is only an example of a single direction. In actual situations, the vibration may be multi-directional. However, based on the previous discussion, it can be known that multi-directional vibrations can also be gradually decomposed into a single direction. Therefore, stretching can be based on The edge of the image is stretched. The image to be detected is stretched from the edge until the edge is stretched to a position that satisfies the stretched length information, indicating that the size of the image has been corrected.
在一种实施例中，根据拉伸方向信息，由边缘拉伸待检测图像，直至满足拉伸长度信息，获得拉伸图像之前，面板缺陷检测方法还包括：In one embodiment, according to the stretching direction information, the image to be detected is stretched from the edge until the stretch length information is satisfied, and before the stretched image is obtained, the panel defect detection method further includes:
将待检测图像的任一边缘与标准图像的对应边缘重合；Overlap any edge of the image to be detected with the corresponding edge of the standard image;
根据拉伸方向信息，由边缘拉伸待检测图像，直至满足拉伸长度信息，获得拉伸图像，包括：According to the stretching direction information, the image to be detected is stretched from the edge until the stretching length information is satisfied, and the stretched image is obtained, including:
根据拉伸方向信息，先拉伸待检测图像未与标准图像重合的边缘，后拉伸待检测图像与标准图像重合的边缘，直至满足拉伸长度信息，获得拉伸图像。According to the stretching direction information, the edge of the image to be detected that does not overlap with the standard image is first stretched, and then the edge of the image to be detected that overlaps with the standard image is stretched until the stretching length information is satisfied, and the stretched image is obtained.
在具体实施过程中，由于振动是微小的，在需要提取缺陷信息的精度下，影响是较大的，但在肉眼观测中，振动带来的面板产品的偏移是微小且不易观测的，也就是说尺寸的偏移是微小的，因此在拉伸还原的过程中，可以直接将待检测图像的任一边缘与标准图像的对应边缘重合，减少拉伸调整的次数，标准图像也即在无振动情况下由同一位置拍摄获得的待检测图像。拉伸还原的过程中，先以重合的边缘为基础拉伸其余边缘，由于重合边缘自身是与标准图像边缘重合了，但是平行于重合边缘的方向也会有振动带来的翻转使其受到了压缩，因此后拉伸重合的边缘，使其长度满足拉伸长度信息后，与标准图像匹配，获得拉伸图像。In the specific implementation process, since the vibration is tiny, the impact is greater when the accuracy of defect information needs to be extracted, but in the naked eye observation, the deviation of the panel product caused by the vibration is small and difficult to observe, and That is to say, the size deviation is small, so in the process of stretching and restoring, any edge of the image to be detected can be directly overlapped with the corresponding edge of the standard image, reducing the number of stretching adjustments, and the standard image is also in the absence of The image to be detected is taken from the same position under vibration. In the process of stretching and restoration, the rest of the edges are first stretched based on the coincident edges. Since the coincident edges themselves coincide with the edges of the standard image, but the direction parallel to the coincident edges will also be flipped by vibration, causing it to be affected. Compression, so after stretching the overlapping edges to make their length meet the stretched length information, match the standard image to obtain a stretched image.
在一种实施例中，根据拉伸参数拉伸待检测图像，获得拉伸图像之前，面板缺陷检测方法还包括：In one embodiment, the image to be detected is stretched according to the stretching parameters, and before obtaining the stretched image, the panel defect detection method further includes:
获取标准图像上的沿面板产品的振动方向分布的若干像素点；Obtain several pixel points distributed along the vibration direction of the panel product on the standard image;
根据面板产品的振动幅度信息，分别获得若干像素点的水平偏移量信息；According to the vibration amplitude information of the panel product, obtain the horizontal offset information of several pixels respectively;
根据若干像素点在振动方向上的相对距离以及若干像素点的水平偏移量信息，构建像素点的水平偏移点图；According to the relative distance of several pixels in the vibration direction and the horizontal offset information of several pixels, construct the horizontal offset point map of the pixels;
对水平偏移点图上的点进行拟合，获得水平偏移曲线图；Fit the points on the horizontal offset point diagram to obtain the horizontal offset curve diagram;
根据水平偏移曲线图，获得拉伸参数。According to the horizontal offset graph, the stretching parameters are obtained.
在具体实施过程中，为了进一步提升拉伸矫正的精度，基于像素点来进行拉伸参数的获取。还是以前述情况为例，分析面板振动时向某一位置发生翻转时的情况可知，如果是面板以左侧为中心，右侧因振动发生翻转，那么相对于左侧来说，距离左侧越远的位置发生的水平偏移是更大的，也图像即不同位置被压缩的程度是不同的，如附图3所示的为一种振动情况下面板产品的示意图，水平的实线框为面板产品，虚线框为其振动情况下的位置示意，在面板不同位置的点位，其振动前后的位置与振动后的正投影点位构成直角三角形，如附图4所示为单个位置点的水平偏移示意图，图中H表示振动前后的高度差，D表示振动前后的水平位移，L为振动前后位置点距离，H、D、L构成直角三角形，由于为同一面板，所以其上的点位振动幅度相同，也即面板振动的夹角一定，越靠近左侧则H越小，那么对应的D就越小。虽然同比例的拉伸在振动幅度较小的情况下，矫正后的缺陷信息的精度可能是在允许范围内，但想要更准确地矫正待检测图像，就需要根据相对翻转中心的距离对不同位置进行不同的拉伸还原。In the specific implementation process, in order to further improve the accuracy of stretch correction, the stretch parameters are acquired based on pixel points. Still taking the aforementioned situation as an example, analyzing the situation when the panel is flipped to a certain position during vibration, it can be known that if the panel is centered on the left side and the right side is flipped due to vibration, then compared with the left side, the distance from the left side is farther away. The horizontal offset that occurs at far positions is greater, and the degree of image compression at different positions is different. As shown in Figure 3, it is a schematic diagram of a panel product under a vibration condition. The horizontal solid line box is For panel products, the dotted line frame indicates its position under vibration. At different positions on the panel, the position before and after vibration and the orthographic projection point after vibration form a right triangle. As shown in Figure 4, it is a single position point Schematic diagram of horizontal offset. In the figure, H represents the height difference before and after vibration, D represents the horizontal displacement before and after vibration, and L represents the distance between points before and after vibration. H, D, and L form a right triangle. Since they are the same panel, the points on it The bit vibration amplitude is the same, that is, the included angle of panel vibration is constant, the closer to the left side, the smaller H, and the corresponding D is smaller. Although the accuracy of the corrected defect information may be within the allowable range when the vibration amplitude is small for the stretching of the same proportion, but to correct the image to be detected more accurately, it is necessary to use different values according to the distance from the flip center. Positions are stretched and restored differently.
因此，本实施例中同前述原理类似，将拉伸还原分解到各个振动方向，获取振动方向上分布的像素点，然后根据振动的幅度信息获得各个像素点的水平偏移量。然后以相对距离与水平偏移量作为横纵坐标构建水平偏移的点图，由于是基于像素进行的，所以横纵坐标均以像素为单位，如附图5所示，其中的相对距离表示的应当是像素点相对因振动发生翻转的翻转中心的距离，随后对点图上的点进行线性拟合，根据拟合得到的曲线图上的曲线，就能够将振动方向上的所有像素点以及其发生的水平位移进行一一的对应，获得的水平偏移曲线图如附图6所示，与相对距离越远，水平偏移量更大的情况对应，可以看出，曲线上相对距离越大的位置的斜率越大，表明越远的位置水平偏移量变化越快，根据该曲线图可以得到拉伸参数，也即水平偏移更大的位置需要更大程度的拉伸还原，如此就能够更贴合实际的待检测图像被压缩的情况，使得待检测图像的拉伸矫正还原更准确，有利于更准确的提取缺陷信息。Therefore, in this embodiment, similar to the aforementioned principles, the stretch reduction is decomposed into each vibration direction, the pixels distributed in the vibration direction are obtained, and then the horizontal offset of each pixel is obtained according to the vibration amplitude information. Then use the relative distance and horizontal offset as the horizontal and vertical coordinates to construct a horizontal offset point map. Since it is based on pixels, the horizontal and vertical coordinates are all in pixels, as shown in Figure 5, where the relative distance represents It should be the distance between the pixel point and the flipping center due to vibration, and then linearly fit the points on the point map. According to the curve on the fitted graph, all the pixels in the vibration direction and the The horizontal displacements that occur are corresponding one by one, and the obtained horizontal displacement curve is shown in Figure 6, which corresponds to the situation that the farther the relative distance is, the larger the horizontal displacement is. It can be seen that the greater the relative distance on the curve The larger the slope of the larger position, the faster the horizontal offset changes at the farther position. According to the graph, the stretch parameter can be obtained, that is, the position with a larger horizontal offset requires a greater degree of stretch restoration, so It can be more suitable for the actual compression of the image to be inspected, making the stretch correction and restoration of the image to be inspected more accurate, and is conducive to more accurate extraction of defect information.
基于前述步骤的基于像素点获得的拉伸参数，根据拉伸参数拉伸待检测图像，获得拉伸图像，包括：Based on the stretching parameters obtained based on the pixel points in the preceding steps, the image to be detected is stretched according to the stretching parameters to obtain a stretched image, including:
根据拉伸参数，基于像素点拉伸待检测图像，获得拉伸图像。According to the stretching parameters, the image to be detected is stretched based on the pixel points to obtain the stretched image.
S20：对拉伸图像进行去模糊处理，获得目标图像。S20: Perform deblurring processing on the stretched image to obtain a target image.
在具体实施过程中，去模糊处理，也即使用图像修复技术或者去模糊技术将图像变得清晰，来解决拍摄的振动状态下的待检测图像模糊的问题。从技术方面来讲，模糊图像处理方法主要分为三大类，分别是图像增强、图像复原和超分辨率重构。利用常用的办公软件如PS、WPS上自带的图像修复功能进行去模糊，也可以使用常用的滤波处理或者卷积神经网络进行，去模糊处理的原理可参考现有技术，此处不再赘述。In the specific implementation process, deblurring processing, that is, using image restoration technology or deblurring technology to make the image clear, solves the blurring problem of the image to be detected under the captured vibration state. From a technical point of view, blurred image processing methods are mainly divided into three categories, namely image enhancement, image restoration and super-resolution reconstruction. Use commonly used office software such as PS, WPS built-in image repair function to perform deblurring, or use commonly used filter processing or convolutional neural network. The principle of deblurring processing can refer to the existing technology, and will not be repeated here. .
S30：将目标图像输入缺陷检测模型，获得缺陷信息；其中，缺陷检测模型包括卷积层，卷积层的卷积核，基于若干并行的分支卷积层的卷积核融合获得。S30: Input the target image into the defect detection model to obtain defect information; wherein, the defect detection model includes a convolution layer, and a convolution kernel of the convolution layer is obtained based on the fusion of convolution kernels of several parallel branch convolution layers.
在具体实施过程中，缺陷检测模型也即利用卷积神经网络训练获得的模型，模型能够通过训练学习到缺陷信息，进而使其具有从输入图像上识别缺陷的能力。从特征表达角度切入，使用若干组并行的卷积层代替原有的卷积，具体为将分支卷积层通过卷积核的融合得到原有卷积层的卷积核，有效利用了非对称结构特征提取能力强和泛化能力强的优势，同时减少了网络参数量。In the specific implementation process, the defect detection model is also a model obtained by using convolutional neural network training. The model can learn defect information through training, so that it has the ability to identify defects from the input image. From the perspective of feature expression, several groups of parallel convolutional layers are used to replace the original convolution. Specifically, the convolution kernel of the original convolutional layer is obtained by fusing the branch convolutional layer through the convolution kernel, which effectively utilizes the asymmetry. It has the advantages of strong structural feature extraction ability and strong generalization ability, while reducing the amount of network parameters.
本申请实施例提供一种由三组分支卷积层替换原有3*3卷积层的实施方式，具体的，将目标图像输入缺陷检测模型，获得缺陷信息之前，面板缺陷检测方法还包括：The embodiment of the present application provides an implementation mode in which the original 3*3 convolutional layer is replaced by three groups of branched convolutional layers. Specifically, before the target image is input into the defect detection model, and before the defect information is obtained, the panel defect detection method also includes:
获取三组并行的分支卷积层；其中，三组分支卷积层的卷积核尺寸分别为3*3、1*3以及3*1，卷积核尺寸为3*3的分支卷积层采用分布位移卷积，卷积核尺寸为1*3与3*1的分支卷积层采用分组卷积；Obtain three sets of parallel branch convolution layers; among them, the convolution kernel sizes of the three sets of branch convolution layers are 3*3, 1*3, and 3*1, respectively, and the branch convolution layer with a convolution kernel size of 3*3 Distributed displacement convolution is adopted, and the branch convolution layer with the convolution kernel size of 1*3 and 3*1 adopts group convolution;
将三组分支卷积层的卷积核融合为3*3尺寸的卷积核，以获得卷积层的卷积核。The convolution kernels of the three groups of branch convolution layers are fused into a 3*3 size convolution kernel to obtain the convolution kernel of the convolution layer.
在具体实施过程中，以增强网络特征提取能力，使用非对称混洗卷积结构的三层并行分支共同提取图像特征，使用3*3、1*3以及3*1的卷积层代替原有的3*3卷积，其中，3*3的分支卷积层采用分布位移卷积，1*3与3*1的分支卷积层采用分组卷积，3*3的卷积层主要用于增大感受野以获取更丰富的特征信息，而分组卷积层主要用于提升模型对翻转或旋转的泛化能力，对空间信息的依赖性降低，可大幅度减小模型参数。In the specific implementation process, in order to enhance the feature extraction ability of the network, the three-layer parallel branch of the asymmetric shuffled convolution structure is used to extract image features together, and the convolution layer of 3*3, 1*3 and 3*1 is used instead of the original The 3*3 convolution, where the 3*3 branch convolution layer uses distributed displacement convolution, the 1*3 and 3*1 branch convolution layers use group convolution, and the 3*3 convolution layer is mainly used for Increase the receptive field to obtain richer feature information, and the group convolution layer is mainly used to improve the generalization ability of the model for flipping or rotating, and the dependence on spatial information is reduced, which can greatly reduce the model parameters.
本实施例中，通过将面板产品的振动参数引入对待检测图像的矫正过程，利用根据振动参数获得的拉伸参数先将待检测图像的尺寸进行矫正，然后再利用去模糊处理将图像变得清晰，以便于缺陷检测模型的识别，由于缺陷检测模型的卷积层的卷积核是通过若干并行的分支卷积层的卷积核融合获得，模型没有增加额外的计算量，还能增强特征提取的能力，从而提升模型的分割精度与泛化能力，由于前述步骤的拉伸消除了振动带来的尺寸偏差，因此检测图像获得的缺陷信息能够与实际面板上的缺陷信息高度匹配，实现了对于振动情况下面板上缺陷的准确提取。In this embodiment, by introducing the vibration parameters of the panel product into the correction process of the image to be detected, the stretching parameters obtained according to the vibration parameters are used to first correct the size of the image to be detected, and then the image is cleared by deblurring , in order to facilitate the recognition of the defect detection model, since the convolution kernel of the convolution layer of the defect detection model is obtained through the fusion of convolution kernels of several parallel branch convolution layers, the model does not add additional calculations, and can also enhance feature extraction ability, thereby improving the segmentation accuracy and generalization ability of the model. Since the stretching in the previous steps eliminates the size deviation caused by vibration, the defect information obtained from the detection image can be highly matched with the defect information on the actual panel, realizing the Accurate extraction of defects on panels under vibration.
在一种实施例中，缺陷检测模型还包括通道混洗层，通道混洗层位于采用分组卷积的分支卷积层之后，通道混洗层用于：In one embodiment, the defect detection model further includes a channel shuffling layer, the channel shuffling layer is located after the branch convolution layer using group convolution, and the channel shuffling layer is used for:
对分支卷积层提取的特征图进行分组，获得分组矩阵；Group the feature maps extracted by the branch convolution layer to obtain a grouping matrix;
转置分组矩阵，获得转置矩阵；Transpose the grouping matrix to obtain the transposed matrix;
将转置矩阵平坦化重分组，获得目标特征图。Flatten and regroup the transposed matrix to obtain the target feature map.
在具体实施过程中，分组卷积会阻碍通道间信息流动，因此设置通道混洗层，将其放在分组卷积层之后，能够有效提高分组卷积中不同通道间的关联性。通道混洗的具体过程为，将输入的特征图分组，分为m组，n为每组的通道数量，输入特征矩阵向量变换为（m，n）；进行转置操作变换为（n，m）的转置矩阵；再将得到的结果平摊扁平化处理，最后结果重新分组，分为n组凭借形成新的特征图作为输出，即目标特征图。In the specific implementation process, group convolution will hinder the flow of information between channels, so setting the channel shuffling layer and placing it after the group convolution layer can effectively improve the correlation between different channels in group convolution. The specific process of channel shuffling is to group the input feature maps into m groups, n is the number of channels in each group, and the input feature matrix vector is transformed into (m, n); the transposition operation is transformed into (n, m ) transpose matrix; and then flatten the obtained results, and finally regroup the results and divide them into n groups by forming a new feature map as the output, that is, the target feature map.
在一种实施例中，缺陷检测模型还包括中间层，中间层基于缺陷检测模型的特征金字塔网络的特征层获得。In an embodiment, the defect detection model further includes an intermediate layer, and the intermediate layer is obtained based on the feature layer of the feature pyramid network of the defect detection model.
在具体实施过程中，由于特征金字塔中的映射在X轴、Y轴方向上都被缩放，并且在相邻两层之间的尺度差距很大，导致两个大小相似的物体被预测并分到不同的层，因此通过在缺陷检测模型的特征金字塔网络的特征层中生成中间层来解决，使得不同尺寸特征图之间的过渡更为平滑，从而提升检测的效果。具体的，提供一种中间层的获取方式，即，将目标图像输入缺陷检测模型，获得缺陷信息之前，面板缺陷检测方法还包括：In the specific implementation process, since the mapping in the feature pyramid is scaled in the X-axis and Y-axis directions, and the scale gap between two adjacent layers is large, two objects of similar size are predicted and divided into Different layers are therefore solved by generating intermediate layers in the feature layer of the feature pyramid network of the defect detection model, making the transition between feature maps of different sizes smoother, thereby improving the detection effect. Specifically, an acquisition method of the middle layer is provided, that is, the target image is input into the defect detection model, and before the defect information is obtained, the panel defect detection method also includes:
分别对缺陷检测模型的特征金字塔网络的特征层进行线性缩放，获得若干第一特征层；Respectively linearly scale the feature layers of the feature pyramid network of the defect detection model to obtain several first feature layers;
将第一特征层进行逐像素相加，获得第二特征层；Adding the first feature layer pixel by pixel to obtain the second feature layer;
对第二特征层进行卷积与融合，获得中间层。Convolve and fuse the second feature layer to obtain the middle layer.
在具体实施过程中，将特征金字塔网络中每一层的相邻层按一定系数进行线性缩放，并将所得结果，即第一特征层进行逐像素相加得到第二特征层，第二特征层通过3*3卷积，从原始层中对特征进行融合，生成中间层，由此使得特征金字塔中相邻层之间都有一个中间层进行过渡，减少尺寸截断的影响，提高模型性能，进而提升缺陷提取的准确。In the specific implementation process, the adjacent layers of each layer in the feature pyramid network are linearly scaled by a certain coefficient, and the result, that is, the first feature layer is added pixel by pixel to obtain the second feature layer, and the second feature layer Through 3*3 convolution, the features are fused from the original layer to generate an intermediate layer, so that there is an intermediate layer for transition between adjacent layers in the feature pyramid, reducing the impact of size truncation and improving model performance. Improve the accuracy of defect extraction.
参照附图7，基于与前述实施例中同样的发明构思，本申请实施例还提供一种面板缺陷检测装置，该装置包括：Referring to Figure 7, based on the same inventive concept as in the previous embodiments, the embodiment of the present application also provides a panel defect detection device, which includes:
拉伸模块，拉伸模块用于根据拉伸参数拉伸待检测图像，获得拉伸图像；其中，待检测图像基于面板产品的正投影方向获得，拉伸参数基于面板产品的振动参数获得；A stretching module, the stretching module is used to stretch the image to be detected according to the stretching parameters to obtain the stretched image; wherein, the image to be detected is obtained based on the orthographic projection direction of the panel product, and the stretching parameter is obtained based on the vibration parameters of the panel product;
去模糊模块，去模糊模块用于对拉伸图像进行去模糊处理，获得目标图像；A deblurring module, the deblurring module is used for deblurring the stretched image to obtain the target image;
检测模块，检测模块用于将目标图像输入缺陷检测模型，获得缺陷信息；其中，缺陷检测模型包括卷积层，卷积层的卷积核，基于若干并行的分支卷积层的卷积核融合获得。Detection module, the detection module is used to input the target image into the defect detection model to obtain defect information; wherein, the defect detection model includes a convolution layer, a convolution kernel of the convolution layer, and a convolution kernel fusion based on several parallel branch convolution layers get.
本领域技术人员应当理解，实施例中的各个模块的划分仅仅是一种逻辑功能的划分，实际应用时可以全部或部分集成到一个或多个实际载体上，且这些模块可以全部以软件通过处理单元调用的形式实现，也可以全部以硬件的形式实现，或是以软件、硬件结合的形式实现，需要说明的是，本实施例中面板缺陷检测装置中各模块是与前述实施例中的面板缺陷检测方法中的各步骤一一对应，因此，本实施例的具体实施方式可参照前述面板缺陷检测方法的实施方式，这里不再赘述。Those skilled in the art should understand that the division of each module in the embodiment is only a division of logical functions, which can be fully or partially integrated into one or more actual carriers in actual application, and these modules can be processed by software It can be implemented in the form of unit calling, or can be implemented in the form of hardware, or in the form of a combination of software and hardware. It should be noted that each module in the panel defect detection device in this embodiment is the same as the panel in the previous embodiment. The steps in the defect detection method are in one-to-one correspondence. Therefore, for the specific implementation manner of this embodiment, reference may be made to the foregoing implementation manners of the panel defect detection method, which will not be repeated here.
基于与前述实施例中同样的发明构思，本申请的实施例还提供一种计算机可读存储介质，储存有计算机程序，计算机程序被处理器加载执行时，实现如本申请实施例提供的面板缺陷检测方法。Based on the same inventive concept as in the foregoing embodiments, the embodiments of the present application also provide a computer-readable storage medium, which stores a computer program. When the computer program is loaded and executed by the processor, the panel defects provided in the embodiments of the present application are realized. Detection method.
基于与前述实施例中同样的发明构思，本申请的实施例还提供一种电子设备，包括处理器及存储器，其中，Based on the same inventive concept as in the foregoing embodiments, the embodiments of the present application further provide an electronic device, including a processor and a memory, wherein,
存储器用于存储计算机程序；memory for storing computer programs;
处理器用于加载执行计算机程序，以使电子设备执行如本申请实施例提供的面板缺陷检测方法。The processor is used to load and execute a computer program, so that the electronic device executes the panel defect detection method provided by the embodiment of the present application.
此外，基于与前述实施例中同样的发明构思，本申请的实施例还提供一种计算机程序产品，包括计算机程序，当计算机程序被执行时，用于执行如本申请实施例提供的面板缺陷检测方法。In addition, based on the same inventive concept as in the foregoing embodiments, the embodiments of the present application also provide a computer program product, including a computer program, which is used to perform panel defect detection as provided in the embodiments of the present application when the computer program is executed. method.
在一些实施例中，计算机可读存储介质可以是FRAM、ROM、PROM、EPROM、EEPROM、闪存、磁表面存储器、光盘、或CD-ROM等存储器；也可以是包括上述存储器之一或任意组合的各种设备。计算机可以是包括智能终端和服务器在内的各种计算设备。In some embodiments, the computer-readable storage medium can be memory such as FRAM, ROM, PROM, EPROM, EEPROM, flash memory, magnetic surface memory, optical disk, or CD-ROM; Various equipment. Computers can be various computing devices including smart terminals and servers.
在一些实施例中，可执行指令可以采用程序、软件、软件模块、脚本或代码的形式，按任意形式的编程语言（包括编译或解释语言，或者声明性或过程性语言）来编写，并且其可按任意形式部署，包括被部署为独立的程序或者被部署为模块、组件、子例程或者适合在计算环境中使用的其它单元。In some embodiments, executable instructions may take the form of programs, software, software modules, scripts, or code written in any form of programming language, including compiled or interpreted languages, or declarative or procedural languages, and its Can be deployed in any form, including as a stand-alone program or as a module, component, subroutine or other unit suitable for use in a computing environment.
作为示例，可执行指令可以但不一定对应于文件系统中的文件，可以可被存储在保存其它程序或数据的文件的一部分，例如，存储在超文本标记语言（HTML，Hyper TextMarkup Language）文档中的一个或多个脚本中，存储在专用于所讨论的程序的单个文件中，或者，存储在多个协同文件（例如，存储一个或多个模块、子程序或代码部分的文件）中。As an example, executable instructions may, but do not necessarily correspond to files in a file system, may be stored as part of files that hold other programs or data, for example, in a Hyper Text Markup Language (HTML) document in one or more scripts of the program in question, in a single file dedicated to the program in question, or in multiple cooperating files (for example, files that store one or more modules, subroutines, or sections of code).
作为示例，可执行指令可被部署为在一个计算设备上执行，或者在位于一个地点的多个计算设备上执行，又或者，在分布在多个地点且通过通信网络互连的多个计算设备上执行。As an example, executable instructions may be deployed to be executed on one computing device, or on multiple computing devices located at one site, or alternatively, on multiple computing devices distributed across multiple sites and interconnected by a communication network. to execute.
需要说明的是，在本文中，术语“包括”、“包含”或者其任何其他变体意在涵盖非排他性的包含，从而使得包括一系列要素的过程、方法、物品或者系统不仅包括那些要素，而且还包括没有明确列出的其他要素，或者是还包括为这种过程、方法、物品或者系统所固有的要素。在没有更多限制的情况下，由语句“包括一个……”限定的要素，并不排除在包括该要素的过程、方法、物品或者系统中还存在另外的相同要素。It should be noted that, as used herein, the term "comprises", "comprises" or any other variation thereof is intended to cover a non-exclusive inclusion such that a process, method, article or system comprising a set of elements includes not only those elements, It also includes other elements not expressly listed, or elements inherent in the process, method, article, or system. Without further limitations, an element defined by the phrase "comprising a..." does not preclude the presence of additional identical elements in the process, method, article or system comprising that element.
上述本申请实施例序号仅仅为了描述，不代表实施例的优劣。The serial numbers of the above embodiments of the present application are for description only, and do not represent the advantages and disadvantages of the embodiments.
通过以上的实施方式的描述，本领域的技术人员可以清楚地了解到上述 实施例方法可借助软件加必需的通用硬件平台的方式来实现，当然也可以通 过硬件，但很多情况下前者是更佳的实施方式。基于这样的理解，本申请的 技术方案本质上或者说对现有技术做出贡献的部分可以以软件产品的形式体 现出来，该计算机软件产品存储在一个存储介质（如只读存储器/随机存取存储器、磁碟、光 盘）中，包括若干指令用以使得一台多媒体终端设备(可以是手机，计算机，电视接收机，或者网络设备等)执行本申请各个实施例所述的方法。Through the description of the above embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus a necessary general-purpose hardware platform, and of course also by hardware, but in many cases the former is better implementation. Based on this understanding, the essence of the technical solution of this application or the part that contributes to the prior art can be embodied in the form of software products, and the computer software products are stored in a storage medium (such as read-only memory/random access Memory, magnetic disk, optical disk), including several instructions to make a multimedia terminal device (which may be a mobile phone, computer, television receiver, or network device, etc.) execute the methods described in various embodiments of the present application.
综上，本申请提供的一种面板缺陷检测方法、装置、存储介质、设备及程序产品，包括：根据拉伸参数拉伸待检测图像，获得拉伸图像；其中，待检测图像基于面板产品的正投影方向获得，拉伸参数基于面板产品的振动参数获得；对拉伸图像进行去模糊处理，获得目标图像；将目标图像输入缺陷检测模型，获得缺陷信息；其中，缺陷检测模型包括卷积层，卷积层的卷积核，基于若干并行的分支卷积层的卷积核融合获得。本申请的方法通过将面板产品的振动参数引入对待检测图像的矫正过程，利用根据振动参数获得的拉伸参数先将待检测图像的尺寸进行矫正，然后再利用去模糊处理将图像变得清晰，以便于缺陷检测模型的识别，由于缺陷检测模型的卷积层的卷积核是通过若干并行的分支卷积层的卷积核融合获得，模型没有增加额外的计算量，还能增强特征提取的能力，从而提升模型的分割精度与泛化能力，由于前述步骤的拉伸消除了振动带来的尺寸偏差，因此检测图像获得的缺陷信息能够与实际面板上的缺陷信息高度匹配，实现了对于振动情况下面板上缺陷的准确提取。To sum up, the present application provides a panel defect detection method, device, storage medium, equipment, and program product, including: stretching the image to be detected according to stretching parameters to obtain a stretched image; wherein, the image to be detected is based on the image of the panel product The orthographic projection direction is obtained, and the stretching parameters are obtained based on the vibration parameters of the panel product; the stretched image is deblurred to obtain the target image; the target image is input into the defect detection model to obtain defect information; the defect detection model includes a convolutional layer , the convolution kernel of the convolution layer is obtained based on the fusion of convolution kernels of several parallel branch convolution layers. The method of the present application introduces the vibration parameters of the panel product into the correction process of the image to be detected, uses the stretching parameters obtained according to the vibration parameters to first correct the size of the image to be detected, and then uses deblurring processing to make the image clear, In order to facilitate the identification of the defect detection model, since the convolution kernel of the convolution layer of the defect detection model is obtained through the fusion of convolution kernels of several parallel branch convolution layers, the model does not add additional calculations, and it can also enhance feature extraction. ability, thereby improving the segmentation accuracy and generalization ability of the model. Since the stretching in the preceding steps eliminates the dimensional deviation caused by vibration, the defect information obtained from the detection image can be highly matched with the defect information on the actual panel, and the vibration Accurate extraction of defects on the panel under certain conditions.
以上所述仅为本申请的较佳实施例，并不用以限制本申请，凡在本申请的精神和原则之内，所作的任何修改、等同替换、改进等，均应包含在本申请的保护范围之内。The above descriptions are only preferred embodiments of the application, and are not intended to limit the application. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the application shall be included in the protection of the application. within range.
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310692268.5ACN116433664B (en) | 2023-06-13 | 2023-06-13 | Panel defect detection method, device, storage medium, equipment and program product |
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310692268.5ACN116433664B (en) | 2023-06-13 | 2023-06-13 | Panel defect detection method, device, storage medium, equipment and program product |
| Publication Number | Publication Date |
|---|---|
| CN116433664A CN116433664A (en) | 2023-07-14 |
| CN116433664Btrue CN116433664B (en) | 2023-09-01 |
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310692268.5AActiveCN116433664B (en) | 2023-06-13 | 2023-06-13 | Panel defect detection method, device, storage medium, equipment and program product |
| Country | Link |
|---|---|
| CN (1) | CN116433664B (en) |
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN118505704B (en)* | 2024-07-18 | 2024-10-22 | 成都数之联科技股份有限公司 | A universal model building and detection method for panel production line defect detection |
| CN119936069B (en)* | 2025-04-03 | 2025-06-24 | 成都九洲电子信息系统股份有限公司 | Product detection method, device, medium and equipment based on machine vision |
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103620928A (en)* | 2011-03-17 | 2014-03-05 | 联合活跃驱动公司 | Asymmetric and general vibration waveforms from a plurality of synchronized vibration actuators |
| CN105046695A (en)* | 2015-07-01 | 2015-11-11 | 华南理工大学 | Projective capacitive touch panel ITO (Indium Tin Oxide) circuit defect detection method based on one-dimensional image |
| CN105510348A (en)* | 2015-12-31 | 2016-04-20 | 南京协辰电子科技有限公司 | Flaw detection method and device of printed circuit board and detection equipment |
| CN107123107A (en)* | 2017-03-24 | 2017-09-01 | 广东工业大学 | Cloth defect inspection method based on neutral net deep learning |
| CN109472760A (en)* | 2019-02-01 | 2019-03-15 | 深兰人工智能芯片研究院（江苏）有限公司 | A kind of method, apparatus of correcting distorted image |
| CN109658823A (en)* | 2019-02-27 | 2019-04-19 | 上海天马微电子有限公司 | Display panel and display device |
| CN110321908A (en)* | 2018-03-29 | 2019-10-11 | 华为技术有限公司 | Image-recognizing method, terminal device and computer readable storage medium |
| CN112508826A (en)* | 2020-11-16 | 2021-03-16 | 哈尔滨工业大学（深圳） | Printed matter defect detection method based on feature registration and gradient shape matching fusion |
| CN113255654A (en)* | 2020-02-12 | 2021-08-13 | 斯凯孚公司 | Imaging processing method of characteristic data and application thereof |
| CN113256485A (en)* | 2021-05-21 | 2021-08-13 | 百果园技术(新加坡)有限公司 | Image stretching method, device, electronic equipment and storage medium |
| CN113496279A (en)* | 2020-04-08 | 2021-10-12 | 脸谱公司 | Packet convolution for channel convolution engine using point-to-point connections |
| WO2021217857A1 (en)* | 2020-04-27 | 2021-11-04 | 平安科技（深圳）有限公司 | Slice defect detection method and apparatus, and electronic device and readable storage medium |
| CN113887491A (en)* | 2021-10-21 | 2022-01-04 | 江南大学 | Human skeleton behavior recognition system and method based on cross-space-time graph convolutional network |
| CN114075751A (en)* | 2020-08-19 | 2022-02-22 | 天津海尔洗涤电器有限公司 | Device with vibration detection and water level sensing functions and washing machine |
| CN114364075A (en)* | 2021-12-29 | 2022-04-15 | 合肥维信诺科技有限公司 | Stretchable display panel, stretchable display device and preparation method |
| CN114577906A (en)* | 2022-02-23 | 2022-06-03 | 韶关东阳光自动化设备有限公司 | Ultrasonic guided wave detection method and system for micro-perforation defects of positive corrosion foil |
| CN115393252A (en)* | 2021-05-25 | 2022-11-25 | 武汉Tcl集团工业研究院有限公司 | Defect detection method and device for display panel, electronic equipment and storage medium |
| CN115471439A (en)* | 2021-06-11 | 2022-12-13 | 武汉Tcl集团工业研究院有限公司 | Method and device for identifying defects of display panel, electronic equipment and storage medium |
| CN115497076A (en)* | 2022-10-09 | 2022-12-20 | 江苏智能无人装备产业创新中心有限公司 | A high-precision and high-efficiency signal identification detection method, device and medium |
| CN115661161A (en)* | 2022-12-29 | 2023-01-31 | 成都数联云算科技有限公司 | Method, device, storage medium, equipment and program product for detecting defects of parts |
| CN116229199A (en)* | 2022-12-15 | 2023-06-06 | 北京市地铁运营有限公司技术创新研究院分公司 | Target detection method based on model light weight |
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| IT201900005826A1 (en)* | 2019-04-16 | 2020-10-16 | Santex Rimar Group S R L | DEVICE AND METHOD FOR REAL TIME DETECTION OF DEFECTS IN FABRICS, DURING WEAVING |
| CN111768386B (en)* | 2020-06-30 | 2024-02-20 | 北京百度网讯科技有限公司 | Product defect detection method, device, electronic equipment and storage medium |
| EP4402034A4 (en)* | 2021-09-15 | 2025-07-09 | Amsted Rail Co Inc | DEVICE AND METHOD FOR MONITORING GOODS ON A MOBILE RAILWAY |
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103620928A (en)* | 2011-03-17 | 2014-03-05 | 联合活跃驱动公司 | Asymmetric and general vibration waveforms from a plurality of synchronized vibration actuators |
| CN105046695A (en)* | 2015-07-01 | 2015-11-11 | 华南理工大学 | Projective capacitive touch panel ITO (Indium Tin Oxide) circuit defect detection method based on one-dimensional image |
| CN105510348A (en)* | 2015-12-31 | 2016-04-20 | 南京协辰电子科技有限公司 | Flaw detection method and device of printed circuit board and detection equipment |
| CN107123107A (en)* | 2017-03-24 | 2017-09-01 | 广东工业大学 | Cloth defect inspection method based on neutral net deep learning |
| CN110321908A (en)* | 2018-03-29 | 2019-10-11 | 华为技术有限公司 | Image-recognizing method, terminal device and computer readable storage medium |
| CN109472760A (en)* | 2019-02-01 | 2019-03-15 | 深兰人工智能芯片研究院（江苏）有限公司 | A kind of method, apparatus of correcting distorted image |
| CN109658823A (en)* | 2019-02-27 | 2019-04-19 | 上海天马微电子有限公司 | Display panel and display device |
| CN113255654A (en)* | 2020-02-12 | 2021-08-13 | 斯凯孚公司 | Imaging processing method of characteristic data and application thereof |
| CN113496279A (en)* | 2020-04-08 | 2021-10-12 | 脸谱公司 | Packet convolution for channel convolution engine using point-to-point connections |
| EP3968237A1 (en)* | 2020-04-08 | 2022-03-16 | Facebook, Inc. | Grouped convolution using point-to-point connected channel convolution engines |
| WO2021217857A1 (en)* | 2020-04-27 | 2021-11-04 | 平安科技（深圳）有限公司 | Slice defect detection method and apparatus, and electronic device and readable storage medium |
| CN114075751A (en)* | 2020-08-19 | 2022-02-22 | 天津海尔洗涤电器有限公司 | Device with vibration detection and water level sensing functions and washing machine |
| CN112508826A (en)* | 2020-11-16 | 2021-03-16 | 哈尔滨工业大学（深圳） | Printed matter defect detection method based on feature registration and gradient shape matching fusion |
| CN113256485A (en)* | 2021-05-21 | 2021-08-13 | 百果园技术(新加坡)有限公司 | Image stretching method, device, electronic equipment and storage medium |
| CN115393252A (en)* | 2021-05-25 | 2022-11-25 | 武汉Tcl集团工业研究院有限公司 | Defect detection method and device for display panel, electronic equipment and storage medium |
| CN115471439A (en)* | 2021-06-11 | 2022-12-13 | 武汉Tcl集团工业研究院有限公司 | Method and device for identifying defects of display panel, electronic equipment and storage medium |
| CN113887491A (en)* | 2021-10-21 | 2022-01-04 | 江南大学 | Human skeleton behavior recognition system and method based on cross-space-time graph convolutional network |
| CN114364075A (en)* | 2021-12-29 | 2022-04-15 | 合肥维信诺科技有限公司 | Stretchable display panel, stretchable display device and preparation method |
| CN114577906A (en)* | 2022-02-23 | 2022-06-03 | 韶关东阳光自动化设备有限公司 | Ultrasonic guided wave detection method and system for micro-perforation defects of positive corrosion foil |
| CN115497076A (en)* | 2022-10-09 | 2022-12-20 | 江苏智能无人装备产业创新中心有限公司 | A high-precision and high-efficiency signal identification detection method, device and medium |
| CN116229199A (en)* | 2022-12-15 | 2023-06-06 | 北京市地铁运营有限公司技术创新研究院分公司 | Target detection method based on model light weight |
| CN115661161A (en)* | 2022-12-29 | 2023-01-31 | 成都数联云算科技有限公司 | Method, device, storage medium, equipment and program product for detecting defects of parts |
| Title |
|---|
| "基于无人机视觉的起重机表面裂纹检测方法";周前飞等;《测控技术》;第41卷(第4期);第28-34页* |
| Publication number | Publication date |
|---|---|
| CN116433664A (en) | 2023-07-14 |
| Publication | Publication Date | Title |
|---|---|---|
| CN116433664B (en) | Panel defect detection method, device, storage medium, equipment and program product | |
| CN108009543B (en) | License plate recognition method and device | |
| JP6688277B2 (en) | Program, learning processing method, learning model, data structure, learning device, and object recognition device | |
| CN107704857A (en) | A kind of lightweight licence plate recognition method and device end to end | |
| CN108986152B (en) | A foreign object detection method and device based on differential image | |
| CN113420756A (en) | Certificate image recognition method and device, storage medium and electronic device | |
| CN111104941B (en) | Image direction correction method and device and electronic equipment | |
| CN105046213A (en) | Method for augmenting reality | |
| CN110033009B (en) | Method for processing image data in a connected network | |
| US12400295B2 (en) | Image processing device and method | |
| CN111461113B (en) | A large-angle license plate detection method based on deformed planar object detection network | |
| CN104252705A (en) | Method and device for splicing images | |
| CN113112511B (en) | Method and device for correcting test paper, storage medium and electronic equipment | |
| CN111179166A (en) | Image processing method, device, equipment and computer readable storage medium | |
| CN112950528A (en) | Certificate posture determining method, model training method, device, server and medium | |
| CN111815628B (en) | Display panel defect detection method, device, equipment and readable storage medium | |
| CN111598084B (en) | Defect segmentation network training method, device, equipment and readable storage medium | |
| CN115619678A (en) | Image deformation correction method and device, computer equipment and storage medium | |
| CN111401365B (en) | OCR image automatic generation method and device | |
| CN115115606A (en) | Image coordinate matching method, terminal, and computer-readable storage medium | |
| CN112308061B (en) | License plate character recognition method and device | |
| WO2024260264A1 (en) | Image processing method and apparatus, device, storage medium, and program product | |
| CN118365693A (en) | Target object size detection method, device, equipment and storage medium | |
| CN117253022A (en) | Object identification method, device and inspection equipment | |
| CN116433610A (en) | Wine cap packaging defect detection method, device, equipment and storage medium |
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| EE01 | Entry into force of recordation of patent licensing contract | Application publication date:20230714 Assignee:Chengdu Shuzhi Innovation Lean Technology Co.,Ltd. Assignor:Chengdu shuzhilian Technology Co.,Ltd. Contract record no.:X2024510000014 Denomination of invention:Panel defect detection methods, devices, storage media, equipment, and program products Granted publication date:20230901 License type:Common License Record date:20240723 | |
| EE01 | Entry into force of recordation of patent licensing contract |