

技术领域technical field
本发明涉及工业智能检测技术领域,尤其是涉及一种多阶段工业视觉缺陷检测方法及检测系统。The invention relates to the technical field of industrial intelligent detection, in particular to a multi-stage industrial visual defect detection method and detection system.
背景技术Background technique
随着计算机视觉领域的发展和硬件计算资源的充裕,各大制造业为缩减劳动力成本、扩大制造规模,开始追求高精度、快速度的自动化工业视觉缺陷检测技术,基于卷积神经网络的缺陷检测算法由于其强大的辨识能力和泛化能力备受技术人员喜爱,现已广泛应用于各类缺陷检测设备中。With the development of computer vision and the abundance of hardware computing resources, in order to reduce labor costs and expand manufacturing scale, major manufacturing industries have begun to pursue high-precision and fast automated industrial visual defect detection technology. Defect detection based on convolutional neural networks The algorithm is favored by technicians due to its powerful identification ability and generalization ability, and has been widely used in various defect detection equipment.
由于生产过程中的杂质颗粒、刻蚀工艺等问题,产品会呈现从未出现过的假缺陷(完全不影响产品性能)外观,导致检测算法过判。该类问题的一般解决方式是收集过判样本重新训练卷积神经网络算法,但该过程耗时较长,导致许多过判问题不能及时解决,影响生产效率;另外,由于神经网络算法的计算速度对显卡硬件的高度依赖,每台工控机都配备显卡的在线部署方案会导致高额的硬件部署成本。Due to problems such as impurity particles and etching process in the production process, the product will show the appearance of false defects that have never appeared before (does not affect product performance at all), which leads to over-judgment of the detection algorithm. The general solution to this type of problem is to collect over-judgment samples and retrain the convolutional neural network algorithm, but this process takes a long time, resulting in many over-judgment problems that cannot be solved in time, affecting production efficiency; in addition, due to the calculation speed of the neural network algorithm Highly dependent on graphics card hardware, the online deployment solution in which every industrial computer is equipped with a graphics card will lead to high hardware deployment costs.
发明内容Contents of the invention
本发明所要解决的主要技术问题是提供了一种多阶段工业视觉缺陷检测方法,能够减少过判样本重新训练时所需的耗时。The main technical problem to be solved by the present invention is to provide a multi-stage industrial visual defect detection method, which can reduce the time-consuming required for retraining over-judged samples.
为了解决上述技术问题,本发明第一方面提供了一种多阶段工业视觉缺陷检测方法,包括:In order to solve the above technical problems, the first aspect of the present invention provides a multi-stage industrial visual defect detection method, including:
S1、采集待检测产品外观的源图像并对所述源图像上的过判位置进行标记;S1. Collect the source image of the appearance of the product to be detected and mark the over-judged position on the source image;
S2、通过卷积神经网络算法对所述源图像以及所述标记的位置进行训练并测试;S2. Train and test the source image and the position of the mark through a convolutional neural network algorithm;
S3、收集测试中所产生的缺陷过判位置的图片,并剔除出图片中的过判位置;S3. Collect pictures of the over-judgment positions of defects generated in the test, and remove the over-judgment positions in the pictures;
S4、通过分类算法对缺陷过判位置的图片以及相同数量的非过判缺陷的图片进行训练;S4. Using a classification algorithm to train the pictures of the defect over-judged position and the same number of pictures of non-judged defects;
S5、将训练得到的分类算法集成在所述卷积神经网络算法后以对所述图片进行二次判定。S5. Integrating the trained classification algorithm into the convolutional neural network algorithm to perform a second judgment on the picture.
优选地,S2中,训练前,首先对S1中采集得到的源图像进行旋转矫正,并对所述源图像的尺寸进行缩放,然后再通过所述卷积神经网络进行训练直至算法收敛。Preferably, in S2, before training, the source image collected in S1 is first rotated and corrected, and the size of the source image is scaled, and then the convolutional neural network is used for training until the algorithm converges.
进一步优选地,所述分类算法为XGBoost分类算法,所述分类算法进行训练时将所述缺陷过判位置的图片与所述非过判缺陷的图片缩放至同一尺寸进行特征提取。Further preferably, the classification algorithm is an XGBoost classification algorithm. During training, the classification algorithm scales the pictures of the over-judgment position of the defect and the picture of the non-over-judgment defect to the same size for feature extraction.
优选地,S5中,卷积神经网络判定所述图片为真缺陷则裁剪出缺陷位置并输入分类算法中进行二次判断。Preferably, in S5, the convolutional neural network judges that the picture is a true defect, then crops out the defect position and inputs it into the classification algorithm for secondary judgment.
进一步优选地,所述特征提取为统计每类像素值的点的个数作为256维特征,或者压缩为一维特征。Further preferably, the feature extraction is to count the number of points of each type of pixel value as a 256-dimensional feature, or to compress it into a one-dimensional feature.
本发明第二方面提供了一种检测系统,所述检测系统包括多条检测线,各检测线至少包括视觉控制器、工业相机、工控机和服务器,各所述视觉控制器依次与所述工业相机、所述工控机以及所述服务器相连以形成一条检测线。The second aspect of the present invention provides a detection system. The detection system includes a plurality of detection lines. Each detection line includes at least a vision controller, an industrial camera, an industrial computer and a server. Each of the vision controllers communicates with the industrial The camera, the industrial computer and the server are connected to form a detection line.
进一步优选地,多条所述检测线共用同一服务器,所述服务器使用Flask框架并在其不同端口开启多个进程的缺陷检测算法。Further preferably, multiple detection lines share the same server, and the server uses the Flask framework and opens multiple processes of defect detection algorithms on different ports.
优选地,多个所述进程采用同一显卡进行计算。Preferably, multiple processes use the same graphics card for calculation.
进一步优选地,多个所述工控机和服务器布置在同一局域网内以接收所述工业相机所摄得的图片。Further preferably, a plurality of said industrial computers and servers are arranged in the same local area network to receive pictures taken by said industrial cameras.
优选地,所述工业相机成像后的图片采用Base64编码将所述图片编为二进制数据,并以POST报文形式传输至所述服务器。Preferably, the picture imaged by the industrial camera is coded with Base64 to encode the picture into binary data, and transmitted to the server in the form of a POST message.
通过上述技术方案,本发明的多阶段工业视觉缺陷检测方法,通过将采集到的源图像以及源图像上的过判位置进行标记,然后将图片中的过判位置剔除,然后再通过分类算法将剔除的过判位置的图片和相同数量的非过判缺陷的图片进行训练,并对训练后的图片进行二次判断,以此来降低过判率,通过上述的训练检测方式,能够在维持原有检测速度的基础上解决工厂检测线的临时需求,并且更加的节约时间;另外,所采用的多条检测线共用同一服务器内的同一显卡,从而保证了有效检测效率的同时能够减少显卡等硬件部署的成本。Through the above technical solution, the multi-stage industrial visual defect detection method of the present invention marks the collected source image and the over-judgment position on the source image, then removes the over-judgment position in the picture, and then classifies the over-judgment position through the classification algorithm The pictures of the rejected positions and the same number of non-judgment defects are trained, and the trained pictures are judged a second time to reduce the passing judgment rate. Through the above training and detection methods, the original On the basis of detection speed, it solves the temporary needs of the factory inspection line, and saves more time; in addition, the multiple inspection lines used share the same graphics card in the same server, thereby ensuring effective detection efficiency and reducing hardware such as graphics cards. The cost of deployment.
本发明的其他特征和优点将在随后的具体实施方式部分予以详细说明。Other features and advantages of the present invention will be described in detail in the following detailed description.
附图说明Description of drawings
图1为本发明具体实施方式的多阶段工业视觉缺陷检测方法的整体工作示意图Fig. 1 is the overall working schematic diagram of the multi-stage industrial visual defect detection method of the specific embodiment of the present invention
图2为本发明具体实施方式的多阶段工业视觉缺陷检测方法的实现流程示意图。Fig. 2 is a schematic diagram of the implementation flow of a multi-stage industrial visual defect detection method according to a specific embodiment of the present invention.
具体实施方式Detailed ways
以下结合附图对本发明的具体实施方式进行详细说明。应当理解的是,此处所描述的具体实施方式仅基于说明和解释本发明,并不基于限制本发明。Specific embodiments of the present invention will be described in detail below in conjunction with the accompanying drawings. It should be understood that the specific embodiments described here are only for illustrating and explaining the present invention, not for limiting the present invention.
在本发明的描述中,需要说明的是,除非另有明确的规定和限定,术语“安装”、“设置”、“连接”应做广义理解,例如,术语“连接”可以是固定连接,也可以是可拆卸连接,或者是一体连接;可以是直接连接,也可以是通过中间媒介间接连接,可以是两个元件内部的连通或两个元件的相互作用关系。对于本领域的普通技术人员而言,可以根据具体情况理解上述术语在本发明中的具体含义。In the description of the present invention, it should be noted that, unless otherwise specified and limited, the terms "installation", "setting", and "connection" should be understood in a broad sense, for example, the term "connection" can be a fixed connection, or It can be a detachable connection or an integrated connection; it can be a direct connection or an indirect connection through an intermediary; it can be the internal communication of two elements or the interaction relationship between two elements. Those of ordinary skill in the art can understand the specific meanings of the above terms in the present invention according to specific situations.
参照图1,本发明具体实施方式的一种多阶段工业视觉缺陷检测方法,包括:S1、采集待检测产品外观的源图像并对所述源图像上的过判位置进行标记;Referring to Fig. 1, a kind of multi-stage industrial visual defect detection method of the specific embodiment of the present invention, comprises: S1, collects the source image of the appearance of the product to be detected and marks the judged position on the source image;
S2、通过卷积神经网络算法对所述源图像以及所述标记的位置进行训练并测试;S2. Train and test the source image and the position of the mark through a convolutional neural network algorithm;
S3、收集测试中所产生的缺陷过判位置的图片,并剔除出图片中的过判位置;S3. Collect pictures of the over-judgment positions of defects generated in the test, and remove the over-judgment positions in the pictures;
S4、通过分类算法对缺陷过判位置的图片以及相同数量的非过判缺陷的图片进行训练;S4. Using a classification algorithm to train the pictures of the defect over-judged position and the same number of pictures of non-judged defects;
S5、将训练得到的分类算法集成在所述卷积神经网络算法后以对所述图片进行二次判定。S5. Integrating the trained classification algorithm into the convolutional neural network algorithm to perform a second judgment on the picture.
具体地,S2中,在训练前,首先对S1中采集得到的源图像进行旋转矫正和尺寸缩放,然后对矫正完成,缩放至设定大小的图片以及图片中标记有过判位置的标签进行训练,直至算法收敛,然后将部署到产线中进行实测。Specifically, in S2, before training, first perform rotation correction and size scaling on the source images collected in S1, and then perform training on the pictures that have been corrected and scaled to the set size and the labels marked with the over-judgment position in the pictures , until the algorithm converges, and then deploy it to the production line for actual measurement.
其中,所采用的分类算法为XGBoost分类算法,当然,所采用的分类算法并不局限于该XGBoost分类算法,也可采用其他类型的分类算法,通过将源图像中的缺陷位置的图片转化为单通道图片,并将尺寸缩放至同一尺寸,在此将设定缩放尺寸为64×64像素,然后对缩放完成后的图片进行特征提取,特征数值提取通过统计每类像素值的点的个数作为256维特征或者直接压缩为一维特征,然后送入XGBoost算法中训练直至算法收敛。Among them, the classification algorithm adopted is the XGBoost classification algorithm. Of course, the classification algorithm adopted is not limited to the XGBoost classification algorithm, and other types of classification algorithms can also be used. By converting the picture of the defect position in the source image into a single channel image, and scale the size to the same size. Here, the zoom size is set to 64×64 pixels, and then feature extraction is performed on the scaled image. The feature value extraction is obtained by counting the number of points of each type of pixel value as 256-dimensional features or directly compressed into one-dimensional features, and then sent to the XGBoost algorithm for training until the algorithm converges.
更具体地,将分类算法集成至卷积神经网络算法后,以能够在卷积神经网络对图片缺陷位置进行判定后,再通过XGBoost算法对图片缺陷位置进行二次判定,通过二次判定来降低过判率,其中,采用分类算法对图片缺陷位置的判定在卷积神经网络判定图片为真缺陷后将缺陷位置剪出,然后再输入分类算法中进行二次判断。More specifically, after the classification algorithm is integrated into the convolutional neural network algorithm, after the convolutional neural network judges the defect position of the picture, the XGBoost algorithm can be used to make a second judgment on the defect position of the picture, and the second judgment can reduce the Over-judgment rate, among them, the classification algorithm is used to determine the defect position of the picture. After the convolutional neural network determines that the picture is a true defect, the defect position is cut out, and then input into the classification algorithm for a second judgment.
另外,基于卷积神经网络算法以及分类算法均需要基于源图像来进行检测,因此源图像的清晰与否对于整体检测结果的准确度较为重要,因此需要采用高精度的工业相机来采集待检测产品外观的源图像,以解决需要检测和区分的缺陷类别,并且保证能够在这些源图像上的缺陷位置和类别进行标记,通常缺陷类别的覆盖面越广、采集的数据量越多,标志质量越好,从而越有利于神经网络去的较好的准确率。In addition, both the convolutional neural network algorithm and the classification algorithm need to be detected based on the source image, so the clarity of the source image is more important for the accuracy of the overall detection result, so it is necessary to use high-precision industrial cameras to collect the products to be detected Appearance source images to solve the defect categories that need to be detected and differentiated, and ensure that the defect positions and categories on these source images can be marked. Generally, the wider the coverage of defect categories, the greater the amount of data collected, and the better the marking quality , which is more conducive to the better accuracy of the neural network.
如图2所示,本发明的一种检测系统采用多阶段工业视觉缺陷检测方法,该检测系统包括多条检测线,多条检测线整体之间采用并联的方式进行连接,且每条检测线至少包括视觉控制器、工业相机、工控机和服务器,一条检测线中,视觉控制器、工业相机、工控机以及服务器依次相连。另外,在一条检测线中,可以采用多个视觉控制器和工业相机来对产品的外观进行检测,多个视觉控制器和工业相机能够分别设置在产品的不同方位,通过对产品外观的多方位进行检测,以能够更加准确的获得产品的外观数据,其中,各视觉控制器与各工业相机之间可以采用串联的方式进行连接并构成为一组摄像装置,在一条检测线中可以设置多组摄像装置,多组摄像装置之间可以采用并联的方式进行连接,多组摄像装置均与同一工控机相连以完成对于图像的传输。As shown in Figure 2, a detection system of the present invention adopts a multi-stage industrial visual defect detection method. The detection system includes a plurality of detection lines, and the multiple detection lines are connected in parallel as a whole, and each detection line It includes at least a vision controller, an industrial camera, an industrial computer, and a server. In a detection line, the vision controller, industrial camera, industrial computer, and server are connected in sequence. In addition, in one inspection line, multiple vision controllers and industrial cameras can be used to inspect the appearance of the product. Multiple vision controllers and industrial cameras can be respectively set in different directions of the product. In order to obtain the appearance data of the product more accurately, each visual controller and each industrial camera can be connected in series to form a group of camera devices, and multiple groups can be set in one detection line As for the camera device, multiple groups of camera devices can be connected in parallel, and the multiple groups of camera devices are all connected to the same industrial computer to complete image transmission.
具体地,多条检测线共用同一服务器,即多条检测线内的工控机均与同一服务器的不同端口相连,并且,使用Flask框架在服务器的不同端口可以开启不同进程的缺陷检测算法,这些进程共用同一张显卡来进行计算,并且能够监听POST请求。Specifically, multiple detection lines share the same server, that is, the industrial computers in multiple detection lines are connected to different ports of the same server, and the defect detection algorithms of different processes can be started on different ports of the server using the Flask framework. Share the same graphics card for calculations and be able to monitor POST requests.
更具体地,多个工控机与一个服务器部署在同一局域网内,或者多个工控机直接采用网线与服务器直接连接,其中,采用局域网无线连接能够便于对于服务器以及多个工控机位置的调整,而采用网线进行连接能够保证传输效率,降低了工控机位置的调整,但减少了传输过程中所可能产生的干扰,保证传输数据的准确性,工控机接收到工业相机成像后的图片后,使用Base64编码将图片编为二进制数据,然后以POST报文形式发给服务器端口,算法服务器接收到该报文后使用Base64解码,并且对解码后的图片进行检测,然后再将缺陷检测的结果发回给工控机。More specifically, multiple industrial computers and a server are deployed in the same local area network, or multiple industrial computers are directly connected to the server using network cables. Among them, the use of wireless connections in the local area network can facilitate the adjustment of the location of the server and multiple industrial computers. The use of network cables to connect can ensure transmission efficiency, reduce the adjustment of the position of the industrial computer, but reduce the interference that may occur during the transmission process, and ensure the accuracy of the transmitted data. After the industrial computer receives the image of the industrial camera, it uses Base64 Encoding encodes the picture into binary data, and then sends it to the server port in the form of a POST message. After receiving the message, the algorithm server uses Base64 to decode, and detects the decoded picture, and then sends the defect detection result back to industrial computer.
另外,所采用的卷积神经网络以YOLOv5卷积神经网络算法为例,采用该卷积神经网络算法作用一阶段的检测算法时,服务器中的12GB显存的显卡足够支持8台工控机的检测需求,因此不需要再将每个工控机对应于一个服务器来进行检测,从而能够节约7/8工控机对显卡的安装要求。In addition, the convolutional neural network used takes the YOLOv5 convolutional neural network algorithm as an example. When the convolutional neural network algorithm is used for the first-stage detection algorithm, the graphics card with 12GB memory in the server is enough to support the detection requirements of 8 industrial computers. , so there is no need to detect each industrial computer corresponding to a server, which can save 7/8 industrial computer installation requirements for graphics cards.
该多检测线的在线部署方案基于多进程方法和HTTP协议,通过使用Flask框架在服务器的不同端口开启不同进程的缺陷检测算法,并且多个进程共用同一张显卡进行计算,并且监听POST请求,另外,多个工控机与一个服务器布置在同一个局域网内,工控机能够以POST报文形式向服务器端口发送检测请求并接收检测结果,并以此作为判断依据,保证检测结果的准确性。The online deployment scheme of the multi-inspection line is based on the multi-process method and the HTTP protocol. By using the Flask framework, the defect detection algorithms of different processes are opened on different ports of the server, and multiple processes share the same graphics card for calculation and monitor POST requests. , multiple industrial computers and a server are arranged in the same local area network, and the industrial computer can send a detection request to the server port in the form of POST message and receive the detection result, and use it as a judgment basis to ensure the accuracy of the detection result.
在本发明的描述中,参考术语“一个实施例”、“一些实施例”、“一种实施方式”等的描述意指结合该实施例或示例描述的具体特征、结构、材料或者特点包含于本发明的至少一个实施例或示例中。在本发明中,对上述术语的示意性表述不一定指的是相同的实施例或示例。而且,描述的具体特征、结构、材料或者特点可以在任何的一个或多个实施例或示例中以合适的方式结合。In the description of the present invention, reference to the terms "one embodiment", "some embodiments", "one implementation" and the like means that a specific feature, structure, material or characteristic described in connection with the embodiment or example is included in the In at least one embodiment or example of the invention. In the present invention, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials or characteristics described may be combined in any suitable manner in any one or more embodiments or examples.
以上结合附图详细描述了本发明的优选实施方式,但是,本发明并不限于此。在本发明的技术构思范围内,可以对本发明的技术方案进行多种简单变型,包括各个具体技术特征以任何合适的方式进行组合,为了避免不必要的重复,本发明对各种可能的组合方式不再另行说明。但这些简单变型和组合同样应当视为本发明所公开的内容,均属于本发明的保护范围。The preferred embodiments of the present invention have been described in detail above with reference to the accompanying drawings, however, the present invention is not limited thereto. Within the scope of the technical concept of the present invention, various simple modifications can be made to the technical solution of the present invention, including combining each specific technical feature in any suitable manner. Not otherwise stated. However, these simple modifications and combinations should also be regarded as the content disclosed by the present invention, and all belong to the protection scope of the present invention.
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310105577.8ACN116091467A (en) | 2023-02-09 | 2023-02-09 | Multi-stage industrial vision defect detection method and detection system |
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310105577.8ACN116091467A (en) | 2023-02-09 | 2023-02-09 | Multi-stage industrial vision defect detection method and detection system |
| Publication Number | Publication Date |
|---|---|
| CN116091467Atrue CN116091467A (en) | 2023-05-09 |
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310105577.8AWithdrawnCN116091467A (en) | 2023-02-09 | 2023-02-09 | Multi-stage industrial vision defect detection method and detection system |
| Country | Link |
|---|---|
| CN (1) | CN116091467A (en) |
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113724233A (en)* | 2021-09-02 | 2021-11-30 | 国网安徽省电力有限公司铜陵供电公司 | Transformer equipment appearance image defect detection method based on fusion data generation and transfer learning technology |
| CN114663352A (en)* | 2022-02-24 | 2022-06-24 | 国网通用航空有限公司 | High-precision detection method and system for defects of power transmission line and storage medium |
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113724233A (en)* | 2021-09-02 | 2021-11-30 | 国网安徽省电力有限公司铜陵供电公司 | Transformer equipment appearance image defect detection method based on fusion data generation and transfer learning technology |
| CN114663352A (en)* | 2022-02-24 | 2022-06-24 | 国网通用航空有限公司 | High-precision detection method and system for defects of power transmission line and storage medium |
| Publication | Publication Date | Title |
|---|---|---|
| CN110648310B (en) | Weak supervision casting defect identification method based on attention mechanism | |
| CN114359283A (en) | Defect detection method based on Transformer and electronic equipment | |
| CN115065708A (en) | Industrial Internet of things based on machine vision detection and control method | |
| CN114697212A (en) | Device parameter processing method, device, system and medium | |
| CN107846472A (en) | The fleet anomaly detection method of extensive power transmission and transforming equipment Monitoring data flow | |
| CN114926385A (en) | Panel defect detection method, storage medium and terminal equipment | |
| CN114895634A (en) | An automatic control system for product production line based on machine vision | |
| CN115456984A (en) | High-speed image recognition defect detection system based on two-dimensional code | |
| CN114202720A (en) | A system and method for real-time detection of egg quantity based on target recognition | |
| CN113868471A (en) | Data matching method and system based on monitoring equipment relationship | |
| CN115546144A (en) | PCB surface defect detection method based on improved Yolov5 algorithm | |
| CN117854402A (en) | Abnormal display detection method and device of display screen and terminal equipment | |
| WO2025060766A9 (en) | Image recognition method and apparatus, storage medium and electronic device | |
| CN120030272B (en) | A robot-based cluster textile machine information acquisition method and system | |
| CN112533168A (en) | Data processing method of wireless sensor network node | |
| CN116091467A (en) | Multi-stage industrial vision defect detection method and detection system | |
| CN112990045B (en) | Method and apparatus for generating image change detection model and image change detection | |
| CN118170116B (en) | Fault analysis and sniffing system of intelligent manufacturing equipment group based on Internet of things | |
| CN116846789B (en) | Operation and maintenance management system for communication link | |
| CN116582447B (en) | An IoT network protocol identification method based on edge computing gateway | |
| CN112364918A (en) | Abnormality recognition method, terminal, and computer-readable storage medium | |
| CN117079286A (en) | Identification method and identification system for pointer instrument | |
| CN116499362A (en) | Steel plate size online measurement system | |
| CN112085722B (en) | Training sample image acquisition method and device | |
| CN108334840A (en) | Pedestrian detection method based on deep neural network under traffic environment |
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| WW01 | Invention patent application withdrawn after publication | ||
| WW01 | Invention patent application withdrawn after publication | Application publication date:20230509 |