CN112991344A - Detection method, storage medium and detection system based on deep transfer learning - Google Patents

Abstract

The invention provides a detection method, a storage medium and a detection system based on deep transfer learning, belongs to the field of image data processing, and particularly relates to a detection technology based on an artificial intelligence algorithm. The scheme adopts a semantic segmentation loss function suitable for the industrial field, is suitable for detecting small defects and solves the problem of missing detection of small objects; the secondary judgment greatly reduces the misjudgment rate of the detection result; by adopting the transfer learning technology, the method can be applied to the detection of various items on the basis of hundreds of small samples by constructing a basic model and introducing the transfer learning, and the detection efficiency is improved.

Description

Detection method, storage medium and detection system based on deep transfer learning

Technical Field

The invention belongs to the field of image data processing, relates to a defect detection technology based on artificial intelligence algorithm and image data processing, can be applied to surface defect detection in the field of semiconductors, and particularly relates to a detection method, a storage medium and a detection system based on deep migration learning.

Background

In artificial intelligence and machine learning, transfer learning is a kind of idea and mode of learning. Machine learning is a large class of important methods for artificial intelligence, and is also the most rapidly and significantly developed method at present. Machine learning solves the problem that a machine autonomously obtains knowledge from data to apply to a new problem, and migration learning is an important branch in machine learning and focuses on applying the learned knowledge migration to the new problem. The core problem of transfer learning is to find the similarity between a new problem and an original problem, so that the transfer of knowledge can be smoothly realized.

As is known, the training and updating of machine learning models both rely on the labeling of data. Although vast amounts of data can be acquired, the data is often in a very rudimentary original form, and few data are correctly manually labeled. The labeling of data is a time-consuming and expensive operation, and there is no effective way to solve this problem. This presents challenges to model training and updating for machine learning and deep learning. For some specific fields, the fields are not well developed because there is not enough calibration data for learning. The problem exists in the industry of detecting the defects of the silicon wafers of the photovoltaic sorting machine, and industrial projects often have the problems of difficulty in acquiring defect images, high difficulty in marking the defect images and the like and lack of sufficient marking data; by utilizing the idea of transfer learning, a better model can be trained from easily obtained big data, and fine adjustment is carried out on a new task, so that the model is transferred to the new task. Furthermore, the models can be adaptively updated according to the new tasks, so that better effect is achieved.

Meanwhile, in view of the main problem existing in the defect detection based on the deep learning algorithm, that the defect detection is easy to go over, it is assumed that the defect detection is carried out on the industrial image by using the semantic segmentation technology in the deep learning, and some abnormal over-detection inevitably occurs in the final detection result.

Taking photovoltaic silicon wafer detection as an example, the difficulty of silicon wafer contamination detection of a sorting machine is that the shape of the contamination is irregular, the gray scale difference of the contamination is large, a lot of contamination can not be seen by naked eyes, the contamination can appear at any position of the silicon wafer, the texture of the silicon wafer interferes, and the like. In contrast, in the conventional method, feature extraction mainly depends on an extractor designed manually, professional knowledge and a complex parameter adjusting process are required, and each method is specific to specific application and has poor generalization capability and robustness. Deep learning mainly includes data-driven feature extraction, deep and data set-specific feature representation can be obtained according to learning of a large number of samples, the data set expression is more efficient and accurate, extracted abstract features are higher in robustness and better in generalization capability, and the end-to-end feature extraction method can be used for end-to-end extraction. The defects are that the sample set has large influence and the computational power requirement is high. Therefore, the contamination detection based on the traditional algorithm is difficult to satisfy the contamination detection of all silicon wafers on the production line through a single algorithm parameter.

Disclosure of Invention

In order to overcome the defects of the prior art, the invention provides a detection method, a storage medium and a detection system based on deep migration learning, which can solve the problems.

The deep learning and traditional method deep fusion algorithm can fully utilize the advantages of current big data, solve some problems in actual projects by using an artificial intelligence algorithm, and can reduce the problem of over-detection caused by black box property of a neural network by adding some artificial judgments and increase some artificial controllability. Meanwhile, a transfer learning technology in deep learning is utilized, so that a good detection effect can be obtained under the condition that only a small sample size exists in the early stage of the project.

Under the condition that the early-stage data quantity is not enough is guaranteed through transfer learning, basic depth features learned by data in other fields are transferred to current data, data collection and marking time of early-stage projects is greatly saved, and meanwhile through the idea that deep learning and a traditional method are combined, the over-detection rate of the projects can be greatly reduced by means of manual design rules and the like under the condition that accuracy is guaranteed.

One of the purposes is to construct a large-scale and universal industrial defect detection database, which comprises various products of different types, pictures of different sizes and defects of different types, so as to ensure that a universal industrial deep learning feature extraction network can be trained, prepare for a subsequent transfer learning technology and practice and apply the universal industrial deep learning feature extraction network to a contamination detection case of a photovoltaic silicon wafer.

The specific scheme for achieving the purpose is as follows.

A detection method based on deep transfer learning comprises the following steps:

s1, constructing a large-scale industrial defect detection gallery: and through gallery management and labeling software, the information of the pictures is labeled and uploaded manually by multiple ports, so that a large-scale high-quality labeled industrial database is constructed together through warehousing of labeled data.

And S2, constructing a basic model, and determining a basic deep learning model suitable for industrial defect detection.

S3, acquiring hundreds of magnitude images of the detection items through an optical system based on transfer learning of a small amount of labels, and carrying out manual region labeling on the defect region of the piece to be detected. And training the marked sample increment to the basic model in S2 to obtain a detection model aiming at the defects of the piece to be detected.

And S4, fusing the traditional algorithm and deep learning, and establishing a secondary judgment rule for the detection model to form a defect detection judgment model for the piece to be detected.

Further, step S1 includes:

and S11, acquiring pictures through the optical system.

And S12, manually screening, labeling and classifying the pictures.

And S13, constructing an industrial database based on the product level small graph and the defect level small graph according to the result in the S12.

Further, step S2 includes:

and S21, constructing a universal segmentation model, wherein the universal segmentation model comprises a training prediction mode module, a model structure module, a loss function module and a preprocessing mode module.

And S22, training the general segmentation model of S21 by using the large-scale industrial defect detection gallery in S1 as a training set.

Further, in the training prediction mode module, the training mode includes cropping training and whole graph training, and the prediction mode includes small graph prediction and whole graph prediction. The model structure module comprises at least one of a classic deeplab series structure, a Unet series structure and a real-time network Blsenet series structure. The penalty function module includes at least one of a BCE function, an IOU function, a BCE + IOU function, and a Recall-BCE function. The preprocessing mode module comprises at least one mode of a turning enhancement mode, an internal gray scale transformation mode and an external gray scale stretching mode.

Further, in step S3, the manual region labeling manner includes at least one of a whole graph labeling manner, a small graph labeling manner, a target box labeling manner, and a pixel level labeling manner. The incremental training method comprises the following steps: and freezing a feature extraction layer of the model, and finely adjusting a feature classification layer through supplemented data to realize incremental training.

Further, step S4 includes:

and S41, inputting the detection image of the piece to be detected collected by the optical system.

And S42, obtaining a probability output graph through the transition learning model in S3, and obtaining a first judgment result through a threshold value of 0.5.

And S43, calculating different evaluation indexes for each detection result in the S42, carrying out secondary judgment according to a preset index threshold value, determining whether the detection result is a real dirt defect, and obtaining a final result.

Further, the evaluation indexes in step S43 include uniformity, intensity, gray value deviation, and width and height of the circumscribed rectangle.

The present invention also provides a computer readable storage medium having stored thereon computer instructions characterized in that: the computer instructions when executed perform the foregoing method.

The invention also provides a detection system based on the depth migration learning, the detection system comprises a computer and an optical detection module which are in telecommunication connection, the optical detection module comprises a detection camera and a light source, and the detection camera transmits the acquired surface image of the piece to be detected to the computer in real time for image processing.

The computer includes a storage unit, an image processing unit, and an output unit.

The storage unit stores the image of the to-be-detected piece collected by the detection camera and sends the collected detection image of the to-be-detected piece to the image processing module according to the requirement.

The image processing unit is used for processing image data of a detection image of a to-be-detected piece, judging defects by operating the method, classifying and labeling the defect pictures and forming a defect labeling graph.

The image output unit outputs the defect labeling diagram obtained by the image processing unit.

Compared with the prior art, the invention has the beneficial effects that:

the scheme 1 adopts a semantic segmentation loss function suitable for the industrial field, is suitable for detecting small defects, and solves the problem of small object omission;

2, highly integrating deep learning with a traditional method, and performing secondary filtering on a deep learning result by utilizing traditional algorithms such as gray scale characteristics (such as gray scale value deviation, non-uniformity and fuzzy entropy) of shape characteristics (such as area, width and height of a circumscribed rectangle and roundness) of defect types on the basis of a deep learning detection result, so that the misjudgment rate of the detection result is greatly reduced;

3, a transfer learning technology is adopted, and the method can be applied to the detection of various items on the basis of hundreds of small samples by constructing a basic model and introducing transfer learning.

Drawings

FIG. 1 is a flow chart of a method for detecting silicon wafer contamination by applying the detection method based on deep migration learning of the present invention;

FIG. 2 is a schematic diagram of a model structure of a large-scale industrial defect detection gallery;

FIG. 3 is a schematic diagram of a model structure of a general segmentation model;

FIG. 4 is a schematic diagram of the introduction of a base model for transfer learning;

FIG. 5 is a diagram illustrating the fusion of deep learning and conventional algorithms;

fig. 6 is a schematic diagram of a silicon wafer contamination detection system based on deep migration learning.

Detailed Description

In order to make the objects, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are some, but not all, embodiments of the present invention. All other embodiments, which can be obtained by a person skilled in the art without any inventive step based on the embodiments of the present invention, are within the scope of the present invention.

As used in this specification and the appended claims, the terms "a," "an," "the," and/or "the" are not intended to be inclusive in the singular, but rather are intended to be inclusive in the plural, unless the context clearly dictates otherwise. In general, the terms "comprises" and "comprising" merely indicate that steps and elements are included which are explicitly identified, that the steps and elements do not form an exclusive list, and that a method or apparatus may include other steps or elements.

Flow charts are used in this description to illustrate operations performed by a system according to embodiments of the present description. It should be understood that the preceding or following operations are not necessarily performed in the exact order in which they are performed. Rather, the various steps may be processed in reverse order or simultaneously. Meanwhile, other operations may be added to the processes, or a certain step or several steps of operations may be removed from the processes.

First embodiment

For a photovoltaic silicon wafer, a description is given of a detection method taking one of defects (scratch, indentation, pitting, dirt, edge chipping, internal crack, notch, foreign matter, miscellaneous spot, etc.) as an example, specifically as follows.

A method for detecting contamination based on deep migration learning, referring to fig. 1, the method includes the following steps:

step S1, constructing a large-scale industrial defect detection gallery: through the graph library management and labeling software, the information labeling and uploading of the pictures are carried out manually through multiple ports, and therefore a large-scale high-quality labeled industrial database is constructed through the warehousing of labeled data, and the method is shown in figure 2. Step S1 specifically includes:

s11, acquiring pictures through an optical system;

s12, manually screening, labeling and classifying the pictures;

and S13, constructing an industrial database based on the product level small graph and the defect level small graph according to the result in the S12.

The method aims to provide large-scale data for constructing the large-scale industrial defect detection gallery and provide a data base for constructing a basic model. The key point of the transfer learning is that a basic model trained on large-scale data exists, and the reason that the transfer learning is rarely chopped in the industry all the time is that few public large-scale defect detection data sets exist, so that the first step of a specific detection item is to construct a large-scale and universal industrial defect detection database which comprises various products of different types, pictures of different sizes and defects of different types, and a universal industrial deep learning feature extraction network can be trained. And preparing for subsequent transfer learning technology.

Referring to fig. 2, the large-scale industrial defect detection gallery is represented as a small map gallery collected for different detection items of different products, including product-level small maps and defect-level small maps. The product level small graph comprises labels of different projects, defect types and different sites. The defect level small graph comprises labels of defect attributes, size attributes, background attributes and label attributes.

Wherein the defect attributes include, but are not limited to, scratches, indentations, stippling, smudging, edge chipping, implosions, gaps, foreign objects, miscellaneous points, and the like.

Wherein the dimensional attributes include, but are not limited to, 96 x 96, 256 x 256, 512 x 512, 1024 x 1024, and the like.

Wherein the background attributes include, but are not limited to, a uniform background, a striped background, etc.

The label attributes include, among others, pixel level (segmentation), region level (detection), image level (classification), and the like.

And S2, constructing a basic model, and determining a basic deep learning model suitable for industrial defect detection. Specifically, step S2 includes:

and S21, constructing a universal segmentation model, wherein the universal segmentation model comprises a training prediction mode module, a model structure module, a loss function module and a preprocessing mode module. Referring to fig. 3, the general segmentation model network is as follows.

In the training prediction mode module, the training mode comprises cropping training and whole graph training, and the prediction mode comprises small graph prediction and whole graph prediction;

the model structure module comprises at least one of a classical deplab series structure, a Unet series structure and a real-time network Blsenet series structure;

the loss function module comprises at least one of a BCE function, an IOU function, a BCE + IOU function and a Recall-BCE function;

wherein the BCE function is:

… … … … … … formula 1;

the Recall-BCE function is as follows:

… … … … … … …formula 2;

in the formula,

the real value of the artificial mark machine value is obtained;

a predicted value for deep learning;

alpha is 0.05, and the value is obtained by manually adjusting the parameters;

is the sum of all OK sample deviations;

is the sum of all NG sample deviations.

The preprocessing mode module comprises at least one mode of a turning enhancement mode, an internal gray scale transformation mode and an external gray scale stretching mode.

In a preferred embodiment, when selecting the general segmentation model, starting with four aspects of a training prediction mode, a model structure, a loss function, a preprocessing mode and the like, a large number of comparison experiments are required in each aspect to obtain an optimal scheme. Firstly, in terms of a training mode, in view of the fact that many industrial defect detection products need strong structural information, whole-image training and whole-image prediction are selected as training prediction modes under the consideration of comprehensive detection rate and time consumption. Secondly, on the network structure of the model structure, the defects such as scratches, dirt and the like which are common detection requirements on industrial defect detection projects are considered to belong to shallow features in the deep neural network; and selecting a U-shaped structure which utilizes a better shallow feature as a basic network. Thirdly, regarding the loss function, considering that the actual ratio of the defect pixel to be actually detected in the defect detection in the whole product is very small, and the gradient decline of the loss function is easy to oscillate when only the IOU function is used, selecting the mixture of the IOU function and the BCE function as the final loss function; finally, in a data preprocessing mode, products placed in various directions may appear in actual production, so that training data are turned over at equal probability, various turning robustness of the products is guaranteed, and meanwhile, due to the fact that image gray scale abnormality may appear in the products under some interference conditions, gray scale stretching is conducted twice on the outside and the inside of the products respectively, and the model is guaranteed to have strong gray scale robustness.

The improved scheme for solving the small object missing detection problem of the loss function is as follows:

improvement for IOU loss: the general OIU loss is training for an abnormal graph, and when the training set includes OK pictures, the IOU loss cannot reflect the learning condition of the network on the current OK sample (numerator is always 0).

Scheme 1, labels 0 and 1 are reversed, and at this time, the IOU can normally perform loss calculation on OK samples, but the gradient characteristics are poor, most samples are predicted correctly, and the IOU cannot completely embody advantages; and in thescheme 2, a smoothing coefficient is added to ensure that the OK sample can be normally calculated.

When calculating the loss, the pixels of the whole image are not averaged, but only the pixels of the defect area are averaged, so that the loss of small dirt and large dirt can be balanced (the loss is balanced from 1: 100 to 3: 1).

Because many normal samples exist in the defect detection sample, the original IOU loss function cannot measure the learning quality of the normal samples; the IOU function is modified, so that the IOU loss can be guaranteed to correctly measure the detection structure of a normal sample; similarly, because the original BCE function generates larger loss on large defect samples, the loss on small defect samples is smaller, and the small defects are usually the more difficult defects in industrial defect detection, the Recall-BCE is provided to strengthen the learning capability of the network on the small defects.

And S22, training the general segmentation model of S21 by using the large-scale industrial defect detection gallery in S1 as a training set.

S3, acquiring images of the silicon wafer with the magnitude of hundreds of dirty pieces through an optical system based on transfer learning with a small amount of labels, and carrying out artificial region labeling on the dirty regions of the silicon wafer; and training the marked sample increment to the basic model in S2 to obtain a detection model aiming at the silicon wafer contamination. A schematic diagram of the introduction of the underlying model by transfer learning is shown in fig. 4.

In step S3, the manual region labeling manner includes at least one of whole graph labeling, small graph labeling, target box labeling and pixel level labeling; the incremental training method comprises the following steps: and freezing a feature extraction layer of the model, and finely adjusting a feature classification layer through supplemented data to realize incremental training. The comparison of the effects before and after the migration training is shown in the following Table-1.

As can be seen from the figure, the effect is much higher than that of the non-transfer learning through the transfer learning of only hundreds of samples, and can exceed 0.5.

In the above comparative experiment using the current migration learning technique, migration is performed on the network of the basic model trained on 1W-sized data sets, and when there are only 105 target data sets, the IOU on the training set can be rapidly increased to 0.5290 (the higher the IOU is, the better the model effect is, and generally, the higher the IOU of semantic segmentation is, the better the current model has an effect when the IOU is above 0.5) by using the current migration learning technique. If a neural network is directly trained from the beginning without adopting the transfer learning, the effect on the training set or the test set is far lower than the result obtained by the transfer learning.

And S4, fusing the traditional algorithm and deep learning, and establishing a secondary judgment rule for the basic model, namely the detection model after the transfer learning in the step S3 to form a contamination detection judgment model for the silicon wafer. The contamination detection judgment model has a final result output terminal. Specifically, step S4 includes:

and S41, inputting the silicon wafer to-be-detected image collected by the optical system.

S42, obtaining a probability output graph through the transfer learning model in S3, and obtaining a first judgment result through a threshold value of 0.5; this step is a detection result based on deep learning.

And S43, calculating different evaluation indexes for each detection result in the S42, carrying out secondary judgment according to a preset index threshold value, determining whether the detection result is a real dirt defect, and obtaining a final result.

The evaluation indexes in step S43 include uniformity, intensity, gray value deviation, and width and height of the circumscribed rectangle.

Referring to fig. 5, a schematic diagram of fusion of deep learning and a conventional algorithm. Although the deep learning algorithm has strong detection capability, due to the actual product complexity in industrial defect detection and the black box property of the neural network, some wrong judgment is inevitably carried out on a normal area, and the condition is generally called over-detection. Therefore, a strong post-processing operation is designed, secondary judgment is carried out on the detection result of deep learning, the over-detection of the algorithm can be reduced to a great extent, and the delivery requirement in actual production is met.

Specifically, on the basis of the deep learning detection result, the deep learning result is subjected to secondary filtering by using traditional algorithms such as shape features (such as area, width and height of circumscribed rectangle and roundness) and gray features (such as gray value deviation, non-uniformity and fuzzy entropy) of defect types, so that the misjudgment rate of the detection result is greatly reduced.

Second embodiment

The present invention also provides a computer readable storage medium having stored thereon computer instructions which, when executed, perform the steps of the aforementioned method. For details, the method is described in the foregoing section, and is not repeated here.

It will be appreciated by those of ordinary skill in the art that all or a portion of the steps of the methods of the embodiments described above may be performed by associated hardware as instructed by a program that may be stored on a computer readable storage medium, which may include non-transitory and non-transitory, removable and non-removable media, to implement information storage by any method or technology. The information may be computer readable instructions, data structures, modules of a program, or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), Static Random Access Memory (SRAM), Dynamic Random Access Memory (DRAM), other types of Random Access Memory (RAM), Read Only Memory (ROM), Electrically Erasable Programmable Read Only Memory (EEPROM), flash memory or other memory technology, compact disc read only memory (CD-ROM), Digital Versatile Discs (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices, or any other non-transmission medium that can be used to store information that can be accessed by a computing device. As defined herein, a computer readable medium does not include a transitory computer readable medium such as a modulated data signal and a carrier wave.

The computer program code represented by the aforementioned computer instructions may be written in any one or more programming languages, including an object oriented programming language such as Java, Scala, Smalltalk, Eiffel, JADE, Emerald, C + +, C #, VB.NET, Python, and the like, a conventional programming language such as C, Visualbasic, Fortran2003, Perl, COBOL2002, PHP, ABAP, a dynamic programming language such as Python, Ruby, and Groovy, or other programming languages, and the like. The program code may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or processing device. In the latter scenario, the remote computer may be connected to the user's computer through any network format, such as a Local Area Network (LAN) or a Wide Area Network (WAN), or the connection may be made to an external computer (for example, through the Internet), or in a cloud computing environment, or as a service, such as a software as a service (SaaS).

Third embodiment

A silicon wafer contamination detection system based on depth migration learning is disclosed, and referring to fig. 6, the silicon wafer contamination detection system comprises a computer 1 and acontamination detection module 2 which are in telecommunication connection, thecontamination detection module 2 comprises adetection camera 21 and alight source 22, and thedetection camera 21 transmits collected silicon wafer surface images to the computer 1 in real time for image processing.

Thedetection camera 21 adopts a linear array camera, and is matched with thelight source 22 to detect dirt in the moving process of the silicon wafer, so that the dirt on the surface of the silicon wafer is detected.

Thelight source 22 includes a light emitting diode and a lamp housing. The light emitting diode is positioned below the lampshade. So inside the lamp shade, the light that emitting diode sent is the wide-angle diffuse reflection state, and diffuse reflection's radiation further reachesdetection camera 21 through the gap of lamp shade, so set up the contrast that can alleviate silicon chip crystal structure to make the silicon chip of illumination evenly in the aspect of the width.

The computer 1 includes a storage unit, an image processing unit, and an output unit. The storage unit stores the silicon chip image collected by the detection camera and sends the collected silicon chip image to the image processing module according to the requirement; the image processing unit is used for processing image data of the silicon wafer image, performing dirt judgment by operating the method in the embodiment 1, and classifying and marking the dirt picture to form a dirt marking picture; wherein,

and the image output unit outputs the dirt labeling diagram obtained by the image processing unit.

Technical terms used and possible to be used in the present application are explained as follows:

the IOU (Intersection ratio-Intersection Union) is a result of dividing the Overlap portion-Overlap of two regions (candidate frame-candidate frame and original mark frame) by the Overlap portion-Intersection of the two regions, i.e., IOU = (Area of Overlap)/(Area of Union).

BCELoss (cross entropy loss function for two classes), when used, needs to add a Sigmoid function in front of the layer.

It is to be noted that different embodiments may produce different advantages, and in different embodiments, any one or combination of the above advantages may be produced, or any other advantages may be obtained. Therefore, in the detection method of the present invention, steps S1 and S2 are general steps, and steps S3 to S4 are directed to different detection items, and the detection method is not limited to the silicon wafer contamination detection in the first embodiment, and can also be directed to other different detection items to achieve the effect of detecting various defects such as corresponding scratches, indentations, stippling, contamination, edge chipping, internal cracking, gaps, foreign matters, and miscellaneous points.

It should also be noted that the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. The use of the phrase "including a" does not exclude the presence of other, identical elements in the process, method, article, or apparatus that comprises the same element, whether or not the same element is present in all of the same element.

Finally, it should be noted that: the above examples are only intended to illustrate the technical solution of the present invention, but not to limit it; although the present invention has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some technical features may be equivalently replaced; and such modifications or substitutions do not depart from the spirit and scope of the corresponding technical solutions of the embodiments of the present invention.

Claims (10)

1. A detection method based on deep transfer learning is characterized by comprising the following steps:

s1, constructing a large-scale industrial defect detection gallery: through gallery management and labeling software, the information of pictures is labeled and uploaded manually by multiple ports, and a large-scale high-quality labeled industrial database is constructed together through warehousing of labeled data;

s2, constructing a basic model, and determining a basic deep learning model suitable for industrial defect detection;

s3, acquiring hundreds of magnitude images of detection items through an optical system based on transfer learning of a small amount of labels, and carrying out manual region labeling on the defect region of the piece to be detected; training the marked sample increment to a basic model in S2 to obtain a detection model aiming at the defects of the piece to be detected;

and S4, fusing the traditional algorithm and deep learning, and establishing a secondary judgment rule for the detection model to form a defect detection judgment model for the piece to be detected.

2. The detection method according to claim 1, characterized in that: step S1 includes:

s11, acquiring pictures through an optical system;

s12, manually screening, labeling and classifying the pictures;

and S13, constructing an industrial database based on the product level small graph and the defect level small graph according to the result in the S12.

3. The detection method according to claim 1, characterized in that: step S2 includes:

s21, constructing a universal segmentation model, wherein the universal segmentation model comprises a training prediction mode module, a model structure module, a loss function module and a preprocessing mode module;

and S22, training the general segmentation model of S21 by using the large-scale industrial defect detection gallery in S1 as a training set.

4. The detection method according to claim 3, characterized in that:

in the training prediction mode module, the training mode comprises cropping training and whole graph training, and the prediction mode comprises small graph prediction and whole graph prediction;

the model structure module comprises at least one of a classical deeplab series structure, a Unet series structure and a real-time network Bisenet series structure;

the loss function module comprises at least one of a BCE function, an IOU function, a BCE + IOU function and a Recall-BCE function;

the preprocessing mode module comprises at least one mode of a turning enhancement mode, an internal gray scale transformation mode and an external gray scale stretching mode.

5. The detection method according to claim 4, characterized in that:

the BCE function is:

… … … … … … formula 1;

the Recall-BCE function is as follows:

… … … … formula 2;

wherein,

the real value of the artificial mark machine value is obtained;

a predicted value for deep learning;

alpha is 0.05, and the value is obtained by manually adjusting the parameters;

is the sum of all OK sample deviations;

is the sum of all NG sample deviations.

6. The detection method according to claim 1, characterized in that: in step S3, the manual region labeling manner includes at least one of whole graph labeling, small graph labeling, target box labeling and pixel level labeling; the incremental training method comprises the following steps: and freezing a feature extraction layer of the model, and finely adjusting a feature classification layer through supplemented data to realize incremental training.

7. The detection method according to claim 1, characterized in that: step S4 includes:

s41, inputting a detection image of the to-be-detected piece collected by the optical system;

s42, obtaining a probability output graph through the transfer learning model in S3, and obtaining a first judgment result through a threshold value of 0.5;

and S43, calculating different evaluation indexes for each detection result in the S42, carrying out secondary judgment according to a preset index threshold value, determining whether the detection result is a real dirt defect, and obtaining a final result.

8. The detection method according to claim 7, characterized in that: the evaluation indexes in step S43 include uniformity, intensity, gray value deviation, and width and height of the circumscribed rectangle.

9. A computer-readable storage medium having stored thereon computer instructions, characterized in that: the computer instructions when executed perform the method of any one of claims 1-8.

10. A detection system based on deep transfer learning is characterized in that: the detection system comprises a computer (1) and an optical detection module (2) which are in telecommunication connection, wherein the optical detection module (2) comprises a detection camera (21) and a light source (22), and the detection camera (21) transmits the acquired surface image of the piece to be detected to the computer (1) in real time for image processing;

the computer (1) comprises a storage unit, an image processing unit and an output unit;

the storage unit stores an image of the to-be-detected piece collected by the detection camera and sends the collected detection image of the to-be-detected piece to the image processing module as required;

the image processing unit is used for processing image data of a detection image of a to-be-detected piece, performing defect judgment by operating the method of any one of claims 1 to 8, and classifying and labeling the defect picture to form a defect labeling graph;

the image output unit outputs the defect labeling diagram obtained by the image processing unit.

Images