Summary of the invention
The object of the present invention is to provide a kind of material microstructure intelligent recognition analysis systems.
Technical solution is as follows:
A kind of material microstructure intelligent recognition analysis system, comprising:
Data acquisition module: image is acquired from microscope device；
Deep learning tranining database:
For the grounding library picture materials of neural network recognization, include the basic apparent material of microstructure to be identifiedDatabase；Increased feature database in machine learning identification process；When the discrimination promotion to the microcosmic composed structure of certain material is arrivedMaturation training library after to a certain degree；
Main interface；
Measurement module；
Image pre-identification module: basic picture recognition module is responsible for examining image gray processing, image border after Image AcquisitionThe primary images preprocessing functions such as survey, image cutting；
Convolutional neural networks: the feature extraction to image microstructure, classified calculating, convolutional calculation deepen learning process, iterationDetection；
Training library: being divided into basic database, study library in depth and adult form tissue characteristics feature database, and training library is software preciousnessEducation resource and the software inevitable outcome and powerful guarantee that more move to maturity, deep learning intelligent image identification technologyIt can be by training library iteration optimization to more accurately microstructure identifies.
Further, main interface includes:
Visual image acquires device attribute reading, set interface, the attributes such as including resolution ratio；
Visualization measurement interface, including manual measurement, scribing line etc.；
Intelligent recognition opens, selection picture, selects the artificial config options such as microstructure sample.
Further, measurement module refers to Measurement Algorithm module, and Measurement Algorithm includes point measurement, straight line measures, circle is surveyedFixed, triangle measurement, two line angle measurements.
The second object of the present invention is to provide a kind of material microstructure intelligent recognition analysis method, steps are as follows:
1) data acquisition module acquires the image data of microscope device in real time first.
2) user is used for the learning training of certain material microstructure identification by subscriber interface module or manual configurationFoundation characteristic picture library.
3) characteristics of image is extracted, image optimization is carried out, obtains characteristic.
4) Classification and Identification is carried out to characteristics of image, to a variety of microstructure classification based trainings of material.
5) with mature neural network convolution algorithm, feature and classification to extraction carry out multitiered network iterative calculation,Obtain the characteristic results and context data of all about material microstructure.
6) export deep learning as a result, and the result and its feature context data are used to grow trained learning database, byStep improves software to the recognition efficiency and accuracy rate of material microstructure.
Compared with prior art, the present invention is theoretical using machine vision image recognition, adopts in original microscope digital imageCollect in Fundamentals of Measurement, increase the intelligent recognition function to material microstructure, more traditional measuring tool software has strongerIt is professional, and software use neural convolutional network can greatly improve the extraction to certain material microstructure characteristic,The classification that deep learning training system enables to software recognition accuracy to step up, identifies gradually increases, this is to meet notCarry out big data trend；It is especially with the obvious advantage in microscope intelligent recognition field in image measurement field, user is improved to materialJudgement and the recognition efficiency for expecting microstructure, reduce manual identified error.
Embodiment:
Referring to Fig. 1, the present embodiment shows a kind of material microstructure intelligent recognition analysis system, comprising:
Data acquisition module: image is acquired from microscope device；
Deep learning tranining database:
For the grounding library picture materials of neural network recognization, include the basic apparent material of microstructure to be identifiedDatabase；Increased feature database in machine learning identification process；When the discrimination promotion to the microcosmic composed structure of certain material is arrivedMaturation training library after to a certain degree；
Main interface；
Measurement module；
Image pre-identification module: basic picture recognition module is responsible for examining image gray processing, image border after Image AcquisitionThe primary images preprocessing functions such as survey, image cutting；
Convolutional neural networks: the feature extraction to image microstructure, classified calculating, convolutional calculation deepen learning process, iterationDetection；
Training library: being divided into basic database, study library in depth and adult form tissue characteristics feature database, and training library is software preciousnessEducation resource and the software inevitable outcome and powerful guarantee that more move to maturity, deep learning intelligent image identification technologyIt can be by training library iteration optimization to more accurately microstructure identifies.
Wherein:
Main interface includes:
Visual image acquires device attribute reading, set interface, the attributes such as including resolution ratio；
Visualization measurement interface, including manual measurement, scribing line etc.；
Intelligent recognition opens, selection picture, selects the artificial config options such as microstructure sample；
Measurement module refers to that Measurement Algorithm module, Measurement Algorithm include point measurement, straight line measurement, circle measurement, triangle measurement, twoLine angle measurement.
Referring to Fig.2, the analysis of material microstructure intelligent recognition is carried out using material microstructure intelligent recognition analysis system,Steps are as follows:
1) data acquisition module acquires the image data of microscope device in real time first.
2) user is used for the learning training of certain material microstructure identification by subscriber interface module or manual configurationFoundation characteristic picture library.
3) characteristics of image is extracted, image optimization is carried out, obtains characteristic.
4) Classification and Identification is carried out to characteristics of image, to a variety of microstructure classification based trainings of material.
5) with mature neural network convolution algorithm, feature and classification to extraction carry out multitiered network iterative calculation,Obtain the characteristic results and context data of all about material microstructure.
6) export deep learning as a result, and the result and its feature context data are used to grow trained learning database, byStep improves software to the recognition efficiency and accuracy rate of material microstructure.
In the present embodiment, following treatment of details also can be used:
(1) it obtains material high definition high magnification image to be detected and is pre-processed.
Pretreatment includes the following steps:
The high definition high magnification image of material to be detected is carried out size change over to default size by a；
Image after size change in step a is carried out edge detection and obtains edge image by b；
C carries out Hough transformation and extracts detection zone to obtain strip image to edge image；
Strip image is carried out slant correction and the strip image after correction is carried out cutting to obtain square with splicing by dImage.
Edge detection is carried out using Canny operator in step b, this method is not described in detail.
(2) pretreated image trained microstructure detection model progress structure detection in advance is input to obtainSpecified type microstructure is obvious on material or position that may be present, and provides the confidence that the position is target microstructureDegree.Wherein, the specified microstructure target detection model of material is the depth network based on deep learning, including successively cascade andAt feature extraction network and classifier and return device network, the feature extraction network carries out pretreated image specialSign, which is extracted, obtains characteristic image, and the classifier and recurrence device network classify returning and obtain material object to characteristic imageMicrostructure position and confidence level.
The training method of target microstructure detection model are as follows:
(a) depth network as described herein is established.
(b) magnanimity or certain material microstructure image as a large amount of as possible are acquired and carries out handmarking, it is emerging to iris out senseThe target area of interest, calculates the RECT (x, y, width, height) of the target area, obtains data sample.
(c) data sample depth network progress feature extraction and classifying is input to return to obtain the position of target to be detectedAnd confidence level；
(d) result of target position and confidence level and handmarking that step (c) obtains is compared, so as to adjust depth netEach link weight in network, and then complete the training of depth network.
It includes 5 successively cascade feature extraction basic units, each feature extraction unit packet that features described above, which extracts network,Include sequentially connected convolutional layer, local acknowledgement's normalization layer, maximum value pond layer and average value pond layer.
Above-mentioned convolutional layer is slided on the image using convolution kernel, convolution operation is carried out to image, to extract input pictureFeature obtains more rough characteristic pattern.
The above-mentioned local acknowledgement normalization layer spy more rough obtained in the convolutional layer using the field of 5 pixel *, 5 pixelIt is slided on sign figure, and carries out the normalization of mean value and variance to the pixel value in each field, not being illuminated by the light variation influencesRough characteristic pattern.
Above-mentioned maximum pond layer normalizes spy rough obtained in layer in local acknowledgement using the field of 5 pixel *, 5 pixelIt is slided on sign figure, and all pixels value in each field is maximized, obtain that there is the more accurate of translation invarianceCharacteristic pattern.
The spy that above-mentioned average pond layer uses the field of 5 pixel *, 5 pixel more accurate obtained in the maximum pond layerIt is slided on sign figure, and all pixels value in each field is averaged, obtain having the accurate of robustness to miniature deformationCharacteristic pattern, above-mentioned accurate characteristic pattern are the characteristic pattern of final corresponding feature extraction basic unit output.
By 5 successively cascade feature extraction basic unit final output characteristic images.
5 convolutional layer calculations successively in cascade feature extraction basic unit are as follows:
In first feature extraction basic unit, convolution kernel size is 7, for extracting biggish feature, exports feature map numberIt is 30.
In second feature extraction basic unit, convolution kernel size is 5, and for extracting medium sized feature, output is specialLevying map number is 60.
In third feature extraction basic unit, convolution kernel size is 3, for extracting lesser feature, exports characteristic patternNumber is 90.
In 4th feature extraction basic unit, convolution kernel size is 3, for extracting minutia, exports characteristic pattern numberMesh is 128.
In 5th feature extraction basic unit, convolution kernel size is 3, for extracting minutia, exports characteristic pattern numberMesh is 256.
Above-mentioned classifier and device network successively the cascade first full articulamentum and the second full articulamentum are returned, described first is completeArticulamentum input feature vector image, the second full articulamentum output end are connected with classifier and return device.
The characteristic image that above-mentioned first full articulamentum exports feature extraction network is weighted, obtain feature toAmount.
The feature vector of the first full articulamentum output is weighted in above-mentioned second full articulamentum, is refined and specialLevy feature vector outstanding.
Above-mentioned classifier is to the refinement of the second full articulamentum output and feature feature vector outstanding judges that judgement isIt is no to belong to material microstructure type to be identified and provide the confidence level for belonging to the type microstructure.
Above-mentioned recurrence device is to the refinement of the second full articulamentum output and feature feature vector outstanding carries out recurrence processing, obtainsTo the target microstructure location information detected.
(3) depth network is built.For a deep learning neural network, it is characterized in the foundation of classification, Feature SelectionMore representative, then classification results are better, therefore the step is that the area to be tested extracted to previous step carries out feature extraction,The validity feature of image to be detected is obtained, so that subsequent network classified, is returned.
This programme is built using the forefront algorithm in deep learning object detection field, Faster-RCNN algorithm ideaThe deep learning network for specifying microstructure to detect suitable for material.By four basic steps of target detection, (candidate region is rawAt feature extraction, classification, position refine) unify within the same depth network frame.Repeated computational expense greatly subtractsIt is small and completed in GPU, to substantially increase the speed of service.
Convolutional layer is slided on the image using convolution kernel, carries out convolution operation to image.Convolution operation can be used to pairImage does edge detection, sharpens, obscure etc., here for the extraction to characteristics of image.In the present solution, passing through control convolutionThe size of core extracts different features to control each layer, and each element passes through training department in each convolution kernel (being exactly a matrix)Divide and determine, that is, needs to modify weight when training.To image carry out convolution operation, be to each pixel in image-region respectively atEach element of convolution kernel is corresponding to be multiplied, new value of all sum of products as regional center pixel.
Local acknowledgement's normalization layer is to each pixel value progress mean value of (5*5 pixel) in image domains and returning for varianceOne changes, and reaches and eliminates background and illumination effect, the effect of prominent features.The normalization of mean value and variance is that is, original image is ledEach pixel value collection is normalized to the data that mean value is 0, variance 1 in domain, and such data have good measurement.NormalizationFormula are as follows:
Z=χ-μ/σ
Wherein z indicates the value after normalization, and x indicates that the pixel value before normalization, μ indicate pixel value in input picture fieldMean value, σ indicate the variance of pixel value in input picture field.
Maximum pond layer can be reduced to (region that this programme uses 3*3) all pixels value maximizing in neighborhoodData volume, while realizing the translation invariance to the feature of extraction.
Average pond layer can be reduced to (region that this programme uses 3*3) all pixels value ball average value in neighborhoodData volume, while enhancing robustness of the feature to miniature deformation of extraction.
In the network, one picture of every input can obtain 256 characteristic pattern outputs, on the characteristic pattern extractedGenerate candidate frame, the effect of candidate frame be for judging whether there may be targets in a certain region, then to these candidate frames intoRow classification and return, whether the effect of classification is judge in each candidate frame including object to be detected, the effect of recurrence be byIt is all to be combined, merge comprising the candidate frame of examined object, and position is modified.Carry out candidate frame classification withThe network structure of recurrence is completed by two full articulamentums, and the number of nodes of each full articulamentum is 1024.So far the software, that is, completeAt the feature extraction to a picture, while candidate frame is generated, and to the candidate frame that may include microstructure to be detectedIt is classified, and gives regression parameter.
Full articulamentum is any one node of this layer, all there is connection with next layer of all nodes, next layer in this wayOutput is just related with upper one layer of all inputs.And since first full articulamentum is connected with the output of feature extraction network,Exactly make final output related with all features extracted.The weight of each node of full articulamentum is needed by reversedLaw of Communication training determines.
Due to the presence of full articulamentum, so that final output is related to each feature, but training sample may not sometimesIdentified all situations can be represented, it is possible to over-fitting situation occur.Dropout layers by randomly hiding some full connectionsThe node increase of layer prevents over-fitting, enhances generalization ability.
SoftmaxWithLoss layers only exist in training, are the evaluations to full articulamentum final output, for countingCalculate the gap between classifier classification results and true value.
Belong to the probability of prospect (examined object) and background on each position of classifier final output；Window device is finalExport the parameter that scaling should be translated on each position.
For each position, classification layer exports the probability for belonging to foreground and background from 256 dimensional features；Window returnsLayer is returned to export 4 translation zooming parameters from 256 dimensional features.
It arrives this completion to the feature extraction of acquisition picture, while generating candidate frame, and to may include micrometer to be checkedThe candidate frame for seeing structure is classified, and gives regression parameter.
(4) sample labeling and model training
It marks by the technological means of front there are the region of target microstructure, provides that there are target microstructured areasAfter starting point coordinate and terminal point coordinate, the sample largely marked is sent into depth network and is trained, training process can be generalIt includes are as follows:
4.1) every image in training set is primarily looked at.
A. it to the true value candidate region of each calibration, overlaps the maximum candidate frame of ratio and is denoted as prospect sample.
B. to a) remaining candidate frame, if it is greater than 0.7 with some calibration overlap proportion, it is denoted as prospect sample.IfThe overlap proportion of itself and any one calibration is both less than 0.3, is denoted as background sample.
C. to a), b) remaining candidate frame, it is discarded.
D. it is discarded across the candidate frame of image boundary.
4.2) all features extracted and the candidate frame of generation are then sent into classification with Recurrent networks, can obtainedThe probability for belonging to foreground and background to each candidate frame and the parameter to return.The result and true value that these are obtained(result of handmarking) compares, and utilization cost function evaluates the difference of discre value and true value, finally by nerveThe general inverse iteration algorithm of network adjusts each link weight in network, so that error in classification and regression error are simultaneously mostSmallization.
(5) microstructure of certain material is identified using the model trained
In the present solution, the microstructure detection that original image carries out material can be directly inputted, it is in the final output imageNo there are the type microstructure samples, and if it exists, then provides the coordinate and its confidence level of the material microstructure position.ArbitrarilyA picture to be detected is inputted, which is the microscope high definition high magnification micro-structure diagram of production material, by above-mentioned allAfter target detection step, obtaining target detection terminates the image of output, mark in the image coordinate position for detect target andBelong to the confidence level of target microstructure.By largely training, the discrimination of target material microstructure reaches 92%.