Intelligent optimization system for fundus low-quality picturesTechnical Field
The invention belongs to the technical field of medical detection, and particularly relates to an intelligent optimization system for fundus oculi low-quality pictures.
Background
The fundus is mainly composed of retina, fundus blood vessel, optic nerve nipple, optic nerve fiber, macula on retina, and choroid behind retina, which are clinically known as fundus disease if lesions occur. The fundus disease is various in variety and complex in etiology, and because the interior of the eyeball is deep in the position, when the fundus disease is caused, many common hospitals cannot check the disease, so that the treatment of the fundus disease cannot be performed directly. Fundus photography examines morphological changes throughout the retina. The principle is to record the view seen under the ophthalmoscope with a special camera. Fundus photography can observe the morphology of retina, optic disc, macular area, retinal blood vessel, whether there is bleeding, exudation, hemangioma, retinal degenerated area, retinal hole, new blood vessel, atrophic plaque, pigment disorder, etc. on retina.
Fundus photography is used as a common checking means for fundus, can quickly acquire information of fundus of a patient, and is used for checking whether the fundus lesion exists in the patient so as to treat the fundus in time and reduce irreversible damage of visual function caused by the fundus lesion. Poor quality pictures are unavoidable due to operator errors, patient mismatch, fundus picture taking under the pupil, refractive interstitial turbidity, etc. Poor picture quality can lead to loss of diagnostic information and thus negatively impact downstream analysis, such as remote medical picture analysis, artificial intelligence automated analysis, and the like. In addition, if lesions are located in these ambiguous areas, they are prone to inadvertent missed diagnosis or similar lesions are hidden from view, but are not clear and misdiagnosis may occur.
Currently, there is no effective method to remove these factors that cause poor picture quality, for the problems of picture quality, particularly blur occlusion and or eyelid covering.
Disclosure of Invention
Aiming at the problems, the invention provides an intelligent optimization system for fundus oculi low-quality pictures.
The technical scheme of the invention is as follows: an intelligent optimization system for fundus oculi low-quality pictures mainly comprises a loading module, a training module, an optimization module, a synthesis module and an output module,
the loading module is used for preprocessing the initial fundus pictures and comprises a loading unit for loading fundus pictures in batches by utilizing an unsupervised learning CycleGAN network, a classifying unit for classifying the loaded fundus pictures into high-quality fundus pictures and low-quality fundus pictures according to definition, a labeling unit for labeling the high-quality fundus pictures and the low-quality fundus pictures respectively and a sorting unit for sorting the labeled high-quality fundus pictures and the low-quality fundus pictures respectively,
the training module is used for carrying out optimized training on the discriminant and comprises a high-quality fundus picture training unit, a low-quality fundus picture training unit and a generator training unit for training the discriminant,
the optimizing module is used for optimizing the low-quality fundus picture and comprises a feature identification unit for identifying fundus features on the low-quality fundus picture through a VGG16 network, a marking unit for marking the blurred or covered feature area on the low-quality fundus picture, a processing unit for removing the blurred or covered feature area marked by the marking unit and a rechecking unit for rechecking the low-quality fundus picture processed by the processing unit by utilizing the feature identification unit,
the synthesizing module is used for synthesizing the low-quality fundus pictures optimized by the optimizing module, the synthesizing module comprises a feature classifying unit used for classifying the features of the low-quality fundus pictures optimized by the optimizing module according to the marking features of the marking unit, a matching unit used for matching the processed low-quality fundus pictures with different marking feature areas according to the classifying features, a synthesizing unit used for superposing and synthesizing the low-quality fundus pictures matched by the matching unit, and a distinguishing unit used for distinguishing the blurred or covered features on the fundus pictures synthesized by the synthesizing unit,
the output module is used for checking and outputting the fundus pictures synthesized by the synthesis module, and comprises a feature comparison unit used for comparing the marking features of the marking unit with the fundus picture synthesized by the synthesis module, a generation unit used for generating clear fundus pictures without fuzzy or covering areas after the feature comparison unit is used for comparing, a storage unit used for storing the clear fundus pictures generated by the generation unit, and a transmission unit used for transmitting and outputting the clear fundus pictures generated by the generation unit.
Furthermore, a loss function is adopted as a classification function of the classification unit, a wasser tein distance is adopted as a classification standard of the loss function, the discrimination unit also adopts the loss function to discriminate, the discrimination standard is the wasser tein distance, the loss function proportion of the classification unit and the loss function proportion of the discrimination unit are 100:1, and the wasser tein distance represents the characteristic difference between the fundus picture after processing and the high-quality fundus picture.
Further, the labeling unit labels the high-quality fundus pictures as 1, labels the low-quality fundus pictures as 0, and the labeling unit can conveniently manage and process the fundus pictures.
Further, the ordering mode of the ordering unit is random ordering, and the random ordering can disturb the ordering method of the original pictures, so that the influence of the ordering of the original pictures on the optimization of the pictures is avoided.
Further, the high-quality fundus picture training unit is used for randomly selecting 5 high-quality fundus pictures to respectively execute 5 times of optimization on the discriminator, the low-quality fundus picture training unit is used for randomly selecting 5 low-quality fundus pictures to respectively execute 5 times of optimization on the discriminator, and updating parameters of the discriminator is closed after the optimization is finished, so that the discriminator cannot be trained, and the discriminator can be trained.
Further, the training optimization content of the generator training unit comprises the capability of optimizing the difference between the generated fundus photo and the high-quality fundus photo and optimizing the confusion judging device, and the joint network output of the generator and the judging device is the judging value of the picture generated by the generator and the judging device, wherein the judging value is 0 or 1.
The intelligent optimization method for performing fundus oculi low-quality pictures by using the intelligent optimization system mainly comprises the following steps:
s1: load classification
The shot fundus pictures are uniformly loaded through a loading unit of a loading module, the loaded fundus pictures are divided into high-quality fundus pictures and low-quality fundus pictures through a classifying unit, the high-quality fundus pictures are marked as 1 mahjong low-quality fundus pictures and marked as 0 by a marking unit, and then the high-quality fundus pictures and the low-quality fundus pictures are respectively randomly ordered by an ordering unit;
s2: optimization training
5 optimizations are carried out on the discriminator by using batch_size training data, each optimization in the 5 optimizations respectively uses a high-quality fundus picture training unit and a low-quality fundus picture training unit to optimize the discriminator for 10 times in total, and then the generator is optimally trained according to the training optimization content of the generator training unit;
s3: optimization processing
Firstly, performing feature recognition on a fundus picture through a feature recognition unit of an optimization module, then marking a blurred or covered feature area on a low-quality fundus picture through a marking unit, then removing the blurred or covered feature area on the low-quality fundus picture through a processing unit, and performing feature recognition on the removed low-quality fundus picture through a rechecking unit again until the blurred or covered feature area disappears;
s4: composite output
And (3) carrying out feature classification on the low-quality fundus pictures subjected to the optimization processing in the S3 through a feature classifying unit of the synthesizing module, matching the processed low-quality fundus pictures with different marked feature areas through a matching unit, superposing and synthesizing the matched processed low-quality fundus pictures through a synthesizing unit, judging the difference between the synthesized fundus pictures and the high-quality fundus pictures through a judging unit, carrying out picture generation on the synthesized fundus pictures meeting the output standard through a generating unit of the output module, and storing the generated high-quality fundus pictures through a storage unit or transmitting the generated high-quality fundus pictures through a transmission unit to be output.
The beneficial effects of the invention are as follows: the intelligent optimization system for fundus oculi low-quality pictures is particularly suitable for picture analysis of a large number of low-quality fundus oculi pictures, the classification unit is arranged in the loading module to classify the fundus oculi pictures with high quality and low quality, the discriminators and the generators are trained respectively through the training module, the discriminators rapidly judge the high-quality fundus oculi pictures and the low-quality fundus oculi pictures, the characteristic differences between the fundus oculi pictures generated by the generators and the high-quality fundus oculi pictures are smaller and smaller, fuzzy or covered characteristic areas in the low-quality fundus oculi pictures are removed through the optimization module, the low-quality fundus oculi pictures after the optimization processing are synthesized into the high-quality fundus oculi pictures through the synthesis module, and the generated high-quality fundus oculi pictures are output through the output module for use. The invention is based on a large number of high-quality and low-quality fundus pictures, and can automatically remove the fuzzy or covered area of the fundus pictures by utilizing the CycleGAN network training model to generate clear high-quality fundus pictures so as to avoid negative influence of the low-quality fundus pictures on downstream diagnosis and treatment analysis.
Drawings
FIG. 1 is a system block diagram of an intelligent optimization system of the present invention.
The device comprises a 1-loading module, a 11-loading unit, a 12-classifying unit, a 13-labeling unit, a 14-sorting unit, a 2-training module, a 21-high-quality fundus picture training unit, a 22-low-quality fundus picture training unit, a 23-generator training unit, a 3-optimizing module, a 31-feature recognition unit, a 32-labeling unit, a 33-processing unit, a 34-detecting unit, a 4-synthesizing module, a 41-feature classifying unit, a 42-matching unit, a 43-synthesizing unit, a 44-distinguishing unit, a 5-output module, a 51-feature comparing unit, a 52-generating unit, a 53-storing unit and a 54-transmitting unit.
Detailed Description
For the convenience of understanding the technical solution of the present invention, the following description will further explain the present invention by referring to fig. 1 and the specific examples, which do not limit the scope of the present invention.
Examples: as shown in fig. 1, an intelligent optimization system for fundus oculi low-quality pictures mainly comprises a loading module 1, a training module 2, an optimization module 3, a synthesis module 4 and an output module 5,
the loading module 1 is used for preprocessing initial fundus pictures, the loading module 1 comprises a loading unit 11 for loading fundus pictures in batches by using an unsupervised learning CycleGAN network, a classifying unit 12 for classifying the loaded fundus pictures into high-quality fundus pictures and low-quality fundus pictures according to definition, a labeling unit 13 for labeling the high-quality fundus pictures and the low-quality fundus pictures respectively, and a sorting unit 14 for sorting the labeled high-quality fundus pictures and low-quality fundus pictures respectively, wherein the labeling unit 13 labels the high-quality fundus pictures as 1, labels the low-quality fundus pictures as 0, the sorting mode of the sorting unit 14 is random sorting,
the training module 2 is used for carrying out optimized training on the discriminator, the training module 2 comprises a high-quality fundus picture training unit 21 used for training the discriminator, a low-quality fundus picture training unit 22 and a generator training unit 23 used for training a generator, the high-quality fundus picture training unit 21 is used for randomly selecting 5 high-quality fundus pictures to respectively carry out 5 times of optimization on the discriminator, the low-quality fundus picture training unit 22 is used for randomly selecting 5 low-quality fundus pictures to respectively carry out 5 times of optimization on the discriminator, updating parameters of the discriminator is closed after the optimization is finished, the discriminator cannot be trained, the training optimization content of the generator training unit 23 comprises the capability of optimizing the difference between the generated fundus picture and the high-quality fundus picture and optimizing the confused discriminator, the combined network output of the generator and the discriminator is the discrimination value of the picture generated by the generator and the discriminator, the discrimination value is 0 or 1,
the optimizing module 3 is used for optimizing the low-quality fundus picture, the optimizing module 3 comprises a feature identifying unit 31 for identifying fundus features on the low-quality fundus picture through a VGG16 network, a marking unit 32 for marking the blurred or covered feature area on the low-quality fundus picture, a processing unit 33 for removing the blurred or covered feature area marked by the marking unit 32, and a rechecking unit 34 for rechecking the low-quality fundus picture processed by the processing unit 33 by utilizing the feature identifying unit 31,
the synthesizing module 4 is used for synthesizing the low-quality fundus pictures optimized by the optimizing module 3, the synthesizing module 4 comprises a feature classifying unit 41 used for classifying the low-quality fundus pictures optimized by the optimizing module 3 according to the marking features of the marking unit 32, a matching unit 42 used for matching the processed low-quality fundus pictures with different marking feature areas according to the classifying features, a synthesizing unit 43 used for superposing and synthesizing the low-quality fundus pictures matched by the matching unit 42, and a distinguishing unit 44 used for distinguishing the blurred or covered features on the fundus pictures synthesized by the synthesizing unit 43,
the classification function of the classification unit 12 adopts a loss function, the classification standard of the loss function adopts a loss function distance, the discrimination unit 44 also adopts the loss function to discriminate, the discrimination standard is the loss function distance, the loss function proportion of the classification unit 12 and the loss function proportion of the discrimination unit 44 are 100:1,
the output module 5 is configured to synthesize the fundus image by the synthesis module 4 to perform checking output, and the output module 5 includes a feature comparison unit 51 configured to compare the marking feature of the marking unit 32 with the feature of the fundus image synthesized by the synthesis module 4, a generating unit 52 configured to generate a clear fundus image without a blurring or covering area after the comparison by the feature comparison unit 51 through a generator, a storage unit 53 configured to store the clear fundus image generated by the generating unit 52, and a transmission unit 54 configured to transmit and output the clear fundus image generated by the generating unit 52.
The intelligent optimization method for fundus oculi low-quality pictures by using the embodiment mainly comprises the following steps:
s1: load classification
The shot fundus pictures are uniformly loaded through a loading unit 11 of the loading module 1, then the loaded fundus pictures are divided into high-quality fundus pictures and low-quality fundus pictures through a classifying unit 12, the high-quality fundus pictures are marked as 1 mahjong low-quality fundus pictures and are marked as 0 by a marking unit 13, and then the high-quality fundus pictures and the low-quality fundus pictures are respectively randomly ordered by an ordering unit 14;
s2: optimization training
5 optimizations are performed on the discriminator by using the batch_size training data, each optimization in the 5 optimizations respectively uses a high-quality fundus picture training unit 21 and a low-quality fundus picture training unit 22 to optimize the discriminator for 10 times in total, and then the generator is optimally trained according to the training optimization content of the generator training unit 23;
s3: optimization processing
Firstly, feature recognition is carried out on an eye fundus picture through a feature recognition unit 31 of an optimization module 3, then a fuzzy or covered feature area on a low-quality eye fundus picture is marked through a marking unit 32, then the fuzzy or covered feature area on the low-quality eye fundus picture is removed through a processing unit 33, and the feature recognition is carried out on the removed low-quality eye fundus picture through a rechecking unit 34 again until the fuzzy or covered feature area disappears;
s4: composite output
The low-quality fundus pictures after the optimization processing in S3 are subjected to feature classification by the feature classification unit 41 of the synthesis module 4, the low-quality fundus pictures after the processing with different marked feature areas are matched by the matching unit 42, the matched low-quality fundus pictures after the processing are overlapped and synthesized by the synthesis unit 43, the difference between the synthesized fundus pictures and the high-quality fundus pictures is judged by the judging unit 44, the synthesized fundus pictures meeting the output standard are subjected to picture generation by the generation unit 52 of the output module 5, and the generated high-quality fundus pictures can be stored by the storage unit 53 or transmitted and output by the transmission unit 54.