Diffraction image measurement and analysis system and method for automatically distinguishing particlesTechnical Field
The invention relates to a diffraction image measurement and analysis system and a method. In particular to a diffraction image measuring and analyzing system and a method for automatically distinguishing particles, which can measure diffraction image signals with adjustable wavelength and polarization formed by particle coherent scattered light, calculate and extract image characteristics of the diffraction image signals highly related to three-dimensional structural characteristics in the particles and automatically, quickly and accurately analyze and distinguish the particles.
Background
Researchers in many fields such as cell biology research, biotechnology research, drug research, environmental pollution monitoring, atmospheric science and the like need a method and an instrument system capable of rapidly and accurately analyzing and distinguishing a large number of single particles with linear scale of one micron to hundreds of microns. In many cases, the function of microparticles including biological microparticles typified by cells or the interaction between the microparticles and the outside world is often closely related to the three-dimensional structure of the microparticles. Therefore, comparing the characteristics and differences of the three-dimensional structural morphology of the particles is one of the best methods for analyzing and identifying the particles. For example, optical microscopy was one of the most commonly used instruments for observing the morphology of particulate structures by humans at the earliest time. However, the method of discriminating fine particles using optical microscope analysis has limitations for the reason that it is difficult to perform rapid analytical discrimination of a fine particle population containing a large number of fine particles. First, a conventional optical microscope (such as a fluorescence microscope, a bright-field or dark-field microscope, etc.) is designed based on the incoherent imaging principle, and an image thereof is formed by two-dimensional projection of a three-dimensional structure of a particle. Second, since the microscopic image is a two-dimensional projection of the three-dimensional structure of the particle, the image analysis for identifying the particle usually requires a very complicated image analysis method, especially a manual analysis when analyzing the cell having a complicated three-dimensional structure morphology, so that the image analysis method based on the optical microscope is difficult to be automated, and the related optical microscope operation and image measurement also require a manual operation, which is time-consuming, prone to errors, and extremely low in analysis speed. Third, many particles, including cells, do not contain molecules that are specifically absorbing or fluorescing in the visible and near infrared wavelength ranges, and therefore must be stained to allow observation of their structural morphology under an optical microscope, often requiring expensive reagents and complicated and time-consuming procedures, and potentially interfering with the biological particles, such as cells, being observed. In recent years, optical microscopy has been developed, for example, by using a confocal technique, a plurality of two-dimensional images with a short depth of field can be acquired, and the three-dimensional structural morphology of the microparticles can be reconstructed by superimposing the two-dimensional images. Confocal optical microscopy only solves the first problem described above, but requires a longer time, while other problems remain unsolved.
Based on the research on optical measurement of particles represented by cells under the laminar flow rapid flow state since the sixties of the last century, a flow cytometer becomes an instrument which is an integrated instrument for rapid measurement and analysis of a large number of single cells and has the advantages of current collector mechanics, laser technology, photoelectric measurement and data processing research results. Flow cytometers utilize concentric nozzles and fluid pressure differentials to create a laminar flow of sample and sheath flows within a sample chamber. The sheath flow surrounding the sample flow reduces the diameter of the sample flow containing the particles by a pressure difference, forcing the particles to flow in a single file through the excitation beam, the particles illuminated by the excitation beam producing scattered light of the same wavelength as the excitation light, the intensity of which varies with the scattering angle. Such scattered light having a wavelength equal to the excitation light wavelength is also called elastic scattered light, and is radiation generated by molecular electric dipoles formed by induction of the electromagnetic field of the excitation beam inside the fine particles. The inductive molecular electric dipole concentration distribution inside the microparticle is expressed by the optical refractive index distribution inside the microparticle, and thus the three-dimensional structure inside the microparticle can be expressed by the optical refractive index three-dimensional distribution thereof. Such as a non-uniform three-dimensional distribution of the optical refractive index inside the particle or a different optical refractive index than the carrier material in which it is suspended, scattered light is present and is generally the strongest of the various optical signals produced under conditions in which the particle is illuminated. The particles irradiated with the excitation light beam, for example, contain fluorescent molecules and also generate fluorescence having a wavelength longer than that of the excitation light, and the fluorescence is emitted by the fluorescent molecules inside the particles after being excited. Many microparticles, including cells, contain no or very little fluorescent molecules, so that they produce a fluorescent signal of sufficient intensity only after staining. The flow cytometer can rapidly analyze and distinguish the particles by measuring fluorescence and scattered light signals generated by the dyed particles, and the processing speed of the flow cytometer can reach thousands of particles per second. Compared with the optical microscopic analysis method, the method has unique advantages in analyzing communities containing a large number of particles and in statistical distribution. Since the eighties of the last century, flow cytometers have found wide application in the fields of cell biology research, pollution monitoring and others.
The current flow cytometry products can be divided into an angle integral type and a noncoherent image type according to the optical signal measurement mode, and most of the current flow cytometry products are angle integral type. In such a flow cytometer, the scattered light signal and the fluorescence signal generated by the flowing particles under the irradiation of the incident light beam are received by different single photosensors (e.g., photodiodes, photomultipliers, etc.) to generate corresponding output electrical signals. The single sensor is a sensor that outputs only 1 electrical signal, and the signal intensity of the single sensor is proportional to the integral value of the scattered light or fluorescence signal intensity within the solid angle formed by the sensor area relative to the light source, which is referred to as the scattered light or fluorescence signal. The fluorescence signal is related to the presence, absence and amount of specific molecules (e.g., certain protein molecules in cells that can bind to the fluorescent molecules) contained within the particle, while the angularly integrated scattered light signal is related only to the particle volume and the internal optical refractive index uniformity, i.e., the particle size, which is the angular integral of the optical refractive index distribution, as opposed to the optical refractive index distribution. The scattered light and the fluorescence signal are combined, and data analysis is carried out through a computer, so that the community containing a large number of particles can be automatically analyzed and distinguished, and the purpose of rapidly distinguishing the types of the particles in the community is achieved. Currently, an angle-integrating flow cytometer can measure 2 to 10 fluorescence signals and 2 scattered light signals. The fluorescence signal contains no structural information, and although 2 scattered light signals (forward and side scattered light signals) can provide information on volume and internal granularity, the content of the structural information is extremely limited, so that the angle integral flow cytometer mainly relies on the fluorescence signal to perform rapid analysis and discrimination on particles.
In recent years, image measurement techniques have been applied to flow cytometers, and incoherent image type flow cytometers have been formed. The flow cytometer measures the spatial angle distribution of incoherent light signals by using an image sensor such as a Charge Coupled Device (CCD) camera based on a traditional optical microscopy method, and can output image data such as fluorescence, bright field and dark field, but all images are two-dimensional projections of a three-dimensional structure of particles. Compared with the angle integral type flow cytometer, the incoherent image type flow cytometer can measure and output a plurality of images for each flowing microparticle, and the structural information contained therein is significantly increased, so that the microparticle structure can be analyzed more finely. However, as with the conventional optical microscopy method, the image-based flow cytometer using incoherent light signal imaging has similar limitations, such as inability to analyze and distinguish the particles according to their three-dimensional structure morphology characteristics, and need to stain the particles to obtain fluorescence images. More importantly, because the relationship between the two-dimensional projection image and the three-dimensional structure is very complex, and manual analysis is usually required, it is difficult to perform automatic image data analysis on a colony containing a large number of particles through computer software, and the purpose of rapidly distinguishing the types of the particles in the colony cannot be achieved. Since the image signal type flow cytometer can measure hundreds to thousands of particles per second, the total amount of image signal data is very large, and if automatic image signal analysis cannot be realized, the application thereof is greatly limited.
As described above, the particles irradiated with the excitation light beam generate scattered light having the same wavelength as the excitation light. If the excitation beam is a highly coherent beam, the scattered light is highly coherent at equal wavelengths. The particles containing fluorescent molecules also produce fluorescence at the same time, and the wavelength of the fluorescence is different from that of the excitation light beam, and the fluorescence does not have coherence. If a laser beam with high coherence is used as the excitation beam, the scattering electromagnetic field with high coherence generated by the induced molecular electric dipole inside the particle forms a diffraction distribution with the light intensity changing with the angle in the space due to the phase difference, the diffraction distribution and the polarization state of the coherent scattering light are determined by the wavelength and the polarization state of the excitation beam and the three-dimensional distribution of the difference between the refractive index of the light inside the particle and the refractive index of the suspending medium, so the diffraction distribution and the polarization state of the coherent scattering light intensity are highly related to the three-dimensional structure form inside the particle and also related to the wavelength and the polarization state of the excitation beam. And measuring the diffraction distribution of the coherent scattered light by using an image sensor to obtain a diffraction image. The three-dimensional structural characteristics of the particles are calculated and analyzed through a plurality of diffraction images, and the three-dimensional structural form of the particles or related information can be obtained. The earliest applications of this method were laser holographic imaging in the visible wavelength range and X-ray diffraction techniques for estimating the three-dimensional structure of biological macromolecules in the X-ray wavelength range. In general, the calculation of the three-dimensional structure of the particle requires obtaining enough multiple (5 to 10 or more) diffraction images under different incidence angles of the excitation light beam, and then performing complicated three-dimensional structure reconstruction calculation. In the flow cytometer, since the particles flow rapidly, it is difficult to obtain a plurality of diffraction image data at different angles at the same time, and even if a plurality of images can be obtained, it is impossible to complete the three-dimensional reconstruction calculation within several seconds or less. In addition, when the particles flow through the incident beam in the laminar flow liquid, there may exist optical interfaces with very small radius of curvature in the vicinity thereof, including interfaces caused by the difference in refractive index between the sheath flow and the fluid chamber material, such as glass. These optical interfaces with very small radii of curvature typically cause scattered light fields that are image noise, which can generally be larger or much larger than the diffracted light intensity distribution produced by the particles, so that the measured diffraction image has very little signal contrast with respect to the particle structure. Obtaining high quality optical diffraction images associated with the particle structure requires reducing or eliminating image noise due to these optical interfaces, a difficult technical problem to solve. In addition, how to obtain information highly related to the three-dimensional structural features of the particles by using the obtained diffraction image data and rapidly analyze and classify colonies containing a large number of particles according to the information is an unsolved problem. Due to these problems, although most of the current commercial flow cytometers use a laser beam as an excitation beam, the particles cannot be distinguished by measuring and analyzing diffraction images. In the angle integration type flow cytometer, the measured scattered light signal is angle integration, so that the diffraction distribution which changes with the angle and is caused by the coherence of the scattered light basically disappears in the signal after the angle integration, and the obtained structural characteristics only comprise simple characteristics of volume and internal granularity; in the incoherent image signal type flow cytometer, the fluorescence image is an incoherent image due to the change of the fluorescence wavelength relative to the excitation light beam wavelength, and the bright field image or the dark field image is generally obtained under the incoherent white light irradiation condition and belongs to an incoherent image.
Based on recent years of research on the theory and experiments on light scattering of microparticles including cells, a new type of diffraction image-based Flow cytometer method has been published, discussing in detail the visual references (e.g., x.h. hu, k.m. jacobs, j.q. lu, "Flow cytometer apparatus for three dimensional diffusion imaging and related methods", PCT Application No. wo 2009/151610 by easy access biology University). This new type of diffraction image type flow cytometer proposes a concept of placing a laminar flow in a fluid chamber formed mainly of a liquid, and uses an image sensor such as a ccd camera to record the angular distribution of coherent scattered light generated by particles, thereby obtaining a diffraction image signal with high contrast. Experimental results show that the novel Diffraction image signal type flow cytometer can distinguish microparticles having different three-dimensional structures according to microparticle Diffraction image signal analysis, and the reference documents are discussed in detail (e.g., k.m. jacobs, l.v. yang, j.ding, a.e.ekPenng, r.castellone, j.q.lu, x.h.hu, "differentiation imaging of spheres and cells with a microscopical object", Journal of biophotonic, vol.2, pp.521-527 (2009); k.m. jacobs, j.q.lu, x.h.hu, "Development of a differential imaging flow meter", Optics Letters, vol.34, vol.2985-2987 (2009)). Two-dimensional diffraction images of microparticles obtained by a diffraction image type flow cytometer have been proven to be highly correlated with their three-dimensional structure by means of microparticle Light scattering models based on classical electrodynamics theory and large-scale numerical calculations, from which many features related to the three-dimensional structural features of microparticles can be extracted, see references (e.g., J.Q.Lu, P.Yang, X.H.Hu, "geometry of Light scattering from a biological captured cell using the FDTD method", Journal of biological Optics, vol.10, 022024 (2005); R.S.Brock, X.H.Hu, D.A.Weidner, J.R.Mount, J.Q.Lu,. offset of transformed cell structure on Light scattering: TD-test of B-cell 3, D.D.3, sample of simulation of Light distribution, Journal of sample distribution, 25. 12, 25. sub.12. volume). Although the above-mentioned research provides a system and a method for rapidly measuring a two-dimensional diffraction image signal of a particle in a flow state and a particle light scattering model based on the classical electrodynamic theory proves that the two-dimensional diffraction image characteristic and the three-dimensional structure characteristic of the particle are highly correlated, extracting the three-dimensional structure parameter inside a single particle from the diffraction image data distribution through the particle light scattering model and numerical calculation requires a large amount of calculation, and even on a large-scale computer, it often takes several hours or more to obtain reliable parameter data of the three-dimensional structure of the single particle, and a large amount of particles cannot be analyzed. How to measure a plurality of diffraction images simultaneously and calculate and obtain image mode characteristics highly related to the three-dimensional structure in the particles from the diffraction images so as to achieve the purpose of rapidly and accurately analyzing and distinguishing the particles, and an effective system design and analysis method is not available.
Disclosure of Invention
The technical problem to be solved by the invention is to provide a diffraction image measurement and analysis system and a method for automatically distinguishing particles, which can measure a plurality of diffraction images with adjustable wavelength and polarization, quickly analyze and extract image mode characteristics highly related to the morphological characteristics of three-dimensional structures in the particles according to the image data, analyze and distinguish communities containing a large number of particles and automatically classify the communities.
The technical scheme adopted by the invention is as follows: a diffraction image measuring and analyzing system and method for automatically distinguishing particles comprises a sample flow consisting of flowing particles, a coherent excitation light beam intersecting the sample flow, a first scattered light receiving objective part for measuring coherent scattered light beams with a first central scattering angle emitted by the particles excited by the coherent excitation light beam, a first light splitting and filtering part, a first imaging measurement and data output part, an image processing circuit, a computer part and a display part connected with the image processing circuit and the computer part; wherein,
the first light splitting and filtering part is used for splitting and filtering the received scattered light emitted by the particles;
the first imaging measurement and data output part is used for performing imaging measurement and data output on split light and filtered scattered light so as to obtain a diffraction image formed by coherent scattered light beams, and comprises a first focusing lens and a second focusing lens which are used for respectively focusing the scattered light in different directions emitted by the splitting and filtering part, a first image sensor positioned behind the first focusing lens and a second image sensor positioned behind the second focusing lens, a first data output input port connected behind the first image sensor for performing data output and input, and a second data output input port connected behind the second image sensor;
the image processing circuit and the computer part are used for receiving the output information of the data output part, extracting the data characteristics of different wavelengths and polarization diffraction images, and calculating, analyzing and distinguishing.
The display part displays the statistical data of the calculation, analysis and identification results.
A second scattered light receiving objective part for measuring a coherent scattered light beam having a second central scattering angle emitted from the particle excited by the coherent excitation light beam; a second light splitting and filtering section for splitting and filtering the received scattered light; a second imaging measurement and data output part for performing imaging measurement on the split and filtered scattered light and outputting a diffraction pattern of the scattered light; the second imaging measurement and data output part is connected with the image processing circuit and the computer part; the second scattered light receiving objective part, the second light splitting and filtering part and the second imaging measurement and data output part have the same structures as the first scattered light receiving objective part, the first light splitting and filtering part and the first imaging measurement and data output part.
The measured solid angle of the coherent scattered beam emerging from the particle excited by the coherent excitation beam is in the range of 0 to pi steradians.
The first scattered light receiving objective part is a microscope objective composed of a plurality of lenses arranged in sequence.
The first light splitting and filtering part comprises a light splitting piece, a first narrow-band filter and a first polarizing filter which are sequentially arranged and used for receiving scattered light transmitted by the light splitting piece, and a second narrow-band filter and a second polarizing filter which are sequentially arranged and used for receiving the scattered light reflected by the light splitting piece.
The first light splitting and filtering part comprises a light splitting piece, a first narrow-band filter and a second narrow-band filter, a first polarizing filter and a second polarizing filter, wherein the first narrow-band filter and the second narrow-band filter are used for receiving scattered light emitted from the light splitting piece in different directions; or the light splitting and filtering part consists of a polarization narrow-band light splitting sheet or consists of a light splitting sheet and a prism or a diffraction grating.
And a light intensity attenuation sheet is arranged behind the first polarizing filter and the second polarizing filter.
The first data output input port and the second data output input port have the same structure, provide bias voltage required by the work of the image sensor and power supply voltage of a cooling power supply of the image sensor, provide control signals of the image sensor and time sequence signals of analog pulse data signals output by the sensor, and provide data signals output by the digitized analog pulse data signals of the sensor.
The image processing circuit and the computer part comprise a first image sensor power supply and a second image sensor power supply which respectively provide bias voltage and power supply voltage for the first image sensor and the second image sensor, a first image processing circuit and a computer which respectively receive image signals through a first data output input port and a second data output input port, a second image processing circuit and a computer,
the first image processing circuit and the computer, and the second image processing circuit and the computer are two computers with completely same structures, or the same computer; the image processing circuit and the computer can store, process and calculate the received image signals, extract characteristic parameters of different wavelengths and polarization diffraction image modes and output display parameter data.
An analysis method of a diffraction image measurement analysis system for automatically distinguishing fine particles, comprising the steps of:
the first step is as follows: measuring the spatial distribution of particle scattered light under the irradiation of the multi-wavelength excitation light beam through an image sensor and a corresponding light path to obtain corresponding adjustable wavelength and polarization diffraction image data;
the second step is as follows: transmitting all particles in the detected particle community to an image processing circuit memory in a computer for image analysis in the next step in different wavelengths, polarization diffraction image data and a population resolution criterion;
the third step: reducing the number of pixel gray value bits from more than 8 bits to 8 bits according to the pixel gray value distribution of the diffraction image and carrying out image space coordinate transformation according to an image distribution mode of the pixel gray value bits;
the fourth step: performing characteristic discrimination on the transformed different wavelengths and polarization diffraction images by using a characteristic discrimination method and selecting a characteristic region;
the fifth step: and selecting diffraction image mode characteristics from the different wavelength and polarization diffraction images after the characteristics are screened in the characteristic region.
A sixth step: forming a diffraction image characteristic parameter vector according to the mode characteristic parameters of different wavelengths and polarization diffraction images, and determining the position of the measured particles in a diffraction image characteristic parameter vector sample space according to the vector;
a seventh step of: after the positions of all particles in the detected particle community in a diffraction image characteristic parameter vector sample space are determined, automatically classifying the detected particle community into different populations according to the position distribution of the particles in the characteristic parameter vector sample space and an input population resolution criterion, and outputting corresponding data.
The first step is that the tunable wavelength and polarization diffraction image data is: when diffraction pattern data is measured, the wavelength and polarization of scattered light emitted from the microparticles under irradiation of a multi-wavelength excitation beam are selected by an optical path element in front of the image sensor.
And the population distinguishing criterion in the second step is obtained by particle community training data.
The diffraction image pattern feature of the fifth step is a diffraction image pattern feature calculated based on pixel gradation correlation.
The pixel-based gray scale correlation calculation in the fifth step is based on a parallel algorithm that divides the input image data into a plurality of data streams and calculates the data streams separately.
The invention relates to a diffraction image measurement and analysis system and a method for automatically distinguishing particles, which utilize coherent scattered light generated by flowing particles under the irradiation of coherent excitation light beams to measure diffraction image data with adjustable wavelength and polarization of spatial distribution, quickly analyze and extract diffraction image mode characteristic parameters to form characteristic parameter vectors highly related to three-dimensional structure characteristics in the particles, and automatically classify the particles in a measured particle group according to the characteristic parameter vectors. The invention has the advantages of fast analysis and discrimination of a large amount of particles according to the three-dimensional structure characteristics in the particles, no need of dyeing the particles, no influence on the internal biochemical process of the detected particles such as cells and the like, and low measurement cost.
Drawings
FIG. 1 is a schematic view of a first embodiment of a diffraction image measurement analysis system for automatically discriminating fine particles according to the present invention;
FIG. 2 is a schematic diagram of a second embodiment of the diffraction image measurement analysis system for automatically discriminating fine particles according to the present invention;
FIG. 3 is a flow chart of a diffraction image measurement analysis method for automatically discriminating fine particles according to the present invention;
FIG. 4 is a flow chart of diffraction image pixel gray scale correlation calculation software according to the present invention;
FIG. 5 is a diagram illustrating the effect of image processing during the calculation of pixel intensity correlation of diffraction image according to the present invention.
Wherein:
1: flowing particles 2: sample flow
3: coherent excitation beam 4: coherent scattered beam
5: and (6) microscope objective lens: light splitting sheet
7: first narrowband filter 8: first polarizing filter
9: first focusing lens 10: first scattered light
11: first image sensor 12: first data output/input port
13: first image sensor power supply 14: first image processing circuit and computer
15: second narrowband filter 16: second polarizing filter
17: second focusing lens 18: second scattered light
19: the second image sensor 20: second data output input port
21: second image sensor power supply 22: second image processing circuit and computer
23: first center scattering angle 24: scattered light beam
25: microscope eyepiece 26: light splitting sheet
27: narrow band filter 28: polarizing filter
29: focusing lens 30: scattered light
31: the image sensor 32: data output input port
35: narrow band filter 36: polarizing filter
37: focusing lens 38: scattered light
39: the image sensor 40: data output input port
43: second center scattering angle a: first scattered light receiving objective lens part
B: first light splitting and filtering section C: first imaging measurement and data output section
D: image processing circuit and computer portion a': scattered light receiving objective lens part
B': splitting and filtering part C': imaging measurement and data output section
Detailed Description
The diffraction image measurement and analysis system and method for automatically identifying particles according to the present invention will be described in detail with reference to the accompanying drawings and examples.
The invention relates to a diffraction image measurement and analysis system and a method for automatically distinguishing particles, which are used for measuring diffraction images with different wavelengths and polarization by using an adjustable imaging system, then rapidly analyzing pixel gray level associated parameters of a plurality of diffraction image data by computer software, extracting image mode statistical parameter vectors and rapidly classifying communities containing a large number of particles according to the parameter vectors and population resolution criteria. The invention can also be implemented by a method for rapid population classification of colonies comprising a large number of particles by computer software on image patterns contained in the diffraction image data. The methods described herein may also be implemented by other optical systems, methods, and computer software.
One of the schemes of the diffraction image measurement and analysis system and the method for automatically distinguishing the particles is to use a microscope objective to receive the respective coherent scattered light on different wavelengths generated by the particles excited by multi-wavelength incident light, select to respectively measure the spatial angle distribution of the scattered light under the conditions of different wavelengths, polarization and angle ranges through the combination of a plurality of image sensors and other optical elements, output a plurality of diffraction image data of different wavelengths and polarization directions, analyze the respective and associated characteristics of the diffraction image data of different wavelengths and polarization by using computer software, and further quickly and accurately analyze and distinguish a large number of particles. Compared with the traditional angle integral type incoherent image type flow cytometer, the diffraction image type method can be used for rapidly and accurately analyzing and distinguishing the particles through the diffraction image characteristics highly related to the morphological characteristics of the three-dimensional structure in the particles.
As shown in fig. 1, the diffraction image measurement and analysis system for automatically identifying particles of the present invention comprises asample flow 2 composed of flowing particles 1, acoherent excitation beam 3 intersecting thesample flow 2, a first scattered light receiving objective part a of a coherent scatteredlight beam 4 having a firstcentral scattering angle 23 emitted from the particles excited by thecoherent excitation beam 3, a first light splitting and filtering part B, a first imaging measurement and data output part C, an image processing circuit and computer part D, and a display part connected to the image processing circuit and computer part D;
wherein,
the solid angle range of the detectedcoherent scatter beam 4 emitted from the excited particle may be from 0 to pi steradians;
the first scattered light receiving objective part A is composed of amicroscope objective 5 which is composed of a plurality of lenses arranged in sequence and has proper numerical aperture and working distance. The design and manufacture of the three-dimensional light source device need to meet the requirement of measuring the distribution of scattered light through diffraction images in different three-dimensional angles.
The light intensity of thecoherent scatter beam 4 to be measured varies with the angle between the beam propagation direction and the incident beam propagation direction, i.e. the scattering angle. The solid angle range of thecoherent scatter beam 4 under measurement, which is collected by themicroscope objective 5, is determined by the numerical aperture of themicroscope objective 5, and the intensity in this angle range forms a diffraction image as a function of the scattering angle, and the firstcentral scattering angle 19 can be used to mark the central angle position of this angle range. The solid angle of thecoherent scatter beam 4 under test may range from 0 to pi steradians and the central scatter angle may range from 5 to 180 degrees, where the central scatter angle is represented by the firstcentral scatter angle 23. Depending on the range of its central scatter angle, thecoherent scatter beam 4 under test may be referred to as a forward, side or backscattered beam. The forward scatter beam is defined as having a central scatter angle between 5 and 90 degrees, the side scatter beam is defined as having a central scatter angle between 45 and 135 degrees, and the back scatter beam is defined as having a central scatter angle between 90 and 180 degrees.
The first light splitting and filtering part B is used for splitting and filtering scattered light emitted by particles excited by the coherentexcitation light beam 3, and comprises alight splitting piece 6, a first narrow-band filter 7 and a firstpolarizing filter 8 which are used for receiving the transmitted scattered light of thelight splitting piece 6 and are sequentially arranged, and a second narrow-band filter 13 and a secondpolarizing filter 14 which are used for receiving the reflected scattered light of thelight splitting piece 6 and are sequentially arranged. A light intensity attenuation sheet can be arranged behind the firstpolarizing filter 8 and the secondpolarizing filter 14, so that light intensity entering the sensor can be automatically adjusted according to light intensity data of the image sensor, and overload and saturation of the sensor are avoided.
The light splitting and filtering part can be obtained according to different light path designs, therefore, the first light splitting and filtering part B can also be: the polarization light source comprises a polarization beam splitter, a first narrow-band filter 7 and a second narrow-band filter 15 which receive scattered light emitted from the polarization beam splitter in different directions, afirst polarization filter 8 positioned behind the first narrow-band filter 7 and asecond polarization filter 16 positioned behind the second narrow-band filter 15; or the light splitting and filtering part B consists of a polarization narrow-band light splitting sheet or consists of a light splitting sheet and a prism or a diffraction grating.
The optical wavelength bandwidth of a general narrow-band filter is between 0.5 nm and 50 nm, the central wavelength of the bandwidth is adjustable, and only light waves with the wavelength within the bandwidth can pass through the narrow-band filter under the condition of small attenuation. Polarizing filters allow only light waves in a certain polarization state to pass with little attenuation, such as a horizontally linear polarization state or a left-handed polarization state, etc. Because the bandwidth center wavelength and polarization of the first filter part and the second filter part can be respectively and independently adjusted, diffraction images with different wavelengths and polarizations can be obtained according to different particle analysis requirements and then are respectively output to the image processing circuit through the first data output andinput port 12 and the second data output andinput port 20.
Thesample flow 2 forces the particles 1 to flow in a single row under the action of sheath flow pressure through a coherentexcitation light beam 3 which can contain a plurality of laser beams, and a coherentscattering light beam 4 is generated and collected by amicroscope objective 5 within a solid angle with a firstcentral scattering angle 23 as a central scattering angle, and then is divided into two parts of transmitted light, namely first scatteredlight 10 and reflected light, namely second scatteredlight 18, by alight splitting plate 6, and then passes through respective filtering parts and an imaging device part to form two diffraction image data outputs.
The first imaging measurement and data output part C is used for performing imaging measurement and data output on the split and filtered scattered light so as to obtain diffraction patterns of the scattered light in different angle ranges such as the forward direction, the lateral direction or the back direction. The first imaging measurement and data output part C comprises a first focusinglens 9 and a second focusinglens 17 which respectively focus the scattered light in different directions output by the light splitting and filtering part B, afirst image sensor 11 positioned behind the first focusinglens 9, asecond image sensor 19 positioned behind the second focusinglens 17, a first dataoutput input port 12 connected behind thefirst image sensor 11 and providing a data output input channel, and a second dataoutput input port 20 connected behind thesecond image sensor 19; the first data output andinput port 12 and the second data output andinput port 20 have the same structure, and are used as signal channels to provide bias voltage required by the operation of the image sensor and power supply voltage of a cooling power supply of the image sensor, provide control signals of the image sensor, provide timing signals of analog pulse data signals output by the sensor, and provide data signals output by the sensor after the analog pulse data signals are digitized. The image sensor converts the scattered light into diffraction image data along with spatial angle distribution and outputs corresponding analog pulse data signals according to control signals. The pixel gray value of the digitalized diffraction image can be selected from 8 bits to 16 bits; generally, the higher the number of bits of the gray-scale value of the pixel, the larger the dynamic range of the image signal measurement, but the larger the amount of data to be stored.
The image processing circuit and the computer part D are used for receiving the output data of the data output part C, calculating and extracting different wavelength and polarization diffraction image mode characteristics and distinguishing particles. Includes a first imagesensor power supply 13 and a second imagesensor power supply 21 for supplying bias voltage and power supply voltage to thefirst image sensor 11 and thesecond image sensor 19, respectively, a first image processing circuit andcomputer 14 and a second image processing circuit andcomputer 22 for receiving image signals through a first dataoutput input port 12 and a second dataoutput input port 20, respectively,
the first image processing circuit and thecomputer 14, the second image processing circuit and thecomputer 22 have the same structure, and the computer part can be two different computers or the same computer; the image processing circuit and the computer can store, process and calculate the received image signals, extract characteristic parameters of different wavelengths and polarization diffraction image modes and output display parameter data.
The first image processing circuit andcomputer 14, the second image processing circuit andcomputer 22 are provided with an image processing circuit which can receive image signals through an image signal transmission line and store and process the received image signals, a circuit which generates image sensor control signals and time sequence signals, and a computer which is connected with the image processing circuit and calculates and extracts characteristic parameters of different wavelengths and polarization diffraction image modes of the received images and outputs display parameter data. The computer display part displays the statistical data of the analysis, calculation and discrimination results.
The image processing circuit may store the input image signal in a circuit internal memory or a memory of an electronic computer, and the image processing circuit may also include a circuit capable of performing a specific mathematical operation on the stored image signal.
Thefirst image sensor 11 and thesecond image sensor 19 of the present invention can both adopt CCD cameras, and the types of the CCD cameras used in this embodiment are: MegaPlus ES2093, manufacturer: princeton Instruments. The signals output by the two CCD cameras through the firstdata output port 12 and the seconddata output port 20, respectively, can be received and stored by a frame receiver card inserted in the computer and then transferred to the memory in the computer. The type of frame grabber is: PIXCI E4, manufacturer: EPIX, Inc.
As shown in fig. 2, the diffraction image measurement and analysis system for automatically distinguishing particles according to the present invention may further include a second scattered light receiving objective part a' for measuring a coherent scatteredlight beam 24 having a secondcentral scattering angle 43 emitted from the particles excited by the coherentexcitation light beam 3, on the basis of fig. 1; a second splitting and filtering section B' that splits and filters the received coherent scatteredbeam 24; a second imaging measurement and data output part C' for performing imaging measurement and data output on the dispersed light beam after the splitting and filtering; the received data output part C' is connected with the image processing circuit and the computer part D; the structures of the second scattered light receiving objective part A ', the second light splitting and filtering part B ' and the second imaging measurement and data output part C ' corresponding to the first scattered light receiving objective part A, the first light splitting and filtering part B and the received data output part C are completely the same. Fig. 2 shows an embodiment of the invention in which the coherent scattered beam generated by the particles excited by thecoherent excitation beam 3 is collected by two microscopic eyepieces, typically any two of the forward, side or back scattered beams, in different scattering angle ranges. The firstcentral scattering angle 19 of the coherent scatteredlight beam 4 shown in fig. 2 is between 5 and 90 degrees, and is a forward scattering light beam, the secondcentral scattering angle 43 of the coherent scatteredlight beam 24 is between 90 and 180 degrees, and is a backscattering light beam, each coherent scattered light beam is divided into two parts of a transmission light and a reflection light by a spectroscope, and four diffraction image data outputs are formed through a corresponding filtering part and an imaging device part.
Thespectroscope 26 shown in fig. 2 divides the scattered light received from themicroscope objective 25 into transmitted scattered light and reflected scattered light, and the transmittedscattered light 30 and the reflected scattered light 38 respectively pass through thenarrow band filter 27 and the narrow band filter 35, thepolarizing filter 28 and thepolarizing filter 36, and the focusinglens 29 and the focusinglens 37 to be converged, and form two pieces of diffraction image data through therespective image sensor 32 and the respective image sensor 40, and then are output to the image processing circuit and the computer through the respective dataoutput input port 32 and the respective data output input port 40.
The computer comprises a computer software part for calculating and extracting different wavelength and polarization diffraction image mode characteristics, and comprises the following steps: reforming transformation, coordinate transformation, mode characteristic analysis and calculation, and image mode characteristic parameter extraction. If the number of bits of the pixel gray value is more than 8 bits, the pixel reforming transformation can reduce the number of the bits of the pixel gray value to 8 bits according to the distribution of the pixel gray value of the diffraction image, thereby reducing the storage capacity required by an image signal file and accelerating the calculation and analysis speed of the subsequent image; the coordinate transformation is to do proper pixel space position transformation according to the light intensity distribution mode in the diffraction image, the mode characteristic analysis calculation can calculate and analyze the diffraction image after the coordinate transformation according to different image analysis algorithms such as pixel gray level correlation and the like, and the image mode characteristic parameter data is obtained by using statistics or other mathematical methods based on the mode characteristic analysis calculation result.
As shown in fig. 3, the analysis method of the diffraction image measurement and analysis system for automatically distinguishing particles of the present invention, namely, the computer software part, specifically comprises the following steps:
the first step is as follows: measuring the spatial distribution of particle scattered light under the irradiation of the multi-wavelength excitation light beam through an image sensor and a corresponding light path to obtain corresponding adjustable wavelength and polarization diffraction image data; the data of the adjustable wavelength and polarization diffraction image is as follows: the particle scattered light under irradiation of the multi-wavelength excitation beam is composed of corresponding multi-wavelength scattered light, the polarization state of the scattered light of each wavelength is generally different from that of incident light, the change of the polarization state is related to the three-dimensional structure form inside the particle, and the wavelength and the polarization of a measured diffraction image are selected through an optical path element in front of an image sensor when measuring diffraction pattern data.
Measuring the spatial distribution of particle scattered light under the irradiation of a multi-wavelength excitation light beam by using an image sensor and a corresponding light path to obtain adjustable wavelength and polarization diffraction image data; the multi-wavelength excitation light beam can be composed of two or more laser beams with different wavelengths, each laser beam is highly coherent in a certain polarization state at the respective wavelength and excites the particles to generate scattered light with the corresponding wavelength, the laser beams are highly coherent in the respective wavelength, the polarization state of the laser beams is generally different from that of the corresponding excitation light beam, the change of the polarization state of the laser beams is related to the three-dimensional structure inside the particles, and corresponding diffraction images can be obtained after the wavelength and the polarization state of the measured scattered light are selected through a narrow-band filter and a polarization filter. (ii) a
The second step is as follows: transmitting all the measured data of the diffraction images of the particles in the measured particle community at different wavelengths and polarization and a population resolution criterion to an image processing circuit memory in a computer for image analysis in the next step;
and the population distinguishing criterion in the second step is obtained by particle community training data. One implementation method is as follows: firstly, obtaining a characteristic parameter vector of each particle diffraction image and the distribution in a characteristic parameter vector sample space by measuring the adjustable wavelength and polarization diffraction image data of a particle community sample with known internal three-dimensional structural morphological characteristics, and finally determining a population resolution criterion related to the internal three-dimensional structural morphological characteristics of the particles. The other realization method comprises the following steps: firstly, obtaining a characteristic parameter vector of each particle diffraction image and the distribution of the characteristic parameter vector in a sample space by measuring adjustable wavelength and polarization diffraction image data of different particle population samples with known characteristics (such as at different stages of cell cycle), and finally determining a population resolution criterion related to the morphological characteristics of the three-dimensional structure in the particles.
The diffraction image data may be alternately transferred from thefirst image sensor 11 and thesecond image sensor 19 directly to the image processing circuit memory in the computer, or may be temporarily stored in a memory provided in the image sensor and then alternately transferred to the image processing circuit memory in the computer in batches.
The third step: reducing the number of bits of the pixel gray value from more than 8 bits to 8 bits according to the distribution of the pixel gray value of the diffraction image, and carrying out image space coordinate transformation according to an image distribution mode of the bits; the number of bits of the pixel gray scale value of the diffraction image can be different according to image sensors produced by different manufacturers, such as 12 bits or 16 bits, the larger the number of the pixel gray scale value is, the higher the calculation accuracy is generally, but the larger the memory and the calculation amount required by the image operation are, when the number of the pixel gray scale value is changed from more than 8 bits through data normalization to 8 bits, the minimum value is set to be 20-1 (equal to 0) and the maximum value is set to be 2 according to the pixel gray scale value distribution8-1(═ 255), other pixel grayscalesThe value is changed into an integer value between 0 and 255 according to a corresponding proportion, and the bit number transformation completed by the normalization of the data can reduce the memory and the calculation amount required by the image operation under the condition of basically not influencing the image calculation precision; the image space coordinate transformation means that rectangular coordinate and polar coordinate transformation is carried out according to a measured diffraction image distribution mode, such as radial divergent light spot distribution, so that the light spot distribution is mainly distributed along the transverse direction or the longitudinal direction.
The fourth step: and (3) performing characteristic discrimination on the transformed different wavelength and polarization diffraction images by using a characteristic discrimination method (such as wavelet transformation, curvelet transformation and Fourier transformation) and selecting a characteristic region. And simultaneously performing noise removal and image enhancement on the image. The processing of this step enhances the feature contrast and reduces the computational complexity of the next step.
The fifth step: and selecting diffraction image mode characteristics from the different wavelength and polarization diffraction images after the characteristics are screened in the characteristic region. The diffraction image mode feature is calculated based on pixel gray scale association. The pixel-based gray scale correlation calculation method is based on a parallel algorithm which divides input image data into a plurality of data streams and then calculates the data streams respectively.
The statistical characteristic selection method can be one or different combinations of methods such as an LAWS texture energy filter, a random domain model, a gray level co-occurrence matrix and the like. This section details how to select statistical characteristics by calculating and obtaining image mode characteristic parameters based on pixel gray level correlation of a gray level co-occurrence matrix, and the calculation of the gray level co-occurrence matrix can be illustrated by the following example.
Let I (m, n) be a diffraction image, m and n be positive integers representing the lateral and vertical coordinates of the pixel position, respectively: 1 is more than or equal to m and less than or equal to Ly, 1 is more than or equal to N and less than or equal to Lx, the total number of pixels is Ly multiplied by Lx, and the level of the gray value I of the pixels is NgI.e. 1. ltoreq. I. ltoreq.NgThen the elements of the gray level co-occurrence matrix P are defined as
Where # denotes the number of pixel pairs (whose positions are represented by the pixel position coordinates before | which satisfy the condition (represented by the system of equations after the symbol | in the set { … | … }), d is a positive integer representing the pixel position difference, the subsequent angle values δ (═ 0 °, 45 °, 90 °, 135 °) are the directional relationships of the pixel pairs, i, j denote the gray values of the pixels within the pixel pairs and represent the lateral and longitudinal coordinates of the gray co-occurrence matrix element positions, respectively, between 1 and NgAn integer in between.
The gray level co-occurrence matrix calculated according to the definition reflects a pixel gray level distribution mode in diffraction image data, and image characteristics highly related to the particle three-dimensional structure form can be extracted through quantitative analysis of factors such as the direction, adjacent interval, change amplitude and the like of each pair of pixel gray level value distribution. It provides a way for analyzing the basic mode information such as texture mode and arrangement rule of diffraction image, and can be represented by multiple two-dimensional gray level co-occurrence images: each gray level co-occurrence matrix element is represented by a gray level co-occurrence image pixel, the horizontal and vertical coordinates of the pixel position are i and j respectively, and the gray value of the pixel is P; each gray level co-occurrence image represents a certain combination of d and delta; if the number of bits of the pixel gray scale value of the diffraction image is 8, the total number of pixels of each corresponding gray scale co-occurrence image is 256 × 256. Therefore, the number of matrix elements of the gray level co-occurrence matrix or the number of pixels of the gray level co-occurrence image and the number of bits of the gray level values of the pixels of the diffraction image form an exponential relationship, and increasing the number of the gray level values of the pixels of the diffraction image causes the storage amount and the calculation amount of the gray level co-occurrence image to increase very quickly. Through the statistical analysis of the gray level co-occurrence matrix or the gray level co-occurrence image, a plurality of diffraction image mode characteristic parameters (including parameters such as a distance/energy, a contrast ratio, a correlation, an autocorrelation, inertia, a sum of squares, a total variance, a total average, a total entropy, a difference, an entropy, homogeneity, a median, a covariance, a variance difference, an entropy difference, a maximum value and a maximum correlation coefficient) can be extracted for the analysis and the identification of the particles.
A sixth step: forming a diffraction pattern characteristic parameter vector according to different wavelengths and polarization diffraction image mode characteristic parameters and determining the position of the measured particles in a characteristic parameter vector sample space according to the vector;
the diffraction image characteristic parameter vector may include a plurality of components, for example, 3 to 20 components, each component value is a certain image characteristic parameter value obtained by calculation from the diffraction image, and a vector obtained by combining all characteristic parameter values of different wavelengths and polarization diffraction images measured from the same particle as component values is a diffraction pattern characteristic parameter vector representing the particle; the characteristic parameter vector sample space is a vector space, the dimension of the vector space is equal to the number of components of the diffraction image characteristic parameter vector, each measured particle is represented by the position of the diffraction image characteristic parameter vector sample space, the position is determined by all the components of the diffraction image characteristic parameter vector of the particle, and the measured particle group is represented by the position point set of all the particles in the characteristic parameter vector sample space.
A seventh step of: after the positions of all the particles in the detected particle community in the diffraction image characteristic parameter vector sample space are determined, automatically classifying the detected particle community into different populations according to the position distribution of the particles in the diffraction image characteristic parameter vector sample space and the input population resolution criterion, and outputting corresponding data.
The population discrimination criterion is obtained by the particle colony training data obtained by the method described above. Data analysis and particle classification in the diffraction image feature parameter vector sample space are generally based on a method combining linear and nonlinear analysis, and can be realized by a hybrid scheme combining a neural network and a support vector machine, for example. The support vector machine belongs to a universal linear classifier, and has the advantages of simultaneously minimizing the empirical error and maximizing the geometric marginal area, so the support vector machine is also commonly called as a maximum marginal classifier. In the method, an extension method of a support vector machine is adopted. In the kernel of the support vector machine, a nonlinear algorithm (comprising a polynomial, hyperbolic tangent, radial basis function based, Gaussian radial basis function based and the like) is introduced, and the method not only keeps the accuracy of the nonlinear algorithm, but also improves the classification efficiency.
Fig. 4 is a schematic flow chart of the calculation of pixel grayscale correlation of diffraction image for automatically identifying particles and the classification of particles based on diffraction image feature parameter vector according to the present invention. Firstly, reducing the number of bits of a pixel gray value from more than 8 bits to 8 bits according to the distribution of the pixel gray value of the diffraction image, then carrying out coordinate transformation from a rectangular coordinate to a polar coordinate, carrying out feature region discrimination and pixel value normalization on the transformed diffraction image, finally calculating a pixel gray co-occurrence matrix and extracting a diffraction image mode feature parameter according to a corresponding gray co-occurrence image. After the diffraction image pixel gray level correlation calculation of all the detected microparticles is completed, the image mode characteristic parameters of the detected microparticles are input into a support vector machine as diffraction image characteristic parameter vectors to automatically classify all the microparticles in the detected microparticle colony in a diffraction image characteristic parameter vector sample space. FIG. 4 also shows that the input diffraction image data can be split into N data streams, using a parallel algorithm to increase the speed of image analysis calculations.
FIG. 5 is a diagram illustrating the effect of image processing during the calculation of pixel intensity correlation of a diffraction image according to the present invention. In the figure: a diffraction image; b, transforming the diffraction image after the coordinate transformation; c, diffraction image characteristic areas after characteristic discrimination; d gray level co-occurrence images.