Disclosure of Invention
In order to solve the problems in the prior art, the application provides a simulation algorithm of a synthetic aperture imaging process, and pupil shake and image shake are added into the algorithm under the condition that the simulation efficiency is not obviously reduced, so that the fitting degree of a simulation result and an experiment result is improved, and the guidance of simulation on the experiment is better realized.
In order to achieve the technical purpose, the invention adopts the following technical scheme:
a simulation algorithm for synthetic aperture imaging, comprising the steps of:
step S1: pretreatment: obtaining complex amplitude distribution of the target image;
step S2: supersampling: the complex amplitude distribution obtained in the step S1 is subjected to oversampling, wherein the oversampling is to increase the density of sampling points on the basis of the complex amplitude distribution, so as to obtain the complex amplitude distribution after the oversampling;
step S3: pupil dithering step: calculating a pupil sequence of the synthetic aperture, and generating a pupil sequence containing jitter by adding random translation to the pupil sequence;
step S4: low resolution image generation: calculating to obtain a low-resolution image sequence corresponding to the target image according to the supersampled complex amplitude distribution and the pupil sequence containing jitter;
step S5: image dithering step: adding random dithering to the low-resolution image sequence to generate a low-resolution image sequence containing dithering;
step S6: and (3) downsampling: and (3) downsampling the low-resolution image sequence containing the jitter, which is obtained in the step (S5), wherein the downsampling is to reduce the density of sampling points, and the downsampled low-resolution image sequence for Fourier stack reconstruction can be obtained.
As a preferred solution, the random translation in step S3 is a uniform random translation, and/or a random translation conforming to a two-dimensional gaussian distribution.
As a preferred solution, the random dithering in step S5 is a uniform random translation, and/or a random translation conforming to a two-dimensional gaussian distribution.
As a preferred solution, the step S5 further comprises adding noise to the low resolution image sequence.
As a preferred solution, the noise comprises multiplicative noise, and/or additive noise.
As a preferred embodiment, step S5 further comprises a step of scaling the low resolution image sequence, or the low resolution image sequence with jitter, said scaling being an equal-scale transformation of the size of the image sequence.
As a preferable scheme, the scaling factor is 0.9-1.1.
As a preferred solution, the scaling is a uniformly distributed random scaling or a random scaling conforming to a gaussian distribution.
As a preferred embodiment, the step S1 includes the steps of:
step S11: inputting an imaging target
;
Step S12: calculating imaging target amplitude
Said->
;
Step S13: adding phase distribution to imaging targets
Obtaining complex amplitude of imaging target
。
As a preferred embodiment, the step S2 includes the steps of:
step S21: the spectrum of the imaged object is calculated,
;
step S22: calculating the size of the imaging target spectrum
Said->
Matrix order for imaging target spectrum;
step S23: reconstructing and denoising an imaging target to generate a high-resolution image, obtaining a reconstructed image, and calculating the size of the reconstructed image
;
Step S24: comparison of
And->
The size of (1)>
≤
For->
Supersampling to obtain the complex amplitude distribution +.>
Returning to step S21 again, according to +.>
Recalculating the oversampled imaging target spectrum +.>
And do->
Performing spectrum post-processing; if->
>
Then the imaging target spectrum obtained in step S21 is +.>
Directly performing spectrum post-processing, and marking the spectrum after spectrum post-processing as +.>
。
In a preferred embodiment, in step S22, the dimensions are as follows
Equal to the resolution of the imaged object.
In a preferred embodiment, in step S23, the dimensions are as follows
=0.9 to 1.1x, where x is the resolution of the reconstructed image.
As a preferred embodiment of the present invention,
=x。
in a preferred embodiment, in step S24, the spectrum post-processing is performed by performing peripheral zero padding on the high-frequency portion of the spectrum.
As a preferable mode, in the step S24, when
When the aperture imaging simulation algorithm is stopped, a new imaging target with higher resolution than the original imaging target is selected, and simulation calculation is performed again from the step S1 until +.>
。
As a preferred embodiment, the step S3 includes the steps of:
step S31: pupil sequence for calculating synthetic aperture
;
Step S32: pupil alignment
Adding random translation to generate pupil sequence +.>
。
As a preferred embodiment, the step S4 includes the steps of:
step S41: calculating a low resolution image sequence spectrum
=
The method comprises the steps of carrying out a first treatment on the surface of the Wherein the method comprises the steps of
For pupil sequences with jitter +.>
Spectrum of the imaging target calculated from the oversampled complex amplitude;
step S42: according to
Calculating to obtain a low resolution image sequence +.>
=
Wherein->
Inverse fourier transform symbols;
as a preferred solution, the simulation algorithm further comprises a reconstruction step S7,
step S7: and (3) carrying out image registration, image denoising and stack restoration on the low-resolution image sequence which can be used for Fourier stack reconstruction and is obtained in the step (S6), and finally outputting a reconstruction target image.
The technical scheme has the advantages that: in the technical scheme, the simulation aiming at pupil shake and image shake is added to simulate the relative shake of the lens and the imaging target in the real environment, so that the simulation can be more complete to investigate the influence of specific shake on the imaging quality of the Fourier lamination.
Detailed Description
Features and exemplary embodiments of various aspects of the present application are described in detail below to make the objects, technical solutions and advantages of the present application more apparent, and to further describe the present application in conjunction with the accompanying drawings and the detailed embodiments. It should be understood that the specific embodiments described herein are intended to be illustrative of the application and are not intended to be limiting. It will be apparent to one skilled in the art that the present application may be practiced without some of these specific details. The following description of the embodiments is merely intended to provide a better understanding of the present application by showing examples of the present application.
Fourier stack imaging, fourier Ptychography Imaging, FP for short, is a new image reconstruction technique, which uses fourier transforms to recover images and has better optical performance, and in general terms, FP is a technique that breaks up a real image into a series of fourier transformed samples, which are then used to reconstruct the original image.
The FP has the core principle that the image is subjected to multiple fourier transform, i.e., the image is first decomposed into different optical apertures, then the different optical apertures are used to detect spectral signals, the original image is combined by an analog simulation algorithm, the FP can realize high-resolution image reconstruction, super-diffraction limit imaging is realized, and the FP has a better dynamic range and signal-to-noise ratio than the traditional imaging technology.
The key technology of FP is the construction of synthetic aperture imaging algorithms. In a real environment, relative jitter between a target surface and a lens is various, and there are up to 9 degrees of freedom of jitter, wherein the number of relative displacement degrees of freedom is 3, and the number of rotational degrees of freedom lenses, namely an imaging system and the target surface, is 6 respectively, but no simulation of the relative jitter between the lens and the target surface exists in the existing algorithm scheme at present, so that the simulated image and the actual image come in and go out. Therefore, the influence of jitter needs to be considered in the actual simulation, thereby ensuring the authenticity of the simulated image.
Based on the above, the application provides a simulation algorithm of synthetic aperture imaging, which comprises the following steps:
step S1: pretreatment: obtaining complex amplitude distribution of the target image;
step S2: supersampling: the complex amplitude distribution obtained in the step S1 is subjected to oversampling, wherein the oversampling is to increase the density of sampling points on the basis of the complex amplitude distribution, so as to obtain the complex amplitude distribution after the oversampling;
step S3: pupil dithering step: calculating a pupil sequence of the synthetic aperture, and generating a pupil sequence containing jitter by adding random translation to the pupil sequence;
step S4: low resolution image generation: calculating to obtain a low-resolution image sequence corresponding to the target image according to the supersampled complex amplitude distribution and the pupil sequence containing jitter;
step S5: image dithering step: adding random dithering to the low-resolution image sequence to generate a low-resolution image sequence containing dithering;
step S6: and (3) downsampling: and (3) downsampling the low-resolution image sequence containing the jitter, which is obtained in the step (S5), wherein the downsampling is to reduce the density of sampling points, and the downsampled low-resolution image sequence for Fourier stack reconstruction can be obtained.
In the above technical solution, the target image is a concept of a spatial domain, and the synthetic aperture is a concept of a spectral domain.
According to the method, the dithering steps for the pupil and for the low-resolution image sequence are introduced in the algorithm process of the synthetic aperture imaging, so that the simulation accuracy of the optical transmission process is effectively improved under the condition that the simulation efficiency is not obviously reduced, and the influence of specific dithering on the imaging quality of the Fourier lamination can be more completely simulated.
In a real environment, the relative shake between the target surface and the lens is various, and there are shake of up to 9 degrees of freedom, wherein the relative displacement freedom is 3, and the rotational freedom lens comprises an imaging system and 6 target surfaces. But in far field imaging, rotational jitter under natural conditions has negligible effect on the imaging. Therefore, only the displacement shake between the target surface and the lens is needed to be simulated.
Under the above conditions, the influence of the shake between the target surface and the lens can be simplified to spectrum shake and image shake. Spectral dithering may be achieved in step S3 by translating each single aperture pupil in the synthetic aperture; image dithering can be realized by translating each low-resolution image sequence in the step S5, and the overall simulation scheme is closer to the actual working condition by adding dithering, so that the simulation accuracy is improved.
The preprocessing step of step S1 is to obtain the complex amplitude of the required target image by calculating simulation according to the target image, or obtain the intensity of the target image when the phase of the target image is 0, and provide a data base for the next supersampling step.
The step of supersampling in the step S2 is performed on the basis of the parameter data obtained in the step S1, and the density of sampling points is further improved for the data obtained in the step S1, so that a data basis is provided for constructing a low-resolution image sequence.
Step S3 is to simulate the actual working condition, and to increase one shake for the pupil to simulate the frequency spectrum shake.
Step S4 is performed by an analog simulation algorithm, and the corresponding low-resolution image sequence can be obtained by calculation based on the supersampled data in step S2 and the pupil subjected to dithering in step S3, so that a low-resolution image can be further obtained.
Step S5 is to add jitter to the low resolution sequence to simulate the image jitter generated in the actual working condition.
Step S6 performs downsampling processing on the obtained low-resolution image sequence. Step S6 corresponds to the simulation of the receiving process of the detector, such as the CCD and the CMOS, and reduces the high sampling rate of the transmission process to the sampling rate or resolution of the image plane of the detector, so as to avoid the excessive data volume while attaching to the actual scene. The resampling step provided by the method effectively improves the simulation sampling rate in the optical transmission process, so that the constructed low-resolution image distortion rate is lower.
In the actual working condition, a large amount of jitter can be added, a small amount of jitter can also be added, and the specific added jitter amount is determined by the actual simulation requirement. Example 1 provides a simulation algorithm result with a small amount of added jitter, and example 2 provides a simulation algorithm result with a large amount of added jitter. The related experimental results are shown in fig. 1, 2, 3 and 4, and the technical scheme of the invention aims to make the simulation reconstruction result and the experimental reconstruction result fit as much as possible. As can be seen from the comparison results of fig. 1 and fig. 2 and the comparison results of fig. 3 and fig. 4, the reconstructed image obtained by the simulation after adding the jitter has higher fitting degree with the target surface image to be simulated due to certain distortion.
As a comparison example, the application also provides a simulation reconstruction result obtained by a simulation algorithm without adding jitter, the simulation reconstruction result is shown in fig. 5, and the experimental reconstruction result corresponding to the simulation reconstruction result is shown in fig. 2. It can be seen that, compared with fig. 2, the simulation image of fig. 5 is too flat compared with the real target surface image, and there is an overfitting, and the difference from the actual experiment is too large, and the simulation result is inaccurate.
In one embodiment of the present application, the random translation in step S3 is a uniform random translation, and/or a random translation conforming to a two-dimensional gaussian distribution.
In one embodiment of the present application, the random dithering in step S5 is a uniform random translation, and/or a random translation conforming to a two-dimensional gaussian distribution.
In one embodiment of the present application, the step S5 further includes adding noise to the low-resolution image sequence, so as to better simulate the actual working condition.
Further, the noise includes multiplicative noise, and/or additive noise. The coherent noise is typically multiplicative noise and the incoherent noise is typically additive noise. Meanwhile, multiplicative noise and additive noise are added, so that the simulation is more close to an experimental result.
In one embodiment of the present application, step S5 further comprises a step of scaling the low resolution image sequence, or the low resolution image sequence with jitter, said scaling being an equal-scale transformation of the size of the image sequence. In this technical solution, when the simulated imaging process is near-field imaging, but not far-field imaging, that is, when the distance change caused by the relative shake between the target surface and the lens causes a non-negligible imaging change, the scaling of the low-resolution image may also be added in step S5, so as to increase the accuracy of the calculation.
Further, the scaling factor is 0.9-1.1.
Further, the scaling is uniformly distributed random scaling or random scaling conforming to a gaussian distribution.
In one embodiment of the present application, the step S1 includes the steps of:
step S11: inputting an imaging target
;
Step S12: calculating imaging target amplitude
Said->
;
Step S13: adding phase distribution to imaging targets
Obtaining complex amplitude of imaging target
。
The imaging target in step S11 corresponds to a target surface in an experiment, and is a picture with higher resolution in the simulation process, and the imaging target is the introduction of the picture.
Step S12 is a step of calculating the imaging target amplitude, because the phase of the coherent light can be superimposed on the amplitude only. Amplitude is the pair
The intensity parameters of (2) are obtained by root marking;
the phase distribution added in step S13 may be controlled to simulate transmission simulation or reflection simulation. Different simulations may be applied for different requirements. If reflection imaging is simulated, a large number of random phases are overlapped to simulate the modulation of the optical phase by the rough surface of the target; if transmission imaging is simulated, no phase distribution is added, or the transmission phase distribution of the required simulation target is added.
In one embodiment of the present application, the step S2 includes the steps of:
step S21: the spectrum of the imaged object is calculated,
;
step S22: calculating the size of the imaging target spectrum
Said->
Matrix order for imaging target spectrum;
step S23: reconstructing and denoising an imaging target to generate a high-resolution image, obtaining a reconstructed image, and calculating the size of the reconstructed image
;
Step S24: comparison of
And->
The size of (1)>
≤
For->
Supersampling to obtain the complex amplitude distribution +.>
Returning to step S21 again, according to +.>
Recalculating the oversampled imaging target spectrum +.>
And do->
Performing spectrum post-processing; if->
>
Then the imaging target spectrum obtained in step S21 is +.>
Directly performing spectrum post-processing, and marking the spectrum after spectrum post-processing as +.>
。
When step S24 is performed, it is explained that the theoretical highest frequency of the image obtained by aperture imaging is already higher than the highest frequency of the imaging target. At this time, the complex amplitude of the imaging target can be calculated by a certain algorithm
The supersampling (upsampling) is equivalent to the supersampling of the target image, so that the upper limit of the effective frequency of the image can be effectively improved, and the definition of the image can be further improved. The supersampling algorithm can be realized by using a nearest and the like, the specific supersampling scheme is determined by the specific requirements of experiments, and the supersampling scheme is determined by the sampling rate or resolution of an input image, the synthetic aperture multiple, the lens focal length, the lens aperture and the like.
In this embodiment, on the basis of the original step S2, steps of image reconstruction and size judgment on the original image and the reconstructed image are added, the setting of the step can judge whether the input image is necessary to be subjected to oversampling, and when the sampling rate or resolution of the reconstructed image is lower than that of the input image, the step of oversampling can be skipped, and related data can be directly applied to the step S4, thereby effectively improving the calculation efficiency.
In one embodiment of the present application, in step S22, the dimensions
Equal to the resolution of the imaged object.
In one embodiment of the present application, in step S23, the dimensions
=0.9 to 1.1x, where x is the resolution of the reconstructed image. Size->
Can be imaged by apertureThe scheme calculation results in that the detailed calculation scheme is related to the specific algorithm selection.
Preferably, the method comprises the steps of,
=x。
the step sequence of step S22 and step S23 may be exchanged,
and->
The calculation sequence of (3) does not affect the simulation result.
In one embodiment of the present application, in step S24, the spectral post-processing is peripheral zero padding of the high frequency portion of the spectrum. The purpose of performing the peripheral zero padding is based on the algorithm requirement, and in order to avoid the index exceeding the boundary, 0 needs to be padded on the periphery of the spectrum, specifically, the rows with 0 and the columns with 0 are padded on the edge of the spectrum matrix, and the number of specific rows and columns is calculated.
In one embodiment of the present application, in step S24, when
When the aperture imaging simulation algorithm is stopped, a new imaging target with higher resolution than the original imaging target is selected, and simulation calculation is performed again from the step S1 until +.>
。
In one embodiment of the present application, the step S3 includes the steps of:
step S31: pupil sequence for calculating synthetic aperture
;
Step S32: pupil alignment
Adding random translation to generate pupil sequence +.>
。
In one embodiment of the present application, the step S4 includes the steps of:
step S41: calculating a low resolution image sequence spectrum
=
The method comprises the steps of carrying out a first treatment on the surface of the Wherein the method comprises the steps of
For pupil sequences with jitter +.>
Spectrum of the imaging target calculated from the oversampled complex amplitude;
step S42: according to
Calculating to obtain a low resolution image sequence +.>
=
Wherein->
Inverse fourier transform symbols;
as a preferred solution, the simulation algorithm further comprises a reconstruction step S7,
step S7: and (3) carrying out image registration, image denoising and stack restoration on the low-resolution image sequence which can be used for Fourier stack reconstruction and is obtained in the step (S6), and finally outputting a reconstruction target image.
Typically, the steps of the simulation of the imaging process and the reconstruction process may be performed separately, step S7 again provides a reconstruction step, and the downsampled low-resolution image is reconstructed again to obtain a resampled reconstruction target image. The physical process corresponding to step S6 is a sampling process of the CCD or CMOS, and the specific downsampling scheme is determined by the requirement, typically the sampling rate or resolution of the imaging CCD or CMOS that needs to be emulated. Through step S4, a low resolution image sequence can be obtained
After adding noise and jitter again, +.>
After downsampling, a downsampled low-resolution image sequence +.>
。
In certain embodiments, the image reconstruction process as described in step S23 is the same as the reconstruction process in step S7 described above.
An exemplary flow provided by this embodiment is shown in fig. 6. In the actual working condition, the imaging target is input into a computer, after the target amplitude of the imaging target is read, phase distribution is added to the imaging target, the frequency spectrum of the imaging target is obtained through calculation, jitter is added to the pupil and the low-resolution image sequence, the actual working condition is simulated, and finally the imaging target is reconstructed to obtain a reconstructed image.
In all the technical schemes provided by the application, the formula only represents the calculation process, but not the scope limited by the formula, and all the simulation schemes or formula calculation schemes which can achieve the technical purpose of the application are considered as the technical scheme protection scope of the application.