技术领域technical field
本发明属于光学领域，涉及一种波前编码成像系统及基于单幅图像放大的超分辨处理方法，尤其涉及一种应用波前编码技术的大焦深成像系统以及针对波前编码成像系统的超分辨率图像处理方法。The invention belongs to the field of optics, and relates to a wavefront coding imaging system and a super-resolution processing method based on single image enlargement, in particular to a large focal depth imaging system applying wavefront coding technology and a super-resolution processing method for the wavefront coding imaging system. resolution image processing methods.
背景技术Background technique
扩展光学系统的焦深一直都是学术界研究的热点，从20世纪80年代中期开始，虽然形形色色的方法被提出用于景深扩展，但是直到美国科罗拉多大学的Dowski博士和Cathey教授于1995年提出波前编码的概念之后，景深延拓才有了真正意义上的突破。The depth of focus of extended optical systems has always been a hotspot in academic research. Since the mid-1980s, although various methods have been proposed for depth of field extension, it was not until Dr. Dowski and Professor Cathey of the University of Colorado in the United States proposed the wave After the concept of pre-coding, the depth of field extension has a real breakthrough.
以一维光学系统为例，其离焦光学传递函数OTF可以通过广义光瞳函数的自相关运算来获得，如下所示：Taking the one-dimensional optical system as an example, its defocus optical transfer function OTF can be obtained through the autocorrelation operation of the generalized pupil function, as follows:
其中，u和x分别是归一化的空间频率与孔径平面横向坐标；W20是最大离焦波像差系数；k是波数；而f则代表相位板通用表达式。Among them, u and x are the normalized spatial frequency and the transverse coordinate of the aperture plane, respectively; W20 is the maximum defocus wave aberration coefficient; k is the wave number; and f represents the general expression of the phase plate.
对于传统成像系统来说，上式中的f项不存在，因此可以轻松得到离焦OTF的具体表达式为：For traditional imaging systems, the f term in the above formula does not exist, so the specific expression of the defocus OTF can be easily obtained as:
可以看到，当系统未引入相位板时，其OTF对离焦是非常敏感的，而且会在频率空间周期性地出现零点，从而造成不可逆的信息损失。但是一旦将E.R.Dowski博士所发明的三次方相位板(f(x)＝αx3)引入到光学系统的入瞳面上之后，通过静态相位近似法就可以得到一个完全不同的离焦OTF，如下：It can be seen that when the system does not introduce a phase plate, its OTF is very sensitive to defocus, and there will be zero points periodically in the frequency space, resulting in irreversible information loss. But once the cubic phase plate (f(x)=αx3 ) invented by Dr. ERDowski is introduced into the entrance pupil plane of the optical system, a completely different defocus OTF can be obtained by static phase approximation, as follows:
显而易见，此时离焦OTF的模，即MTF与离焦波像差系数是无关的，也就是说三次方相位板可以使系统MTF对离焦不敏感；虽然OTF的相位部分与离焦参量W20有关，但是只要调制因子α增大，其对W20的依赖度就会显著降低。同时最为重要的是，在添加了相位掩膜板之后，MTF在有效频率范围之内只是在幅度上有一定程度的下降，而不存在零点或近零点，即系统出现离焦时，超出原始系统景深范围的信息并没有丢失，之后通过数字图像复原算法就可以被有效地恢复。同时，由于相位板对系统的通光量和分辨率都不会造成影响，所以波前编码是一种非常不同于缩小孔径法、中心遮拦法或切趾法的新型大景深成像方法。Obviously, the modulus of the defocused OTF at this time, that is, the MTF has nothing to do with the defocused wave aberration coefficient, that is to say, the cubic phase plate can make the system MTF insensitive to defocus; although the phase part of the OTF is related to the defocus parameter W20 , but as long as the modulation factor α increases, its dependence on W20 will decrease significantly. At the same time, the most important thing is that after adding the phase mask, the MTF only has a certain degree of decline in the amplitude within the effective frequency range, and there is no zero point or near zero point, that is, when the system is out of focus, it exceeds the original system The information of the depth of field is not lost, and can be effectively restored by digital image restoration algorithms afterwards. At the same time, since the phase plate will not affect the light throughput and resolution of the system, wavefront encoding is a new imaging method with large depth of field that is very different from the aperture reduction method, central obscuration method or apodization method.
毫无疑问，相位掩膜板是实现系统光学传递函数OTF离焦不敏感的关键。然而，如果没有复原滤波，就无法获得清晰的大焦深图像。传统的复原流程仅能获得与图像传感器有效像素数目相对应的解码图像，此时像元的实际物理大小决定了以每毫米线对数表征的系统空间分辨能力。如果要进一步提升系统的空间分辨力，选用像元更小的探测器是最直接的方法。但是，图像传感器像元的物理大小受制造工艺以及应用条件等诸多因素的限制，不可能无限地小下去。因此，在不改变传感器物理像元尺寸的前提下实现空间分辨率能力的提升是超分辨率研究领域的热点之一。Undoubtedly, the phase mask is the key to realize the defocus insensitivity of the optical transfer function OTF of the system. However, without restoration filtering, it is impossible to obtain sharp images with large depth of focus. The traditional restoration process can only obtain a decoded image corresponding to the effective number of pixels of the image sensor. At this time, the actual physical size of the pixel determines the spatial resolution of the system represented by the logarithm of lines per millimeter. If the spatial resolution of the system is to be further improved, the most direct method is to select a detector with a smaller pixel. However, the physical size of an image sensor pixel is limited by many factors such as manufacturing process and application conditions, and cannot be infinitely small. Therefore, improving the spatial resolution capability without changing the physical pixel size of the sensor is one of the hotspots in the field of super-resolution research.
发明内容Contents of the invention
为了解决背景技术中存在的上述技术问题，本发明提供了一种在不改变图像传感器硬件条件的前提下，不但能够实现超大焦深的清晰成像，而且可以获得对应于更小物理像元尺寸探测器的超分辨率重构图像的波前编码成像系统及基于单幅图像放大的超分辨处理方法。In order to solve the above-mentioned technical problems in the background technology, the present invention provides a method that can not only realize clear imaging with a super-large depth of focus, but also obtain detection corresponding to a smaller physical pixel size without changing the hardware conditions of the image sensor. A wavefront coding imaging system for super-resolution reconstructed images of a device and a super-resolution processing method based on single image enlargement.
本发明的技术解决方案是：本发明提供了一种波前编码成像系统，包括波前编码成像镜头、1/3英寸图像探测器以及解码处理单元，所述波前编码成像镜头包括第一镜片、相位掩膜板、第二镜片以及第三镜片；所述第一镜片、相位掩膜板、第二镜片、第三镜片，1/3英寸图像探测器以及解码处理单元依次设置在同一光路上；其特殊之处在于：The technical solution of the present invention is: the present invention provides a wavefront coding imaging system, including a wavefront coding imaging lens, a 1/3 inch image detector and a decoding processing unit, and the wavefront coding imaging lens includes a first lens , a phase mask, a second lens and a third lens; the first lens, the phase mask, the second lens, the third lens, the 1/3-inch image detector and the decoding processing unit are sequentially arranged on the same optical path ; is special in that:
所述第一镜片的前表面的曲率半径是18.91mm，第一镜片的前表面的X方向通光半孔径以及Y方向通光半孔径均是6.22mm；所述第一镜片的后表面的曲率半径是234mm，第一镜片的后表面的X方向通光半孔径以及Y方向通光半孔径均是5.85mm；所述第一镜片的前表面与第一镜片的后表面之间的距离，即第一镜片的中心厚度是3.0mm；The radius of curvature of the front surface of the first eyeglass is 18.91mm, and the X-direction light semi-aperture and the Y-direction light half-aperture of the front surface of the first eyeglass are both 6.22mm; the curvature of the back surface of the first eyeglass The radius is 234mm, and the X-direction light semi-aperture and the Y-direction light semi-aperture of the rear surface of the first lens are both 5.85mm; the distance between the front surface of the first lens and the rear surface of the first lens is, The central thickness of the first lens is 3.0mm;
所述相位掩膜板的前表面的X方向通光半孔径以及Y方向通光半孔径均是4.95mm；所述相位掩膜板的后表面的X方向通光半孔径以及Y方向通光半孔径均是4.42mm；所述第一镜片的后表面与相位掩膜板的前表面之间的距离，即第一镜片与相位掩膜板之间的距离是3.22mm；所述相位掩膜板的前表面与相位掩膜板的后表面之间的距离，即相位掩膜板的厚度是2.0mm；The X direction light semi-aperture and the Y direction light semi-aperture of the front surface of the phase mask are 4.95mm; the X direction light semi-aperture and the Y direction light semi-aperture of the back surface of the phase mask are The apertures are all 4.42mm; the distance between the rear surface of the first eyeglass and the front surface of the phase mask, that is, the distance between the first eyeglass and the phase mask is 3.22mm; the phase mask The distance between the front surface of the phase mask and the back surface of the phase mask, that is, the thickness of the phase mask is 2.0mm;
所述第二镜片的前表面的曲率半径是-34.973mm，第二镜片的前表面的X方向通光半孔径以及Y方向通光半孔径均是4.43mm；所述第二镜片的后表面的曲率半径是15.776mm，第二镜片的后表面的X方向通光半孔径以及Y方向通光半孔径均是5.46mm；所述相位掩膜板的后表面与第二镜片的前表面之间的距离，即相位掩膜板与第二镜片之间的距离是2.0mm；所述第二镜片的前表面与第二镜片的后表面之间的距离，即第二镜片的中心厚度是3.0mm；The radius of curvature of the front surface of the second lens is -34.973mm, the X direction light semi-aperture and the Y direction light semi-aperture of the front surface of the second lens are both 4.43mm; the rear surface of the second lens is The radius of curvature is 15.776mm, and the X-direction light semi-aperture and Y-direction light semi-aperture of the rear surface of the second lens are both 5.46mm; the distance between the rear surface of the phase mask and the front surface of the second lens Distance, that is, the distance between the phase mask plate and the second lens is 2.0mm; the distance between the front surface of the second lens and the rear surface of the second lens, that is, the central thickness of the second lens is 3.0mm;
所述第三镜片的前表面的曲率半径是33.6240mm，第三镜片的前表面的X方向通光半孔径以及Y方向通光半孔径均是5.57mm；所述第三镜片的后表面的曲率半径是-26.53mm，第三镜片的后表面的X方向通光半孔径以及Y方向通光半孔径均是4.24mm；所述第二镜片的后表面与第三镜片的前表面之间的距离，即第二镜片与第三镜片之间的距离是5.17mm；所述第三镜片的前表面与第三镜片的后表面之间的距离，即第三镜片的中心厚度是3.0mm；The radius of curvature of the front surface of the third lens is 33.6240mm, the X-direction light semi-aperture and the Y-direction light semi-aperture of the front surface of the third lens are both 5.57mm; the curvature of the rear surface of the third lens is The radius is -26.53mm, and the X-direction light semi-aperture and the Y-direction light semi-aperture of the rear surface of the third lens are both 4.24mm; the distance between the rear surface of the second lens and the front surface of the third lens , that is, the distance between the second lens and the third lens is 5.17mm; the distance between the front surface of the third lens and the rear surface of the third lens, that is, the central thickness of the third lens is 3.0mm;
所述第三镜片的后表面与1/3英寸图像探测器之间的距离是39.3135mm。The distance between the rear surface of the third optic and the 1/3 inch image detector is 39.3135mm.
上述相位掩膜板的2D掩膜函数形式是：The 2D mask function form of the above phase mask is:
其中：in:
α表征三次方形相位掩膜板的相位调制强度，所述α取值是0.0130mm；α represents the phase modulation intensity of the cubic phase mask, and the value of α is 0.0130mm;
x以及y均为归一化的孔径坐标，单位mm，所述x以及y取值范围均为[-4.9513，4.9513]。Both x and y are normalized aperture coordinates, in mm, and the value ranges of x and y are both [-4.9513, 4.9513].
上述波前编码成像镜头的焦距是50mm，相对孔径1：4.5，全视场角约是10°，工作谱段480um～680um。The focal length of the above-mentioned wavefront encoding imaging lens is 50mm, the relative aperture is 1:4.5, the full field of view is about 10°, and the working spectrum is 480um～680um.
一种基于如上所述的波前编码成像系统的基于单幅图像放大的超分辨处理方法，其特征在于：所述基于单幅图像放大的超分辨处理方法包括以下步骤：A super-resolution processing method based on the enlargement of a single image based on the above-mentioned wavefront encoding imaging system, characterized in that: the super-resolution processing method based on the enlargement of a single image comprises the following steps:
1)选定超分辨所希望得到的传感器实际像元的大小pnew，并据此确定超分辨率网格图像Y与解码之前呈均匀模糊的原始分辨率图像I之间的比例系数，即所述比例系数就是目标图像的放大倍数，所述目标图像的放大倍数是δ＝[p/pnew]，1) Select the actual pixel size pnew of the sensor desired for super-resolution, and determine the proportional coefficient between the super-resolution grid image Y and the uniformly blurred original resolution image I before decoding, that is, The scale factor is exactly the magnification of the target image, and the magnification of the target image is δ=[p/pnew ],
其中：in:
[]代表取整；[] stands for rounding;
p则代表与解码之前呈均匀模糊的原始分辨率图像I对应的传感器像元大小；p represents the sensor pixel size corresponding to the uniformly blurred original resolution image I before decoding;
2)将解码之前呈均匀模糊的原始分辨率图像I的行方向以及列方向均放大δ倍，此时超分辨率网格图像Y的有效像素数将变为解码之前呈均匀模糊的原始分辨率图像I的δ2倍；2) Enlarge the row direction and column direction of the original resolution image I that was uniformly blurred before decoding by δ times, at this time the effective number of pixels of the super-resolution grid image Y will become the original resolution that was uniformly blurred before decoding δ2 times of image I;
3)利用如上所示的波前编码成像镜头的参数获得所希望得到的具有更小像元图像传感器采样之前的原始点扩散函数Psfoptical；3) Utilize the parameters of the wavefront encoding imaging lens as shown above to obtain the desired original point spread function Psfoptical before the sampling of the image sensor with smaller pixels;
4)根据目标图像的放大倍数δ，通过反向填充重采样，利用Psfoptical计算对应于更小的传感器实际物理像元pnew的采样点扩散函数Psfcalculated；4) According to the magnification δ of the target image, by reverse filling and resampling, use Psfoptical to calculate the sampling point spread function Psfcalculated corresponding to the actual physical pixel pnew of the sensor;
5)以Psfcalculated作为先验知识，对行方向以及列方向均已经放大到目标图像的放大倍数δ倍的模糊图像进行复原滤波，实现基于单幅图像放大的超分辨率重建。5) Using Psfcalculated as prior knowledge, restore and filter the blurred image that has been enlarged to δ times the magnification of the target image in both the row direction and the column direction, and realize super-resolution reconstruction based on single image enlargement.
上述步骤2)中将解码之前呈均匀模糊的原始分辨率图像I的行方向以及列方向均放大δ倍时所采用的插值方法是最邻近插值。In the above step 2), the interpolation method adopted when the row direction and the column direction of the original resolution image I that was uniformly blurred before decoding is enlarged by δ times is nearest neighbor interpolation.
上述步骤3)中获得所希望得到的具有更小像元图像传感器采样之前的原始点扩散函数Psfoptical的方式是由光学系统设计软件导出。The way to obtain the desired original point spread function Psfoptical before sampling of the image sensor with smaller pixels in the above step 3) is derived by the optical system design software.
上述放大倍数不大于4。The above-mentioned magnification factor is not greater than 4.
本发明的优点是：The advantages of the present invention are:
本发明在深入研究波前编码成像系统物理特性的基础上，提出了一种适合应用于波前编码系统的、基于图像放大的超分辨率处理方法，在不改变探测器像元尺寸的前提下，不但可以实现超大焦深成像，而且能够达到超分辨率成像的目的。图像放大是最简单的将低分辨率图像映射到高分辨率网格的方法。传统的基于插值的图像放大并不能带来信息量的增加，而只是单纯地增加了像素数目，如果处理不当，还会对细节的分辨带来不利的影响。本发明所提出的超分辨处理方法以波前编码成像机理为理论基础，不但能够增加图像的有效像素数目，而且可以消除传统图像放大对局部细节，如边缘等特征的破坏，是对波前编码成像技术潜在特性的深入挖掘。据此，本发明设计了搭载经典三次方形相位板的波前编码成像系统用于试验验证。结果表明，这种基于波前编码机理的超分辨率处理方法在局部放大细节的精确描述方面具有极大的优势。Based on the in-depth study of the physical characteristics of the wavefront coding imaging system, the present invention proposes a super-resolution processing method based on image amplification suitable for wavefront coding systems, without changing the pixel size of the detector, Not only can ultra-large focal depth imaging be achieved, but also super-resolution imaging can be achieved. Image upscaling is the easiest way to map a low-resolution image to a high-resolution grid. Traditional image enlargement based on interpolation does not increase the amount of information, but simply increases the number of pixels. If it is not handled properly, it will also adversely affect the resolution of details. The super-resolution processing method proposed by the present invention is based on the wavefront encoding imaging mechanism, which can not only increase the number of effective pixels of the image, but also eliminate the destruction of local details, such as edges and other features, caused by traditional image amplification. A deep dive into the underlying properties of imaging technologies. Accordingly, the present invention designs a wavefront encoding imaging system equipped with a classical cubic phase plate for experimental verification. The results show that this super-resolution processing method based on the wavefront coding mechanism has great advantages in the accurate description of locally enlarged details.
附图说明Description of drawings
图1是波前编码大焦深成像系统的原理示意图；Figure 1 is a schematic diagram of the principle of the wavefront encoding large depth of focus imaging system;
图2是波前编码大焦深成像系统对应于不同成像距离(不同离焦量下)的光学传递函数；Fig. 2 is the optical transfer function corresponding to different imaging distances (under different defocus amounts) of the wavefront encoded large focal depth imaging system;
图3是由光学设计软件导出的光学点扩散函数通过模拟采样计算得到的对应于不同传感器物理像元大小的采样点扩散函数；Figure 3 is the sampling point spread function corresponding to different sensor physical pixel sizes calculated by the optical point spread function derived from the optical design software through analog sampling;
图4是波前编码大焦深成像系统的实物以及在平行光管下的经5.2um像元大小图像传感器采样得到的实际点扩散函数；Fig. 4 is the real object of the wavefront encoding large depth of focus imaging system and the actual point spread function obtained by sampling the 5.2um pixel size image sensor under the collimator;
图5是本发明所提供的针对波前编码成像系统的超分辨率处理流程图；Fig. 5 is a flow chart of super-resolution processing for the wavefront coding imaging system provided by the present invention;
图6是波前编码大焦深成像系统的一组外场试验结果以及相应的超分辨率处理后的局部效果。Fig. 6 is a set of field test results of the wavefront coding large focal depth imaging system and the corresponding local effects after super-resolution processing.
具体实施方式detailed description
本发明涉及一套完整的基于波前编码机理的大焦深成像系统以及针对波前编码成像系统的基于单幅图像放大的超分辨率处理方法。The invention relates to a complete set of large focal depth imaging system based on wavefront coding mechanism and a super-resolution processing method based on single image enlargement for the wavefront coding imaging system.
本发明提供了一种波前编码成像系统，包括波前编码成像镜头、1/3英寸图像探测器以及解码处理单元，波前编码成像镜头包括第一镜片(图1中的镜片1)、相位掩膜板、第二镜片(图1中的镜片2)以及第三镜片(图1中的镜片3)；第一镜片、相位掩膜板、第二镜片、第三镜片，1/3英寸图像探测器以及解码处理单元依次设置在同一光路上；The present invention provides a wavefront coding imaging system, comprising a wavefront coding imaging lens, a 1/3 inch image detector and a decoding processing unit, the wavefront coding imaging lens includes a first lens (glass 1 in Fig. 1), a phase Mask, second mirror (mirror 2 in Figure 1), and third mirror (mirror 3 in Figure 1); first mirror, phase mask, second mirror, third mirror, 1/3-inch image The detector and the decoding processing unit are sequentially arranged on the same optical path;
第一镜片的前表面的曲率半径是18.91mm，第一镜片的前表面的X方向通光半孔径以及Y方向通光半孔径均是6.22mm；第一镜片的后表面的曲率半径是234mm，第一镜片的后表面的X方向通光半孔径以及Y方向通光半孔径均是5.85mm；第一镜片的前表面与第一镜片的后表面之间的距离，即第一镜片的中心厚度是3.0mm；The radius of curvature of the front surface of the first lens is 18.91mm, the semi-aperture of the X direction and the semi-aperture of the Y direction of the front surface of the first lens are both 6.22mm; the radius of curvature of the rear surface of the first lens is 234mm, The X-direction light semi-aperture and the Y-direction light semi-aperture of the rear surface of the first lens are both 5.85mm; the distance between the front surface of the first lens and the rear surface of the first lens, that is, the central thickness of the first lens is 3.0mm;
相位掩膜板的前表面的X方向通光半孔径以及Y方向通光半孔径均是4.95mm；相位掩膜板的后表面的X方向通光半孔径以及Y方向通光半孔径均是4.42mm；第一镜片的后表面与相位掩膜板的前表面之间的距离，即第一镜片与相位掩膜板之间的距离是3.22mm；相位掩膜板的前表面与相位掩膜板的后表面之间的距离，即相位掩膜板的厚度是2.0mm；The X-direction light semi-aperture and the Y-direction light semi-aperture of the front surface of the phase mask are both 4.95mm; the X-direction light semi-aperture and the Y direction light semi-aperture of the back surface of the phase mask are both 4.42mm mm; the distance between the rear surface of the first lens and the front surface of the phase mask, that is, the distance between the first lens and the phase mask is 3.22mm; the front surface of the phase mask and the phase mask The distance between the rear surfaces, that is, the thickness of the phase mask is 2.0mm;
第二镜片的前表面的曲率半径是-34.973mm，第二镜片的前表面的X方向通光半孔径以及Y方向通光半孔径均是4.43mm；第二镜片的后表面的曲率半径是15.776mm，第二镜片的后表面的X方向通光半孔径以及Y方向通光半孔径均是5.46mm；相位掩膜板的后表面与第二镜片的前表面之间的距离，即相位掩膜板与第二镜片之间的距离是2.0mm；第二镜片的前表面与第二镜片的后表面之间的距离，即第二镜片的中心厚度是3.0mm；The radius of curvature of the front surface of the second lens is -34.973mm, the X direction light semi-aperture and the Y direction light semi-aperture of the front surface of the second lens are both 4.43mm; the curvature radius of the rear surface of the second lens is 15.776 mm, the X-direction light semi-aperture and the Y-direction light semi-aperture of the rear surface of the second lens are both 5.46mm; the distance between the rear surface of the phase mask plate and the front surface of the second lens, that is, the phase mask The distance between the plate and the second lens is 2.0 mm; the distance between the front surface of the second lens and the rear surface of the second lens, ie the central thickness of the second lens is 3.0 mm;
第三镜片的前表面的曲率半径是33.6240mm，第三镜片的前表面的X方向通光半孔径以及Y方向通光半孔径均是5.57mm；第三镜片的后表面的曲率半径是-26.53mm，第三镜片的后表面的X方向通光半孔径以及Y方向通光半孔径均是4.24mm；第二镜片的后表面与第三镜片的前表面之间的距离，即第二镜片与第三镜片之间的距离是5.17mm；第三镜片的前表面与第三镜片的后表面之间的距离，即第三镜片的中心厚度是3.0mm；The radius of curvature of the front surface of the third lens is 33.6240mm, the semi-aperture of the X direction and the semi-aperture of the Y direction of the front surface of the third lens are both 5.57mm; the radius of curvature of the rear surface of the third lens is -26.53 mm, the X direction light semi-aperture and the Y direction light semi-aperture of the rear surface of the third eyeglass are both 4.24mm; the distance between the back surface of the second eyeglass and the front surface of the third eyeglass, that is, the second eyeglass and The distance between the third lens is 5.17mm; the distance between the front surface of the third lens and the rear surface of the third lens, that is, the central thickness of the third lens is 3.0mm;
第三镜片的后表面与1/3英寸图像探测器之间的距离是39.3135mm。The distance between the rear surface of the third optic and the 1/3 inch image detector is 39.3135mm.
相位掩膜板的2D掩膜函数形式是：The 2D mask function form of the phase mask is:
其中：in:
α表征三次方形相位掩膜板的相位调制强度，α取值是0.0130mm；α represents the phase modulation intensity of the cubic phase mask, and the value of α is 0.0130mm;
x以及y均为归一化的孔径坐标，单位mm，x以及y取值范围均为[-4.9513，4.9513]。Both x and y are normalized aperture coordinates, in mm, and the value range of x and y is [-4.9513, 4.9513].
波前编码成像镜头的焦距是50mm，相对孔径1：4.5，全视场角约是10°，工作谱段480um～680um。The focal length of the wavefront encoding imaging lens is 50mm, the relative aperture is 1:4.5, the full field of view is about 10°, and the working spectrum is 480um～680um.
本发明在提供如上波前编码成像系统的同时，还提供了一种基于该波前编码成像系统的基于单幅图像放大的超分辨处理方法，基于单幅图像放大的超分辨处理方法包括以下步骤：While providing the above wavefront encoding imaging system, the present invention also provides a super-resolution processing method based on single image enlargement based on the wavefront encoding imaging system. The super-resolution processing method based on single image enlargement includes the following steps :
1)选定超分辨所希望得到的传感器实际像元的大小pnew，并据此确定超分辨率网格图像Y与解码之前呈均匀模糊的原始分辨率图像I之间的比例系数，即比例系数就是目标图像的放大倍数，目标图像的放大倍数是δ＝[p/pnew]，1) Select the actual pixel size pnew of the sensor desired for super-resolution, and accordingly determine the ratio coefficient between the super-resolution grid image Y and the uniformly blurred original resolution image I before decoding, that is, the ratio The coefficient is the magnification of the target image, and the magnification of the target image is δ=[p/pnew ],
其中：in:
[]代表取整；[] stands for rounding;
p则代表与解码之前呈均匀模糊的原始分辨率图像I对应的传感器像元大小；p represents the sensor pixel size corresponding to the uniformly blurred original resolution image I before decoding;
2)将解码之前呈均匀模糊的原始分辨率图像I的行方向以及列方向均放大δ倍，此时超分辨率网格图像Y的有效像素数将变为解码之前呈均匀模糊的原始分辨率图像I的δ2倍；采用的插值方法是最邻近插值；2) Enlarge the row direction and column direction of the original resolution image I that was uniformly blurred before decoding by δ times, at this time the effective number of pixels of the super-resolution grid image Y will become the original resolution that was uniformly blurred before decoding δ2 times of image I; the interpolation method adopted is nearest neighbor interpolation;
3)利用如上所示的波前编码成像镜头的参数获得所希望得到的具有更小像元图像传感器采样之前的原始点扩散函数Psfoptical；步骤3)中获得所希望得到的具有更小像元图像传感器采样之前的原始点扩散函数Psfoptical的方式是由光学系统设计软件导出。3) Utilize the parameters of the wavefront encoding imaging lens as shown above to obtain the desired original point spread function Psfoptical before the sampling of the image sensor with smaller pixels; in step 3), obtain the desired image sensor with smaller pixels The original point spread function Psfoptical of the image sensor before sampling is derived by the optical system design software.
4)根据目标图像的放大倍数δ，通过反向填充重采样，利用Psfoptical计算对应于更小的传感器实际物理像元pnew的采样点扩散函数Psfcalculated；4) According to the magnification δ of the target image, by reverse filling and resampling, use Psfoptical to calculate the sampling point spread function Psfcalculated corresponding to the actual physical pixel pnew of the sensor;
5)以Psfcalculated作为先验知识，对行方向以及列方向均已经放大到目标图像的放大倍数δ倍的模糊图像进行复原滤波，实现基于单幅图像放大的超分辨率重建。5) Using Psfcalculated as prior knowledge, restore and filter the blurred image that has been enlarged to δ times the magnification of the target image in both the row direction and the column direction, and realize super-resolution reconstruction based on single image enlargement.
参考图1(波前编码大焦深成像系统的原型设计框架)，在本发明所提出的系统中，成像目标1通过常规波前编码成像镜头2后，在1/3英寸图像探测器3上形成模糊的中间像，然后解码处理单元4进行去卷积处理，最终得到聚焦清晰的大焦深图像。除了能够获得原始分辨率大焦深图像之外，解码处理单元4还可以根据需要实施基于单幅图像放大的超分辨率重建，通过计算得到对应于更小物理像元尺寸传感器的图像。With reference to Fig. 1 (prototype design frame of wavefront encoding large depth of focus imaging system), in the system proposed by the present invention, after imaging target 1 passes through conventional wavefront encoding imaging lens 2, on 1/3 inch image detector 3 A blurred intermediate image is formed, and then the decoding processing unit 4 performs deconvolution processing to finally obtain a clearly focused image with a large depth of focus. In addition to being able to obtain images with original resolution and large depth of focus, the decoding processing unit 4 can also implement super-resolution reconstruction based on single image enlargement as required, and obtain images corresponding to sensors with smaller physical pixel sizes through calculation.
本发明所提出的波前编码成像镜头是在经典Cook三片物镜的基础上搭载具有三次方形相位分布的相位掩膜板获得的。三片常规球面透镜采用一般的玻璃透镜加工工艺即可获得，三次方形相位掩膜板则需要由五自由度自由曲面加工设备完成。目标场景所发出的成像光线在经过波前编码成像镜头以后，其所具携带的场景不同位置的深度信息被编码，随之而来的是系统对离焦的不敏感，由图2(波前编码大焦深成像系统对应于不同成像距离(5m(a)，7m(b)，10m(c)，50m(d)以及无穷远(e))所示。其中横坐标代表以每毫米线对数表征的空间频率，纵坐标则代表归一化后的MTF幅值。)所示的调制传递函数MTF反应出来。可以看到，不同离焦量、不同视场以及不同波长所对应的MTF均具有极好的一致性。虽然编码后的MTF值要小于编码前的值，但是在存在离焦量的情况下它不会出现零值，因此不会造成图像细节信息的丢失。图像探测器上是一幅呈现均匀一致模糊的图像，用解码处理单元对它进行解码，将系统的MTF值提升至接近衍射受限，从而恢复出锐利清晰的图像。The wavefront encoding imaging lens proposed by the present invention is obtained by carrying a phase mask with a cubic phase distribution on the basis of the classic three-piece Cook objective lens. Three conventional spherical lenses can be obtained by general glass lens processing technology, and the cubic phase mask plate needs to be completed by five-degree-of-freedom free-form surface processing equipment. After the imaging light emitted by the target scene passes through the wavefront encoding imaging lens, the depth information of different positions of the scene carried by it is encoded, followed by the insensitivity of the system to defocus, as shown in Figure 2 (wavefront The coded large focal depth imaging system corresponds to different imaging distances (5m (a), 7m (b), 10m (c), 50m (d) and infinity (e)). The abscissa represents the line pair per millimeter The spatial frequency represented by the number, and the ordinate represents the normalized MTF amplitude.) The modulation transfer function MTF shown is reflected. It can be seen that the MTFs corresponding to different defocus amounts, different fields of view and different wavelengths have excellent consistency. Although the MTF value after encoding is smaller than the value before encoding, it will not have a zero value in the presence of defocus, so it will not cause loss of image detail information. On the image detector is a uniform blurred image, which is decoded by the decoding processing unit, and the MTF value of the system is raised to be close to diffraction-limited, thereby restoring a sharp and clear image.
本发明所涉及的波前编码成像系统在投入使用之前，需要在平行光管的帮助下进行标定工作。研究表明：对于矩形可分的相位掩膜板来说，其所对应的点扩散函数并非旋转对称，其主要能量分布在正交的x方向和y方向。如果波前编码成像镜头点扩散函数的x轴y轴与图像传感器的x轴y轴不能良好对应，那么就会在复原图像中产生强烈的伪信息。因此，本发明所设计的相位掩膜板安装在一个具有机械旋动装置的夹持框上(如图4所示)，可以在0～90°的范围内实现相位掩膜板以光轴为中心的旋转。这一结构就能够做到使相位板的两轴与图像传感器两轴的较为准确的对齐。Before the wavefront encoding imaging system involved in the present invention is put into use, it needs to be calibrated with the help of a collimator. The research shows that for a rectangular separable phase mask, the corresponding point spread function is not rotationally symmetric, and its main energy is distributed in the orthogonal x and y directions. If the x-axis and y-axis of the point spread function of the wavefront encoded imaging lens do not correspond well to the x-axis and y-axis of the image sensor, strong false information will be generated in the restored image. Therefore, the phase mask designed by the present invention is installed on a clamping frame with a mechanical rotating device (as shown in Figure 4), and the phase mask can be realized within the range of 0-90° with the optical axis as the Center of rotation. This structure can achieve relatively accurate alignment of the two axes of the phase plate and the two axes of the image sensor.
鉴于波前编码技术的特殊性，本发明提出了一种基于单幅图像放大的超分辨率重建处理方法，可以在不改变图像传感器硬件的前提下获得对应于更小物理像元的重建图像。这一方法的提出是以下述研究结果为基础的。In view of the particularity of wavefront coding technology, the present invention proposes a super-resolution reconstruction processing method based on single image enlargement, which can obtain reconstructed images corresponding to smaller physical pixels without changing the hardware of the image sensor. This method is proposed based on the following research results.
任何一个成像镜头在经过良好的像差校正之后，其所具有的光学点扩散函数的支持域将非常紧凑。此时，利用光学点扩散函数模拟经探测器采样后的系统点扩散函数时，不同像元大小探测器所对应的结果之间的差异很小。然而，由于波前编码系统的光学点扩散函数具有极大的支持域，所以像元大小的不同将产生截然不同的系统点扩散函数。像元越小，采样点扩散函数就越接近光学点扩散函数。换句话说，像元越小，采样点扩散函数对系统的描述就越精确。如图3(图3给出了与该波前编码成像镜头对应的、由光学设计软件导出的光学点扩散函数通过模拟采样计算得到的对应于不同传感器物理像元大小(5.2um，3.2um，2.6um以及1.7um)的采样点扩散函数)所示，通过模拟计算，由光学系统设计软件导出的光学点扩散函数能够获得与不同大小像元传感器(5.2um，3.2um，2.6um以及1.7um)相对应的采样点扩散函数。通过比较图3与图4(图4给出了该波前编码大焦深成像系统的实物以及在平行光管下的经5.2um像元大小图像传感器采样得到的实际点扩散函数)，不难发现，模拟产生的5.2um像元对应的点扩散函数与平行光管下的实测点扩散函数具有很高的相似度。因此，利用光学系统设计软件提取光学点扩散函数以后，就可以较为精确地计算出经不同大小像元采样后的实际点扩散函数，该研究结果就为不改变传感器硬件条件而获得对应于更小像元的实际采样点扩散函数奠定了理论基础。After any imaging lens has undergone good aberration correction, the support domain of its optical point spread function will be very compact. At this time, when using the optical point spread function to simulate the system point spread function sampled by the detector, the difference between the results corresponding to the detectors with different pixel sizes is very small. However, since the optical point spread function of the wavefront encoding system has a very large support domain, the difference in pixel size will produce completely different system point spread functions. The smaller the pixel, the closer the sampling point spread function is to the optical point spread function. In other words, the smaller the pixel, the more accurate the sampling point spread function describes the system. As shown in Figure 3 (Figure 3 shows the optical point spread function derived by the optical design software corresponding to the wavefront encoding imaging lens and obtained by analog sampling corresponding to different sensor physical pixel sizes (5.2um, 3.2um, 2.6um and 1.7um) sampling point spread function), through simulation calculation, the optical point spread function derived by the optical system design software can be obtained with different size pixel sensors (5.2um, 3.2um, 2.6um and 1.7um ) corresponding to the sampling point spread function. By comparing Figure 3 and Figure 4 (Figure 4 shows the real object of the wavefront encoding large focal depth imaging system and the actual point spread function obtained by sampling the 5.2um pixel size image sensor under the collimator), it is not difficult It is found that the point spread function corresponding to the simulated 5.2um pixel has a high similarity with the measured point spread function under the collimator. Therefore, after using the optical system design software to extract the optical point spread function, the actual point spread function after sampling by pixels of different sizes can be calculated more accurately. The actual sampling point spread function of the pixel lays the theoretical foundation.
基于波前编码成像系统的这一特性，本发明提出的针对波前编码系统的超分辨率处理流程如图5所示。首先，确定所希望得到的探测器物理像元大小，并通过与当前传感器实际像元大小的比较获得目标图像的放大倍率。比如当前图像传感器的像元大小是5.2um，假设想要获得2.6um传感器的成像效果，那么此时目标图像的放大倍率就等于2。其次，由光学点扩散函数出发，通过模拟采样获得与具有所需像元大小的探测器对应的实际采样点扩散函数。再次，将经相位板编码之后的模糊中间图像放大到相应的倍数。最后，利用模拟计算获得的与更小像元探测器对应的实际采样点扩散函数，通过忙卷积结合Richardson-Lucy算法对放大后的中间模糊图像进行复原重建，从而获得超分辨重建的效果。Based on this characteristic of the wavefront coding imaging system, the super-resolution processing flow for the wavefront coding system proposed by the present invention is shown in FIG. 5 . First, determine the desired physical pixel size of the detector, and obtain the magnification of the target image by comparing it with the actual pixel size of the current sensor. For example, the pixel size of the current image sensor is 5.2um. Suppose you want to obtain the imaging effect of the 2.6um sensor, then the magnification of the target image at this time is equal to 2. Secondly, starting from the optical point spread function, the actual sampling point spread function corresponding to the detector with the required pixel size is obtained by analog sampling. Again, the blurred intermediate image encoded by the phase plate is enlarged to a corresponding multiple. Finally, using the actual sampling point spread function corresponding to the smaller pixel detector obtained by simulation calculation, the enlarged intermediate blurred image is restored and reconstructed through busy convolution combined with the Richardson-Lucy algorithm, so as to obtain the effect of super-resolution reconstruction.
图6给出了一组利用本发明提出的波前编码超分辨成像系统进行的外场试验结果，其中，原始分辨率图像所对应的传感器物理像元大小为5.2um。可以看到，解码之后的图像非常清晰，而且场景中的边缘都显著地增强了，这是因为相位掩膜板在消除离焦敏感性的同时也对系统的波前畸变起到了抑制作用，所以波前编码系统的滤波解码图像由于消除了波前畸变带来的模糊退化而产生了场景增强的效果。利用图5所述的超分辨率重建流程，图6也给出了相应的处理实例，其中4倍超分辨率的处理效果等价于由像元大小为1.3um的图像传感器所捕获的图像。可以看到，与单纯的放大相比，本发明所提出的超分辨处理方法可以显著增强放大后的效果，消除常规放大带来的锯齿效应，可以更加精确地对场景信息进行描述。这就证明，本发明所提出的针对波前编码系统的超分辨处理流程是非常有效的。然而，考虑到小像元带来的能量不足以及信噪比降低等问题，图5所示的超分辨处理流程所能够获得的放大倍数是存在限制的。研究表明，本发明所提出的超分辨率处理流程适用的放大倍数一般不超过4。Fig. 6 shows a group of field test results using the wavefront coding super-resolution imaging system proposed by the present invention, wherein the sensor physical pixel size corresponding to the original resolution image is 5.2um. It can be seen that the decoded image is very clear, and the edges of the scene are significantly enhanced. This is because the phase mask can suppress the wavefront distortion of the system while eliminating the defocus sensitivity, so The filtered decoded image of the wavefront coding system has the effect of scene enhancement due to the elimination of the blur degradation caused by the wavefront distortion. Using the super-resolution reconstruction process described in Fig. 5, Fig. 6 also shows a corresponding processing example, where the processing effect of 4 times super-resolution is equivalent to the image captured by an image sensor with a pixel size of 1.3um. It can be seen that, compared with simple zoom-in, the super-resolution processing method proposed in the present invention can significantly enhance the zoom-in effect, eliminate the jagged effect caused by conventional zoom-in, and describe the scene information more accurately. This proves that the super-resolution processing flow for the wavefront coding system proposed by the present invention is very effective. However, considering the problems of insufficient energy and reduced signal-to-noise ratio caused by small pixels, the magnification that can be obtained by the super-resolution process shown in Figure 5 is limited. Research shows that the applicable magnification factor of the super-resolution processing flow proposed by the present invention generally does not exceed 4.
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201510166370.7ACN104834088B (en) | 2015-04-09 | 2015-04-09 | Wavefront coding imaging system and super-resolution processing method based on single image amplification |
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201510166370.7ACN104834088B (en) | 2015-04-09 | 2015-04-09 | Wavefront coding imaging system and super-resolution processing method based on single image amplification |
| Publication Number | Publication Date |
|---|---|
| CN104834088A CN104834088A (en) | 2015-08-12 |
| CN104834088Btrue CN104834088B (en) | 2017-12-05 |
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201510166370.7AActiveCN104834088B (en) | 2015-04-09 | 2015-04-09 | Wavefront coding imaging system and super-resolution processing method based on single image amplification |
| Country | Link |
|---|---|
| CN (1) | CN104834088B (en) |
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US10795168B2 (en) | 2017-08-31 | 2020-10-06 | Metalenz, Inc. | Transmissive metasurface lens integration |
| US11906698B2 (en) | 2017-05-24 | 2024-02-20 | The Trustees Of Columbia University In The City Of New York | Broadband achromatic flat optical components by dispersion-engineered dielectric metasurfaces |
| US11978752B2 (en) | 2019-07-26 | 2024-05-07 | Metalenz, Inc. | Aperture-metasurface and hybrid refractive-metasurface imaging systems |
| US12140778B2 (en) | 2018-07-02 | 2024-11-12 | Metalenz, Inc. | Metasurfaces for laser speckle reduction |
| US12276807B2 (en) | 2022-03-31 | 2025-04-15 | Metalenz, Inc. | Polarization sorting metasurface microlens array device |
| US12416752B2 (en) | 2018-01-24 | 2025-09-16 | President And Fellows Of Harvard College | Polarization state generation with a metasurface |
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2022021425A1 (en)* | 2020-07-31 | 2022-02-03 | Huawei Technologies Co., Ltd. | Optical apparatus |
| CN113031260A (en)* | 2021-03-12 | 2021-06-25 | 中国人民解放军国防科技大学 | Wavefront coding imaging system based on arcsine type phase mask plate |
| CN113393380B (en)* | 2021-05-27 | 2024-04-09 | 中国科学院西安光学精密机械研究所 | High-resolution imaging camera with large F number diffraction real-time correction based on pixel coding |
| CN114488524B (en)* | 2022-02-19 | 2024-07-02 | 复旦大学 | Wavefront coding imaging system based on high-order polynomial free-form surface phase plate |
| CN116681595B (en)* | 2023-08-01 | 2023-11-03 | 长春理工大学 | A long-distance computing super-resolution imaging device based on multimodal PSF |
| CN118590773A (en)* | 2024-05-27 | 2024-09-03 | 哈尔滨工业大学 | A computational imaging system for backlit targets based on wavefront coding |
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2003009041A2 (en)* | 2001-07-20 | 2003-01-30 | The Regents Of The University Of Colorado | Wavefront coding zoom lens imaging systems |
| CN101449193A (en)* | 2006-03-06 | 2009-06-03 | 全视Cdm光学有限公司 | Zoom lens systems with wavefront coding |
| CN102243329A (en)* | 2010-05-14 | 2011-11-16 | 中国科学院西安光学精密机械研究所 | Phase plate for wavefront coded imaging techniques |
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN204719330U (en)* | 2015-04-09 | 2015-10-21 | 中国科学院西安光学精密机械研究所 | Wavefront coded imaging system |
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2003009041A2 (en)* | 2001-07-20 | 2003-01-30 | The Regents Of The University Of Colorado | Wavefront coding zoom lens imaging systems |
| CN101449193A (en)* | 2006-03-06 | 2009-06-03 | 全视Cdm光学有限公司 | Zoom lens systems with wavefront coding |
| CN102243329A (en)* | 2010-05-14 | 2011-11-16 | 中国科学院西安光学精密机械研究所 | Phase plate for wavefront coded imaging techniques |
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US11906698B2 (en) | 2017-05-24 | 2024-02-20 | The Trustees Of Columbia University In The City Of New York | Broadband achromatic flat optical components by dispersion-engineered dielectric metasurfaces |
| US10795168B2 (en) | 2017-08-31 | 2020-10-06 | Metalenz, Inc. | Transmissive metasurface lens integration |
| US11579456B2 (en) | 2017-08-31 | 2023-02-14 | Metalenz, Inc. | Transmissive metasurface lens integration |
| US11988844B2 (en) | 2017-08-31 | 2024-05-21 | Metalenz, Inc. | Transmissive metasurface lens integration |
| US12411348B2 (en) | 2017-08-31 | 2025-09-09 | Metalenz, Inc. | Transmissive metasurface lens integration |
| US12416752B2 (en) | 2018-01-24 | 2025-09-16 | President And Fellows Of Harvard College | Polarization state generation with a metasurface |
| US12140778B2 (en) | 2018-07-02 | 2024-11-12 | Metalenz, Inc. | Metasurfaces for laser speckle reduction |
| US11978752B2 (en) | 2019-07-26 | 2024-05-07 | Metalenz, Inc. | Aperture-metasurface and hybrid refractive-metasurface imaging systems |
| US12389700B2 (en) | 2019-07-26 | 2025-08-12 | Metalenz, Inc. | Aperture-metasurface and hybrid refractive-metasurface imaging systems |
| US12276807B2 (en) | 2022-03-31 | 2025-04-15 | Metalenz, Inc. | Polarization sorting metasurface microlens array device |
| Publication number | Publication date |
|---|---|
| CN104834088A (en) | 2015-08-12 |
| Publication | Publication Date | Title |
|---|---|---|
| CN104834088B (en) | Wavefront coding imaging system and super-resolution processing method based on single image amplification | |
| CN104834089B (en) | Wavefront coding imaging system and super-resolution processing method | |
| CN204719330U (en) | Wavefront coded imaging system | |
| US11721002B2 (en) | Imaging system and method for imaging objects with reduced image blur | |
| US8432479B2 (en) | Range measurement using a zoom camera | |
| JP5824113B2 (en) | System, method, and media for recording an image using an optical diffuser | |
| US20080089598A1 (en) | Apparatus and Method For Extended Depth of Field Imaging | |
| US8254714B2 (en) | Methods and systems for designing electromagnetic wave filters and electromagnetic wave filters designed using same | |
| WO2021103696A1 (en) | Transmission-based infrared coded aperture imaging system and super-resolution method therefor | |
| US9473700B2 (en) | Camera systems and methods for gigapixel computational imaging | |
| JP2013531268A (en) | Measuring distance using coded aperture | |
| Li et al. | Efficient assessment method of on-board modulation transfer function of optical remote sensing sensors | |
| CN102170526A (en) | Method for calculation of defocus fuzzy core and sharp processing of defocus fuzzy image of defocus fuzzy core | |
| Jiang et al. | Annular computational imaging: Capture clear panoramic images through simple lens | |
| CN107395933B (en) | Programmable aperture imaging system based on LCOS spatial light modulator and super-resolution method | |
| Hoffmire et al. | Deep learning for anisoplanatic optical turbulence mitigation in long-range imaging | |
| CN103743484B (en) | A kind of spectrum coding imaging device | |
| KR20170099959A (en) | Method for determining the complex amplitude of the electromagnetic field associated with a scene | |
| CN110018564B (en) | Optical imaging system for large depth of field continuation of wavefront coding space | |
| CN204613515U (en) | Wavefront coding imaging system | |
| Tang et al. | Utilizing optical aberrations for extended-depth-of-field panoramas | |
| CN115308894B (en) | Large focal depth microscopic imaging system and image restoration method | |
| CN103885176A (en) | Phase mask plate and wavefront coding imaging system capable of adjusting quality of intermediate coding image | |
| Zhou et al. | DR-UNet: dynamic residual U-Net for blind correction of optical degradation | |
| Zhang et al. | Analysis of radially restored images for spherical single lens cellphone camera |
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| EXSB | Decision made by sipo to initiate substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |