CN114494258A - Lens aberration prediction and image reconstruction method and device - Google Patents
Lens aberration prediction and image reconstruction method and deviceDownload PDFInfo
- Publication number
- CN114494258A CN114494258ACN202210394888.6ACN202210394888ACN114494258ACN 114494258 ACN114494258 ACN 114494258ACN 202210394888 ACN202210394888 ACN 202210394888ACN 114494258 ACN114494258 ACN 114494258A
- Authority
- CN
- China
- Prior art keywords
- aberration
- disparity
- prediction
- image reconstruction
- view
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/0002—Inspection of images, e.g. flaw detection
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/50—Image enhancement or restoration using two or more images, e.g. averaging or subtraction
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10052—Images from lightfield camera
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20081—Training; Learning
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20084—Artificial neural networks [ANN]
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20228—Disparity calculation for image-based rendering
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Quality & Reliability (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Image Processing (AREA)
- Studio Devices (AREA)
Abstract
Description
Translated fromChinese技术领域technical field
本申请涉及图像数据处理或产生技术领域，特别涉及一种镜头的像差预测和图像重建方法及装置。The present application relates to the technical field of image data processing or generation, and in particular, to a lens aberration prediction and image reconstruction method and device.
背景技术Background technique
二维成像传感器已经彻底改变了几乎所有领域，包括工业检测、移动设备、自动驾驶、监控、医疗诊断、生物学和天文学，受益于半导体产业的快速发展，数字传感器的像素大小在过去十年迅速增长。然而，大多数成像系统的实际性能已经达到了光学而不是电子器件的瓶颈，例如，对于一个千兆像素传感器，由于不完善的镜头或环境干扰引起的光学像差，正常成像系统的有效像素数通常会被限制在百万像素级别，进而导致从一个点发出的光散布在一个很大的区域上在二维传感器上。同时，将3D场景投影到2D平面会导致LF（Light Field，光场）的各种自由度的损失，例如深度和局部相干性。因此，使用集成传感器获取高密度深度图一直是一个挑战。2D imaging sensors have revolutionized almost all fields, including industrial inspection, mobile devices, autonomous driving, surveillance, medical diagnostics, biology, and astronomy. Benefiting from the rapid development of the semiconductor industry, the pixel size of digital sensors has rapidly increased in the past decade. increase. However, the practical performance of most imaging systems has reached the bottleneck of optics rather than electronics. For example, for a gigapixel sensor, the effective pixel count of a normal imaging system is due to optical aberrations caused by imperfect lenses or environmental disturbances. Usually limited to the megapixel level, which in turn results in light from one point being spread over a large area on a 2D sensor. At the same time, projecting a 3D scene to a 2D plane results in the loss of various degrees of freedom of the LF (Light Field), such as depth and local coherence. Therefore, obtaining high-density depth maps using integrated sensors has always been a challenge.
相关技术利用扫描光场成像，基于波动光学的数字AO（Adaptive optics，自适应光学）算法，即DAO（Digital Adaptive Optics，数字自适应光学）算法，以低成本实现稳健、通用和高性能的3D成像。具体而言，它通过物理方式模拟光传播的前向和反向过程，设置像差初值，计算镜头点扩散函数的，之后通过反卷积过程计算高分辨率图片，再通过图片和点扩散函数计算应该拍摄到的图片和现有传感器信号进行对比，进而修正像差，不断迭代直至收敛。The related technology utilizes scanning light field imaging, wave optics-based digital AO (Adaptive optics, adaptive optics) algorithm, namely DAO (Digital Adaptive Optics, digital adaptive optics) algorithm, to achieve robust, general and high-performance 3D at low cost imaging. Specifically, it simulates the forward and reverse process of light propagation by physical means, sets the initial value of aberration, calculates the point spread function of the lens, and then calculates the high-resolution image through the deconvolution process, and then uses the image and point spread The function calculation should compare the captured image with the existing sensor signal, and then correct the aberration, and iterate continuously until convergence.
然而，相关技术仍存在一定的缺陷，其主要分为两种：However, the related technologies still have certain defects, which are mainly divided into two types:
第一种，基于物理，前向后向传播过程复杂，计算机模拟时间较长，是否收敛不确定，对不同过程迭代方式，迭代过程不同，鲁棒性差，且反卷积过程耗时耗能巨大，迭代造成这一过程多次进行无法避免且难以并行。The first one is based on physics, the forward and backward propagation process is complex, the computer simulation time is long, and the convergence is uncertain. For different process iteration methods, the iteration process is different, the robustness is poor, and the deconvolution process is time-consuming and energy-consuming , the iteration causes this process to be unavoidable and difficult to parallelize multiple times.
第二种，基于简单的神经网络，例如3D CNN（Convolutional Neural Networks，卷积神经网络）等，整个过程网络负担过重，网络预测点扩散函数或者像差的性能不稳定，或精度较低，从而导致重建的效果较差。The second is based on simple neural networks, such as 3D CNN (Convolutional Neural Networks, convolutional neural networks), etc., the network is overburdened in the whole process, the performance of the network prediction point spread function or aberration is unstable, or the accuracy is low, As a result, the reconstruction effect is poor.
综上所述，相关技术无法同时保证视差估计精度和速度，亟需改善。To sum up, the related technology cannot guarantee the accuracy and speed of disparity estimation at the same time, and it is in urgent need of improvement.
发明内容SUMMARY OF THE INVENTION
本申请提供一种镜头的像差预测和图像重建方法及装置，以解决相关技术基于物理手段或简单的神经网络，传播过程复杂，鲁棒性较差且耗时耗能巨大，从而导致重建效果较差的技术问题。The present application provides a lens aberration prediction and image reconstruction method and device, to solve the problem that the related technology is based on physical means or simple neural network, the propagation process is complex, the robustness is poor, and the time-consuming and energy-consuming is huge, resulting in reconstruction effects. Poor technical issues.
本申请第一方面实施例提供一种镜头的像差预测和图像重建方法，包括以下步骤：扫描光场相机的采集数据，得到多视角图片；从所述多视角图片中提取每个视角的视角特征；基于所述每个视角的视角特征，将所述多视角图片按照预设顺序输入至像差预测模型，得到像差预测结果；以及根据所述像差预测结果获取高于第一预设辨率的像差，并根据所述像差得到图像重建图片。An embodiment of the first aspect of the present application provides a method for predicting aberration and image reconstruction of a lens, including the following steps: scanning data collected by a light field camera to obtain a multi-view image; and extracting the angle of view of each view from the multi-view image feature; based on the viewing angle features of each viewing angle, inputting the multi-view images into the disparity prediction model in a preset order to obtain a disparity prediction result; The aberration of the resolution is obtained, and the image reconstruction picture is obtained according to the aberration.
可选地，在本申请的一个实施例中，所述根据所述像差预测结果获取高于预设辨率的像差，包括：基于所述像差预测结果，利用泽尼克多项式的拟合，以生成高于所述预设辨率的像差。Optionally, in an embodiment of the present application, the obtaining an aberration higher than a preset resolution according to the aberration prediction result includes: using the Zernike polynomial fitting based on the aberration prediction result , to generate aberrations higher than the preset resolution.
可选地，在本申请的一个实施例中，所述根据所述像差得到图像重建图片，包括：根据所述像差利用预设点扩散函数计算实际点扩散函数，并和所述多视角图片进行反卷积，得到图像重建图片。Optionally, in an embodiment of the present application, obtaining the reconstructed picture according to the aberration includes: calculating an actual point spread function by using a preset point spread function according to the aberration, and comparing the multi-view angle with the The image is deconvolved to obtain an image reconstruction image.
可选地，在本申请的一个实施例中，所述基于所述每个视角的视角特征，将所述多视角图片按照预设顺序输入至像差预测模型，得到像差预测结果，包括：利用多头注意力机制对每个视角的像差进行预测，其中，所述像差预测模型学习所述像差的有关局部特征，得到低于第二预设分辨率的像差。Optionally, in an embodiment of the present application, inputting the multi-view pictures into a disparity prediction model in a preset order based on the viewing angle characteristics of each viewing angle, to obtain a disparity prediction result, including: The multi-head attention mechanism is used to predict the disparity of each viewing angle, wherein the disparity prediction model learns the relevant local features of the disparity, and obtains the disparity lower than the second preset resolution.
本申请第二方面实施例提供一种镜头的像差预测和图像重建装置，包括：扫描模块，用于扫描光场相机的采集数据，得到多视角图片；提取模块，用于从所述多视角图片中提取每个视角的视角特征；像差预测模块，用于基于所述每个视角的视角特征，将所述多视角图片按照预设顺序输入至像差预测模型，得到像差预测结果；以及重建模块，用于根据所述像差预测结果获取高于第一预设辨率的像差，并根据所述像差得到图像重建图片。An embodiment of the second aspect of the present application provides a lens aberration prediction and image reconstruction device, including: a scanning module for scanning data collected by a light field camera to obtain a multi-view image; an extraction module for scanning from the multi-view Extracting the viewing angle feature of each viewing angle from the picture; the disparity prediction module is used to input the multi-viewing view picture into the disparity prediction model according to the preset order based on the viewing angle characteristic of each viewing angle to obtain the disparity prediction result; and a reconstruction module, configured to acquire an aberration higher than the first preset resolution according to the aberration prediction result, and obtain an image reconstruction picture according to the aberration.
可选地，在本申请的一个实施例中，所述重建模块包括：拟合单元，用于基于所述像差预测结果，利用泽尼克多项式的拟合，以生成高于所述预设辨率的像差。Optionally, in an embodiment of the present application, the reconstruction module includes: a fitting unit, configured to use Zernike polynomial fitting based on the aberration prediction result to generate a higher-than-preset discrimination result. rate aberrations.
可选地，在本申请的一个实施例中，所述重建模块，包括：计算单元，用于根据所述像差利用预设点扩散函数计算实际点扩散函数，并和所述多视角图片进行反卷积，得到图像重建图片。Optionally, in an embodiment of the present application, the reconstruction module includes: a calculation unit, configured to use a preset point spread function to calculate an actual point spread function according to the aberration, and perform a calculation with the multi-view picture. Deconvolution to get the image reconstruction picture.
可选地，在本申请的一个实施例中，所述像差预测模块包括：预测单元，用于利用多头注意力机制对每个视角的像差进行预测，其中，所述像差预测模型学习所述像差的有关局部特征，得到低于第二预设分辨率的像差。Optionally, in an embodiment of the present application, the disparity prediction module includes: a prediction unit, configured to use a multi-head attention mechanism to predict the disparity of each viewing angle, wherein the disparity prediction model learns The aberrations lower than the second preset resolution are obtained from the relevant local features of the aberrations.
本申请第三方面实施例提供一种电子设备，包括：存储器、处理器及存储在所述存储器上并可在所述处理器上运行的计算机程序，所述处理器执行所述程序，以实现如上述实施例所述的镜头的像差预测和图像重建方法。An embodiment of a third aspect of the present application provides an electronic device, including: a memory, a processor, and a computer program stored on the memory and executable on the processor, where the processor executes the program to achieve The lens aberration prediction and image reconstruction method as described in the above embodiments.
本申请第四方面实施例提供一种计算机可读存储介质，所述计算机可读存储介质存储计算机指令，所述计算机指令用于使所述计算机执行如上述实施例所述的镜头的像差预测和图像重建方法。Embodiments of the fourth aspect of the present application provide a computer-readable storage medium, where the computer-readable storage medium stores computer instructions, where the computer instructions are used to cause the computer to perform the lens aberration prediction according to the above embodiments and image reconstruction methods.
本申请实施例可以利用扫描光场相机获取多视角图片，并提取每个视角的视角特征，利用像差预测模型获取像差预测结果，进而根据像差获得图像重建照片，可以在保证视差估计精度的同时，实现高速无迭代、内存负担小的图片重建，性能稳定，鲁棒性高且可以并行。由此，解决了相关技术基于物理手段或简单的神经网络，传播过程复杂，鲁棒性较差且耗时耗能巨大，从而导致重建效果较差的技术问题。In this embodiment of the present application, a scanning light field camera can be used to obtain multi-view pictures, and the feature of each view angle can be extracted, and the aberration prediction model can be used to obtain the aberration prediction result, and then the image reconstruction photo can be obtained according to the aberration, which can ensure the accuracy of the parallax estimation. At the same time, it achieves high-speed image reconstruction without iteration and low memory burden, with stable performance, high robustness and parallelism. Thereby, the technical problems that the related technology is based on physical means or simple neural network, the propagation process is complex, the robustness is poor, and the time-consuming and energy-consuming are huge, resulting in poor reconstruction effect are solved.
本申请附加的方面和优点将在下面的描述中部分给出，部分将从下面的描述中变得明显，或通过本申请的实践了解到。Additional aspects and advantages of the present application will be set forth, in part, in the following description, and in part will be apparent from the following description, or learned by practice of the present application.
附图说明Description of drawings
本申请上述的和/或附加的方面和优点从下面结合附图对实施例的描述中将变得明显和容易理解，其中：The above and/or additional aspects and advantages of the present application will become apparent and readily understood from the following description of embodiments taken in conjunction with the accompanying drawings, wherein:
图1为根据本申请实施例提供的一种镜头的像差预测和图像重建方法的流程图；FIG. 1 is a flowchart of a method for aberration prediction and image reconstruction of a lens provided according to an embodiment of the present application;
图2为根据本申请一个实施例的镜头的像差预测和图像重建方法的流程图；FIG. 2 is a flowchart of a method for aberration prediction and image reconstruction of a lens according to an embodiment of the present application;
图3为根据本申请实施例提供的一种镜头的像差预测和图像重建装置的结构示意图；3 is a schematic structural diagram of a lens aberration prediction and image reconstruction device according to an embodiment of the present application;
图4为根据本申请实施例提供的电子设备的结构示意图。FIG. 4 is a schematic structural diagram of an electronic device provided according to an embodiment of the present application.
具体实施方式Detailed ways
下面详细描述本申请的实施例，所述实施例的示例在附图中示出，其中自始至终相同或类似的标号表示相同或类似的元件或具有相同或类似功能的元件。下面通过参考附图描述的实施例是示例性的，旨在用于解释本申请，而不能理解为对本申请的限制。The following describes in detail the embodiments of the present application, examples of which are illustrated in the accompanying drawings, wherein the same or similar reference numerals refer to the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary, and are intended to be used to explain the present application, but should not be construed as a limitation to the present application.
下面参考附图描述本申请实施例的镜头的像差预测和图像重建方法及装置。针对上述背景技术中心提到的相关技术基于物理手段或简单的神经网络，传播过程复杂，鲁棒性较差且耗时耗能巨大，从而导致重建效果较差的技术问题，本申请提供了一种镜头的像差预测和图像重建方法，在该方法中，可以利用扫描光场相机获取多视角图片，并提取每个视角的视角特征，利用像差预测模型获取像差预测结果，进而根据像差获得图像重建照片，可以在保证视差估计精度的同时，实现高速无迭代、内存负担小的图片重建，性能稳定，鲁棒性高且可以并行。由此，解决了相关技术基于物理手段或简单的神经网络，传播过程复杂，鲁棒性较差且耗时耗能巨大，从而导致重建效果较差的技术问题。The following describes the method and apparatus for aberration prediction and image reconstruction of a lens according to the embodiments of the present application with reference to the accompanying drawings. In view of the technical problems that the related technologies mentioned in the above-mentioned background technology center are based on physical means or simple neural networks, the propagation process is complex, the robustness is poor, and the time-consuming and energy-consuming is huge, resulting in poor reconstruction effect. A method of aberration prediction and image reconstruction of a lens, in this method, a scanning light field camera can be used to obtain multi-view images, and the viewing angle features of each view can be extracted, and the aberration prediction results can be obtained by using the aberration prediction model, and then based on the image The image reconstruction photo obtained from the difference can achieve high-speed image reconstruction without iteration and low memory burden while ensuring the accuracy of disparity estimation, with stable performance, high robustness and parallelization. As a result, the technical problems that the related technology is based on physical means or simple neural network, the propagation process is complex, the robustness is poor, and the time-consuming and energy-consuming are huge, resulting in poor reconstruction effect.
具体而言，图1为本申请实施例所提供的一种镜头的像差预测和图像重建方法的流程示意图。Specifically, FIG. 1 is a schematic flowchart of a method for aberration prediction and image reconstruction of a lens according to an embodiment of the present application.
如图1所示，该镜头的像差预测和图像重建方法包括以下步骤：As shown in Figure 1, the aberration prediction and image reconstruction method for this lens includes the following steps:
在步骤S101中，扫描光场相机的采集数据，得到多视角图片。In step S101, the collected data of the light field camera is scanned to obtain a multi-view image.
在实际执行过程中，本申请实施例可以通过扫描光场相机的数据，获得数据图片，并对数据图片进行对齐，从而得到多视角图片，本申请实施例利用扫描光场相机部分视角图片，可以去掉数据图片中因光线较暗无法获取信息的视角，从而提升图像重建的效果。In the actual execution process, the embodiments of the present application can obtain data pictures by scanning the data of the light field camera, and align the data pictures to obtain multi-view images. The viewing angle that cannot obtain information due to low light in the data picture is removed, thereby improving the effect of image reconstruction.
在步骤S102中，从多视角图片中提取每个视角的视角特征。In step S102, view features of each view are extracted from the multi-view pictures.
可以理解的是，网络是一种较为常用的特征提取器，可以快速并行化实现特征提取，因此，本申请实施例可以将多个视角的图片，输入到残差神经网络ResNet101中，提取每个视角特征。It can be understood that the network is a relatively common feature extractor, which can quickly implement feature extraction in parallel. Therefore, in this embodiment of the present application, pictures from multiple perspectives can be input into the residual neural network ResNet101 to extract each image. point of view features.
在步骤S103中，基于每个视角的视角特征，将多视角图片按照预设顺序输入至像差预测模型，得到像差预测结果。In step S103 , the multi-view pictures are input into the disparity prediction model in a preset order based on the viewpoint feature of each viewpoint, to obtain a disparity prediction result.
作为一种可能实现的方式，本申请实施例可以将多个视角的图片按预设顺序，例如，从上到小从左到右的顺序，输入像差预测模型transformer当中，像差预测模型transformer利用较短的2层2多头的注意力机制预测每张图对应像差的低分辨率的梯度，从而得到像差预测结果。As a possible implementation manner, in this embodiment of the present application, pictures from multiple viewing angles may be input into the disparity prediction model transformer in a preset order, for example, from top to bottom and from left to right, and the disparity prediction model transformer A short 2-layer 2-multiple attention mechanism is used to predict the low-resolution gradient of each image corresponding to the disparity, so as to obtain the disparity prediction result.
需要注意的是，预设顺序可以由本领域技术人员根据实际情况进行设置，在此不做具体限制。It should be noted that the preset sequence may be set by those skilled in the art according to actual conditions, which is not specifically limited herein.
可选地，在本申请的一个实施例中，基于每个视角的视角特征，将多视角图片按照预设顺序输入至像差预测模型，得到像差预测结果，包括：利用多头注意力机制对每个视角的像差进行预测，其中，像差预测模型学习像差的有关局部特征，得到低于第二预设分辨率的像差。Optionally, in an embodiment of the present application, based on the perspective characteristics of each perspective, the multi-view pictures are input into the disparity prediction model in a preset order, and the disparity prediction result is obtained, including: using a multi-head attention mechanism to The disparity of each viewing angle is predicted, wherein the disparity prediction model learns the relevant local features of the disparity, and obtains the disparity lower than the second preset resolution.
在实际执行过程中，本申请实施例可以采用多头注意力机制对每个视角的像差进行预测，并学习其像差的有关局部特征，从而得到低于第二预设分辨率的像差，本申请实施例通过像差预测模型transformer，处理多视角图片的相关性，并有效利用注意力机制学习，从而更加准确有效的预测像差的梯度，减少了迭代过程，极大的缩短反卷积过程的耗时，且不存在病态问题，鲁棒性较高，提高图像重建的效率。In the actual implementation process, the embodiment of the present application can use the multi-head attention mechanism to predict the aberration of each viewing angle, and learn the relevant local features of the aberration, so as to obtain the aberration lower than the second preset resolution, The embodiment of the present application processes the correlation of multi-view images through the aberration prediction model transformer, and effectively uses the attention mechanism to learn, so as to predict the gradient of the aberration more accurately and effectively, reduce the iterative process, and greatly shorten the deconvolution The process is time-consuming, there is no ill-conditioned problem, the robustness is high, and the efficiency of image reconstruction is improved.
需要注意的是，第二预设分辨率可以由本领域技术人员根据实际情况进行设置，在此不做具体限制。It should be noted that, the second preset resolution can be set by those skilled in the art according to the actual situation, which is not specifically limited here.
在步骤S104中，根据像差预测结果获取高于第一预设辨率的像差，并根据像差得到图像重建图片。In step S104, an aberration higher than the first preset resolution is obtained according to the aberration prediction result, and an image reconstruction picture is obtained according to the aberration.
具体地，本申请实施例可以根据像差预测结果获取高于第一预设辨率的像差，利用理想的点扩散函数进行计算，并对多视角图片进行反卷积，进而重建出原图。本申请实施例通过像差预测模型transformer，处理多视角图片的相关性，并有效利用注意力机制学习，从而更加准确有效的预测像差的梯度，减少了迭代过程，极大的缩短反卷积过程的耗时，且不存在病态问题，鲁棒性较高。Specifically, in this embodiment of the present application, an aberration higher than the first preset resolution can be obtained according to the aberration prediction result, an ideal point spread function can be used for calculation, and the multi-view image can be deconvolved to reconstruct the original image. . The embodiment of the present application processes the correlation of multi-view images through the aberration prediction model transformer, and effectively uses the attention mechanism to learn, so as to predict the gradient of the aberration more accurately and effectively, reduce the iterative process, and greatly shorten the deconvolution The process is time-consuming, there is no ill-conditioned problem, and the robustness is high.
需要注意的是，第一预设分辨率可以由本领域技术人员根据实际情况进行设置，在此不做具体限制。It should be noted that, the first preset resolution can be set by those skilled in the art according to the actual situation, which is not specifically limited here.
可选地，在本申请的一个实施例中，根据像差预测结果获取高于预设辨率的像差，包括：基于像差预测结果，利用泽尼克多项式的拟合，以生成高于预设辨率的像差。Optionally, in an embodiment of the present application, obtaining an aberration higher than a preset resolution according to the aberration prediction result includes: based on the aberration prediction result, using Zernike polynomial fitting to generate a higher resolution than the prediction result. Set the aberration of the resolution.
进一步地，本申请实施例可以利用低分辨率的像差进行泽尼克多项式的拟合，以生成高于预设辨率的像差。本申请实施例利用泽尼克多项式进行图像重建，可以涉及大部分镜头的像差，从而有效提高图像重建的效率。Further, in this embodiment of the present application, a low-resolution aberration can be used to perform Zernike polynomial fitting, so as to generate an aberration higher than a preset resolution. The embodiment of the present application uses the Zernike polynomial to perform image reconstruction, which can involve most of the lens aberrations, thereby effectively improving the efficiency of image reconstruction.
可选地，在本申请的一个实施例中，根据像差得到图像重建图片，包括：根据像差利用预设点扩散函数计算实际点扩散函数，并和多视角图片进行反卷积，得到图像重建图片。Optionally, in an embodiment of the present application, obtaining the image reconstruction picture according to the aberration includes: calculating the actual point spread function by using a preset point spread function according to the aberration, and performing deconvolution with the multi-view picture to obtain the image. Rebuild the picture.
在实际执行过程中，本申请实施例可以根据像差利用预设点扩散函数RL，计算实际点扩散函数，并和多视角图片进行快速反卷积，得到图像重建图片，从而实现较快的部署和运行，有效提高图像重建的效率。In the actual execution process, the embodiment of the present application can use the preset point spread function RL according to the aberration to calculate the actual point spread function, and perform fast deconvolution with the multi-view picture to obtain the image reconstruction picture, thereby realizing faster deployment. and operation, effectively improving the efficiency of image reconstruction.
下面结合图2所示，以一个具体实施例对本申请实施例的镜头的像差预测和图像重建方法进行详细阐述。The aberration prediction and image reconstruction method of the lens according to the embodiment of the present application will be described in detail below with reference to FIG. 2 with a specific embodiment.
如图2所示，本申请实施例包括以下步骤：As shown in Figure 2, the embodiment of the present application includes the following steps:
步骤S201：扫描光场相机的采集数据。在实际执行过程中，本申请实施例可以通过扫描光场相机的数据，获得数据图片。Step S201: Scan the collected data of the light field camera. In the actual execution process, the embodiment of the present application can obtain a data picture by scanning the data of the light field camera.
步骤S202：多视角图片，保存成多帧图片。本申请实施例可以对数据图片进行对齐，从而得到多视角图片，进而保存成多帧图片。Step S202: Multi-view pictures are saved as multi-frame pictures. In this embodiment of the present application, data pictures can be aligned to obtain multi-view pictures, which are then saved as multi-frame pictures.
步骤S203：ResNet101提取多视角图片特征。本申请实施例可以将多个视角的图片，输入到残差神经网络ResNet101中，提取每个视角特征。Step S203: ResNet101 extracts multi-view image features. In this embodiment of the present application, pictures from multiple perspectives can be input into the residual neural network ResNet101 to extract features of each perspective.
步骤S204：Transformer预测相差梯度。作为一种可能实现的方式，本申请实施例可以将多个视角的图片按预设顺序，例如，从上到小从左到右的顺序，输入像差预测模型transformer当中，像差预测模型transformer利用较短的2层2多头的注意力机制预测每张图对应像差的低分辨率的梯度，从而得到像差预测结果。Step S204: Transformer predicts the phase difference gradient. As a possible implementation manner, in this embodiment of the present application, pictures from multiple viewing angles may be input into the disparity prediction model transformer in a preset order, for example, from top to bottom and from left to right, and the disparity prediction model transformer A short 2-layer 2-multiple attention mechanism is used to predict the low-resolution gradient of each image corresponding to the disparity, so as to obtain the disparity prediction result.
需要注意的是，预设顺序可以由本领域技术人员根据实际情况进行设置，在此不做具体限制。It should be noted that the preset sequence may be set by those skilled in the art according to actual conditions, which is not specifically limited herein.
步骤S205：相差泽尼克多项式系数。进一步地，本申请实施例可以利用低分辨率的像差进行泽尼克多项式的拟合，以生成高于预设辨率的像差。本申请实施例利用泽尼克多项式进行图像重建，可以涉及大部分镜头的像差，从而有效提高图像重建的效率。Step S205: Difference Zernike polynomial coefficients. Further, in this embodiment of the present application, a low-resolution aberration can be used to perform Zernike polynomial fitting, so as to generate an aberration higher than a preset resolution. The embodiment of the present application uses the Zernike polynomial to perform image reconstruction, which can involve most of the lens aberrations, thereby effectively improving the efficiency of image reconstruction.
步骤S206：实际的点扩散函数。在实际执行过程中，本申请实施例可以根据像差利用预设点扩散函数RL，计算实际点扩散函数，并和多视角图片进行快速反卷积，得到图像重建图片，从而实现较快的部署和运行，有效提高图像重建的效率。Step S206: the actual point spread function. In the actual execution process, the embodiment of the present application can use the preset point spread function RL according to the aberration to calculate the actual point spread function, and perform fast deconvolution with the multi-view picture to obtain the image reconstruction picture, thereby realizing faster deployment. and operation, effectively improving the efficiency of image reconstruction.
步骤S207：计算实际图片。本申请实施例可以通过多视角图片反卷积计算并重建图像，也可以通过实际点扩散函数计算并重建图像。Step S207: Calculate the actual picture. In this embodiment of the present application, the image can be calculated and reconstructed through multi-view image deconvolution, and the image can also be calculated and reconstructed through an actual point spread function.
根据本申请实施例提出的镜头的像差预测和图像重建方法，可以利用扫描光场相机获取多视角图片，并提取每个视角的视角特征，利用像差预测模型获取像差预测结果，进而根据像差获得图像重建照片，可以在保证视差估计精度的同时，实现高速无迭代、内存负担小的图片重建，性能稳定，鲁棒性高且可以并行。由此，解决了相关技术基于物理手段或简单的神经网络，传播过程复杂，鲁棒性较差且耗时耗能巨大，从而导致重建效果较差的技术问题。According to the method for aberration prediction and image reconstruction of a lens proposed in the embodiment of the present application, a scanning light field camera can be used to obtain multi-view images, and the feature of each view angle can be extracted, and the aberration prediction result can be obtained by using an aberration prediction model, and then according to Image reconstruction photos obtained by disparity can achieve high-speed image reconstruction without iteration and low memory burden while ensuring the accuracy of disparity estimation, with stable performance, high robustness and parallelization. Thereby, the technical problems that the related technology is based on physical means or simple neural network, the propagation process is complex, the robustness is poor, and the time-consuming and energy-consuming are huge, resulting in poor reconstruction effect are solved.
其次参照附图描述根据本申请实施例提出的镜头的像差预测和图像重建装置。Next, the aberration prediction and image reconstruction device for a lens according to the embodiments of the present application will be described with reference to the accompanying drawings.
图3是本申请实施例的镜头的像差预测和图像重建装置的方框示意图。FIG. 3 is a schematic block diagram of a lens aberration prediction and image reconstruction apparatus according to an embodiment of the present application.
如图3所示，该镜头的像差预测和图像重建装置10包括：扫描模块100、提取模块200、像差预测模块300和重建模块400。As shown in FIG. 3 , the lens aberration prediction and
具体地，扫描模块100，用于扫描光场相机的采集数据，得到多视角图片。Specifically, the
提取模块200，用于从多视角图片中提取每个视角的视角特征。The
像差预测模块300，用于基于每个视角的视角特征，将多视角图片按照预设顺序输入至像差预测模型，得到像差预测结果。The
重建模块400，用于根据像差预测结果获取高于第一预设辨率的像差，并根据像差得到图像重建图片。The
可选地，在本申请的一个实施例中，重建模块400包括：拟合单元。Optionally, in an embodiment of the present application, the
其中，拟合单元，用于基于像差预测结果，利用泽尼克多项式的拟合，以生成高于预设辨率的像差。Wherein, the fitting unit is configured to use Zernike polynomial fitting based on the aberration prediction result to generate aberrations higher than a preset resolution.
可选地，在本申请的一个实施例中，重建模块400包括：计算单元。Optionally, in an embodiment of the present application, the
其中，计算单元，用于根据像差利用预设点扩散函数计算实际点扩散函数，并和多视角图片进行反卷积，得到图像重建图片。The calculation unit is used to calculate the actual point spread function by using the preset point spread function according to the aberration, and perform deconvolution with the multi-view picture to obtain the image reconstruction picture.
可选地，在本申请的一个实施例中，像差预测模块300包括：预测单元。Optionally, in an embodiment of the present application, the
其中，预测单元，用于利用多头注意力机制对每个视角的像差进行预测，其中，像差预测模型学习像差的有关局部特征，得到低于第二预设分辨率的像差。The prediction unit is used to predict the disparity of each viewing angle by using the multi-head attention mechanism, wherein the disparity prediction model learns the relevant local features of the disparity, and obtains the disparity lower than the second preset resolution.
需要说明的是，前述对镜头的像差预测和图像重建方法实施例的解释说明也适用于该实施例的镜头的像差预测和图像重建装置，此处不再赘述。It should be noted that the foregoing explanations of the embodiments of the lens aberration prediction and image reconstruction method are also applicable to the lens aberration prediction and image reconstruction apparatus of this embodiment, and are not repeated here.
根据本申请实施例提出的镜头的像差预测和图像重建装置，可以利用扫描光场相机获取多视角图片，并提取每个视角的视角特征，利用像差预测模型获取像差预测结果，进而根据像差获得图像重建照片，可以在保证视差估计精度的同时，实现高速无迭代、内存负担小的图片重建，性能稳定，鲁棒性高且可以并行。由此，解决了相关技术基于物理手段或简单的神经网络，传播过程复杂，鲁棒性较差且耗时耗能巨大，从而导致重建效果较差的技术问题。According to the lens aberration prediction and image reconstruction device proposed in the embodiment of the present application, a scanning light field camera can be used to obtain a multi-view image, and the viewing angle feature of each view can be extracted, and the aberration prediction result can be obtained by using the aberration prediction model, and then according to the Image reconstruction photos obtained by disparity can achieve high-speed image reconstruction without iteration and low memory burden while ensuring the accuracy of disparity estimation, with stable performance, high robustness and parallelization. Thereby, the technical problems that the related technology is based on physical means or simple neural network, the propagation process is complex, the robustness is poor, and the time-consuming and energy-consuming are huge, resulting in poor reconstruction effect are solved.
图4为本申请实施例提供的电子设备的结构示意图。该电子设备可以包括：FIG. 4 is a schematic structural diagram of an electronic device provided by an embodiment of the present application. The electronic device may include:
存储器401、处理器402及存储在存储器401上并可在处理器402上运行的计算机程序。
处理器402执行程序时实现上述实施例中提供的镜头的像差预测和图像重建方法。When the
进一步地，电子设备还包括：Further, the electronic device also includes:
通信接口403，用于存储器401和处理器402之间的通信。The
存储器401，用于存放可在处理器402上运行的计算机程序。The
存储器401可能包含高速RAM存储器，也可能还包括非易失性存储器（non-volatile memory），例如至少一个磁盘存储器。The
如果存储器401、处理器402和通信接口403独立实现，则通信接口403、存储器401和处理器402可以通过总线相互连接并完成相互间的通信。总线可以是工业标准体系结构（Industry Standard Architecture，简称为ISA）总线、外部设备互连（PeripheralComponent，简称为PCI）总线或扩展工业标准体系结构（Extended Industry StandardArchitecture，简称为EISA）总线等。总线可以分为地址总线、数据总线、控制总线等。为便于表示，图4中仅用一条粗线表示，但并不表示仅有一根总线或一种类型的总线。If the
可选地，在具体实现上，如果存储器401、处理器402及通信接口403，集成在一块芯片上实现，则存储器401、处理器402及通信接口403可以通过内部接口完成相互间的通信。Optionally, in terms of specific implementation, if the
处理器402可能是一个中央处理器（Central Processing Unit，简称为CPU），或者是特定集成电路（Application Specific Integrated Circuit，简称为ASIC），或者是被配置成实施本申请实施例的一个或多个集成电路。The
本实施例还提供一种计算机可读存储介质，其上存储有计算机程序，该程序被处理器执行时实现如上的镜头的像差预测和图像重建方法。This embodiment also provides a computer-readable storage medium on which a computer program is stored, and when the program is executed by a processor, implements the above method for predicting aberration of a lens and reconstructing an image.
在本说明书的描述中，参考术语“一个实施例”、“一些实施例”、“示例”、“具体示例”、或“一些示例”等的描述意指结合该实施例或示例描述的具体特征、结构、材料或者特点包含于本申请的至少一个实施例或示例中。在本说明书中，对上述术语的示意性表述不必须针对的是相同的实施例或示例。而且，描述的具体特征、结构、材料或者特点可以在任一个或N个实施例或示例中以合适的方式结合。此外，在不相互矛盾的情况下，本领域的技术人员可以将本说明书中描述的不同实施例或示例以及不同实施例或示例的特征进行结合和组合。In the description of this specification, description with reference to the terms "one embodiment," "some embodiments," "example," "specific example," or "some examples", etc., mean specific features described in connection with the embodiment or example , structure, material or feature is included in at least one embodiment or example of the present application. In this specification, schematic representations of the above terms are not necessarily directed to the same embodiment or example. Furthermore, the particular features, structures, materials or characteristics described may be combined in any suitable manner in any one or N of the embodiments or examples. Furthermore, those skilled in the art may combine and combine the different embodiments or examples described in this specification, as well as the features of the different embodiments or examples, without conflicting each other.
此外，术语“第一”、“第二”仅用于描述目的，而不能理解为指示或暗示相对重要性或者隐含指明所指示的技术特征的数量。由此，限定有“第一”、“第二”的特征可以明示或者隐含地包括至少一个该特征。在本申请的描述中，“N个”的含义是至少两个，例如两个，三个等，除非另有明确具体的限定。In addition, the terms "first" and "second" are only used for descriptive purposes, and should not be construed as indicating or implying relative importance or implying the number of indicated technical features. Thus, a feature delimited with "first", "second" may expressly or implicitly include at least one of that feature. In the description of the present application, "N" means at least two, such as two, three, etc., unless otherwise expressly and specifically defined.
流程图中或在此以其他方式描述的任何过程或方法描述可以被理解为，表示包括一个或更N个用于实现定制逻辑功能或过程的步骤的可执行指令的代码的模块、片段或部分，并且本申请的优选实施方式的范围包括另外的实现，其中可以不按所示出或讨论的顺序，包括根据所涉及的功能按基本同时的方式或按相反的顺序，来执行功能，这应被本申请的实施例所属技术领域的技术人员所理解。Any process or method description in the flowchart or otherwise described herein may be understood to represent a module, segment or portion of code comprising one or N more executable instructions for implementing custom logical functions or steps of the process , and the scope of the preferred embodiments of the present application includes alternative implementations in which the functions may be performed out of the order shown or discussed, including performing the functions substantially concurrently or in the reverse order depending upon the functions involved, which should It is understood by those skilled in the art to which the embodiments of the present application belong.
在流程图中表示或在此以其他方式描述的逻辑和/或步骤，例如，可以被认为是用于实现逻辑功能的可执行指令的定序列表，可以具体实现在任何计算机可读介质中，以供指令执行系统、装置或设备（如基于计算机的系统、包括处理器的系统或其他可以从指令执行系统、装置或设备取指令并执行指令的系统）使用，或结合这些指令执行系统、装置或设备而使用。就本说明书而言，"计算机可读介质"可以是任何可以包含、存储、通信、传播或传输程序以供指令执行系统、装置或设备或结合这些指令执行系统、装置或设备而使用的装置。计算机可读介质的更具体的示例（非穷尽性列表）包括以下：具有一个或N个布线的电连接部（电子装置），便携式计算机盘盒（磁装置），随机存取存储器（RAM），只读存储器（ROM），可擦除可编辑只读存储器（EPROM或闪速存储器），光纤装置，以及便携式光盘只读存储器（CDROM）。另外，计算机可读介质甚至可以是可在其上打印所述程序的纸或其他合适的介质，因为可以例如通过对纸或其他介质进行光学扫描，接着进行编辑、解译或必要时以其他合适方式进行处理来以电子方式获得所述程序，然后将其存储在计算机存储器中。The logic and/or steps represented in flowcharts or otherwise described herein, for example, may be considered an ordered listing of executable instructions for implementing the logical functions, may be embodied in any computer-readable medium, For use with, or in conjunction with, an instruction execution system, apparatus, or device (such as a computer-based system, a system including a processor, or other system that can fetch instructions from and execute instructions from an instruction execution system, apparatus, or apparatus) or equipment. For the purposes of this specification, a "computer-readable medium" can be any device that can contain, store, communicate, propagate, or transport the program for use by or in conjunction with an instruction execution system, apparatus, or apparatus. More specific examples (non-exhaustive list) of computer readable media include the following: electrical connections (electronic devices) with one or N wires, portable computer disk cartridges (magnetic devices), random access memory (RAM), Read Only Memory (ROM), Erasable Editable Read Only Memory (EPROM or Flash Memory), Fiber Optic Devices, and Portable Compact Disc Read Only Memory (CDROM). In addition, the computer readable medium may even be paper or other suitable medium on which the program may be printed, as the paper or other medium may be optically scanned, for example, followed by editing, interpretation, or other suitable medium as necessary process to obtain the program electronically and then store it in computer memory.
应当理解，本申请的各部分可以用硬件、软件、固件或它们的组合来实现。在上述实施方式中，N个步骤或方法可以用存储在存储器中且由合适的指令执行系统执行的软件或固件来实现。如，如果用硬件来实现和在另一实施方式中一样，可用本领域公知的下列技术中的任一项或他们的组合来实现：具有用于对数据信号实现逻辑功能的逻辑门电路的离散逻辑电路，具有合适的组合逻辑门电路的专用集成电路，可编程门阵列（PGA），现场可编程门阵列（FPGA）等。It should be understood that various parts of this application may be implemented in hardware, software, firmware, or a combination thereof. In the above-described embodiments, the N steps or methods may be implemented in software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware as in another embodiment, it can be implemented by any one of the following techniques known in the art, or a combination thereof: discrete with logic gates for implementing logic functions on data signals Logic circuits, ASICs with suitable combinational logic gates, Programmable Gate Arrays (PGA), Field Programmable Gate Arrays (FPGA), etc.
本技术领域的普通技术人员可以理解实现上述实施例方法携带的全部或部分步骤是可以通过程序来指令相关的硬件完成，所述的程序可以存储于一种计算机可读存储介质中，该程序在执行时，包括方法实施例的步骤之一或其组合。Those of ordinary skill in the art can understand that all or part of the steps carried by the methods of the above embodiments can be completed by instructing the relevant hardware through a program, and the program can be stored in a computer-readable storage medium, and the program is stored in a computer-readable storage medium. When executed, one or a combination of the steps of the method embodiment is included.
此外，在本申请各个实施例中的各功能单元可以集成在一个处理模块中，也可以是各个单元单独物理存在，也可以两个或两个以上单元集成在一个模块中。上述集成的模块既可以采用硬件的形式实现，也可以采用软件功能模块的形式实现。所述集成的模块如果以软件功能模块的形式实现并作为独立的产品销售或使用时，也可以存储在一个计算机可读取存储介质中。In addition, each functional unit in each embodiment of the present application may be integrated into one processing module, or each unit may exist physically alone, or two or more units may be integrated into one module. The above-mentioned integrated modules can be implemented in the form of hardware, and can also be implemented in the form of software function modules. If the integrated modules are implemented in the form of software functional modules and sold or used as independent products, they may also be stored in a computer-readable storage medium.
上述提到的存储介质可以是只读存储器，磁盘或光盘等。尽管上面已经示出和描述了本申请的实施例，可以理解的是，上述实施例是示例性的，不能理解为对本申请的限制，本领域的普通技术人员在本申请的范围内可以对上述实施例进行变化、修改、替换和变型。The above-mentioned storage medium may be a read-only memory, a magnetic disk or an optical disk, and the like. Although the embodiments of the present application have been shown and described above, it should be understood that the above embodiments are exemplary and should not be construed as limitations to the present application. Embodiments are subject to variations, modifications, substitutions and variations.
Claims (10)
Translated fromChinesePriority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210394888.6ACN114494258B (en) | 2022-04-15 | 2022-04-15 | Lens aberration prediction and image reconstruction method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210394888.6ACN114494258B (en) | 2022-04-15 | 2022-04-15 | Lens aberration prediction and image reconstruction method and device |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN114494258Atrue CN114494258A (en) | 2022-05-13 |
| CN114494258B CN114494258B (en) | 2022-08-30 |
Family
ID=81489631
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210394888.6AActiveCN114494258B (en) | 2022-04-15 | 2022-04-15 | Lens aberration prediction and image reconstruction method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114494258B (en) |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115205459A (en)* | 2022-07-14 | 2022-10-18 | 中国科学院苏州生物医学工程技术研究所 | Medical image processing and displaying method and system |
| WO2023231138A1 (en)* | 2022-05-30 | 2023-12-07 | 元潼(北京)技术有限公司 | Multi-angle-of-view image super-resolution reconstruction method and apparatus based on meta-imaging |
| CN117541519A (en)* | 2024-01-09 | 2024-02-09 | 清华大学 | Light field aberration correction method, device, electronic equipment and storage medium |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20030103189A1 (en)* | 2001-09-11 | 2003-06-05 | The Regents Of The University Of California | Characterizing aberrations in an imaging lens and applications to visual testing and integrated circuit mask analysis |
| US20150178923A1 (en)* | 2013-12-24 | 2015-06-25 | Lytro, Inc. | Light-field aberration correction |
| CN109640830A (en)* | 2016-07-14 | 2019-04-16 | 医视特有限公司 | Focus ultrasonic based on precedent |
| CN110675451A (en)* | 2019-09-17 | 2020-01-10 | 北京超放信息技术有限公司 | Digital self-adaptive correction method and system based on phase space optics |
| CN111915526A (en)* | 2020-08-05 | 2020-11-10 | 湖北工业大学 | Photographing method based on brightness attention mechanism low-illumination image enhancement algorithm |
| CN113256741A (en)* | 2021-07-14 | 2021-08-13 | 清华大学 | Lens calibration method and device for scanning light field imaging system |
- 2022
- 2022-04-15CNCN202210394888.6Apatent/CN114494258B/enactiveActive
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20030103189A1 (en)* | 2001-09-11 | 2003-06-05 | The Regents Of The University Of California | Characterizing aberrations in an imaging lens and applications to visual testing and integrated circuit mask analysis |
| US20150178923A1 (en)* | 2013-12-24 | 2015-06-25 | Lytro, Inc. | Light-field aberration correction |
| CN109640830A (en)* | 2016-07-14 | 2019-04-16 | 医视特有限公司 | Focus ultrasonic based on precedent |
| CN110675451A (en)* | 2019-09-17 | 2020-01-10 | 北京超放信息技术有限公司 | Digital self-adaptive correction method and system based on phase space optics |
| CN111915526A (en)* | 2020-08-05 | 2020-11-10 | 湖北工业大学 | Photographing method based on brightness attention mechanism low-illumination image enhancement algorithm |
| CN113256741A (en)* | 2021-07-14 | 2021-08-13 | 清华大学 | Lens calibration method and device for scanning light field imaging system |
Non-Patent Citations (2)
| Title |
|---|
| 吴志云等: "用泽尼克多项式消除三维轮廓术中的条纹载频", 《光学学报》* |
| 潘德馨: "光学像差的图像校正算法", 《中国优秀博硕士学位论文全文数据库(硕士)信息科技辑》* |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2023231138A1 (en)* | 2022-05-30 | 2023-12-07 | 元潼(北京)技术有限公司 | Multi-angle-of-view image super-resolution reconstruction method and apparatus based on meta-imaging |
| CN115205459A (en)* | 2022-07-14 | 2022-10-18 | 中国科学院苏州生物医学工程技术研究所 | Medical image processing and displaying method and system |
| CN117541519A (en)* | 2024-01-09 | 2024-02-09 | 清华大学 | Light field aberration correction method, device, electronic equipment and storage medium |
| CN117541519B (en)* | 2024-01-09 | 2024-05-31 | 清华大学 | Optical field aberration correction method, optical field aberration correction device, electronic equipment and storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN114494258B (en) | 2022-08-30 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN114494258B (en) | Lens aberration prediction and image reconstruction method and device | |
| Pang et al. | Cascade residual learning: A two-stage convolutional neural network for stereo matching | |
| US11663691B2 (en) | Method and apparatus for restoring image | |
| US10326981B2 (en) | Generating 3D images using multi-resolution camera set | |
| US10268901B2 (en) | Quasi-parametric optical flow estimation | |
| CN112396645A (en) | Monocular image depth estimation method and system based on convolution residual learning | |
| CN110443874B (en) | Viewpoint data generation method and device based on convolutional neural network | |
| CN111327887A (en) | Electronic device, method of operating the same, and method of processing image of the electronic device | |
| KR102319643B1 (en) | Method for processing microscopy image using artificial neural network with point spread function layer and apparatus therefore | |
| CN114511609A (en) | Unsupervised light field disparity estimation system and method based on occlusion perception | |
| Jin et al. | Light field super-resolution via attention-guided fusion of hybrid lenses | |
| EP4016447A1 (en) | Method and apparatus with image processing | |
| CN115035235A (en) | Three-dimensional reconstruction method and device | |
| CN117274514A (en) | Remote sensing image generation method and device based on ground-air visual angle geometric transformation | |
| CN116611501A (en) | Model training method, model application method, model training device and training equipment | |
| WO2017096384A1 (en) | Quasi-parametric optical flow estimation | |
| Wijayasingha et al. | Camera-Independent Single Image Depth Estimation from Defocus Blur | |
| KR102680384B1 (en) | Method and apparatus for correction of aberration | |
| Wu et al. | Sparse spn: Depth completion from sparse keypoints | |
| CN118015179A (en) | Three-dimensional light field display method, device, equipment and medium based on camera in-loop | |
| CN117197256A (en) | Calibration method, device and electronic equipment | |
| TWI417810B (en) | Image enhancement method, image enhancement device and image processing circuit | |
| CN115205112A (en) | A model training method and device for super-resolution of real complex scene images | |
| CN115965647A (en) | Background image generation method, image fusion method, device, electronic equipment and readable medium | |
| Van der Walt | Super-resolution imaging |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |