Movatterモバイル変換

[0]ホーム
URL:

Skip to main content

Advertisement

Springer Nature Link
Log in

Blind Image Deblurring with Unknown Kernel Size and Substantial Noise

  • Published:
International Journal of Computer Vision Aims and scope Submit manuscript

Abstract

Blind image deblurring (BID) has been extensively studied in computer vision and adjacent fields. Modern methods for BID can be grouped into two categories: single-instance methods that deal with individual instances using statistical inference and numerical optimization, and data-driven methods that train deep-learning models to deblur future instances directly. Data-driven methods can be free from the difficulty in deriving accurate blur models, but are fundamentally limited by the diversity and quality of the training data—collecting sufficiently expressive and realistic training data is a standing challenge. In this paper, we focus on single-instance methods that remain competitive and indispensable. However, most such methods do not prescribe how to deal with unknown kernel size and substantial noise, precluding practical deployment. Indeed, we show that several state-of-the-art (SOTA) single-instance methods are unstable when the kernel size is overspecified, and/or the noise level is high. On the positive side, we propose a practical BID method that is stable against both, the first of its kind. Our method builds on the recent ideas of solving inverse problems by integrating physical models and structured deep neural networks, without extra training data. We introduce several crucial modifications to achieve the desired stability. Extensive empirical tests on standard synthetic datasets, as well as real-worldNTIRE2020 andRealBlur datasets, show the superior effectiveness and practicality of our BID method compared to SOTA single-instance as well as data-driven methods. The code of our method is available athttps://github.com/sun-umn/Blind-Image-Deblurring.

This is a preview of subscription content,log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
¥17,985 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price includes VAT (Japan)

Instant access to the full article PDF.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17
Algorithm 1
Fig. 18
Fig. 19
Fig. 20
Fig. 21
Fig. 22
Fig. 23
Fig. 24
Fig. 25
Fig. 26
Fig. 27
Fig. 28
Fig. 29
Fig. 30
Fig. 31
Fig. 32
Fig. 33
Fig. 34
Fig. 35

Similar content being viewed by others

Explore related subjects

Discover the latest articles, news and stories from top researchers in related subjects.

Data Availability

Part of the code and datasets used during the current study, necessary to interpret, replicate and build upon the findings reported in the article, are available in the Github repositoryhttps://github.com/sun-umn/Blind-Image-Deblurring

Notes

  1. Indeed, by Young’s convolution inequality and the fact\({\Vert {\varvec{k}} \Vert _1 = 1}\),.

  2. See also similar ideas for the inverse filtering approach in Cabrelli (1985) and Sun and Donoho (2021).

  3. In particular, if they form a measure-zero set.

  4. The result in Eq. (11) assumes a circular convolution model:\(\varvec{y} = \varvec{a} \circledast \varvec{z}\), but it is well known that the linear convolution can be written as circular convolution by appropriate zero-padding to the two convolving components.

  5. We note in passing that the reason we do not use FBC directly is that it may be misleading: the correspondence ratio as they define it can be larger than 1, so in principle the average approaching 1 does not imply that recovery is good. When checking their code (https://github.com/shizenglin/Measure-and-Control-Spectral-Bias), we find that they actually truncate values greater than 1, which potentially make the metric more misleading.

  6. The existing synthetic BID datasets are too small to support training data-driven methods.

  9. LAI16 has 4 trajectories to synthesize non-uniform motion blur also, which we do not consider in this paper. Moreover, it also includes 100 real-world blurry images without groundtruth kernels.

  10. Available at (registration needed to download the dataset):https://competitions.codalab.org/competitions/22233#learn_the_details. We suspect that this is a superset of the REDS (REalistic and Dynamic Scenes) dataset (available athttps://seungjunnah.github.io/Datasets/reds.html), at least with the same generation procedure as that of REDS.

  12. NTIRE2020 is developed for data-driven approaches that require an extensive training set.

  18. InDONG17 the loss consists in applying\(h(z) = z^2/2 - \log {(a+e^{bz^2})}/(2b)\) element-wise to\(\varvec{y} - {\varvec{k}} *{\varvec{x}}\), where\(a, b > 0\) and so that\(h(z) \le 0\). Note that\(h(z) \sim O(z^2)\) as\(z \rightarrow 0\), andh(z) approaches the constant 0 whenz is large.

References

  • Ahmed, A., Recht, B., & Romberg, J. (2014). Blind deconvolution using convex programming.IEEE Transactions on Information Theory,60(3), 1711–1732.https://doi.org/10.1109/tit.2013.2294644

    Article MathSciNet  Google Scholar 

  • Aljadaany, R., Pal, D. K., & Savvides, M. (2019). Douglas-rachford networks: Learning both the image prior and data fidelity terms for blind image deconvolution. In: 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). IEEE.https://doi.org/10.1109/cvpr.2019.01048

  • Asim, M., Shamshad, F., & Ahmed, A. (2020). Blind image deconvolution using deep generative priors.IEEE Transactions on Computational Imaging,6, 1493–1506.https://doi.org/10.1109/tci.2020.3032671

    Article MathSciNet  Google Scholar 

  • Benichoux, A., Vincent, E., & Gribonval, R. (2013). A fundamental pitfall in blind deconvolution with sparse and shift-invariant priors. In: IEEE International Conference on Acoustics, Speech and Signal Processing. IEEE .https://doi.org/10.1109/icassp.2013.6638838

  • Bostan, E., Heckel, R., Chen, M., Kellman, M., & Waller, L. (2020). Deep phase decoder: Self-calibrating phase microscopy with an untrained deep neural network.Optica,7(6), 559–562.

    Article  Google Scholar 

  • Cabrelli, C. A. (1985). Minimum entropy deconvolution and simplicity: A noniterative algorithm.Geophysics,50(3), 394–413.https://doi.org/10.1190/1.1441919

    Article  Google Scholar 

  • Chan, T., & Wong, C. K. (1998). Total variation blind deconvolution.IEEE Transactions on Image Processing,7(3), 370–375.https://doi.org/10.1109/83.661187

    Article  Google Scholar 

  • Chen, L., Fang, F., Wang, T., Zhang, G.(2019). Blind image deblurring with local maximum gradient prior. In: IEEE Conference on computer vision and pattern recognition (CVPR). IEEE .https://doi.org/10.1109/cvpr.2019.00184

  • Chen, L., Fang, F., Zhang, J., Liu, J., & Zhang, G. (2020). OID: Outlier identifying and discarding in blind image deblurring. In: European Conference on Computer Vision (ECCV), pp. 598–613. Springer International Publishing.https://doi.org/10.1007/978-3-030-58595-2_36

  • Chen, L., Zhang, J., Lin, S., Fang, F., & Ren, J. S. (2021). Blind deblurring for saturated images. In: IEEE Conference on computer vision and pattern recognition (CVPR). IEEE .https://doi.org/10.1109/cvpr46437.2021.00624

  • Cheung, S. C., Shin, J. Y., Lau, Y., Chen, Z., Sun, J., Zhang, Y., Müller, M. A., Eremin, I. M., Wright, J. N., & Pasupathy, A. N.(2020). Dictionary learning in fourier-transform scanning tunneling spectroscopy. Nature Communications11(1):1081.https://doi.org/10.1038/s41467-020-14633-1

  • Chi, Y. (2016). Guaranteed blind sparse spikes deconvolution via lifting and convex optimization.IEEE Journal of Selected Topics in Signal Processing,10(4), 782–794.https://doi.org/10.1109/jstsp.2016.2543462

    Article  Google Scholar 

  • Cho, S., & Lee, S. (2009). Fast motion deblurring. In: ACM Trans. Graph. ACM Press .https://doi.org/10.1145/1661412.1618491

  • Cho, S., Lee, S.(2017). Convergence analysis of MAP based blur kernel estimation. In: IEEE International Conference on Computer Vision (ICCV). IEEEhttps://doi.org/10.1109/iccv.2017.515

  • Choudhary, S., & Mitra, U. (2014). Sparse blind deconvolution: What cannot be done. InIEEE international symposium on information theory. IEEE .https://doi.org/10.1109/isit.2014.6875385

  • Choudhary, S., & Mitra, U. (2018). On the properties of the rank-two null space of nonsparse and canonical-sparse blind deconvolution.IEEE Transactions on Signal Processing,66(14), 3696–3709.https://doi.org/10.1109/tsp.2018.2815014

    Article MathSciNet  Google Scholar 

  • Darestani, M. Z., & Heckel, R. (2021). Accelerated MRI with un-trained neural networks.IEEE Transactions on Computational Imaging,7, 724–733.https://doi.org/10.1109/tci.2021.3097596

    Article  Google Scholar 

  • Ding, Z., & Luo, Z. Q. (2000). A fast linear programming algorithm for blind equalization.IEEE Transactions on Communications,48(9), 1432–1436.https://doi.org/10.1109/26.870004

    Article  Google Scholar 

  • Dong, J., Pan, J., Su, Z., & Yang, M. H.(2017). Blind image deblurring with outlier handling. InIEEE international conference on computer vision (ICCV). IEEE .https://doi.org/10.1109/iccv.2017.271

  • Donoho, D. (1981). ON minimum entropy deconvolution. InApplied time series analysis II, pp. 565–608. Elsevier .https://doi.org/10.1016/b978-0-12-256420-8.50024-1

  • Ekanadham, C., Tranchina, D., Simoncelli, E.(2011). A blind sparse deconvolution method for neural spike identification. InAdvances in neural information processing systems

  • Fang, L., Liu, H., Wu, F., Sun, X., & Li, H. (2014). Separable kernel for image deblurring. InIEEE conference on computer vision and pattern recognition (CVPR). IEEE .https://doi.org/10.1109/cvpr.2014.369

  • Gandelsman, Y., Shocher, A., & Irani, M. (2019). “double-DIP”: Unsupervised image decomposition via coupled deep-image-priors. InIEEE conference on computer vision and pattern recognition (CVPR). IEEE (2019).https://doi.org/10.1109/cvpr.2019.01128

  • Gong, D., Tan, M., Zhang, Y., van den Hengel, A., & Shi, Q. (2017). Self-paced kernel estimation for robust blind image deblurring. InIEEE International conference on computer vision (ICCV). IEEE .https://doi.org/10.1109/iccv.2017.184

  • Gong, D., Tan, M., Zhang, Y., Hengel, A. V. D., & Shi, Q. (2016). Blind image deconvolution by automatic gradient activation. InIEEE Conference on computer vision and pattern recognition (CVPR). IEEE .https://doi.org/10.1109/cvpr.2016.202

  • Heckel, R., Hand, P. (2019). Deep decoder: Concise image representations from untrained non-convolutional networks. InInternational conference on learning representations

  • Heckel, R., & Soltanolkotabi, M. (2019). Denoising and regularization via exploiting the structural bias of convolutional generators. arXiv preprintarXiv:1910.14634

  • Heckel, R., & Soltanolkotabi, M. (2020). Compressive sensing with un-trained neural networks: Gradient descent finds the smoothest approximation.arXiv:2005.03991

  • Hendrycks, D., & Dietterich, T. (2019). Benchmarking neural network robustness to common corruptions and perturbations. InInternational conference on learning representations.https://openreview.net/forum?id=HJz6tiCqYm

  • Huber, P. J. (1964). Robust estimation of a location parameter.The Annals of Mathematical Statistics,35(1), 73–101.https://doi.org/10.1214/aoms/1177703732

    Article MathSciNet  Google Scholar 

  • Hurley, N., & Rickard, S. (2009). Comparing measures of sparsity.IEEE Transactions on Information Theory,55(10), 4723–4741.https://doi.org/10.1109/tit.2009.2027527

    Article MathSciNet  Google Scholar 

  • Jin, M., Roth, S., & Favaro, P. (2018). Normalized blind deconvolution. InEuropean conference on computer vision (ECCV), pp. 694–711. Springer International Publishing .https://doi.org/10.1007/978-3-030-01234-2_41

  • Joshi, N., Szeliski, R., Kriegman, D.J.(2008). PSF estimation using sharp edge prediction. InIEEE conference on computer vision and pattern recognition (CVPR). IEEE .https://doi.org/10.1109/cvpr.2008.4587834

  • Joshi, N., Zitnick, C.L., Szeliski, R., & Kriegman, D. J. (2009). Image deblurring and denoising using color priors. InIEEE conference on computer vision and pattern recognition (CVPR). IEEE .https://doi.org/10.1109/cvpr.2009.5206802

  • Kech, M., & Krahmer, F. (2017). Optimal injectivity conditions for bilinear inverse problems with applications to identifiability of deconvolution problems.SIAM Journal on Applied Algebra and Geometry,1(1), 20–37.https://doi.org/10.1137/16m1067469

    Article MathSciNet  Google Scholar 

  • Köhler, R., Hirsch, M., Mohler, B., Schölkopf, B., & Harmeling, S.(2012). Recording and playback of camera shake: Benchmarking blind deconvolution with a real-world database. InEuropean conference on computer vision (ECCV), pp. 27–40. Springer Berlin Heidelberg .https://doi.org/10.1007/978-3-642-33786-4_3

  • Koh, J., Lee, J., & Yoon, S. (2021). Single-image deblurring with neural networks: A comparative survey.Computer Vision and Image Understanding,203, 103134.https://doi.org/10.1016/j.cviu.2020.103134

    Article  Google Scholar 

  • Komodakis, N., & Paragios, N. (2013). MRF-based blind image deconvolution. InAsian conference on computer vision (ACCV), pp. 361–374. Springer Berlin Heidelberg .https://doi.org/10.1007/978-3-642-37431-9_28

  • Krishnan, D., & Fergus, R.(2009) Fast image deconvolution using hyper-laplacian priors. InAdvances in Neural Information Processing Systems.https://proceedings.neurips.cc/paper/2009/file/3dd48ab31d016ffcbf3314df2b3cb9ce-Paper.pdf

  • Krishnan, D., Tay, T., & Fergus, R. (2011). Blind deconvolution using a normalized sparsity measure. InIEEE conference on computer vision and pattern recognition (CVPR). IEEE .https://doi.org/10.1109/cvpr.2011.5995521

  • Kundur, D., & Hatzinakos, D. (1996). Blind image deconvolution.IEEE Signal Processing Magazine,13(3), 43–64.https://doi.org/10.1109/79.489268

    Article  Google Scholar 

  • Kuo, H. W., Zhang, Y., Lau, Y., & Wright, J. (2020). Geometry and symmetry in short-and-sparse deconvolution.SIAM Journal on Mathematics of Data Science,2(1), 216–245.https://doi.org/10.1137/19m1237569

    Article MathSciNet  Google Scholar 

  • Kupyn, O., Martyniuk, T., Wu, J., & Wang, Z. (2019). Deblurgan-v2: Deblurring (orders-of-magnitude) faster and better. InProceedings of the IEEE/CVF International Conference on Computer Vision, pp. 8878–8887

  • Lai, W.S., Huang, J.B., Hu, Z., Ahuja, N., Yang, M.H.(2016). A comparative study for single image blind deblurring. InIEEE conference on computer vision and pattern recognition (CVPR). IEEE .https://doi.org/10.1109/cvpr.2016.188

  • Lawrence, H., Bramherzig, D., Li, H., Eickenberg, M., & Gabrié, M. (2020). Phase retrieval with holography and untrained priors: Tackling the challenges of low-photon nanoscale imaging. arXiv preprintarXiv:2012.07386

  • Levin, A., Weiss, Y., Durand, F., & Freeman, W. T. (2011). Understanding blind deconvolution algorithms.IEEE Transactionson Pattern Analysis and Machine Intelligence,33(12), 2354–2367.https://doi.org/10.1109/tpami.2011.148

    Article  Google Scholar 

  • Lewicki, M. S. (1998). A review of methods for spike sorting: The detection and classification of neural action potentials.Network: Computation in Neural Systems,9(4), R53–R78.https://doi.org/10.1088/0954-898x_9_4_001

    Article  Google Scholar 

  • Li, L., Pan, J., Lai, W. S., Gao, C., Sang, N., Yang, M. H. (2018). Learning a discriminative prior for blind image deblurring. In2018 IEEE/CVF conference on computer vision and pattern recognition. IEEE .https://doi.org/10.1109/cvpr.2018.00692

  • Li, T., Wang, H., Zhuang, Z., & Sun, J. (2023). Deep random projector: Accelerated deep image prior. InProceedings of the IEEE/CVF conference on computer vision and pattern recognition (CVPR), pp. 18176–18185.

  • Li, T., Zhuang, Z., Liang, H., Peng, L., Wang, H., & Sun, J. (2021). Self-validation: Early stopping for single-instance deep generative priors. InBritish Machine Vision Conference (BMVC).

  • Li, X., Ling, S., Strohmer, T., & Wei, K. (2019). Rapid, robust, and reliable blind deconvolution via nonconvex optimization.Applied and Computational Harmonic Analysis,47(3), 893–934.https://doi.org/10.1016/j.acha.2018.01.001

    Article MathSciNet  Google Scholar 

  • Li, Y., Lee, K., & Bresler, Y.(2015). A unified framework for identifiability analysis in bilinear inverse problems with applications to subspace and sparsity models.arXiv:1501.06120

  • Li, Y., Lee, K., & Bresler, Y. (2017). Identifiability and stability in blind deconvolution under minimal assumptions.IEEE Transactions on Information Theory,63(7), 4619–4633.https://doi.org/10.1109/tit.2017.2689779

    Article MathSciNet  Google Scholar 

  • Li, Y., Tofighi, M., Geng, J., Monga, V., & Eldar, Y. C. (2019). Deep algorithm unrolling for blind image deblurring.arXiv:1902.03493

  • Liu, Y., Dong, W., Gong, D., Zhang, L., & Shi, Q. (2018). Deblurring natural image using super-gaussian fields. InEuropean conference on computer vision (ECCV), pp. 467–484. Springer International Publishing .https://doi.org/10.1007/978-3-030-01246-5_28

  • Ma, X., Hill, P., & Achim, A. (2021). Unsupervised image fusion using deep image priors.arXiv:2110.09490

  • Michaeli, T., & Irani, M. (2014). Blind deblurring using internal patch recurrence. InEuropean conference on computer vision (ECCV), pp. 783–798. Springer International Publishing .https://doi.org/10.1007/978-3-319-10578-9_51

  • Michelashvili, M., & Wolf, L. (2019). Speech denoising by accumulating per-frequency modeling fluctuations.arXiv:1904.07612

  • Nah, S., Baik, S., Hong, S., Moon, G., Son, S., Timofte, R., & Lee, K. M. (2019). NTIRE 2019 challenge on video deblurring and super-resolution: Dataset and study. In2019 IEEE/CVF conference on computer vision and pattern recognition workshops (CVPRW). IEEE .https://doi.org/10.1109/cvprw.2019.00251

  • Nah, S., Kim, T. H., & Lee, K. M.(2017). Deep multi-scale convolutional neural network for dynamic scene deblurring. In2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). IEEE .https://doi.org/10.1109/cvpr.2017.35

  • Nah, S., Son, S., Lee, S., Timofte, R., & Lee, K. M. (2021). Ntire 2021 challenge on image deblurring.arXiv:2104.14854

  • Nah, S., Son, S., Timofte, R., & Lee, K. M. (2020). NTIRE 2020 challenge on image and video deblurring.arXiv:2005.01244

  • Ongie, G., Jalal, A., Metzler, C. A., Baraniuk, R. G., Dimakis, A. G., & Willett, R. (2020). Deep learning techniques for inverse problems in imaging.IEEE Journal on Selected Areas in Information Theory,1(1), 39–56.https://doi.org/10.1109/jsait.2020.2991563

    Article  Google Scholar 

  • Pan, J., Dong, J., Liu, Y., Zhang, J., Ren, J., Tang, J., Tai, Y. W., & Yang, M. H. (2021). Physics-based generative adversarial models for image restoration and beyond.IEEE Transactions on Pattern Analysis and Machine Intelligence,43(7), 2449–2462.https://doi.org/10.1109/tpami.2020.2969348

    Article  Google Scholar 

  • Pan, J., Hu, Z., Su, Z., Yang, M. H. (2014). Deblurring text images via l0-regularized intensity and gradient prior. InIEEE Conference on computer vision and pattern recognition (CVPR). IEEE .https://doi.org/10.1109/cvpr.2014.371

  • Pan, J., Lin, Z., Su, Z., & Yang, M. H. (2016). Robust kernel estimation with outliers handling for image deblurring. InIEEE conference on computer vision and pattern recognition (CVPR). IEEE .https://doi.org/10.1109/cvpr.2016.306

  • Pan, J., Sun, D., Pfister, H., & Yang, M. H. (2016). Blind image deblurring using dark channel prior. InIEEE conference on computer vision and pattern recognition (CVPR). IEEE.https://doi.org/10.1109/cvpr.2016.180

  • Perrone, D., Favaro, P.(2014). Total variation blind deconvolution: The devil is in the details. InIEEE conference on computer vision and pattern recognition (CVPR). IEEE .https://doi.org/10.1109/cvpr.2014.372

  • Qayyum, A., Ilahi, I., Shamshad, F., Boussaid, F., Bennamoun, M., & Qadir, J. (2021). Untrained neural network priors for inverse imaging problems: A survey. TechRxiv .https://doi.org/10.36227/techrxiv.14208215

  • Ravula, S., & Dimakis, A. G. (2019). One-dimensional deep image prior for time series inverse problems.arXiv:1904.08594

  • Ren, D., Zhang, K., Wang, Q., Hu, Q., & Zuo, W. (2020). Neural blind deconvolution using deep priors. InIEEE conference on computer vision and pattern recognition (CVPR). IEEE.https://doi.org/10.1109/cvpr42600.2020.00340

  • Rim, J., Lee, H., Won, J., & Cho, S. (2020). Real-world blur dataset for learning and benchmarking deblurring algorithms. InEuropean conference on computer vision (ECCV), pp. 184–201. Springer International Publishing.https://doi.org/10.1007/978-3-030-58595-2_12

  • Schuler, C. J., Hirsch, M., Harmeling, S., & Scholkopf, B. (2016). Learning to deblur.IEEE Transactions on Pattern Analysis and Machine Intelligence,38(7), 1439–1451.https://doi.org/10.1109/tpami.2015.2481418

    Article  Google Scholar 

  • Sheikh, H., & Bovik, A. (2006). Image information and visual quality.IEEE Transactions on Image Processing,15(2), 430–444.https://doi.org/10.1109/tip.2005.859378

    Article  Google Scholar 

  • Shi, Z., Mettes, P., Maji, S., & Snoek, C. G. M. (2022). On measuring and controlling the spectral bias of the deep image prior.International Journal of Computer Vision,130(4), 885–908.https://doi.org/10.1007/s11263-021-01572-7

    Article  Google Scholar 

  • Si-Yao, L., Ren, D., & Yin, Q. (2019). Understanding kernel size in blind deconvolution. InIEEE winter conference on applications of computer vision (WACV). IEEE.https://doi.org/10.1109/wacv.2019.00224

  • Sitzmann, V., Martel, J., Bergman, A., Lindell, D., & Wetzstein, G. (2020). Implicit neural representations with periodic activation functions.Advances in Neural Information Processing Systems,33, 7462–7473.

    Google Scholar 

  • Sun, L., Cho, S., Wang, J., & Hays, J. (2013). Edge-based blur kernel estimation using patch priors. InIEEE international conference on computational photography (ICCP). IEEE.https://doi.org/10.1109/iccphot.2013.6528301

  • Sun, Q., & Donoho, D. (2021). Convex sparse blind deconvolution.arXiv:2106.07053

  • Szeliski, R. (2021).Computer vision: Algorithms and applications (2nd ed.). London: Springer.

    Google Scholar 

  • Tai, Y.W., & Lin, S. (2012). Motion-aware noise filtering for deblurring of noisy and blurry images. InIEEE conference on computer vision and pattern recognition (CVPR). IEEE .https://doi.org/10.1109/cvpr.2012.6247653

  • Tancik, M., Srinivasan, P., Mildenhall, B., Fridovich-Keil, S., Raghavan, N., Singhal, U., Ramamoorthi, R., Barron, J., & Ng, R.(2020). Fourier features let networks learn high frequency functions in low dimensional domains. InAdvances in neural information processing systems.

  • Tao, X., Gao, H., Shen, X., Wang, J., & Jia, J. (2018). Scale-recurrent network for deep image deblurring. InProceedings of the IEEE conference on computer vision and pattern recognition, pp. 8174–8182.

  • Tayal, K., Manekar, R., Zhuang, Z., Yang, D., Kumar, V., Hofmann, F., & Sun, J. (2021). Phase retrieval using single-instance deep generative prior. InOSA optical sensors and sensing congress 2021 (AIS, FTS, HISE, SENSORS, ES). OSA .https://doi.org/10.1364/ais.2021.jw2a.37

  • Tran, P., Tran, A., Phung, Q., & Hoai, M. (2021). Explore image deblurring via encoded blur kernel space. InProceedings of the IEEE conference on computer vision and pattern recognition (CVPR).https://doi.org/10.1109/CVPR46437.2021.01178

  • Ulyanov, D., Vedaldi, A., & Lempitsky, V. (2020). Deep image prior.International Journal of Computer Vision,128(7), 1867–1888.https://doi.org/10.1007/s11263-020-01303-4

    Article  Google Scholar 

  • Vasu, S.(2021). Image and video deblurring: A curated list of resources for image and video deblurring.https://github.com/subeeshvasu/Awesome-Deblurring. Accessed on Dec 12 2021

  • Vembu, S., Verdu, S., Kennedy, R., & Sethares, W. (1994). Convex cost functions in blind equalization.IEEE Transactions on Signal Processing,42(8), 1952–1960.https://doi.org/10.1109/78.301833

    Article  Google Scholar 

  • Wang, H., Li, T., Zhuang, Z., Chen, T., Liang, H., & Sun, J. (2021). Early stopping for deep image prior.arXiv:2112.06074

  • Wang, Z., Wang, Z., Li, Q., & Bilen, H. (2019). Image deconvolution with deep image and kernel priors. In2019 IEEE/CVF international conference on computer vision workshop (ICCVW). IEEE .https://doi.org/10.1109/iccvw.2019.00127

  • Wiggins, R. A. (1978). Minimum entropy deconvolution.Geoexploration,16(1–2), 21–35.https://doi.org/10.1016/0016-7142(78)90005-4

    Article  Google Scholar 

  • Williams, F., Schneider, T., Silva, C., Zorin, D., Bruna, J., & Panozzo, D. (2019). Deep geometric prior for surface reconstruction.arXiv:1811.10943

  • Wipf, D., & Zhang, H. (2014). Revisiting bayesian blind deconvolution.Journal of Machine Learning Research,15(111), 3775–3814.

    MathSciNet  Google Scholar 

  • Xu, L., & Jia, J. (2010). Two-phase kernel estimation for robust motion deblurring. InEuropean Conference on Computer Vision, pp. 157–170. Springer Berlin Heidelberg .https://doi.org/10.1007/978-3-642-15549-9_12

  • Xu, L., Zheng, S., & Jia, J.(2013). Unnatural l0 sparse representation for natural image deblurring. InIEEE conference on computer vision and pattern recognition (CVPR). IEEE.https://doi.org/10.1109/cvpr.2013.147

  • Yan, Y., Ren, W., Guo, Y., Wang, R., & Cao, X. (2017). Image deblurring via extreme channels prior. InIEEE conference on computer vision and pattern recognition (CVPR). IEEE .https://doi.org/10.1109/cvpr.2017.738

  • Yang, D., Zhuang, Z., Phillips, N.W., KaySong, Zdora, M.C., Harder, R., Cha, W., Liu, W., Barmherzig, D.A., Sun, J., & Hofmann, F. (2022). Application of single-instance deep generative priors for reconstruction of highly strained gold microcrystals in bragg coherent x-ray diffraction. In preparation 25

  • Yang, L., & Ji, H. (2019). A variational EM framework with adaptive edge selection for blind motion deblurring. InIEEE conference on computer vision and pattern recognition (CVPR). IEEE .https://doi.org/10.1109/cvpr.2019.01041

  • Zhang, K., Luo, W., Zhong, Y., Ma, L., Stenger, B., Liu, W., & Li, H. (2020). Deblurring by realistic blurring. InProceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 2737–2746.

  • Zhang, K., Ren, W., Luo, W., Lai, W. S., Stenger, B., Yang, M. H., & Li, H. (2022). Deep image deblurring: A survey.International Journal of Computer Vision,130(9), 2103–2130.https://doi.org/10.1007/s11263-022-01633-5

  • Zhang, K., Zuo, W., & Zhang, L. (2019). Deep plug-and-play super-resolution for arbitrary blur kernels. In2019 IEEE/CVF conference on computer vision and pattern recognition (CVPR). IEEE.https://doi.org/10.1109/cvpr.2019.00177

  • Zhang, R., Isola, P., Efros, A.A., Shechtman, E., & Wang, O.(2018). The unreasonable effectiveness of deep features as a perceptual metric 16.

  • Zhang, Y., Kuo, H. W., & Wright, J. (2020). Structured local optima in sparse blind deconvolution.IEEE Transactions on Information Theory,66(1), 419–452.https://doi.org/10.1109/tit.2019.2940657

    Article MathSciNet  Google Scholar 

  • Zhang, Y., Lau, Y., Kuo, H.W., Cheung, S., Pasupathy, A., & Wright, J. (2017). On the global geometry of sphere-constrained sparse blind deconvolution. InIEEE conference on computer vision and pattern recognition (CVPR). IEEE .https://doi.org/10.1109/cvpr.2017.466

  • Zhong, L., Cho, S., Metaxas, D., Paris, S., & Wang, J. (2013). Handling noise in single image deblurring using directional filters. InIEEE conference on computer vision and pattern recognition (CVPR). IEEE .https://doi.org/10.1109/cvpr.2013.85

  • Zhou, K. C., & Horstmeyer, R. (2020). Diffraction tomography with a deep image prior.Optics Express,28(9), 12872.https://doi.org/10.1364/oe.379200

    Article  Google Scholar 

  • Zhuang, Z., Yang, D., Hofmann, F., Barmherzig, D., & Sun, J. (2022). Practical phase retrieval using double deep image priors. arXiv preprintarXiv:2211.00799

Download references

Acknowledgements

Zhong Zhuang, Hengkang Wang, and Ju Sun are partially supported by NSF CMMI 2038403. We thank the anonymous reviewers and the associate editor for their insightful comments that have substantially helped us improve the presentation of this paper. We thank Le Peng and Wenjie Zhang for allowing us to use the e-scooter image of Fig. 1 that they captured. The authors acknowledge the Minnesota Supercomputing Institute (MSI) at the University of Minnesota for providing resources that contributed to the research results reported within this paper.

Author information

Authors and Affiliations

  1. Electrical and Computer Engineering, University of Minnesota, Minneapolis, USA

    Zhong Zhuang

  2. Computer Science and Engineering, University of Minnesota, Minneapolis, USA

    Taihui Li, Hengkang Wang & Ju Sun

Authors
  1. Zhong Zhuang

    You can also search for this author inPubMed Google Scholar

  2. Taihui Li

    You can also search for this author inPubMed Google Scholar

  3. Hengkang Wang

    You can also search for this author inPubMed Google Scholar

  4. Ju Sun

    You can also search for this author inPubMed Google Scholar

Corresponding author

Correspondence toJu Sun.

Additional information

Communicated by Jiaya Jia.

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Appendix

Appendix

1.1List of common acronyms

Table 4 List of acronyms (in alphabetic order)

See Table4

Fig. 36
figure 36

Contrast-enhanced version of Fig. 30 after histogram equalization

Fig. 37
figure 37

Contrast-enhanced version of Fig. 32 after histogram equalization

1.2Contrast-Enhanced Version of Figs. 30 and32

To reveal more details for images in Figs. 30 and32 that are about extremely dark scenes, we perform histogram equalization to enhance the contrast and display the results as follows (Figs.36,37).

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zhuang, Z., Li, T., Wang, H.et al. Blind Image Deblurring with Unknown Kernel Size and Substantial Noise.Int J Comput Vis132, 319–348 (2024). https://doi.org/10.1007/s11263-023-01883-x

Download citation

Keywords

Access this article

Subscribe and save

Springer+ Basic
¥17,985 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price includes VAT (Japan)

Instant access to the full article PDF.

Advertisement

[8]ページ先頭
©2009-2025 Movatter.jp