Movatterモバイル変換


[0]ホーム

URL:


Skip to main content

Advertisement

Springer Nature Link
Log in

Thermal Image Super-Resolution: A Novel Unsupervised Approach

  • Conference paper
  • First Online:

Abstract

This paper proposes the use of a CycleGAN architecture for thermal image super-resolution under a transfer domain strategy, where middle-resolution images from one camera are transferred to a higher resolution domain of another camera. The proposed approach is trained with a large dataset acquired using three thermal cameras at different resolutions. An unsupervised learning process is followed to train the architecture. Additional loss function is proposed trying to improve results from the state of the art approaches. Following the first thermal image super-resolution challenge (PBVS-CVPR2020) evaluations are performed. A comparison with previous works is presented showing the proposed approach reaches the best results.

This is a preview of subscription content,log in via an institution to check access.

Access this chapter

Subscribe and save

Springer+ Basic
¥17,985 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
JPY 3498
Price includes VAT (Japan)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
JPY 11439
Price includes VAT (Japan)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
JPY 14299
Price includes VAT (Japan)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide -see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Similar content being viewed by others

Notes

  1. 1.

    FREE FLIR Thermal Dataset for Algorithm Traininghttps://www.flir.in/oem/adas/adas-dataset-form/.

References

  1. Arbel, P., Maire, M., Fowlkes, C., Malik, J.: Contour detection and hierarchical image segmentation. IEEE Trans. Pattern Anal. Mach. Intell.33(5), 898–916 (2011)

    Article  Google Scholar 

  2. Bevilacqua, M., Roumy, A., Guillemot, C., Alberi-Morel, M.L.: Low-complexity single-image super-resolution based on nonnegative neighbor embedding (2012)

    Google Scholar 

  3. Chang, H., Lu, J., Yu, F., Finkelstein, A.: PairedCycleGAN: asymmetric style transfer for applying and removing makeup. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 40–48 (2018)

    Google Scholar 

  4. Chen, Y.S., Wang, Y.C., Kao, M.H., Chuang, Y.Y.: Deep photo enhancer: unpaired learning for image enhancement from photographs with GANs. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 6306–6314 (2018)

    Google Scholar 

  5. Choi, Y., Kim, N., Hwang, S., Kweon, I.S.: Thermal image enhancement using convolutional neural network. In: 2016 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 223–230. IEEE (2016)

    Google Scholar 

  6. Chudasama, V., et al.: TheriSuRNet-a computationally efficient thermal image super-resolution network. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 86–87 (2020)

    Google Scholar 

  7. Davis, J.W., Keck, M.A.: A two-stage template approach to person detection in thermal imagery. In: 2005 Seventh IEEE Workshops on Applications of Computer Vision (WACV/MOTION 2005), vol. 1, pp. 364–369. IEEE (2005)

    Google Scholar 

  8. Ding, M., Zhang, X., Chen, W.H., Wei, L., Cao, Y.F.: Thermal infrared pedestrian tracking via fusion of features in driving assistance system of intelligent vehicles. Proc. Inst. Mech. Eng. Part G: J. Aerosp. Eng.233(16), 6089–6103 (2019)

    Article  Google Scholar 

  9. Dong, C., Loy, C.C., He, K., Tang, X.: Image super-resolution using deep convolutional networks. IEEE Trans. Pattern Anal. Mach. Intell.38(2), 295–307 (2015)

    Article  Google Scholar 

  10. Dong, C., Loy, C.C., Tang, X.: Accelerating the super-resolution convolutional neural network. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) ECCV 2016. LNCS, vol. 9906, pp. 391–407. Springer, Cham (2016).https://doi.org/10.1007/978-3-319-46475-6_25

    Chapter  Google Scholar 

  11. Gade, R., Moeslund, T.B.: Thermal cameras and applications: a survey. Mach. Vis. Appl.25(1), 245–262 (2013).https://doi.org/10.1007/s00138-013-0570-5

    Article  Google Scholar 

  12. Girshick, R., Donahue, J., Darrell, T., Malik, J.: Region-based convolutional networks for accurate object detection and segmentation. IEEE Trans. Pattern Anal. Mach. Intell.38(1), 142–158 (2015)

    Article  Google Scholar 

  13. Goodfellow, I., et al.: Generative adversarial nets. In: Advances in Neural Information Processing Systems, pp. 2672–2680 (2014)

    Google Scholar 

  14. Herrmann, C., Ruf, M., Beyerer, J.: CNN-based thermal infrared person detection by domain adaptation. In: Autonomous Systems: Sensors, Vehicles, Security, and the Internet of Everything, vol. 10643, p. 1064308. International Society for Optics and Photonics (2018)

    Google Scholar 

  15. Huang, J.B., Singh, A., Ahuja, N.: Single image super-resolution from transformed self-exemplars. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 5197–5206 (2015)

    Google Scholar 

  16. Hwang, S., Park, J., Kim, N., Choi, Y., So Kweon, I.: Multispectral pedestrian detection: benchmark dataset and baseline. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1037–1045 (2015)

    Google Scholar 

  17. Kansal, P., Nathan, S.: A multi-level supervision model: a novel approach for thermal image super resolution. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 94–95 (2020)

    Google Scholar 

  18. Kim, J., Kwon Lee, J., Mu Lee, K.: Accurate image super-resolution using very deep convolutional networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1646–1654 (2016)

    Google Scholar 

  19. Kittler, J.: On the accuracy of the Sobel edge detector. Image Vis. Comput.1(1), 37–42 (1983)

    Article  Google Scholar 

  20. Lim, B., Son, S., Kim, H., Nah, S., Mu Lee, K.: Enhanced deep residual networks for single image super-resolution. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, pp. 136–144 (2017)

    Google Scholar 

  21. Mandanici, E., Tavasci, L., Corsini, F., Gandolfi, S.: A multi-image super-resolution algorithm applied to thermal imagery. Appl. Geomat.11(3), 215–228 (2019).https://doi.org/10.1007/s12518-019-00253-y

    Article  Google Scholar 

  22. Martin, D., Fowlkes, C., Tal, D., Malik, J., et al.: A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics. In: ICCV, Vancouver (2001)

    Google Scholar 

  23. Matsui, Y., et al.: Sketch-based manga retrieval using manga109 dataset. Multimed. Tools Appl.76(20), 21811–21838 (2016).https://doi.org/10.1007/s11042-016-4020-z

    Article  Google Scholar 

  24. Mehri, A., Sappa, A.D.: Colorizing near infrared images through a cyclic adversarial approach of unpaired samples. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops (2019)

    Google Scholar 

  25. Mudunuri, S.P., Biswas, S.: Low resolution face recognition across variations in pose and illumination. IEEE Trans. Pattern Anal. Mach. Intell.38(5), 1034–1040 (2015)

    Article  Google Scholar 

  26. Olmeda, D., Premebida, C., Nunes, U., Armingol, J.M., de la Escalera, A.: Pedestrian detection in far infrared images. Integr. Comput.-Aided Eng.20(4), 347–360 (2013)

    Article  Google Scholar 

  27. Park, S.J., Son, H., Cho, S., Hong, K.S., Lee, S.: SRFeat: single image super-resolution with feature discrimination. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 439–455 (2018)

    Google Scholar 

  28. Rivadeneira, R.E., et al.: Thermal image super-resolution challenge-PBVS 2020. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 96–97 (2020)

    Google Scholar 

  29. Rivadeneira, R.E., Sappa, A.D., Vintimilla, B.X.: Thermal image super-resolution: a novel architecture and dataset. In: VISIGRAPP (4: VISAPP), pp. 111–119 (2020)

    Google Scholar 

  30. Rivadeneira, R.E., Suárez, P.L., Sappa, A.D., Vintimilla, B.X.: Thermal image superresolution through deep convolutional neural network. In: Karray, F., Campilho, A., Yu, A. (eds.) ICIAR 2019. LNCS, vol. 11663, pp. 417–426. Springer, Cham (2019).https://doi.org/10.1007/978-3-030-27272-2_37

    Chapter  Google Scholar 

  31. Shamsolmoali, P., Zareapoor, M., Jain, D.K., Jain, V.K., Yang, J.: Deep convolution network for surveillance records super-resolution. Multimed. Tools Appl.78(17), 23815–23829 (2018).https://doi.org/10.1007/s11042-018-5915-7

    Article  Google Scholar 

  32. Shi, W., Ledig, C., Wang, Z., Theis, L., Huszar, F.: Super resolution using a generative adversarial network, 15 March 2018. US Patent App. 15/706,428

    Google Scholar 

  33. Suarez, P.L., Sappa, A.D., Vintimilla, B.X., Hammoud, R.I.: Image vegetation index through a cycle generative adversarial network. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops (2019)

    Google Scholar 

  34. Timofte, R., Agustsson, E., Van Gool, L., Yang, M.H., Zhang, L.: NTIRE 2017 challenge on single image super-resolution: methods and results. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, pp. 114–125 (2017)

    Google Scholar 

  35. Wang, Z., Bovik, A.C., Sheikh, H.R., Simoncelli, E.P., et al.: Image quality assessment: from error visibility to structural similarity. IEEE Trans. Image Process.13(4), 600–612 (2004)

    Article  Google Scholar 

  36. Wu, Z., Fuller, N., Theriault, D., Betke, M.: A thermal infrared video benchmark for visual analysis. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, pp. 201–208 (2014)

    Google Scholar 

  37. Zeyde, R., Elad, M., Protter, M.: On single image scale-up using sparse-representations. In: Boissonnat, J.-D., Chenin, P., Cohen, A., Gout, C., Lyche, T., Mazure, M.-L., Schumaker, L. (eds.) Curves and Surfaces 2010. LNCS, vol. 6920, pp. 711–730. Springer, Heidelberg (2012).https://doi.org/10.1007/978-3-642-27413-8_47

    Chapter  Google Scholar 

  38. Zhang, K., Zuo, W., Gu, S., Zhang, L.: Learning deep CNN denoiser prior for image restoration. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3929–3938 (2017)

    Google Scholar 

  39. Zhang, L., Zhang, H., Shen, H., Li, P.: A super-resolution reconstruction algorithm for surveillance images. Signal Process.90(3), 848–859 (2010)

    Article MATH  Google Scholar 

  40. Zhang, Y., Li, K., Li, K., Wang, L., Zhong, B., Fu, Y.: Image super-resolution using very deep residual channel attention networks. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 286–301 (2018)

    Google Scholar 

  41. Zhu, J.Y., Park, T., Isola, P., Efros, A.A.: Unpaired image-to-image translation using cycle-consistent adversarial networks. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2223–2232 (2017)

    Google Scholar 

Download references

Acknowledgements

This work has been partially supported by the Spanish Government under Project TIN2017-89723-P; and the “CERCA Programme/Generalitat de Catalunya”. The first author has been supported by Ecuador government under a SENESCYT scholarship contract.

Author information

Authors and Affiliations

  1. Escuela Superior Politécnica del Litoral, ESPOL, Facultad de Ingeniería en Electricidad y Computación, CIDIS, Campus Gustavo Galindo Km. 30.5 Vía Perimetral, P.O. Box 09-01-5863, Guayaquil, Ecuador

    Rafael E. Rivadeneira, Angel D. Sappa & Boris X. Vintimilla

  2. Computer Vision Center, Edifici O, Campus UAB, Bellaterra, 08193, Barcelona, Spain

    Angel D. Sappa

Authors
  1. Rafael E. Rivadeneira

    You can also search for this author inPubMed Google Scholar

  2. Angel D. Sappa

    You can also search for this author inPubMed Google Scholar

  3. Boris X. Vintimilla

    You can also search for this author inPubMed Google Scholar

Corresponding author

Correspondence toRafael E. Rivadeneira.

Editor information

Editors and Affiliations

  1. IRISA, University of Rennes 1, Rennes, France

    Kadi Bouatouch

  2. Universidade do Porto, Porto, Portugal

    A. Augusto de Sousa

  3. University of Genova, Genova, Italy

    Manuela Chessa

  4. Mines ParisTech, Paris, France

    Alexis Paljic

  5. Linnaeus University, Växjö, Sweden

    Andreas Kerren

  6. French Civil Aviation University (ENAC), Toulouse, France

    Christophe Hurter

  7. Università di Catania, Catania, Italy

    Giovanni Maria Farinella

  8. Universitat de Barcelona, Barcelona, Spain

    Petia Radeva

  9. Escola Superior de Tecnologia de Setúbal, Setúbal, Portugal

    Jose Braz

Rights and permissions

Copyright information

© 2022 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Rivadeneira, R.E., Sappa, A.D., Vintimilla, B.X. (2022). Thermal Image Super-Resolution: A Novel Unsupervised Approach. In: Bouatouch, K.,et al. Computer Vision, Imaging and Computer Graphics Theory and Applications. VISIGRAPP 2020. Communications in Computer and Information Science, vol 1474. Springer, Cham. https://doi.org/10.1007/978-3-030-94893-1_23

Download citation

Publish with us

Access this chapter

Subscribe and save

Springer+ Basic
¥17,985 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
JPY 3498
Price includes VAT (Japan)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
JPY 11439
Price includes VAT (Japan)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
JPY 14299
Price includes VAT (Japan)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide -see info

Tax calculation will be finalised at checkout

Purchases are for personal use only


[8]ページ先頭

©2009-2025 Movatter.jp