- Rafael E. Rivadeneira ORCID:orcid.org/0000-0002-5327-204814,
- Angel D. Sappa ORCID:orcid.org/0000-0003-2468-003114,15 &
- Boris X. Vintimilla ORCID:orcid.org/0000-0001-8904-020914
Part of the book series:Communications in Computer and Information Science ((CCIS,volume 1474))
Included in the following conference series:
1061Accesses
Abstract
This paper proposes the use of a CycleGAN architecture for thermal image super-resolution under a transfer domain strategy, where middle-resolution images from one camera are transferred to a higher resolution domain of another camera. The proposed approach is trained with a large dataset acquired using three thermal cameras at different resolutions. An unsupervised learning process is followed to train the architecture. Additional loss function is proposed trying to improve results from the state of the art approaches. Following the first thermal image super-resolution challenge (PBVS-CVPR2020) evaluations are performed. A comparison with previous works is presented showing the proposed approach reaches the best results.
This is a preview of subscription content,log in via an institution to check access.
Access this chapter
Subscribe and save
- Get 10 units per month
- Download Article/Chapter or eBook
- 1 Unit = 1 Article or 1 Chapter
- Cancel anytime
Buy Now
- Chapter
- JPY 3498
- Price includes VAT (Japan)
- eBook
- JPY 11439
- Price includes VAT (Japan)
- Softcover Book
- JPY 14299
- Price includes VAT (Japan)
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
Notes
- 1.
FREE FLIR Thermal Dataset for Algorithm Traininghttps://www.flir.in/oem/adas/adas-dataset-form/.
References
Arbel, P., Maire, M., Fowlkes, C., Malik, J.: Contour detection and hierarchical image segmentation. IEEE Trans. Pattern Anal. Mach. Intell.33(5), 898–916 (2011)
Bevilacqua, M., Roumy, A., Guillemot, C., Alberi-Morel, M.L.: Low-complexity single-image super-resolution based on nonnegative neighbor embedding (2012)
Chang, H., Lu, J., Yu, F., Finkelstein, A.: PairedCycleGAN: asymmetric style transfer for applying and removing makeup. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 40–48 (2018)
Chen, Y.S., Wang, Y.C., Kao, M.H., Chuang, Y.Y.: Deep photo enhancer: unpaired learning for image enhancement from photographs with GANs. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 6306–6314 (2018)
Choi, Y., Kim, N., Hwang, S., Kweon, I.S.: Thermal image enhancement using convolutional neural network. In: 2016 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 223–230. IEEE (2016)
Chudasama, V., et al.: TheriSuRNet-a computationally efficient thermal image super-resolution network. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 86–87 (2020)
Davis, J.W., Keck, M.A.: A two-stage template approach to person detection in thermal imagery. In: 2005 Seventh IEEE Workshops on Applications of Computer Vision (WACV/MOTION 2005), vol. 1, pp. 364–369. IEEE (2005)
Ding, M., Zhang, X., Chen, W.H., Wei, L., Cao, Y.F.: Thermal infrared pedestrian tracking via fusion of features in driving assistance system of intelligent vehicles. Proc. Inst. Mech. Eng. Part G: J. Aerosp. Eng.233(16), 6089–6103 (2019)
Dong, C., Loy, C.C., He, K., Tang, X.: Image super-resolution using deep convolutional networks. IEEE Trans. Pattern Anal. Mach. Intell.38(2), 295–307 (2015)
Dong, C., Loy, C.C., Tang, X.: Accelerating the super-resolution convolutional neural network. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) ECCV 2016. LNCS, vol. 9906, pp. 391–407. Springer, Cham (2016).https://doi.org/10.1007/978-3-319-46475-6_25
Gade, R., Moeslund, T.B.: Thermal cameras and applications: a survey. Mach. Vis. Appl.25(1), 245–262 (2013).https://doi.org/10.1007/s00138-013-0570-5
Girshick, R., Donahue, J., Darrell, T., Malik, J.: Region-based convolutional networks for accurate object detection and segmentation. IEEE Trans. Pattern Anal. Mach. Intell.38(1), 142–158 (2015)
Goodfellow, I., et al.: Generative adversarial nets. In: Advances in Neural Information Processing Systems, pp. 2672–2680 (2014)
Herrmann, C., Ruf, M., Beyerer, J.: CNN-based thermal infrared person detection by domain adaptation. In: Autonomous Systems: Sensors, Vehicles, Security, and the Internet of Everything, vol. 10643, p. 1064308. International Society for Optics and Photonics (2018)
Huang, J.B., Singh, A., Ahuja, N.: Single image super-resolution from transformed self-exemplars. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 5197–5206 (2015)
Hwang, S., Park, J., Kim, N., Choi, Y., So Kweon, I.: Multispectral pedestrian detection: benchmark dataset and baseline. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1037–1045 (2015)
Kansal, P., Nathan, S.: A multi-level supervision model: a novel approach for thermal image super resolution. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 94–95 (2020)
Kim, J., Kwon Lee, J., Mu Lee, K.: Accurate image super-resolution using very deep convolutional networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1646–1654 (2016)
Kittler, J.: On the accuracy of the Sobel edge detector. Image Vis. Comput.1(1), 37–42 (1983)
Lim, B., Son, S., Kim, H., Nah, S., Mu Lee, K.: Enhanced deep residual networks for single image super-resolution. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, pp. 136–144 (2017)
Mandanici, E., Tavasci, L., Corsini, F., Gandolfi, S.: A multi-image super-resolution algorithm applied to thermal imagery. Appl. Geomat.11(3), 215–228 (2019).https://doi.org/10.1007/s12518-019-00253-y
Martin, D., Fowlkes, C., Tal, D., Malik, J., et al.: A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics. In: ICCV, Vancouver (2001)
Matsui, Y., et al.: Sketch-based manga retrieval using manga109 dataset. Multimed. Tools Appl.76(20), 21811–21838 (2016).https://doi.org/10.1007/s11042-016-4020-z
Mehri, A., Sappa, A.D.: Colorizing near infrared images through a cyclic adversarial approach of unpaired samples. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops (2019)
Mudunuri, S.P., Biswas, S.: Low resolution face recognition across variations in pose and illumination. IEEE Trans. Pattern Anal. Mach. Intell.38(5), 1034–1040 (2015)
Olmeda, D., Premebida, C., Nunes, U., Armingol, J.M., de la Escalera, A.: Pedestrian detection in far infrared images. Integr. Comput.-Aided Eng.20(4), 347–360 (2013)
Park, S.J., Son, H., Cho, S., Hong, K.S., Lee, S.: SRFeat: single image super-resolution with feature discrimination. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 439–455 (2018)
Rivadeneira, R.E., et al.: Thermal image super-resolution challenge-PBVS 2020. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 96–97 (2020)
Rivadeneira, R.E., Sappa, A.D., Vintimilla, B.X.: Thermal image super-resolution: a novel architecture and dataset. In: VISIGRAPP (4: VISAPP), pp. 111–119 (2020)
Rivadeneira, R.E., Suárez, P.L., Sappa, A.D., Vintimilla, B.X.: Thermal image superresolution through deep convolutional neural network. In: Karray, F., Campilho, A., Yu, A. (eds.) ICIAR 2019. LNCS, vol. 11663, pp. 417–426. Springer, Cham (2019).https://doi.org/10.1007/978-3-030-27272-2_37
Shamsolmoali, P., Zareapoor, M., Jain, D.K., Jain, V.K., Yang, J.: Deep convolution network for surveillance records super-resolution. Multimed. Tools Appl.78(17), 23815–23829 (2018).https://doi.org/10.1007/s11042-018-5915-7
Shi, W., Ledig, C., Wang, Z., Theis, L., Huszar, F.: Super resolution using a generative adversarial network, 15 March 2018. US Patent App. 15/706,428
Suarez, P.L., Sappa, A.D., Vintimilla, B.X., Hammoud, R.I.: Image vegetation index through a cycle generative adversarial network. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops (2019)
Timofte, R., Agustsson, E., Van Gool, L., Yang, M.H., Zhang, L.: NTIRE 2017 challenge on single image super-resolution: methods and results. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, pp. 114–125 (2017)
Wang, Z., Bovik, A.C., Sheikh, H.R., Simoncelli, E.P., et al.: Image quality assessment: from error visibility to structural similarity. IEEE Trans. Image Process.13(4), 600–612 (2004)
Wu, Z., Fuller, N., Theriault, D., Betke, M.: A thermal infrared video benchmark for visual analysis. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, pp. 201–208 (2014)
Zeyde, R., Elad, M., Protter, M.: On single image scale-up using sparse-representations. In: Boissonnat, J.-D., Chenin, P., Cohen, A., Gout, C., Lyche, T., Mazure, M.-L., Schumaker, L. (eds.) Curves and Surfaces 2010. LNCS, vol. 6920, pp. 711–730. Springer, Heidelberg (2012).https://doi.org/10.1007/978-3-642-27413-8_47
Zhang, K., Zuo, W., Gu, S., Zhang, L.: Learning deep CNN denoiser prior for image restoration. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3929–3938 (2017)
Zhang, L., Zhang, H., Shen, H., Li, P.: A super-resolution reconstruction algorithm for surveillance images. Signal Process.90(3), 848–859 (2010)
Zhang, Y., Li, K., Li, K., Wang, L., Zhong, B., Fu, Y.: Image super-resolution using very deep residual channel attention networks. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 286–301 (2018)
Zhu, J.Y., Park, T., Isola, P., Efros, A.A.: Unpaired image-to-image translation using cycle-consistent adversarial networks. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2223–2232 (2017)
Acknowledgements
This work has been partially supported by the Spanish Government under Project TIN2017-89723-P; and the “CERCA Programme/Generalitat de Catalunya”. The first author has been supported by Ecuador government under a SENESCYT scholarship contract.
Author information
Authors and Affiliations
Escuela Superior Politécnica del Litoral, ESPOL, Facultad de Ingeniería en Electricidad y Computación, CIDIS, Campus Gustavo Galindo Km. 30.5 Vía Perimetral, P.O. Box 09-01-5863, Guayaquil, Ecuador
Rafael E. Rivadeneira, Angel D. Sappa & Boris X. Vintimilla
Computer Vision Center, Edifici O, Campus UAB, Bellaterra, 08193, Barcelona, Spain
Angel D. Sappa
- Rafael E. Rivadeneira
You can also search for this author inPubMed Google Scholar
- Angel D. Sappa
You can also search for this author inPubMed Google Scholar
- Boris X. Vintimilla
You can also search for this author inPubMed Google Scholar
Corresponding author
Correspondence toRafael E. Rivadeneira.
Editor information
Editors and Affiliations
IRISA, University of Rennes 1, Rennes, France
Kadi Bouatouch
Universidade do Porto, Porto, Portugal
A. Augusto de Sousa
University of Genova, Genova, Italy
Manuela Chessa
Mines ParisTech, Paris, France
Alexis Paljic
Linnaeus University, Växjö, Sweden
Andreas Kerren
French Civil Aviation University (ENAC), Toulouse, France
Christophe Hurter
Università di Catania, Catania, Italy
Giovanni Maria Farinella
Universitat de Barcelona, Barcelona, Spain
Petia Radeva
Escola Superior de Tecnologia de Setúbal, Setúbal, Portugal
Jose Braz
Rights and permissions
Copyright information
© 2022 Springer Nature Switzerland AG
About this paper
Cite this paper
Rivadeneira, R.E., Sappa, A.D., Vintimilla, B.X. (2022). Thermal Image Super-Resolution: A Novel Unsupervised Approach. In: Bouatouch, K.,et al. Computer Vision, Imaging and Computer Graphics Theory and Applications. VISIGRAPP 2020. Communications in Computer and Information Science, vol 1474. Springer, Cham. https://doi.org/10.1007/978-3-030-94893-1_23
Download citation
Published:
Publisher Name:Springer, Cham
Print ISBN:978-3-030-94892-4
Online ISBN:978-3-030-94893-1
eBook Packages:Computer ScienceComputer Science (R0)
Share this paper
Anyone you share the following link with will be able to read this content:
Sorry, a shareable link is not currently available for this article.
Provided by the Springer Nature SharedIt content-sharing initiative