Movatterモバイル変換

[0]ホーム
URL:

Skip to main content

Advertisement

Springer Nature Link
Log in

AIM 2020 Challenge on Image Extreme Inpainting

  • Conference paper
  • First Online:

Part of the book series:Lecture Notes in Computer Science ((LNIP,volume 12537))

Included in the following conference series:

Abstract

This paper reviews the AIM 2020 challenge on extreme image inpainting. This report focuses on proposed solutions and results for two different tracks on extreme image inpainting: classical image inpainting and semantically guided image inpainting. The goal of track 1 is to inpaint large part of the image with no supervision. Similarly, the goal of track 2 is to inpaint the image by having access to the entire semantic segmentation map of the input. The challenge had 88 and 74 participants, respectively. 11 and 6 teams competed in the final phase of the challenge, respectively. This report gauges current solutions and set a benchmark for future extreme image inpainting methods.

E. Ntavelis (entavelis@ethz.ch, ETH Zurich and CSEM SA), A. Romero, S. Bigdeli, and R. Timofte are the AIM 2020 challenge organizers, while the other authors participated in the challenge.

Appendix A contains the authors’teams and affiliations.

AIM webpage:http://www.vision.ee.ethz.ch/aim20/.

Github webpage:https://github.com/vglsd/AIM2020-Image-Inpainting-Challenge.

This is a preview of subscription content,log in via an institution to check access.

Access this chapter

Subscribe and save

Springer+ Basic
¥17,985 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
JPY 3498
Price includes VAT (Japan)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
JPY 14871
Price includes VAT (Japan)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
JPY 18589
Price includes VAT (Japan)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide -see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Similar content being viewed by others

References

  1. A tensorflow implementation of SRGAN.https://github.com/tensorlayer/srgan.git

  2. Alom, M.Z., Hasan, M., Yakopcic, C., Taha, T.M., Asari, V.K.: Recurrent residual convolutional neural network based on U-Net (R2U-Net) for medical image segmentation. arXiv preprintarXiv:1802.06955 (2018)

  3. Bai, M., Li, S., Fan, J., Zhou, C., Zuo, L., Na, J., Jeong, M.: Fast light-weight network for extreme image inpainting challenge. In: Bartoli, A., Fusiello, A. (eds.) ECCV 2020 Workshops. LNCS, vol. 12537, pp.742–757 (2020)

    Google Scholar 

  4. Bau, D., et al.: Semantic photo manipulation with a generative image prior. ACM Trans. Graph. (Proc. ACM SIGGRAPH)38(4), 1–11 (2019)

    Google Scholar 

  5. Caesar, H., Uijlings, J., Ferrari, V.: Coco-stuff: thing and stuff classes in context. In: CVPR (2018).https://arxiv.org/abs/1612.03716

  6. Cho, K., et al.: Learning phrase representations using RNN encoder-decoder for statistical machine translation. arXiv preprintarXiv:1406.1078 (2014)

  7. El Helou, M., Zhou, R., Süsstrunk, S., Timofte, R., et al.: AIM 2020: scene relighting and illumination estimation challenge. In: Bartoli, A., Fusiello, A. (eds.) ECCV 2020 Workshops. LNCS, vol. 12537, pp. 499–518 (2020)

    Google Scholar 

  8. Fritsche, M., Gu, S., Timofte, R.: Frequency separation for real-world super-resolution. In: 2019 IEEE/CVF International Conference on Computer Vision Workshop (ICCVW), pp. 3599–3608. IEEE (2019)

    Google Scholar 

  9. Fuoli, D., Huang, Z., Gu, S., Timofte, R., et al.: AIM 2020 challenge on video extreme super-resolution: methods and results. In: European Conference on Computer Vision Workshops (2020)

    Google Scholar 

  10. Goodfellow, I., et al.: Generative adversarial nets. In: Advances in Neural Information Processing Systems, pp. 2672–2680 (2014)

    Google Scholar 

  11. Hong, S., Yan, X., Huang, T.E., Lee, H.: Learning hierarchical semantic image manipulation through structured representations. In: Advances in Neural Information Processing Systems, pp. 2713–2723 (2018)

    Google Scholar 

  12. Hui, Z., Li, J., Wang, X., Gao, X.: Image fine-grained inpainting. arXiv preprintarXiv:2002.02609 (2020)

  13. Ignatov, A., Timofte, R., et al.: AIM 2020 challenge on learned image signal processing pipeline. In: Bartoli, A., Fusiello, A. (eds.) ECCV 2020 Workshops. LNCS, vol. 12537, pp. 152–170 (2020)

    Google Scholar 

  14. Ignatov, A., Timofte, R., et al.: AIM 2020 challenge on rendering realistic bokeh. In: Bartoli, A., Fusiello, A. (eds.) ECCV 2020 Workshops. LNCS, vol. 12537, pp. 213–228 (2020)

    Google Scholar 

  15. Isola, P., Zhu, J.Y., Zhou, T., Efros, A.A.: Image-to-image translation with conditional adversarial networks (2016)

    Google Scholar 

  16. Johnson, J., Alahi, A., Fei-Fei, L.: Perceptual losses for real-time style transfer and super-resolution. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) ECCV 2016. LNCS, vol. 9906, pp. 694–711. Springer, Cham (2016).https://doi.org/10.1007/978-3-319-46475-6_43

    Chapter  Google Scholar 

  17. Li, C.T., Siu, W.C., Liu, Z.S., Wang, L.W., Lun, D.P.K.: DeepGIN: deep generative inpainting network for extreme image inpainting. In: European Conference on Computer Vision Workshops (2020)

    Google Scholar 

  18. Li, C., Wand, M.: Precomputed real-time texture synthesis with Markovian generative adversarial networks. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) ECCV 2016. LNCS, vol. 9907, pp. 702–716. Springer, Cham (2016).https://doi.org/10.1007/978-3-319-46487-9_43

    Chapter  Google Scholar 

  19. Lim, J.H., Ye, J.C.: Geometric GAN. arXiv preprintarXiv:1705.02894 (2017)

  20. Nazeri, K., Ng, E., Joseph, T., Qureshi, F., Ebrahimi, M.: EdgeConnect: structure guided image inpainting using edge prediction. In: The IEEE International Conference on Computer Vision (ICCV) Workshops, October 2019

    Google Scholar 

  21. Nazeri, K., Ng, E., Joseph, T., Qureshi, F.Z., Ebrahimi, M.: EdgeConnect: generative image inpainting with adversarial edge learning. arXiv preprintarXiv:1901.00212 (2019)

  22. Ntavelis, E., Romero, A., Kastanis, I., Van Gool, L., Timofte, R.: Sesame: semantic editing of scenes by adding, manipulating or erasing objects. In: Proceedings of the European Conference on Computer Vision (ECCV) (2020)

    Google Scholar 

  23. Ntavelis, E., Romero, A., Bigdeli, S.A., Timofte, R., et al.: AIM 2020 challenge on image extreme inpainting. In: Bartoli, A., Fusiello, A. (eds.) ECCV 2020 Workshops. LNCS, vol. 12537, pp. 716–741 (2020)

    Google Scholar 

  24. Park, T., Liu, M.Y., Wang, T.C., Zhu, J.Y.: Semantic image synthesis with spatially-adaptive normalization. In: CVPR, pp. 2337–2346 (2019)

    Google Scholar 

  25. Shaker, N., Liapis, A., Togelius, J., Lopes, R., Bidarra, R.: Constructive generation methods for dungeons and levels. Procedural Content Generation in Games. CSCS, pp. 31–55. Springer, Cham (2016).https://doi.org/10.1007/978-3-319-42716-4_3

    Chapter  Google Scholar 

  26. Son, S., Lee, J., Nah, S., Timofte, R., Lee, K.M., et al.: AIM 2020 challenge on video temporal super-resolution. In: European Conference on Computer Vision Workshops (2020)

    Google Scholar 

  27. Wang, X., et al.: ESRGAN: enhanced super-resolution generative adversarial networks. In: Leal-Taixé, L., Roth, S. (eds.) ECCV 2018. LNCS, vol. 11133, pp. 63–79. Springer, Cham (2019).https://doi.org/10.1007/978-3-030-11021-5_5

    Chapter  Google Scholar 

  28. Wei, P., Lu, H., Timofte, R., Lin, L., Zuo, W., et al.: AIM 2020 challenge on real image super-resolution : methods and results. In: Bartoli, A., Fusiello, A. (eds.) ECCV 2020 Workshops. LNCS, vol. 12537, pp. 392–422 (2020)

    Google Scholar 

  29. Xiangli, Y., Deng, Y., Dai, B., Loy, C.C., Lin, D.: Real or not real, that is the question. In: ICLR (2020)

    Google Scholar 

  30. Xu, D., Chu, Y., Sun, Q.: Moire pattern removal via attentive fractal network. In: The IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops, June 2020

    Google Scholar 

  31. Yi, Z., Tang, Q., Azizi, S., Jang, D., Xu, Z.: Contextual residual aggregation for ultra high-resolution image inpainting. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 7508–7517 (2020)

    Google Scholar 

  32. Yu, J., Lin, Z., Yang, J., Shen, X., Lu, X., Huang, T.S.: Generative image inpainting with contextual attention. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 5505–5514 (2018)

    Google Scholar 

  33. Yu, J., Lin, Z., Yang, J., Shen, X., Lu, X., Huang, T.S.: Free-form image inpainting with gated convolution. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 4471–4480 (2019)

    Google Scholar 

  34. Zeng, Y., Lin, Z., Yang, J., Zhang, J., Shechtman, E., Lu, H.: High-resolution image inpainting with iterative confidence feedback and guided upsampling. In: European Conference on Computer Vision. Springer (2020)

    Google Scholar 

  35. Zhang, K., Danelljan, M., Li, Y., Timofte, R., et al.: AIM 2020 challenge on efficient super-resolution: Methods and results. In: Bartoli, A., Fusiello, A. (eds.) ECCV 2020 Workshops. LNCS, vol. 12537, pp. 5–40 (2020)

    Google Scholar 

  36. Zhou, B., Lapedriza, A., Khosla, A., Oliva, A., Torralba, A.: Places: a 10 million image database for scene recognition. IEEE Trans. Pattern Anal. Mach. Intell.40(6), 1452–1464 (2017)

    Article  Google Scholar 

  37. Zhou, B., Zhao, H., Puig, X., Fidler, S., Barriuso, A., Torralba, A.: Scene parsing through ade20k dataset. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 633–641 (2017)

    Google Scholar 

Download references

Acknowledgements

We thank the AIM 2020 sponsors: Huawei, MediaTek, Qualcomm AI Research, NVIDIA, Google and Computer Vision Lab/ETH Zürich.

Author information

Authors and Affiliations

  1. Computer Vision Lab, ETH Zürich, Zürich, Switzerland

    Evangelos Ntavelis, Andrés Romero & Radu Timofte

  2. CSEM, Neuchâtel, Switzerland

    Evangelos Ntavelis & Siavash Bigdeli

  3. School of Electronic Engineering, Xidian University, Xi’an, China

    Zheng Hui, Xiumei Wang & Xinbo Gao

  4. Image and Video Pattern Recognition Laboratory, School of Electrical and Electronic Engineering, Yonsei University, Seoul, South Korea

    Chajin Shin, Taeoh Kim, Hanbin Son & Sangyoun Lee

  5. Department of Computer Vision (VIS), Baidu Inc., Beijing, China

    Chao Li, Fu Li, Dongliang He, Shilei Wen & Errui Ding

  6. Samsung R&D Institute China-Beijing (SRC-Beijing), Beijing, China

    Mengmeng Bai & Shuchen Li

  7. Dalian University of Technology, Dalian, China

    Yu Zeng & Huchuan Lu

  8. Adobe, San Jose, USA

    Zhe Lin, Jimei Yang, Jianming Zhang & Eli Shechtman

  9. Rensselaer Polytechnic Institute, Troy, USA

    Weijian Zeng, Haopeng Ni, Yiyang Cai & Chenghua Li

  10. Peking University, Beijing, China

    Dejia Xu, Haoning Wu & Yu Han

  11. Computer Vision and Image Processing (CVIP) Lab, Gachon University, Seongnam, South Korea

    Uddin S. M. Nadim, Hae Woong Jang, Soikat Hasan Ahmed, Jungmin Yoon & Yong Ju Jung

  12. Centre for Multimedia Signal Processing, Department of Electronic and Information Engineering, The Hong Kong Polytechnic University, Hong Kong, China

    Chu-Tak Li, Zhi-Song Liu, Li-Wen Wang, Wan-Chi Siu & Daniel P. K. Lun

  13. Indian Institute of Technology Madras, Chennai, India

    Maitreya Suin, Kuldeep Purohit & A. N. Rajagopalan

  14. BITS Pilani, Pilani, India

    Pratik Narang & Pranjal Singh Chauhan

  15. MNIT Jaipur, Jaipur, India

    Murari Mandal

Authors
  1. Evangelos Ntavelis

    You can also search for this author inPubMed Google Scholar

  2. Andrés Romero

    You can also search for this author inPubMed Google Scholar

  3. Siavash Bigdeli

    You can also search for this author inPubMed Google Scholar

  4. Radu Timofte

    You can also search for this author inPubMed Google Scholar

  5. Zheng Hui

    You can also search for this author inPubMed Google Scholar

  6. Xiumei Wang

    You can also search for this author inPubMed Google Scholar

  7. Xinbo Gao

    You can also search for this author inPubMed Google Scholar

  8. Chajin Shin

    You can also search for this author inPubMed Google Scholar

  9. Taeoh Kim

    You can also search for this author inPubMed Google Scholar

  10. Hanbin Son

    You can also search for this author inPubMed Google Scholar

  11. Sangyoun Lee

    You can also search for this author inPubMed Google Scholar

  12. Chao Li

    You can also search for this author inPubMed Google Scholar

  13. Fu Li

    You can also search for this author inPubMed Google Scholar

  14. Dongliang He

    You can also search for this author inPubMed Google Scholar

  15. Shilei Wen

    You can also search for this author inPubMed Google Scholar

  16. Errui Ding

    You can also search for this author inPubMed Google Scholar

  17. Mengmeng Bai

    You can also search for this author inPubMed Google Scholar

  18. Shuchen Li

    You can also search for this author inPubMed Google Scholar

  19. Yu Zeng

    You can also search for this author inPubMed Google Scholar

  20. Zhe Lin

    You can also search for this author inPubMed Google Scholar

  21. Jimei Yang

    You can also search for this author inPubMed Google Scholar

  22. Jianming Zhang

    You can also search for this author inPubMed Google Scholar

  23. Eli Shechtman

    You can also search for this author inPubMed Google Scholar

  24. Huchuan Lu

    You can also search for this author inPubMed Google Scholar

  25. Weijian Zeng

    You can also search for this author inPubMed Google Scholar

  26. Haopeng Ni

    You can also search for this author inPubMed Google Scholar

  27. Yiyang Cai

    You can also search for this author inPubMed Google Scholar

  28. Chenghua Li

    You can also search for this author inPubMed Google Scholar

  29. Dejia Xu

    You can also search for this author inPubMed Google Scholar

  30. Haoning Wu

    You can also search for this author inPubMed Google Scholar

  31. Yu Han

    You can also search for this author inPubMed Google Scholar

  32. Uddin S. M. Nadim

    You can also search for this author inPubMed Google Scholar

  33. Hae Woong Jang

    You can also search for this author inPubMed Google Scholar

  34. Soikat Hasan Ahmed

    You can also search for this author inPubMed Google Scholar

  35. Jungmin Yoon

    You can also search for this author inPubMed Google Scholar

  36. Yong Ju Jung

    You can also search for this author inPubMed Google Scholar

  37. Chu-Tak Li

    You can also search for this author inPubMed Google Scholar

  38. Zhi-Song Liu

    You can also search for this author inPubMed Google Scholar

  39. Li-Wen Wang

    You can also search for this author inPubMed Google Scholar

  40. Wan-Chi Siu

    You can also search for this author inPubMed Google Scholar

  41. Daniel P. K. Lun

    You can also search for this author inPubMed Google Scholar

  42. Maitreya Suin

    You can also search for this author inPubMed Google Scholar

  43. Kuldeep Purohit

    You can also search for this author inPubMed Google Scholar

  44. A. N. Rajagopalan

    You can also search for this author inPubMed Google Scholar

  45. Pratik Narang

    You can also search for this author inPubMed Google Scholar

  46. Murari Mandal

    You can also search for this author inPubMed Google Scholar

  47. Pranjal Singh Chauhan

    You can also search for this author inPubMed Google Scholar

Corresponding author

Correspondence toEvangelos Ntavelis.

Editor information

Editors and Affiliations

  1. University of Clermont Auvergne, Clermont Ferrand, France

    Adrien Bartoli

  2. Università degli Studi di Udine, Udine, Italy

    Andrea Fusiello

Appendix A: Teams and affiliations

Appendix A: Teams and affiliations

1.1AIM2020 organizers

Members: Evangelos Ntavelis1,2 (entavelis@ethz.ch), Siavash Bigdeli2 (siavash.bigdeli@csem.ch), Andrés Romero1 (roandres@ethz.ch), Radu Timofte1 (radu.timofte@vision.ee.ethz.ch).

Affiliations:1Computer Vision Lab, ETH Zürich.2CSEM.

1.2Rainbow

Title: Image fine-grained inpainting.

Members: Zheng Hui, Xiumei Wang, Xinbo Gao.

Affiliations: School of Electronic Engineering, Xidian University.

1.3Yonsei-MVPLab

Title: Image Inpainting based on Edge and Frequency Guided Recurrent Convolutions.

Members: Chajin Shin, Taeoh Kim, Hanbin Son, Sangyoun Lee.

Affiliations: Image and Video Pattern Recognition Lab., School of Electrical and Electronic Engineering, Yonsei University, Seoul, South Korea.

1.4BossGao

Title: Image Inpainting With Mask Awareness

Members: Chao Li, Fu Li, Dongliang He, Shilei Wen, Errui Ding

Affiliations: Department of Computer Vision (VIS), Baidu Inc.

1.5ArtIst

Title: Fast Light-Weight Network for Image Inpainting

Members: Mengmeng Bai, Shuchen Li

Affiliations: Samsung R&D Institute China-Beijing (SRC-Beijing)

1.6DLUT

Title: Iterative Confidence Feedback and Guided Upsampling for filling large holes and inpainting high-resolution images

Members: Yu Zeng1, Zhe Lin2, Jimei Yang2, Jianming Zhang2, Eli Shechtman2, Huchuan Lu1

Affiliations:1Dalian University of Technology,2Adobe

1.7AI-Inpainting Group

Title: MSEM: Multi-Scale Semantic-Edge Merged Model for Image Inpainting

Members: Weijian Zeng, Haopeng Ni, Yiyang Cai, Chenghua Li

Affiliations: Rensselaer Polytechnic Institute

1.8qwq

Title: Markovian Discriminator guided Attentive Fractal Network

Members: Dejia Xu, Haoning Wu, Yu Han

Affiliations: Peking University

1.9CVIP Inpainting Team

Title: Global Spatial-Channel Attention and Inter-layer GRU-based Image Inpainting

Members: Uddin S. M. Nadim, Hae Woong Jang, Soikat Hasan Ahmed, Jungmin Yoon, and Yong Ju Jung

Affiliations: Computer Vision and Image Processing (CVIP) Lab, Gachon University.

1.10DeepInpaintingT1

Title: Deep Generative Inpainting Network for Extreme Image Inpainting

Members: Chu-Tak Li, Zhi-Song Liu, Li-Wen Wang, Wan-Chi Siu, Daniel P.K. Lun

Affiliations: Centre for Multimedia Signal Processing, Department of Electronic and Information Engineering, The Hong Kong Polytechnic University, Hong Kong

1.11IPCV IITM

Title: Contextual Residual Aggregation Network

Members: Maitreya Suin, Kuldeep Purohit, A. N. Rajagopalan

Affiliations: Indian Institute of Technology Madras, India

1.12MultiCog

Title: Pix2Pix for Image Inpainting

Members: Pratik Narang1, Murari Mandal2, Pranjal Singh Chauhan1

Affiliations:1BITS Pilani,2MNIT Jaipur

Rights and permissions

Copyright information

© 2020 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Ntavelis, E.et al. (2020). AIM 2020 Challenge on Image Extreme Inpainting. In: Bartoli, A., Fusiello, A. (eds) Computer Vision – ECCV 2020 Workshops. ECCV 2020. Lecture Notes in Computer Science(), vol 12537. Springer, Cham. https://doi.org/10.1007/978-3-030-67070-2_43

Download citation

Publish with us

Access this chapter

Subscribe and save

Springer+ Basic
¥17,985 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
JPY 3498
Price includes VAT (Japan)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
JPY 14871
Price includes VAT (Japan)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
JPY 18589
Price includes VAT (Japan)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide -see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

[8]ページ先頭
©2009-2025 Movatter.jp