Movatterモバイル変換

[0]ホーム
URL:

Skip to main content

Advertisement

Springer Nature Link
Log in

Error-Tolerant Image Compositing

  • Published:
International Journal of Computer Vision Aims and scope Submit manuscript

Abstract

Gradient-domain compositing is an essential tool in computer vision and its applications, e.g., seamless cloning, panorama stitching, shadow removal, scene completion and reshuffling. While easy to implement, these gradient-domain techniques often generate bleeding artifacts where the composited image regions do not match. One option is to modify the region boundary to minimize such mismatches. However, this option may not always be sufficient or applicable, e.g., the user or algorithm may not allow the selection to be altered. We propose a new approach to gradient-domain compositing that is robust to inaccuracies and prevents color bleeding without changing the boundary location. Our approach improves standard gradient-domain compositing in two ways. First, we define the boundary gradients such that the produced gradient field is nearly integrable. Second, we control the integration process to concentrate residuals where they are less conspicuous. We show that our approach can be formulated as a standard least-squares problem that can be solved with a sparse linear system akin to the classical Poisson equation. We demonstrate results on a variety of scenes. The visual quality and run-time complexity compares favorably to other approaches.

This is a preview of subscription content,log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
¥17,985 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price includes VAT (Japan)

Instant access to the full article PDF.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15

Similar content being viewed by others

Explore related subjects

Discover the latest articles and news from researchers in related subjects, suggested using machine learning.

Notes

  1. Note that for a 2D vector fieldu=(ux,uy), the curl is a scalar value that corresponds to thez component of the 3D curl applied to the 3D vector field (ux,uy,0).

References

  • Agarwala, A. (2007). Efficient gradient-domain compositing using quadtrees. InACM transactions on graphics: Vol. 26.Proceedings of the ACM SIGGRAPH conference.

    Google Scholar 

  • Agarwala, A., Dontcheva, M., Agrawala, M., Drucker, S., Colburn, A., Curless, B., Salesin, D. H., & Cohen, M. F. (2004). Interactive digital photomontage. InACM transactions on graphics: Vol. 23.Proceedings of the ACM SIGGRAPH conference (pp. 294–302).

    Google Scholar 

  • Agrawal, A., Raskar, R., & Chellappa, R. (2006). What is the range of surface reconstructions from a gradient field? InProceedings of the European conference on computer vision.

    Google Scholar 

  • Aubert, G., & Kornprobst, P. (2002).Applied mathematical sciences: Vol. 147.Mathematical problems in image processing: partial differential equations and the calculus of variations. Berlin: Springer.

    MATH  Google Scholar 

  • Bae, S., Paris, S., & Durand, F. (2006). Two-scale tone management for photographic look. InACM transactions on graphics: Vol. 25.Proceedings of the ACM SIGGRAPH conference (pp. 637–645).

    Google Scholar 

  • Bhat, P., Zitnick, C. L., Cohen, M., & Curless, B. (2009). Gradientshop: a gradient-domain optimization framework for image and video filtering. InACM transactions on graphics.

    Google Scholar 

  • Cho, T. S., Avidan, S., & Freeman, W. T. (2010). The patch transform.IEEE Transactions on Pattern Analysis and Machine Intelligence,32(8), 1489–1501.

    Article  Google Scholar 

  • Drettakis, G., Bonneel, N., Dachsbacher, C., Lefebvre, S., Schwarz, M., & Viaud-Delmon, I. (2007). An interactive perceptual rendering pipeline using contrast and spatial masking.Rendering Techniques.

  • Farbman, Z., Fattal, R., Lischinski, D., & Szeliski, R. (2008). Edge-preserving decompositions for multi-scale tone and detail manipulation. InACM transactions on graphics: Vol. 27.Proceedings of the ACM SIGGRAPH conference.

    Google Scholar 

  • Farbman, Z., Hoffer, G., Lipman, Y., Cohen-Or, D., Fattal, R., & Lischinski, D. (2009). Coordinates for instant image cloning. InACM transactions on graphics: Vol. 28.Proceedings of the ACM SIGGRAPH conference.

    Google Scholar 

  • Finlayson, G. D., Hordley, S. D., Lu, C., & Drew, M. S. (2006). On the removal of shadows from images.IEEE Transactions on Pattern Analysis and Machine Intelligence,28, 59–68.

    Article  Google Scholar 

  • Finlayson, G. D., Drew, M. S., & Lu, C. (2009). Entropy minimization for shadow removal.International Journal of Computer Vision,85(1), 35–57.

    Article  Google Scholar 

  • Georgiev, T. (2006). Covariant derivatives and vision. InProceedings of the European conference on computer vision.

    Google Scholar 

  • Hays, J., & Efros, A. A. (2007). Scene completion using millions of photographs. InACM transactions on graphics: Vol. 26.Proceedings of the ACM SIGGRAPH conference.

    Google Scholar 

  • Jia, J., Sun, J., Tang, C. K., & Shum, H. Y. (2006). Drag-and-drop pasting. InACM transactions on graphics: Vol. 25.Proceedings of the ACM SIGGRAPH conference.

    Google Scholar 

  • Lalonde, J. F., Hoiem, D., Efros, A., Rother, C., Winn, J., & Criminisi, A. (2007). Photo clip art. InACM transactions on graphics: Vol. 26.Proceedings of the ACM SIGGRAPH conference.

    Google Scholar 

  • Levin, A., Zomet, A., Peleg, S., & Weiss, Y. (2006). Seamless image stitching in the gradient domain. InProceedings of the European conference on computer vision.

    Google Scholar 

  • Perona, P., & Malik, J. (1990). Scale-space and edge detection using anisotropic diffusion.IEEE Transactions on Pattern Analysis and Machine Intelligence,12, 629–639.

    Article  Google Scholar 

  • Prez, P., Gangnet, M., & Blake, A. (2003). Poisson image editing. InACM transactions on graphics: Vol. 22.Proceedings of the ACM SIGGRAPH conference.

    Google Scholar 

  • Ramanarayanan, G., Ferwerda, J., Walter, B., & Bala, K. (2007). Visual equivalence: towards a new standard for image fidelity. InACM transactions on graphics: Vol. 26.Proceedings of the ACM SIGGRAPH conference.

    Google Scholar 

  • Ramanarayanan, G., Bala, K., & Ferwerda, J. (2008). Perception of complex aggregates. InACM transactions on graphics: Vol. 27.Proceedings of the ACM SIGGRAPH conference.

    Google Scholar 

  • Reddy, D., Agrawal, A., & Chellappa, R. (2009). Enforcing integrability by error correction using L-1 minimization. InProceedings of the conference on computer vision and pattern recognition.

    Google Scholar 

  • Sivic, J., Kaneva, B., Torralba, A., Avidan, S., & Freeman, W. T. (2008). Creating and exploring a large photorealistic virtual space. InProceedings of the IEEE workshop on internet vision.

    Google Scholar 

  • Su, S., Durand, F., & Agrawala, M. (2005). De-emphasis of distracting image regions using texture power maps. InProceedings of the ICCV workshop on texture analysis and synthesis.

    Google Scholar 

  • Tappen, M. F., Adelson, E. H., & Freeman, W. T. (2005). Recovering intrinsic images from a single image.IEEE Transactions on Pattern Analysis and Machine Intelligence,27, 1459–1472.

    Article  Google Scholar 

  • Vangorp, P., Laurijssen, J., & Dutr, P. (2007). The influence of shape on the perception of material reflectance. InACM transactions on graphics: Vol. 26.Proceedings of the ACM SIGGRAPH conference.

    Google Scholar 

  • Wang, Z., Bovik, A. C., Sheikh, H. R., & Simoncelli, E. P. (2004). Image quality assessment: from error visibility to structural similarity.IEEE Transactions on Image Processing,13(14), 600–612.

    Article  Google Scholar 

  • Whyte, O., Sivic, J., & Zisserman, A. (2009). Get out of my picture! Internet-based inpainting. InProceedings of the British machine vision conference.

    Google Scholar 

Download references

Acknowledgements

The authors thank Todor Georgiev for the link with the Poisson equation, Kavita Bala and George Drettakis for their discussion about visual masking, Aseem Agarwala and Bill Freeman for their help with the paper, Tim Cho and Biliana Kaneva for helping with the validation, Medhat H. Ibrahim for the image of the Egyption pyramids, Adobe Systems, Inc. for supporting Micah K. Johnson’s research, and Ravi Ramamoorthi for supporting Michael Tao’s work. This material is based upon work supported by the National Science Foundation under Grant Nos. 0739255 and 0924968.

Author information

Authors and Affiliations

  1. University of California, Berkeley, CA, USA

    Michael W. Tao

  2. Massachusetts Institute of Technology, Cambridge, MA, USA

    Micah K. Johnson

  3. Adobe Systems, Inc., Cambridge, MA, USA

    Sylvain Paris

Authors
  1. Michael W. Tao

    You can also search for this author inPubMed Google Scholar

  2. Micah K. Johnson

    You can also search for this author inPubMed Google Scholar

  3. Sylvain Paris

    You can also search for this author inPubMed Google Scholar

Corresponding author

Correspondence toMichael W. Tao.

Electronic Supplementary Material

Below is the link to the electronic supplementary material.

Rights and permissions

About this article

Cite this article

Tao, M.W., Johnson, M.K. & Paris, S. Error-Tolerant Image Compositing.Int J Comput Vis103, 178–189 (2013). https://doi.org/10.1007/s11263-012-0579-7

Download citation

Keywords

Access this article

Subscribe and save

Springer+ Basic
¥17,985 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price includes VAT (Japan)

Instant access to the full article PDF.

Advertisement

[8]ページ先頭
©2009-2025 Movatter.jp