- Daoyi Gao12,
- Yitong Li12,
- Patrick Ruhkamp12,
- Iuliia Skobleva12,
- Magdalena Wysocki12,
- HyunJun Jung12,
- Pengyuan Wang12,
- Arturo Guridi12 &
- …
- Benjamin Busam12
Part of the book series:Lecture Notes in Computer Science ((LNCS,volume 13669))
Included in the following conference series:
4566Accesses
Abstract
Light has many properties that vision sensors can passively measure. Colour-band separated wavelength and intensity are arguably the most commonly used for monocular 6D object pose estimation. This paper explores how complementary polarisation information, i.e. the orientation of light wave oscillations, influences the accuracy of pose predictions. A hybrid model that leverages physical priors jointly with a data-driven learning strategy is designed and carefully tested on objects with different levels of photometric complexity. Our design significantly improves the pose accuracy compared to state-of-the-art photometric approaches and enables object pose estimation for highly reflective and transparent objects. A new multi-modal instance-level 6D object pose dataset with highly accurate pose annotations for multiple objects with varying photometric complexity is introduced as a benchmark.
D. Gao, Y. Li, P. Ruhkamp, I. Skobleva and M. Wysocki—Equal contribution; Alphabetical order.
This is a preview of subscription content,log in via an institution to check access.
Access this chapter
Subscribe and save
- Get 10 units per month
- Download Article/Chapter or eBook
- 1 Unit = 1 Article or 1 Chapter
- Cancel anytime
Buy Now
- Chapter
- JPY 3498
- Price includes VAT (Japan)
- eBook
- JPY 12583
- Price includes VAT (Japan)
- Softcover Book
- JPY 15729
- Price includes VAT (Japan)
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
Notes
- 1.
Latin for “let there be light”.
- 2.
Dataset and code publicly available at:https://daoyig.github.io/PPPNet/.
References
Atkinson, G.A.: Polarisation photometric stereo. Comput. Vis. Image Underst.160, 158–167 (2017)
Atkinson, G.A., Hancock, E.R.: Multi-view surface reconstruction using polarization. In: IEEE International Conference on Computer Vision (ICCV), pp. 309–316 (2005)
Atkinson, G.A., Hancock, E.R.: Recovery of surface orientation from diffuse polarization. Trans. Image Process.15(6), 1653–1664 (2006)
Ba, Y., Gilbert, A., Wang, F., Yang, J., Chen, R., Wang, Y., Yan, L., Shi, B., Kadambi, A.: Deep shape from polarization. In: Vedaldi, A., Bischof, H., Brox, T., Frahm, J.-M. (eds.) ECCV 2020. LNCS, vol. 12369, pp. 554–571. Springer, Cham (2020).https://doi.org/10.1007/978-3-030-58586-0_33
Besl, P.J., McKay, N.D.: Method for registration of 3d shapes. In: Sensor Fusion IV: Control Paradigms and Data Structures, vol. 1611, pp. 586–606. International Society for Optics and Photonics (1992)
Birdal, T., Ilic, S.: Point pair features based object detection and pose estimation revisited. In: IEEE International Conference on 3D Vision (3DV), pp. 527–535 (2015)
Brachmann, E., Krull, A., Michel, F., Gumhold, S., Shotton, J., Rother, C.: Learning 6D object pose estimation using 3D object coordinates. In: Fleet, D., Pajdla, T., Schiele, B., Tuytelaars, T. (eds.) ECCV 2014. LNCS, vol. 8690, pp. 536–551. Springer, Cham (2014).https://doi.org/10.1007/978-3-319-10605-2_35
Brachmann, E., Michel, F., Krull, A., Yang, M.Y., Gumhold, S., et al.: Uncertainty-driven 6d pose estimation of objects and scenes from a single rgb image. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 3364–3372 (2016)
Busam, B., Jung, H.J., Navab, N.: I like to move it: 6d pose estimation as an action decision process. arXiv preprintarXiv:2009.12678 (2020)
Cui, Z., Gu, J., Shi, B., Tan, P., Kautz, J.: Polarimetric multi-view stereo. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1558–1567 (2017)
Cui, Z., Larsson, V., Pollefeys, M.: Polarimetric relative pose estimation. In: Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pp. 2671–2680 (2019)
Di, Y., Manhardt, F., Wang, G., Ji, X., Navab, N., Tombari, F.: So-pose: exploiting self-occlusion for direct 6d pose estimation. In: Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pp. 12396–12405 (2021)
Drost, B., Ulrich, M., Bergmann, P., Hartinger, P., Steger, C.: Introducing mvtec itodd-a dataset for 3d object recognition in industry. In: Proceedings of the IEEE International Conference on Computer Vision (ICCV) Workshops, pp. 2200–2208 (2017)
Drost, B., Ulrich, M., Navab, N., Ilic, S.: Model globally, match locally: efficient and robust 3d object recognition. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 998–1005 (2010)
Fließbach, T.: Elektrodynamik: Lehrbuch zur Theoretischen Physik II, vol. 2. Springer-Verlag (2012)
Garcia, N.M., De Erausquin, I., Edmiston, C., Gruev, V.: Surface normal reconstruction using circularly polarized light. Opt. Express23(11), 14391–14406 (2015)
He, K., Gkioxari, G., Dollár, P., Girshick, R.: Mask r-cnn. In: Proceedings of the IEEE International Conference on Computer Vision (ICCV), pp. 2961–2969 (2017)
He, Y., Huang, H., Fan, H., Chen, Q., Sun, J.: Ffb6d: a full flow bidirectional fusion network for 6d pose estimation. In: IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) (2021)
He, Y., Sun, W., Huang, H., Liu, J., Fan, H., Sun, J.: Pvn3d: A deep point-wise 3d keypoints voting network for 6dof pose estimation. In: IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) (2020)
Heikkila, J., Silvén, O.: A four-step camera calibration procedure with implicit image correction. In: Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1106–1112 (1997)
Hinterstoisser, S., et al.: Model based training, detection and pose estimation of texture-less 3d objects in heavily cluttered scenes. In: Asian Conference on Computer Vision (ACCV), pp. 548–562 (2012)
Hodan, T., Barath, D., Matas, J.: Epos: estimating 6d pose of objects with symmetries. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 11703–11712 (2020)
Hodan, T., Haluza, P., Obdržálek, Š., Matas, J., Lourakis, M., Zabulis, X.: T-less: An rgb-d dataset for 6d pose estimation of texture-less objects. In: IEEE Winter Conference on Applications of Computer Vision (WACV), pp. 880–888 (2017)
Hu, Y., Fua, P., Wang, W., Salzmann, M.: Single-stage 6d object pose estimation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 2930–2939 (2020)
Hu, Y., Hugonot, J., Fua, P., Salzmann, M.: Segmentation-driven 6d object pose estimation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 3385–3394 (2019)
Islam, M.N., Tahtali, M., Pickering, M.: Specular reflection detection and inpainting in transparent object through msplfi. Remote Sens13(3), 455 (2021)
Kadambi, A., Taamazyan, V., Shi, B., Raskar, R.: Depth sensing using geometrically constrained polarization normals. Int. J. Comput. Vis. (IJCV)125(1–3), 34–51 (2017)
Kalra, A., Taamazyan, V., Rao, S.K., Venkataraman, K., Raskar, R., Kadambi, A.: Deep polarization cues for transparent object segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 8602–8611 (2020)
Kaskman, R., Zakharov, S., Shugurov, I., Ilic, S.: Homebreweddb: Rgb-d dataset for 6d pose estimation of 3d objects. In: International Conference on Computer Vision (ICCV) Workshops (2019)
Kehl, W., Manhardt, F., Tombari, F., Ilic, S., Navab, N.: Ssd-6d: Making rgb-based 3d detection and 6d pose estimation great again. In: Proceedings of the IEEE International Conference on Computer Vision (ICCV). pp. 1521–1529 (2017)
Kingma, D.P., Ba, J.: Adam: a method for stochastic optimization. arXiv preprintarXiv:1412.6980 (2014)
Kundu, A., Li, Y., Rehg, J.M.: 3d-rcnn: instance-level 3d object reconstruction via render-and-compare. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 3559–3568 (2018)
Labbé, Y., Carpentier, J., Aubry, M., Sivic, J.: CosyPose: consistent multi-view multi-object 6D pose estimation. In: Vedaldi, A., Bischof, H., Brox, T., Frahm, J.-M. (eds.) ECCV 2020. LNCS, vol. 12362, pp. 574–591. Springer, Cham (2020).https://doi.org/10.1007/978-3-030-58520-4_34
Lei, C., Huang, X., Zhang, M., Yan, Q., Sun, W., Chen, Q.: Polarized reflection removal with perfect alignment in the wild. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1750–1758 (2020)
Li, Y., Wang, G., Ji, X., Xiang, Yu., Fox, D.: DeepIM: deep iterative matching for 6D pose estimation. In: Ferrari, V., Hebert, M., Sminchisescu, C., Weiss, Y. (eds.) ECCV 2018. LNCS, vol. 11210, pp. 695–711. Springer, Cham (2018).https://doi.org/10.1007/978-3-030-01231-1_42
Li, Z., Wang, G., Ji, X.: Cdpn: coordinates-based disentangled pose network for real-time rgb-based 6-dof object pose estimation. In: Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pp. 7678–7687 (2019)
Liu, X., Iwase, S., Kitani, K.M.: Stereobj-1m: large-scale stereo image dataset for 6d object pose estimation. In: Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pp. 10870–10879 (2021)
Liu, X., Jonschkowski, R., Angelova, A., Konolige, K.: Keypose: multi-view 3d labeling and keypoint estimation for transparent objects. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 11602–11610 (2020)
Manhardt, F., et al.: Explaining the ambiguity of object detection and 6d pose from visual data. In: Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pp. 6841–6850 (2019)
Park, K., Patten, T., Vincze, M.: Pix2pose: pixel-wise coordinate regression of objects for 6d pose estimation. In: Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pp. 7668–7677 (2019)
Peng, S., Liu, Y., Huang, Q., Zhou, X., Bao, H.: Pvnet: pixel-wise voting network for 6dof pose estimation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4561–4570 (2019)
Phillips, C.J., Lecce, M., Daniilidis, K.: Seeing glassware: From edge detection to pose estimation and shape recovery. In: Robotics: Science and Systems, vol. 3 (2016)
Rad, M., Lepetit, V.: Bb8: a scalable, accurate, robust to partial occlusion method for predicting the 3d poses of challenging objects without using depth. In: Proceedings of the IEEE International Conference on Computer Vision (ICCV), pp. 3828–3836 (2017)
Sajjan, S., et al.: Clear grasp: 3d shape estimation of transparent objects for manipulation. In: IEEE International Conference on Robotics and Automation (ICRA), pp. 3634–3642 (2020)
Saxena, A., Driemeyer, J., Ng, A.Y.: Robotic grasping of novel objects using vision. Int. J. Robot. Res.27(2), 157–173 (2008)
Smith, W.A., Ramamoorthi, R., Tozza, S.: Height-from-polarisation with unknown lighting or albedo. IEEE Trans. Pattern Anal. Mach. Intell. (T-PAMI)41(12), 2875–2888 (2018)
Song, C., Song, J., Huang, Q.: Hybridpose: 6d object pose estimation under hybrid representations. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 431–440 (2020)
Sundermeyer, M., Durner, M., Puang, E.Y., Marton, Z.C., Vaskevicius, N., Arras, K.O., Triebel, R.: Multi-path learning for object pose estimation across domains. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). pp. 13916–13925 (2020)
Sundermeyer, M., Marton, Z.C., Durner, M., Brucker, M., Triebel, R.: Implicit 3d orientation learning for 6d object detection from rgb images. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 699–715 (2018)
Verdie, Y., Song, J., Mas, B., Benjamin, B., Leonardis, A., McDonagh, S.: Cromo: cross-modal learning for monocular depth estimation. In: IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) (2022)
Wang, C., et al.: Densefusion: 6d object pose estimation by iterative dense fusion. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 3343–3352 (2019)
Wang, G., Manhardt, F., Tombari, F., Ji, X.: Gdr-net: geometry-guided direct regression network for monocular 6d object pose estimation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 16611–16621 (2021)
Wang, P., et al.: Phocal: a multimodal dataset for category-level object pose estimation with photometrically challenging objects. In: IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) (2022)
Wohlhart, P., Lepetit, V.: Learning descriptors for object recognition and 3d pose estimation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 3109–3118 (2015)
Xiang, Y., Schmidt, T., Narayanan, V., Fox, D.: Posecnn: a convolutional neural network for 6d object pose estimation in cluttered scenes. arXiv preprintarXiv:1711.00199 (2017)
Yu, Y., Zhu, D., Smith, W.A.: Shape-from-polarisation: a nonlinear least squares approach. In: Proceedings of the IEEE International Conference on Computer Vision (ICCV) Workshops, pp. 2969–2976 (2017)
Zakharov, S., Shugurov, I., Ilic, S.: Dpod: 6d pose object detector and refiner. In: Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pp. 1941–1950 (2019)
Zhang, Y., Morel, O., Blanchon, M., Seulin, R., Rastgoo, M., Sidibé, D.: Exploration of deep learning-based multimodal fusion for semantic road scene segmentation. In: VISIGRAPP (5: VISAPP), pp. 336–343 (2019)
Zhang, Z.: A flexible new technique for camera calibration. Transactions on Pattern Analysis and Machine Intelligence (T-PAMI) 22(11), 1330–1334 (2000)
Zhou, Y., Barnes, C., Lu, J., Yang, J., Li, H.: On the continuity of rotation representations in neural networks. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 5745–5753 (2019)
Zhu, D., Smith, W.A.: Depth from a polarisation + rgb stereo pair. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 7586–7595 (2019)
Author information
Authors and Affiliations
Technical University of Munich, Munich, Germany
Daoyi Gao, Yitong Li, Patrick Ruhkamp, Iuliia Skobleva, Magdalena Wysocki, HyunJun Jung, Pengyuan Wang, Arturo Guridi & Benjamin Busam
- Daoyi Gao
You can also search for this author inPubMed Google Scholar
- Yitong Li
You can also search for this author inPubMed Google Scholar
- Patrick Ruhkamp
You can also search for this author inPubMed Google Scholar
- Iuliia Skobleva
You can also search for this author inPubMed Google Scholar
- Magdalena Wysocki
You can also search for this author inPubMed Google Scholar
- HyunJun Jung
You can also search for this author inPubMed Google Scholar
- Pengyuan Wang
You can also search for this author inPubMed Google Scholar
- Arturo Guridi
You can also search for this author inPubMed Google Scholar
- Benjamin Busam
You can also search for this author inPubMed Google Scholar
Corresponding author
Correspondence toPatrick Ruhkamp.
Editor information
Editors and Affiliations
Tel Aviv University, Tel Aviv, Israel
Shai Avidan
University College London, London, UK
Gabriel Brostow
Google AI, Accra, Ghana
Moustapha Cissé
University of Catania, Catania, Italy
Giovanni Maria Farinella
Facebook (United States), Menlo Park, CA, USA
Tal Hassner
1Electronic supplementary material
Below is the link to the electronic supplementary material.
Rights and permissions
Copyright information
© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Gao, D.et al. (2022). Polarimetric Pose Prediction. In: Avidan, S., Brostow, G., Cissé, M., Farinella, G.M., Hassner, T. (eds) Computer Vision – ECCV 2022. ECCV 2022. Lecture Notes in Computer Science, vol 13669. Springer, Cham. https://doi.org/10.1007/978-3-031-20077-9_43
Download citation
Published:
Publisher Name:Springer, Cham
Print ISBN:978-3-031-20076-2
Online ISBN:978-3-031-20077-9
eBook Packages:Computer ScienceComputer Science (R0)
Share this paper
Anyone you share the following link with will be able to read this content:
Sorry, a shareable link is not currently available for this article.
Provided by the Springer Nature SharedIt content-sharing initiative