1. Amir, S., Gandelsman, Y., Bagon, S., Dekel, T.: Deep ViT features as dense visual descriptors. In: European Conference on Computer Vision (ECCV) Workshops (2022)

   Google Scholar 

2. Bachmann, R., Mizrahi, D., Atanov, A., Zamir, A.: MultiMAE: multi-modal multi-task masked autoencoders. In: Avidan, S., Brostow, G., Cissé, M., Farinella, G.M., Hassner, T. (eds.) ECCV 2022, Part XXXVII, pp. 348–367. Springer, Cham (2022).https://doi.org/10.1007/978-3-031-19836-6_20

3. Bao, H., Dong, L., Piao, S., Wei, F.: BEiT: BERT pre-training of Image Transformers. In: International Conference on Learning Representations (ICLR) (2022)

   Google Scholar 

4. Bengio, Y., Courville, A., Vincent, P.: Representation learning: a review and new perspectives. IEEE Trans. Pattern Anal. Mach. Intell.35(8), 1798–1828 (2013)

   Google Scholar 

5. Bhat, S.F., Alhashim, I., Wonka, P.: Adabins: depth estimation using adaptive bins. In: International Conference on Computer Vision and Pattern Recognition (CVPR) (2021)

   Google Scholar 

6. Caron, M., Misra, I., Mairal, J., Goyal, P., Bojanowski, P., Joulin, A.: Unsupervised learning of visual features by contrasting cluster assignments. In: International Conference on Neural Information Processing Systems (NeurIPS) (2020)

   Google Scholar 

7. Caron, M., et al.: Emerging properties in self-supervised vision transformers. In: International Conference on Computer Vision (ICCV) (2021)

   Google Scholar 

8. Chen, M., et al.: Generative pretraining from pixels. In: International Conference on Machine Learning (ICML) (2020)

   Google Scholar 

9. Dai, A., Chang, A.X., Savva, M., Halber, M., Funkhouser, T., Nießner, M.: ScanNet: richly-annotated 3D reconstructions of indoor scenes. In: International Conference on Computer Vision and Pattern Recognition (CVPR) (2017)

   Google Scholar 

10. Darcet, T., Oquab, M., Mairal, J., Bojanowski, P.: Vision transformers need registers. In: International Conference on Learning Representations (ICLR) (2024)

    Google Scholar 

11. Devlin, J., Chang, M.W., Lee, K., Toutanova, K.: BERT: pre-training of deep bidirectional transformers for language understanding. In: NAACL (2018)

    Google Scholar 

12. Doersch, C., Gupta, A., Efros, A.A.: Unsupervised visual representation learning by context prediction. In: International Conference on Computer Vision (ICCV) (2015)

    Google Scholar 

13. Dosovitskiy, A., et al.: An image is worth 16x16 words: transformers for image recognition at scale. In: International Conference on Learning Representations (ICLR) (2020)

    Google Scholar 

14. Dosovitskiy, A., Springenberg, J.T., Riedmiller, M., Brox, T.: Discriminative unsupervised feature learning with convolutional neural networks. In: International Conference on Neural Information Processing Systems (NeurIPS) (2014)

    Google Scholar 

15. El Banani, M., et al.: Probing the 3D awareness of visual foundation models. In: International Conference on Computer Vision and Pattern Recognition (CVPR) (2024)

    Google Scholar 

16. Engelmann, F., Manhardt, F., Niemeyer, M., Tateno, K., Tombari, F.: OpenNeRF: open set 3D neural scene segmentation with pixel-wise features and rendered novel views. In: International Conference on Learning Representations (ICLR) (2024)

    Google Scholar 

17. Everingham, M., Eslami, S.A., Van Gool, L., Williams, C.K., Winn, J., Zisserman, A.: The Pascal visual object classes challenge: a retrospective. Int. J. Comput. Vision (2015)

    Google Scholar 

18. Geiger, A., Lenz, P., Stiller, C., Urtasun, R.: Vision meets robotics: the Kitti dataset. Int. J. Robot. Res. (2013)

    Google Scholar 

19. Ghiasi, G., Gu, X., Cui, Y., Lin, T.Y.: Scaling open-vocabulary image segmentation with image-level labels. In: European Conference on Computer Vision (ECCV) (2022)

    Google Scholar 

20. Gidaris, S., Singh, P., Komodakis, N.: Unsupervised representation learning by predicting image rotations. In: International Conference on Learning Representations (ICLR) (2018)

    Google Scholar 

21. Grill, J.B., et al.: Bootstrap your own latent-a new approach to self-supervised learning. In: International Conference on Neural Information Processing Systems (NeurIPS) (2020)

    Google Scholar 

22. Ha, H., Song, S.: Semantic abstraction: open-world 3D scene understanding from 2D vision-language models. In: Conference on Robot Learning (CoRL) (2022)

    Google Scholar 

23. Hadsell, R., Chopra, S., LeCun, Y.: Dimensionality reduction by learning an invariant mapping. In: International Conference on Computer Vision and Pattern Recognition (CVPR) (2006)

    Google Scholar 

24. He, K., Chen, X., Xie, S., Li, Y., Dollár, P., Girshick, R.: Masked autoencoders are scalable vision learners. In: International Conference on Computer Vision and Pattern Recognition (CVPR) (2022)

    Google Scholar 

25. He, K., Fan, H., Wu, Y., Xie, S., Girshick, R.: Momentum contrast for unsupervised visual representation learning. In: International Conference on Computer Vision and Pattern Recognition (CVPR) (2020)

    Google Scholar 

26. Hou, J., Dai, X., He, Z., Dai, A., Nießner, M.: Mask3D: pre-training 2D vision transformers by learning masked 3D priors. In: International Conference on Computer Vision and Pattern Recognition (CVPR) (2023)

    Google Scholar 

27. Hou, J., Xie, S., Graham, B., Dai, A., Nießner, M.: Pri3D: can 3D priors help 2D representation learning? In: International Conference on Computer Vision (ICCV) (2021)

    Google Scholar 

28. Huang, R., et al.: Segment3D: learning fine-grained class-agnostic 3D segmentation without manual labels. In: European Conference on Computer Vision (ECCV) (2024)

    Google Scholar 

29. Jatavallabhula, K.M., et al.: ConceptFusion: open-set multimodal 3D mapping. Sci. Syst. (RSS) Robot. (2023)

    Google Scholar 

30. Ke, B., Obukhov, A., Huang, S., Metzger, N., Daudt, R.C., Schindler, K.: Repurposing diffusion-based image generators for monocular depth estimation. In: International Conference on Computer Vision and Pattern Recognition (CVPR) (2024)

    Google Scholar 

31. Kerbl, B., Kopanas, G., Leimkühler, T., Drettakis, G.: 3D Gaussian splatting for real-time radiance field rendering. ACM Trans. Graph. (2023)

    Google Scholar 

32. Kerr, J., Kim, C.M., Goldberg, K., Kanazawa, A., Tancik, M.: LERF: language embedded radiance fields. In: International Conference on Computer Vision (ICCV) (2023)

    Google Scholar 

33. Kirillov, A., et al.: Segment anything. In: International Conference on Computer Vision (ICCV) (2023)

    Google Scholar 

34. Kobayashi, S., Matsumoto, E., Sitzmann, V.: Decomposing Nerf for editing via feature field distillation. In: International Conference on Neural Information Processing Systems (NeurIPS) (2022)

    Google Scholar 

35. Li, B., Weinberger, K.Q., Belongie, S., Koltun, V., Ranftl, R.: Language-driven Semantic Segmentation. In: International Conference on Learning Representations (ICLR) (2022)

    Google Scholar 

36. Li, F., et al.: Mask DINO: towards a unified transformer-based framework for object detection and segmentation. In: International Conference on Computer Vision and Pattern Recognition (CVPR) (2023)

    Google Scholar 

37. Liu, Z., et al.: Swin transformer: hierarchical vision transformer using shifted windows. In: International Conference on Computer Vision (ICCV) (2021)

    Google Scholar 

38. Loshchilov, I., Hutter, F.: Decoupled weight decay regularization. In: International Conference on Learning Representations (ICLR) (2019)

    Google Scholar 

39. Mazur, K., Sucar, E., Davison, A.J.: Feature-realistic neural fusion for real-time, open set scene understanding. In: International Conference on Robotics and Automation (ICRA) (2023)

    Google Scholar 

40. Mildenhall, B., Srinivasan, P.P., Tancik, M., Barron, J.T., Ramamoorthi, R., Ng, R.: NeRF: representing scenes as neural radiance fields for view synthesis. In: European Conference on Computer Vision (ECCV) (2020)

    Google Scholar 

41. Noroozi, M., Favaro, P.: Unsupervised learning of visual representations by solving Jigsaw puzzles. In: European Conference on Computer Vision (ECCV) (2016)

    Google Scholar 

42. Oquab, M., et al.: DINOv2: learning robust visual features without supervision. Trans. Mach. Learn. Res. (2023)

    Google Scholar 

43. Pathak, D., Girshick, R., Dollár, P., Darrell, T., Hariharan, B.: Learning features by watching objects move. In: International Conference on Computer Vision and Pattern Recognition (CVPR) (2017)

    Google Scholar 

44. Peng, S., Genova, K., Jiang, C., Tagliasacchi, A., Pollefeys, M., Funkhouser, T.: OpenScene: 3D scene understanding with open vocabularies. In: International Conference on Computer Vision and Pattern Recognition (CVPR) (2023)

    Google Scholar 

45. Qin, M., Li, W., Zhou, J., Wang, H., Pfister, H.: LangSplat: 3D language Gaussian splatting. In: International Conference on Computer Vision and Pattern Recognition (CVPR) (2024)

    Google Scholar 

46. Radford, A., et al.: Learning transferable visual models from natural language supervision. In: International Conference on Machine Learning (ICML) (2021)

    Google Scholar 

47. Rombach, R., Blattmann, A., Lorenz, D., Esser, P., Ommer, B.: High-resolution image synthesis with latent diffusion models. In: International Conference on Computer Vision and Pattern Recognition (CVPR) (2022)

    Google Scholar 

48. Saxena, S., et al.: The surprising effectiveness of diffusion models for optical flow and monocular depth estimation. In: International Conference on Neural Information Processing Systems (NeurIPS) (2024)

    Google Scholar 

49. Shen, W., Yang, G., Yu, A., Wong, J., Kaelbling, L.P., Isola, P.: Distilled feature fields enable few-shot language-guided manipulation. In: Conference on Robot Learning (CoRL) (2023)

    Google Scholar 

50. Shi, J.C., Wang, M., Duan, H.B., Guan, S.H.: Language embedded 3D Gaussians for open-vocabulary scene understanding. In: International Conference on Computer Vision and Pattern Recognition (CVPR) (2024)

    Google Scholar 

51. Silberman, N., Hoiem, D., Kohli, P., Fergus, R.: Indoor segmentation and support inference from RGBD images. In: Fitzgibbon, A., Lazebnik, S., Perona, P., Sato, Y., Schmid, C. (eds.) ECCV 2012. LNCS, vol. 7576, pp. 746–760. Springer, Heidelberg (2012).https://doi.org/10.1007/978-3-642-33715-4_54

    Chapter  Google Scholar 

52. Takmaz, A., Fedele, E., Sumner, R.W., Pollefeys, M., Tombari, F., Engelmann, F.: OpenMask3D: open-vocabulary 3D instance segmentation. In: International Conference on Neural Information Processing Systems (NeurIPS) (2023)

    Google Scholar 

53. Tan, H., Wu, S., Pi, J.: Semantic diffusion network for semantic segmentation. In: International Conference on Neural Information Processing Systems (NeurIPS) (2022)

    Google Scholar 

54. Touvron, H., Cord, M., Jégou, H.: DeiT III: revenge of the ViT. In: Avidan, S., Brostow, G., Cissé, M., Farinella, G.M., Hassner, T. (eds.) ECCV 2022, Part XXIV, pp. 516–533. Springer, Cham (2022).https://doi.org/10.1007/978-3-031-20053-3_30

55. Tschernezki, V., Laina, I., Larlus, D., Vedaldi, A.: Neural feature fusion fields: 3D distillation of self-supervised 2D image representations. In: International Conference on 3D Vision (3DV) (2022)

    Google Scholar 

56. Wang, X., Gupta, A.: Unsupervised learning of visual representations using videos. In: International Conference on Computer Vision (ICCV) (2015)

    Google Scholar 

57. Weder, S., Blum, H., Engelmann, F., Pollefeys, M.: LabelMaker: automatic semantic label generation from RGB-D trajectories. In: International Conference on 3D Vision (3DV) (2024)

    Google Scholar 

58. Weinzaepfel, P., et al.: CroCo: self-supervised pre-training for 3D vision tasks by cross-view completion. In: International Conference on Neural Information Processing Systems (NeurIPS) (2022)

    Google Scholar 

59. Yang, L., Kang, B., Huang, Z., Xu, X., Feng, J., Zhao, H.: Depth anything: unleashing the power of large-scale unlabeled data. In: International Conference on Computer Vision and Pattern Recognition (CVPR) (2024)

    Google Scholar 

60. Yeshwanth, C., Liu, Y.C., Nießner, M., Dai, A.: ScanNet++: a high-fidelity dataset of 3D indoor scenes. In: International Conference on Computer Vision (ICCV) (2023)

    Google Scholar 

61. Zhang, J., et al.: A tale of two features: stable diffusion complements DINO for zero-shot semantic correspondence. In: International Conference on Neural Information Processing Systems (NeurIPS) (2023)

    Google Scholar 

62. Zhou, B., Zhao, H., Puig, X., Fidler, S., Barriuso, A., Torralba, A.: Scene parsing through ade20K dataset. In: International Conference on Computer Vision and Pattern Recognition (CVPR) (2017)

    Google Scholar 

63. Zhou, S., et al.: Feature 3DGS: supercharging 3D Gaussian splatting to enable distilled feature fields. In: International Conference on Computer Vision and Pattern Recognition (CVPR) (2024)

    Google Scholar 

Download references