multi-modal-learning

A curated list of Visual Question Answering(VQA)(Image/Video Question Answering),Visual Question Generation ,Visual Dialog ,Visual Commonsense Reasoning and related area.

vqa awesome-list multi-modal multi-modal-learning attention-networks

UpdatedJul 6, 2023

InternRobotics /EmbodiedScan

Star646

[CVPR 2024 & NeurIPS 2024] EmbodiedScan: A Holistic Multi-Modal 3D Perception Suite Towards Embodied AI

computer-vision robotics 3d-vision multi-modal-learning

UpdatedJun 13, 2025
Python

zjukg /KG-MM-Survey

Star463

Knowledge Graphs Meet Multi-Modal Learning: A Comprehensive Survey

information-extraction survey knowledge-graph awsome image-classification image-generation surveys entity-linking knowledge-graph-embeddings visual-question-answering entity-alignment paper-list awsome-list cross-modal-retrieval multi-modal-learning multi-modal-fusion large-language-models multi-modal-knowledge-graph

UpdatedDec 10, 2024

DmitryRyumin /CVPR-2023-24-Papers

Star457

CVPR 2023-2024 Papers: Dive into advanced research presented at the leading computer vision conference. Keep up to date with the latest developments in computer vision and deep learning. Code included. ⭐ support visual intelligence development!

computer-vision deep-learning biometrics segmentation face-recognition pattern-recognition datasets autonomous-driving cvpr action-recognition gesture-recognition medical-image-processing shape-analysis image-synthesis self-supervised-learning scene-analysis multi-modal-learning video-synthesis cvpr2023 cvpr2024

UpdatedJul 15, 2024
Python

zhengli97 /PromptKD

Star345

[CVPR 2024] Official PyTorch Code for "PromptKD: Unsupervised Prompt Distillation for Vision-Language Models"

clip knowledge-distillation multi-modal-learning prompt-learning vision-language-model cvpr2024

UpdatedDec 14, 2025
Python

Ysz2022 /NeRCo

Star260

[ICCV 2023] Implicit Neural Representation for Cooperative Low-light Image Enhancement

iccv low-light-image multi-modal-learning low-light-image-enhancement neural-representation iccv2023

UpdatedMar 18, 2024
Python

moabarar /nemar

Star190

[CVPR2020] Unsupervised Multi-Modal Image Registration via Geometry Preserving Image-to-Image Translation

deep-learning cnn pytorch multi-modal image-registration affine-transformation stn image-to-image-translation multimodal deformable-transformation multi-modal-learning cvpr2020 registartion multimodal-image-registration

UpdatedAug 2, 2020
Python

GuanRunwei /Achelous

Star161

The official repository of Achelous and Achelous++

object-detection object-tracking semantic-segmentation multi-task-learning point-cloud-segmentation multi-modal-learning multi-modal-fusion panoptic-perception 4d-mmwave-radar

UpdatedJul 8, 2024
Python

huggingface /chug

Star160

Minimal sharded dataset loaders, decoders, and utils for multi-modal document, image, and text datasets.

computer-vision pdf-document datasets distributed-training dataloading document-understanding multi-modal-learning webdataset

UpdatedApr 3, 2024
Python

qizekun /ReCon

Star149

[ICML 2023] Contrast with Reconstruct: Contrastive 3D Representation Learning Guided by Generative Pretraining

representation-learning 3d-point-clouds self-supervised-learning multi-modal-learning

UpdatedJul 21, 2024
Python

wjun0830 /CGDETR

Star145

Official pytorch repository for CG-DETR "Correlation-guided Query-Dependency Calibration in Video Representation Learning for Temporal Grounding"

computer-vision video-summarization pytorch video-understanding video-grounding multi-modal-learning detr moment-retrieval highlight-detection detection-transformer temporal-grounding text-video-retrieval

UpdatedAug 21, 2024
Python

shikras /d-cube

Star138

A detection/segmentation dataset with labels characterized by intricate and flexible expressions. "Described Object Detection: Liberating Object Detection with Flexible Expressions" (NeurIPS 2023).

dataset object-detection vision-language multi-modal-learning referring-expression-comprehension open-vocabulary-detection