multimodality

Star

Here are 176 public repositories matching this topic...

Language:All

Filter by language

All176 Python103 Jupyter Notebook25 HTML7 Kotlin4 JavaScript3 TeX3 TypeScript3 Java2 MATLAB2 R2

Sort:Most stars

Sort options

Most stars Fewest stars Most forks Fewest forks Recently updated Least recently updated

lucidrains /big-sleep

Star2.6k

A simple command line tool for text to image generation, using OpenAI's CLIP and a BigGAN. Technique was originally created byhttps://twitter.com/advadnoun

deep-learning artificial-intelligence multimodality generative-adversarial-networks text-to-image

UpdatedFeb 6, 2022
Python

The Cradle framework is a first attempt at General Computer Control (GCC). Cradle supports agents to ace any computer task by enabling strong reasoning abilities, self-improvment, and skill curation, in a standardized general environment with minimal requirements.

ai gcc multimodality vlm cradle computer-control lmm grounding ai-agent large-language-models llm generative-ai vision-language-model ai-agents-framework general-computer-control personoid foundation-agent

UpdatedNov 7, 2024
Python

hymie122 /RAG-Survey

Star1.8k

Collecting awesome papers of RAG for AIGC. We propose a taxonomy of RAG foundations, enhancements, and applications in paper "Retrieval-Augmented Generation for AI-Generated Content: A Survey".

survey multimodality rag diffusion-models aigc llm

UpdatedAug 20, 2024

AIDC-AI /Ovis

Star1.4k

A novel Multimodal Large Language Model (MLLM) architecture, designed to structurally align visual and textual embeddings.

chatbot multimodality multimodal vision-language-model multimodal-large-language-models vision-language-learning qwen llama3

UpdatedSep 22, 2025
Python

ArrowLuo /CLIP4Clip

Star1k

An official implementation for "CLIP4Clip: An Empirical Study of CLIP for End to End Video Clip Retrieval"

search retrieval ranking clip multimodality multimodal-learning multimodal activitynet retrieval-model msvd msrvtt video-text-retrieval lsmdc didemo video-clip-retrieval

UpdatedApr 12, 2024
Python

PreferredAI /cornac

Star997

A Comparative Framework for Multimodal Recommender Systems

collaborative-filtering matrix-factorization recommendation-system recommendation-engine recommender-system recommendation-algorithms multimodality multimodal-learning

UpdatedOct 25, 2025
Python

fnzhan /Generative-AI

Star755

[TPAMI 2023] Multimodal Image Synthesis and Editing: The Generative AI Era

gans multimodality diffusion-model nerfs aigc

UpdatedNov 21, 2023
TeX

aimclub /FEDOT

Star697

Automated modeling and machine learning framework FEDOT

machine-learning automation genetic-programming hyperparameter-optimization evolutionary-algorithms multimodality automl automated-machine-learning parameter-tuning structural-learning fedot

UpdatedOct 14, 2025
Python

VITA-MLLM /Woodpecker

Star638

✨✨Woodpecker: Hallucination Correction for Multimodal Large Language Models

multimodality hallucination hallucinations large-language-models llm mllm multimodal-large-language-models

UpdatedDec 23, 2024
Python

microsoft /LLM2CLIP

Star561

LLM2CLIP makes SOTA pretrained CLIP model more SOTA ever.

clip multimodality fundation-models

UpdatedJul 1, 2025
Python

jshilong /GPT4RoI

Star547

GPT4RoI: Instruction Tuning Large Language Model on Region-of-Interest

computer-vision gpt roi multimodality llm

UpdatedJun 3, 2025
Python

YingqingHe /Awesome-LLMs-meet-Multimodal-Generation

Star514

🔥🔥🔥 A curated list of papers on LLMs-based multimodal generation (image, video, 3D and audio).

text-to-speech multimodality text-to-image text-to-audio text-to-video text-to-music multimodal-models aigc large-language-models llm text-to-3d multimodal-generation mllm text-to-sound large-vision-language-models multimodal-large-language-models lvlm

UpdatedApr 4, 2025
HTML

MMMU-Benchmark /MMMU

Star512

This repo contains evaluation code for the paper "MMMU: A Massive Multi-discipline Multimodal Understanding and Reasoning Benchmark for Expert AGI"

machine-learning natural-language-processing deep-neural-networks computer-vision deep-learning evaluation question-answering stem multimodality multimodal-learning visual-question-answering multimodal multimodal-deep-learning foundation-models large-language-models llm llms large-multimodal-models

UpdatedMay 19, 2025
Python

zengyan-97 /X-VLM

Star484

X-VLM: Multi-Grained Vision Language Pre-Training (ICML 2022)

multimodality vision-and-language x-vlm

UpdatedNov 25, 2022
Python

afiaka87 /clip-guided-diffusion

Star462

A CLI tool/python module for generating images from text using guided diffusion and CLIP from OpenAI.

deep-learning artificial-intelligence openai image-generation multimodality text-to-image diffusion multimodal text-to-image-synthesis openai-clip

UpdatedFeb 8, 2022
Python

kyegomez /Med-PaLM

Sponsor

Star425

Towards Generalist Biomedical AI

opensource deep-learning multimodality biomedical multimodal multimodal-deep-learning gpt4

UpdatedFeb 17, 2024
Python

HazyResearch /fonduer

Star410

A knowledge base construction engine for richly formatted data

machine-learning multimodality knowledge-base-construction

UpdatedJun 23, 2021
Python

lium-lst /nmtpytorch

Star391

Sequence-to-Sequence Framework in PyTorch

deep-learning cnn pytorch speech-recognition seq2seq neural-machine-translation nmt multimodality asr

UpdatedJan 5, 2023
Jupyter Notebook

OmicsML /dance

Star384

DANCE: a deep learning library and benchmark platform for single-cell analysis

python data-science benchmark machine-learning bioinformatics deep-learning computational-biology dance single-cell multimodality single-cell-rna-seq graph-neural-networks spatial-transcriptomics single-cell-rna-sequencing

UpdatedOct 20, 2025
Python

UCSC-VLAA /MedTrinity-25M

Star380

[ICLR 2025] This is the official repository of our paper "MedTrinity-25M: A Large-scale Multimodal Dataset with Multigranular Annotations for Medicine“

dataset multimodality mllms

UpdatedJul 11, 2025
Python

Improve this page

Add a description, image, and links to themultimodality topic page so that developers can more easily learn about it.

Curate this topic

Add this topic to your repo

To associate your repository with themultimodality topic, visit your repo's landing page and select "manage topics."

Learn more

Movatterモバイル変換

Navigation Menu

Search code, repositories, users, issues, pull requests...

Provide feedback

Saved searches

Use saved searches to filter your results more quickly