visual-language-learning

🦦 Otter, a multi-modal model based on OpenFlamingo (open-sourced version of DeepMind's Flamingo), trained on MIMIC-IT and showcasing improved instruction-following and in-context learning ability.

machine-learning deep-learning multi-modality artificial-inteligence embodied-ai gpt-4 foundation-models large-scale-models visual-language-learning chatgpt instruction-tuning

UpdatedMar 5, 2024
Python

InternLM /InternLM-XComposer

Star2.9k

InternLM-XComposer2.5-OmniLive: A Comprehensive Multimodal System for Long-term Streaming Video and Audio Interactions

foundation gpt language-model multimodal multi-modality vision-transformer gpt-4 visual-language-learning llm chatgpt instruction-tuning large-language-model supervised-finetuning mllm vision-language-model large-vision-language-model

UpdatedMay 26, 2025
Python

xiaoachen98 /Open-LLaVA-NeXT

Star427

An open-source implementation for training LLaVA-NeXT.

chatbot llama multimodal multi-modality gpt-4 visual-language-learning chatgpt vision-language-model llava large-multimodal-models llama3 gpt4o llava-next

UpdatedOct 23, 2024
Python

RLHF-V /RLHF-V

Star299

[CVPR'24] RLHF-V: Towards Trustworthy MLLMs via Behavior Alignment from Fine-grained Correctional Human Feedback

chatbot llama multimodal multi-modality gpt-4 visual-language-learning rlhf-v

UpdatedSep 11, 2024
Python

mlpc-ucsd /BLIVA

Star259

(AAAI 2024) BLIVA: A Simple Multimodal LLM for Better Handling of Text-rich Visual Questions

chatbot llama lora multimodal visual-language-learning llm instruction-tuning blip2 bliva

UpdatedApr 14, 2024
Python

thomas-yanxin /KarmaVLM

Star88

🧘🏻‍♂️KarmaVLM (相生)：A family of high efficiency and powerful visual language model.

vlm multimodel visual-language-learning vision-language-model llava llama2 qwen2

UpdatedApr 29, 2024
Python

AdrianBZG /llama-multimodal-vqa

Star50

Multimodal Instruction Tuning for Llama 3

chatbot vqa llama language-models visual-question-answering multimodal huggingface gpt-4 visual-language-learning chatgpt instruction-tuning multimodal-instruction-tuning llama2 llama3

UpdatedApr 25, 2024
Python

xinyanghuang7 /Basic-Visual-Language-Model

Star48

Build a simple basic multimodal large model from scratch. 从零搭建一个简单的基础多模态大模型🤖

visual-language-learning large-language-models visual-language-models multimodel-large-language-model

UpdatedJun 19, 2024
Python

Skyline-9 /Shotluck-Holmes

Star13

[ACM MMGR '24] 🔍 Shotluck Holmes: A family of small-scale LLVMs for shot-level video understanding

python nlp video-summarization video-captioning multi-modality visual-language-learning llm vision-language-model shotluck-holmes

UpdatedOct 26, 2024
Python

ashleykleynhans /llava-docker

Sponsor

Star4

Docker image for LLaVA: Large Language and Vision Assistant

docker ai docker-image chatbot llama multimodal gpt-4 foundation-models visual-language-learning llm runpod chatgpt instruction-tuning vision-language-model llava llama2 llama-2

UpdatedMay 17, 2025
Shell

MuhammadAliS /CLIP

Star3

PyTorch implementation of OpenAI's CLIP model for image classification, visual search, and visual question answering (VQA).

deep-neural-networks transformers visual-question-answering pytorch-implementation huggingface visual-language-learning

UpdatedSep 14, 2024
Jupyter Notebook

ecoxial2007 /EffVideoQA

Star1

Efficient Video Question Answering

computer-vision video-question-answering visual-language-learning

UpdatedJan 19, 2023
Python

Improve this page

Add a description, image, and links to thevisual-language-learning topic page so that developers can more easily learn about it.

Curate this topic

Add this topic to your repo

To associate your repository with thevisual-language-learning topic, visit your repo's landing page and select "manage topics."

Learn more

Movatterモバイル変換

Navigation Menu

Search code, repositories, users, issues, pull requests...

Provide feedback

Saved searches

Use saved searches to filter your results more quickly