Repository files navigation

💡 Get help -❓FAQ💭Discussions💬 Discord📖 Documentation website

💻 Quickstart🖼️ Models🚀 Roadmap🥽 Demo🌍 Explorer🛫 Examples Try on

LocalAI is the free, Open Source OpenAI alternative. LocalAI act as a drop-in replacement REST API that's compatible with OpenAI (Elevenlabs, Anthropic... ) API specifications for local AI inferencing. It allows you to run LLMs, generate images, audio (and not only) locally or on-prem with consumer grade hardware, supporting multiple model families. Does not require GPU. It is created and maintained byEttore Di Giacinto.

🆕 LocalAI is now part of a comprehensive suite of AI tools designed to work together:

	LocalAGI A powerful Local AI agent management platform that serves as a drop-in replacement for OpenAI's Responses API, enhanced with advanced agentic capabilities.
	LocalRecall A REST-ful API and knowledge base management system that provides persistent memory and storage capabilities for AI agents.

Talk Interface	Generate Audio

Models Overview	Generate Images

Chat Interface	Home

Login	Swarm

Run the installer script:

# Basic installationcurl https://localai.io/install.sh| sh

For more installation options, seeInstaller Options.

Or run with docker:

docker run -ti --name local-ai -p 8080:8080 localai/localai:latest

# CUDA 12.0docker run -ti --name local-ai -p 8080:8080 --gpus all localai/localai:latest-gpu-nvidia-cuda-12# CUDA 11.7docker run -ti --name local-ai -p 8080:8080 --gpus all localai/localai:latest-gpu-nvidia-cuda-11# NVIDIA Jetson (L4T) ARM64docker run -ti --name local-ai -p 8080:8080 --gpus all localai/localai:latest-nvidia-l4t-arm64

docker run -ti --name local-ai -p 8080:8080 --device=/dev/kfd --device=/dev/dri --group-add=video localai/localai:latest-gpu-hipblas

# Intel GPU with FP16 supportdocker run -ti --name local-ai -p 8080:8080 --device=/dev/dri/card1 --device=/dev/dri/renderD128 localai/localai:latest-gpu-intel-f16# Intel GPU with FP32 supportdocker run -ti --name local-ai -p 8080:8080 --device=/dev/dri/card1 --device=/dev/dri/renderD128 localai/localai:latest-gpu-intel-f32

docker run -ti --name local-ai -p 8080:8080 localai/localai:latest-gpu-vulkan

# CPU versiondocker run -ti --name local-ai -p 8080:8080 localai/localai:latest-aio-cpu# NVIDIA CUDA 12 versiondocker run -ti --name local-ai -p 8080:8080 --gpus all localai/localai:latest-aio-gpu-nvidia-cuda-12# NVIDIA CUDA 11 versiondocker run -ti --name local-ai -p 8080:8080 --gpus all localai/localai:latest-aio-gpu-nvidia-cuda-11# Intel GPU versiondocker run -ti --name local-ai -p 8080:8080 localai/localai:latest-aio-gpu-intel-f16# AMD GPU versiondocker run -ti --name local-ai -p 8080:8080 --device=/dev/kfd --device=/dev/dri --group-add=video localai/localai:latest-aio-gpu-hipblas

For more information about the AIO images and pre-downloaded models, seeContainer Documentation.

To load models:

# From the model gallery (see available models with `local-ai models list`, in the WebUI from the model tab, or visiting https://models.localai.io)local-ai run llama-3.2-1b-instruct:q4_k_m# Start LocalAI with the phi-2 model directly from huggingfacelocal-ai run huggingface://TheBloke/phi-2-GGUF/phi-2.Q8_0.gguf# Install and run a model from the Ollama OCI registrylocal-ai run ollama://gemma:2b# Run a model from a configuration filelocal-ai run https://gist.githubusercontent.com/.../phi-2.yaml# Install and run a model from a standard OCI registry (e.g., Docker Hub)local-ai run oci://localai/phi-2:latest

For more information, see💻 Getting started

• June 2025:Backend management has been added. Attention: extras images are going to be deprecated from the next release! Readthe backend management PR.
• May 2025:Audio input andReranking in llama.cpp backend,Realtime API, Support to Gemma, SmollVLM, and more multimodal models (available in the gallery).
• May 2025: Important: image name changesSee release
• Apr 2025: Rebrand, WebUI enhancements
• Apr 2025:LocalAGI andLocalRecall join the LocalAI family stack.
• Apr 2025: WebUI overhaul, AIO images updates
• Feb 2025: Backend cleanup, Breaking changes, new backends (kokoro, OutelTTS, faster-whisper), Nvidia L4T images
• Jan 2025: LocalAI model release:https://huggingface.co/mudler/LocalAI-functioncall-phi-4-v0.3, SANA support in diffusers:#4603
• Dec 2024: stablediffusion.cpp backend (ggml) added (#4289 )
• Nov 2024: Bark.cpp backend added (#4287 )
• Nov 2024: Voice activity detection models (VAD) added to the API:#4204
• Oct 2024: examples moved toLocalAI-examples
• Aug 2024: 🆕 FLUX-1,P2P Explorer
• July 2024: 🔥🔥 🆕 P2P Dashboard, LocalAI Federated mode and AI Swarms:#2723. P2P Global community pools:#3113
• May 2024: 🔥🔥 Decentralized P2P llama.cpp:#2343 (peer2peer llama.cpp!) 👉 Docshttps://localai.io/features/distribute/
• May 2024: 🔥🔥 Distributed inferencing:#2324
• April 2024: Reranker API:#2121

Roadmap items:List of issues

• 🧩Backend Gallery: Install/remove backends on the fly, powered by OCI images — fully customizable and API-driven.
• 📖Text generation with GPTs (llama.cpp,transformers,vllm ...📖 and more)
• 🗣Text to Audio
• 🔈Audio to Text (Audio transcription withwhisper.cpp)
• 🎨Image generation
• 🔥OpenAI-alike tools API
• 🧠Embeddings generation for vector databases
• ✍️Constrained grammars
• 🖼️Download Models directly from Huggingface
• 🥽Vision API
• 📈Reranker API
• 🆕🖧P2P Inferencing
• Agentic capabilities
• 🔊 Voice activity detection (Silero-VAD support)
• 🌍 Integrated WebUI!

Build and deploy custom containers:

• https://github.com/sozercan/aikit

WebUIs:

• https://github.com/Jirubizu/localai-admin
• https://github.com/go-skynet/LocalAI-frontend
• QA-Pilot(An interactive chat project that leverages LocalAI LLMs for rapid understanding and navigation of GitHub code repository)https://github.com/reid41/QA-Pilot

Model galleries

• https://github.com/go-skynet/model-gallery

Other:

• Helm charthttps://github.com/go-skynet/helm-charts
• VSCode extensionhttps://github.com/badgooooor/localai-vscode-plugin
• Langchain:https://python.langchain.com/docs/integrations/providers/localai/
• Terminal utilityhttps://github.com/djcopley/ShellOracle
• Local Smart assistanthttps://github.com/mudler/LocalAGI
• Home Assistanthttps://github.com/sammcj/homeassistant-localai /https://github.com/drndos/hass-openai-custom-conversation /https://github.com/valentinfrlch/ha-gpt4vision
• Discord bothttps://github.com/mudler/LocalAGI/tree/main/examples/discord
• Slack bothttps://github.com/mudler/LocalAGI/tree/main/examples/slack
• Shell-Pilot(Interact with LLM using LocalAI models via pure shell scripts on your Linux or MacOS system)https://github.com/reid41/shell-pilot
• Telegram bothttps://github.com/mudler/LocalAI/tree/master/examples/telegram-bot
• Another Telegram Bothttps://github.com/JackBekket/Hellper
• Auto-documentationhttps://github.com/JackBekket/Reflexia
• Github bot which answer on issues, with code and documentation as contexthttps://github.com/JackBekket/GitHelper
• Github Actions:https://github.com/marketplace/actions/start-localai
• Examples:https://github.com/mudler/LocalAI/tree/master/examples/

• LLM finetuning guide
• How to build locally
• How to install in Kubernetes
• Projects integrating LocalAI
• How tos section (curated by our community)

• Run Visual studio code with LocalAI (SUSE)
• 🆕Run LocalAI on Jetson Nano Devkit
• Run LocalAI on AWS EKS with Pulumi
• Run LocalAI on AWS
• Create a slackbot for teams and OSS projects that answer to documentation
• LocalAI meets k8sgpt
• Question Answering on Documents locally with LangChain, LocalAI, Chroma, and GPT4All
• Tutorial to use k8sgpt with LocalAI

If you utilize this repository, data in a downstream project, please consider citing it with:

@misc{localai,  author = {Ettore Di Giacinto},  title = {LocalAI: The free, Open source OpenAI alternative},  year = {2023},  publisher = {GitHub},  journal = {GitHub repository},  howpublished = {\url{https://github.com/go-skynet/LocalAI}},

Do you find LocalAI useful?

Support the project by becominga backer or sponsor. Your logo will show up here with a link to your website.

A huge thank you to our generous sponsors who support this project covering CI expenses, and ourSponsor list:

LocalAI is a community-driven project created byEttore Di Giacinto.

MIT - Author Ettore Di Giacintomudler@localai.io

LocalAI couldn't have been built without the help of great software already available from the community. Thank you!

• llama.cpp
• https://github.com/tatsu-lab/stanford_alpaca
• https://github.com/cornelk/llama-go for the initial ideas
• https://github.com/antimatter15/alpaca.cpp
• https://github.com/EdVince/Stable-Diffusion-NCNN
• https://github.com/ggerganov/whisper.cpp
• https://github.com/rhasspy/piper

This is a community project, a special thanks to our contributors! 🤗