Repository files navigation

• 2025/07:
  • ND Parallelism support has been added into Axolotl. Compose Context Parallelism (CP), Tensor Parallelism (TP), and Fully Sharded Data Parallelism (FSDP) within a single node and across multiple nodes. Check out theblog post for more info.
  • Axolotl adds more models:GPT-OSS,Gemma 3n,Liquid Foundation Model 2 (LFM2), andArcee Foundation Models (AFM).
  • FP8 finetuning with fp8 gather op is now possible in Axolotl viatorchao. Get startedhere!
  • Voxtral,Magistral 1.1, andDevstral with mistral-common tokenizer support has been integrated in Axolotl!
  • TiledMLP support for single-GPU to multi-GPU training with DDP, DeepSpeed and FSDP support has been added to support Arctic Long Sequence Training. (ALST). Seeexamples for using ALST with Axolotl!
• 2025/05: Quantization Aware Training (QAT) support has been added to Axolotl. Explore thedocs to learn more!
• 2025/03: Axolotl has implemented Sequence Parallelism (SP) support. Read theblog anddocs to learn how to scale your context length when fine-tuning.

Axolotl is a free and open-source tool designed to streamline post-training and fine-tuning for the latest large language models (LLMs).

Features:

• Multiple Model Support: Train various models like GPT-OSS, LLaMA, Mistral, Mixtral, Pythia, and many more models available on the Hugging Face Hub.
• Multimodal Training: Fine-tune vision-language models (VLMs) including LLaMA-Vision, Qwen2-VL, Pixtral, LLaVA, SmolVLM2, and audio models like Voxtral with image, video, and audio support.
• Training Methods: Full fine-tuning, LoRA, QLoRA, GPTQ, QAT, Preference Tuning (DPO, IPO, KTO, ORPO), RL (GRPO), and Reward Modelling (RM) / Process Reward Modelling (PRM).
• Easy Configuration: Re-use a single YAML configuration file across the full fine-tuning pipeline: dataset preprocessing, training, evaluation, quantization, and inference.
• Performance Optimizations:Multipacking,Flash Attention,Xformers,Flex Attention,Liger Kernel,Cut Cross Entropy,Sequence Parallelism (SP),LoRA optimizations,Multi-GPU training (FSDP1, FSDP2, DeepSpeed),Multi-node training (Torchrun, Ray), and many more!
• Flexible Dataset Handling: Load from local, HuggingFace, and cloud (S3, Azure, GCP, OCI) datasets.
• Cloud Ready: We shipDocker images and alsoPyPI packages for use on cloud platforms and local hardware.

Requirements:

• NVIDIA GPU (Ampere or newer forbf16 and Flash Attention) or AMD GPU
• Python 3.11
• PyTorch ≥2.7.1

pip3 install -U packaging==23.2 setuptools==75.8.0 wheel ninjapip3 install --no-build-isolation axolotl[flash-attn,deepspeed]# Download example axolotl configs, deepspeed configsaxolotl fetch examplesaxolotl fetch deepspeed_configs# OPTIONAL

Installing with Docker can be less error prone than installing in your own environment.

docker run --gpus'"all"' --rm -it axolotlai/axolotl:main-latest

Other installation approaches are describedhere.

# Fetch axolotl examplesaxolotl fetch examples# Or, specify a custom pathaxolotl fetch examples --dest path/to/folder# Train a model using LoRAaxolotl train examples/llama-3/lora-1b.yml

That's it! Check out ourGetting Started Guide for a more detailed walkthrough.

• Installation Options - Detailed setup instructions for different environments
• Configuration Guide - Full configuration options and examples
• Dataset Loading - Loading datasets from various sources
• Dataset Guide - Supported formats and how to use them
• Multi-GPU Training
• Multi-Node Training
• Multipacking
• API Reference - Auto-generated code documentation
• FAQ - Frequently asked questions

• Join ourDiscord community for support
• Check out ourExamples directory
• Read ourDebugging Guide
• Need dedicated support? Please contact✉️wing@axolotl.ai for options

Contributions are welcome! Please see ourContributing Guide for details.

Interested in sponsoring? Contact us atwing@axolotl.ai

If you use Axolotl in your research or projects, please cite it as follows:

@software{axolotl,title ={Axolotl: Open Source LLM Post-Training},author ={{Axolotl maintainers and contributors}},url ={https://github.com/axolotl-ai-cloud/axolotl},license ={Apache-2.0},year ={2023}}

This project is licensed under the Apache 2.0 License - see theLICENSE file for details.