Production ready LLM model compression/quantization toolkit with hw accelerated inference support for both cpu/gpu via HF, vLLM, and SGLang.

A (nicer) tokenizer you want to use for model inference and training: with all known peventable gotchas normalized or auto-fixed.

A framework for few-shot evaluation of language models.

🤗 PEFT: State-of-the-art Parameter-Efficient Fine-Tuning.

SGLang is a fast serving framework for large language models and vision language models.

Self-contained Python lib with zero-dependencies that give you a unified device properties for gpu, cpu, and npu. No more calling separate tools such as nvidia-smi or /proc/cpuinfo and parsing it yourself.

A unified Logger and ProgressBar util with zero dependencies.

🤗 Transformers: State-of-the-art Machine Learning for Pytorch, TensorFlow, and JAX.