SOTA low-bit LLM quantization (INT8/FP8/INT4/FP4/NF4) & sparsity; leading model compression techniques on TensorFlow, PyTorch, and ONNX Runtime

• UpdatedMar 18, 2025
• Python

Large Language Models for All, 🦙 Cult and More, Stay in touch !

• UpdatedJun 1, 2023
• HTML

🦖 X—LLM: Cutting Edge & Easy LLM Finetuning

• UpdatedJan 17, 2024
• Python

Advanced Quantization Algorithm for LLMs/VLMs.

• UpdatedMar 18, 2025
• Python

Production ready LLM model compression/quantization toolkit with hw accelerated inference support for both cpu/gpu via HF, vLLM, and SGLang.

• UpdatedMar 17, 2025
• Python

Run any Large Language Model behind a unified API

• UpdatedNov 13, 2023
• Python

🪶 Lightweight OpenAI drop-in replacement for Kubernetes

• UpdatedFeb 5, 2024
• Python

A guide about how to use GPTQ models with langchain

• UpdatedAug 19, 2023
• Jupyter Notebook

zero零训练llm调参

• UpdatedJul 20, 2023

ChatSakura：Open-source multilingual conversational model.（开源多语言对话大模型）

• UpdatedApr 2, 2023
• Python

An OpenAI Compatible API which integrates LLM, Embedding and Reranker. 一个集成 LLM、Embedding 和 Reranker 的 OpenAI 兼容 API

• UpdatedJul 13, 2024
• Python

Private self-improvement coaching with open-source LLMs

• UpdatedMar 7, 2024
• Python

This repository is for profiling, extracting, visualizing and reusing generative AI weights to hopefully build more accurate AI models and audit/scan weights at rest to identify knowledge domains for risk(s).

• UpdatedDec 18, 2023
• Python

Run gguf LLM models in Latest Version TextGen-webui

• UpdatedOct 11, 2024
• Jupyter Notebook

A.L.I.C.E (Artificial Labile Intelligence Cybernated Existence). A REST API of A.I companion for creating more complex system

• UpdatedFeb 6, 2025
• Python

Conversation AI model for open domain dialogs

• UpdatedNov 15, 2023
• Python

This project will develop a NEPSE chatbot using an open-source LLM, incorporating sentence transformers, vector database and reranking.

• UpdatedDec 31, 2023
• Jupyter Notebook

LLM quantization techniques: absmax, zero-point, GPTQ and GGUF

• UpdatedAug 2, 2024
• Jupyter Notebook

Quantizing LLMs using GPTQ

• UpdatedDec 31, 2023
• Jupyter Notebook

Personal GitHub repository for stashing resources on Large Language Models (LLM), including Jupyter Notebooks on open source LLMs, use-cases with Langchain and R&D paper review.

• UpdatedJun 20, 2023
• Jupyter Notebook