gptq

Star

Here are 26 public repositories matching this topic...

Language:All

Filter by language

All26 Python16 Jupyter Notebook9 HTML1

Sort:Most stars

Sort options

Most stars Fewest stars Most forks Fewest forks Recently updated Least recently updated

intel /neural-compressor

Star2.5k

SOTA low-bit LLM quantization (INT8/FP8/MXFP8/INT4/MXFP4/NVFP4) & sparsity; leading model compression techniques on PyTorch, TensorFlow, and ONNX Runtime

sparsity pruning quantization knowledge-distillation auto-tuning int8 low-precision quantization-aware-training post-training-quantization awq int4 large-language-models gptq smoothquant sparsegpt fp4 mxformat

UpdatedDec 18, 2025
Python

ModelCloud /GPTQModel

Star933

LLM model quantization (compression) toolkit with hw acceleration support for Nvidia CUDA, AMD ROCm, Intel XPU and Intel/AMD/Apple CPU via HF, vLLM, and SGLang.

transformers quantization optimum peft vllm gptq sglang

UpdatedDec 18, 2025
Python

shm007g /LLaMA-Cult-and-More

Star450

Large Language Models for All, 🦙 Cult and More, Stay in touch !

tensorflow transformers pytorch llama gpt alpaca loralib vicuna deepspeed gpt4 llm chatgpt ggml gptq

UpdatedJun 1, 2023
HTML

bobazooba /xllm

Star408

🦖 X—LLM: Cutting Edge & Easy LLM Finetuning

deep-neural-networks deep-learning torch pytorch openai llama gpt alpaca zephyr mistral vicuna gpt-4 large-language-models llm chatgpt cerebras gptq bitsandbytes llama2

UpdatedJan 17, 2024
Python

1b5d /llm-api

Star170

Run any Large Language Model behind a unified API

python machine-learning llama huggingface llm chatgpt langchain llamacpp gptq llm-inference

UpdatedNov 13, 2023
Python

chenhunghan /ialacol

Star146

🪶 Lightweight OpenAI drop-in replacement for Kubernetes

python kubernetes ai gpu helm cuda openai cloudnative llm langchain llm-serving llamacpp ggml gptq llm-inference

UpdatedFeb 5, 2024
Python

taishan1994 /LLM-Quantization

Star49

记录量化LLM中的总结。

quantization llm gptq quarot qwen3

UpdatedDec 16, 2025
Python

abhinand5 /gptq_for_langchain

Star40

A guide about how to use GPTQ models with langchain

ai gpt quantization language-model llm langchain gptq wizardlm

UpdatedAug 19, 2023
Jupyter Notebook

seyf1elislam /LocalLLM_OneClick_Colab

Star19

Run gguf LLM models in Latest Version TextGen-webui and koboldcpp

python colab-notebook llm llms gptq localllm exllama gguf localllama

UpdatedAug 6, 2025
Jupyter Notebook

hcd233 /Aris-AI-Model-Server

Star18

An OpenAI Compatible API which integrates LLM, Embedding and Reranker. 一个集成 LLM、Embedding 和 Reranker 的 OpenAI 兼容 API

ai embedding mlx reranker rag fastapi sentence-transformers awq llm vllm gptq openai-compatible-api

UpdatedAug 21, 2025
Python

tripathiarpan20 /self-improvement-4all

Star15

Private self-improvement coaching with open-source LLMs

python transformers faiss langchain text-generation-webui gptq

UpdatedMar 7, 2024
Python

chinoll /chatsakura

Star13

ChatSakura：Open-source multilingual conversational model.（开源多语言对话大模型）

bloom transformers pytorch gradio llm chatgpt bloomz instruct-gpt gptq

UpdatedApr 2, 2023
Python

This repository is for profiling, extracting, visualizing and reusing generative AI weights to hopefully build more accurate AI models and audit/scan weights at rest to identify knowledge domains for risk(s).

ai deep-learning blender tiff transformers weights image-to-image blender-python llm stable-diffusion foundational-models generative-ai safetensors blip2 gptq

UpdatedDec 18, 2023
Python

Aqirito /A.L.I.C.E

Star9

A.L.I.C.E (Artificial Labile Intelligence Cybernated Existence). A REST API of A.I companion for creating more complex system

text-to-speech anime rest-api text-generation artificial-intelligence tts waifu otaku pygmalion fastapi huggingface-transformers genshin-impact vits llm llms langchain gptq langchain-python exllama

UpdatedFeb 6, 2025
Python

Abeshith /FineTuning_LanguageModels

Star4

🎯 Fine-tune large language models and use them for text-related tasks. This repository provides a straightforward approach to fine-tuning models like Gemma, Llama 🦙, and Mistral 🌪️ for various NLP tasks. 🔧 It includes training 📚, fine-tuning 🛠️, and inference pipelines ⚙️. 🚀

transformer lora quantization knowledge-distillation finetuning quantization-aware-training post-quantization ptq large-language-models gptq bitsandbytes unsloth multi-lora