Popular repositoriesLoading

1. gptqgptqPublic

   Code for the ICLR 2023 paper "GPTQ: Accurate Post-training Quantization of Generative Pretrained Transformers".

   Python 2.1k 164

2. sparsegptsparsegptPublic

   Code for the ICML 2023 paper "SparseGPT: Massive Language Models Can Be Accurately Pruned in One-Shot".

   Python 776 102

3. marlinmarlinPublic

   FP16xINT4 LLM inference kernel that can achieve near-ideal ~4x speedups up to medium batchsizes of 16-32 tokens.

   Python 771 62

4. PanzaMailPanzaMailPublic

   Python 284 17

5. qmoeqmoePublic

   Code for the paper "QMoE: Practical Sub-1-Bit Compression of Trillion-Parameter Models".

   Python 271 22

6. QUIKQUIKPublic

   Repository for the QUIK project, enabling the use of 4bit kernels for generative inference - EMNLP 2024

   C++ 177 14