A highly optimized LLM inference acceleration engine for Llama and its variants.

TLLM_QMM strips the implementation of quantized kernels of Nvidia's TensorRT-LLM, removing NVInfer dependency and exposes ease of use Pytorch module. We modified the dequantation and weight preprocessing to align with popular quantization alogirthms such as AWQ and GPTQ, and combine them with new FP8 quantization.

An orm library support nGQL for Golang

A React-based web video player

Rust port of Lucene

🎆 A well-designed local image and video selector for Android

Redis sharding client library

Zetta Table Store

Go client for Zetta Table Store