Foundation Model Inference

Inference Systems for Foundation Models

PinnedLoading

FlexLLMGenFlexLLMGenPublic archive
Running large language models on a single GPU for throughput-oriented scenarios.
Python 9.3k 565

Showing 3 of 3 repositories

FlexLLMGen Public archive
Running large language models on a single GPU for throughput-oriented scenarios.
FMInference/FlexLLMGen’s past year of commit activity
Python 9,278Apache-2.0 565 52 (3 issues need help) 6 UpdatedOct 28, 2024
H2O Public
[NeurIPS'23] H2O: Heavy-Hitter Oracle for Efficient Generative Inference of Large Language Models.
FMInference/H2O’s past year of commit activity
Python 432 54 33 1 UpdatedAug 1, 2024
DejaVu Public
FMInference/DejaVu’s past year of commit activity
Python 310 41 26 1 UpdatedApr 2, 2024

This organization has no public members. You must be a member to see who’s a part of this organization.

Loading…

Loading…

You can’t perform that action at this time.