inference-server
Here are 52 public repositories matching this topic...
Language:All
Sort:Most stars
⚡ Python-free Rust inference server — OpenAI-API compatible. GGUF + SafeTensors, hot model swap, auto-discovery, single binary. FREE now, FREE forever.
- Updated
Dec 17, 2025 - Rust
RamaLama is an open-source developer tool that simplifies the local serving of AI models from any source and facilitates their use for inference in production, all through the familiar language of containers.
- Updated
Dec 17, 2025 - Python
Turn any computer or edge device into a command center for your computer vision projects.
- Updated
Dec 17, 2025 - Python
The simplest way to serve AI/ML models in production
- Updated
Dec 17, 2025 - Python
An open-source computer vision framework to build and deploy apps in minutes
- Updated
May 8, 2024 - Rust
Python + Inference - Model Deployment library in Python. Simplest model inference server ever.
- Updated
Feb 14, 2023 - Python
A REST API for Caffe using Docker and Go
- Updated
Jul 20, 2018 - C++
This is a repository for an nocode object detection inference API using the Yolov3 and Yolov4 Darknet framework.
- Updated
Jun 28, 2022 - Python
Work with LLMs on a local environment using containers
- Updated
Dec 17, 2025 - TypeScript
This is a repository for an nocode object detection inference API using the Yolov4 and Yolov3 Opencv.
- Updated
Jun 28, 2022 - Python
This is a repository for an object detection inference API using the Tensorflow framework.
- Updated
Jun 28, 2022 - Python
ONNX Runtime Server: The ONNX Runtime Server is a server that provides TCP and HTTP/HTTPS REST APIs for ONNX inference.
- Updated
Oct 30, 2025 - C++
Serving AI/ML models in the open standard formats PMML and ONNX with both HTTP (REST API) and gRPC endpoints
- Updated
Dec 17, 2025 - Scala
Orkhon: ML Inference Framework and Server Runtime
- Updated
Feb 1, 2021 - Rust
K3ai is a lightweight, fully automated, AI infrastructure-in-a-box solution that allows anyone to experiment quickly with Kubeflow pipelines. K3ai is perfect for anything from Edge to laptops.
- Updated
Nov 2, 2021 - PowerShell
Deploy DL/ ML inference pipelines with minimal extra code.
- Updated
Nov 20, 2024 - Python
A standalone inference server for trained Rubix ML estimators.
- Updated
Mar 28, 2025 - PHP
[⛔️ DEPRECATED] Friendli: the fastest serving engine for generative AI
- Updated
Jun 25, 2025 - Python
Wingman is the fastest and easiest way to run Llama models on your PC or Mac.
- Updated
Jun 2, 2024 - TypeScript
Advanced inference pipeline using NVIDIA Triton Inference Server for CRAFT Text detection (Pytorch), included converter from Pytorch -> ONNX -> TensorRT, Inference pipelines (TensorRT, Triton server - multi-format). Supported model format for Triton inference: TensorRT engine, Torchscript, ONNX
- Updated
Aug 18, 2021 - Python
Improve this page
Add a description, image, and links to theinference-server topic page so that developers can more easily learn about it.
Add this topic to your repo
To associate your repository with theinference-server topic, visit your repo's landing page and select "manage topics."