Movatterモバイル変換

Skip to content

#

model-serving

Here are 166 public repositories matching this topic...

Language:All

Filter by language

All166 Python89 Jupyter Notebook22 Scala9 C++7 Go7 Java7 Rust3 TypeScript3 HTML2 JavaScript2

Sort:Most stars

Sort options

Most stars Fewest stars Most forks Fewest forks Recently updated Least recently updated

vllm-project /vllm

A high-throughput and memory-efficient inference and serving engine for LLMs

amd cuda inference pytorch transformer llama gpt rocm model-serving tpu hpu mlops xpu llm inferentia llmops llm-serving qwen deepseek trainium

UpdatedJul 19, 2025
Python

BentoML

bentoml /BentoML

The easiest way to serve AI apps and models - Build Model Inference APIs, Job queues, LLM apps, Multi-model pipelines, and more!

python machine-learning deep-learning model-serving multimodal mlops ml-engineering ai-inference llm generative-ai llmops llm-serving model-inference-service llm-inference inference-platform

UpdatedJul 18, 2025
Python

kserve /kserve

Standardized Serverless ML Inference Platform on Kubernetes

kubernetes machine-learning tensorflow sklearn pytorch artificial-intelligence xgboost k8s service-mesh hacktoberfest istio model-serving kubeflow mlops knative model-interpretability kserve genai llm-inference

UpdatedJul 16, 2025
Python

ahkarami /Deep-Learning-in-Production

In this repository, I will share some useful notes and references about deploying deep learning-based models in production.

react python angularjs flask c-plus-plus tutorial deep-neural-networks deep-learning mxnet rest-api production keras pytorch tensorflow-models tesnorflow model-serving caffe2 serving serving-pytorch-models convert-pytorch-models

UpdatedNov 9, 2024

FedML-AI /FedML

FEDML - The unified and scalable ML library for large-scale distributed training, model serving, and federated learning. FEDML Launch, a cross-cloud scheduler, further enables running any AI jobs on any GPU cloud or on-premise cluster. Built on this library, TensorOpera AI (https://TensorOpera.ai) is your generative AI platform at scale.

machine-learning deep-learning inference-engine model-deployment model-serving distributed-training federated-learning mlops edge-ai ai-agent on-device-training

UpdatedJul 7, 2025
Python

ModelTC /LightLLM

LightLLM is a Python-based LLM (Large Language Model) inference and serving framework, notable for its lightweight design, easy scalability, and high-speed performance.

nlp deep-learning llama gpt model-serving llm openai-triton

UpdatedJul 18, 2025
Python

predibase /lorax

Multi-LoRA inference server that scales to 1000s of fine-tuned LLMs

transformers pytorch llama gpt lora model-serving fine-tuning llm llmops llm-serving llm-inference

UpdatedMay 21, 2025
Python

HuaizhengZhang /AI-Infra-from-Zero-to-Hero

🚀 Awesome System for Machine Learning ⚡️ AI System Papers and Industry Practice. ⚡️ System for Machine Learning, LLM (Large Language Model), GenAI (Generative AI). 🍻 OSDI, NSDI, SIGCOMM, SoCC, MLSys, etc. 🗃️ Llama3, Mistral, etc. 🧑‍💻 Video Tutorials.

model-serving model-training mlsys ai-infra large-language-models genai llmsys

UpdatedMay 24, 2025

tensorchord /envd

🏕️ Reproducible development environment

docker developer-tools development-environment hacktoberfest model-serving buildkit mlops mlops-workflow llmops

UpdatedJul 1, 2025
Go

Olares

beclab /Olares

Olares: An Open-Source Personal Cloud to Reclaim Your Data

kubernetes home-automation mcp homeserver self-hosted homelab personal-cloud ai-agents model-serving edge-ai home-server ai-privacy home-cloud local-ai

UpdatedJul 19, 2025
Go

microsoft /aici

AICI: Prompts as (Wasm) Programs

rust ai wasm inference transformer language-model model-serving wasmtime llm llmops llm-serving llm-inference llm-framework

UpdatedJan 22, 2025
Rust

mlrun /mlrun

MLRun is an open source MLOps platform for quickly building and managing continuous ML applications across their lifecycle. MLRun integrates into your development and CI/CD environment and automates the delivery of production data, ML pipelines, and online applications.

python kubernetes workflow data-science machine-learning data-engineering model-serving mlops experiment-tracking mlops-workflow

UpdatedJul 18, 2025
Python

logicalclocks /hopsworks

Hopsworks - Data-Intensive AI platform with a Feature Store

python aws data-science machine-learning serverless azure gcp ml pyspark feature-engineering governance model-serving mlops feature-store feature-management hopsworks kserve

UpdatedFeb 10, 2025
Java

thu-pacman /chitu

High-performance inference framework for large language models, focusing on efficiency, flexibility, and availability.

gpu pytorch model-serving llm llm-serving deepseek

UpdatedJul 15, 2025
Python

kitops

kitops-ml /kitops

An open source DevOps tool for packaging and versioning AI/ML models, datasets, code, and configuration into an OCI artifact.

kubernetes devops opensource ai code tensorflow sklearn models ml pytorch datasets devops-tools hacktoberfest kubernetes-deployment model-serving platform-engineering mlops model-interpretability gguf mlops-tools

UpdatedJul 17, 2025
Go

truss

basetenlabs /truss

The simplest way to serve AI/ML models in production

open-source machine-learning packaging artificial-intelligence falcon easy-to-use whisper inference-server model-serving inference-api stable-diffusion wizardlm

UpdatedJul 19, 2025
Python

zhihu /ZhiLight

A highly optimized LLM inference acceleration engine for Llama and its variants.

cuda pytorch llama gpt inference-engine model-serving llm llm-serving llm-inference deepseek-r1

UpdatedJul 10, 2025
C++

vllm-project /vllm-ascend

Community maintained hardware plugin for vLLM on Ascend

inference transformer model-serving mlops ascend llm llmops llm-serving vllm

UpdatedJul 19, 2025
Python

mosec

mosecorg /mosec

A high-performance ML model serving framework, offers dynamic batching and CPU/GPU pipelines to fully exploit your compute machine

python rust machine-learning deep-learning mxnet tensorflow gpu cv pytorch tts hacktoberfest model-serving nerual-network machine-learning-platform jax mlops llm llm-serving

UpdatedJul 11, 2025
Python

efeslab /Nanoflow

A throughput-oriented high-performance serving framework for LLMs

cuda inference model-serving llm llm-serving llama2

UpdatedJul 9, 2025
Jupyter Notebook

Improve this page

Add a description, image, and links to themodel-serving topic page so that developers can more easily learn about it.

Curate this topic

Add this topic to your repo

To associate your repository with themodel-serving topic, visit your repo's landing page and select "manage topics."

[8]ページ先頭

©2009-2025 Movatter.jp