Movatterモバイル変換

Skip to content

#

llm-inference

Here are 1,377 public repositories matching this topic...

Language:All

Filter by language

All1,377 Python694 Jupyter Notebook200 TypeScript80 JavaScript49 C++47 HTML42 Go31 Rust23 Shell21 Java14

Sort:Most stars

Sort options

Most stars Fewest stars Most forks Fewest forks Recently updated Least recently updated

nomic-ai /gpt4all

GPT4All: Run Local LLMs on Any Device. Open-source and available for commercial use.

ai-chat llm-inference

UpdatedMay 27, 2025
C++

ray-project /ray

Ray is an AI compute engine. Ray consists of a core distributed runtime and a set of AI Libraries for accelerating ML workloads.

python data-science machine-learning reinforcement-learning deep-learning deployment tensorflow optimization parallel pytorch distributed hyperparameter-optimization ray hyperparameter-search serving rllib large-language-models llm llm-serving llm-inference

UpdatedDec 17, 2025
Python

gitleaks /gitleaks

Find secrets with Gitleaks 🔑

git go cli golang open-source security secret ci-cd cicd hacktoberfest dlp security-tools devsecops data-loss-prevention gitleaks ai-powered nhi llm llm-training llm-inference

UpdatedDec 9, 2025
Go

liguodongiot /llm-action

本项目旨在分享大模型相关技术原理以及实战经验（大模型工程化、大模型应用落地）

llm llmops llm-serving llm-training llm-inference

UpdatedDec 3, 2025
HTML

Lightning-AI /litgpt

20+ high-performance LLMs with recipes to pretrain, finetune and deploy at scale.

ai deep-learning artificial-intelligence large-language-models llm llms llm-inference

UpdatedDec 17, 2025
Python

bentoml /OpenLLM

Run any open-source LLMs, such as DeepSeek and Llama, as OpenAI compatible API endpoint in the cloud.

llama mistral fine-tuning mlops bentoml vicuna llm model-inference llmops llm-serving llm-inference open-source-llm llama2 openllm llm-ops llama3-1 llama3-2 llama3-2-vision

UpdatedDec 15, 2025
Python

mistralai /mistral-inference

Official inference library for Mistral models

llm llm-inference mistralai

UpdatedNov 21, 2025
Jupyter Notebook

openvinotoolkit /openvino

OpenVINO™ is an open source toolkit for optimizing and deploying AI inference

nlp natural-language-processing ai computer-vision deep-learning transformers inference speech-recognition yolo recommendation-system performance-boost good-first-issue openvino diffusion-models stable-diffusion generative-ai llm-inference optimize-ai deploy-ai

UpdatedDec 17, 2025
C++

SJTU-IPADS /PowerInfer

High-speed Large Language Model Serving for Local Deployment

llama large-language-models llm local-inference llm-inference

UpdatedAug 2, 2025
C++

BentoML

bentoml /BentoML

The easiest way to serve AI apps and models - Build Model Inference APIs, Job queues, LLM apps, Multi-model pipelines, and more!

python machine-learning deep-learning model-serving multimodal mlops ml-engineering ai-inference llm generative-ai llmops llm-serving model-inference-service llm-inference inference-platform

UpdatedDec 15, 2025
Python

InternLM /lmdeploy

LMDeploy is a toolkit for compressing, deploying, and serving LLMs.

llama cuda-kernels deepspeed llm fastertransformer llm-inference turbomind internlm llama2 codellama llama3

UpdatedDec 17, 2025
Python

superduper

superduper-io /superduper

Superduper: End-to-end framework for building custom AI applications and agents.

python data database ai mongodb chatbot ml transformers inference torch pytorch pretrained-models semantic-search rag mlops distributed-ml vector-search llmops llm-serving llm-inference

UpdatedSep 1, 2025
Python

kserve /kserve

Standardized Distributed Generative and Predictive AI Inference Platform for Scalable, Multi-Framework Deployment on Kubernetes

kubernetes machine-learning tensorflow cncf pytorch artificial-intelligence xgboost k8s service-mesh hacktoberfest istio model-serving kubeflow mlops knative model-interpretability kserve vllm genai llm-inference

UpdatedDec 16, 2025
Shell

algorithmicsuperintelligence /openevolve

Open-source implementation of AlphaEvolve

genetic-algorithm discovery optimize evolutionary-algorithms deepmind-lab deepmind iterative-methods genetic-algorithms evolutionary-computation alphacode distributed-evolutionary-algorithms iterative-refinement llm-inference llm-engineering llm-ensemble coding-agent alpha-evolve alphaevolve openevolve

UpdatedDec 15, 2025
Python

Awesome-LLM-Inference

xlite-dev /Awesome-LLM-Inference

📚A curated list of Awesome LLM/VLM Inference Papers with Codes: Flash-Attention, Paged-Attention, WINT8/4, Parallelism, etc.🎉

mla vllm llm-inference awesome-llm flash-attention tensorrt-llm paged-attention deepseek flash-attention-3 deepseek-v3 minimax-01 deepseek-r1 flash-mla qwen3

UpdatedNov 28, 2025
Python

FellouAI /eko

Eko (Eko Keeps Operating) - Build Production-ready Agentic Workflow with Natural Language - eko.fellou.ai

agent workflow agents natural-language-inference browser-automation ai-agents rag computer-automation prompt-engineering chain-of-thought genai llm-inference llm-agents llmapi agentic-framework agentic-workflow agentic-ai agentic-ai-development computeruse browseruse

UpdatedDec 17, 2025
TypeScript

katanemo /archgw

Delivery infrastructure for agents. Arch is a models-native proxy and data plane for agents that handles plumbing work in AI - like agent routing and orchestration, guardrails, zero-code logs and traces, and unified access to LLMs (OpenAI, Anthropic, Ollama, etc.). Build agents faster and deliver them reliably to prod.

proxy routing gateway prompt proxy-server openai envoy envoyproxy llms generative-ai llmops llm-inference llm-proxy ai-gateway llm-gateway llm-routing ai-gateway-support

UpdatedDec 17, 2025
Rust

flashinfer-ai /flashinfer

FlashInfer: Kernel Library for LLM Serving

gpu cuda jit pytorch nvidia moe attention llm-inference large-large-models distributed-inference

UpdatedDec 17, 2025
Cuda

gpustack /gpustack

GPU cluster manager for optimized AI model deployment

cuda inference openai llama maas rocm ascend llm llm-serving vllm genai llm-inference qwen deepseek sglang distributed-inference high-performance-inference mindie

UpdatedDec 17, 2025
Python

cactus-compute /cactus

Kernels & AI inference engine for mobile devices.

android ios mobile framework ai edge transformer smartphone llm llms llamacpp llm-inference

UpdatedDec 17, 2025
C++

Improve this page

Add a description, image, and links to thellm-inference topic page so that developers can more easily learn about it.

Curate this topic

Add this topic to your repo

To associate your repository with thellm-inference topic, visit your repo's landing page and select "manage topics."

[8]ページ先頭

©2009-2025 Movatter.jp