llm-router

Star

Here are 17 public repositories matching this topic...

Language:All

Filter by language

All17 Python8 Go3 TypeScript2 Jupyter Notebook1 Rust1

Sort:Most stars

Sort options

Most stars Fewest stars Most forks Fewest forks Recently updated Least recently updated

NVIDIA-AI-Blueprints /llm-router

Star135

Route LLM requests to the best model for the task at hand.

llm-router

UpdatedNov 17, 2025
Jupyter Notebook

thushan /olla

Sponsor

Star119

High-performance lightweight proxy and load balancer for LLM infrastructure. Intelligent routing, automatic failover and unified model discovery across local and remote inference backends.

golang ai amd proxy intel self-hosted nvidia mlx llamacpp llama-cpp vllm llm-inference local-ai ollama llm-proxy lmstudio sglang llm-router llm-routing self-hosted-ai

UpdatedNov 23, 2025
Go

lordmathis /llamactl

Star53

Unified management and routing for llama.cpp, MLX and vLLM models with web dashboard.

self-hosted mlx openai-api llm llamacpp llama-cpp vllm llm-inference localllm localllama llama-server llm-router mlx-lm

UpdatedNov 25, 2025
Go

RouteWorks /RouterArena

Star43

RouterArena: An open framework for evaluating LLM routers with standardized datasets, metrics, an automated framework, and a live leaderboard.

arena routing multi-agent multi-agent-systems router-benchmark llm llm-router llm-routing router-evaluation router-leaderboard

UpdatedNov 28, 2025
Python

Fully agentic LLM orchestrator with autonomous decision-making. This agentic system self-discovers models, learns from usage, and adapts routing strategies. Save 67% on API costs through intelligent agentic behavior. Production-ready with monitoring and self-healing.

python machine-learning artificial-intelligence multi-model self-learning cost-optimization openai-api ai-agent llm autonomous-ai gpt-5 claude-ai claude-api ollama gemini-ai llm-router agentic-ai ai-orchestration model-routing

UpdatedNov 10, 2025
Python

llm-use /llm-use

Star12

Intelligent routing automatically selects the optimal model (GPT-4/Claude/Llama) for each prompt based on complexity. Production-ready with streaming, caching, and A/B testing.

python streaming monitoring api-gateway artificial-intelligence openai ab-testing claude cost-optimization fastapi groq llm langchain llmops gpt-5 anthropic llm-router ai-orchestration

UpdatedOct 9, 2025
Python

LM4eu /goinfer

Star8

Local LLM proxy, DevOps friendly

inference inference-server inference-api openai-api llm openaiapi llamacpp llama-cpp local-llm localllm local-ai llm-proxy llama-api llama-server llm-router language-model-api local-lm local-llm-integration

UpdatedNov 25, 2025
Go

samadpls /archdata

Star7

Open-source pipeline for generating and augmenting Arch-Router-style conversational routing datasets.

opensource dataset-generation llm-router archgw

UpdatedOct 27, 2025
Python

meet447 /Unio

Star2

your all in one llm api

api router openai openai-api llm llm-router

UpdatedNov 15, 2025
TypeScript

SeiyaKobayashi /credible-llm-markets

Star2

Securing markets of dLLMs (Decentralized LLMs) utilizing cryptographic technologies.

nlp machine-learning ethereum smart-contracts artificial-intelligence mpc cryptoeconomics fhe zkp tokenomics llm llm-router

UpdatedMay 29, 2024

radlab-dev-group /llm-router

Star2

LLM Router is a service that can be deployed on‑premises or in the cloud. It adds a layer between any application and the LLM provider. In real time it controls traffic, distributes a load among providers of a specific LLM, and enables analysis of outgoing requests from a security perspective (masking, anonymization, prohibited content).

automation cloud rest-api model-management load-balancing on-prem genai llm-router llm-gateway llm-gateway-system llm-balancing

UpdatedNov 28, 2025
Python

SayamAlt /Personalized-Health-and-Fitness-Coach-using-Langchain

Star1

Successfully developed an LLM application that provides AI-powered, structured insights based on user queries. The app features a dynamic response generator with progress indicators, interactive upvote/downvote options, and a clean, engaging user interface built using Streamlit. Ideal for personalized meal, fitness, and health-related advice.

streamlit-webapp openai-api prompt-engineering generative-ai langchain llm-chain llm-application chatopenai llm-router output-parsers llm-router-chain multi-prompt-chain

UpdatedMar 27, 2025
Python

bselleslagh /v-router

Star1

A lightweight AI model router for seamlessly switching between multiple AI providers (OpenAI, Anthropic, Google AI) with unified API interface.

python gemini openai developer-tools anthropic ai-gateway llm-router

UpdatedJun 13, 2025
Python

maariogutierrez /GreenChoiceAI

Star0

GreenChoiceAI routes questions to the most cost-efficient LLM based on predicted category and heuristic difficulty. Backend (FastAPI) computes difficulty, predicts category with sentence embeddings + SVM, selects a model using performance CSV, and returns concise LLM answers; frontend is a Vite+React chat UI.

react svm chatbot embeddings fastapi llm llm-router

UpdatedNov 22, 2025
TypeScript

dexmac221 /AiSociety

Star0

Hybrid LLM router: local+cloud models with meta-routing,memory, FastAPI UI, and FAISS; integrates Ollama and OpenAI.

web chat-application hybrid-application faiss fastapi llm llm-router prompt-optimizer

UpdatedSep 5, 2025
Python

m3mentomor1 /Input-based-LLM-Routing-Agent

Star0

A dynamic input-based LLM routing AI-agent built with n8n. It selects the most suitable language model based on user queries, and uses the selected model to answer, solve, or address the input accordingly.

ai-agents n8n llm-router