🦍 The Cloud-Native API Gateway and AI Gateway.

• UpdatedMar 23, 2025
• Lua

Python SDK, Proxy Server (LLM Gateway) to call 100+ LLM APIs in OpenAI format - [Bedrock, Azure, OpenAI, VertexAI, Cohere, Anthropic, Sagemaker, HuggingFace, Replicate, Groq]

• UpdatedMar 23, 2025
• Python

A blazing fast AI Gateway with integrated guardrails. Route to 200+ LLMs, 50+ AI Guardrails with 1 fast & friendly API.

• UpdatedMar 22, 2025
• TypeScript

AI模型接口管理与分发系统，支持将多种大模型转为统一格式调用，支持OpenAI、Claude等格式，可供个人或者企业内部管理与分发渠道使用，本项目基于One API二次开发。🍥 The next-generation LLM gateway and AI asset management system supports multiple languages.

• UpdatedMar 20, 2025
• Go

🤖 AI Gateway | AI Native API Gateway

• UpdatedMar 23, 2025
• Go

The AI-native proxy server for agents. Arch handles the pesky heavy lifting in building agentic apps - routing prompts to agents or specifc tools, clarifying user inputs, unifying access and observability to any LLM - so you can build smarter and ship faster.

• UpdatedMar 21, 2025
• Rust

🦄云原生、超高性能 AI&API网关，LLM API 管理、分发系统、开放平台，支持所有AI API，不限于OpenAI、Azure、Anthropic Claude、Google Gemini、DeepSeek、字节豆包、ChatGLM、文心一言、讯飞星火、通义千问、360 智脑、腾讯混元等主流模型，统一 API 请求和返回，API申请与审批，调用统计、负载均衡、多模型灾备。一键部署，开箱即用。Cloud native, ultra-high performance AI&API gateway, LLM API management, distribution system, open platform, supporting all AI APIs.

• UpdatedMar 19, 2025
• TypeScript

CodeGate: Security, Workspaces and Muxing for AI Applications, coding assistants, and agentic frameworks.

• UpdatedMar 23, 2025
• Python

LLMs and Machine Learning done easily

• UpdatedMar 10, 2024
• Python

Govern, Secure, and Optimize your AI Traffic. AI Gateway provides unified interface to all LLMs using OpenAI API format with a focus on performance and reliability. Built in Rust.

• UpdatedMar 21, 2025
• Rust

Envoy AI Gateway is an open source project for using Envoy Gateway to handle request traffic from application clients to Generative AI services.

• UpdatedMar 22, 2025
• Go

Modular, open source LLMOps stack that separates concerns: LiteLLM unifies LLM APIs, manages routing and cost controls, and ensures high-availability, while Langfuse focuses on detailed observability, prompt versioning, and performance evaluations.

• UpdatedFeb 15, 2025

Build reliable, secure, and production-ready AI apps easily.

• UpdatedMar 20, 2025
• Python

Built for demanding AI workflows, this gateway offers low-latency, provider-agnostic access, ensuring your AI applications run smoothly and quickly.

• UpdatedMar 7, 2025
• Rust

Inference Hub for AI at Scale

• UpdatedMar 23, 2025
• Go

Build reliable, secure, and production-ready AI apps easily.

• UpdatedMar 6, 2025
• TypeScript

A powerful, AI Gateway designed from scratch for AI

• UpdatedMar 21, 2025
• Go

Developer focused AI Gateway

• UpdatedMar 7, 2025
• Ballerina

• UpdatedOct 4, 2024

Set of Otoroshi plugins to interact with LLMs

• UpdatedMar 21, 2025
• Scala