llmops

Star

Here are 642 public repositories matching this topic...

Language:All

Filter by language

All642 Python309 Jupyter Notebook83 TypeScript67 Go27 Rust23 JavaScript14 HTML13 Shell8 MDX5 CSS4

Sort:Most stars

Sort options

Most stars Fewest stars Most forks Fewest forks Recently updated Least recently updated

pathwaycom /llm-app

Star47.7k

Ready-to-run cloud templates for RAG, AI pipelines, and enterprise search with live data. 🐳Docker-friendly.⚡Always in sync with Sharepoint, Google Drive, S3, Kafka, PostgreSQL, real-time data APIs, and more.

machine-learning real-time chatbot pathway open-ai rag vector-database hugging-face llm llmops vector-index llm-prompting llm-security llm-local retrieval-augmented-generation

UpdatedOct 23, 2025
Jupyter Notebook

BerriAI /litellm

Star31.8k

Python SDK, Proxy Server (AI Gateway) to call 100+ LLM APIs in OpenAI (or native) format, with cost tracking, guardrails, loadbalancing and logging. [Bedrock, Azure, OpenAI, VertexAI, Cohere, Anthropic, Sagemaker, HuggingFace, VLLM, NVIDIA NIM]

gateway bedrock openai vertex-ai azure-openai llm langchain llmops anthropic openai-proxy litellm ai-gateway llm-gateway mcp-gateway

UpdatedNov 28, 2025
Python

ComposioHQ /composio

Star26.2k

Composio equips your AI agents & LLMs with 100+ high-quality integrations via function calling

javascript python typescript ai js mcp sse developer-tools agents ai-agents gpt-4 llm llmops function-calling aiagents agentic-ai remote-mcp-server

UpdatedNov 28, 2025
TypeScript

mlflow /mlflow

Star23.1k

The open source developer platform to build AI agents and models with confidence. Enhance your AI applications with end-to-end tracking, observability, and evaluations, all in one integrated platform.

open-source machine-learning ai apache-spark evaluation ml openai agents observability model-management mlops mlflow agentops prompt-engineering ai-governance langchain llmops llm-evaluation

UpdatedNov 29, 2025
Python

liguodongiot /llm-action

Star22k

本项目旨在分享大模型相关技术原理以及实战经验（大模型工程化、大模型应用落地）

llm llmops llm-serving llm-training llm-inference

UpdatedNov 25, 2025
HTML

jina-ai /serve

Star21.8k

☁️ Build multimodal AI applications with cloud-native stack

docker kubernetes machine-learning framework deep-learning microservice pipeline cncf grpc prometheus orchestration cloud-native jaeger multimodal mlops fastapi opentelemetry neural-search generative-ai llmops

UpdatedMar 24, 2025
Python

langfuse /langfuse

Star18.8k

🪢 Open source LLM engineering platform: LLM Observability, metrics, evals, prompt management, playground, datasets. Integrates with OpenTelemetry, Langchain, OpenAI SDK, LiteLLM, and more. 🍊YC W23

open-source playground monitoring analytics evaluation self-hosted ycombinator openai observability autogen large-language-models llm prompt-engineering langchain llmops llama-index prompt-management llm-evaluation llm-observability

UpdatedNov 28, 2025
TypeScript

TransformerOptimus /SuperAGI

Star16.9k

<⚡️> SuperAGI - A dev-first open source autonomous AI agent framework. Enabling developers to build, manage & run useful autonomous agents quickly and reliably.

python ai nextjs agi artificial-intelligence openai artificial-general-intelligence agents hacktoberfest autonomous-agents pinecone gpt-4 llm llmops superagi

UpdatedJan 22, 2025
Python

comet-ml /opik

Star16.2k

Debug, evaluate, and monitor your LLM applications, RAG systems, and agentic workflows with comprehensive tracing, automated evaluations, and production-ready dashboards.

open-source playground evaluation openai hacktoberfest llm prompt-engineering hacktoberfest2025 langchain llmops llama-index llm-evaluation llm-observability

UpdatedNov 28, 2025
Python

raga-ai-hub /RagaAI-Catalyst

Star16.1k

Python SDK for Agent AI Observability, Monitoring and Evaluation Framework. Includes features like agent, llm and tools tracing, debugging multi-agentic system, self-hosted dashboard and advanced analytics with timeline and execution graph view

agents llmops agentneo ai-performance-optimization agentic-ai llm-testing ai-agent-monitoring llm-tracing ai-application-debugging agentic-ai-development ai-tool-interaction-monitoring ai-evaluation-tools

UpdatedNov 24, 2025
Python

bentoml /OpenLLM

Star12k

Run any open-source LLMs, such as DeepSeek and Llama, as OpenAI compatible API endpoint in the cloud.

llama mistral fine-tuning mlops bentoml vicuna llm model-inference llmops llm-serving llm-inference open-source-llm llama2 openllm llm-ops llama3-1 llama3-2 llama3-2-vision

UpdatedNov 24, 2025
Python

vibrantlabsai /ragas

Star11.6k

Supercharge Your LLM Application Evaluations 🚀

evaluation llm llmops

UpdatedNov 28, 2025
Python

tensorzero /tensorzero

Star10.6k

TensorZero is an open-source stack for industrial-grade LLM applications. It unifies an LLM gateway, observability, optimization, evaluation, and experimentation.

python rust machine-learning ai deep-learning ml artificial-intelligence openai llama gpt mlops ml-engineering ai-engineering large-language-models llm llms generative-ai llmops anthropic genai

UpdatedNov 29, 2025
Rust

dataelement /bisheng

Star10.4k

BISHENG is an open LLM devops platform for next generation Enterprise AI applications. Powerful and comprehensive features include: GenAI workflow, RAG, Agent, Unified model management, Evaluation, SFT, Dataset Management, Enterprise-level System Management, Observability and more.

react python agent enterprise workflow ocr ai chatbot orchestration openai llama gpt finetune rag sft llm llmops genai langchian llmdevops

UpdatedNov 28, 2025
TypeScript

Portkey-AI /gateway

Star10k

A blazing fast AI Gateway with integrated guardrails. Route to 200+ LLMs, 50+ AI Guardrails with 1 fast & friendly API.

mcp gateway openai hacktoberfest llm llms generative-ai langchain llmops ai-gateway llm-gateway mcp-servers mcp-client mcp-gateway model-router

UpdatedNov 29, 2025
TypeScript

Netflix /metaflow

Star9.7k

Build, Manage and Deploy AI/ML Systems

python kubernetes aws machine-learning ai azure gcp ml datascience high-performance-computing agents model-management cost-optimization distributed-training ml-infrastructure mlops ml-platform llm generative-ai llmops

UpdatedNov 27, 2025
Python

promptfoo /promptfoo

Star9.2k

Test your prompts, agents, and RAGs. AI Red teaming, pentesting, and vulnerability scanning for LLMs. Compare performance of GPT, Claude, Gemini, Llama, and more. Simple declarative configs with command line and CI/CD integration.

testing ci evaluation ci-cd pentesting cicd vulnerability-scanners prompts evaluation-framework red-teaming rag llm prompt-engineering llmops prompt-testing llm-eval llm-evaluation llm-evaluation-framework

UpdatedNov 29, 2025
TypeScript

bentoml /BentoML

Star8.3k

The easiest way to serve AI apps and models - Build Model Inference APIs, Job queues, LLM apps, Multi-model pipelines, and more!

python machine-learning deep-learning model-serving multimodal mlops ml-engineering ai-inference llm generative-ai llmops llm-serving model-inference-service llm-inference inference-platform

UpdatedNov 28, 2025
Python

Arize-ai /phoenix

Star7.8k

AI Observability & Evaluation

openai datasets agents ai-monitoring ai-observability prompt-engineering llms langchain llmops anthropic llamaindex llm-eval evals llm-evaluation aiengineering smolagents

UpdatedNov 25, 2025
Jupyter Notebook

evidentlyai /evidently

Star6.9k

Evidently is an open-source ML and LLM observability framework. Evaluate, test, and monitor any AI-powered system or data pipeline. From tabular data to Gen AI. 100+ metrics.

data-science machine-learning data-validation pandas-dataframe jupyter-notebook html-report hacktoberfest data-quality mlops model-monitoring data-drift llm generative-ai llmops

UpdatedNov 28, 2025
Jupyter Notebook

Improve this page

Add a description, image, and links to thellmops topic page so that developers can more easily learn about it.

Curate this topic

Add this topic to your repo

To associate your repository with thellmops topic, visit your repo's landing page and select "manage topics."

Learn more

Movatterモバイル変換

Navigation Menu

Search code, repositories, users, issues, pull requests...

Provide feedback

Saved searches

Use saved searches to filter your results more quickly