llm-observability

Star

Here are 29 public repositories matching this topic...

Language:All

Filter by language

All29 Python9 TypeScript8 Go6 HTML1 Java1 JavaScript1

Sort:Most stars

Sort options

Most stars Fewest stars Most forks Fewest forks Recently updated Least recently updated

langfuse /langfuse

Star18.8k

🪢 Open source LLM engineering platform: LLM Observability, metrics, evals, prompt management, playground, datasets. Integrates with OpenTelemetry, Langchain, OpenAI SDK, LiteLLM, and more. 🍊YC W23

open-source playground monitoring analytics evaluation self-hosted ycombinator openai observability autogen large-language-models llm prompt-engineering langchain llmops llama-index prompt-management llm-evaluation llm-observability

UpdatedNov 29, 2025
TypeScript

comet-ml /opik

Star16.2k

Debug, evaluate, and monitor your LLM applications, RAG systems, and agentic workflows with comprehensive tracing, automated evaluations, and production-ready dashboards.

open-source playground evaluation openai hacktoberfest llm prompt-engineering hacktoberfest2025 langchain llmops llama-index llm-evaluation llm-observability

UpdatedNov 28, 2025
Python

Next-generation AI Agent Optimization Platform: Cozeloop addresses challenges in AI agent development by providing full-lifecycle management capabilities from development, debugging, and evaluation to monitoring.

agent open-source playground ai monitoring evaluation openai observability agentops coze langchain llmops prompt-management llm-observability agent-evaluation eino agent-observability

UpdatedNov 29, 2025
Go

Helicone /helicone

Star4.8k

🧊 Open source LLM observability platform. One line of code to monitor, evaluate, and experiment. YC W23 🍓

open-source playground monitoring analytics evaluation ycombinator openai gpt large-language-models llm prompt-engineering langchain llmops llama-index prompt-management llm-evaluation llm-observability agent-monitoring llm-cost

UpdatedNov 27, 2025
TypeScript

VoltAgent /voltagent

Sponsor

Star4.1k

Open Source TypeScript AI Agent Framework with built-in LLM Observability

nodejs javascript open-source framework typescript ai mcp tts openai chatbots multiagent agents observability ai-agents rag llm chatgpt ai-agents-framework llm-observability aiagentframework

UpdatedNov 29, 2025
TypeScript

Agenta-AI /agenta

Star3.4k

The open-source LLMOps platform: prompt playground, prompt management, LLM evaluation, and LLM observability all in one place.

prompt-engineering prompt-management llm-tools llm-framework llm-playground llm-platform llm-evaluation rag-evaluation llm-monitoring llm-as-a-judge llm-observability llmops-platform

UpdatedNov 28, 2025
Python

lmnr-ai /lmnr

Star2.4k

Laminar - open-source all-in-one platform for engineering AI products. Create data flywheel for your AI app. Traces, Evals, Datasets, Labels. YC S24.

rust open-source typescript ai monitoring analytics evaluation ts self-hosted rust-lang developer-tools agents observability aiops ai-observability llmops evals llm-evaluation llm-observability llm-workflow

UpdatedNov 27, 2025
TypeScript

JudgmentLabs /judgeval

Star1k

The open source post-building layer for agents. Our environment data and evals power agent post-training (RL, SFT) and monitoring.

agent open-source reinforcement-learning openai rl agents llm prompt-engineering langchain llama-index llm-evaluation langgraph llm-observability agentic-ai grpo

UpdatedNov 29, 2025
Python

palico-ai /palico-ai

Star340

Build, Improve Performance, and Productionize your LLM Application with an Integrated Framework

nodejs javascript docker typescript ai full-stack openai autogen rag llm portkey langchain anthropic llamaindex langchain-js llm-agent llm-framework llm-evaluation llm-observability

UpdatedNov 26, 2024
TypeScript

evilmartians /agent-prism

Star251

React components for visualizing traces from AI agents

react open-source typescript monitoring react-components tracing developer-tools component-library ui-components observability distributed-tracing ai-agents trace-visualization tailwindcss opentelemetry ai-monitoring llm generative-ai langchain llm-observability

UpdatedNov 14, 2025
TypeScript

langfuse /oss-llmops-stack

Star127

Modular, open source LLMOps stack that separates concerns: LiteLLM unifies LLM APIs, manages routing and cost controls, and ensures high-availability, while Langfuse focuses on detailed observability, prompt versioning, and performance evaluations.

open-source monitoring gateway self-hosted ycombinator llm llmops openai-proxy prompt-management llm-evaluation ai-gateway llm-observability llm-gateway

UpdatedFeb 15, 2025

cyberark /agentwatch

Star102

A powerful AI observability framework that provides comprehensive insights into agent interactions across platforms, enabling developers to monitor, analyze, and optimize AI-driven applications with minimal integration effort.

agent security machine-learning ai monitoring cybersecurity observability large-language-models llm agentic llm-tools llm-framework llm-observability agentic-ai

UpdatedMay 14, 2025
Python

radicalbit /radicalbit-ai-monitoring

Star82

A comprehensive solution for monitoring your AI models in production

data-science machine-learning ai monitoring artificial-intelligence observability machine-learning-engineering data-drift ai-monitoring ml-observability ai-observability llm-observability

UpdatedNov 13, 2025
Python

myscale /myscale-telemetry

Star53

Open-source observability for your LLM application.

python monitoring callback langchain llm-observability

UpdatedJan 2, 2025
Python

langfuse /langfuse-java

Star34

🪢 Auto-generated Java Client for Langfuse API

java open-source monitoring analytics self-hosted ycombinator openai observability sdk-java large-language-models llm prompt-engineering llmops langfuse llm-observability

UpdatedAug 21, 2025
Java

teilomillet /hapax

Star22

The reliability layer between your code and LLM providers.

infrastructure open-source monitoring openai infrastructure-as-code observability llm llmops anthropic llm-observability llm-gateway

UpdatedJan 6, 2025
Go

AndrMoura /streamlit-chatbot-analytics

Star7

Streamlit-based chatbot leveraging Ollama via LangChain and PostHog-LLM for advanced logging and monitoring

analytics chatbot chatbots chatbot-application streamlit llm langchain ollama llm-ops llama3 llm-observability

UpdatedMay 8, 2024
Python

matdev83 /llm-accounting

Star5

A Python package for tracking and analyzing LLM usage across different models and applications. It is primarily designed as a library for integration into development process of LLM-based agentic workflow tooling, providing robust tracking capabilities.

mlops mlops-workflow llms llm-ops llm-observability agentic-workflow agentic-ai agentic-ai-development

UpdatedJul 7, 2025
Python

VoltAgent /ai-agent-platform

Sponsor

Star5

AI agent platform for building multi-agent systems with orchestration, memory, RAG, workflows, and enterprise observability.

ai ai-agents llm-agent ai-agents-framework llm-observability ai-agent-platform

UpdatedOct 27, 2025

justinGrosvenor /alignmenter

Star4

Check if your AI sounds like your brand, stays safe, and behaves consistently. Works with your custom GPTs, hosted APIs, and local models. Get detailed reports in minutes, not days.

evaluation-metrics ai-safety evaluation-framework llm ai-testing llm-training llm-evaluation llm-observability brand-voice

UpdatedNov 12, 2025
HTML

Improve this page

Add a description, image, and links to thellm-observability topic page so that developers can more easily learn about it.

Curate this topic

Add this topic to your repo

To associate your repository with thellm-observability topic, visit your repo's landing page and select "manage topics."

Learn more

Movatterモバイル変換

Navigation Menu

Search code, repositories, users, issues, pull requests...

Provide feedback

Saved searches

Use saved searches to filter your results more quickly