llm-monitoring

Here are 8 public repositories matching this topic...

Language:All

Filter by language

All8 TypeScript4 Python2 JavaScript1 Jupyter Notebook1

Agenta-AI /agenta

The open-source LLMOps platform: prompt playground, prompt management, LLM evaluation, and LLM observability all in one place.

prompt-engineering prompt-management llm-tools llm-framework llm-playground llm-platform llm-evaluation rag-evaluation llm-monitoring llm-as-a-judge llm-observability llmops-platform

UpdatedDec 17, 2025
Python

bluewave-labs /flagwise

Sponsor

Star34

Monitor, detect, and analyze security threats and Shadow AI in your LLM applications with comprehensive analytics and real-time alerting.

ai-compliance ai-observability llm ai-governance ai-analytics llm-security llm-monitoring shadow-ai llm-tracker ai-traffic-monitoring shadow-ai-traction shadow-ai-detector

UpdatedNov 12, 2025
JavaScript

aichatwatch /aicw

Star14

AI Chat Watch (AICW) - free open-source tool for GEO marketers that track what & how AI mentions brands, products, companies.

marketing observability competitive-intelligence ai-agents claude martech brand-monitoring ai-observability chatgpt llm-evaluation llm-monitoring ai-transparency

UpdatedOct 30, 2025
TypeScript

pyladiesams /eval-llm-based-apps-jan2025

Star8

Create an evaluation framework for your LLM based app. Incorporate it into your test suite. Lay the monitoring foundation.

workshop llm llms llmops llm-eval llm-test llm-evaluation-framework llm-evaluation-metrics llm-monitoring llm-testing llm-evals

UpdatedMay 6, 2025
Jupyter Notebook

OpenTelemetry wrapper for Claude Code CLI that logs tool calls, token usage, costs, and execution traces to Logfire, Sentry, Honeycomb, or Datadog. Drop-in replacement that swaps 'claude' command for 'claudia'.

python cli sentry datadog tracing observability honeycomb opentelemetry anthropic-claude llm-monitoring logfire claude-code

UpdatedOct 24, 2025
Python

subodhkc /llmverify-npm

Star2

AI model health monitor for LLM apps – runtime checks for drift, hallucination risk, latency, and JSON/format quality on any OpenAI, Anthropic, or local client.

ai runtime-metrics ai-safety drift-detection llm ai-quality ai-code-review hallucination-detection llm-monitoring ai-observalibility