Repository files navigation

promptfoo is a developer-friendly local tool for testing LLM applications. Stop the trial-and-error approach - start shipping secure, reliable AI apps.

Website ·Getting Started ·Red Teaming ·Documentation ·Discord

# Install and initialize projectnpx promptfoo@latest init# Run your first evaluationnpx promptfooeval

SeeGetting Started (evals) orRed Teaming (vulnerability scanning) for more.

• Test your prompts and models withautomated evaluations
• Secure your LLM apps withred teaming and vulnerability scanning
• Compare models side-by-side (OpenAI, Anthropic, Azure, Bedrock, Ollama, andmore)
• Automate checks inCI/CD
• Share results with your team

Here's what it looks like in action:

It works on the command line too:

It also can generatesecurity vulnerability reports:

• 🚀Developer-first: Fast, with features like live reload and caching
• 🔒Private: LLM evals run 100% locally - your prompts never leave your machine
• 🔧Flexible: Works with any LLM API or programming language
• 💪Battle-tested: Powers LLM apps serving 10M+ users in production
• 📊Data-driven: Make decisions based on metrics, not gut feel
• 🤝Open source: MIT licensed, with an active community

• 📚Full Documentation
• 🔐Red Teaming Guide
• 🎯Getting Started
• 💻CLI Usage
• 📦Node.js Package
• 🤖Supported Models

We welcome contributions! Check out ourcontributing guide to get started.

Join ourDiscord community for help and discussion.