Loading open source…
by promptfoo
Test your prompts, agents, and RAGs. Red teaming/pentesting/vulnerability scanning for AI. Compare performance of GPT, Claude, Gemini, DeepSeek, and more. Simple declarative configs with command line and CI/CD integration. Used by OpenAI and Anthropic.
[npm] [npm] [GitHub Workflow Status] [MIT license] [Discord]
promptfoo is a CLI and library for evaluating and red-teaming LLM-based apps. Stop using trial-and-error... start shipping secure, reliable agents
Website · Getting Started · Red Teaming · Documentation · Discord
Promptfoo is now part of OpenAI. Promptfoo remains open source and MIT licensed. Read the company update.
npm install -g promptfoo
promptfoo init --example getting-started
Also available via brew install promptfoo and pip install promptfoo. Or use npx promptfoo@latest to run any command without installing.
Most LLM providers require an API key. Set yours as an environment variable:
export OPENAI_API_KEY=sk-abc123
Run an eval and view results:
cd getting-started
promptfoo eval
promptfoo view
See Getting Started (evals) or Red Teaming (vulnerability scanning) for more!
Here's what it looks like in action:
It works on the command line too:
[promptfoo command line]It also can generate security vulnerability reports:
[gen ai red team]We welcome contributions! Check out our contributing guide to get started.
Join our Discord community for help and discussion.
[image]Isometric line figures that answer the pointer. For React and for anything with a DOM.