Send a request

promptfoo

by Promptfoo (OpenAI)

Test AI features like regular code

Automatically tests prompts, agents and RAG apps and runs red-teaming and vulnerability scans locally. Now part of OpenAI, still open source.

Open SourceTrust 97Some practice neededTerminal · Python · Docker · macOS
Get it via OROBO

Checked sourcegithub.com/promptfoo/promptfooLicence: MITchecked 3 hours agoGitHub lists a SHA-256 checksum for every release file.

Have it set up for you instead:Request setup by OROBOOr start with the AI Compass: a written assessment at a fixed price of €490 (businesses only).

How it runs for you

Simulation starting …

Illustration, not a real test run

What you get

What you get

  1. 01Tests prompts, agents and RAG apps automatically
  2. 02Red-team scans run on your own machine
  3. 03Fits into your CI pipeline

For you if …

  • Developers shipping AI features
  • Teams with automated tests

Pick something else if …

  • If you only use ready-made AI apps

How to start

In three steps.

What you need
  • Node.js or Python
  • API access to your model or endpoint
  1. 1Run npx promptfoo@latest redteam setup
  2. 2Choose the target and attack types
  3. 3Run it and view the report in your browser

Part of these packages

promptfoo works together with other software.

Use AI safely

Your team uses AI — with far less risk of data leaks or hijacked chatbots.

5 stepsone day for the basicsMedium

Safety check

6 of 7 checks passed · 1 note

No known malicious patterns in the install instructionsactively maintainedLicence: MIT

Trust 97
Who is behind it?
  • Taken by OROBO from the official source. The maker has not joined OROBO yet.
Is the code clean?
  • Source code publicly visible
  • Licence: MIT
  • No known malicious patterns in the install instructions
Is it maintained?
  • Website reachable
  • Updated today
Technical details & what we don’t check
  • Website loads fast
  • Website protection settings: 2 of 6 (website only, not the software)nosniff, Referrer-Policy

Claims about protection come from the maker. The trust score checks public signals, not effectiveness.

We automatically check origin, code patterns, licence, maintenance and reachability — again every day. We do not check whether the software fits your purpose, how it handles your data, or whether it works without errors. Last checked 3 hours ago. How OROBO checks

Alternatives for the same goal

Snyk Agent Scan (formerly Invariant MCP-Scan)Make it safe · Snyk (ex-Invariant Labs team)Scan installed MCP servers and skills for trapsOpen SourceTrust high, 100For developersCisco Skill ScannerMake it safe · Cisco AI DefenseCheck agent skills before you install themOpen SourceTrust high, 100For developersDocker MCP GatewayMake it safe · DockerRun MCP servers sealed off in containersOpen SourceTrust high, 100For developersCisco MCP ScannerMake it safe · Cisco AI DefenseInspect MCP servers before you plug them inOpen SourceTrust high, 100For developersAktoMake it safe · AktoFind and test MCP servers and APIs in your companyOpen SourceTrust high, 97For developersgarakMake it safe · NVIDIAHave your language model attacked on purposeOpen SourceTrust high, 100For developersGuardrails AIMake it safe · Guardrails AI (Harvey)Check AI answers before they move onOpen SourceTrust high, 100For developersNeMo GuardrailsMake it safe · NVIDIAFirm guardrails for your chatbotOpen SourceTrust high, 100For developers

Goes well with

OllamaBuild it yourself · OllamaRun language models locally with one commandOpen SourceTrust high, 97Runs offlineLakera GuardMake it safe · Lakera (Check Point Software)Catch prompt attacks before your AI respondsFree versionTrust high, 100For businessesCisco Skill ScannerMake it safe · Cisco AI DefenseCheck agent skills before you install themOpen SourceTrust high, 100For developersReality DefenderMake it safe · Reality Defender Inc.Detect deepfakes in voice, images and videoFree versionTrust high, 100Beginner-friendly

Facts

AreaMake it safe
PriceOpen Source
LicenceMIT
Runs onTerminal, Python, Docker, macOS, Linux, Windows
Popular with developers★ 26.9K GitHub
Last updated05 Nov 2026
Sourcepromptfoo/promptfoo ↗
Websitepromptfoo.dev ↗
I am the maker