Migrate from Promptfoo in an afternoon.
Your Promptfoo YAML imports in one command. You get 257 red-team plugins (5× Promptfoo), 201 scorers, and the same CLI ergonomics — plus firewall, gateway, observability, and FinOps in the same platform. One bill. Zero stitching.
Why move now
Your red-team tool shouldn't be owned by the model you're testing
OpenAI now owns Promptfoo. That's a conflict if you rely on it to test OpenAI models for jailbreaks, prompt injection, or policy violations. EvalGuard is independent — 91 providers, zero model-vendor ownership.
257plugins vs Promptfoo's ~50
Every OWASP LLM Top 10 category plus indirect prompt injection, data exfiltration, multi-turn jailbreaks, PII leakage, policy violations, and many more. Kept up to date with threat-intel feed sync — not static. Plus 47 adversarial strategies and 200 DLP patterns.
Eval + firewall + gateway + observability + FinOps — same workspace
Promptfoo does eval. Then you stitch Helicone for observability, Portkey for gateway, Langfuse for traces. EvalGuard replaces all of them with one auth, one bill, one SLA.
Your promptfoo.yaml maps 1:1
Most fields need no rename. The ones that differ are listed below with the exact replacement.
| Promptfoo | EvalGuard | Notes |
|---|---|---|
| providers: [openai:gpt-4o] | model: gpt-4o | Provider auto-detected from model prefix. |
| prompts: [...] | prompt: "{{input}}" | Inline prompt template; {{input}} interpolation supported. |
| tests: [...] | cases: [...] | Same shape: { input, expectedOutput, metadata }. |
| assert: [{type: contains, value: 'X'}] | scorers: ["contains"] | Simple string array. 201 built-in scorers; contains/regex/similarity/etc. |
| assert config via type+value | scorerOptions: { contains: { value: 'X' } } | Per-scorer config is a separate object. |
| redteam: {plugins: [...]} | `evalguard scan` command | Red team lives in a separate config for scans; same platform, separate flow. |
CLI commands
| Promptfoo CLI | EvalGuard CLI |
|---|---|
| promptfoo eval | evalguard eval |
| promptfoo eval --no-cache | evalguard eval:local |
| promptfoo view | evalguard view |
| promptfoo redteam run | evalguard scan |
| promptfoo share | evalguard share |
| promptfoo init | evalguard init |
Migration in four commands
- 1Install the CLI
npm i -g @evalguard/cli - 2Authenticate
evalguard login --key <your-eg_key> - 3Import your Promptfoo config
evalguard import:promptfoo promptfoo.yaml - 4Run the eval
evalguard eval
Stuck on a Promptfoo feature that doesn't map cleanly? Tell us — we'll add the shim within a business day.
Your tests don't belong to the vendor you're testing
Move in an afternoon. Free forever tier — 50K traces/month, unlimited projects, AI Gateway included.
Start free — no credit card