Datasets and logs. Plus security and guardrails.
Braintrust does datasets, experiments, scorers, and production logging well. EvalGuard imports both — your log spans and your eval datasets — and adds red-team scans (300+ plugins), an LLM firewall, and a SOC 2 evidence engine on the same data. No sign-up needed to run your first eval.
Honest positioning
Where Braintrust stops, EvalGuard keeps going
Braintrust is a solid eval and observability tool — datasets, experiments, scorers, and logs done well. EvalGuard overlaps on all of that, then extends into the security, firewall, and compliance work you'd otherwise buy separately.
| Capability | Braintrust | EvalGuard |
|---|---|---|
| Datasets & experiments | Yes | Yes — versioned datasets + run history |
| Scorers / autoevals | Yes | Yes — 200+ scorers (LLM-as-judge, pairwise, rubric) |
| Production logging & tracing | Yes | Yes — OTel trace ingest + cost attribution |
| Prompt playground & versioning | Yes | Yes — Prompt IDE + optimizer across 90+ providers |
| Human review / annotation | Yes | Yes — annotation queues + Krippendorff's alpha |
| AI proxy / gateway | AI proxy | Yes — BYOK gateway, 90+ providers, semantic cache |
| Red-team & security scans | Not offered | Yes — 300+ attack plugins with threat-feed sync |
| LLM firewall / guardrails | Not offered | Yes — real-time input + output firewall |
| SOC 2 evidence engine | Not offered | Yes — live evidence engine + audit log |
| Self-hosting | Hybrid / enterprise | Yes — self-host available |
Comparison reflects each product's core offering; check current Braintrust plans for the latest on hybrid/enterprise deployment.
Migration paths
Braintrust is two things. So the move is two paths.
Braintrust owns your export end-to-end — we never touch your Braintrust account. Convert what you download with the EvalGuard CLI.
Observability — bring your log spans
Export your logs from Braintrust (the SDK span.export() / project-logs fetch, or the API), then convert them to neutral-shape spans — model, tokens, scores, and latency all map over.
Datasets & evals — convert and run
Export a Braintrust dataset, convert it to a runnable EvalGuard config, then run it — keyless on your machine, or on the EvalGuard cloud.
# SDK: initDataset(project, { dataset }).fetch() → JSON (or the dataset fetch API)npx @evalguard/cli import:braintrust braintrust-dataset.json -o evalguard.config.jsonnpx @evalguard/cli eval:local evalguard.config.json --provider echoThe echo provider runs the whole eval loop with no API key — perfect for validating the config before you spend a token. Swap in --provider openai (with your key) for real model calls, or run on the cloud for shared dashboards and run history:
Want a hand with the migration?
If you have a large log history or a dataset with custom scorers, send us your Braintrust export and we'll help you map it and validate the first run. Free.