Keep your Weave calls. Add security and compliance.
Weights & Biases Weave is a great LLM tracing + evaluation platform. EvalGuard imports your Weave calls — op names, token usage (OpenAI and Anthropic naming), cost, and exceptions all carry over — and puts a 300+-plugin red-team platform, a real-time input + output firewall, a BYOK gateway, and a SOC 2 evidence engine on the same data. No sign-up needed to run your first import.
Honest positioning
Where Weave stops, EvalGuard keeps going
Weave is a first-class tracing + eval tool with a clean call tree and the W&B ecosystem behind it. EvalGuard overlaps on tracing and evals, then extends into the hosted security platform, runtime firewall, AI gateway, and compliance work you'd otherwise buy separately.
| Capability | W&B Weave | EvalGuard |
|---|---|---|
| LLM call tracing (trace tree) | Yes | Yes — OTLP + OpenInference ingestion, governed |
| Evaluation + scorers | Yes | Yes — 200+ scorers (LLM-as-judge, pairwise, rubric) |
| Native token / cost tracking | Yes | Yes — per-org cost ledger + budgets |
| Open source & self-hosting | Apache-2.0 incl. the trace server; no packaged self-host recipe in-repo | Yes — Apache-2.0 core, self-host available |
| Red-team & security scans | Security scorers (prompt-injection, Presidio PII, Bedrock); no attack generator | Yes — 300+ attack plugins + 100+ adversarial strategies |
| Runtime LLM firewall / guardrails | Yes — guardrail scorers (prompt-injection, Presidio PII, Bedrock) + Monitors | Yes — real-time input + output firewall |
| AI gateway / BYOK proxy | Yes — completions proxy + custom providers with key custody | Yes — BYOK gateway, 15 proxied providers, similarity response cache, firewall on the same hop |
| SOC 2 evidence automation | Not a documented product feature | Yes — live evidence engine + audit log |
W&B Weave claims on this page were verified against their source — github.com/wandb/weave @ c28011c940 (2026-08-07) — on 2026-08-10. Capabilities move; check their current docs before you decide. “SOC 2 evidence automation” describes a product capability, not any vendor’s own certification status.
Migration path
Bring your calls in one command
Everything stays local until you choose to run it — we never touch your W&B project. Export your calls, then convert them to neutral-shape spans with the EvalGuard CLI.
# Python: calls = client.get_calls(include_costs=True)
# pd.DataFrame([c.to_dict() for c in calls]).to_json('weave-calls.json', orient='records')npx @evalguard/cli import:traces --from weave weave-calls.json --output spans.jsonnpx @evalguard/cli init --template security-scan # scaffolds a scan config
npx @evalguard/cli scan:local <that-config> # no API key neededToken usage keyed by model, per-model cost (with include_costs), and exceptions all map over — a failed call becomes an error span, a normal one stays ok. Then layer on red team, a runtime firewall, and a SOC 2 evidence engine.
Want a hand with the migration?
Send us your Weave export and we'll help you map it and validate the first import. Free.