Keep your MLflow traces. Add red team and guardrails.
MLflow is the MLOps standard, with first-class LLM tracing and mlflow.evaluate(). EvalGuard imports your MLflow traces — every span's model, provider, token usage, and inputs/outputs carry over (V2 and V3 formats) — then adds hosted red-team scans (300+ plugins), a runtime LLM firewall, an AI gateway, and a SOC 2 evidence engine on the same data. No sign-up needed to run your first import.
Honest positioning
Where MLflow stops, EvalGuard keeps going
MLflow is the MLOps backbone — experiment tracking, model registry, tracing, and evaluate(). EvalGuard overlaps on LLM tracing and evals, then extends into the hosted security platform, runtime firewall, AI gateway, and compliance work you'd otherwise buy separately (and doesn't try to replace MLflow's model registry).
| Capability | MLflow | EvalGuard |
|---|---|---|
| LLM tracing (spans, token usage) | Yes | Yes — OTLP + OpenInference ingestion, governed |
| Evaluation (mlflow.evaluate / genai) | Yes | Yes — 200+ scorers (LLM-as-judge, pairwise, rubric) |
| Experiment tracking / model registry | MLflow core | Focused on LLM eval + security, not model registry |
| Open source & self-hosting | Yes | Yes — Apache-2.0 core, self-host available |
| Red-team & security scans | Not offered | Yes — 300+ attack plugins with threat-feed sync |
| Runtime LLM firewall / guardrails | Not offered | Yes — real-time input + output firewall |
| AI gateway / BYOK proxy | Not offered | Yes — BYOK gateway, 15 proxied providers, semantic cache |
| SOC 2 evidence engine | Not offered | Yes — live evidence engine + audit log |
Migration path
Bring your traces in one command
Everything stays local until you choose to run it — we never touch your MLflow server. Export a trace, then convert it to neutral-shape spans with the EvalGuard CLI.
# Python: trace = mlflow.get_trace(trace_id)
# open('mlflow-trace.json','w').write(trace.to_json())npx @evalguard/cli import:traces --from mlflow mlflow-trace.json --output spans.jsonnpx @evalguard/cli scan # 300+ red-team attacks on your migrated dataBoth the MLflow 2.x and 3.x trace formats are handled — nanosecond timing and JSON-encoded span attributes decoded automatically, so model, provider, token usage, and inputs/outputs land clean. Then layer on red team, a runtime firewall, and a SOC 2 evidence engine.
Want a hand with the migration?
Send us your MLflow trace export and we'll help you map it and validate the first import. Free.