Keep your MLflow traces. Add an attack library and DLP depth.
MLflow is the MLOps standard, with first-class LLM tracing and mlflow.evaluate(). EvalGuard imports your MLflow traces — every span's model, provider, token usage, and inputs/outputs carry over (V2 and V3 formats) — and puts a 300+-plugin red-team platform, a firewall carrying 439 DLP patterns, and a SOC 2 evidence engine on the same data. No sign-up needed to run your first import.
Honest positioning
Where MLflow stops, EvalGuard keeps going
MLflow is the MLOps backbone — experiment tracking, model registry, tracing, and evaluate(). EvalGuard overlaps on LLM tracing and evals, then extends into the hosted security platform, runtime firewall, AI gateway, and compliance work you'd otherwise buy separately (and doesn't try to replace MLflow's model registry).
| Capability | MLflow | EvalGuard |
|---|---|---|
| LLM tracing (spans, token usage) | Yes | Yes — OTLP + OpenInference ingestion, governed |
| Evaluation (mlflow.evaluate / genai) | Yes | Yes — 200+ scorers (LLM-as-judge, pairwise, rubric) |
| Experiment tracking / model registry | MLflow core | Focused on LLM eval + security, not model registry |
| Open source & self-hosting | Yes | Yes — Apache-2.0 core, self-host available |
| Red-team & security scans | Safety / PII / jailbreak-detection scorers; no attack generator | Yes — 300+ attack plugins + 100+ adversarial strategies |
| Runtime LLM firewall / guardrails | Yes — gateway guardrails block and sanitize requests + responses | Yes — real-time input + output firewall, 439 DLP patterns |
| AI gateway / BYOK proxy | Yes — MLflow AI Gateway (unified proxy, key custody, traffic splitting, fallback) | Yes — BYOK gateway, 15 proxied providers, similarity response cache |
| SOC 2 evidence automation | Not a documented product feature | Yes — live evidence engine + audit log |
MLflow claims on this page were verified against their source — github.com/mlflow/mlflow @ c2153b2f6 (2026-08-09) — on 2026-08-10. “No attack generator” was settled by searching that whole tree for adversarial prompt generation; MLflow ships safety detectors, not an attack library. Capabilities move; check their current docs before you decide. “SOC 2 evidence automation” describes a product capability, not any vendor’s own certification status.
Migration path
Bring your traces in one command
Everything stays local until you choose to run it — we never touch your MLflow server. Export a trace, then convert it to neutral-shape spans with the EvalGuard CLI.
# Python: trace = mlflow.get_trace(trace_id)
# open('mlflow-trace.json','w').write(trace.to_json())npx @evalguard/cli import:traces --from mlflow mlflow-trace.json --output spans.jsonnpx @evalguard/cli init --template security-scan # scaffolds a scan config
npx @evalguard/cli scan:local <that-config> # no API key neededBoth the MLflow 2.x and 3.x trace formats are handled — nanosecond timing and JSON-encoded span attributes decoded automatically, so model, provider, token usage, and inputs/outputs land clean. Then layer on red team, a runtime firewall, and a SOC 2 evidence engine.
Want a hand with the migration?
Send us your MLflow trace export and we'll help you map it and validate the first import. Free.