Bring your executions. Get security with your evals.
Vellum runs your prompt and workflow executions. EvalGuard imports that run history and adds red team (300+ plugins), a firewall on the gateway hop, and a BYOK gateway alongside prompt and eval tooling — one workspace, one bill.
Prompts + evals were the start
What you keep — and what you gain
- ✓ Prompt- + workflow-execution history
- ✓ Model per execution (provider too, when your export carries one)
- ✓ Inputs + outputs (variables and completion blocks)
- ✓ Token usage — prompt / completion
- ✓ Cost + latency per run
- ✓ Execution state, and every other column you export
- + Firewall on the gateway hop rather than a node inside a workflow — 439 DLP patterns, input and output
- + 300+ red-team plugins + 100+ adversarial strategies
- + 200+ eval scorers (LLM-as-judge, pairwise, rubric)
- + Prompt IDE + optimizer across 90+ providers
- + Managed BYOK gateway with similarity response caching
- + 50 compliance frameworks mapped to live evidence + a tamper-evident audit log
Backfill your history
Bring your Vellum executions along
Export your executions from Vellum (the Executions tables in the deployment UI at app.vellum.ai, or their workflow- and prompt-deployment executions APIs), then convert them with the EvalGuard CLI. Both shapes are accepted, no record is dropped, and re-running the same import never double-counts, because spans use a stable content hash.
Prints an import summary (spans imported / duplicates skipped / parse errors) and writes the neutral spans to spans.json. This is an execution-history import — it moves your run logs, not your Vellum prompt deployments or workflow definitions.
| What imports | Where it lands |
|---|---|
| Workflow + prompt executions | One EvalGuard span each, grouped by trace id |
| Model (and provider, when present) | span.model / span.provider |
| Inputs / outputs | span.input / span.output |
| Token usage (prompt / completion) | span.promptTokens / completionTokens / totalTokens |
| Cost | span.costUsd |
| Latency | span.durationMs (latency, latency_ms, or start / end) |
| Execution state + every other column | span.attributes (vellum.* namespace) |
Vellum's terminal failure state is REJECTED; those executions import with status: error so your failure history survives the move. Everything else the importer recognizes is preserved verbatim under span.attributes.
Vellum claims on this page were checked against their published documentation — docs.vellum.ai + the public vellum-ai SDK types (raw) — on 2026-08-10. Capabilities move; check their current docs before you decide.
Note that vellum.ai now fronts a consumer personal-AI assistant; the developer platform this page is about is at docs.vellum.ai and app.vellum.ai.
One platform, one bill
Ship prompts. Secure, eval, optimize — everywhere.
Prompt tooling is the hook. The platform is why you stay.
Start free