Cost budgets & FinOps

Cap LLM spend per API key, per team, per run, per project, and per org. Caps are enforced in the gateway proxy: when a key is over budget the proxy returns 402 Payment Required before the upstream provider is ever called.

Spend is metered from real gateway traffic and estimated per-model, so the counters you cap against are the same numbers the cost dashboards report.

What ships today

Per-key, per-team, and per-run caps are enforced live in the gateway proxy (budget-enforcer.ts) — over-cap traffic is blocked with a 402. Configure them from the API or the evalguard budget CLI. The org-level budget and the effective-cascade resolvership as configuration + read-only resolution today; wiring those two levels into the hot path is a deliberately separate soft→hard rollout.

The levels

Caps are set independently at each level. Under enforcement the resolver applies tightest-cap-wins: the level with the least remaining headroom binds, and any exceeded hard cap blocks the request.

scope → where the cap lives

org        org_budgets.monthly_budget_usd          (config + read-only cascade)
project    project_settings key=cost_budget         (monthlyLimit + thresholds)
team       team_budgets(org_id, team_id)            (enforced in gateway)
key        api_keys.monthly_budget_usd              (enforced in gateway)
run        agent_runs.per_run_budget_usd            (enforced in gateway)

A "team" is the same identifier the rest of the Cost axis attributes spend by — api_keys.agent_tag (mirrored onto each run). A key/run with no agent_tag, or no team_budgets row, simply falls through to per-key behavior — zero regression.

Set a per-key spend cap

The per-key budget is the workhorse: hand different services or tenants different caps under one org account. Set it from the CLI or the REST endpoint. Both are admin-only — spend and caps are billing data.

evalguard budget

# Set a $100 cap that resets monthly (the default cadence)
evalguard budget set <keyId> 100.00

# Same cap, but reset the counter every day (UTC-aligned)
evalguard budget set <keyId> 100.00 --reset-period daily

# Change only the reset cadence: daily | weekly | monthly
evalguard budget period <keyId> weekly

# Inspect current spend + cap + percent-used
evalguard budget get <keyId>
#
#   my-service-key
#   Key       6f1c…-…-…
#   Cap       $100.00
#   Cadence   monthly
#   Spent     $37.4200
#   Remaining $62.5800
#   Usage     37.4%
#   Period    2026-07-01T00:00:00.000Z

# Preview then remove the cap (key becomes uncapped/unlimited)
evalguard budget clear <keyId> --dry-run
evalguard budget clear <keyId>

Setting 0 blocks all gateway traffic on the key. The hard ceiling is $1,000,000 per period to catch typos.

REST — GET / PATCH / DELETE /api/v1/api-keys/:keyId/budget

# PATCH sets or updates the cap and/or the reset cadence.
# monthlyBudgetUsd: number sets it, null removes it (unlimited).
curl -X PATCH https://evalguard.ai/api/v1/api-keys/<keyId>/budget \
  -H "Authorization: Bearer $EVALGUARD_API_KEY" \
  -H "content-type: application/json" \
  -d '{ "monthlyBudgetUsd": 100.00, "resetPeriod": "monthly" }'

# GET returns the live budget state:
# {
#   "monthlyBudgetUsd": 100,
#   "resetPeriod": "monthly",
#   "currentPeriodSpentUsd": 37.42,
#   "currentPeriodStartedAt": "2026-07-01T00:00:00.000Z",
#   "remainingUsd": 62.58,
#   "percentUsed": 37.42,
#   "staleReset": false
# }

# DELETE is a convenience alias for PATCH { "monthlyBudgetUsd": null }.

How enforcement works

Enforcement lives in the gateway proxy, not the config API. On each request checkBudget()reads the key's cap and period-to-date spend before forwarding upstream; after a successful call recordSpend()atomically increments the counter via a Postgres RPC (read + compare + write in one transaction, so concurrent requests don't race).

over-cap response

HTTP/1.1 402 Payment Required
x-evalguard-budget-usd: 100.00
x-evalguard-spent-usd:  100.41

# reason codes surfaced by checkBudget():
#   exceeded        per-key hard cap hit
#   team_exceeded   team HARD cap hit (blocks even if the key is under)
#   no_cap          no per-key cap set (team cap may still block)

Deliberate design choices

Fail-open on DB errors. A flaky Supabase returns allowed: true — an accounting hiccup must never black-hole all gateway traffic.
No cron for resets. The period counter resets amortized on the first request of each new period (daily/weekly/monthly, UTC-aligned) — the stale counter reads as 0 for enforcement and recordSpend writes the new period start.
Per-run caps too. agent_runs.per_run_budget_usd flips a run to status='budget_exceeded' when an increment pushes it over, and the gateway 402s subsequent calls on that run.

Org & team budgets and the effective cascade

The org budget is CRUD at the top of the cascade. Setting the row is the opt-in; deleting it disables org enforcement. It carries an enforcementMode (hard blocks, soft only warns). Writes are admin-only and audited.

REST — org budget + effective cascade

# Set the org ceiling (admin-only, audit-logged)
curl -X PUT https://evalguard.ai/api/v1/budgets/org \
  -H "Authorization: Bearer $EVALGUARD_API_KEY" \
  -H "content-type: application/json" \
  -d '{ "orgId": "<orgId>", "monthlyBudgetUsd": 5000, "enforcementMode": "hard" }'

# Resolve the effective cascade for a context (READ-ONLY).
# Applies tightest-cap-wins across the levels that track spend and
# reports the binding cap + per-level breakdown + whether a HARD cap
# WOULD block — before the soft→hard rollout changes anything.
curl "https://evalguard.ai/api/v1/budgets/effective?orgId=<orgId>&teamId=<teamId>" \
  -H "Authorization: Bearer $EVALGUARD_API_KEY"
# → { "binding": {...}, "levels": [...], "blocked": false, "blockingLevels": [] }

Honest about scope: /budgets/org and /budgets/effective are configuration and read-only resolution. They surface which cap binds so operators can preview impact; they do not yet change what the gateway enforces. Per-key, per-team, and per-run caps are the levels enforced live today.

Project budgets & alerts

Project budgets are evaluation-oriented: a monthly limit plus warning and critical thresholds, stored in project_settings. A check action runs the BudgetEnforceragainst the current month's eval_runs spend and returns an allow/deny decision (respecting mode: soft | hard).

REST — /api/v1/cost/budget · /api/v1/cost/alerts

# Save a project budget: $500/mo, warn at 80%, critical at 95%
curl -X POST https://evalguard.ai/api/v1/cost/budget \
  -H "Authorization: Bearer $EVALGUARD_API_KEY" -H "content-type: application/json" \
  -d '{ "projectId": "<id>", "monthlyLimit": 500,
        "warningThreshold": 0.8, "criticalThreshold": 0.95, "mode": "soft" }'

# Ask whether a projected cost is within budget before running
curl -X POST https://evalguard.ai/api/v1/cost/budget \
  -H "Authorization: Bearer $EVALGUARD_API_KEY" -H "content-type: application/json" \
  -d '{ "projectId": "<id>", "action": "check", "estimatedCost": 12.50 }'

# Alert status → level: info | warning | critical | exceeded
curl "https://evalguard.ai/api/v1/cost/alerts?projectId=<id>" \
  -H "Authorization: Bearer $EVALGUARD_API_KEY"

# Configure (and test) Slack / email alert channels
curl -X POST https://evalguard.ai/api/v1/cost/alerts \
  -H "Authorization: Bearer $EVALGUARD_API_KEY" -H "content-type: application/json" \
  -d '{ "projectId": "<id>", "slackWebhook": "https://hooks.slack.com/…",
        "notifyEmail": "finops@acme.com" }'

When the alert level reaches warning or above, EvalGuard fires the configured Slack/email channels via the notification bridge (fire-and-forget, so it never blocks the response). POST with action: "test" to verify a channel is wired correctly.

Cost tracking & FinOps exports

Budgets cap spend; the cost APIs explain it. These are read-side analytics over the same metered spend the caps enforce against:

cost analytics + interchange connectors

GET /api/v1/cost/breakdown   ?orgId=…&groupBy=team|cost_center|owner|environment|user
                             multi-dimensional tag breakdown over cost_entries
GET /api/v1/cost/burn-rate   ?orgId=…&windowSeconds=300
                             live $/hr projection from recent gateway_proxy_logs
GET /api/v1/cost/forecast    ?projectId=…&daysAhead=30    projected spend
GET /api/v1/cost/export      ?format=focus|openmeter|lago  FinOps interchange

The export connectors emit row-level LLM spend in the standard formats a FinOps team already ingests — FinOps Foundation FOCUS 1.0 CSV, OpenMeter CloudEvents, or Lago usage events — so spend lands in the same data lake as your cloud bill. Pull them on the terminal:

evalguard cost-export

# Row-level FOCUS 1.0 CSV to stdout
evalguard cost-export <orgId> --format focus

# Lago NDJSON for a date range, written to a file
evalguard cost-export <orgId> --format lago \
  --start 2026-05-01 --end 2026-05-31 --out spend.ndjson

Roadmap, stated plainly

Enforced today: per-key, per-team, and per-run caps in the gateway proxy (hard block + 402; soft caps warn).
Config + read-only today: org budgets and the effective-cascade resolver. They preview which cap binds but don't yet gate the hot path — that's the pending soft→hard rollout.
Opt-in, off by default: budget-triggered model downgrade (budget_downgrade_enabled) transparently drops to a cheaper model near cap exhaustion and logs the decision to the FinOps dashboard.