Fine-tuning

Cross-provider fine-tuning ledger. Pin training runs to immutable dataset snapshots for reproducibility; submit + cancel jobs on OpenAI, Vertex, Azure-OpenAI, Fireworks, and Together, and watch progress in one place.

Dashboard: /dashboard/evals/fine-tune · REST:/api/v1/fine-tune/jobs

Submit + cancel are live

Submit and cancel ship today for OpenAI, Vertex, Azure-OpenAI, Fireworks, and Together via your org's BYOK provider keys. Anthropic is intentionally not a submit target — it has no public fine-tune API — but Anthropic jobs created elsewhere still surface in the read-only ledger. Every submitted job lands in the same ledger and dashboard.

Reproducible training data

Every job pins to a dataset_versions snapshot via dataset_version_id. Re-runs use the same case set even if the live dataset has been edited since. The case count, content hash, and snapshot description carry through to the job's audit trail.

Lifecycle

Jobs flow through a 7-state machine mirroring OpenAI's naming for familiarity:

status transitions

pending → validating → queued → running → succeeded
                                          → failed
                                          → cancelled

(any state) → cancelled  // operator-initiated cancel

The webhook handler is the authoritative writer for the vendor- driven transitions (validating → queued → running → succeeded/failed). Operators (admin role) can cancel an in-flight job via POST /api/v1/fine-tune/jobs/<id>/cancel— it calls the vendor adapter's cancel() and is idempotent: already-terminal jobs (succeeded / failed / cancelled) are returned unchanged. The CLI exposes it as evalguard fine-tune cancel <jobId>.

CLI

evalguard fine-tune

# List jobs (newest first; filter by status or provider)
evalguard fine-tune list
evalguard fine-tune list --status running
evalguard fine-tune list --provider openai

# Inspect one job
evalguard fine-tune get <jobId>
evalguard fine-tune get <jobId> --json | jq .hyperparameters

# Estimate cost before submitting
evalguard fine-tune estimate --provider openai --base-model gpt-4o-mini \
  --dataset <datasetId> --version <versionId>

# Submit a job (admin+) pinned to an immutable dataset_versions snapshot
evalguard fine-tune submit --provider openai --base-model gpt-4o-mini \
  --dataset <datasetId> --version <versionId>

# Cancel an in-flight job (idempotent — terminal jobs return as-is)
evalguard fine-tune cancel <jobId>

REST

POST /api/v1/fine-tune/jobs — submit (admin+)

# Submits to the vendor with the org's BYOK key, then writes the ledger row.
{
  "provider": "openai",          // openai | vertex | azure-openai | fireworks | together
  "baseModel": "gpt-4o-mini",
  "datasetId": "uuid…",
  "datasetVersionId": "uuid…",   // pins to an immutable dataset_versions snapshot
  "suffix": "support-v3",        // optional vendor-side label
  "hyperparameters": { "n_epochs": 3 }  // optional
}

# 200 OK -> { "data": { "job": { "id": "…", "status": "queued", … } } }

POST /api/v1/fine-tune/jobs/<id>/cancel — cancel (admin+, idempotent)

# Calls the vendor adapter's cancel(). Already-terminal jobs are a no-op.
# 200 OK -> { "data": { "job": { …, "status": "cancelled" }, "alreadyTerminal": false } }

GET /api/v1/fine-tune/jobs

# Query params: status, provider, limit (max 200), offset
{
  "data": {
    "jobs": [
      {
        "id": "ff70…",
        "provider": "openai",
        "base_model": "gpt-4o-mini",
        "fine_tuned_model": "ft:gpt-4o-mini:org:abc:1A2B",
        "status": "succeeded",
        "case_count": 1200,
        "trained_tokens": 480000,
        "estimated_cost": 38.40,
        "dataset_version_id": "uuid…",
        "created_at": "2026-05-22T12:00:00Z",
        "finished_at": "2026-05-22T13:42:11Z"
      }
    ],
    "total": 1
  }
}

Authorization

SELECT — any org member can view jobs.
INSERT (submit) — admin+ only.
UPDATE (cancel, manual corrections) — admin+.
DELETE — owner only. Cancel via status=cancelled is the recommended path; DELETE drops the audit trail.