Evidence Engine

What the engine produces

Every artifact the evidence engine emits is integrity-protected. The three building blocks are:

• Evidence items -- each carries a SHA-256 hash of its own data, so any post-hoc edit is detectable.
• Evidence bundles -- deterministic, per-framework rollups with a bundle digest and an optional Ed25519 issuance signature.
• Audit log -- an append-only, HMAC-SHA256-signed hash chain of AI decisions, overrides, and guardrail events.

Across all frameworks, EvalGuard ships 50 compliance framework definitions that evidence maps to.

How evidence is generated

A compliance check runs a fixed pipeline. Given a framework, EvalGuard:

1. Resolves the framework and its requirements.
2. Collects the attackTypes from every automatable requirement.
3. Runs a security scan targeting those attack types.
4. Runs specialized requirement validators.
5. Optionally re-grades each requirement with an LLM judge tier (fail-closed — a judge error downgrades toward not-met, never silently passes).
6. Determines a per-requirement status: met, partial, not-met, or untested.
7. Calculates category and overall scores.
8. Generates a prioritized remediation plan.
9. Packages each piece of evidence with a SHA-256 hash of its data.
10. Computes a delta against a previous assessment, if supplied.

Evidence item types

Each collected evidence item is one of four types, and every item stores a hash field (SHA-256 of its data) alongside a requirementId, description, and timestamp:

The whole set is folded into an audit package whose package-level integrityHash is the SHA-256 of every item hash concatenated — recomputing it detects a swapped or removed item.

Live control evidence

Beyond scan-driven evidence, EvalGuard collects live control evidence from your organization's actual system state — not stubs. Each call inspects real tables and configuration and returns a status of pass, fail, partial, or not_applicable per control.

curl -X GET "https://evalguard.ai/api/v1/compliance/evidence?orgId=YOUR_ORG_ID&framework=soc2" \
  -H "Authorization: Bearer eg_live_your_api_key"

The framework parameter accepts soc2, gdpr, eu_ai_act, hipaa, iso_42001, and nist_ai_rmf. Controls checked include access control, audit logging, encryption at rest (BYOK), data retention, webhook integrity (HMAC), automated security testing, SSO/MFA enforcement, eval governance, rate limiting, and incident response.

{
  "framework": "soc2",
  "orgId": "YOUR_ORG_ID",
  "collectedAt": "2026-07-16T12:00:00.000Z",
  "summary": {
    "totalControls": 10,
    "pass": 7,
    "fail": 2,
    "partial": 1,
    "complianceScore": 70
  },
  "evidence": [
    {
      "controlId": "soc2-SC-1",
      "framework": "soc2",
      "controlName": "Encryption at Rest (BYOK)",
      "status": "pass",
      "evidence": "AES-256-GCM encryption with PBKDF2 key derivation (310K iterations) is configured.",
      "collectedAt": "2026-07-16T12:00:00.000Z"
    }
  ]
}

SOC 2 control coverage & rows

SOC 2 maps to five Trust Services Criteria categories — Security (CC6), Availability (A1), Processing Integrity (PI1), Confidentiality (C1), and Privacy (P1). Two endpoints back an auditor's SOC 2 workflow:

The coverage summary returns a totals object (total, covered, stale, uncovered) so you can surface gaps before an audit window opens.

Deterministic evidence bundles

To hand an auditor a single artifact per framework, POST a list of evidence items and a reporting period to the bundle endpoint. The result is deterministic: the same input always yields the same bundleId and digest, so auditors can re-derive and diff bundles across reporting periods without false-positive drift.

Each evidence item declares a kind — one of audit-trail, scorer-run, guardrail-decision, policy-decision, manual-attestation, or other — and joins to a control via controlIds, pluginId, or attackType.

curl -X POST "https://evalguard.ai/api/v1/compliance/evidence-bundle/soc2" \
  -H "Authorization: Bearer eg_live_your_api_key" \
  -H "Content-Type: application/json" \
  -d '{
    "orgId": "00000000-0000-0000-0000-000000000000",
    "periodStart": "2026-01-01T00:00:00.000Z",
    "periodEnd": "2026-04-01T00:00:00.000Z",
    "evidence": [
      {
        "id": "ev-1",
        "kind": "guardrail-decision",
        "controlIds": ["soc2-sec-01"],
        "at": 1735689600000,
        "description": "Prompt-injection guardrail blocked an override attempt",
        "outcome": "pass"
      }
    ]
  }'

The response is a bundle with per-control rollups (met / partial / not-met / untested), a severity-weighted overallScore, a byCategory breakdown, and a digest. When signed, it also carries a base64 signature, signatureAlgorithm: "ed25519", and signaturePublicKeyPem.

Tamper-evident audit log

The audit trail is an append-only log of AI decisions, human overrides, guardrail triggers, and lifecycle events. Every entry is both signed and hash-chained to the entry before it.

HMAC-SHA256 signature + hash chain

• Each entry is signed with HMAC-SHA256 over a canonical payload: sequence number, timestamp, event type, actor id, resource, description, previous hash, and metadata. The current payload version also binds the tenant scope (orgId / projectId) into the signed bytes.
• The SHA-256 hash of a complete entry becomes the previousHash of the next — a broken link reveals any insertion, deletion, or reordering.
• The chain starts from a fixed genesis hash (sha256("evalguard-genesis-block")).
• The HMAC key comes from AUDIT_SIGNING_KEY. In production the engine fails closed — it refuses to sign with a committed default rather than produce a forgeable chain.

Recorded event types include ai-decision, model-response, human-override, guardrail-trigger, eval-run, security-scan, deployment, config-change, data-access, export, and deletion.

Ed25519 asymmetric signing — publicly verifiable

HMAC is verifiable only by a holder of the secret key. To make the log provable to a customer, auditor, or court, you export the chain and sign it once with an Ed25519 key. The exported bundle embeds the public key, so a third party runs verifyAuditChain with no secret at all.

import {
  auditTrail,
  generateAuditKeyPair,
  signAuditChain,
  verifyAuditChain,
} from "@evalguard/core";

// 1. Append tamper-evident entries (HMAC-SHA256 + hash chain).
auditTrail.append({
  eventType: "ai-decision",
  actor: { type: "user", id: "user_123" },
  orgId: "org_abc",
  resourceType: "ai-decision",
  description: "Loan approval generated by gpt-4o",
});

// 2. Ed25519-sign the exported chain for public verification.
const keys = generateAuditKeyPair();
const entries = auditTrail.getEntries();
const signed = signAuditChain(entries, keys);

// 3. Anyone can verify with ONLY the public key — no shared secret.
const result = verifyAuditChain(entries, signed);
console.log(result.valid, result.entriesChecked);

verifyAuditChain (and the HMAC-based AuditTrail.verifyChain) return an AuditVerificationResult with valid, entriesChecked, errors, and — on failure — firstInvalidEntry and brokenChainAt, pinpointing exactly where a chain was tampered.

Retention policies

Audit evidence carries built-in, framework-driven retention policies. The engine resolves the effective retention for each event type by taking the strictest applicable minimum and preferring the safest deletion strategy (archive over anonymize over hard-delete).

When two policies conflict — for example GDPR's maximum below EU AI Act's minimum — detectRetentionConflicts surfaces the clash with a suggested resolution (anonymize personal data at the earlier bound while retaining non-personal audit data for the longer term).

Exporting evidence

Compliance evidence exports as an audit package in the format your auditor prefers. The report carries a generated reportId (EG-AUDIT-YYYYMMDD-XXXXXX), an executive summary with a risk level, a per-category breakdown, an optional remediation plan, an optional model card, and an evidence log.

curl -X POST "https://evalguard.ai/api/v1/compliance/export" \
  -H "Authorization: Bearer eg_live_your_api_key" \
  -H "Content-Type: application/json" \
  -d '{
    "framework": "hipaa",
    "format": "html",
    "organizationName": "Acme Health",
    "systemName": "Triage Assistant",
    "includeRemediationPlan": true,
    "includeEvidenceLog": true
  }'

Independent verification

Any compliance document — an evidence bundle, audit bundle, SoA snapshot, or SOC 2 evidence row — can be verified against its own hash through a public, unauthenticated endpoint. It recomputes the SHA-256 over the canonical JSON of the document (with the hash field stripped) and reports whether it matches the claimed hash. No data is stored or logged.

curl -X POST "https://evalguard.ai/api/v1/compliance/verify" \
  -H "Content-Type: application/json" \
  -d '{
    "document": { "...": "the full bundle exactly as downloaded" },
    "claimedHash": "THE_64_CHAR_SHA256_FROM_THE_DOCUMENT",
    "hashField": "digest"
  }'

Pass the field name that holds the hash via hashField — digest for an evidence bundle, bundleHash for an audit bundle, snapshotHash (the default) for a SoA snapshot, or payload_hash for a SOC 2 evidence row. Supply publicKeyPem to also verify the Ed25519 signature. The response returns matchesClaimed, computedHash, and — when a key was given — signatureValid.