Compliance

Supported Frameworks

EU AI Act

The EU AI Act classifies AI systems by risk level and imposes obligations accordingly. EvalGuard maps your security scan and eval results to specific EU AI Act requirements.

Requirement Categories

• Risk Classification -- Prohibited, high-risk, limited, minimal risk determination
• Transparency -- AI interaction disclosure, deepfake labeling, content marking
• Human Oversight -- Human-in-the-loop, intervention capabilities, override mechanisms
• Data Governance -- Data quality, representativeness, bias testing
• Technical Robustness -- Accuracy, security, adversarial resilience
• Accountability -- Registration, documentation, conformity assessment
• Fundamental Rights -- Impact assessment on fundamental rights

Automated Checks

EvalGuard can automatically verify certain EU AI Act requirements through security scans. For example, the system-prompt-leak plugin verifies transparency requirement EU-TR-1 (AI Interaction Disclosure).

ISO 42001

ISO 42001 provides requirements for establishing, implementing, and improving an AI Management System (AIMS).

Requirement Categories

• AI Policy & Leadership -- Organizational commitment and governance
• Risk Assessment -- AI-specific risk identification and treatment
• Data Management -- Data quality, lifecycle, and provenance
• AI System Development -- Design, testing, and validation
• Performance Evaluation -- Monitoring, measurement, and audit
• Improvement -- Nonconformity, corrective action, continual improvement

NIST AI RMF

The NIST AI Risk Management Framework organizes risk management into four core functions.

Core Functions

• Govern -- Cultivate a culture of risk management, establish policies and processes
• Map -- Understand the context, identify and assess AI risks
• Measure -- Analyze, assess, and track identified risks
• Manage -- Prioritize, respond to, and monitor risks

EvalGuard maps eval scorers and security plugins to NIST AI RMF subcategories, providing automated measurement for the Measure function.

MITRE ATLAS

MITRE ATLAS (Adversarial Threat Landscape for AI Systems) catalogs real-world adversarial techniques targeting AI. EvalGuard's security plugins directly map to ATLAS techniques.

Tactic Coverage

• Reconnaissance -- system-reconnaissance, model-identification plugins
• Initial Access -- prompt-injection, indirect-injection, few-shot-attack plugins
• Execution -- jailbreak, roleplay-exploit, chain-of-thought-exploit plugins
• Persistence -- memory-poisoning, rag-poisoning plugins
• Exfiltration -- data-extraction, pii-leak, system-prompt-leak plugins

Running a Gap Analysis

Use the API or dashboard to run a compliance gap analysis. It compares your existing eval and security scan results against a framework's requirements.

Via API

curl -X GET "https://evalguard.ai/api/v1/compliance/gaps?framework=eu-ai-act" \
  -H "Authorization: Bearer eg_your_api_key"

Response

{
  "framework": "EU AI Act",
  "overallScore": 72,
  "totalRequirements": 25,
  "metCount": 14,
  "partialCount": 4,
  "notMetCount": 3,
  "untestedCount": 4,
  "byCategory": {
    "risk-classification": { "total": 3, "met": 1, "partial": 0, "notMet": 0, "untested": 2 },
    "transparency": { "total": 5, "met": 3, "partial": 1, "notMet": 0, "untested": 1 },
    "technical-robustness": { "total": 6, "met": 4, "partial": 2, "notMet": 0, "untested": 0 }
  }
}

Via Dashboard

Navigate to Dashboard > Compliance to see a visual breakdown of your compliance posture across all frameworks.

Remediation Guidance

For each gap, EvalGuard provides:

• Priority -- Based on severity and business impact
• Action -- Specific steps to address the gap
• Effort -- Low, medium, or high implementation effort
• Automatable -- Whether EvalGuard can verify the fix automatically

Run the relevant security scan plugins or eval scorers to automatically verify remediation. Re-run the gap analysis to see updated scores.