Shadow AI discovery

Find the unsanctioned AI tools your workforce is already using. Ship egress, SSO, or CASB logs to EvalGuard, and it classifies every domain against a known-AI-tool catalog, rolls sightings up per tool, and ranks them by usage and data risk.

"Shadow AI" is AI-SaaS used outside IT approval — ChatGPT, Claude, Otter, Fireflies, and hundreds more. Discovery answers the question a policy alone can't: who is actually pasting company data into which tool.

What ships today

The full discovery pipeline is live: log ingest from Zscaler / Netskope / Cloudflare Gateway / Okta, domain classification, the detections rollup, per-domain policy overrides, a browsable AI-app catalog, and the evalguard shadow-ai CLI. The single-request PII/sensitive-data analyzer is live too. The aggregate GET /api/v1/shadow-ai dashboard is still a thin roll-up over gateway logs — see Honest status below.

How discovery works

You already have the raw signal — your secure web gateway, CASB, or IdP logs every outbound request. EvalGuard turns that noise into a ranked inventory in three steps:

pipeline

1. INGEST    POST egress-log rows → classify each domain against the
             AI-tool catalog (chat.openai.com → ChatGPT, restricted, high risk)
             → upsert into shadow_ai_sightings, summing request_count + bytes.

2. DETECT    GET rolls deduped sightings up per tool/domain, ranked by
             request volume + data risk, with affected users / teams /
             departments and an "unsanctioned" flag.

3. GOVERN    Set a per-domain policy (approved | blocked | pending). It
             propagates to sightings instantly and silences (or raises)
             violation alerts. New sightings fire shadow_ai_detected playbooks.

Classification is hostname-first (what external logs actually give you) and walks up the domain tree, so chat.openai.com resolves to the same tool as openai.com. Domains that aren't in the catalog are counted as domain_not_in_catalog skips rather than silently dropped.

1 · Ingest egress logs

POST /api/v1/shadow-ai/ingest accepts up to 10,000 rows per call from five source formats: zscaler, netskope, cloudflare, okta, or generic. The parser is tolerant — it maps each vendor's field names to a normalized sighting and skips (never throws on) malformed rows. Callers typically run this on a cron pulling the last N minutes of logs.

curl — Cloudflare Gateway rows

curl -X POST https://evalguard.ai/api/v1/shadow-ai/ingest \
  -H "Authorization: Bearer $EVALGUARD_API_KEY" \
  -H "content-type: application/json" \
  -d '{
    "source": "cloudflare",
    "projectId": "proj_123",
    "rows": [
      { "UserEmail": "dana@acme.com", "HTTPHost": "chat.openai.com",
        "Timestamp": "2026-07-15T09:12:00Z", "BytesReceived": 20481 },
      { "UserEmail": "lee@acme.com",  "HTTPHost": "otter.ai",
        "Timestamp": "2026-07-15T09:20:00Z", "BytesReceived": 88213 }
    ]
  }'

# → 201
# {
#   "success": true,
#   "data": {
#     "ingested": 2, "newSightings": 2, "updatedSightings": 0,
#     "parsedRows": 2, "skipped": 0, "byReason": {}
#   }
# }

Re-ingesting the same (project, domain, user, source) tuple sums counts rather than overwriting them (via the shadow_ai_upsert_sightings RPC), so daily log-shipping produces correct cumulative totals. Rate-limited to 10 req/min; the supplied projectId is verified to belong to the authenticated org before any write.

2 · Read the detections rollup

GET /api/v1/shadow-ai/detections aggregates the deduped sightings per tool and ranks them by request volume. Filter with category (approved / restricted / blocked / unclassified), risk (minimum data risk), status, and limit.

curl — high-risk tools only

curl -H "Authorization: Bearer $EVALGUARD_API_KEY" \
  "https://evalguard.ai/api/v1/shadow-ai/detections?projectId=proj_123&risk=high"

# → { "success": true, "data": {
#   "detections": [
#     {
#       "toolName": "Otter.ai", "domain": "otter.ai",
#       "category": "restricted", "dataRisk": "critical",
#       "policyStatus": "unreviewed", "unsanctioned": true,
#       "userCount": 14, "requestCount": 2210, "bytesTransferred": 41288810,
#       "departments": ["Sales"], "teams": ["AE-West"],
#       "firstSeen": "2026-07-01T...", "lastSeen": "2026-07-15T..."
#     }
#   ],
#   "summary": {
#     "totalTools": 23, "unsanctionedTools": 9, "totalUsers": 118,
#     "totalRequests": 40213, "highRiskTools": 6,
#     "scannedSightings": 8801, "capped": false
#   }
# } }

A tool is flagged unsanctioned when its catalog category is restricted or blocked AND no explicit approved policy exists for it. The rollup scans up to 10,000 sightings per request; summary.capped tells you when a very large tenant has hit that ceiling (a SQL-side rollup is a future optimization).

3 · Govern with per-domain policy

A policy override pins a (project, domain) pair to approved, blocked, or pending, taking precedence over the catalog default. Setting approved on grammarly.com stops it surfacing as a violation; the change propagates to existing sightings immediately, so the detections view flips without a re-ingest.

policy — set / list / delete

# Approve a tool (admin-only, audited as shadow_ai_policy_set)
curl -X POST https://evalguard.ai/api/v1/shadow-ai/policy \
  -H "Authorization: Bearer $EVALGUARD_API_KEY" \
  -H "content-type: application/json" \
  -d '{ "projectId": "proj_123", "domain": "grammarly.com",
        "status": "approved", "rationale": "Enterprise DLP-covered plan" }'

# List every override for a project
curl -H "Authorization: Bearer $EVALGUARD_API_KEY" \
  "https://evalguard.ai/api/v1/shadow-ai/policy?projectId=proj_123"

# Remove an override (reverts affected sightings to 'unreviewed')
curl -X DELETE -H "Authorization: Bearer $EVALGUARD_API_KEY" \
  "https://evalguard.ai/api/v1/shadow-ai/policy?projectId=proj_123&domain=grammarly.com"

POST and DELETE require the admin role and are written to the audit log — deciding what AI a company may use is a governance action, not a casual toggle.

Do it all from the CLI

The evalguard shadow-ai command wraps the same endpoints — point it at an NDJSON export (one JSON object per line) and it chunks the upload for you.

evalguard shadow-ai

# Upload an egress-log export, classified on ingest
evalguard shadow-ai upload egress.ndjson --source cloudflare

# Rank detected tools by request volume (add --json to pipe it)
evalguard shadow-ai detections --project proj_123 --risk high
#   Otter.ai              critical users=14    reqs=2210     ⚠ unsanctioned
#   ChatGPT               high     users=41    reqs=8830     ⚠ unsanctioned

# Set / list per-domain policy
evalguard shadow-ai policy set chat.openai.com --status blocked \
  --project proj_123 --rationale "No enterprise agreement"
evalguard shadow-ai policy list --project proj_123

Browse the AI-app catalog

GET /api/v1/shadow-ai/catalog exposes the reference set of known AI applications behind discovery — searchable by name, provider, or domain, and filterable by category (chatbot, coding, image, audio, and more). Each entry carries a data-risk rating and a trainOnData flag so you can reason about exposure per tool.

curl — search the catalog

curl -H "Authorization: Bearer $EVALGUARD_API_KEY" \
  "https://evalguard.ai/api/v1/shadow-ai/catalog?category=chatbot&search=deep"

# → { "success": true, "data": {
#   "apps": [ { "id": "deepseek", "name": "DeepSeek", "provider": "DeepSeek",
#               "domain": "chat.deepseek.com", "category": "chatbot",
#               "dataRisk": "critical", "trainOnData": true } ],
#   "totalResults": 1, "totalCatalog": 200 } }

Inline request analysis

Separately from log-based discovery, POST /api/v1/shadow-aiscores a single prompt for leak risk — useful when EvalGuard's gateway is inline and you want a verdict on one request. It runs PII and sensitive-data detection and returns a 0–100 risk score plus an action.

curl — analyze one request

curl -X POST https://evalguard.ai/api/v1/shadow-ai \
  -H "Authorization: Bearer $EVALGUARD_API_KEY" \
  -H "content-type: application/json" \
  -d '{ "input": "Summarize this record: Jane Doe, SSN 123-45-6789",
        "provider": "openai", "model": "gpt-4o" }'

# → { "success": true, "data": {
#   "event": { "authorized": true, "piiDetected": true, "piiTypes": ["ssn"],
#              "sensitiveDataDetected": false, "riskScore": 35,
#              "action": "allowed" },
#   "piiDetails": { "detected": true, "types": ["ssn"] },
#   "sensitiveDataDetails": { "detected": false, "types": [] }
# } }

Risk weighting: unauthorized model +30, PII +25, sensitive data +25, extra bumps for private keys / AWS keys / large payloads. Score ≥ 50 is flagged; an unauthorized model under a blockUnknown policy is blocked.

Acting on findings

Discovery is only useful if it drives action. The intended loop:

Triage by risk, not volume alone. A single Otter.ai user (critical, records meetings) can outrank a busy but low-risk translation tool.
Decide per domain. Approve tools you can govern (with a rationale for the audit trail), block the ones you can't, and leave the rest pending while you evaluate.
Automate the alert. Newly seen tools/users fire the shadow_ai_detected playbook with a severity derived from how many new sightings landed — wire it to Slack, a ticket, or a webhook. Re-ingests of known tuples don't re-alert, so you avoid a flood.
Close the loop. Once a policy is set, the detections dashboard reflects it immediately, so approved tools drop out of the unsanctioned count on the next view.

Honest status

What's partial

The aggregate GET /api/v1/shadow-ai dashboard currently returns model/user counts over recent gateway_logs only. Its unauthorizedUsage, piiLeakAttempts, and requestsBlocked fields are placeholders (all traffic lands in the low risk bucket). The real, risk-weighted signal lives in the ingest → detections pipeline above — use that.
Ingest classification runs against a focused catalog of the highest-risk AI tools. The larger browsable set behind /catalog is a superset; a domain only becomes a sighting if the classifier recognizes it, so unknown AI domains show up as domain_not_in_catalog skips until added.
The detections rollup is computed in-handler with a 10k-sighting scan cap per request. A SQL-side materialized rollup for very large tenants is on the roadmap; summary.capped signals when you've hit the ceiling today.