Skip to content
Model · Artifact security

Scan model filesbefore you load them.

One pickle'd os.system and your inference cluster runs arbitrary code as it loads. EvalGuard parses pickle / SafeTensors / ONNX / GGUF artifacts statically — no deserialization — and surfaces dangerous opcodes, external-data escapes, and obfuscated payloads.

Read the API referenceCLI docs
Static parse — never deserializesCLI · API · GitHub App
4
Artifact formats
0
Bytes deserialized — static parse
100
Free scans / month
2 GB
Max per URL fetch

01 · The run

Every byte inspected. None executed.

The scanner walks the artifact's structure statically — opcode by opcode, header by header — and resolves each check to a clean or flagged verdict. Clean means the structure held; flagged means a dangerous opcode or escape was found before any model loaded it.

  • Static parse — the artifact is never deserialized or run.
  • Finding-level severity counts, persisted for audit.
  • Fail your CI on any critical / high finding.
model-scan · pytorch_model.bin
REDUCE opcode
→ posix.system('…')
scanning…
STACK_GLOBAL
__builtin__.eval
safetensors header
size within bounds
onnx external-data
no path traversal
gguf metadata
magic + lengths valid
zip-wrapped pickle
state_dict scanned
running… · static parse · never executedclean 0%

02 · Coverage

Four formats. Static parsing only.

From the pickle opcode stream to GGUF magic bytes — each format has its own structural checks, and not one of them loads the artifact to find them.

Pickle / PyTorch

  • REDUCE / NEWOBJ / GLOBAL / STACK_GLOBAL opcodes
  • os.system, subprocess, eval, exec imports
  • ZIP-wrapped pickles (PyTorch state-dicts)
  • Base64 / zlib obfuscation
.pkl · .pickle · .pt · .pth · .bin

SafeTensors

  • Oversized header (DoS / parser bypass)
  • Tensor offset overflow (read-past-EOF exfiltration)
  • Metadata schema deviation
.safetensors

ONNX

  • External-data path traversal (.. / absolute paths)
  • Custom-op imports that load .so / .dll
  • Suspicious metadata producer fields
.onnx

GGUF

  • Magic-bytes mismatch
  • Metadata length overflow
  • Tensor dtype / shape anomalies
.gguf

03 · Quickstart

Run it from the terminal or the API.

CLI quickstart

Static parse — never deserializes. Runs fully offline against a local model file or directory.

bash
# Scan a local file (no API key required — runs offline)
$ evalguard model-scan ./checkpoint.pkl

# Scan everything in a directory (recurses automatically)
$ evalguard model-scan ./models/ --severity high

# JSON output for CI pipelines
$ evalguard model-scan ./checkpoint.pkl --format json

API quickstart

Scan by HuggingFace repo, S3 URI, or generic URL. Persisted to model_scans with finding-level severity counts.

curl
# Scan a HuggingFace artifact and persist the result
$ curl -X POST https://evalguard.ai/api/v1/security/model-scan \
    -H "Authorization: Bearer $EVALGUARD_API_KEY" \
    -H "Content-Type: application/json" \
    -d '{
      "projectId": "<uuid>",
      "source": "huggingface",
      "sourceRef": "meta-llama/Llama-2-7b-hf/pytorch_model.bin"
    }'

# Response:
# {
#   "scanId": "<uuid>",
#   "format": "pytorch",
#   "verdict": "warn",
#   "findings": [
#     { "severity": "high", "category": "suspicious_import",
#       "detail": "Found 'urllib.request.urlopen' in pickle stream" }
#   ],
#   "stats": { "durationMs": 412, "bytesScanned": 13476283 }
# }

04 · Honest limits

Where the scanner stops

  • Static parse only. We never deserialize the artifact. That's safer, but means we don't catch payloads that depend on a specific runtime path.
  • Heuristic, not formal. A skilled adversary can pickle-via-non-REDUCE; we flag obfuscation hints, but the final call is yours.
  • SafeTensors are mostly safe by design. The scanner still runs structural checks, but most findings will be info-level.
  • 2 GB cap per URL fetch. Larger artifacts should use the upload variant or stream directly to S3 and scan via the S3 source.