Provider Catalog

Providers

77+ first-party LLM providers through a unified interface, plus 800+ models reachable via the gateway-aggregator providers (OpenRouter / LiteLLM / CometAPI). Switch models with one line of config.

Tip — Need a model not in our first-party catalog?

Wire OpenRouter (200+ models), LiteLLM (self-host, 100+ providers),or CometAPI (500+ models) as a provider and the entire aggregator catalog becomes available to evals, gateway, and red-team — no provider count growth, no integration work. Equivalent total reach to Promptfoo and Portkey, single billing relationship.

Configuration

Set the provider in your eval or scan config. Each provider reads its API key from environment variables.

eval-config.json

{
  "model": "gpt-4o",
  "provider": "openai"
}

Provider-Specific Setup

OpenAI

terminal

export OPENAI_API_KEY="sk-..."
# Config: { "provider": "openai", "model": "gpt-4o" }

Anthropic

terminal

export ANTHROPIC_API_KEY="sk-ant-..."
# Config: { "provider": "anthropic", "model": "claude-sonnet-4-20250514" }

Google Gemini

terminal

export GEMINI_API_KEY="AIza..."
# Config: { "provider": "gemini", "model": "gemini-2.5-pro" }

Groq

terminal

export GROQ_API_KEY="gsk_..."
# Config: { "provider": "groq", "model": "llama-3.3-70b-versatile" }

Azure OpenAI

terminal

export AZURE_OPENAI_API_KEY="..."
export AZURE_OPENAI_ENDPOINT="https://your-resource.openai.azure.com"
export AZURE_OPENAI_DEPLOYMENT="gpt-4o"
# Config: { "provider": "azure-openai", "model": "gpt-4o" }

AWS Bedrock

Bedrock credentials are not read from environment variables. Add your AWS access key ID, secret access key, and region (default us-east-1) via the dashboard BYOK form at Settings > Providers, then reference the provider in your config. Use the full Bedrock model identifier (e.g. anthropic.claude-3-sonnet-20240229-v1:0).

eval-config.json

{
  "provider": "bedrock",
  "model": "anthropic.claude-3-sonnet-20240229-v1:0"
}

BYOK (Bring Your Own Key)

EvalGuard never stores your provider API keys. All keys are passed at runtime via environment variables or the dashboard settings.

For cloud evals/scans, configure your provider keys in the dashboard at Settings > Providers. Keys are encrypted at rest and never logged.

Self-Hosted Providers

Ollama

terminal

# Start Ollama
ollama serve

# Pull a model
ollama pull llama3.2

# Config: { "provider": "ollama", "model": "llama3.2" }
# Default endpoint: http://localhost:11434

vLLM

terminal

# Start vLLM server
python -m vllm.entrypoints.openai.api_server --model meta-llama/Llama-3-8B-Instruct

# Config: { "provider": "vllm", "model": "meta-llama/Llama-3-8B-Instruct" }
# Set VLLM_BASE_URL if not localhost:8000

OpenAI-Compatible Endpoints

To point EvalGuard at a self-hosted endpoint that implements the OpenAI API format, use one of the registered self-hosting providers. The localai provider defaults to http://localhost:8080/v1:

eval-config.json

{
  "provider": "localai",
  "model": "my-custom-model"
}

For other OpenAI-compatible servers use vllm, llamacpp, or llamafile. The base URL is taken from the provider default or its environment override (e.g. VLLM_BASE_URL for vLLM, which defaults to http://localhost:8000/v1).

Major Cloud Providers

openai

OpenAI

GPT-4o, GPT-4o-mini, GPT-4, o1, o3

anthropic

Anthropic

Claude Opus 4, Claude Sonnet 4, Claude Haiku

gemini

Google Gemini

Gemini 2.5 Pro, Flash, Ultra

mistral

Mistral

Mistral Large, Medium, Small, Codestral

cohere

Cohere

Command R+, Command R, Embed

deepseek

DeepSeek

DeepSeek-V3, DeepSeek-R1

xai

xAI

Grok-2, Grok-2 Mini

perplexity

Perplexity

Sonar Pro, Sonar

Cloud Platforms

azure-openai

Azure OpenAI

All OpenAI models via Azure

bedrock

AWS Bedrock

Claude, Llama, Titan, Mistral

aws-bedrock-agents

AWS Bedrock Agents

Agent runtime

vertex

Google Vertex AI

Gemini, PaLM, custom models

sagemaker

AWS SageMaker

Custom deployed models

databricks

Databricks

DBRX, custom fine-tunes

snowflake

Snowflake Cortex

Arctic, Mistral, Llama

watsonx

IBM WatsonX

Granite, Llama

ibm-bam

IBM BAM

Granite, Flan

Inference APIs

groq

Groq

Llama, Mixtral (ultra-fast inference)

together

Together AI

Llama, Mixtral, DBRX, code models

fireworks

Fireworks AI

Llama, Mixtral, custom

cerebras

Cerebras

Llama (fastest inference)

sambanova

SambaNova

Llama, Mistral

replicate

Replicate

Any open model

huggingface

Hugging Face

Inference API models

anyscale

Anyscale

Llama, Mixtral

baseten

Baseten

Custom deployed models

hyperbolic

Hyperbolic

Open models

lepton

Lepton AI

LLMs and image models

modal

Modal

Custom serverless models

nscale

Nscale

GPU cloud models

Routers & Gateways

openrouter

OpenRouter

200+ models — OpenAI / Anthropic / Google / Meta / Mistral / DeepSeek / Qwen, one endpoint

litellm

LiteLLM

Self-hosted proxy aggregating 100+ providers under one OpenAI-compatible API

cometapi

CometAPI

500+ models across all major providers — single billing relationship

helicone

Helicone

Observability proxy with caching + cost tracking

portkey

Portkey

AI gateway with fallbacks + semantic cache (competitor; we wrap it for migration)

vercel-ai-gateway

Vercel AI Gateway

Vercel AI SDK with edge routing

cloudflare-ai-gateway

Cloudflare AI Gateway

Edge AI proxy with caching

envoy-ai-gateway

Envoy AI Gateway

Service mesh AI for k8s deployments

f5-gateway

F5 Gateway

Enterprise gateway with WAF integration

Self-Hosted / Local

ollama

Ollama

Any GGUF model locally

vllm

vLLM

High-throughput serving

localai

LocalAI

Local OpenAI-compatible

llamacpp

llama.cpp

GGUF models via HTTP

llamafile

Llamafile

Single-file executable models

text-generation-webui

Text Generation WebUI

oobabooga models

docker-model-runner

Docker Model Runner

Docker-based inference

docker

Docker Provider

Custom container models

transformers-js

Transformers.js

In-browser inference

openllm

OpenLLM

BentoML serving

Specialized & SDK Providers

openai-agents

OpenAI Agents

Agent SDK

openai-chatkit

OpenAI ChatKit

Chat completions kit

openai-codex-sdk

OpenAI Codex SDK

Codex API

claude-agent-sdk

Claude Agent SDK

Anthropic agent framework

mcp

MCP Provider

Model Context Protocol

elevenlabs

ElevenLabs

Text-to-speech

fal

Fal

Image generation

gradio

Gradio

Gradio app endpoints

modelslab

ModelsLab

Image/video models

voyage

Voyage AI

Embeddings

Other Platforms

ai21

AI21 Labs

Jamba

aiml-api

AIML API

Various models

alibaba

Alibaba Qwen

Qwen models

cloudflare

Cloudflare Workers AI

Edge models

cloudera

Cloudera

Enterprise AI

github-models

GitHub Models

GitHub-hosted models

jfrog-ml

JFrog ML

Model registry

llama-api

Llama API

Meta Llama hosting

openclaw

OpenClaw

Open models

quiverai

QuiverAI

AI platform

truefoundry

TrueFoundry

ML platform

Protocol & Custom

custom-http

Custom HTTP

Any REST endpoint

webhook

Webhook

Custom webhook endpoint

websocket

WebSocket

WebSocket streaming

python

Python Script

Run Python for inference

go

Go Script

Run Go for inference

ruby

Ruby Script

Run Ruby for inference

script

Script

Any shell script

echo

Echo

Echo input back (testing)

manual-input

Manual Input

Human-in-the-loop

simulated-user

Simulated User

LLM-based user simulation

web-browser

Web Browser

Browser-based interaction

sequence

Sequence

Chain multiple providers

slack

Slack

Slack bot responses