Providers
84 LLM providers supported through a unified interface. Switch models with one line of config.
Categories
Configuration
Set the provider in your eval or scan config. Each provider reads its API key from environment variables.
{
"model": "gpt-4o",
"provider": "openai"
}Provider-Specific Setup
OpenAI
export OPENAI_API_KEY="sk-..."
# Config: { "provider": "openai", "model": "gpt-4o" }Anthropic
export ANTHROPIC_API_KEY="sk-ant-..."
# Config: { "provider": "anthropic", "model": "claude-sonnet-4-20250514" }Google Gemini
export GOOGLE_API_KEY="AIza..."
# Config: { "provider": "gemini", "model": "gemini-2.5-pro" }Groq
export GROQ_API_KEY="gsk_..."
# Config: { "provider": "groq", "model": "llama-3.3-70b-versatile" }Azure OpenAI
export AZURE_OPENAI_API_KEY="..."
export AZURE_OPENAI_ENDPOINT="https://your-resource.openai.azure.com"
export AZURE_OPENAI_DEPLOYMENT="gpt-4o"
# Config: { "provider": "azure-openai", "model": "gpt-4o" }AWS Bedrock
export AWS_ACCESS_KEY_ID="..."
export AWS_SECRET_ACCESS_KEY="..."
export AWS_REGION="us-east-1"
# Config: { "provider": "bedrock", "model": "anthropic.claude-3-sonnet-20240229-v1:0" }BYOK (Bring Your Own Key)
EvalGuard never stores your provider API keys. All keys are passed at runtime via environment variables or the dashboard settings.
For cloud evals/scans, configure your provider keys in the dashboard at Settings > Providers. Keys are encrypted at rest and never logged.
Self-Hosted Providers
Ollama
# Start Ollama
ollama serve
# Pull a model
ollama pull llama3.2
# Config: { "provider": "ollama", "model": "llama3.2" }
# Default endpoint: http://localhost:11434vLLM
# Start vLLM server
python -m vllm.entrypoints.openai.api_server --model meta-llama/Llama-3-8B-Instruct
# Config: { "provider": "vllm", "model": "meta-llama/Llama-3-8B-Instruct" }
# Set VLLM_BASE_URL if not localhost:8000OpenAI-Compatible
Use the openai-compatible provider to connect to any endpoint that implements the OpenAI API format.
{
"provider": "openai-compatible",
"model": "my-custom-model",
"providerOptions": {
"baseUrl": "http://localhost:8080/v1",
"apiKey": "optional-key"
}
}Major Cloud Providers
openaiOpenAI
GPT-4o, GPT-4o-mini, GPT-4, o1, o3
anthropicAnthropic
Claude Opus 4, Claude Sonnet 4, Claude Haiku
geminiGoogle Gemini
Gemini 2.5 Pro, Flash, Ultra
mistralMistral
Mistral Large, Medium, Small, Codestral
cohereCohere
Command R+, Command R, Embed
deepseekDeepSeek
DeepSeek-V3, DeepSeek-R1
xaixAI
Grok-2, Grok-2 Mini
perplexityPerplexity
Sonar Pro, Sonar
Cloud Platforms
azure-openaiAzure OpenAI
All OpenAI models via Azure
bedrockAWS Bedrock
Claude, Llama, Titan, Mistral
aws-bedrock-agentsAWS Bedrock Agents
Agent runtime
vertexGoogle Vertex AI
Gemini, PaLM, custom models
sagemakerAWS SageMaker
Custom deployed models
databricksDatabricks
DBRX, custom fine-tunes
snowflakeSnowflake Cortex
Arctic, Mistral, Llama
watsonxIBM WatsonX
Granite, Llama
ibm-bamIBM BAM
Granite, Flan
Inference APIs
groqGroq
Llama, Mixtral (ultra-fast inference)
togetherTogether AI
Llama, Mixtral, DBRX, code models
fireworksFireworks AI
Llama, Mixtral, custom
cerebrasCerebras
Llama (fastest inference)
sambanovaSambaNova
Llama, Mistral
replicateReplicate
Any open model
huggingfaceHugging Face
Inference API models
anyscaleAnyscale
Llama, Mixtral
basetenBaseten
Custom deployed models
hyperbolicHyperbolic
Open models
leptonLepton AI
LLMs and image models
modalModal
Custom serverless models
nscaleNscale
GPU cloud models
Routers & Gateways
openrouterOpenRouter
200+ models via single API
litellmLiteLLM
Unified proxy for 100+ providers
heliconeHelicone
Observability proxy
portkeyPortkey
AI gateway with fallbacks
vercel-ai-gatewayVercel AI Gateway
Vercel AI SDK
cloudflare-ai-gatewayCloudflare AI Gateway
Edge AI
envoy-ai-gatewayEnvoy AI Gateway
Service mesh AI
f5-gatewayF5 Gateway
Enterprise gateway
Self-Hosted / Local
ollamaOllama
Any GGUF model locally
vllmvLLM
High-throughput serving
localaiLocalAI
Local OpenAI-compatible
llamacppllama.cpp
GGUF models via HTTP
llamafileLlamafile
Single-file executable models
text-generation-webuiText Generation WebUI
oobabooga models
docker-model-runnerDocker Model Runner
Docker-based inference
docker-providerDocker Provider
Custom container models
transformers-jsTransformers.js
In-browser inference
openllmOpenLLM
BentoML serving
Specialized & SDK Providers
openai-agentsOpenAI Agents
Agent SDK
openai-chatkitOpenAI ChatKit
Chat completions kit
openai-codex-sdkOpenAI Codex SDK
Codex API
claude-agent-sdkClaude Agent SDK
Anthropic agent framework
mcp-providerMCP Provider
Model Context Protocol
elevenlabsElevenLabs
Text-to-speech
falFal
Image generation
gradioGradio
Gradio app endpoints
modelslabModelsLab
Image/video models
voyageVoyage AI
Embeddings
Other Platforms
ai21AI21 Labs
Jamba
aiml-apiAIML API
Various models
alibabaAlibaba Qwen
Qwen models
cloudflareCloudflare Workers AI
Edge models
clouderaCloudera
Enterprise AI
cometapiComet API
ML tracking
github-modelsGitHub Models
GitHub-hosted models
jfrog-mlJFrog ML
Model registry
llama-apiLlama API
Meta Llama hosting
openclawOpenClaw
Open models
quiveraiQuiverAI
AI platform
truefoundryTrueFoundry
ML platform
Protocol & Custom
openai-compatibleOpenAI Compatible
Any OpenAI-compatible API
custom-httpCustom HTTP
Any REST endpoint
webhookWebhook
Custom webhook endpoint
websocket-providerWebSocket
WebSocket streaming
python-providerPython Script
Run Python for inference
go-providerGo Script
Run Go for inference
ruby-providerRuby Script
Run Ruby for inference
scriptScript
Any shell script
echoEcho
Echo input back (testing)
manual-inputManual Input
Human-in-the-loop
simulated-userSimulated User
LLM-based user simulation
web-browserWeb Browser
Browser-based interaction
sequenceSequence
Chain multiple providers
slack-providerSlack
Slack bot responses