Azure OpenAI

azure-openai

YAML config

providers:
  - id: azure-openai:<your-model>
    config:
      apiKey: ${AZURE_OPENAI_API_KEY}

TypeScript usage

import { createProvider } from "@evalguard/core";

const provider = createProvider("azure-openai", process.env.AZURE_OPENAI_API_KEY);
const response = await provider.complete({
  model: "<your-model>",
  messages: [{ role: "user", content: "Hello" }],
});

Authentication

Set AZURE_OPENAI_API_KEY in your environment. EvalGuard validates the key on first call and surfaces typed errors for 401 / 403 / rate-limit responses (with Retry-After parsing).

Setup walkthrough

  1. 1. Create an Azure OpenAI resource in your tenant (https://portal.azure.com → Create resource → Azure OpenAI). Pick a region with the model family you need (e.g. eastus2 for gpt-4o).
  2. 2. Deploy a model: Resource → Model deployments → Create. Note the *deployment name* — Azure puts this in the URL, NOT a model identifier.
  3. 3. Get the key + endpoint: Resource → Keys and Endpoint. Endpoint format is `https://{resource-name}.openai.azure.com`.
  4. 4. Configure: `providers: [{id: 'azure-openai:my-gpt4o-deployment', config: {apiKey, endpoint: 'https://myres.openai.azure.com', apiVersion: '2024-10-21'}}]`. The `model` ID is the *deployment name*, not the underlying model.
  5. 5. Smoke test the URL: `curl -H 'api-key: $AZURE_KEY' https://myres.openai.azure.com/openai/deployments/my-gpt4o-deployment/chat/completions?api-version=2024-10-21 -d '{"messages":[{"role":"user","content":"hi"}]}'`.

Gotchas

  • Auth uses `api-key` header, NOT `Authorization: Bearer`. Common 401 cause.
  • The deployment name is per-region. A deployment that exists in eastus2 doesn't exist in westus.
  • Azure's content-filter intercepts BEFORE the model. A 400 with `content_filter` in the body means Azure's safety layer rejected the prompt, not the model. EvalGuard's ProviderError surfaces this via the `body` field.
  • `api-version` is mandatory in the query string. 2024-10-21 is the current GA spec; older versions miss tool calling / vision / response_format.

Cost note

Same per-token pricing as OpenAI, BUT enterprise discounts apply (negotiate via your AE). Provisioned Throughput Units (PTU) flatten cost into a monthly bucket — ideal for steady eval workloads.

Recommended models

Eval / judge
gpt-4o-mini deployment — same pricing as OpenAI direct
Agent / tool-use
gpt-4o deployment — full feature parity with OpenAI direct
Vision
gpt-4o deployment — vision support depends on the regional model variant

Hand-written · 2026-05-21