Gemini

gemini

YAML config

providers:
  - id: gemini:gemini-2.0-flash
    config:
      apiKey: ${GEMINI_API_KEY}

TypeScript usage

import { createProvider } from "@evalguard/core";

const provider = createProvider("gemini", process.env.GEMINI_API_KEY);
const response = await provider.complete({
  model: "gemini-2.0-flash",
  messages: [{ role: "user", content: "Hello" }],
});

Authentication

Set GEMINI_API_KEY in your environment. EvalGuard validates the key on first call and surfaces typed errors for 401 / 403 / rate-limit responses (with Retry-After parsing).

Setup walkthrough

  1. 1. Get an API key at https://aistudio.google.com/app/apikey (Google AI Studio — separate from Vertex).
  2. 2. Set in env: `export GEMINI_API_KEY=...` or `GOOGLE_API_KEY=...`.
  3. 3. Configure: `providers: [{id: 'gemini:gemini-2.0-flash', config: {apiKey: '${GEMINI_API_KEY}'}}]`.
  4. 4. For enterprise / GCP-tenant deployments, switch to the Vertex provider instead — same wire format but OAuth Bearer auth + project-scoped quotas.

Gotchas

  • Gemini API and Vertex AI share the same wire format BUT different auth (API key vs OAuth Bearer) and different URL structure. EvalGuard's Gemini and Vertex providers are separate classes — Vertex subclasses Gemini.
  • Image input requires base64-encoded `inlineData` parts — Gemini does NOT accept hosted image URLs in the request body. EvalGuard collapses `image_url` parts to a text placeholder; pre-fetch + base64 client-side if you need vision.
  • `promptFeedback.blockReason: 'SAFETY'` returns a 200 OK with no candidates. EvalGuard maps this to `finishReason: 'content_filter'` so you can branch on it.
  • Streaming uses a separate URL (`:streamGenerateContent`) instead of a body flag. EvalGuard's stream() handles this transparently — `isStreamingProvider(p)` returns true.

Cost note

gemini-2.0-flash: $0.10/M input, $0.40/M output (cheapest among major providers). gemini-2.5-pro: $1.25/M input, $10/M output. Google's free tier on AI Studio: 1500 requests/day (great for dev).

Recommended models

Eval / judge
gemini-2.0-flash — cheapest + 1M context
Agent / tool-use
gemini-2.5-pro — best for multi-step reasoning + tool use
Code
gemini-2.5-pro — solid at large-codebase reasoning
Vision
gemini-2.5-pro — strongest at video + multi-image

Hand-written · 2026-05-21