Gateway API mode

In API mode the PromptGlass Gateway exposes OpenAI-compatible and Anthropic-compatible endpoints. Your internal apps, agents and scripts change one setting, the base URL, and every model call then gets attribution, AI DLP, budgets, rate limits and cost accounting. Streaming, tool calls, vision inputs and embeddings pass through unchanged.

Endpoints

Protocol	Base URL	Covers
OpenAI	`https://ai-gateway.example.com/v1`	Chat Completions, Responses, Embeddings, Images, Audio, Models
Anthropic	`https://ai-gateway.example.com/anthropic`	Messages (incl. streaming, tools), Token counting
Azure OpenAI	`https://ai-gateway.example.com/azure/<deployment>`	Deployment-style routes
Bedrock / Vertex	`https://ai-gateway.example.com/bedrock`, `/vertex`	Signed passthrough with attribution

Virtual API keys

Apps don’t hold real provider keys. Create a virtual key per team or app. The gateway maps it to the real key, kept in the gateway’s secret store, and attaches the owner, cost center and policy scope.

promptglass keys create --name "support-bot" --team "Customer Support" \
  --cost-center CC-220 --budget-monthly 1500 --models "gpt-4.1-mini,claude-haiku-4-5"
# → pgk_live_9f2c…   (shown once)

Use it from code

from openai import OpenAI
import anthropic

openai_client = OpenAI(
    base_url="https://ai-gateway.example.com/v1",
    api_key="pgk_live_9f2c...",          # PromptGlass virtual key
)
r = openai_client.chat.completions.create(
    model="gpt-4.1-mini",
    messages=[{"role": "user", "content": "Summarize this ticket..."}],
    extra_headers={"x-promptglass-user": "jane.doe@example.com"},  # optional end-user attribution
)

claude = anthropic.Anthropic(
    base_url="https://ai-gateway.example.com/anthropic",
    api_key="pgk_live_9f2c...",
)
msg = claude.messages.create(
    model="claude-sonnet-4-5", max_tokens=1024,
    messages=[{"role": "user", "content": "Draft a reply..."}],
)

import OpenAI from 'openai';
import Anthropic from '@anthropic-ai/sdk';

const openai = new OpenAI({
  baseURL: 'https://ai-gateway.example.com/v1',
  apiKey: process.env.PROMPTGLASS_KEY,
});
const stream = await openai.chat.completions.create({
  model: 'gpt-4.1-mini',
  stream: true,
  messages: [{ role: 'user', content: 'Classify this email…' }],
});
for await (const chunk of stream) process.stdout.write(chunk.choices[0]?.delta?.content ?? '');

const claude = new Anthropic({
  baseURL: 'https://ai-gateway.example.com/anthropic',
  apiKey: process.env.PROMPTGLASS_KEY,
});

# Most SDKs and tools read these variables, so no code change is needed
export OPENAI_BASE_URL=https://ai-gateway.example.com/v1
export OPENAI_API_KEY=pgk_live_9f2c...
export ANTHROPIC_BASE_URL=https://ai-gateway.example.com/anthropic
export ANTHROPIC_API_KEY=pgk_live_9f2c...

Routing, budgets and limits

routes:
  - match: { model: "gpt-4o" }
    target: azure-openai-eu        # same model, company tenant, EU data residency
  - match: { model: "claude-*" }
    target: anthropic-direct
    fallback: bedrock-claude       # used on 5xx/429 from primary
budgets:
  - scope: { team: "Marketing" }
    monthly_usd: 4000
    on_exceed: coach               # coach | block | route:gpt-4.1-mini
rate_limits:
  - scope: { key: "*" }
    requests_per_minute: 600
cache:
  enabled: true                    # identical prompt+params within TTL return the cached response
  ttl: 10m
  exclude_data_classes: [pii, phi]

Response headers

Every response includes x-promptglass-event-id, x-promptglass-action (allow, redact, …) and x-promptglass-cost-usd, so apps can log them alongside their own traces.

Errors returned to callers

Situation	HTTP	Body `error.type`
Policy block	403	`promptglass_policy_block` (includes the user-facing message)
Budget exceeded (block mode)	429	`promptglass_budget_exceeded`
Upstream unavailable, no fallback	502	`upstream_error`

Errors follow the provider’s error schema, so existing SDK error handling keeps working.