AI
gateway & proxy
The PromptGlass Gateway is an AI-aware proxy. As a forward proxy it inspects traffic to AI domains only. As an API gateway it speaks the OpenAI and Anthropic APIs, so apps switch by changing a base URL and gain virtual keys, budgets, model routing, caching and inline data-leakage protection.
Outcomes
- ✓Every LLM call attributed and governed
- ✓Provider keys out of app configs
- ✓Freedom to switch models without code changes
What you get
Inside PromptGlass ai gateway & proxy
Drop-in compatible
Set OPENAI_BASE_URL or ANTHROPIC_BASE_URL. Streaming, tools and batch work unchanged.
Virtual keys
Issue keys per team or app, rotate them centrally, and never hand out provider keys again.
Routing and fallback
Route by policy, cost or availability across OpenAI, Anthropic, Azure OpenAI, Bedrock and Vertex AI.
Scales horizontally
Stateless pods behind your ingress with autoscaling. Runs in your cluster or ours.
Rollout
Four steps, no big bang
Start in observe-only mode and tighten controls as you learn what your organization needs.
- 1
Deploy
Helm install into your cluster, or use PromptGlass Cloud.
- 2
Register providers
Add upstream keys once, in the gateway vault.
- 3
Issue virtual keys
One per team or app, with budget and policy attached.
- 4
Switch base URLs
Apps keep their SDKs; only the endpoint changes.
AI gateway & proxy: FAQ
What is an AI gateway?
+
An AI gateway is a proxy between your applications or users and AI model providers. It adds authentication, cost tracking, routing, caching and security controls to every model call without changing application code.
Does the gateway add latency?
+
Inline inspection adds a small overhead measured in milliseconds, and the gateway can run in the same cluster or region as your applications to keep it low.
Find out how your company really uses AI.
Start a free 30-day AI usage assessment: a complete shadow AI inventory, the sensitive data leaving, and the AI seats you are paying for but not using.