What is an AI gateway?
Definition
An AI gateway (also called an LLM proxy or LLM gateway) is a proxy that sits between users or applications and AI model providers. It authenticates callers, records usage and cost, routes requests to the right model, and applies security controls such as data-leakage detection, without changing application code.
How an AI gateway works
Applications send requests to the gateway instead of the provider, usually by changing a base URL such as OPENAI_BASE_URL. The gateway checks the caller’s key, applies policy, optionally rewrites or redacts the prompt, forwards it to the chosen provider, streams the response back, and logs tokens, cost and latency.
Forward proxy vs. API gateway
A forward proxy intercepts browser and desktop traffic to AI websites. An API gateway exposes a provider-compatible API for software. PromptGlass runs both modes in one gateway.
When you need one
When several teams share provider keys, when you cannot attribute AI spend, when prompts may contain customer data, or when you want to switch models without code changes.