How do LLM token costs work?
Definition
LLM token costs are the charges model providers bill for API use, calculated from the number of input tokens (the prompt and context) and output tokens (the response), each at a per-model rate. Tokens are word fragments; long contexts, agent loops and large models drive cost up.
What drives cost
Model choice, context length (system prompts, documents, conversation history), output length, retries and agent loops that make many calls per task. Prompt caching and batch APIs can lower the effective rate.
Seats vs. tokens
Most organizations pay twice: per-seat subscriptions for chat and copilot apps, and per-token API spend for internal apps. Both need attribution to teams to be managed.
Reducing spend
Right-size models per task, cache repeated prompts, trim context, cap agent loops, set budgets per team, and reclaim idle seats.