For the complete documentation index, see llms.txt. Markdown versions of all docs pages are available by appending .md to any docs URL.
Cost controls
Attribute, observe, enforce, and optimize LLM spend through agentgateway.
Control LLM spend end to end, in three stages:
- Attribute: Tie usage to teams and keys with virtual keys, and carry the attribution value onto the cloud provider’s own bill with invoice-grade attribution.
- Observe: Compute the realized USD cost of every request with a model cost catalog, see spend by model, provider, and user in the built-in cost dashboard, and view it in metrics, logs, and traces with Observe traffic.
- Enforce: Cap token usage and dollar spend with budget and spend limits, either as a per-key budget or as a rate limit token budget.
Virtual key management
Issue API keys with per-key token budgets and cost tracking (also known as virtual keys).
Invoice-grade attribution
Carry a validated caller identity into AWS and Google Cloud billing records, so LLM spend is …
Model costs
Price LLM requests with a model cost catalog and expose realized USD costs in logs, traces, metrics, …
Cost dashboard
View LLM spend, tokens, and traffic in the built-in UI, grouped by model, provider, and user.
Budget and spend limits
Cap LLM spend per API key in dollars or tokens, with per-key budgets or with rate limiting.
