Custom AI gateway allocation is in Preview.
Overview
If your teams send LLM requests through a shared AI gateway, the provider bill shows that traffic under a single API key or project. Connect your AI gateway to split that spend across the teams, users, or services that sent the requests, using token usage reported by your gateway.
The Connect your AI Gateway flow generates Custom Allocation Rules that divide each provider bill in proportion to the token usage your gateway reports.
Requirements
To connect your AI gateway, you need:
- At least one of the following AI providers set up for Cloud Cost Management:
- A source of gateway token usage:
- The
cloud_cost_management_write permission
A provider is available to allocate only if its bill is connected to Cloud Cost Management and your usage source reports requests to that provider.
Create gateway allocation rules
Navigate to Cloud Cost > Summarize > AI and click Connect your AI Gateway.
Select usage metric: Choose where your usage telemetry comes from.
- Select Agent Observability or LiteLLM.
- Filter the usage metric to your gateway traffic (for example,
service:ai_gateway). For LiteLLM, you can instead select Allocate my costs based on the entire metric. - Choose up to three tags to break your AI spend down by, such as
team, user, or service.
Match to bills: Select the provider bills to split. For each provider, add a bill filter that identifies the gateway’s spend on that bill (for example, api_key_name:gateway_api_key for Anthropic or project_name:ai_gateway for OpenAI). Bill filters support the is operator.
Confirm coverage: Review the percentage of provider-reported tokens that your usage source accounts for, for each token type, over a seven-day window. If coverage exceeds 100% for a token type, check that your usage filter and bill filter cover the same API key or project. For more information, see How gateway costs are allocated.
The flow creates one allocation rule for each provider and token type, such as input, output, cache read, and cache write tokens.
After the rules are processed, the tags you selected are added to your provider costs. Use these tags across Cloud Cost Management, including Cloud Cost Explorer, dashboards, Cost Reports, monitors, and budgets.
How gateway costs are allocated
Gateway allocation rules compare your gateway’s token usage against the token usage reported by the provider for the same bill scope. For each model and day, coverage is the gateway’s tokens divided by the provider-reported tokens. For example, 80,000 output tokens reported by your gateway out of 100,000 output tokens reported by the provider is 80% coverage.
- The covered share of cost is split across the tags you selected in proportion to each tag value’s token usage.
- The remaining share of cost is attributed to
uninstrumented. This represents spend on the bill scope that your gateway did not report, such as requests that bypass the gateway. - Coverage is capped at 100%. If your gateway reports more tokens than the provider, all of the cost is split across your selected tags.
- If the provider reports no token usage for a model on a given day, those costs are not allocated.
Manage gateway allocation rules
Rules created by the Connect your AI Gateway flow appear in the AI Gateway Rules section of the Custom Allocation Rules page, grouped by provider. You can view or delete gateway rules, but you cannot edit or disable them. To change a gateway configuration, delete its rules and run the Connect your AI Gateway flow again.
Further reading
Additional helpful documentation, links, and articles: