How to log context size (prompt-tokens response header) in azure apim policy

Hello @Das, D. (Debjyoti) - The thread's already pointing at per-request logging, which is the right destination. What's worth spelling out is why the metric policy can't be stretched to cover context size - otherwise it reads as a preference rather than a constraint.

Two hard limits in the llm-emit-token-metric reference. You get at most 5 custom dimensions per policy. And the active time series a policy produces is the product of the unique values across those dimensions, measured against an Azure Monitor ceiling of 50,000 active time series per region per subscription in any 12-hour window.

Prompt-token counts have effectively unbounded cardinality - every distinct value creates its own series. Multiply that against your existing client and model dimensions and you'd cross the limit almost immediately.

So per-request logging isn't just the cleaner option here - it's the only one that works.

One thing worth checking on your current setup. The same reference notes token counts are inaccurate if a stream is interrupted, and that some OpenAI models don't return them at all when streaming unless you set include_usage: true in the request. If any of your traffic streams, the numbers you're billing from today may already be incomplete.

We have Azure OpenAI instances behind Azure API Management and use the llm-emit-token-metric policy to track aggregated prompt, completion, and total tokens by client and model.

If your GPT-5 pricing varies based on context size (prompt length), you should log the prompt token count for each request and associate it with metadata such as client ID, subscription, model, and deployment.

For GPT-5 models behind Azure API Management (APIM), the recommended approach is to capture and persist the actual token usage returned by Azure OpenAI (prompt_tokens, completion_tokens, and total_tokens) rather than relying solely on the aggregated metrics generated by the llm-emit-token-metric policy. APIM token metrics are primarily intended for monitoring, analytics, and throttling scenarios, while detailed chargeback calculations typically require per-request token data.

If GPT-5 pricing tiers depend on prompt/context length, you should log the actual prompt token count for every request and categorize it into the applicable pricing bucket during ingestion or reporting. The token usage information is available in the Azure OpenAI response payload and can be captured through APIM policies, custom logging, Application Insights, Log Analytics, or Event Hub integration. A common enterprise pattern is:

1. Route all Azure OpenAI traffic through APIM.
2. Capture per-request token usage and client identifier.
3. Store detailed logs in Log Analytics/Application Insights.
4. Calculate chargeback externally using KQL, Power BI, or a billing pipeline based on the model-specific pricing rules.