Bemærk
Adgang til denne side kræver godkendelse. Du kan prøve at logge på eller ændre mapper.
Adgang til denne side kræver godkendelse. Du kan prøve at ændre mapper.
Observability helps you understand how model router handles requests, verify routing behavior, and investigate latency, errors, and fallback. Request-level signals complement aggregate metrics and logs, giving developers and operators context to evaluate application performance.
This article covers the per-request routing metadata preview for the Chat Completions API. The metadata identifies the serving model and describes routing attempts for an individual request. For aggregate metrics and logs, see Monitor model deployments.
Prerequisites
- Python 3.9 or later.
- The
openai>=1.75.0andpython-dotenvpackages. Install them by runningpip install "openai>=1.75.0" python-dotenv. - A model router deployment that you can access through an Azure OpenAI endpoint.
- The endpoint and API key for your Azure OpenAI resource. The complete sample reads them from the
AZURE_OPENAI_ENDPOINTandAZURE_OPENAI_API_KEYenvironment variables. - Azure OpenAI API version
2024-10-21.
Enable per-request routing metadata
After your application reads the endpoint and API key into endpoint and api_key, create the client with the preview feature header:
client = AzureOpenAI(
azure_endpoint=endpoint,
api_key=api_key,
api_version="2024-10-21",
default_headers={"Foundry-Features": "ModelRouterControls=V1Preview"},
)
The Foundry-Features: ModelRouterControls=V1Preview header requests per-request routing metadata. Because this feature is in preview, the metadata presence and response schema can vary by request and service version.
Send a Chat Completions request
Use the model router deployment name to send a request. The response includes the completion and, when available, the per-request routing metadata:
response = client.chat.completions.create(
model=deployment,
messages=[
{"role": "system", "content": "You are a helpful assistant."},
{
"role": "user",
"content": "In one sentence, name the most popular tourist destination in Seattle.",
},
],
)
Understand routing metadata
The following model_selection_details fragment illustrates a request with two ordered model attempts:
{
"model_selection_details": {
"model_router_details": {
"mode": "balanced",
"routing_trace": [
{
"latency_ms": 19,
"attempts": [
{
"model": "example-model-a",
"result": {
"status": 404,
"error": {
"code": "NotFound",
"message": "The request failed."
}
}
},
{
"model": "example-model-b",
"result": {
"status": 200
}
}
]
}
]
}
}
}
modeis the routing mode returned for the request.routing_tracecontains the routing entries returned for the request.latency_msis the latency reported for a routing-trace entry.attemptslists model attempts in order.- Each attempt contains a model and an HTTP status in
result.status. - A failed attempt can include an optional
errorwith a code and message.
Extract routing and fallback information
After the Chat Completions request returns response, inspect the serving model and model selection details:
print(f"\nRouted to model: {response.model}")
print("--- Model Selection Details ---")
model_selection_details = getattr(response, "model_selection_details", None)
if not model_selection_details:
print("No model selection details were returned.")
else:
model_router_details = model_selection_details.get("model_router_details", {})
print(f"Routing mode: {model_router_details.get('mode', 'unknown')}")
routing_trace = model_router_details.get("routing_trace", [])
if not routing_trace:
print("No routing trace was returned.")
for decision_number, routing_decision in enumerate(routing_trace, start=1):
latency_ms = routing_decision.get("latency_ms")
latency = f"{latency_ms} ms" if latency_ms is not None else "not reported"
print(f"Routing decision {decision_number} (latency: {latency})")
for attempt_number, attempt in enumerate(
routing_decision.get("attempts", []), start=1
):
result = attempt.get("result", {})
status = result.get("status", "unknown")
outcome = (
"selected"
if isinstance(status, int) and 200 <= status < 300
else "failed"
)
print(
f" Attempt {attempt_number}: {attempt.get('model', 'unknown')} - HTTP {status} ({outcome})"
)
error = result.get("error")
if error:
print(
f" Error: {error.get('code', 'unknown')} - {error.get('message', 'No message')}"
)
print("\n")
Ordered attempts can reveal automatic fallback for an individual request. In the example response, the failed attempt followed by a successful attempt is evidence of fallback for that request. Requests don't always include multiple attempts, so don't expect fallback on every request.
For complete application setup and runnable examples, see the Foundry Model Router samples.
Interpret the results
The following output shows the response and routing metadata for an example request:
--- Chat Completions Response ---
Response:Pike Place Market is Seattle's most popular tourist destination.
Usage: 29 prompt + 278 completion = 307 total tokens
Routed to model: gpt-5-mini-2025-08-07
--- Model Selection Details ---
Routing mode: balanced
Routing decision 1 (latency: 19 ms)
Attempt 1: grok-4-1-fast-reasoning - HTTP 404 (failed)
Error: NotFound - The request failed with HTTP status code 404 (NotFound).
Attempt 2: gpt-5-mini - HTTP 200 (selected)
- If
model_selection_detailsis absent, the sample reports that no model selection details were returned. Don't infer routing details that aren't present. - If
routing_traceis empty, the sample reports that no routing trace was returned. - An attempt can omit
error. The extraction code prints an error only when the response includes one. - Model names, HTTP statuses, attempt counts, and reported latency can vary by request and service version.
response.modelidentifies the serving model for the demonstrated request.