Monitor real-time voice agents by using Application Insights

By integrating Application Insights with your real-time voice agent, you can monitor, troubleshoot, and analyze conversations in near real-time.

By using this integration, you can:

• Track end-to-end voice conversations from ring to hang-up.
• Measure caller-perceived response latency.
• Diagnose tool execution and failures.
• Understand how your agent behaves across different scenarios.

Prerequisites

Before using diagnostics for real-time voice agents, make sure that you:

• Have an Azure Application Insights resource.
• Enable real-time voice on your agent.

Set up Application Insights integration

To enable diagnostics, configure Application Insights for your environment.

How the data is organized

All real-time voice telemetry lands in the traces table in Application Insights. Before you query, keep in mind the following points:

• The meaningful content is in the customDimensions column. The top-level message field is a placeholder (log record), so don't rely on it.
• Every row represents one event from a call, identified by customDimensions.Subject.
• Every event a call produces shares a single correlationId. That's how you pull one call's rows out of the table.
• Always filter by Subject; rows without one belong to other telemetry pipelines, not the voice agent.

Event types

Each row's customDimensions.Subject tells you what kind of event it is:

Field reference

Every event carries the base CallLifecycle context in addition to its own fields in the following sections.

CallLifecycle (base context on every event)

CallLifecycle is the base event that every other event carries. Every row in the traces table—DialogLifecycle, SpeechPipeline, LlmInvocation, ToolExecution—includes the CallLifecycle context. CallLifecycle is also emitted as a standalone event when the call ends.

• correlationId: The single ID that ties all of a call's rows together. Use this in every query to isolate a call.
• OrganizationId: The identifier of the organization to which the call belongs. This value also serves as the Application Insights routing key.
• Timestamp: When the event happened.
• EventType: The specific moment or event this row marks. The value depends on whether CallLifecycle is composed with another event or standalone:
  • When composed with another event, EventType takes the value of that event: DialogStarted, DialogCompleted, SpeechPipeline, LlmInvocation, or ToolExecution.
  • When CallLifecycle is emitted standalone, the only value is CallDisconnected (the call ended).
• ChannelType: Voice. Currently populated only on SpeechPipeline rows.

DialogLifecycle

• DialogId: ID of this dialog (agent session).
• BotId: Which AI agent handled the dialog. Filter on this to isolate a specific agent's calls.
• DialogStatus: Started or Completed (emitted as a numeric enum: 0 = Started, 1 = Completed).
• DialogDuration_Ms: How long the dialog ran, in milliseconds. Populated only on DialogCompleted rows.
• EndReason: Why the dialog ended. Populated only on DialogCompleted rows.
  • CallDisconnected: The call ended (caller hangup or graceful agent end; the two aren't distinguished).
  • StopDialog: The dialog ended without the call ending (for example, a transfer to another dialog).

Emitted as a numeric enum (1 = CallDisconnected). Convert with iff(toint(customDimensions.EndReason) == 1, "CallDisconnected", "StopDialog").

SpeechPipeline

• PipelineType: The voice pipeline used, such as GenAI_Realtime for real-time speech-to-speech. The value is emitted as a numeric enum.
• AudioLatency_Ms: A setup marker for how long the call's audio session took to start, typically a few milliseconds. This value isn't an audio delay or a network round-trip. It's an internal readiness signal at call start.

LlmInvocation (one row per model reply)

• Duration_Ms: Caller-perceived reply latency, also called Time to First Audio Back (TTFAB). Measured from when the user stops speaking to when the first audio byte reaches the caller from any source. This source can be the model's spoken reply, a tool's TTS prompt, or a system-generated message. Smaller values are snappier, and sub-second is typical.
• Model: The model deployment that produced the reply, such as gpt-realtime-2025-08-28.
• InputTokens: Tokens the model reads for this reply, including the system prompt, conversation history, and latest user input. This number usually grows over a call as history accumulates.
• OutputTokens: Tokens the model produces for this reply.

For more information about how Duration_Ms behaves on multistep turns, see How latency is measured (voice-specific).

ToolExecution (one row per tool the model calls)

• ToolName: The tool or function invoked.
• ToolResult: Success or Failed, indicating whether the tool call threw an error.
• TotalDuration_Ms: Total time the tool took from the agent runtime's perspective.
• McsProcessingTime_Ms: Processing time on the platform side.
• ExternalCallTime_Ms: The external-service portion of a tool call. This value is populated only when the tool makes an external call. It might be empty for internal tools.

Sample queries

Use the following Kusto queries in Application Insights to analyze real-time voice agent telemetry.

Find a specific conversation

When you know the correlationId of the call you're investigating, use this query to return every row from that call in order:

let corr = "<paste-correlationId>";
traces
| extend cl = parse_json(tostring(customDimensions.CallLifecycle))
| where tostring(cl.TrackingContext.CorrelationId) == corr
| project
    timestamp,
    subject   = tostring(customDimensions.Subject),
    eventType = tostring(cl.EventType)
| order by timestamp asc

Conversation timing

traces
| where customDimensions.Subject == "DialogLifecycle"
| extend cl = parse_json(tostring(customDimensions.CallLifecycle))
| project
    ts        = timestamp,
    eventType = tostring(cl.EventType),                    // DialogStarted | DialogCompleted
    status    = tostring(customDimensions.DialogStatus),
    dialogId  = tostring(customDimensions.DialogId),
    botId     = tostring(customDimensions.BotId),
    corrId    = tostring(cl.TrackingContext.CorrelationId),
    duration  = toint(customDimensions.DialogDuration_Ms), // DialogCompleted only
    endReason = tostring(customDimensions.EndReason)       // DialogCompleted only

LLM response latency per invocation

traces
| where customDimensions.Subject == "LlmInvocation"
| extend cl = parse_json(tostring(customDimensions.CallLifecycle))
| project
    ts     = timestamp,
    corrId = tostring(cl.TrackingContext.CorrelationId),
    model  = tostring(customDimensions.Model),
    ttfabMs = toint(customDimensions.Duration_Ms),
    inTok  = toint(customDimensions.InputTokens),
    outTok = toint(customDimensions.OutputTokens)

Tool call time and outcome

traces
| where customDimensions.Subject == "ToolExecution"
| extend cl = parse_json(tostring(customDimensions.CallLifecycle))
| project
    ts      = timestamp,
    corrId  = tostring(cl.TrackingContext.CorrelationId),
    tool    = tostring(customDimensions.ToolName),
    totalMs = toint(customDimensions.TotalDuration_Ms),
    mcsMs   = toint(customDimensions.McsProcessingTime_Ms),
    extMs   = toint(customDimensions.ExternalCallTime_Ms),
    ok      = tostring(customDimensions.ToolResult) == "Success"

How latency is measured (voice-specific)

For real-time voice agents, Duration_Ms on LlmInvocation rows measures caller-perceived response latency, which is Time to First Audio Back (TTFAB):

• Start: The moment the user stops speaking.
• End: The moment the first audio byte reaches the caller regardless of source. This moment includes the real-time model speaking, a tool playing a TTS prompt, or a platform-provided message. All three count because they're what the caller actually hears.

Multi-step turns

A single user turn can produce multiple LlmInvocation rows. For example, when the agent says "Let me check that for you," calls a tool, and then delivers the answer. Each step gets its own row, but every row's timer anchors back to the same user-stop-speaking moment.

This means:

• Several rows in the same turn can share the same Duration_Ms value. They all point back to the first sound the caller heard.
• A step that completed before any sound played can show a smaller, less meaningful value.
• Don't read each row as its own wait time.

To get a single "how long until the caller heard something" value per turn, group by correlationId (and turn boundary) and take the maximum Duration_Ms. Tool duration itself is reported separately on the corresponding ToolExecution row.

What one call looks like

Every following row shares the same correlationId; that's one call end-to-end:

SpeechPipeline    pipeline ready 
DialogLifecycle   DialogStarted 
LlmInvocation     TTFAB 603 ms    in 13,034 / out 204 
LlmInvocation     TTFAB 254 ms    in 13,275 / out 387 
ToolExecution     Rewards-Program-FAQ    Success    4,101 ms 
LlmInvocation     TTFAB 280 ms    in 13,956 / out 996 
DialogLifecycle   DialogCompleted    duration 134,831 ms 
CallLifecycle     CallDisconnected 

Key scenarios

• Analyze conversation lifecycle: How long was the call? Did it end early? Was it a caller hangup or a dialog transfer?
• Measure AI response latency: Identify slow turns, compare TTFAB across models, and analyze end-to-end caller wait time. For multi-step turns, use the maximum Duration_Ms per turn.
• Diagnose tool execution problems: Which tools are being called, how long they take, and whether they succeed or fail. Compare McsProcessingTime_Ms and ExternalCallTime_Ms to see where time is spent.
• Analyze multi-step interactions: Correlate the sequence of LlmInvocation and ToolExecution rows within a single correlationId to understand full turn behavior.

Best practices

• Enable diagnostics for all production voice agents.
• Monitor latency and failures on an ongoing basis, not just during incidents.
• Always filter by correlationId when investigating a specific conversation.
• Filter by customDimensions.Subject to exclude unrelated telemetry.

Limitations

• Some fields defined in the schema aren't populated yet, including per-utterance speech timing (STT/TTS durations), per-invocation sequence numbers, and dedicated error diagnostics events. These fields are planned for future updates.
• Cancelled responses aren't included.
• Voice and text latency semantics differ. For real-time voice, Duration_Ms is caller-perceived TTFAB, not full-response duration.