pydantic_ai.realtime.azure
The Azure OpenAI realtime model reuses the OpenAI Realtime codec and connection, authenticates with
AzureProvider, and so needs the realtime and
openai optional groups (pip install "pydantic-ai-slim[realtime,openai]").
Azure realtime support using the OpenAI GA or Azure AI Voice Live protocol.
Bases: RealtimeModelProfile
A RealtimeModelProfile with the Azure-specific facts.
Read via AzureRealtimeModel.profile. Pass a partial
one as profile= to correct what’s inferred from a deployment name that doesn’t match its model.
Which Azure realtime APIs serve this model, when only one does — the constraint that routes it.
A model served only by Voice Live carries {'voice_live'} and routes there automatically; a GA-only
model carries {'azure_openai'} and rejects
azure_voice_live=True.
Absent for a model served by both (e.g. gpt-realtime) or a name the table below doesn’t recognize
(e.g. a future gpt-realtime-3): either way it defaults to GA and reaches Voice Live only when
azure_voice_live=True is set. Pass a profile= override to constrain a deployment named after
something the table can’t place.
Type: frozenset[AzureRealtimeApi]
Whether Voice Live serves this model as a cascade (Azure speech-to-text, then a chat model, then Azure text-to-speech) rather than as a native-audio model.
Stamped for the recognized cascade models (e.g. gpt-5, gpt-4.1, phi4-mm-realtime). A cascade
rejects OpenAI’s semantic VAD, so a SemanticVAD turn detection is sent as Voice Live’s own
azure_semantic_vad, which has no eagerness. Pass it in a profile= override for a cascade
deployment named after something the table can’t place.
Type: bool
Bases: OpenAIRealtimeConnection
A live WebSocket connection to Azure OpenAI’s realtime API.
Reuses OpenAIRealtimeConnection for the
shared GA wire protocol, naming Azure as the vendor so a connection that drops or rejects content
doesn’t send someone debugging an Azure session to OpenAI’s status page.
Bases: Protocol
Structural type for a synchronous Microsoft Entra ID token credential.
Bases: TypedDict
An Azure voice for an Azure AI Voice Live session.
Sent as the session’s voice object as-is, so it takes the fields Voice Live documents for each
voice type. Set it with
azure_voice_live_voice.
The kind of voice: an Azure text-to-speech azure-standard (including HD) voice, an azure-custom
or azure-personal voice, or one of the azure-realtime-native voices of the azure-realtime model.
Type: Required[Literal[‘azure-standard’, ‘azure-custom’, ‘azure-personal’, ‘azure-realtime-native’]]
The voice name, e.g. en-US-AvaMultilingualNeural, en-US-Ava:DragonHDLatestNeural, or ava
for azure-realtime-native.
The deployment endpoint ID of an azure-custom voice.
Type: str
The base model of an azure-personal voice, e.g. DragonLatestNeural.
Type: str
Variability in intonation and prosody, for Azure HD voices.
Type: float
Speaking rate, from '0.5' to '1.5'.
Type: str
Pitch adjustment, e.g. '+5%'.
Type: str
Volume, e.g. 'loud'.
Type: str
Speaking style, e.g. 'cheerful', for voices that support styles.
Type: str
Locale to speak in, e.g. 'en-US', for multilingual voices.
Type: str
Locales to prefer when a multilingual voice detects the language, e.g. ['en-US', 'en-GB'].
URL of a custom lexicon that sets how the voice pronounces specific words.
Type: str
Bases: TypedDict
Azure AI Voice Live’s end-of-utterance detection, which lets the user pause mid-sentence without ending their turn.
It reads the transcribed speech, so it works only on a cascade model (see
azure_voice_live_cascade);
setting it for any other model raises UserError before connecting.
The end-of-utterance detection model.
Type: Required[Literal[‘semantic_detection_v1’, ‘semantic_detection_v1_en’, ‘semantic_detection_v1_multilingual’]]
How sure the model must be that the user is done; 'default' is 'medium'.
Type: Literal[‘low’, ‘medium’, ‘high’, ‘default’]
The longest to wait for more speech, in milliseconds. Defaults to 1000.
Type: int
Bases: ServerVAD
Server VAD with the options only Azure AI Voice Live offers.
Speech required to start a turn, in milliseconds.
Type: int
Whether the server truncates the interrupted response when the user barges in.
Type: bool
End-of-utterance detection, on a cascade model only.
Type: AzureEndOfUtteranceDetection
Bases: TypedDict
Azure AI Voice Live’s own semantic turn detection, which works on every model it serves.
'azure_semantic_vad' is tuned for English; 'azure_semantic_vad_multilingual' for the languages it lists.
Type: Required[Literal[‘azure_semantic_vad’, ‘azure_semantic_vad_multilingual’]]
Activation threshold (0.0-1.0).
Type: float
Audio to include before detected speech, in milliseconds.
Type: int
Silence required to detect the end of speech, in milliseconds.
Type: int
Speech required to start a turn, in milliseconds.
Type: int
Whether filler words like “um” are ignored rather than taken as the user barging in.
Type: bool
The languages to expect, e.g. ['en', 'fr'].
End-of-utterance detection, on a cascade model only.
Type: AzureEndOfUtteranceDetection
Whether to automatically generate a response when the user stops speaking. Defaults to True.
Type: bool
Whether to interrupt an in-progress response when the user starts speaking. Defaults to True.
Type: bool
Whether the server truncates the interrupted response when the user barges in.
Type: bool
Bases: OpenAIRealtimeModelSettings
Settings specific to Azure realtime models.
This inherits every OpenAIRealtimeModelSettings
field. When azure_voice_live
is set, the Voice Live session config is built from the fields Voice Live’s beta session object
has a counterpart for: instructions, openai_voice (by name, unless azure_voice_live_voice is
set), turn_detection (or openai_turn_detection, or azure_voice_live_turn_detection),
openai_input_noise_reduction (or azure_voice_live_noise_reduction), thinking (as
reasoning_effort, on models whose profile reports
supports_thinking),
input_transcription_model, output_modality, max_tokens, parallel_tool_calls, tool_choice,
and tools, plus the azure_voice_live_* settings.
The remaining inherited fields — openai_output_speed and openai_truncation — are silently
ignored under Voice Live; they still apply on the GA path.
Voice Live’s own truncation_strategy takes different values from openai_truncation ('auto' or
'last_messages', not a retention ratio), so the two don’t map onto each other.
Use the Azure AI Voice Live endpoint and beta session protocol instead of the GA endpoint.
Voice Live is a distinct Azure resource; AzureProvider
reads its AZURE_VOICELIVE_ENDPOINT / AZURE_VOICELIVE_API_KEY / AZURE_VOICELIVE_API_VERSION
credentials as a fallback to the AZURE_OPENAI_* variables.
Type: bool
Voice Live turn detection; only applies when the session uses Voice Live.
When present, it overrides openai_turn_detection and turn_detection on Voice Live. Besides the
OpenAI ServerVAD and
SemanticVAD, it takes Voice Live’s own
AzureServerVAD and
AzureSemanticVAD, including
end-of-utterance detection. On a cascade
model (see azure_voice_live_cascade),
which rejects OpenAI’s semantic VAD, SemanticVAD is sent as azure_semantic_vad, which has no
eagerness setting.
Type: ServerVAD | SemanticVAD | AzureServerVAD | AzureSemanticVAD
Input noise reduction on Voice Live; only applies when the session uses Voice Live.
'azure_deep_noise_suppression' is Azure’s own and works on every model; this setting is sent as
given. It overrides openai_input_noise_reduction, which is sent as azure_deep_noise_suppression
on a cascade model because its near_field/far_field are for the native-audio gpt-realtime models.
Type: Literal[‘azure_deep_noise_suppression’, ‘near_field’, ‘far_field’]
Whether Voice Live removes the model’s own voice from the input audio; only applies when the session uses Voice Live.
Useful when the reply plays through a speaker the microphone can hear, so the model doesn’t interrupt itself.
Type: bool
The Azure voice for a Voice Live session; only applies when the session uses Voice Live.
A string is an Azure text-to-speech standard voice name, such as 'en-US-AvaMultilingualNeural' or
the HD 'en-US-Ava:DragonHDLatestNeural'. Pass an
AzureVoiceLiveVoice for a custom, personal, or
azure-realtime-native voice, or to tune the rate, pitch, style, and so on.
Cascade models (e.g. gpt-5, gpt-4.1) and phi4-mm-realtime speak only through Azure voices and
reject openai_voice, so this is how to choose their voice. It takes precedence over
openai_voice, which native-audio models like gpt-realtime also accept under Voice Live.
Type: str | AzureVoiceLiveVoice
Sampling temperature for a Voice Live session; only applies when the session uses Voice Live.
Microsoft documents a range of 0.6 to 1.2 (default 0.8), but Voice Live accepts 0 to 2. The GA
realtime API has no temperature setting. As with a standard OpenAI run, it’s dropped with a
warning while a reasoning model like gpt-5 is reasoning, since those models then accept only the
default.
Type: float
Bases: OpenAIRealtimeModel
Azure realtime model using the OpenAI GA protocol or Azure AI Voice Live.
The existing AzureProvider supplies the Azure
resource endpoint and API key. The WebSocket transport does not use its OpenAI SDK client or
api_version. By default it connects to the GA /openai/v1/realtime endpoint; set
azure_voice_live
to connect to /voice-live/realtime with the Voice Live beta session protocol. Both use an
api-key header.
Pass a Microsoft Entra ID credential (e.g. azure.identity.DefaultAzureCredential()) to
authenticate every request to the resource — the realtime WebSocket session and the browser
WebRTC signaling calls — with a bearer token instead of the api-key (needed when the resource is
locked to managed identity). For browser WebRTC the browser still only ever receives the short-lived
ephemeral secret, never the Entra token or the API key.
A model served only by Voice Live (e.g. the cascade chat models like gpt-5, or phi4-mm-realtime)
routes there automatically; a model served by both defaults to GA and needs azure_voice_live=True
for Voice Live; a GA-only model rejects the setting. See
AzureRealtimeModelProfile.azure_realtime_apis.
The Azure realtime profile, with the model’s serving APIs and minus what Voice Live can’t do.
Stamps azure_realtime_apis
for a recognized model (a profile= override wins), which routes between the GA API and Voice
Live. And because Voice Live negotiates WebRTC over its own WebSocket control channel rather than
the GA signaling endpoints this model inherits, a model configured for Voice Live reports no
WebRTC support and the signaling methods refuse. Voice Live selected per session instead can’t be
seen from here, so those calls still refuse at the point of use.
Type: RealtimeModelProfile
def __init__(
model: AzureRealtimeModelName,
*,
provider: Provider[AsyncOpenAI] | str = 'azure',
settings: RealtimeModelSettings | None = None,
profile: RealtimeModelProfileSpec | None = None,
credential: AzureTokenCredential | None = None,
) -> None
Create an Azure OpenAI realtime model.
The Azure deployment name, which is what the realtime URL and the profile lookup
use. Azure deployments are conventionally named after their model; when yours isn’t,
profile is how to correct the facts inferred from the name.
provider : Provider[AsyncOpenAI] | str Default: 'azure'
The provider supplying the resource endpoint and API key. Defaults to 'azure'.
settings : RealtimeModelSettings | None Default: None
Model settings used as defaults for realtime sessions.
profile : RealtimeModelProfileSpec | None Default: None
Optional override for the realtime model profile, merged over the provider’s — a partial dict, or a callable taking the resolved profile and returning the one to use.
credential : AzureTokenCredential | None Default: None
Optional Microsoft Entra ID credential. When set, realtime requests use its bearer tokens instead of the resource API key.
An Azure realtime speech-to-speech API a model can be reached through: the Azure OpenAI GA realtime
API (/openai/v1/realtime) or Azure AI Voice Live
(/voice-live/realtime, selected with azure_voice_live=True).
Default: Literal['azure_openai', 'voice_live']