The Gemini Live API provider. Requires the google optional group
(pip install "pydantic-ai-slim[google]").
GoogleRealtimeModel runs over the google-genai
SDK (which manages the WebSocket transport). Gemini expects 16 kHz PCM input (output is 24 kHz),
produces one response modality per session, and natively accepts live video frames sent as
BinaryImage. It exposes Gemini Live’s session and generation
configuration through GoogleRealtimeModelSettings —
shared turn-taking via TurnDetection, with finer Gemini-specific
control via AutomaticVAD in google_vad plus
google_activity_handling/google_turn_coverage, voice via google_voice or a
MultiSpeaker in google_multi_speaker,
and long-session ContextCompression — with
resilience via session resumption + a ReconnectPolicy in the
reconnect setting.
Gemini Live API provider for realtime speech-to-speech (and live video) sessions.
Built on the google-genai SDK, which manages the WebSocket transport for you. Available via the
google optional group:
pip install “pydantic-ai-slim[google-realtime]”
Unlike the OpenAI provider, Gemini wants 16 kHz PCM input audio (output is 24 kHz), produces a
single response modality per session (audio or text), and natively accepts a stream of video
frames sent as BinaryImage.
Use provider='google' for the Gemini Developer API, or provider='google-cloud' /
GoogleCloudProvider for Google Cloud with
Application Default Credentials.
Bases: TypedDict
Server-side voice activity detection — the default turn-taking mode for Gemini Live.
Turn off automatic VAD entirely. Defaults to False.
Do not set this through RealtimeSession: Pydantic AI does not expose Gemini activity markers or
manual turn controls. Use automatic VAD instead; the shared turn_detection=False setting is
rejected for the same reason.
Type: bool
How readily speech onset is detected. high triggers on quieter audio; low is stricter.
Defaults to the provider default.
Type: Literal[‘high’, ‘low’]
How readily the end of speech is detected. high ends turns sooner; low waits longer.
Defaults to the provider default.
Type: Literal[‘high’, ‘low’]
Audio to include before detected speech, in milliseconds. Defaults to the provider default.
Type: int
Silence required to detect the end of speech, in milliseconds. Defaults to the provider default.
Type: int
Bases: TypedDict
Assign prebuilt voices to named speakers for multi-speaker audio output.
Mapping of speaker label to prebuilt voice name, e.g. {'Joe': 'Puck', 'Jane': 'Kore'}.
Defaults to an empty mapping.
Bases: TypedDict
Sliding-window context compression so long sessions don’t exceed the context window.
Compress once the context passes this many tokens. Defaults to the provider default.
Type: int
Target size (in tokens) of the retained sliding window after compression. Defaults to the provider default.
Type: int
Bases: RealtimeModelSettings
Settings used for a Gemini Live session.
Amount of randomness injected into the response.
Type: float
Nucleus sampling probability mass.
Type: float
Only sample from the top K options for each subsequent token.
Type: int
The random seed to use for the session.
Type: int
The thinking configuration to use for the model.
Type: genai_types.ThinkingConfigDict
The video resolution to use for the model.
Type: genai_types.MediaResolution
BCP-47 language code for audio output.
Type: str
Prebuilt voice used for audio output, e.g. Puck.
Type: str
Per-speaker voice assignments; takes precedence over google_voice.
Type: MultiSpeaker
Whether to enable emotion-aware delivery (native-audio models only).
Type: bool
Whether the model may decide when to respond, including staying silent on input not addressed to it (native-audio models only). Useful for “react to the camera” experiences.
Type: bool
Whether to transcribe input audio. Defaults to True.
When False, user turns are recorded as retained audio when available, or as content-less
placeholders otherwise. Takes precedence over the shared
input_transcription_model,
whose None also turns transcription off here.
Type: bool
Whether to transcribe output audio. Defaults to True.
When False, retain output audio if assistant audio turns need to appear in history. Assistant
audio without a transcript cannot be handed off or seeded.
Type: bool
Language hints applied to input and output transcription.
Gemini-specific server-side voice activity detection settings.
When present, this fully overrides the cross-provider turn_detection setting.
google_vad={'disabled': True} raises a UserError, like turn_detection=False: Pydantic AI does
not expose Gemini activity markers or manual turn controls, so the resulting session could not
drive turns.
Type: AutomaticVAD
Whether detected user activity interrupts the model.
Type: Literal[‘interrupts’, ‘no_interruption’]
Which realtime input is attached to a turn — 'activity_only', 'all_input' (everything
between turns too), or 'all_video' (all video frames plus audio during activity; ideal for
live-camera use). Absent uses the provider default.
Type: Literal[‘activity_only’, ‘all_input’, ‘all_video’]
Sliding-window context compression for long-running sessions.
Type: ContextCompression
Raw values merged last into the Google LiveConnectConfig.
Whether to request session-resumption handles, which let a re-dial restore the server-side conversation.
When absent, handles are requested exactly when a
reconnect policy is set. An explicit
False cannot be combined with a reconnect policy: a re-dial without resumption would lose the
conversation, so connect raises UserError rather than
silently reconnecting into a model that remembers nothing.
Type: bool
Whether tool calls may run without pausing the model’s speech. Defaults to False.
By default Gemini stops generating while a tool call is outstanding, so the caller hears silence
for as long as the tool takes. Enabling this declares tools NON_BLOCKING and returns their
results with INTERRUPT scheduling, so the model keeps talking (typically narrating what it’s
doing) and the result cuts into that speech when it arrives.
This pays off for tools that take a noticeable moment. It is a poor trade for fast tools: the
result interrupts a reply the model has barely started, leaving an extra interrupted turn in
history with nothing in it. Verified live against gemini-2.5-flash-native-audio-latest.
Supported by Gemini native-audio models (see
supports_async_tool_calls).
Other models silently ignore it.
Type: bool
Bases: RealtimeModel
Gemini Live API model.
Session and generation configuration is read from
GoogleRealtimeModelSettings, passed
through settings as model-level defaults or as model_settings when opening a session.
Authentication and the underlying google-genai client come from a
Provider, mirroring GoogleModel.
Pass provider='google' (the default) for the Gemini Developer API (reads GOOGLE_API_KEY /
GEMINI_API_KEY), provider='google-cloud' for Vertex AI (Application Default Credentials, useful
where org policy disallows API keys), or a GoogleProvider /
GoogleCloudProvider instance for a custom
key, client, or region. Gemini Live is available on both surfaces.
The model name, e.g. gemini-2.5-flash-native-audio-latest (an alias that tracks the
newest native-audio Live model) or gemini-3.1-flash-live-preview.
provider : Literal[‘google’, ‘google-cloud’, ‘gateway’] | Provider[Client] Default: 'google'
The provider to use for authentication and API access — 'google' (Gemini Developer
API, the default) or 'google-cloud' (Vertex AI), or a Provider instance.
settings : RealtimeModelSettings | None Default: None
Model-level defaults for session and generation configuration.
profile : RealtimeModelProfileSpec | None Default: None
Optional override for the realtime model profile,
merged over the provider’s — a partial dict, or a callable taking the resolved profile and
returning the one to use. Mirrors profile= on a standard
Model, and is the escape hatch when a model name doesn’t
identify the model (e.g. an Azure deployment named something other than its model).
The underlying google.genai.Client from the provider.
Type: Client
Bases: RealtimeConnection
A live connection to the Gemini Live API, backed by a google-genai session.
@async
def send(content: RealtimeInput) -> None
Send content to the Gemini Live API.
Accepts BinaryAudio (raw PCM16, 16kHz, mono), a str text turn, BinaryImage (a live
video frame), and ToolResult. The manual turn-taking verbs are not supported (Gemini uses
automatic VAD).
Sample rate (Hz) Gemini expects for PCM16 input audio.
Default: 16000