Connection lifecycle

A realtime model uses one persistent provider connection. Your backend owns that session and the media bridge to the user (see Deployment); a reconnect policy can recover dropped connections and provider session limits without changing the application event loop.

The session lifecycle

stateDiagram-v2
    [*] --> Connecting: session() opens
    Connecting --> Listening: handshake complete
    Listening --> UserTurn: speech detected /<br>audio committed
    UserTurn --> ModelResponse: turn detection /<br>create_response()
    ModelResponse --> ToolCalls: model calls a tool
    ToolCalls --> ModelResponse: result returned
    ModelResponse --> Listening: turn complete
    Listening --> Reconnecting: connection drops
    ModelResponse --> Reconnecting: connection drops
    Reconnecting --> Listening: redial succeeds
    Reconnecting --> [*]: attempts exhausted
    Listening --> [*]: close()

Opening the session performs the provider handshake, after which the session listens for input. Turn detection (or manual push-to-talk control) moves a user turn into a model response, which may loop through tool calls before RealtimeTurnCompleteEvent marks the turn boundary and the session listens again. A dropped connection enters the reconnect loop below — emitting RealtimeSessionReconnectEvent on recovery — until close() (or leaving the async with block) ends the session, including from a tool that hangs up.

Connection and handshake

The connection is opened when the session() context is entered, and the shared handshake_timeout setting (default 30 seconds) bounds how long the session waits for the provider handshake: each handshake event on OpenAI, Azure OpenAI, and xAI, the whole session setup on Gemini, and the conversation initiation metadata on ElevenLabs Agents. A handshake that times out raises RealtimeError; a rejected WebSocket upgrade raises ModelHTTPError (see Errors).

Reconnecting

Set the reconnect shared setting to a ReconnectPolicy to redial with exponential backoff, reapply configuration, and emit RealtimeSessionReconnectEvent. Like any realtime model setting, it can be a default on the model or passed for one session:

from pydantic_ai import Agent

agent = Agent()
realtime = agent.realtime(
    'openai:gpt-realtime-2.1',
    model_settings={'reconnect': {'max_attempts': 5}},
)

max_attempts bounds retries for one drop. max_reconnects bounds recoveries across the entire session, preventing an endpoint that repeatedly accepts and closes connections from redialing forever.

While the policy is replacing a dropped connection, an audio chunk sent with send_audio() is dropped instead of raising, so a microphone capture task survives the reconnect (live audio is no use once late). Other sends made during the reconnect still raise RealtimeError. A reconnect in the middle of an utterance commits only the audio sent after it, so with push-to-talk, prompt the user to repeat themselves on RealtimeSessionReconnectEvent.

Without a policy, an unexpected provider close raises RealtimeError from the session iterator.

On a WebRTC sideband the same policy applies to an unexpected drop, but a clean close is treated as the browser hanging up: the sideband is a control channel, so a normal close ends iteration without a session error or reconnect attempt even when a reconnect policy is set. The close frame alone can’t distinguish a hangup from a WebSocket-terminating proxy closing the sideband cleanly mid-call (a restart or graceful rotation), which would end the agent side while the browser keeps talking to the provider — drain such connections at the infrastructure layer rather than relying on the reconnect policy to cover them.

State restoration

On OpenAI, Azure OpenAI, and xAI, Pydantic AI replays local message history into the new session. Prior transcript turns survive; in-flight audio does not. GPT-Live does the same by default, or forks a session stored with openai_live_store=True, so the new session has the whole conversation on OpenAI’s side. A GPT-Live WebRTC sideband re-attaches to the call’s own session instead, which carried on while it was away: the conversation, the reply under way, and the tool calls waiting for their results all carry over, so it reports state_restored=True. Live sends a re-attached sideband the last few seconds of events again, which Pydantic AI skips where it already handled them; what happened before that, in a longer gap, is lost. So on a sideband re-attach, state_restored=True means the call’s own session carried on, not that nothing was missed: a user turn or tool call from a longer gap is not in the history. A tool call the backend asked for in such a gap is never answered, and as Live can’t cancel the delegation waiting on it, Live then delegates nothing more for the rest of the call. Delegated work the drop cut off is sent again, and if the backend still never picks it up, it is given up with a recoverable live_continuation_lost error (see GPT-Live’s quirks).

Replay sends text, not audio, so a spoken user turn with no transcript (input transcription is off, or its transcript never arrived) replays as the text [The user spoke; no transcript is available.]. The new session then sees that the user said something before each answer, rather than an answer out of nowhere. The marker is only sent to the provider: the session’s message history keeps the turn as the SpeechPart it was.

Gemini uses native in-process session resumption, enabled automatically when a reconnect policy is present (an explicit google_enable_session_resumption=False alongside a policy raises UserError instead of silently losing the conversation); see the Gemini resumption settings. Its handles live only in memory and cannot be persisted for another process.

RealtimeSessionReconnectEvent.state_restored reports whether the reconnect carried the conversation through without cutting a turn off.

How a reply the drop caught in flight is handled depends on the mechanism. Gemini reports True once the server has issued a resumption handle (shortly after connect) but closes the cut reply as an interrupted response (keeping any partial transcript in history) before the RealtimeSessionReconnectEvent and stays quiet until the next input. A drop before the first handle dials a fresh session instead, and Pydantic AI replays the session’s message history into it, so a reply in flight is handled as with local replay (below). Gemini issues no handle while a tool call is running, so a resumed session never has a call still running at the drop, and never answers its result. Such a call is cancelled with an interrupted return, like a call Gemini cancels itself, the resumed session is told the call was interrupted (on the Gemini Developer API, only if the call came with an id, which an answer there requires), and state_restored is False. Gemini 2.5 also withholds handles while it works on a turn, so a typed turn not yet followed by a handle after its reply is missing from the resumed session too: the turn stays in history, state_restored is False, and you can send it again. When its reply hadn’t started and no spoken reply was in progress, wait_for_reply() also stops waiting for it. A turn typed while the model was answering speech can’t be told apart from that spoken reply, so wait_for_reply() may keep waiting for it, and the reconnect may report state_restored=True.

Local replay (OpenAI, Azure OpenAI, xAI, and Gemini without a resumption handle) restores only the finalized turns, so a reply in flight when the socket dropped cannot continue. The session settles it before emitting the event — the partial reply becomes an interrupted response, running tool calls get cancelled returns, and the turn ends so queued messages waiting for the boundary still flush — and state_restored is False to say the turn was cut off. An answer that was solicited but had not started streaming is instead re-requested on the new connection, and a drop with nothing in flight restores the whole call as-is; both lose no output, so state_restored stays True.

Provider session limits

Providers cap individual connection duration. A reconnect policy is also how an application survives those limits. Exact limits and provider behavior can change, so provider pages are canonical:

• OpenAI session behavior
• Azure OpenAI session behavior
• Gemini session resumption
• xAI session length
• ElevenLabs Agents session behavior

Gemini sends GoAway shortly before its cap but Pydantic AI currently reconnects only after the connection drops, so a long call can briefly drop mid-turn.

ElevenLabs Agents is the exception to surviving the cap with a policy: a conversation ends at the agent’s configured maximum duration and cannot be resumed, so a reconnect policy is ignored there, with an UnsupportedSettingWarning, rather than silently opening a conversation that remembers nothing.

Ending a call

Leaving the async with block closes the session. To hang up from elsewhere — a watchdog, a stop button, or a tool — await close() from any task. The teardown runs to completion even if that task is cancelled while it waits, and both a concurrent close() and the async with exit wait for the same teardown, so the session is fully closed by the time the block is left. While the session is being iterated the loop ends, leaving the async with block does not raise, and session.result is settled. On a WebRTC sideband, close() only detaches from the browser’s call; hang_up() ends it.

For external policy such as an idle timeout or maximum call duration, run a watchdog task that calls close():

import asyncio

from pydantic_ai import Agent
from pydantic_ai.realtime import RealtimeSession

agent = Agent(instructions='You are a helpful voice assistant.')


async def close_after(session: RealtimeSession, seconds: float) -> None:
    await asyncio.sleep(seconds)
    await session.close()


async def main():
    async with agent.realtime('openai:gpt-realtime-2.1').session() as session:
        watchdog = asyncio.create_task(close_after(session, 300))
        try:
            async for event in session:
                ...  # handle events as usual; the loop ends when the watchdog closes the session
        finally:
            watchdog.cancel()

For an idle rather than absolute timeout, reset the watchdog on RealtimeInputSpeechStartEvent and RealtimeInputSpeechEndEvent, which providers whose profile declares emits_input_speech_events send. Where that flag is unset, as on Gemini, GPT-Live, and ElevenLabs Agents, reset the watchdog from your own input path as well — whenever you send audio that is not silence — plus assistant PartDeltaEvent activity and RealtimeTurnCompleteEvent; resetting on output alone would hang up on a person mid-sentence. With input transcription enabled, user parts from stream_transcripts() work too.

Errors

Realtime sessions use the standard Pydantic AI exception hierarchy:

Exception	Raised when
UserError	The application requests an unsupported operation, passes incompatible settings, lacks credentials, or misuses the session.
ModelHTTPError	The provider rejects the WebSocket upgrade with an HTTP status. A provider that accepts the upgrade and then closes the socket during the handshake (Gemini’s 1007 for a rejected config, for example) raises RealtimeError instead, since a WebSocket close code isn’t an HTTP status.
RealtimeError	The connection fails, times out, closes unexpectedly, returns an invalid frame, or exhausts reconnect attempts.
UsageLimitExceeded	A configured usage limit is exceeded.

RealtimeError subclasses ModelAPIError, so except ModelAPIError covers HTTP and non-HTTP provider failures together.

Recoverable failures arrive as events: RealtimeSessionErrorEvent for provider operations and RealtimeInputTranscriptionErrorEvent for one failed user transcription. The session remains usable after either event.

A RealtimeSessionErrorEvent can be the provider refusing something you sent: OpenAI Realtime refuses a text longer than 256,000 characters, for example. When the error identifies the refused input, as OpenAI-protocol providers do by echoing the client event’s id for a malformed or oversized one, the session takes back what the send assumed, the way it does when the send itself raises: refused content is removed from history, and a refused request for a response stops wait_for_reply() waiting for it. An error that doesn’t identify an input changes neither, because the reply can still come: on OpenAI and xAI, a request for a response sent after a refused item is still answered.

One refusal only means “not yet”: OpenAI Realtime and Azure OpenAI refuse a request for a response while another is under way, as when server-side voice activity detection starts answering the user’s speech just as you send text. That response never saw the text, so the session asks for the text’s reply again once it ends, and reports nothing. It asks up to three times in a row; if the provider refuses the request a fourth time, the refusal is reported like any other, and wait_for_reply() stops waiting for that reply.

Failures surface from the responsible call where possible; a failed send_audio() raises there. Receive-loop and tool failures are raised from async for while the event stream is being iterated. A receive-loop failure ends the session; a tool failure doesn’t (the model is told the call failed and answers it), so a consumer that catches it and iterates again keeps a working session (see Tools). Without an active iterator, the session stops receiving, the audio and transcript views end, and the failure is raised when the async with block exits (from close()). If the receive side has already failed, the next outbound session method raises that failure instead; it is delivered only once.

For symptom-first debugging, see Troubleshooting.