Skip to content
Docs Pydantic Docs

pydantic_ai.realtime.openai_live

The OpenAI GPT-Live API provider. Requires the openai-realtime optional group (pip install "pydantic-ai-slim[openai-realtime]"), which floors openai at 3.12, the release that added Live’s event types.

GPT-Live is a different protocol from the OpenAI Realtime API, not a model served by it, so OpenAILiveModel shares no event mapping with OpenAIRealtimeModel; the model name is what picks between them. The Live model runs the spoken conversation and delegates the work to a backend model configured through OpenAILiveResponsesDelegation in OpenAILiveModelSettings, which receives the agent’s instructions and tools and calls them as ordinary ToolCalls.

Live owns turn-taking entirely, so there is no manual turn control, interruption, or truncation, and no turn-detection setting. It sends no end-of-response frame either: the connection synthesizes ResponseDone after openai_live_turn_silence_ms of silence, and the profile reports synthesizes_turn_boundary=True. Text is delivered as context rather than as a user turn, seeding is text-only, and usage is reported as billable audio seconds instead of tokens. Authentication comes from an OpenAIProvider; Azure OpenAI does not serve Live. See the GPT-Live documentation for the full story.

OpenAI GPT-Live realtime support.

GPT-Live is a separate protocol from the OpenAI Realtime API, not a model served by it, so none of its event mapping is shared: what it borrows from _openai_protocol.py is the parts that are about the provider rather than the protocol (deriving the WebSocket URL, resolving authentication, and mapping handshake failures). The differences that shape the adapter:

  • Audio drives the session. Live has no user-turn event for text: text arrives as context through session.commentary.append (speakable) and session.thinking.append (silent), each capped at 500 tokens. Both are placed on the session’s audio timeline, which only advances while audio flows — so a session whose microphone isn’t streaming silently defers everything sent to it.
  • There is no turn terminal. Live has no response.done equivalent and no transcript-done event; transcripts arrive as timeline fragments. The connection synthesizes ResponseDone once the model has been quiet for openai_live_turn_silence_ms and no delegated work is outstanding, and the profile reports synthesizes_turn_boundary=True so consumers know the boundary is inferred.
  • Work is delegated, not tool-called. The Live model hands a task either to your application (client delegation) or to a Responses backend it drives itself (responses delegation). Only the latter produces typed function calls, so it is what this adapter configures: the agent’s tools are advertised to the backend, its calls arrive nested inside response.event, and their results go back as Responses input items. See the Live docs for the split.
  • Usage has two meters. Live reports its own audio as a cumulative duration in seconds and no tokens at all; the backend it delegates to reports ordinary Responses token usage, which is where most of a call’s token cost is.

OpenAILiveResponsesDelegation

Bases: TypedDict

Settings for the Responses backend a Live session delegates work to.

The Live model runs the conversation; the backend model does the reasoning and calls the agent’s tools. OpenAI’s prompting guide asks for the two prompts to stay separate, and Pydantic AI keeps them separate by construction: the agent’s instructions become the backend prompt, because they describe the work, while live_instructions describes how to speak.

Attributes

model

The Responses model that handles delegated work.

When unset, the backend is, first match wins: the model named after a + in the Live model name ('gpt-live-1+gpt-5.6-sol'); the agent’s own model, when it is an OpenAI model reached the same way as the Live model (directly, or through the same gateway route); then 'auto', which resolves to AUTO_BACKEND_MODEL. There is always a backend, so nothing raises for want of one.

Type: str

instructions

Extra backend instructions, appended after the agent’s own instructions.

Type: str

max_output_tokens

Maximum output tokens per delegated response.

Type: int

parallel_tool_calls

Whether the backend may request several tool calls in one response.

Type: bool

reasoning_effort

Reasoning effort for the backend model.

Type: Literal[‘none’, ‘minimal’, ‘low’, ‘medium’, ‘high’, ‘xhigh’]

verbosity

How much detail the backend generates. Does not affect the Live model’s spoken delivery.

Type: Literal[‘low’, ‘medium’, ‘high’]

service_tier

Service tier for delegated Responses requests.

Type: Literal[‘auto’, ‘default’, ‘flex’, ‘priority’]

OpenAILiveModelSettings

Bases: RealtimeModelSettings

Settings for OpenAI GPT-Live sessions.

See RealtimeModelSettings for the shared settings. Live has no turn-detection, truncation, token-limit, or temperature controls: it owns turn-taking entirely, and voice, audio format and instructions are immutable once the session has started.

Attributes

openai_voice

The voice used for Live speech, e.g. marin (the provider default). Immutable after startup.

Type: str

openai_live_instructions

Instructions for the Live model’s spoken behavior: voice, pacing, interruptions, and when to delegate.

The agent’s own instructions describe the work and become the backend prompt, so this is where conversational style belongs. Defaults to a short prompt that tells the model to delegate anything it can’t answer from the conversation itself.

Type: str

openai_live_delegation

Configuration for the Responses backend the Live session delegates work to.

Type: OpenAILiveResponsesDelegation

openai_live_turn_silence_ms

How long the model must stay quiet, in milliseconds, before the session reports the turn complete. Defaults to 2000.

Live has no end-of-turn frame, so this threshold is the turn boundary. Lower it for snappier turn-taking at the risk of ending a turn during a dramatic pause; raise it when replies contain long silences.

Type: int

openai_live_store

Whether OpenAI stores the session so it can later be forked or downloaded. Defaults to False.

Type: bool

OpenAILiveConnection

Bases: RealtimeConnection

A live WebSocket connection to the OpenAI GPT-Live API.

Translates Live’s session-timeline events into the shared codec vocabulary, including the two boundaries Live itself never sends: the end of the user’s turn and the end of the model’s reply.

Live streams output audio as a continuous telephony-style track — ten 100 ms frames a second for as long as the session is open, digitally silent when the model isn’t speaking. So a frame arriving means nothing about whether anyone is talking, and voice (a transcript fragment or a non-silent audio frame) is what drives the turn clock.

Attributes

input_transcription_enabled

Live always transcribes both directions; there is no way to turn it off.

Type: bool

reconnect_restores_in_flight_state

A redialed Live session starts empty: its history is re-seeded, not resumed.

Type: bool

Methods

aclose

@async

def aclose() -> None

Cancel the read in flight so closing the socket doesn’t strand its exception.

Returns

None

OpenAILiveModel

Bases: RealtimeModel

OpenAI GPT-Live model.

GPT-Live runs the spoken conversation and delegates the thinking to a Responses backend that Pydantic AI configures with the agent’s instructions and tools, so tools, dependencies, validation, and message history stay on the Pydantic AI side. Reach it through Agent.realtime as 'openai:gpt-live-1', or construct it directly for model-level configuration.

Live differs from the Realtime API in ways that change what a session can do — no text input, an inferred turn boundary, no manual turn control or interruption, and duration-based usage. See the Live docs.

Constructor Parameters

model : OpenAILiveModelName

The model name, e.g. gpt-live-1.

provider : Provider[AsyncOpenAI] | str Default: 'openai'

The provider to use for authentication and the base URL. Defaults to 'openai'.

settings : RealtimeModelSettings | None Default: None

Model settings used as defaults for realtime sessions.

profile : RealtimeModelProfileSpec | None Default: None

Optional override for the realtime model profile.

Attributes

client

The underlying AsyncOpenAI client from the provider.

Type: AsyncOpenAI

tool_def_to_live 

def tool_def_to_live(tool: ToolDefinition) -> dict[str, Any]

Convert a ToolDefinition to a Live backend function tool.

Returns

dict[str, Any]

seed_input_items 

def seed_input_items(
    messages: Sequence[ModelMessage],
    *,
    provider_name: str,
) -> list[dict[str, Any]]

Map prior history to Live’s startup input list.

Live seeds from text only: user and assistant messages with one text part each. Tool rounds are rendered as readable text — as Gemini Live does for the same reason — because the protocol has no place to put function parts in seeded history. Audio, images, and other media cannot be seeded at all, and the profile says so, which is what makes the session reject them before we get here.

Returns

list[dict[str, Any]]

LatestOpenAILiveModelNames

Latest OpenAI GPT-Live model names.

Default: Literal['gpt-live-1']

OpenAILiveModelName

Possible OpenAI GPT-Live model names.

Since OpenAI supports a variety of date-stamped models, we explicitly list the latest models but allow any name in the type hints.

Default: str | LatestOpenAILiveModelNames

DEFAULT_TURN_SILENCE_MS

How long the model must stay quiet before a turn is considered complete.

Matches the assistant_silence_ms default of OpenAI’s own TranscriptGrouper, which solves the same problem for display transcripts.

Default: 2000

AUTO_BACKEND_MODEL

What a backend model of 'auto' resolves to: the Responses model Pydantic AI currently recommends.

Used when nothing names one: not openai_live_delegation, not the model name, and not the agent. It moves with new OpenAI models, so pin a backend explicitly when its behavior needs to stay put.

Default: 'gpt-6-sol'