Skip to content
Docs Pydantic Docs

pydantic_ai.realtime.xai

The xAI Grok Voice realtime API provider. Requires the realtime, xai, and openai optional groups (pip install "pydantic-ai-slim[realtime,xai,openai]") — openai because the model reuses the OpenAI Realtime codec, whose event types come from the OpenAI SDK.

xAI’s realtime API is a clone of the OpenAI Realtime protocol, so XaiRealtimeModel reuses the OpenAI codec (event mapping, seeding, the WebSocket connection). Turn-taking uses the shared TurnDetection (or False for push-to-talk); for exact server-VAD control, xai_turn_detection accepts ServerVAD and fully overrides the shared setting. It diverges only where xAI does: it supports cancellation-based interruption but not output truncation, has no image input, and streams input transcription as cumulative snapshots that may revise earlier text, rather than as incremental deltas. Authentication comes from an XaiProvider, mirroring XaiModel.

xAI Grok Voice realtime API provider for speech-to-speech sessions.

Connects to wss://api.x.ai/v1/realtime over a WebSocket. xAI’s realtime API is a deliberate clone of the OpenAI Realtime protocol, so this provider reuses the OpenAI codec from pydantic_ai.realtime.openai — event mapping, session seeding, tool conversion, server-VAD config, and the WebSocket connection itself — and diverges only where xAI does:

  • the session.update shape (voice/turn_detection sit at the session top level, not nested under audio as on OpenAI’s GA surface);
  • input audio transcription, delivered as cumulative conversation.item.input_audio_transcription.updated snapshots plus a final .completed, rather than OpenAI’s incremental .delta events (see map_event);
  • the provider’s conversation ID, reported in conversation.created (see XaiRealtimeConnection.conversation_id). A reconnect replays the local message history into the new session, as on OpenAI: xAI’s own conversation resumption leaves user text and function calls out of the conversation it restores;
  • no output truncation at barge-in: xAI’s conversation.item.truncate only lands once the response has ended, and unreliably even then, so RealtimeModelProfile.supports_output_truncation is False while cancellation-based interruption still works;
  • no text output — the API accepts a response-modality setting but ignores it and always speaks — so RealtimeModelProfile.supports_text_output is False and output_modality='text' raises rather than silently coming back as audio.

Requires the websockets package (the realtime optional group), xai-sdk (the xai group, for XaiProvider), and openai (the openai group, whose SDK supplies the event types the shared OpenAI codec is built on):

pip install “pydantic-ai-slim[xai-realtime]“

XaiRealtimeModelSettings

Bases: RealtimeModelSettings

Settings specific to xAI realtime models.

Grok Voice always produces audio, so its profile reports supports_text_output=False and the inherited output_modality='text' is rejected up front rather than quietly ignored.

Attributes

xai_voice

Voice used for audio output, e.g. eve, or a custom voice ID.

Type: str

xai_turn_detection

xAI-specific server-VAD configuration, for an exact threshold, prefix padding, or silence duration.

When present, this fully overrides the cross-provider turn_detection setting. xAI echoes create_response: False back but still responds when the user stops speaking; set turn_detection=False for push-to-talk instead.

Type: ServerVAD

XaiRealtimeConnection

Bases: OpenAIRealtimeConnection

A live WebSocket connection to the xAI Grok Voice realtime API.

Reuses OpenAIRealtimeConnection for the shared wire protocol, including reconnecting by replaying the local message history, while mapping xAI’s cumulative input transcription and conversation lifecycle events.

Attributes

conversation_id

The ID xAI gave the conversation on the current socket, once its conversation.created arrives.

Type: str | None

XaiRealtimeModel

Bases: RealtimeModel

xAI Grok Voice realtime API model.

Pass provider='xai' (the default, which reads XAI_API_KEY) or an XaiProvider constructed with api_key=. A custom api_host is not supported, and a provider constructed only with xai_client= cannot be used because the WebSocket connection needs access to the API key. The realtime WebSocket URL is wss://api.x.ai/v1/realtime.

Constructor Parameters

model : XaiRealtimeModelName

The model name, e.g. grok-voice-latest (which tracks the current model) or a pinned version like grok-voice-think-fast-1.0. The model query parameter is required by the server, which otherwise falls back to a default silently.

provider : Provider[AsyncClient] | str Default: 'xai'

The provider to use for authentication and the base URL. Defaults to 'xai'.

settings : RealtimeModelSettings | None Default: None

Model settings used as defaults for realtime sessions. With a reconnect policy, a dropped connection is re-dialed and the local message history replayed into it.

profile : RealtimeModelProfileSpec | None Default: None

Optional override for the realtime model profile, merged over the provider’s — a partial dict, or a callable taking the resolved profile and returning the one to use. Mirrors profile= on a standard Model, and is the escape hatch when a model name doesn’t identify the model (e.g. an Azure deployment named something other than its model).

map_conversation_event 

def map_conversation_event(
    data: dict[str, Any],
    *,
    replayed: bool | None = None,
) -> ConversationCreated | ConversationItemCreated | None

Map xAI’s conversation handshake and item lifecycle events to codec control events.

Returns

ConversationCreated | ConversationItemCreated | None

map_event 

def map_event(data: dict[str, Any]) -> RealtimeCodecEvent | None

Map a raw xAI Grok Voice realtime event to a RealtimeCodecEvent.

xAI clones the OpenAI Realtime protocol, so most events map identically via the OpenAI codec. The first exception is input audio transcription: xAI emits cumulative conversation.item.input_audio_transcription.updated snapshots (which may retroactively correct earlier text — 'Hello?' becomes 'Hello, my name is') plus cumulative .completed snapshots, rather than OpenAI’s incremental .delta. The partials are surfaced as cumulative InputTranscripts so a live transcript can render the user’s words as they are spoken; the session adopts each snapshot wholesale, appending when it merely extends and replacing when xAI revises itself. The shared codec still drops interim .completed snapshots. The other exception is xAI’s conversation lifecycle events, which are surfaced as codec control events, so the connection can report the provider’s conversation.id. Finally, xAI’s max_duration error ends the conversation, so it is reported as non-recoverable.

Returns

RealtimeCodecEvent | None