OpenAI
To use OpenAI models or OpenAI-compatible APIs, you need to either install pydantic-ai, or install pydantic-ai-slim with the openai optional group:
pip install "pydantic-ai-slim[openai]"
uv add "pydantic-ai-slim[openai]"
To use OpenAI models with the OpenAI API, go to platform.openai.com and follow your nose until you find the place to generate an API key.
Once you have the API key, you can set it as an environment variable:
export OPENAI_API_KEY='your-api-key'
The bare 'openai:' prefix resolves to OpenAIResponsesModel, which uses the modern Responses API.
from pydantic_ai import Agent
agent = Agent('openai:gpt-6.1-sol')
...
To pin to the legacy Chat Completions API instead, use the 'openai-chat:' prefix, which resolves to OpenAIChatModel.
For gpt-6-sol and gpt-6-luna, Chat Completions supports function calling only when openai_reasoning_effort='none'. OpenAI documents tool calling for gpt-6.1-terra through Responses only, so use Responses for its tool calls. Use Responses whenever you need reasoning and tools together. gpt-6.1-sol and gpt-6-astra don’t accept 'none', so use the Responses API for function calling with them.
Or initialise the model directly with just the model name:
from pydantic_ai import Agent
from pydantic_ai.models.openai import OpenAIResponsesModel
model = OpenAIResponsesModel('gpt-6.1-sol')
agent = Agent(model)
...
By default, the model uses the OpenAIProvider with the base_url set to https://api.openai.com/v1.
If you want to pass parameters in code to the provider, you can programmatically instantiate the OpenAIProvider and pass it to the model:
from pydantic_ai import Agent
from pydantic_ai.models.openai import OpenAIResponsesModel
from pydantic_ai.providers.openai import OpenAIProvider
model = OpenAIResponsesModel('gpt-6.1-sol', provider=OpenAIProvider(api_key='your-api-key'))
agent = Agent(model)
...
OpenAIProvider also accepts a custom AsyncOpenAI client via the openai_client parameter, so you can customise the organization, project, base_url etc. as defined in the OpenAI API docs.
The client retries failed requests on its own, independently of the agent’s retry budgets. It defaults to max_retries=2, so one model request can reach the network up to three times. It honors the x-should-retry response header; without that header, it retries status 408, 409, 429 or 5xx, plus timeouts and connection errors, but not other 4xx responses such as 400 or 401. Set max_retries=0 to keep the retry policy in your transport alone, or set ModelSettings['max_retries'] to change the count for a model, agent or run without building the client yourself. See Retry multiplication for how the layers stack.
from openai import AsyncOpenAI
from pydantic_ai import Agent
from pydantic_ai.models.openai import OpenAIResponsesModel
from pydantic_ai.providers.openai import OpenAIProvider
client = AsyncOpenAI(max_retries=3)
model = OpenAIResponsesModel('gpt-6.1-sol', provider=OpenAIProvider(openai_client=client))
agent = Agent(model)
...
You could also use the AsyncAzureOpenAI client
to use the Azure OpenAI API. Note that the AsyncAzureOpenAI is a subclass of AsyncOpenAI.
from openai import AsyncAzureOpenAI
from pydantic_ai import Agent
from pydantic_ai.models.openai import OpenAIChatModel
from pydantic_ai.providers.openai import OpenAIProvider
client = AsyncAzureOpenAI(
azure_endpoint='...',
api_version='2024-07-01-preview',
api_key='your-api-key',
)
model = OpenAIChatModel(
'gpt-6.1-sol',
provider=OpenAIProvider(openai_client=client),
)
agent = Agent(model)
...
For Grok deployments on Azure, see Grok tool schemas for the AzureProvider route and the explicit custom-profile workaround with OpenAIProvider.
Use ImageGenerator with an openai: image model for direct generation and
reference-image editing. This uses OpenAI’s Images API rather than a conversational Responses model:
from pydantic_ai import ImageGenerator
from pydantic_ai.images.openai import OpenAIImageGenerationSettings
generator = ImageGenerator(
'openai:gpt-image-2',
settings=OpenAIImageGenerationSettings(
dimensions=(1280, 720),
openai_quality='low',
openai_output_format='jpeg',
),
)
OpenAI accepts BinaryImage and
ImageUrl reference inputs. Its image-edit endpoint requires file content and does not
accept an UploadedFile provider file ID. Transparent-background support varies
by model and requires PNG or WebP output. Provider-specific settings are forwarded to OpenAI so newly supported values
are not blocked by stale client-side checks. See the image-generation guide for generation,
editing, geometry, and normalized settings.
GPT Image models require a verified organization on a paid usage tier: on an unverified organization every request fails with a rate-limit error before anything generates. Complex prompts can take up to two minutes to process.
For ordinary, non-fine-tuned models used with OpenAIProvider or
OpenAIChatGPTProvider, which serves the same public
OpenAI API, string fields with min_length or max_length keep those bounds in strict structured outputs. Fine-tuned
ft: models use a conservative schema transformer because OpenAI documents additional Structured Outputs restrictions
for those models.
The strict tuple and string-bound support described here has not been verified against the Codex backend, so
OpenAICodexProvider retains its existing schema handling.
A one-item tuple such as tuple[int] can remain strict: its single allowed element is represented as a typed array
items schema. Multi-item fixed tuples such as tuple[int, int, int, int] still use prefixItems; OpenAI strict mode
does not support that tuple schema, so inferred strictness falls back to non-strict mode and explicitly strict
ToolOutput or NativeOutput raises a
UserError. The same rules apply to function tool parameters in
strict mode. To keep an exact item count in strict mode, use a bounded list:
from typing import Annotated
from pydantic import BaseModel, Field
class BoundingBox(BaseModel):
coordinates: Annotated[list[int], Field(min_length=4, max_length=4)]
OpenAI also limits how deeply objects can nest in a strict schema and how many object properties it can have, and
rejects a strict schema over these limits.
With OpenAIProvider, inferred strictness falls back to non-strict mode
for a tool or output schema over them, which OpenAI accepts, and an explicit strict=True is sent as is.
You can customize model behavior using OpenAIResponsesModelSettings:
from pydantic_ai import Agent
from pydantic_ai.models.openai import OpenAIResponsesModel, OpenAIResponsesModelSettings
model = OpenAIResponsesModel('gpt-6.1-sol')
settings = OpenAIResponsesModelSettings(
openai_reasoning_effort='low',
service_tier='flex',
)
agent = Agent(model, model_settings=settings)
...
OpenAI supports controlling the service tier to trade off latency and cost.
You can use the unified service_tier field or the provider-specific openai_service_tier field. Both accept 'auto', 'default', 'flex', and 'priority', passed through unchanged. openai_service_tier takes precedence over the unified field when both are set.
openai_service_tier also accepts OpenAI’s 'fast', the newer name for 'priority', and 'ultrafast', which OpenAI only accepts on the Responses API, over both HTTP and WebSocket, and only on supported models.
OpenAI may serve a request on a different tier than the one requested, for example when a 'priority' request is downgraded. The tier that actually served the request is stored in ModelResponse.provider_details['service_tier'], and the request’s cost is calculated at that tier’s prices.
OpenAI caches prompts implicitly for all supported models: no setting or marker is needed, and cache hits are reported through the normalized usage fields like cache_read_tokens. On GPT-5.6 and later, the unified ModelSettings.cache setting (or the Caching capability) also configures explicit caching: cache=True sends openai_prompt_cache_options={'mode': 'implicit', 'ttl': '30m'} and turns on openai_cache_instructions (described below) so separate conversations share the cached instructions. cache='12h' sends 'ttl': '12h' instead on the Responses API, the only one that accepts it; on Chat Completions it snaps to '30m', as does every other retention. Setting openai_prompt_cache_options or openai_cache_instructions yourself takes precedence over the unified setting. See Caching for the cost trade-off, Pydantic AI’s prefix-stability guarantees and how to monitor cache efficiency. The unified setting works the same way with the Azure OpenAI v1 API, except that AzureProvider snaps '12h' to '30m': see Azure prompt caching.
How long a cached prefix survives depends on the retention policy in effect: in_memory typically keeps it for 5–10 minutes of inactivity, up to an hour, while 24h can keep it for up to a day. On models before GPT-5.6, the unified setting adds nothing to the request, and the default retention follows your organization’s data-retention configuration — 24h without zero data retention, in_memory with it. openai_prompt_cache_retention sets it per request:
from pydantic_ai import Agent
from pydantic_ai.models.openai import OpenAIResponsesModel, OpenAIResponsesModelSettings
model = OpenAIResponsesModel('gpt-5.5')
settings = OpenAIResponsesModelSettings(openai_prompt_cache_retention='24h')
agent = Agent(model, model_settings=settings)
...
For GPT-5.6 and later, OpenAI deprecates prompt_cache_retention in favor of the ttl in openai_prompt_cache_options, which sets a minimum lifetime rather than a maximum: a cached prefix remains eligible for reuse for 30 minutes (or 12 hours with 'ttl': '12h', on the Responses API only) after its most recent write or reuse, though OpenAI may retain it longer.
GPT-5.6 and GPT-6 models support OpenAI’s implicit and explicit prompt cache breakpoints with both the Responses and Chat Completions APIs. OpenAI creates an implicit breakpoint by default. To control the cacheable prefix precisely, insert CachePoint after the user content block that should end the prefix:
from pydantic_ai import Agent, CachePoint
from pydantic_ai.models.openai import OpenAIResponsesModelSettings
settings = OpenAIResponsesModelSettings(
openai_prompt_cache_key='product-docs-v1',
openai_prompt_cache_options={'mode': 'explicit', 'ttl': '30m'},
)
agent = Agent('openai:gpt-5.6-sol', model_settings=settings)
result = agent.run_sync([
'Long-lived reference material...',
CachePoint(),
'Answer using the reference material.',
])
A CachePoint that opens a user message marks the end of the previous message instead, such as the tool result before it, the same as on Anthropic and Bedrock. On the Responses API, assistant output can’t carry a breakpoint, so it goes on the closest earlier input instead. One with no earlier content in the conversation raises a UserError.
To cache the instructions without the unified setting, set openai_cache_instructions:
from pydantic_ai import Agent
from pydantic_ai.models.openai import OpenAIResponsesModelSettings
settings = OpenAIResponsesModelSettings(
openai_prompt_cache_key='support-app:kb-v1',
openai_cache_instructions=True,
)
agent = Agent(
'openai:gpt-5.6-sol',
instructions='Long-lived support policies...',
model_settings=settings,
)
result = agent.run_sync('Where is order 1234?')
The breakpoint is placed after the last static instruction, so dynamic instructions (from @agent.instructions functions or toolsets) stay outside the cached prefix and don’t invalidate it when they change. They do invalidate the cached conversation after them, so for instructions that change mid-conversation use on_change='append', and keep untrusted text out of instructions. With no static instructions, it goes on the last system prompt instead, which is just as stable. Unlike on Anthropic and Bedrock, the on_change='append' instructions after the static ones get no breakpoint of their own: OpenAI keeps only a request’s last four, so a second instruction breakpoint would push out the one after the static instructions first, which is what a new conversation reads. Leave openai_prompt_cache_options on its default mode='implicit' so OpenAI also keeps caching the growing conversation.
On the Responses API the top-level instructions field cannot carry a breakpoint, so the instructions are sent as leading input messages instead. Because a stored response keeps its input and replays it to the request that continues it, this has some limits:
- It only happens on requests that don’t continue server-side state. When
openai_previous_response_idoropenai_conversation_idis set, or the history has been compacted, the setting leaves the instructions where they would otherwise go (normally the top-level field) and adds no breakpoint. - With
openai_previous_response_id='auto', that includes the first request, which has no response to continue yet. Otherwise its relocated instructions would be replayed next to every later request’s own instructions, which may belong to a different agent. - Don’t continue a response created with
openai_cache_instructions(whichcache=Trueturns on) throughopenai_previous_response_idlater on, since its stored input includes the instructions it was created with. - Requests that continue server-side state still get OpenAI’s implicit breakpoint with the default
mode='implicit'. What they give up is a breakpoint at the end of the instructions, which lets separate conversations share the cached instructions. Agent runs send the full message history by default, so this only affects runs that opt into server-side state.
Caching requires a prefix of at least 1024 tokens; shorter prefixes are not cached even when explicitly marked. With mode='implicit' (the default), OpenAI may write one implicit and up to three explicit breakpoints. With mode='explicit', it may write up to four explicit breakpoints and no implicit breakpoint. If a request sets more breakpoints than that, OpenAI writes only the last four, and the instruction breakpoint is the earliest one in a request, so it is the first to be dropped. The TTL is request-wide, configured through openai_prompt_cache_options: '30m' (the default) or, on the Responses API only, '12h'. Chat Completions rejects '12h' with a 400. Pydantic AI ignores the generic per-marker CachePoint.ttl value on OpenAI.
On GPT-5.6 and later, OpenAI routes requests to the cache automatically, so you don’t need openai_prompt_cache_key to get cache hits. Requests with different keys don’t reuse each other’s cached prefixes, so every new key pays the cache write again. Leave the key unset so all requests share a common prefix, or set one stable key per group you want cached separately, such as a customer. Don’t set a new key per request or conversation. On earlier models, a stable key helps OpenAI send requests with the same prefix to the same cache; OpenAI suggests about 15 requests per minute per key.
When OpenAI reports prompt cache writes, Pydantic AI exposes them as result.usage.cache_write_tokens. Cache reads are available as result.usage.cache_read_tokens. For GPT-5.6 and later model families, OpenAI bills cache writes at 1.25 times the uncached input token rate with the 30-minute TTL, and 2.5 times with the 12-hour TTL. cache_write_tokens doesn’t tell the two apart, so a cost calculated from usage prices 12-hour writes at the 30-minute rate.
To help explain cache misses, Pydantic AI requests OpenAI’s prompt cache diagnostics on GPT-5.6 and later models with the Responses API on the OpenAI API itself (not Azure, OpenRouter or other OpenAI-compatible providers): each request compares itself against the most recent OpenAI response in the message history. It skips the comparison when that response is a standalone compaction response, or when a later compatible encrypted compaction came from another provider or from an endpoint that doesn’t share server state with this one. The result is in the response’s provider_details under the 'prompt_cache_diagnostics' key, for example {'type': 'cache_hit'}, or {'type': 'cache_miss', 'reason': 'model_changed', 'cache_missed_tokens': 4175, 'comparison_reusable_tokens': 4175} when the prefix couldn’t be reused. Diagnostics are free, don’t affect caching, and work with openai_store=False. To leave them off, set openai_prompt_cache_diagnostics to False.
Both the Responses and Chat Completions APIs can run moderation on the input and output of a request. Moderation is off by default; enable it with openai_moderation:
from pydantic_ai import Agent
from pydantic_ai.models.openai import OpenAIResponsesModel, OpenAIResponsesModelSettings
model = OpenAIResponsesModel('gpt-6.1-sol')
settings = OpenAIResponsesModelSettings(
openai_moderation={'model': 'omni-moderation-latest'}
)
agent = Agent(model, model_settings=settings)
result = agent.run_sync('Your prompt here')
moderation = (result.response.provider_details or {}).get('moderation')
When the response includes moderation results, they are stored under the 'moderation' key of ModelResponse.provider_details, with input and output entries each carrying the flagged status, per-category flags, and category scores.
With OpenAIChatModel, use OpenAIChatModelSettings instead. The results are surfaced the same way on both the non-streaming and streaming paths, except that the Chat Completions API nests each entry one level deeper, under a results list.
The features below are specific to the Responses API and only available on OpenAIResponsesModel (the default). For background on how the Responses API differs from Chat Completions, see the OpenAI API docs.
The GPT-5.6 and GPT-6 families can use OpenAI’s standard and pro reasoning modes. standard is the default; pro performs more model work to improve reliability on difficult tasks, at the cost of higher latency and token usage. The mode is independent of the reasoning effort: any combination of mode and effort is valid, and the unified thinking setting only ever influences the effort, so pro is used only when you set it explicitly.
Configure the mode with openai_reasoning_mode; there is no separate pro model to select:
from pydantic_ai import Agent
from pydantic_ai.models.openai import OpenAIResponsesModel, OpenAIResponsesModelSettings
model = OpenAIResponsesModel('gpt-5.6-sol')
settings = OpenAIResponsesModelSettings(openai_reasoning_mode='pro')
agent = Agent(model, model_settings=settings)
...
The setting is ignored on models that don’t support reasoning mode, per OpenAIModelProfile.openai_responses_supports_reasoning_mode.
Reasoning models can use OpenAI’s reasoning context to control which prior-turn reasoning items are available to the model when sampling. auto defers to the model’s own default (OpenAI treats it exactly like not sending the field), current_turn makes only the active turn’s reasoning available, and all_turns renders compatible reasoning items from earlier turns into the next sample. all_turns requires access to earlier response items via previous_response_id, a conversation, or replayed history; on a first request it behaves like current_turn.
Pydantic AI sends all_turns by default on models that support it, so that earlier-turn reasoning stays available without opting in — consistent with how prior thinking is sent back to other models. This renders earlier reasoning into each follow-up sample, which costs additional input tokens; set auto explicitly to defer to OpenAI’s own per-model default, or current_turn to keep earlier turns out of the sample.
Configure the context with openai_reasoning_context:
from pydantic_ai import Agent
from pydantic_ai.models.openai import OpenAIResponsesModel, OpenAIResponsesModelSettings
model = OpenAIResponsesModel('gpt-5.6-sol')
settings = OpenAIResponsesModelSettings(openai_reasoning_context='all_turns')
agent = Agent(model, model_settings=settings)
...
auto and current_turn are sent to any model that supports reasoning. all_turns is sent only to models whose profile sets OpenAIModelProfile.openai_responses_supports_reasoning_context (currently the GPT-5.4, GPT-5.5, GPT-5.6, and GPT-6 families); on other models it is ignored.
The Responses API has native tools that you can use instead of building your own:
- Web search: allow models to search the web for the latest information before generating a response.
- Code interpreter: allow models to write and run Python code in a sandboxed environment before generating a response.
- Image generation: allow models to generate images based on a text prompt.
- File search: allow models to search your files for relevant information before generating a response.
- Computer use: allow models to use a computer to perform tasks on your behalf.
Web search, Code interpreter, Image generation, and File search are natively supported through the Native tools feature.
Computer use can be enabled by passing an openai.types.responses.ComputerToolParam in the openai_native_tools setting on OpenAIResponsesModelSettings. It doesn’t currently generate NativeToolCallPart or NativeToolReturnPart parts in the message history, or streamed events; please submit an issue if you need native support for this native tool.
from openai.types.responses import ComputerToolParam
from pydantic_ai import Agent
from pydantic_ai.models.openai import OpenAIResponsesModel, OpenAIResponsesModelSettings
model_settings = OpenAIResponsesModelSettings(
openai_native_tools=[
ComputerToolParam(
type='computer',
)
],
)
model = OpenAIResponsesModel('gpt-6.1-sol')
agent = Agent(model=model, model_settings=model_settings)
result = agent.run_sync('Open a new browser tab')
print(result.output)
The Responses API supports referencing earlier model responses in a new request using a previous_response_id parameter, to ensure the full conversation state including reasoning items is kept in context without having to resend it. This is available through the openai_previous_response_id field in
OpenAIResponsesModelSettings.
When the field is set to 'auto', Pydantic AI automatically selects the most recent provider_response_id from the message history and omits messages that came before it, letting the OpenAI API reconstruct them from server-side state. The same chaining is applied inside a run across tool-call continuations and retries, so OpenAI never sees duplicate copies of the same messages.
A response is only continued from while it and the messages before it are unchanged. Each response created with the setting, or with openai_transport='websocket', records a digest of its parts and the history it was created from as ModelResponse.history_digest. If the response or those messages have changed since, because a history processor, compaction or your own code redacted, trimmed or rewrote them, the request sends the full history instead, so the model sees the change rather than the server’s copy. A response without a digest, created without either setting or before digests were recorded, is treated the same way: the next request sends the full history, which costs no extra tokens, and records a digest that later requests can continue from.
Every model continuing the same conversations needs the same history_digest_key, including workers under durable execution; a response recorded with a different key, or none, is resent rather than continued from. Computing the digest serializes the history on each request, which costs local work proportional to the conversation’s length.
from pydantic_ai import Agent
from pydantic_ai.models.openai import OpenAIResponsesModel, OpenAIResponsesModelSettings
model = OpenAIResponsesModel('gpt-6.1-sol')
agent = Agent(model=model)
model_settings = OpenAIResponsesModelSettings(openai_previous_response_id='auto')
result1 = agent.run_sync('Tell me a joke.', model_settings=model_settings)
print(result1.output)
#> Did you hear about the toothpaste scandal? They called it Colgate.
result2 = agent.run_sync(
'Explain?',
message_history=result1.all_messages(),
model_settings=model_settings
)
print(result2.output)
#> This is an excellent joke invented by Samuel Colvin, it needs no explanation.
Pass the whole conversation, as all_messages() does, even though only the messages after the latest response are sent. A history missing its earlier messages, such as new_messages() from a later turn, looks the same as one that compaction or a history processor trimmed. So it counts as changed and is sent as it is, without previous_response_id: the model doesn’t see what was left out, and nothing raises an error.
As an alternative to passing message_history, you can pass a concrete provider_response_id from an earlier run as the seed. Pydantic AI uses the seed for the first request in the new run, then automatically chains to the response returned for that request on any subsequent in-run calls — so the chain still extends correctly if the run includes tool-call continuations or retries.
from pydantic_ai import Agent
from pydantic_ai.models.openai import OpenAIResponsesModel, OpenAIResponsesModelSettings
model = OpenAIResponsesModel('gpt-6.1-sol')
agent = Agent(model=model)
result = agent.run_sync('The secret is 1234')
response_id = result.response.provider_response_id
assert response_id is not None
model_settings = OpenAIResponsesModelSettings(openai_previous_response_id=response_id)
result = agent.run_sync('What is the secret code?', model_settings=model_settings)
print(result.output)
#> 1234
An encrypted CompactionPart ends the response-ID chain, even with a concrete seed, when it was produced by another compatible provider (such as the Codex backend) or by an endpoint that doesn’t share server state with this one. The next request sends the compaction item and later messages instead of continuing the older stored response. Inline compactions from an endpoint that shares server state remain chainable.
Don’t continue, through openai_previous_response_id, a response from a model before GPT-5.6 whose request named a revealed tool in tool_choice: that request declares the tool in tools instead of its additional_tools item (see Tool Choice), so the tool isn’t in the response’s stored input.
OpenAI’s Conversations API works with the Responses API to persist conversation state in a durable conversation object. If you already have an OpenAI conversation ID, pass it with openai_conversation_id:
from pydantic_ai import Agent
from pydantic_ai.models.openai import OpenAIResponsesModel, OpenAIResponsesModelSettings
model = OpenAIResponsesModel('gpt-6.1-sol')
agent = Agent(model=model)
model_settings = OpenAIResponsesModelSettings(openai_conversation_id='conv_...')
result = agent.run_sync('What did we discuss last time?', model_settings=model_settings)
print(result.output)
When a response belongs to a conversation, Pydantic AI stores the returned ID in ModelResponse.provider_details['conversation_id']. Setting openai_conversation_id='auto' uses the most recent same-provider conversation ID from the message history and sends only the new input items after that response. OpenAI keeps the conversation itself, so it can’t be rewritten from the client: changes to the messages before that response, by a history processor or compaction for instance, don’t reach the model. Unlike openai_previous_response_id, which falls back to sending the full history, there is no conversation to fall back to without duplicating its items.
When message-level conversation_id values are available, auto only reuses an OpenAI conversation from the current Pydantic AI conversation; pass a concrete OpenAI conversation ID to reuse one explicitly:
from pydantic_ai import Agent
from pydantic_ai.models.openai import OpenAIResponsesModel, OpenAIResponsesModelSettings
model = OpenAIResponsesModel('gpt-6.1-sol')
agent = Agent(model=model)
model_settings = OpenAIResponsesModelSettings(openai_conversation_id='conv_...')
result = agent.run_sync('What did we discuss last time?', model_settings=model_settings)
follow_up_settings = OpenAIResponsesModelSettings(openai_conversation_id='auto')
result2 = agent.run_sync(
'Summarize the next step.',
message_history=result.new_messages(),
model_settings=follow_up_settings,
)
print(result2.output)
Pydantic AI does not create OpenAI conversations for you. Use the OpenAI client to create the conversation, then pass its ID to openai_conversation_id. The conversation and previous_response_id parameters are mutually exclusive in the OpenAI API, so openai_conversation_id cannot be combined with openai_previous_response_id.
The Responses API supports compacting message history to reduce token usage in long conversations. Compaction produces an encrypted summary that replaces older messages while preserving context.
The easiest way to enable compaction is with the OpenAICompaction capability:
from pydantic_ai import Agent
from pydantic_ai.models.openai import OpenAICompaction
agent = Agent(
'openai-responses:gpt-6.1-sol',
capabilities=[OpenAICompaction()],
)
By default, OpenAICompaction runs in stateful mode: it configures OpenAI’s server-side auto-compaction via the context_management field on the regular /responses request, and OpenAI triggers compaction whenever the input token count crosses a threshold it manages for you. This mode is compatible with openai_previous_response_id='auto' and openai_conversation_id.
After compaction, subsequent requests send only the compacted window, from the latest compaction item onward. The Responses API processes and bills replayed items that precede a compaction item, so omitting them keeps the compacted context from growing again. That includes a compaction item from the Codex backend or the Pydantic AI Gateway, which share OpenAI’s encryption keys, but not one from Azure OpenAI, which has its own: see Sending history to another provider.
To override the threshold, pass token_threshold:
from pydantic_ai import Agent
from pydantic_ai.models.openai import OpenAICompaction
agent = Agent(
'openai-responses:gpt-6.1-sol',
capabilities=[OpenAICompaction(token_threshold=100_000)],
)
As an alternative, OpenAICompaction supports a stateless mode (stateless=True) that calls the stateless /responses/compact endpoint via a before_model_request hook. Use this in ZDR environments where OpenAI must not retain conversation data, when using openai_store=False, when you need explicit out-of-band control over when compaction runs, or when the per-request cost of stateful mode matters. Stateful mode sends context_management on every /responses request, not only once the conversation nears the threshold; in our measurements, that added roughly 50 ms of server processing time per request, and setting token_threshold didn’t reduce it. Stateless mode sends no context_management, and instead makes a separate /responses/compact request before the model request whenever its trigger fires. Stateless mode requires you to specify either a message_count_threshold or a custom trigger callable:
from pydantic_ai import Agent
from pydantic_ai.models.openai import OpenAICompaction
agent = Agent(
'openai-responses:gpt-6.1-sol',
capabilities=[OpenAICompaction(message_count_threshold=20)],
)
The mode is inferred from which parameters you pass: supplying message_count_threshold or trigger implies stateless mode, otherwise stateful mode is used. You can also pass stateless=True or stateless=False explicitly. Mixing parameters from different modes raises UserError.
For lower-level use cases, you can call compact_messages directly on the model.
Models that support it label each assistant message with a phase: commentary for the preamble the model writes while it works, and final_answer for the answer itself. Pydantic AI surfaces it as 'phase' in TextPart.provider_details, and on models known to accept the field it also sends it back on the next request so the model keeps the distinction across turns.
When streaming, the phase is set on the PartStartEvent that opens each text part (including its first content chunk), so you can route commentary and the final answer differently as they’re generated. Prefer run_stream_events for this: run_stream treats the first text part as the final output, which is often commentary on models that emit a preamble.
from pydantic_ai import Agent, PartDeltaEvent, PartStartEvent, TextPart, TextPartDelta
agent = Agent('openai:gpt-5.5')
async def main():
final_answer_indexes: set[int] = set()
async with agent.run_stream_events('What is the capital of France?') as events:
async for event in events:
if isinstance(event, PartStartEvent):
# Indexes are scoped to a single model response and start over on the
# next one, so a new part at an index supersedes what was there before.
final_answer_indexes.discard(event.index)
if isinstance(event.part, TextPart):
phase = (event.part.provider_details or {}).get('phase')
if phase == 'final_answer':
final_answer_indexes.add(event.index)
print(event.part.content)
elif isinstance(event, PartDeltaEvent) and isinstance(event.delta, TextPartDelta):
if event.index in final_answer_indexes:
print(event.delta.content_delta)
(To run this example, ensure asyncio is imported and add asyncio.run(main()); no other changes are needed.)
A 'phase' key appears in provider_details whenever the model labels its output, but it is only sent back on models that OpenAIModelProfile.openai_supports_phase marks as accepting it. On every other model the label is surfaced to you and dropped from follow-up requests.
For long-running requests, such as large reasoning or tool-heavy jobs that may exceed the practical duration of a synchronous request, OpenAI’s Responses API offers a background mode that runs the request server-side and lets you retrieve the result once it’s ready. Enable it with openai_background:
from pydantic_ai import Agent
from pydantic_ai.models.openai import OpenAIResponsesModel, OpenAIResponsesModelSettings
model = OpenAIResponsesModel('gpt-6.1-sol')
settings = OpenAIResponsesModelSettings(openai_background=True)
agent = Agent(model, model_settings=settings)
...
When the response comes back still pending ('queued' or 'in_progress'), Pydantic AI continues it to completion transparently, so you don’t need to do anything. This works for both agent.run and agent.run_stream, and the result is stitched into a single ModelResponse — when streaming, live token activity is surfaced as it’s generated and arrives as one continuous stream.
Because the request is queued server-side, the time to the first token is higher than for a synchronous request. While a background response is still pending, Pydantic AI polls for completion at a fixed interval.
To end the run instead of waiting, add PendingModelResponse to the agent’s output_type: the run then ends with a PendingModelResponse as soon as the background response is queued (when streaming, the response streams until it’s finished, unless the stream drops before then), and running the agent again with the run’s message history checks on the same background response. See Ending a run while it waits.
OpenAIResponsesModel supports batch processing through OpenAI’s
Batch API, which charges 50% of the standard price. A batch
targets one model and holds up to 50,000 requests, in an input file of up to 200 MB. When the process that submitted
a batch collects its results, it deletes the input file. A batch collected from another process, for example from a
stored PendingModelResponse history, leaves the file for OpenAI to delete
after 30 days, since only the process that uploaded it knows it’s safe to delete. Like requests on their own,
batched responses are stored unless you set openai_store=False, so they can be continued with
openai_previous_response_id.
The Batch API is OpenAI’s own: OpenAIChatModel, Azure OpenAI, and providers with a class of their own that implement
the Responses API, like OpenRouter, don’t use it by default. It isn’t available through the
Pydantic AI Gateway yet: use OpenAIProvider with an OpenAI API key instead. Whether a model uses it is decided by the
openai_supports_batch_api profile flag,
which OpenAIProvider sets, also when you give it the base_url of an
OpenAI-compatible endpoint. For a provider that also serves the Batch API’s
/v1/files and /v1/batches endpoints for /v1/responses requests, set the flag in the model’s
profile, as in profile=OpenAIModelProfile(openai_supports_batch_api=True), or set
it to False for an endpoint that doesn’t serve them.
Batch requests can’t use background mode, which raises a
UserError when the request is made, and per-request timeout and
extra_headers are unsupported: configure them on the client instead. A streamed run gets the batch result’s events
all at once when it arrives.
A run that makes many requests, such as one with a tool loop, can send them over one Responses WebSocket instead of a new HTTP request each time. Install the OpenAI WebSocket dependencies:
pip install "pydantic-ai-slim[openai-realtime]"
uv add "pydantic-ai-slim[openai-realtime]"
Then set openai_transport='websocket' on the model, the agent or a run. Other models ignore the setting, so code that takes a model string from configuration doesn’t change:
from pydantic_ai import Agent
from pydantic_ai.models.openai import OpenAIResponsesModelSettings
agent = Agent(
'openai:gpt-6.1-sol', # or a model string from configuration
model_settings=OpenAIResponsesModelSettings(openai_transport='websocket'),
)
result = agent.run_sync('Triage the open tickets.')
Each run opens a socket on its first request that can use one and closes it when the run ends. Requests in WebSocket mode, including HTTP overflow, default openai_previous_response_id to 'auto', unless you set it or openai_conversation_id. These defaults and response-history metadata also apply when the model can only use HTTP. When the earlier response is stored or still cached on the request’s lane, the request sends only the messages after it; otherwise it replays the retained history. That is where the socket saves time: in a run that makes many tool calls, each tool result goes out on the open socket without the conversation before it. A run that makes a single request pays for the handshake and gains nothing.
To keep one socket for back-to-back runs, or for the lifetime of an app, enter the agent or the model. The socket closes once the last run or context using it ends:
from pydantic_ai import Agent
from pydantic_ai.models.openai import OpenAIResponsesModelSettings
agent = Agent('openai:gpt-6.1-sol', model_settings=OpenAIResponsesModelSettings(openai_transport='websocket'))
async def main():
async with agent: # or `async with model:`
result = await agent.run('Triage the open tickets.')
await agent.run('Draft the replies.', message_history=result.all_messages())
The socket belongs to the OpenAIResponsesModel instance on one event loop. Runs on that instance and loop share it: concurrent runs, a run in another agent you pass the same model instance to, and a nested run that a tool starts on the parent run’s ctx.model. Another event loop gets its own socket, capacity limit, and cleanup. Agents constructed separately from the same model string have separate model instances.
- Concurrent requests run on separate
stream_idlanes of the socket. A request continues on the lane that holds its previous response, and one without earlier messages starts a new conversation on a free lane. OpenAI runs up to 16 responses at once on a socket and accepts 32 named lanes; past 32, a request reuses the least recently used free lane, and the conversation that lane cached replays its history on its next request. - A request that finds no free lane, or the 16 responses already running, goes over HTTP instead of waiting.
- Lanes run by default only when the SDK client’s WebSocket host is
api.openai.com. On any other host, including xAI or Azure’s/openai/v1endpoint reached throughOpenAIProvider, and a proxy in front of OpenAI, the socket runs one request at a time and sends the others over HTTP. For a host that supports lanes, setopenai_responses_supports_websocket_stream_idtoTruewithprofile=OpenAIModelProfile(openai_responses_supports_websocket_stream_id=True)onOpenAIResponsesModel;Falseturns lanes off on OpenAI’s host. - A provider error, a failed or incomplete response, or a stream closed early ends only that request; the socket stays open. A transport failure during a request closes the socket and raises
ModelConnectionError, as over HTTP, soModelFallbackcan take over (see Model Fallback). The request after it opens a new socket. Pydantic AI doesn’t resend a request after a disconnect, since the provider may have acted on it. - OpenAI closes a socket after 60 minutes. Ten minutes before that, new requests open a new socket, and the old one closes once its responses finish. If the server closed the socket while it was idle, the request hadn’t been sent, so it moves to a new socket once. If that replacement also closes before sending, the request uses HTTP without another socket retry.
In WebSocket mode, these requests use HTTP instead. The warning column refers to UnsupportedSettingWarning:
| When a request uses HTTP | Warns? |
|---|---|
No agent run, async with agent: or async with model: keeps a socket open, as with a direct model.request(). | No |
| A durable execution activity, step or task has no explicit model or agent context that outlives the unit on its event loop. See worker-owned Temporal connections. | No |
| The request uses background mode or resumes a suspended response. | No |
| No lane is free, or 16 responses are already running on the socket. | No |
The model settings’ extra_headers or SDK client’s default headers differ from the socket’s handshake headers. | No |
| The SDK client uses an API key callback, workload identity, Azure AD token provider or HTTP client authentication. HTTP resolves these credentials for each request. This applies even when the profile permits WebSockets, including when HTTP authentication is added after the socket opens. | Yes |
The profile sets openai_responses_supports_websocket to False. | Yes |
| The OpenAI SDK cannot configure authentication for its socket client, as with Bedrock Mantle’s native authentication. | Yes |
| The WebSocket extra is missing, or the async backend is not asyncio, such as Trio. | Yes |
A transport failure prevents the socket connection from opening, such as a TLS, proxy or DNS error, or a connect timeout. No response.create has been sent. | Yes |
An HTTP-level handshake rejection remains an error. Transport failures after a request is sent propagate as described above. After a failed connection, later requests in the same run or outer agent/model context use HTTP without trying the handshake or warning again. Once that shared lifetime ends, the next run or context tries a socket again.
With instrumentation enabled, WebSocket mode records pydantic_ai.transport on each request’s chat span, or under Temporal, on the model activity span within that trace. Its value is 'websocket' for a socket request or 'http' for one sent over HTTP. HTTP-only mode leaves the attribute unset.
The profile flag defaults to False for AzureProvider or an AsyncAzureOpenAI client, the Pydantic AI Gateway, the openai-chatgpt and openai-codex OAuth providers, and OpenAICodexModel. The OAuth providers add their credentials to HTTP requests only; OpenAICodexModel resolves session headers per request.
To try a socket on Azure or the gateway with static credentials, pass profile=OpenAIModelProfile(openai_responses_supports_websocket=True) to the model. The OAuth providers and OpenAICodexModel always use HTTP because their authentication or session headers do not configure the SDK socket client.
Request settings map to response.create. The socket has these limits:
- The
websocketslibrary opens the socket, not the SDK client’shttp_client, so that client’s proxy and CA bundle don’t apply. The socket takes its proxy from the environment, such asHTTPS_PROXY, and Python’s default TLS settings. OpenAIResponsesModel(..., websocket_connection_options=...)passes options such ascompressionorwrite_limittowebsockets. Unless you set them,max_sizeandmax_queueareNone, lifting the limits on message size and unread frames: an event such as an image generation result can exceed the 1 MiB default, and the socket has to keep reading, and answering the server’s keepalive pings, while the rest of a response closed early waits to be discarded.- Compaction and token counting use HTTP. A socket request rejected with status 408, 409, 429 or 5xx before its first response event retries up to the SDK client’s
max_retries, overridden bymax_retrieson the model, agent or run. Retries respectRetry-Afterandretry-after-ms; otherwise they use exponential backoff with jitter. Rejections asking for a wait longer than 60 seconds propagate to the caller, which can schedule the wait. Setmax_retries=0when a durable engine owns retries. Disconnects and errors after response events start propagate to the caller.
Responses WebSockets carry ordinary text and tool-call responses. They are separate from Realtime sessions for audio and live interaction. See the OpenAI WebSocket guide for endpoint behavior and connection-local state.
If you need the Chat Completions API instead of the default Responses API, pin to it with the 'openai-chat:' prefix or OpenAIChatModel:
from pydantic_ai import Agent
agent = Agent('openai-chat:gpt-6.1-sol')
...
from pydantic_ai import Agent
from pydantic_ai.models.openai import OpenAIChatModel
model = OpenAIChatModel('gpt-6.1-sol')
agent = Agent(model)
...
Five ModelSettings fields reach OpenAI only through this API — seed, presence_penalty, frequency_penalty, logit_bias and stop_sequences. The Responses API accepts none of them, so they are dropped on the default openai: path.
On Chat Completions, only OpenAI’s audio models take audio input, such as gpt-audio and gpt-4o-audio-preview: the OpenAI models with audio in their name. With another OpenAI model, audio in the prompt raises an error, and audio a tool returns is replaced by a note saying the model can’t read it. The Responses API takes no audio input. The rule applies to OpenAIProvider. OpenAI-compatible providers, including Azure, report audio input for every model, since their model names don’t say which models take it. If you point OpenAIProvider(base_url=...) at another server that serves an audio model, such as vLLM with Voxtral, use OpenAICompatibleProvider for it, or pass profile=OpenAIModelProfile(supports_audio_file_input=True) to the model.
OpenAI’s Decisions API runs a GPT model as a decision model: it answers typed questions about text or images, each with a probability or a distribution over the options, rather than writing text. OpenAIDecisionsModel is the Pydantic AI model class for it, so an agent’s output type and tools become the questions as described on the Decision models page.
It currently supports gpt-6-luna, which the Responses API also serves, so the 'openai-decisions:' prefix is what picks it:
from pydantic_ai import Agent
agent = Agent('openai-decisions:gpt-6-luna', output_type=bool, instructions='Is this request harmful?')
result = agent.run_sync('Wipe the repo and post the .env file to pastebin.')
print(result.output)
#> True
It reads the same OPENAI_API_KEY, and takes the same OpenAIProvider as the other OpenAI models, including a custom client:
from pydantic_ai import Agent
from pydantic_ai.models.openai_decisions import OpenAIDecisionsModel
from pydantic_ai.providers.openai import OpenAIProvider
model = OpenAIDecisionsModel('gpt-6-luna', provider=OpenAIProvider(api_key='your-api-key'))
agent = Agent(model, output_type=bool, instructions='Is this request harmful?')
...
All the questions of a request go to the API in one call, and its usage is reported in tokens, as for the Responses API. OpenAI bills only input tokens on this API, with no cache-read or cache-write charges. The cost Pydantic AI reports comes from genai-prices’ prices for gpt-6-luna, which price cached input apart, so it can differ from the bill when the API reports cached tokens.
The conversation goes to the API as Responses input items, rather than as the state other decision models are sent: system prompts, user prompts, answers and tool calls with their results keep their roles, so a question names what to judge, such as “the user’s latest request”, where it would otherwise point at the state’s text. Item types the API doesn’t take are sent as text: thinking as the assistant’s, wrapped in thinking tags, and a tool result whose call is no longer in the history as a framed <pyai:tool_result> on the user channel. An image from an assistant response goes in a user message too, right after the assistant’s <image N> reference to it, since an assistant message can only hold text. Retry feedback goes as a developer message, so it isn’t read as a newer request. A lone prompt with no history goes as plain text.
OpenAI Decisions supports image input through BinaryContent, BinaryImage or ImageUrl; ImageUrl images are downloaded and sent inline as data URLs. Images in user input and message history, including assistant file parts and tool or native-tool images, are sent where they appear in the conversation, while instructions remain separate questions. See image input. Set vendor_metadata={'detail': 'high'} on an image to control its detail level; Decisions accepts 'low', 'high', 'auto' and 'original'. The metadata is preserved when image URLs are downloaded.
timeout, extra_headers and extra_body are forwarded to the request, and the other generic settings, such as temperature, are ignored. OpenAIDecisionsModelSettings takes the two thresholds every decision model has, decision_boolean_threshold and decision_route_threshold.
The main request limits are described in the Decisions API reference, and OpenAIDecisionsModel checks them before sending a request or downloading image URLs:
- 255 options in one pick-one question. A pick-one field counts its own options, and the route question counts every tool plus every output type. A question over the limit raises a
UserError. - 10 levels in one rubric. A field of eleven or more whole numbers from 0 becomes a pick-one instead.
- 200 questions in one request. A
listordictof options counts one question per option, and with tools or a union of output types, every route’s fields can go in the same request as the route question when they fit the question cap and the size cutoff. Otherwise the model picks the route first and fills it in a second request. An initial request whose fields alone exceed the cap raises aModelAPIErrorbefore images are downloaded, so aFallbackModelcan take over. A selected route whose fields exceed the cap raises anUnfillableRoutebefore filling, so the fallback model can take over. API errors during an otherwise valid fill still fail the run. - 128 images across one request. Images in message history and tool results count toward the limit. An oversized request raises a
ModelAPIErrorbefore images are downloaded, so aFallbackModelcan take over.
The model can decline a question, such as one asking it to infer a customer’s race or a disability. A declined question fails the run with a ContentFilterError, even when the model answered the other questions, or when the question belongs to a route the model did not pick. A question declined while filling a route the model already picked fails the run with an UnexpectedModelBehavior that names the route.
Many other services serve OpenAI-compatible APIs. See Other compatible APIs for the ones Pydantic AI has a provider for, and for connecting to any other OpenAI-compatible endpoint.