pydantic_ai.settings

ModelSettings

Bases: TypedDict

Settings to configure an LLM.

Includes only settings which apply to multiple models / model providers, though not all of these settings are supported by all models.

Each field’s Supported by: list names the model classes that put the setting on the wire. A bare name covers every interface that model serves, so OpenAI means both OpenAIChatModel and OpenAIResponsesModel; a name qualified with an interface, like OpenAI Chat Completions, covers only that one, because the Responses API does not accept the setting at all.

These lists are parsed and checked against the wire by tests/models/test_model_settings_support.py, so keep the * Name bullet shape and put any nuance in parentheses after the name.

A model ignores a setting it doesn’t support, so the same settings work across models. For the settings whose absence changes what the request guarantees (timeout, max_retries, stop_sequences, parallel_tool_calls=False, extra_headers and extra_body), it emits an UnsupportedSettingWarning; the others, like temperature, are ignored silently. A model class declares what it ignores in Model.unsupported_settings.

Being listed means Pydantic AI sends the setting, not that the service honors it: the OpenAI-compatible model classes forward whatever the OpenAI schema accepts, and an individual provider behind one of them may ignore a field its own API doesn’t define, or reject it. Where we know of such a case it is noted on the entry, but the provider’s own API reference is the authority.

All types must be serializable using Pydantic.

Attributes

max_tokens

The maximum number of tokens to generate before stopping.

Supported by:

OpenAI
Anthropic
Google
Groq
Cohere
Mistral
Bedrock
MCP Sampling
xAI
HuggingFace
Cerebras
Crusoe
GitHub Copilot
Ollama
OpenRouter
Snowflake
Z.AI
Bedrock Mantle

Type: int

temperature

Amount of randomness injected into the response.

Use temperature closer to 0.0 for analytical / multiple choice, and closer to a model’s maximum temperature for creative and generative tasks.

Note that even with temperature of 0.0, the results will not be fully deterministic.

Supported by:

OpenAI
Anthropic
Google
Groq
Cohere
Mistral
Bedrock
MCP Sampling
xAI
HuggingFace
Cerebras
Crusoe
GitHub Copilot (not sent for the Anthropic models that reject sampling settings, e.g. claude-opus-4.8)
Ollama
OpenRouter
Snowflake
Z.AI
Bedrock Mantle
System One

Type: float

top_p

An alternative to sampling with temperature, called nucleus sampling, where the model considers the results of the tokens with top_p probability mass.

So 0.1 means only the tokens comprising the top 10% probability mass are considered.

You should either alter temperature or top_p, but not both.

Supported by:

OpenAI
Anthropic
Google
Groq
Cohere
Mistral
Bedrock
xAI
HuggingFace
Cerebras
Crusoe
GitHub Copilot (not sent for the Anthropic models that reject sampling settings, e.g. claude-opus-4.8)
Ollama
OpenRouter
Snowflake
Z.AI
Bedrock Mantle

Type: float

top_k

Only sample from the top K options for each subsequent token.

Used to remove “long tail” low probability responses.

Supported by:

Anthropic
Google
Cohere
Bedrock (Anthropic and Amazon Nova models only)

Type: int

timeout

Override the client-level default timeout for a request, in seconds.

Numeric seconds work everywhere. A legacy httpx.Timeout is also accepted and is converted to an httpx2.Timeout on the paths whose SDK expects one. httpx2.Timeout is deliberately not part of this contract, because some SDKs behind these settings still reject it.

On an HTTP client Pydantic AI created (including one from create_async_httpx2_client()), a number of seconds (or an httpx.Timeout whose phases are all equal) can shorten but never lengthen the client’s own connect timeout (5 seconds by default) and pool timeout, so a long request timeout doesn’t also allow a long wait to connect. An httpx.Timeout whose phases differ, and any timeout on another client you pass in yourself, is used as given.

Supported by:

OpenAI
OpenAI ChatGPT
OpenAI Codex
Anthropic
Google (numeric seconds only, not httpx.Timeout)
Groq
Mistral (numeric seconds only, not httpx.Timeout)
Cerebras
Crusoe
GitHub Copilot
Ollama
OpenRouter
Snowflake
Z.AI
Bedrock Mantle
TypeSafe
System One
OpenAI Decisions

Type: int | float | Timeout

request_timeout

A deadline for one request to one model, in seconds, enforced by Pydantic AI for every model.

timeout is applied by the provider client to each attempt: an SDK retry re-arms it, and on a streamed response it only limits the wait for each chunk. This one clock instead covers every attempt the provider SDK and HTTP transport make, any continuation of a paused response, and reading a streamed response through its last chunk, including the time your code takes to consume it. In Temporal’s terms, timeout is the start-to-close timeout of one attempt and request_timeout the schedule-to-close timeout of the whole request.

When it runs out, the request is cancelled and ModelRequestTimeout is raised. A FallbackModel gives each model it tries a fresh deadline, and by default falls back to the next model when one runs out.

It is not sent to the provider. See Timeouts.

Type: float

max_retries

Override how many times the provider SDK’s client retries a failed request, for one request.

0 makes a single attempt. The SDK still decides which failures to retry and how long to wait between attempts; this setting changes only how many retries it makes, so a request is sent at most 1 + max_retries times. Without it, the client’s own configuration applies: 2 retries for the OpenAI, Anthropic, Groq and Cohere clients Pydantic AI builds, and none for Google’s.

Set it to 0 when something else owns retrying, such as a retrying HTTP transport, a durable execution engine’s retry policy, or a FallbackModel that should move to its next model as soon as one fails. See Provider SDK retries.

Supported by:

OpenAI
OpenAI ChatGPT
OpenAI Codex
Anthropic
Google (keeps the delays and status codes of the provider’s retry_options)
Groq
Mistral (0 only, which turns off a retry_config set on the client; other values are ignored without a warning)
Cohere
Cerebras
Crusoe
GitHub Copilot
Ollama
OpenRouter
Snowflake
Z.AI
Bedrock Mantle
TypeSafe (the request’s other RetryPolicy options are the SDK defaults)
OpenAI Decisions

Type: int

parallel_tool_calls

Whether to allow parallel tool calls.

Supported by:

OpenAI (some models, not o1)
OpenAI ChatGPT
OpenAI Codex
Anthropic
Groq
Mistral
xAI
Cerebras
Crusoe
GitHub Copilot
Ollama
OpenRouter
Snowflake
Z.AI
Bedrock Mantle

Type: bool

tool_choice

Control which tools the model can use.

See the Tool Choice guide for detailed documentation and examples.

None (default): Defaults to 'auto' behavior
'auto': All tools available, model decides whether to use them
'none': Disables function tools; model responds with text only (output tools remain for structured output)
'required': Forces tool use; excludes output tools so the agent cannot produce a final response when set statically
list[str]: Force a call to one of the specified function and/or output tools; name an output tool to let an agent run finish
ToolOrOutput: Specified function tools plus output tools/text/image

Note: setting 'required' statically (via the model_settings argument of Agent.run or the agent’s own model_settings) raises a UserError, because it excludes output tools and forces a function tool call on every step. A static list[str] is accepted only when it names at least one output tool for the run; otherwise it also prevents the agent from producing a final response. To vary tool_choice per step (e.g. force a tool on the first step only), return a callable from a capability’s get_model_settings — those values are trusted to adapt across steps. For single API calls without an agent loop, use pydantic_ai.direct.model_request.

Supported by:

OpenAI
Anthropic ('required' and specific tools not supported with thinking enabled)
Google
Groq
Cohere (a named subset is honored by filtering the tool list, not sent as a parameter)
Mistral (a single forced tool is sent as a parameter; a larger named subset is honored by filtering the tool list)
Bedrock
xAI
HuggingFace
Cerebras
Crusoe
GitHub Copilot
Ollama (sent, but Ollama documents tool_choice as unsupported)
OpenRouter
Snowflake
Z.AI
Bedrock Mantle

Type: ToolChoice

seed

The random seed to use for the model, theoretically allowing for deterministic results.

Supported by:

OpenAI Chat Completions
Google
Groq
Cohere
Mistral
xAI
HuggingFace
Cerebras
Crusoe
GitHub Copilot
Ollama
OpenRouter
Snowflake
Z.AI
Bedrock Mantle Chat Completions

Type: int

presence_penalty

Penalize new tokens based on whether they have appeared in the text so far.

Supported by:

OpenAI Chat Completions
Google
Groq
Cohere
Mistral
xAI
HuggingFace
Cerebras
Crusoe
GitHub Copilot
Ollama
OpenRouter
Snowflake
Z.AI
Bedrock Mantle Chat Completions

Type: float

frequency_penalty

Penalize new tokens based on their existing frequency in the text so far.

Supported by:

OpenAI Chat Completions
Google
Groq
Cohere
Mistral
xAI
HuggingFace
Cerebras
Crusoe
GitHub Copilot
Ollama
OpenRouter
Snowflake
Z.AI
Bedrock Mantle Chat Completions

Type: float

logit_bias

Modify the likelihood of specified tokens appearing in the completion.

Supported by:

OpenAI Chat Completions
Groq
HuggingFace
Crusoe
GitHub Copilot
Ollama (sent, but Ollama documents logit_bias as unsupported)
OpenRouter
Snowflake
Z.AI
Bedrock Mantle Chat Completions

Type: dict[str, int]

stop_sequences

Sequences that will cause the model to stop generating.

Supported by:

OpenAI Chat Completions
Anthropic
Google
Groq
Cohere
Mistral
Bedrock
MCP Sampling
xAI
HuggingFace
Cerebras
Crusoe
GitHub Copilot
Ollama
OpenRouter
Snowflake
Z.AI
Bedrock Mantle Chat Completions

Type: list[str]

extra_headers

Extra headers to send to the model.

Supported by:

OpenAI
OpenAI ChatGPT
OpenAI Codex
Anthropic
Google
Groq
Bedrock
Cerebras
Crusoe
GitHub Copilot
Ollama
OpenRouter
Snowflake
Z.AI
Bedrock Mantle
TypeSafe
System One
OpenAI Decisions

Type: dict[str, str]

thinking

Enable or configure thinking/reasoning for the model.

True: Enable thinking with the provider’s default effort level.
False: Disable thinking. On a model that can’t turn thinking off, use the lowest effort it accepts; a model that offers no thinking setting at all keeps its default.
'minimal'/'low'/'medium'/'high'/'xhigh'/'max': Enable thinking at a specific effort level.

When omitted, the model uses its default behavior (which may include thinking for reasoning models).

Provider-specific thinking settings (e.g., anthropic_thinking, openai_reasoning_effort) take precedence over this unified field. When one comes from a broader scope than this field, such as the model’s settings while this is set for a run, a UserWarning names both.

Listed below are the model classes that translate this field onto the request. Where some of a class’s models take no thinking parameter at all, the per-model behavior is on the Thinking page (Mistral’s dated magistral ids).

Supported by:

OpenAI
Anthropic
Google
Groq
Mistral
Cohere (as thinking, with effort levels as a token_budget)
Bedrock
xAI
Cerebras (only False is forwarded, as reasoning_effort='none', or 'low' on gpt-oss, which can’t turn reasoning off; the enable levels are not sent because Cerebras models reason by default)
Crusoe
GitHub Copilot (as reasoning_effort; which ids accept it, and which levels, is per-model and enforced by Copilot, so an unsupported value comes back as a 400 invalid_reasoning_effort)
Ollama
OpenRouter (as extra_body['reasoning'])
Snowflake (as extra_body['reasoning'] on Claude models, otherwise as reasoning_effort)
Z.AI (as extra_body['thinking'])
Bedrock Mantle (the Responses interface only; the Chat Completions interface serves only the gpt-oss-safeguard models, which take no thinking parameter)

Type: ThinkingLevel

citations

Whether to ask for citations from providers that only cite when asked.

Citations go on TextPart.citations.

Supported by:

Anthropic (documents, including those returned by tools, and Web Fetch)
Bedrock (TXT and PDF documents, including those returned by tools, on Anthropic models)
xAI (inline citations)

Only used when the output is text. On Anthropic models, a tool_choice that forces a tool call raises a UserError when the request has something to cite. Providers that always cite ignore this setting. See Citations, tools and structured output.

Type: bool

cache

Enable, configure, or disable library-managed prompt caching for the model request.

Prompt caching lets the provider serve a long or multi-turn prompt’s stable prefix from its cache instead of re-processing it. Cache writes cost more than uncached input (1.25x for Anthropic’s 5-minute and OpenAI’s GPT-5.6 caches, 2x for Anthropic’s 1-hour cache) while cache reads cost about 0.1x, so a 1.25x write breaks even after one read and a 2x write after two.

True: Cache the stable prompt prefix (tool definitions and static instructions) and the growing conversation, with the provider’s default retention. The library places cache breakpoints, using the provider’s automatic caching mode for the conversation where one exists.
False: Disable library-managed caching, the same as leaving the setting unset but also overriding a cache value in the model’s default settings. Explicit CachePoint markers and provider-specific cache settings still apply, and providers that cache implicitly (e.g. OpenAI, Gemini) still do.
'5m'/'30m'/'1h': Cache with a specific retention, snapped to the nearest tier the provider supports (down where a shorter tier exists).
A CacheConfig: Cache with an optional retention, and with messages=False cache only the stable prefix (tool definitions and static instructions), not the conversation: useful for many one-off conversations that share long instructions or tools.

Explicit CachePoint markers in the message history can be combined with this setting; when a request would exceed the provider’s maximum number of cache breakpoints, the oldest message breakpoints are dropped first (on OpenAI, the server drops the earliest breakpoints first, starting with the instruction breakpoint). Provider-specific cache settings (e.g. anthropic_cache, bedrock_cache_instructions) take precedence: if any is set, including to False, this unified field is ignored entirely for that request.

Silently ignored by model classes not listed below, and by models that cache implicitly without request-side configuration.

Supported by:

Anthropic (as anthropic_cache_instructions, anthropic_cache_tool_definitions and anthropic_cache, with anthropic_cache_messages instead of anthropic_cache on the Bedrock and Vertex SDK clients)
Bedrock (Anthropic and Amazon Nova models only; as bedrock_cache_instructions, bedrock_cache_tool_definitions and bedrock_cache_messages)
OpenRouter (Anthropic and Gemini models only; as openrouter_cache_instructions, openrouter_cache_tool_definitions and openrouter_cache_messages)
OpenAI (GPT-5.6 and later on the OpenAI API only; as openai_prompt_cache_options and openai_cache_instructions)

Type: CacheSetting

service_tier

The cross-provider service tier to use for the model request.

See ServiceTier for the value semantics and the per-provider mapping table. Provider-specific settings (openai_service_tier, anthropic_service_tier, bedrock_service_tier, google_cloud_service_tier) take precedence over this unified field when set.

Supported by:

OpenAI
OpenAI Codex
Anthropic
Google (Gemini API and Google Cloud)
Bedrock
Cerebras (sent and enum-validated — HTTP 200 on an ordinary key; the tiers are in private preview, so no effect is observable without access)
Crusoe
GitHub Copilot
Ollama
OpenRouter
Snowflake (sent, but Snowflake Cortex rejects service_tier with an error)
Z.AI
Bedrock Mantle

The OpenAI-derived model classes send the OpenAI value unchanged, so the OpenAI column of the mapping table applies to them.

Type: ServiceTier

extra_body

Extra body to send to the model.

Supported by:

OpenAI
OpenAI ChatGPT
OpenAI Codex
Anthropic
Groq
HuggingFace
Cerebras
Crusoe
GitHub Copilot
Ollama
OpenRouter
Snowflake
Z.AI
Bedrock Mantle
TypeSafe
System One
OpenAI Decisions

On the OpenAI-derived models that build their own extra_body (Cerebras, OpenRouter, Snowflake, Z.AI), the model’s own derived keys overwrite yours when the keys collide.

Type: object

ToolOrOutput

Restricts function tools while keeping output tools and direct text/image output available.

Use this when you want to control which function tools the model can use in an agent run while still allowing all output tools and direct text/image output. A plain list instead forces a call to one of the named function and/or output tools.

See the Tool Choice guide for examples.

Attributes

function_tools

The names of function tools available to the model.

Type: list[str]

CacheConfig

Bases: TypedDict

Detailed prompt-cache configuration, for the cache setting.

{} is the same as cache=True, and {'retention': '1h'} the same as cache='1h'.

Attributes

retention

The retention to cache with, snapped to the nearest tier the provider supports.

When omitted, the provider’s default retention is used.

Type: CacheRetention

messages

Whether to also cache the growing conversation, not only the stable prompt prefix. Default: True.

With False, only the tool definitions and static instructions are cached: useful for many short, one-off conversations that share long instructions or tools, where writing each conversation to the cache would cost more than it saves because it’s never read back. On OpenAI requests that can’t carry an instruction breakpoint, such as those continuing server-side state, the conversation is cached as well, as caching only the prefix would cache nothing.

Type: bool

ModelSettingName

The name of a ModelSettings field, as listed in a model class’s unsupported_settings.

Type: TypeAlias Default: Literal['max_tokens', 'temperature', 'top_p', 'top_k', 'timeout', 'request_timeout', 'max_retries', 'parallel_tool_calls', 'tool_choice', 'seed', 'presence_penalty', 'frequency_penalty', 'logit_bias', 'stop_sequences', 'extra_headers', 'thinking', 'citations', 'cache', 'service_tier', 'extra_body']

ToolChoice

Type alias for all valid tool_choice values.

Default: ToolChoiceScalar | list[str] | ToolOrOutput | None

ServiceTier

Cross-provider value set for ModelSettings.service_tier.

Values:

'auto': Let the provider decide — typically means “use a higher tier (scale credits, priority capacity) when available, otherwise standard.” On providers without a server-side auto concept the field is omitted so the provider’s natural default applies.
'default': Explicitly request the provider’s standard tier — opts out of any server-side auto-promotion to premium tiers.
'flex': Lower-cost, latency-tolerant tier where the provider offers one. Silently ignored on providers that don’t (e.g. Anthropic) — though a few reject the field outright rather than ignore it, as noted on the service_tier entries.
'priority': Higher-priority / lower-latency tier where the provider offers one. Silently ignored on providers that don’t.

Per-provider mapping:

value	OpenAI	Anthropic	Bedrock	Google (Gemini API)	Google Cloud
`'auto'`	`'auto'`	`'auto'`	(omitted)	(omitted)	no headers (PT then on-demand)
`'default'`	`'default'`	`'standard_only'`	`{'type': 'default'}`	`'standard'`	no headers (PT then on-demand)
`'flex'`	`'flex'`	(omitted)	`{'type': 'flex'}`	`'flex'`	header `Shared-Request-Type: flex` (PT then Flex PayGo)
`'priority'`	`'priority'`	(omitted)	`{'type': 'priority'}`	`'priority'`	header `Shared-Request-Type: priority` (PT then Priority PayGo)

On Google Cloud the unified field maps only to safe PT-with-spillover variants so customers with Provisioned Throughput keep using their reserved capacity first; to bypass PT entirely use google_cloud_service_tier with 'flex_only' or 'priority_only'. Likewise, provider-specific values not in the unified set (Bedrock’s 'reserved', Anthropic’s 'standard_only', Google Cloud’s PT routing tiers) are reachable only through the per-provider field.

Per-provider settings (openai_service_tier, anthropic_service_tier, bedrock_service_tier, google_cloud_service_tier) always take precedence over this unified field when set.

Type: TypeAlias Default: Literal['auto', 'default', 'flex', 'priority']