pydantic_ai.settings

ModelSettings

Bases: TypedDict

Settings to configure an LLM.

Includes only settings which apply to multiple models / model providers, though not all of these settings are supported by all models.

Each field’s Supported by: list names the model classes that put the setting on the wire. A bare name covers every interface that model serves, so OpenAI means both OpenAIChatModel and OpenAIResponsesModel; a name qualified with an interface, like OpenAI Chat Completions, covers only that one, because the Responses API does not accept the setting at all.

These lists are parsed and checked against the wire by tests/models/test_model_settings_support.py, so keep the * Name bullet shape and put any nuance in parentheses after the name.

Being listed means Pydantic AI sends the setting, not that the service honors it: the OpenAI-compatible model classes forward whatever the OpenAI schema accepts, and an individual provider behind one of them may ignore a field its own API doesn’t define, or reject it. Where we know of such a case it is noted on the entry, but the provider’s own API reference is the authority.

All types must be serializable using Pydantic.

Attributes

max_tokens

The maximum number of tokens to generate before stopping.

Supported by:

OpenAI
Anthropic
Google
Groq
Cohere
Mistral
Bedrock
MCP Sampling
xAI
HuggingFace
Cerebras
Crusoe
Ollama
OpenRouter
Snowflake
Z.AI
Bedrock Mantle

Type: int

temperature

Amount of randomness injected into the response.

Use temperature closer to 0.0 for analytical / multiple choice, and closer to a model’s maximum temperature for creative and generative tasks.

Note that even with temperature of 0.0, the results will not be fully deterministic.

Supported by:

OpenAI
Anthropic
Google
Groq
Cohere
Mistral
Bedrock
MCP Sampling
xAI
HuggingFace
Cerebras
Crusoe
Ollama
OpenRouter
Snowflake
Z.AI
Bedrock Mantle

Type: float

top_p

An alternative to sampling with temperature, called nucleus sampling, where the model considers the results of the tokens with top_p probability mass.

So 0.1 means only the tokens comprising the top 10% probability mass are considered.

You should either alter temperature or top_p, but not both.

Supported by:

OpenAI
Anthropic
Google
Groq
Cohere
Mistral
Bedrock
xAI
HuggingFace
Cerebras
Crusoe
Ollama
OpenRouter
Snowflake
Z.AI
Bedrock Mantle

Type: float

top_k

Only sample from the top K options for each subsequent token.

Used to remove “long tail” low probability responses.

Supported by:

Anthropic
Google
Cohere
Bedrock (Anthropic and Amazon Nova models only)

Type: int

timeout

Override the client-level default timeout for a request, in seconds.

Numeric seconds work everywhere. A legacy httpx.Timeout is also accepted and is converted to an httpx2.Timeout on the paths whose SDK expects one. httpx2.Timeout is deliberately not part of this contract, because some SDKs behind these settings still reject it.

Supported by:

OpenAI
Anthropic
Google (numeric seconds only, not httpx.Timeout)
Groq
Mistral (numeric seconds only, not httpx.Timeout)
Cerebras
Crusoe
Ollama
OpenRouter
Snowflake
Z.AI
Bedrock Mantle

Type: int | float | Timeout

parallel_tool_calls

Whether to allow parallel tool calls.

Supported by:

OpenAI (some models, not o1)
Anthropic
Groq
Mistral
xAI
Crusoe
Ollama
OpenRouter
Snowflake
Z.AI
Bedrock Mantle

Type: bool

tool_choice

Control which function tools the model can use.

See the Tool Choice guide for detailed documentation and examples.

None (default): Defaults to 'auto' behavior
'auto': All tools available, model decides whether to use them
'none': Disables function tools; model responds with text only (output tools remain for structured output)
'required': Forces tool use; excludes output tools so the agent cannot produce a final response when set statically
list[str]: Only specified tools; excludes output tools so the agent cannot produce a final response when set statically
ToolOrOutput: Specified function tools plus output tools/text/image

Note: setting 'required' or list[str] statically (via the model_settings argument of Agent.run or the agent’s own model_settings) raises a UserError, because it would force a tool call on every step and prevent the agent from producing a final response. To vary tool_choice per step (e.g. force a tool on the first step only), return a callable from a capability’s get_model_settings — those values are trusted to adapt across steps. For single API calls without an agent loop, use pydantic_ai.direct.model_request.

Supported by:

OpenAI
Anthropic ('required' and specific tools not supported with thinking enabled)
Google
Groq
Cohere (a named subset is honored by filtering the tool list, not sent as a parameter)
Mistral (a named subset is honored by filtering the tool list, not sent as a parameter)
Bedrock
xAI
HuggingFace
Cerebras
Crusoe
Ollama (sent, but Ollama documents tool_choice as unsupported)
OpenRouter
Snowflake
Z.AI
Bedrock Mantle

Type: ToolChoice

seed

The random seed to use for the model, theoretically allowing for deterministic results.

Supported by:

OpenAI Chat Completions
Google
Groq
Cohere
Mistral
xAI
HuggingFace
Cerebras
Crusoe
Ollama
OpenRouter
Snowflake
Z.AI
Bedrock Mantle Chat Completions

Type: int

presence_penalty

Penalize new tokens based on whether they have appeared in the text so far.

Supported by:

OpenAI Chat Completions
Google
Groq
Cohere
Mistral
xAI
HuggingFace
Crusoe
Ollama
OpenRouter
Snowflake
Z.AI
Bedrock Mantle Chat Completions

Type: float

frequency_penalty

Penalize new tokens based on their existing frequency in the text so far.

Supported by:

OpenAI Chat Completions
Google
Groq
Cohere
Mistral
xAI
HuggingFace
Crusoe
Ollama
OpenRouter
Snowflake
Z.AI
Bedrock Mantle Chat Completions

Type: float

logit_bias

Modify the likelihood of specified tokens appearing in the completion.

Supported by:

OpenAI Chat Completions
Groq
HuggingFace
Crusoe
Ollama (sent, but Ollama documents logit_bias as unsupported)
OpenRouter
Snowflake
Z.AI
Bedrock Mantle Chat Completions

Type: dict[str, int]

stop_sequences

Sequences that will cause the model to stop generating.

Supported by:

OpenAI Chat Completions
Anthropic
Google
Groq
Cohere
Mistral
Bedrock
MCP Sampling
xAI
HuggingFace
Cerebras
Crusoe
Ollama
OpenRouter
Snowflake
Z.AI
Bedrock Mantle Chat Completions

Type: list[str]

extra_headers

Extra headers to send to the model.

Supported by:

OpenAI
Anthropic
Google
Groq
Bedrock
Cerebras
Crusoe
Ollama
OpenRouter
Snowflake
Z.AI
Bedrock Mantle

Type: dict[str, str]

thinking

Enable or configure thinking/reasoning for the model.

True: Enable thinking with the provider’s default effort level.
False: Disable thinking (silently ignored if the model always thinks).
'minimal'/'low'/'medium'/'high'/'xhigh': Enable thinking at a specific effort level.

When omitted, the model uses its default behavior (which may include thinking for reasoning models).

Provider-specific thinking settings (e.g., anthropic_thinking, openai_reasoning_effort) take precedence over this unified field.

Listed below are the model classes that translate this field onto the request. A class whose models always reason and take no thinking parameter is not listed at all (Cohere); where only some of a class’s models are always-on it stays listed, and the per-model behavior is on the Thinking page (Mistral’s magistral).

Supported by:

OpenAI
Anthropic
Google
Groq
Mistral
Bedrock
xAI
Cerebras (only False is forwarded, as reasoning_effort='none'; the enable levels are not sent because Cerebras models reason by default, and gpt-oss ignores the disable too)
Crusoe
Ollama
OpenRouter (as extra_body['reasoning'])
Snowflake (as extra_body['reasoning'] on Claude models, otherwise as reasoning_effort)
Z.AI (as extra_body['thinking'])
Bedrock Mantle (the Responses interface only; the Chat Completions interface serves only the gpt-oss-safeguard models, which take no thinking parameter)

Type: ThinkingLevel

service_tier

The cross-provider service tier to use for the model request.

See ServiceTier for the value semantics and the per-provider mapping table. Provider-specific settings (openai_service_tier, anthropic_service_tier, bedrock_service_tier, google_cloud_service_tier) take precedence over this unified field when set.

Supported by:

OpenAI
Anthropic
Google (Gemini API and Google Cloud)
Bedrock
Crusoe
Ollama
OpenRouter
Snowflake (sent, but Snowflake Cortex rejects service_tier with an error)
Z.AI
Bedrock Mantle

The OpenAI-derived model classes send the OpenAI value unchanged, so the OpenAI column of the mapping table applies to them.

Type: ServiceTier

extra_body

Extra body to send to the model.

Supported by:

OpenAI
Anthropic
Groq
HuggingFace
Cerebras
Crusoe
Ollama
OpenRouter
Snowflake
Z.AI
Bedrock Mantle

On the OpenAI-derived models that build their own extra_body (Cerebras, OpenRouter, Snowflake, Z.AI), the model’s own derived keys overwrite yours when the keys collide.

Type: object

ToolOrOutput

Restricts function tools while keeping output tools and direct text/image output available.

Use this when you want to control which function tools the model can use in an agent run while still allowing the agent to complete with structured output, text, or images.

See the Tool Choice guide for examples.

Attributes

function_tools

The names of function tools available to the model.

Type: list[str]

ServiceTier

Cross-provider value set for ModelSettings.service_tier.

Values:

'auto': Let the provider decide — typically means “use a higher tier (scale credits, priority capacity) when available, otherwise standard.” On providers without a server-side auto concept the field is omitted so the provider’s natural default applies.
'default': Explicitly request the provider’s standard tier — opts out of any server-side auto-promotion to premium tiers.
'flex': Lower-cost, latency-tolerant tier where the provider offers one. Silently ignored on providers that don’t (e.g. Anthropic) — though a few reject the field outright rather than ignore it, as noted on the service_tier entries.
'priority': Higher-priority / lower-latency tier where the provider offers one. Silently ignored on providers that don’t.

Per-provider mapping:

value	OpenAI	Anthropic	Bedrock	Google (Gemini API)	Google Cloud
`'auto'`	`'auto'`	`'auto'`	(omitted)	(omitted)	no headers (PT then on-demand)
`'default'`	`'default'`	`'standard_only'`	`{'type': 'default'}`	`'standard'`	no headers (PT then on-demand)
`'flex'`	`'flex'`	(omitted)	`{'type': 'flex'}`	`'flex'`	header `Shared-Request-Type: flex` (PT then Flex PayGo)
`'priority'`	`'priority'`	(omitted)	`{'type': 'priority'}`	`'priority'`	header `Shared-Request-Type: priority` (PT then Priority PayGo)

On Google Cloud the unified field maps only to safe PT-with-spillover variants so customers with Provisioned Throughput keep using their reserved capacity first; to bypass PT entirely use google_cloud_service_tier with 'flex_only' or 'priority_only'. Likewise, provider-specific values not in the unified set (Bedrock’s 'reserved', Anthropic’s 'standard_only', Google Cloud’s PT routing tiers) are reachable only through the per-provider field.

Per-provider settings (openai_service_tier, anthropic_service_tier, bedrock_service_tier, google_cloud_service_tier) always take precedence over this unified field when set.

Type: TypeAlias Default: Literal['auto', 'default', 'flex', 'priority']