# pydantic\_ai.settings

### ModelSettings

**Bases:** [`TypedDict`](https://docs.python.org/3/library/typing.html#typing.TypedDict)

Settings to configure an LLM.

Includes only settings which apply to multiple models / model providers, though not all of these settings are supported by all models.

All types must be serializable using Pydantic.

#### Attributes

##### max\_tokens

The maximum number of tokens to generate before stopping.

Supported by:

-   Gemini
-   Anthropic
-   OpenAI
-   Groq
-   Cohere
-   Mistral
-   Bedrock
-   MCP Sampling
-   Outlines (all providers)
-   xAI

**Type:** [`int`](https://docs.python.org/3/library/functions.html#int)

##### temperature

Amount of randomness injected into the response.

Use `temperature` closer to `0.0` for analytical / multiple choice, and closer to a model's maximum `temperature` for creative and generative tasks.

Note that even with `temperature` of `0.0`, the results will not be fully deterministic.

Supported by:

-   Gemini
-   Anthropic
-   OpenAI
-   Groq
-   Cohere
-   Mistral
-   Bedrock
-   Outlines (Transformers, LlamaCpp, SgLang, VLLMOffline)
-   xAI

**Type:** [`float`](https://docs.python.org/3/library/functions.html#float)

##### top\_p

An alternative to sampling with temperature, called nucleus sampling, where the model considers the results of the tokens with top\_p probability mass.

So 0.1 means only the tokens comprising the top 10% probability mass are considered.

You should either alter `temperature` or `top_p`, but not both.

Supported by:

-   Gemini
-   Anthropic
-   OpenAI
-   Groq
-   Cohere
-   Mistral
-   Bedrock
-   Outlines (Transformers, LlamaCpp, SgLang, VLLMOffline)
-   xAI

**Type:** [`float`](https://docs.python.org/3/library/functions.html#float)

##### top\_k

Only sample from the top K options for each subsequent token.

Used to remove "long tail" low probability responses.

Supported by:

-   Gemini
-   Anthropic
-   Cohere

**Type:** [`int`](https://docs.python.org/3/library/functions.html#int)

##### timeout

Override the client-level default timeout for a request, in seconds.

Supported by:

-   Gemini
-   Anthropic
-   OpenAI
-   Groq
-   Mistral
-   xAI

**Type:** [`float`](https://docs.python.org/3/library/functions.html#float) | `Timeout`

##### parallel\_tool\_calls

Whether to allow parallel tool calls.

Supported by:

-   OpenAI (some models, not o1)
-   Groq
-   Anthropic
-   xAI

**Type:** [`bool`](https://docs.python.org/3/library/functions.html#bool)

##### tool\_choice

Control which function tools the model can use.

See the [Tool Choice guide](/docs/ai/tools-toolsets/tools-advanced#tool-choice) for detailed documentation and examples.

-   `None` (default): Defaults to `'auto'` behavior
-   `'auto'`: All tools available, model decides whether to use them
-   `'none'`: Disables function tools; model responds with text only (output tools remain for structured output)
-   `'required'`: Forces tool use; excludes output tools so the agent cannot produce a final response when set statically
-   `list[str]`: Only specified tools; excludes output tools so the agent cannot produce a final response when set statically
-   [`ToolOrOutput`](/docs/ai/api/pydantic-ai/settings/#pydantic_ai.settings.ToolOrOutput): Specified function tools plus output tools/text/image

Note: setting `'required'` or `list[str]` _statically_ (via the `model_settings` argument of `Agent.run` or the agent's own `model_settings`) raises a `UserError`, because it would force a tool call on every step and prevent the agent from producing a final response. To vary `tool_choice` per step (e.g. force a tool on the first step only), return a callable from a capability's [`get_model_settings`](/docs/ai/api/pydantic-ai/capabilities/#pydantic_ai.capabilities.AbstractCapability.get_model_settings) -- those values are trusted to adapt across steps. For single API calls without an agent loop, use [`pydantic_ai.direct.model_request`](/docs/ai/api/pydantic-ai/direct/#pydantic_ai.direct.model_request).

Supported by:

-   OpenAI
-   Anthropic (`'required'` and specific tools not supported with thinking enabled)
-   Google
-   Groq
-   Mistral
-   HuggingFace
-   Bedrock
-   xAI

**Type:** `ToolChoice`

##### seed

The random seed to use for the model, theoretically allowing for deterministic results.

Supported by:

-   OpenAI
-   Groq
-   Cohere
-   Mistral
-   Gemini
-   Outlines (LlamaCpp, VLLMOffline)
-   xAI

**Type:** [`int`](https://docs.python.org/3/library/functions.html#int)

##### presence\_penalty

Penalize new tokens based on whether they have appeared in the text so far.

Supported by:

-   OpenAI
-   Groq
-   Cohere
-   Gemini
-   Mistral
-   Outlines (LlamaCpp, SgLang, VLLMOffline)
-   xAI

**Type:** [`float`](https://docs.python.org/3/library/functions.html#float)

##### frequency\_penalty

Penalize new tokens based on their existing frequency in the text so far.

Supported by:

-   OpenAI
-   Groq
-   Cohere
-   Gemini
-   Mistral
-   Outlines (LlamaCpp, SgLang, VLLMOffline)
-   xAI

**Type:** [`float`](https://docs.python.org/3/library/functions.html#float)

##### logit\_bias

Modify the likelihood of specified tokens appearing in the completion.

Supported by:

-   OpenAI
-   Groq
-   Outlines (Transformers, LlamaCpp, VLLMOffline)

**Type:** [`dict`](https://docs.python.org/3/reference/expressions.html#dict)\[[`str`](https://docs.python.org/3/library/stdtypes.html#str), [`int`](https://docs.python.org/3/library/functions.html#int)\]

##### stop\_sequences

Sequences that will cause the model to stop generating.

Supported by:

-   OpenAI
-   Anthropic
-   Bedrock
-   Mistral
-   Groq
-   Cohere
-   Google
-   xAI

**Type:** [`list`](https://docs.python.org/3/glossary.html#term-list)\[[`str`](https://docs.python.org/3/library/stdtypes.html#str)\]

##### extra\_headers

Extra headers to send to the model.

Supported by:

-   OpenAI
-   Anthropic
-   Gemini
-   Groq
-   xAI

**Type:** [`dict`](https://docs.python.org/3/reference/expressions.html#dict)\[[`str`](https://docs.python.org/3/library/stdtypes.html#str), [`str`](https://docs.python.org/3/library/stdtypes.html#str)\]

##### thinking

Enable or configure thinking/reasoning for the model.

-   `True`: Enable thinking with the provider's default effort level.
-   `False`: Disable thinking (silently ignored if the model always thinks).
-   `'minimal'`/`'low'`/`'medium'`/`'high'`/`'xhigh'`: Enable thinking at a specific effort level.

When omitted, the model uses its default behavior (which may include thinking for reasoning models).

Provider-specific thinking settings (e.g., `anthropic_thinking`, `openai_reasoning_effort`) take precedence over this unified field.

Supported by:

-   Anthropic
-   OpenAI
-   Gemini
-   Groq
-   Bedrock
-   OpenRouter
-   Cerebras
-   xAI

**Type:** `ThinkingLevel`

##### service\_tier

The cross-provider service tier to use for the model request.

See `ServiceTier` for the value semantics and the per-provider mapping table. Provider-specific settings (`openai_service_tier`, `anthropic_service_tier`, `bedrock_service_tier`, `google_cloud_service_tier`) take precedence over this unified field when set.

Supported by:

-   OpenAI
-   Anthropic
-   Bedrock
-   Google (Gemini API and Google Cloud)

**Type:** `ServiceTier`

##### extra\_body

Extra body to send to the model.

Supported by:

-   OpenAI
-   Anthropic
-   Groq
-   Outlines (all providers)

**Type:** [`object`](https://docs.python.org/3/glossary.html#term-object)

### ToolOrOutput

Restricts function tools while keeping output tools and direct text/image output available.

Use this when you want to control which function tools the model can use in an agent run while still allowing the agent to complete with structured output, text, or images.

See the [Tool Choice guide](/docs/ai/tools-toolsets/tools-advanced#tool-choice) for examples.

#### Attributes

##### function\_tools

The names of function tools available to the model.

**Type:** [`list`](https://docs.python.org/3/glossary.html#term-list)\[[`str`](https://docs.python.org/3/library/stdtypes.html#str)\]