pydantic_ai.models
Logic related to making requests to an LLM.
The aim here is to make a common interface for different LLMs, so that the rest of the code can be agnostic to the specific LLM being used.
Configuration for an agent’s request to a model, specifically related to tools and output handling.
Maps each function tool name to its resolved ToolVisibility.
None on authored parameters; Model.prepare_request
populates an entry for every function tool, so a resolved request always carries a dict — empty
exactly when there are no function tools. Output tools never get entries because they are always
plain tools entries; visibility_of
treats their absent entries like 'visible'. The no-defaults repr omits the field until
resolution, so authored parameters print as authored and resolved state stays visible.
Type: dict[str, ToolVisibility] | None Default: None
Names history has revealed so far, derived from the outgoing message list before each request.
Discovered means evidenced by history; revealed means represented on this request’s wire state.
Input to visibility resolution: ToolDefinition.defer_loading records what the author asked
for and stays set after a reveal, so this answers the separate question of what the model can
see now. History can name tools that no longer exist in the current run’s definitions; those
are dropped where this is derived, so it is a subset of function_tools’ names by construction.
Type: set[str] Default: field(default_factory=(set[str]), repr=False)
Immediately-available function tools that joined the run’s tool population after its first request.
tools is the first cache section, so a tool that appears mid-run (added to a FunctionToolset
from inside a tool, returned by a dynamic toolset, listed by an MCP server after
tools/list_changed, let through by prepare_tools) is not added to it. It travels like a
revealed deferred tool instead: the run records a
ToolAvailabilityDeltaPart naming it, and visibility
resolution puts it on the provider’s tool-addition channel once that delta is in the outgoing
history. Without the delta it stays an ordinary visible tools entry, because it is callable either way.
This changes only how the definition reaches the model. ToolDefinition.defer_loading stays as
authored, and whether the tool can be called is decided exactly as before.
Type: set[str] Default: field(default_factory=(set[str]), repr=False)
IDs of the run’s capabilities that defer their loading.
Read from the capability instances themselves, so it means what it says. It cannot be derived
from the function tools: ToolDefinition.capability_id records which capability contributed a
tool, and defer_loading is set both by a deferred capability and by a search-gated tool inside
an always-on one — so the two cases are indistinguishable from the definitions alone.
Used to answer “may this tool be revealed yet?”: a tool whose capability_id is in this set is
gated on that capability being loaded, while one whose owner is absent here is gated only on its
own discovery.
Type: set[str] Default: field(default_factory=(set[str]), repr=False)
Structured instruction parts with metadata about their origin (static vs dynamic).
Static instructions (dynamic=False) come from literal strings passed to Agent(instructions=...).
Dynamic instructions (dynamic=True) come from @agent.instructions functions, TemplateStr,
or toolset get_instructions() methods.
Models that support granular caching (e.g. Anthropic, Bedrock) use this to place cache boundaries at the static/dynamic instruction boundary.
Type: list[InstructionPart] | None Default: None
Resolved thinking/reasoning configuration for this request.
None means the model should use its default behavior. Set by the base
Model.prepare_request() from the unified thinking field in ModelSettings,
after checking that the model’s profile supports thinking.
Type: ThinkingLevel | None Default: None
Resolved unified prompt-caching configuration for this request.
Set by the base Model.prepare_request() from the unified cache field in ModelSettings,
after checking that the model’s profile supports caching configuration
(supports_cache) and snapping a retention
to a supported tier. None means the setting is unset or the model’s profile doesn’t support caching
configuration.
Like thinking, this is the requested
unified value: provider-specific cache settings take precedence in the model’s request mapping
without changing it.
Type: CacheSetting | None Default: None
Definitions represented in the provider’s ordinary tools collection.
The visibility filter applies to function tools only: output tools are always plain
tools entries, so they are included unconditionally rather than keyed through a
name-indexed filter a hidden function tool could shadow.
Type: dict[str, ToolDefinition]
Function tools represented in the provider’s ordinary tools collection.
Type: list[ToolDefinition]
def visibility_of(tool_name: str) -> ToolVisibility
The resolved ToolVisibility for tool_name.
For parameters constructed directly rather than resolved by Model.prepare_request,
deferred function tools default to 'withheld' and every other name defaults to 'visible'.
ToolVisibility
def with_default_output_mode(
output_mode: StructuredOutputMode,
) -> ModelRequestParameters
Set the default output mode if the current mode is ‘auto’, atomically updating allow_text_output.
No-op if the current output_mode is not ‘auto’. This ensures the two fields stay in sync — output_mode=‘tool’ implies allow_text_output=False, while ‘native’ and ‘prompted’ imply allow_text_output=True.
ModelRequestParameters
Bases: ABC
Shared identity for request-response and realtime models.
The model name.
Type: str
The model provider, ex: openai.
Use to populate the gen_ai.system OpenTelemetry semantic convention attribute,
so should use well-known values listed in
https://opentelemetry.io/docs/specs/semconv/attributes-registry/gen-ai/#gen-ai-system
when applicable.
Type: str
The base URL for the provider API, if available.
The fully qualified model name in 'provider:model_name' format.
Type: str
The maximum number of tokens the model can handle at once, input and output combined, or None when unknown.
Models with a profile read it from the profile’s context_window, which is filled from
genai-prices when no profile layer sets it. Wrapper
models report their wrapped model’s; a FallbackModel
reports the smallest among its candidates.
Human-friendly display label for the model.
Handles common patterns:
- gpt-5 -> GPT 5
- claude-sonnet-4-5 -> Claude Sonnet 4.5
- gemini-2.5-pro -> Gemini 2.5 Pro
- meta-llama/llama-3-70b -> Llama 3 70b (OpenRouter style)
Type: str
@async
def __aenter__() -> Self
Enter the model context.
@async
def __aexit__(
exc_type: type[BaseException] | None,
exc_val: BaseException | None,
exc_tb: TracebackType | None,
) -> bool | None
Exit the model context.
Context for model request hooks.
Wrapping these parameters in a dataclass instead of a tuple makes the signature future-proof: new fields can be added without breaking existing implementations.
A before_model_request hook
returns a context, so modifying one is normally dataclasses.replace(request_context, ...). Every
field is therefore settable through replace(), including model_id and streaming: the agent
graph sets those two immediately before calling the hook, and replace() re-initializes any
init=False field to its default, so declaring them that way silently zeroed both for any hook
that copied its context — costing a streamed run its streaming flag and a durable-execution
worker the selection token it re-resolves an aliased model from. They are still set by the graph
rather than by a caller; passing either to a fresh ModelRequestContext is not meaningful.
Messages to send for this model request.
This is an independently owned shallow list, so top-level mutations and assignments change
only the current request. To rewrite the persistent
message history, update RunContext.messages
instead, or update both explicitly when both effects are intended.
To add a message the model sees now and that stays in the conversation, such as a steering message
or a reminder, call RunContext.enqueue from
before_model_request: it goes into this request and into the message history. See
Changing what the model sees.
Messages a before_model_request hook appends here with append, extend or += are also
added to the message history, but this is deprecated and emits a warning.
This is an independent top-level list, not an independent object graph. Retained messages and their parts may be the same objects as those in persistent history. Filtering, reordering, or assigning the outer sequence is request-only, but mutating a contained message or part in place can affect persistent history. Construct replacements down to the level being changed when isolation is required.
Type: list[ModelMessage]
The tool, output, and instruction configuration for this request.
instruction_parts is the source of
truth for the instructions in the agent flow: a
before_model_request hook that
rewrites them changes what the model receives, and the ModelRequest
recorded in message history is re-rendered from them afterwards, so history and traces keep showing
what was sent. Assigning to that message’s instructions instead is not propagated back into the
parts, and so does not reach the model.
Type: ModelRequestParameters
The model-name string this request’s model was selected/resolved from, if any.
This is the selection token — e.g. 'openai:gpt-5.6-sol', or an alias like 'tenant-x' that a
resolve_model_id capability
turned into a concrete model — so it can differ from the resolved model’s own
model_id. None when the model was supplied as an
instance rather than resolved from a string.
Durable-execution capabilities carry this across the activity/step/task boundary in preference
to the resolved model’s own model_id, so an aliased model round-trips as the original string
the worker-side resolution chain can re-resolve. Only meaningful while model is still the run’s
resolved model — a model swapped in by a hook invalidates it.
Type: str | None Default: None
Whether the agent loop expects to iterate the model response as a stream.
Set for streamed runs — run_stream(), run_stream_events(), iter()’s node streaming — and
for run() when an event_stream_handler is set or a capability overrides
wrap_run_event_stream (e.g. ProcessEventStream, or a durability capability’s
event_stream_handler=). There is no separate before_model_request_stream hook — streaming
and non-streaming requests share the same hooks — so this field is how a hook can tell them
apart. Read-only from hooks: reassigning it doesn’t change how the loop consumes the response.
Type: bool Default: False
Bases: Generic[ModelContextDepsT]
Context used to resolve a model ID before a model is available.
This is narrower than RunContext because model
resolution happens before a run context can contain its resolved model.
The agent whose model is being resolved.
Type: AbstractAgent[ModelContextDepsT, Any]
The dependencies supplied for this run.
Type: ModelContextDepsT
Bases: ModelResolutionContext[ModelContextDepsT]
Context used by a capability to select the model for a request step.
The lower-precedence model on the first step, then the model used for the previous step.
Type: Model | None
The request step being selected, starting at 1.
Type: int
The run’s user prompt, as RunContext.prompt holds it.
When a run resumes from a history ending in a request, without a new prompt, this is that request’s prompt.
Type: str | Sequence[UserContent] | None Default: None
The messages the selected model will be sent for this step, ending with the request being routed.
This is what RunContext.messages holds for the step, minus
what is only added once the model is selected: the request’s
instructions and, on a fresh run’s first step,
its system prompt parts. Earlier requests keep the instructions they were sent with.
When a run resumes from a response with tool calls still to run, the step’s request is their results, which don’t exist before the model is selected, so the messages end with that response. They also end with the response when it’s a suspended one being continued, as no request is sent.
It’s a new list, so adding or removing messages doesn’t change the run’s. Don’t change the messages in it: they’re the run’s own, except for the request being routed on a run’s first step, which is built for selection, so changing it has no effect on what’s sent.
Type: list[ModelMessage]
Usage accumulated by the run before this request step.
Type: RunUsage
The number of output retries the run has used before this request step.
This is what output validators see as RunContext.retry. It goes up
each time the run asks the model to try again for its output: an output tool’s arguments failed validation,
an output function or validator raised ModelRetry, the response had
no usable output, or a capability’s model request hook raised ModelRetry. Steps in between, such as
function tool calls, don’t reset it, and function tool retries don’t count towards it.
Type: int Default: 0
The run’s output retry budget, as output validators see it in RunContext.max_retries.
An output tool with its own budget, set with ToolOutput(max_retries=...),
is limited by that budget instead, while retry still
counts the retries of every output.
Type: int Default: 0
The UsageLimits enforced for this run, as
RunContext.usage_limits holds them.
During a run this is always set: if no limits were passed, the run enforces the default
UsageLimits(). It is only None when a model is selected outside a
run, as by system_prompt_parts(). Treat it as
read-only: it is the live object the run enforces against.
Type: UsageLimits | None Default: None
The time (timezone-aware, in UTC) by which this run must finish, or None if it has no deadline,
as RunContext.deadline holds it.
Set from timeout= on the agent run methods, or inherited from an enclosing run. The run’s clock
starts before its first model is selected, so the time a selector takes counts toward it.
Type: datetime | None Default: None
Unique identifier for the agent run, as RunContext.run_id holds it.
The run_id argument to Agent.run (etc.), or a fresh UUID7. Inside a durable workflow or flow,
the default is derived from the workflow or flow run instead, so it stays the same when the engine
re-executes the run. None when a model is selected outside a run.
Type: str | None Default: None
Unique identifier for the conversation this run belongs to, as
RunContext.conversation_id holds it.
None when a model is selected outside a run.
Type: str | None Default: None
Bases: AbstractModel, Generic[InterfaceClient]
Abstract class for a model.
tool_deferral_mode values this adapter’s renderer implements.
A profile may claim a mode for the model family, but the claim only takes effect when the
adapter class declares it here: Model.tool_deferral_mode intersects the two, so a Model
subclass that declares nothing (the default) never resolves tools to a wire shape it cannot
render, no matter what a pass-through vendor profile claims.
Type: frozenset[ToolDeferralMode] Default: frozenset()
tool_addition_mode values this adapter’s renderer implements. See supported_tool_deferral_modes.
Type: frozenset[ToolAdditionMode] Default: frozenset()
Whether this adapter’s API only honors a CompactionPart
that carries encrypted content.
When set, a part without it isn’t a wire boundary: the adapter would omit it, so letting it hide the earlier history would send nothing in its place.
Declared by the adapter rather than the model profile: how an API carries compaction state is a
property of the API, not of the model behind it — the same model reached through OpenAI’s Chat
Completions and Responses APIs answers differently, and eight providers route a profile of their
own through OpenAIResponsesModel. Independent of compaction_retains_standing_prompt, which
today’s two adapters happen to answer the same way.
Type: bool Default: False
Whether this adapter’s compaction item keeps serving the leading system items of the window it replaced.
When set, re-sending the standing prompt after the boundary would duplicate it. When not (the
default), the standing prompt travels in a per-request channel rebuilt from those items, so the
trim has to re-insert them or it is silently dropped from every subsequent request. See
compaction_requires_encrypted_content for why this is declared here and not on the profile.
Type: bool Default: False
The ModelSettings fields this model class ignores.
When a request sets one of them that changes what the request guarantees, like timeout or
stop_sequences, prepare_request emits an
UnsupportedSettingWarning; the tuning knobs among them,
like temperature, are ignored silently. Empty by default, so a model class that declares nothing never warns.
This covers the fields a model class sends to its provider. request_timeout is enforced by Pydantic AI for
every model, and thinking, cache and tool_choice are resolved through the model profile, so they’re not
listed here.
It’s declared on the model class because whether a setting reaches the provider is mostly decided by the API and
SDK the class speaks. The OpenAI model classes add the ignored settings their profile lists in
openai_unsupported_model_settings.
Type: frozenset[ModelSettingName] Default: frozenset()
The provider for this model, if any.
Type: Provider[InterfaceClient] | None
Get the model settings.
Type: ModelSettings | None
The effective schema-deferral mode: the profile’s claim, if this adapter renders it.
Type: ToolDeferralMode | None
The effective tool-addition mode: the profile’s claim, if this adapter renders it.
Type: ToolAdditionMode | None
The resolved profile’s context_window.
The model profile.
Resolution order (later layers override earlier ones):
DEFAULT_PROFILE— base values for every key inModelProfile.- The provider’s
model_profile(model_name)result — provider-specific defaults for this model — or what_provider_model_profile()returns in its place. - A best-effort
context_windowvalue from genai-prices, unless the provider or a partial user profile explicitly set the field (including toNone). 3b. Defaults for this model instance from_instance_profile_defaults(), such as narrowing that depends on the provider’s SDK client. Unlike narrowing in a subclass’sprofileoverride, which runs after the user’s layer, the user’sprofile=can still override these. - The user’s
profile=argument — partial dict merged on top, OR a callable(default) -> profilefor full control.
After resolution we compute the intersection of the profile’s supported_native_tools
and the model class’s implemented tools, ensuring model.profile['supported_native_tools']
is the single source of truth for what’s actually usable. The file-input flags are narrowed the
same way, to the kinds of file the model class’s message mapping can send.
Type: ModelProfile
def __init__(
*,
settings: ModelSettings | None = None,
profile: ModelProfileSpec | None = None,
) -> None
Initialize the model with optional settings and profile.
settings : ModelSettings | None Default: None
Model-specific settings that will be used as defaults for this model.
profile : ModelProfileSpec | None Default: None
The model profile to use.
@async
def __aenter__() -> Self
Enter the model context, delegating to the provider to manage its HTTP client lifecycle.
@async
def __aexit__(
exc_type: type[BaseException] | None,
exc_val: BaseException | None,
exc_tb: TracebackType | None,
) -> bool | None
Exit the model context, closing the provider’s HTTP client if it owns one.
def __eq__(other: object) -> bool
Compare settings, profile, and the subclass’s dataclass fields; a subclass that hashes by identity compares by identity.
def resolve_cache_retention(model_settings: ModelSettings | None) -> timedelta | None
Resolve prompt cache retention requested by model settings.
The model’s default settings are merged with the per-request model_settings. Both the unified
cache setting and provider-specific
settings are considered, as far as they change what’s sent to the provider. If multiple active settings
request different retention periods, the longest period wins because any longer-lived cache breakpoint can
keep the corresponding prompt prefix available. Models without an active retention setting return None,
in which case the provider’s
default_cache_retention applies.
timedelta | None
@deprecated
def resolve_prompt_cache_retention(
model_settings: ModelSettings | None,
) -> timedelta | None
Deprecated alias of resolve_cache_retention.
timedelta | None
@abstractmethod
@async
def request(
messages: list[ModelMessage],
model_settings: ModelSettings | None,
model_request_parameters: ModelRequestParameters,
) -> ModelResponse
Make a request to the model.
This is ultimately called by pydantic_ai._agent_graph.ModelRequestNode._make_request(...).
@async
def count_tokens(
messages: list[ModelMessage],
model_settings: ModelSettings | None,
model_request_parameters: ModelRequestParameters,
) -> RequestUsage
Make a request to the model for counting tokens.
@async
def compact_messages(
request_context: ModelRequestContext,
*,
instructions: str | None = None,
) -> ModelResponse
Compact messages to reduce conversation context size.
This method is optional and only supported by specific providers (e.g. OpenAI Responses API). Providers that support compaction override this method with their implementation.
@async
def request_stream(
messages: list[ModelMessage],
model_settings: ModelSettings | None,
model_request_parameters: ModelRequestParameters,
run_context: RunContext[Any] | None = None,
) -> AsyncGenerator[StreamedResponse]
Make a request to the model and return a streaming response.
AsyncGenerator[StreamedResponse]
@async
def cancel_suspended_response(response: ModelResponse) -> None
Cancel a server-side suspended/background response (e.g. an OpenAI background job).
Called when a continuation is abandoned via cancellation or error. No-op by default; model classes with cancellable server-side jobs override this.
def continuation_delay(response: ModelResponse) -> float | None
Seconds to wait before continuing a suspended response, or None to continue immediately.
Called between the segments of a suspended turn. None by default (e.g. Anthropic pause_turn
continues immediately); a model that polls a server-side job (e.g. OpenAI background mode)
overrides this to return a poll interval so the graph doesn’t busy-poll.
def customize_request_parameters(
model_request_parameters: ModelRequestParameters,
) -> ModelRequestParameters
Customize the request parameters for the model.
This method can be overridden by subclasses to modify the request parameters before sending them to the model. In particular, this method can be used to make modifications to the generated tool JSON schemas if necessary for vendor/model-specific reasons.
ModelRequestParameters
def prepare_request(
model_settings: ModelSettings | None,
model_request_parameters: ModelRequestParameters,
) -> tuple[ModelSettings | None, ModelRequestParameters]
Prepare request inputs before they are passed to the provider.
This merges the given model_settings with the model’s own settings attribute and ensures
customize_request_parameters is applied to the resolved
ModelRequestParameters. Subclasses can override this method if
they need to customize the preparation flow further, but most implementations should simply call
self.prepare_request(...) at the start of their request (and related) methods.
tuple[ModelSettings | None, ModelRequestParameters]
def prepare_messages(
messages: list[ModelMessage],
model_request_parameters: ModelRequestParameters | None = None,
) -> list[ModelMessage]
Pre-process the message history before it’s handed to the adapter’s message-prep step.
Translates typed NativeToolSearch*Part instances carried over from a
different provider (e.g. Anthropic to OpenAI Responses), or any native
provider when the active model doesn’t support ToolSearchTool, into the
local-shape ToolSearch*Part instances. This splits the single
ModelResponse(call+return) carrying the inline server-side result into
ModelResponse(call) + ModelRequest(return) so the adapter can render the
provider-agnostic exchange.
Translates each retry part into the part that carries it on the wire, by its retry.cause: a
RetryFeedbackPart for a 'validation_error' or 'model_retry' becomes a UserPromptPart
fenced in <validation_errors> or <retry_feedback> behind an application attribution line,
since either can quote the model’s own output, and only 'no_output', whose text is the
framework’s own, becomes a mid-conversation SystemPromptPart. A legacy RetryPromptPart becomes a ToolReturnPart with outcome='retried'
when it names a tool, and the RetryFeedbackPart it always meant when it doesn’t.
Renders InstructionDeltaParts as
SystemPromptParts for this model’s delivery path, dropping deltas from before the latest
instruction baseline or compaction.
Where the profile’s supports_request_scoped_system_prompts is False, leaves out every
request-scoped SystemPromptPart whose request has been answered
and moves the current request’s ones to the end of the last request.
Also wraps non-leading SystemPromptParts as <system>-tagged UserPromptParts when
the profile’s supports_inline_system_prompts is False, converts
SpeechParts from realtime session history into UserPromptParts /
TextParts that any model can consume, replaces each file a tool returned that the
profile’s file-input flags say the model can’t read with a note naming it, and puts the
tool results in each request in the order of the calls they answer, so the request bytes
(and the provider’s prompt cache) don’t depend on the order a history happened to store
them in. When the profile sets tool_call_id_length, tool call IDs that don’t fit it are
replaced with stable hashes that do.
Subclasses normally don’t need to override this; the framework calls it on the
agent’s behalf in _agent_graph._make_request so per-adapter message-prep code
sees a homogeneous shape regardless of which provider produced the prior turn.
messages : list[ModelMessage]
The history to pre-process.
model_request_parameters : ModelRequestParameters | None Default: None
The parameters this history will be sent with. Optional, and
only needed to render a ToolAvailabilityDeltaPart on a model with no native way to
express one: whether that reveal has to be a mechanism or can just be a statement
depends on whether any tool actually goes on the wire with its schema withheld, which
the profile alone can’t answer. Omitting it falls back to the adapter’s effective mode,
which differs only for a corpus mixing capability-gated and standalone deferred tools.
Framework callers pass it.
@classmethod
def supported_native_tools(cls) -> frozenset[type[AbstractNativeTool]]
Return the set of native tool types this model class can handle.
Subclasses should override this to reflect their actual capabilities. Default is empty set - subclasses must explicitly declare support.
frozenset[type[AbstractNativeTool]]
def unsupported_native_tool_settings(native_tool: AbstractNativeTool) -> frozenset[str]
Return the names of the settings set on native_tool that this model can’t apply.
Only restricting_settings count:
a native tool with any of them is treated as unsupported for the request, so its local fallback is used
when one is provided, it is left out when it is optional, and
UserError is raised otherwise.
Subclasses that send a native tool without one of its restricting settings should override this. The default is an empty set, meaning every restricting setting is applied.
Bases: ABC
Streamed response from an LLM when calling a tool.
Lifecycle state of the response.
Type: ModelResponseState Default: field(default='complete', init=False)
Earlier attempts at this request that failed before this stream was opened, see ModelResponse.failed_attempts.
Type: list[ModelRequestAttempt] | None Default: field(default=None, init=False)
Get the usage of the response so far. This will not be the final usage until the stream is exhausted.
Type: RequestUsage
Get the model name of the response.
Type: str
Get the provider name.
Get the provider base URL.
Get the timestamp of the response.
Type: datetime
Whether the stream has been cancelled via cancel().
Type: bool
def __aiter__() -> AsyncIterator[ModelResponseStreamEvent]
Stream the response as an async iterable of ModelResponseStreamEvents.
This proxies the _event_iterator() and emits all events, while also checking for matches
on the result schema and emitting a FinalResultEvent if/when the
first match is found.
AsyncIterator[ModelResponseStreamEvent]
@async
def cancel() -> None
Cancel local stream consumption and request provider shutdown.
Sets self._cancelled = True before delegating to close_stream()
so the flag is visible to any iterator that observes the transport error
raised when the underlying connection is torn down, even if
close_stream() itself raises.
def get_stream_cancel_errors() -> tuple[type[BaseException], ...]
Return transport errors caused by cancel() tearing down the stream.
The default covers model classes whose SDKs iterate HTTP responses
directly (Anthropic, OpenAI, Groq, Mistral, Google GenAI, and HuggingFace),
since they let bare httpx2 (or legacy httpx) errors propagate from
chunk reads. Model classes that use other transports (for example gRPC or
botocore) should override this method.
tuple[type[BaseException], …]
@async
def close_stream() -> None
Close the provider stream and any exposed HTTP or gRPC transport.
Model classes must override this to close the local stream and, where the
provider SDK exposes one, its transport. Integrations that cannot support
local cancellation should leave the default implementation so cancel()
fails clearly.
def get() -> ModelResponse
Build a ModelResponse from the data received from the stream so far.
def time_to_first_chunk(request_start: float) -> float | None
Seconds from request_start to the first chunk surfaced to the consumer, or None if nothing was yielded.
request_start must be a time.perf_counter() reading taken when the request was issued.
The first-chunk instant is stamped on the first async for pull, so the result reflects when
the consumer received the first event: it includes any consumer-side iteration delay
(debouncing, batching, or awaiting other work) on top of the chunk’s transit time, which for
eager consumers is negligible.
@cached
def known_model_names() -> tuple[str, ...]
Return every model name known to KnownModelName.
This is the public, stable way to enumerate the known model ids. Prefer it over introspecting
the KnownModelName type alias directly (e.g. get_args(KnownModelName.__value__)), which is
not part of the public API and would break if the alias were ever recomposed.
def check_allow_model_requests() -> None
Check if model requests are allowed.
If you’re defining your own models that have costs or latency associated with their use, you should call this at the
top of each method that sends a request to the provider: Model.request,
Model.request_stream,
Model.count_tokens,
Model.compact_messages,
EmbeddingModel.embed,
EmbeddingModel.count_tokens and
ImageGenerationModel.generate.
Methods that produce their result locally don’t need it — for example
OpenAIEmbeddingModel’s count_tokens, which tokenizes with
tiktoken and never calls the provider. Neither does
Model.cancel_suspended_response, which deliberately omits it
so an already-started job can still be cancelled after the flag is flipped.
RuntimeError— If model requests are not allowed.
def infer_model(
model: Model | KnownModelName | str,
provider_factory: Callable[[str], Provider[Any]] = infer_provider,
) -> Model
Infer the model from the name.
Model
model : Model | KnownModelName | str
Model name to instantiate, in the format of provider:model. Use the string “test” to instantiate TestModel.
Function that instantiates a provider object. The provider name is passed into the function parameter. Defaults to provider.infer_provider.
@async
def download_item(
item: FileUrl,
data_format: Literal['bytes'],
type_format: Literal['mime', 'extension'] = 'mime',
) -> DownloadedItem[bytes]
def download_item(
item: FileUrl,
data_format: Literal['base64', 'base64_uri', 'text'],
type_format: Literal['mime', 'extension'] = 'mime',
) -> DownloadedItem[str]
Download an item by URL and return the content as a bytes object or a (base64-encoded) string.
This function includes SSRF (Server-Side Request Forgery) protection:
- Only http:// and https:// protocols are allowed
- Private/internal IP addresses are blocked by default
- Cloud metadata endpoints (169.254.169.254) are always blocked
- Hostnames are resolved before requests to prevent DNS rebinding
- Response bodies are limited to 50 MiB
Set item.force_download='allow-local' to allow private IP addresses.
DownloadedItem[str] | DownloadedItem[bytes]
item : FileUrl
The item to download.
data_format : Literal[‘bytes’, ‘base64’, ‘base64_uri’, ‘text’] Default: 'bytes'
The format to return the content in:
bytes: The raw bytes of the content.base64: The base64-encoded content.base64_uri: The base64-encoded content as a data URI.text: The content as a string.
type_format : Literal[‘mime’, ‘extension’] Default: 'mime'
The format to return the media type in:
mime: The media type as a MIME type.extension: The media type as an extension.
UserError— If the URL points to a YouTube video.ValueError— If the URL uses an unsupported protocol or targets a private/internal IP address (unless allow-local is set), or the body exceeds 50 MiB.
def create_async_httpx2_client(
*,
timeout: float | httpx2.Timeout = httpx2.Timeout(DEFAULT_HTTP_TIMEOUT, connect=5),
limits: httpx2.Limits = httpx2.Limits(max_connections=DEFAULT_MAX_CONNECTIONS, max_keepalive_connections=DEFAULT_MAX_KEEPALIVE_CONNECTIONS),
) -> httpx2.AsyncClient
Create an httpx2.AsyncClient with Pydantic AI’s default timeouts, connection limits and user agent.
This is the client a provider creates when you don’t pass your own http_client; the Google
providers create it with a 20-minute timeout instead. Call it yourself to adjust the timeouts or
connection pool limits, and pass the result to the provider as http_client. A client you create
this way is yours to close.
A request timeout whose phases are all equal, such as a number of seconds, can shorten the
client’s connect and pool timeouts but never lengthen them; see
ModelSettings.timeout.
httpx2.AsyncClient
timeout : float | httpx2.Timeout Default: httpx2.Timeout(DEFAULT_HTTP_TIMEOUT, connect=5)
The client’s timeout, in seconds or as an httpx2.Timeout that sets each phase
separately. Defaults to 600 seconds, with a 5-second connect timeout.
limits : httpx2.Limits Default: httpx2.Limits(max_connections=DEFAULT_MAX_CONNECTIONS, max_keepalive_connections=DEFAULT_MAX_KEEPALIVE_CONNECTIONS)
The connection pool limits. Defaults to 1000 connections, of which up to 100 are kept alive while idle, matching the OpenAI and Anthropic SDKs’ own clients.
def override_allow_model_requests(allow_model_requests: bool) -> Generator[None]
Context manager to temporarily override ALLOW_MODEL_REQUESTS.
allow_model_requests : bool
Whether to allow model requests within the context.
Known model names that can be used with the model parameter of Agent.
KnownModelName is provided as a concise way to specify a model.
Default: TypeAliasType('KnownModelName', Literal['anthropic:claude-fable-5', 'anthropic:claude-fable-5-1', 'anthropic:claude-haiku-4-5', 'anthropic:claude-haiku-4-5-20251001', 'anthropic:claude-haiku-5-5', 'anthropic:claude-mythos-5', 'anthropic:claude-mythos-5-1', 'anthropic:claude-mythos-preview', 'anthropic:claude-opus-4-5', 'anthropic:claude-opus-4-5-20251101', 'anthropic:claude-opus-4-6', 'anthropic:claude-opus-4-7', 'anthropic:claude-opus-4-8', 'anthropic:claude-opus-5', 'anthropic:claude-opus-5-5', 'anthropic:claude-sonnet-4-5', 'anthropic:claude-sonnet-4-5-20250929', 'anthropic:claude-sonnet-4-6', 'anthropic:claude-sonnet-5', 'anthropic:claude-sonnet-5-5', 'bedrock-mantle:openai.gpt-5.4', 'bedrock-mantle:openai.gpt-5.4-2026-03-05', 'bedrock-mantle:openai.gpt-5.5', 'bedrock-mantle:openai.gpt-5.5-2026-04-23', 'bedrock-mantle:openai.gpt-5.6-luna', 'bedrock-mantle:openai.gpt-5.6-sol', 'bedrock-mantle:openai.gpt-5.6-terra', 'bedrock-mantle:openai.gpt-oss-120b', 'bedrock-mantle:openai.gpt-oss-20b', 'bedrock-mantle:openai.gpt-oss-safeguard-120b', 'bedrock-mantle:openai.gpt-oss-safeguard-20b', 'bedrock:amazon.titan-text-express-v1', 'bedrock:amazon.titan-text-lite-v1', 'bedrock:amazon.titan-tg1-large', 'bedrock:anthropic.claude-3-5-haiku-20241022-v1:0', 'bedrock:anthropic.claude-3-5-sonnet-20240620-v1:0', 'bedrock:anthropic.claude-3-5-sonnet-20241022-v2:0', 'bedrock:anthropic.claude-3-7-sonnet-20250219-v1:0', 'bedrock:anthropic.claude-3-haiku-20240307-v1:0', 'bedrock:anthropic.claude-3-opus-20240229-v1:0', 'bedrock:anthropic.claude-3-sonnet-20240229-v1:0', 'bedrock:anthropic.claude-haiku-4-5-20251001-v1:0', 'bedrock:anthropic.claude-haiku-5-5', 'bedrock:anthropic.claude-instant-v1', 'bedrock:anthropic.claude-opus-4-20250514-v1:0', 'bedrock:anthropic.claude-sonnet-4-20250514-v1:0', 'bedrock:anthropic.claude-sonnet-4-5-20250929-v1:0', 'bedrock:anthropic.claude-sonnet-4-6', 'bedrock:anthropic.claude-v2', 'bedrock:anthropic.claude-v2:1', 'bedrock:au.anthropic.claude-haiku-5-5', 'bedrock:cohere.command-light-text-v14', 'bedrock:cohere.command-r-plus-v1:0', 'bedrock:cohere.command-r-v1:0', 'bedrock:cohere.command-text-v14', 'bedrock:deepseek.r1-v1:0', 'bedrock:deepseek.v3.2', 'bedrock:eu.anthropic.claude-haiku-4-5-20251001-v1:0', 'bedrock:eu.anthropic.claude-haiku-5-5', 'bedrock:eu.anthropic.claude-sonnet-4-20250514-v1:0', 'bedrock:eu.anthropic.claude-sonnet-4-5-20250929-v1:0', 'bedrock:eu.anthropic.claude-sonnet-4-6', 'bedrock:global.amazon.nova-2-lite-v1:0', 'bedrock:global.anthropic.claude-fable-5', 'bedrock:global.anthropic.claude-fable-5-1', 'bedrock:global.anthropic.claude-haiku-5-5', 'bedrock:global.anthropic.claude-opus-4-5-20251101-v1:0', 'bedrock:global.anthropic.claude-opus-4-6-v1', 'bedrock:global.anthropic.claude-opus-4-7', 'bedrock:global.anthropic.claude-opus-4-8', 'bedrock:global.anthropic.claude-opus-5', 'bedrock:global.anthropic.claude-opus-5-5', 'bedrock:global.anthropic.claude-sonnet-5', 'bedrock:global.anthropic.claude-sonnet-5-5', 'bedrock:global.openai.gpt-5.6-luna', 'bedrock:global.openai.gpt-5.6-sol', 'bedrock:global.openai.gpt-5.6-terra', 'bedrock:global.xai.grok-4.6', 'bedrock:global.xai.grok-4.7', 'bedrock:google.gemma-3-12b-it', 'bedrock:google.gemma-3-27b-it', 'bedrock:google.gemma-3-4b-it', 'bedrock:in.openai.gpt-5.6-luna', 'bedrock:in.openai.gpt-5.6-terra', 'bedrock:jp.anthropic.claude-haiku-5-5', 'bedrock:meta.llama3-1-405b-instruct-v1:0', 'bedrock:meta.llama3-1-70b-instruct-v1:0', 'bedrock:meta.llama3-1-8b-instruct-v1:0', 'bedrock:meta.llama3-70b-instruct-v1:0', 'bedrock:meta.llama3-8b-instruct-v1:0', 'bedrock:minimax.minimax-m2', 'bedrock:minimax.minimax-m2.1', 'bedrock:minimax.minimax-m2.5', 'bedrock:mistral.devstral-2-123b', 'bedrock:mistral.magistral-small-2509', 'bedrock:mistral.ministral-3-14b-instruct', 'bedrock:mistral.ministral-3-3b-instruct', 'bedrock:mistral.ministral-3-8b-instruct', 'bedrock:mistral.mistral-7b-instruct-v0:2', 'bedrock:mistral.mistral-large-2402-v1:0', 'bedrock:mistral.mistral-large-2407-v1:0', 'bedrock:mistral.mistral-large-3-675b-instruct', 'bedrock:mistral.mistral-small-2402-v1:0', 'bedrock:mistral.mixtral-8x7b-instruct-v0:1', 'bedrock:mistral.pixtral-large-2502-v1:0', 'bedrock:moonshot.kimi-k2-thinking', 'bedrock:moonshotai.kimi-k2.5', 'bedrock:nvidia.nemotron-nano-12b-v2', 'bedrock:nvidia.nemotron-nano-3-30b', 'bedrock:nvidia.nemotron-nano-9b-v2', 'bedrock:nvidia.nemotron-super-3-120b', 'bedrock:qwen.qwen3-32b-v1:0', 'bedrock:qwen.qwen3-coder-30b-a3b-v1:0', 'bedrock:qwen.qwen3-coder-next', 'bedrock:qwen.qwen3-next-80b-a3b', 'bedrock:qwen.qwen3-vl-235b-a22b', 'bedrock:us.amazon.nova-2-lite-v1:0', 'bedrock:us.amazon.nova-lite-v1:0', 'bedrock:us.amazon.nova-micro-v1:0', 'bedrock:us.amazon.nova-premier-v1:0', 'bedrock:us.amazon.nova-pro-v1:0', 'bedrock:us.anthropic.claude-3-5-haiku-20241022-v1:0', 'bedrock:us.anthropic.claude-3-5-sonnet-20240620-v1:0', 'bedrock:us.anthropic.claude-3-5-sonnet-20241022-v2:0', 'bedrock:us.anthropic.claude-3-7-sonnet-20250219-v1:0', 'bedrock:us.anthropic.claude-3-haiku-20240307-v1:0', 'bedrock:us.anthropic.claude-3-opus-20240229-v1:0', 'bedrock:us.anthropic.claude-3-sonnet-20240229-v1:0', 'bedrock:us.anthropic.claude-fable-5', 'bedrock:us.anthropic.claude-fable-5-1', 'bedrock:us.anthropic.claude-haiku-4-5-20251001-v1:0', 'bedrock:us.anthropic.claude-haiku-5-5', 'bedrock:us.anthropic.claude-opus-4-1-20250805-v1:0', 'bedrock:us.anthropic.claude-opus-4-20250514-v1:0', 'bedrock:us.anthropic.claude-opus-4-5-20251101-v1:0', 'bedrock:us.anthropic.claude-opus-4-6-v1', 'bedrock:us.anthropic.claude-opus-4-7', 'bedrock:us.anthropic.claude-opus-4-8', 'bedrock:us.anthropic.claude-opus-5', 'bedrock:us.anthropic.claude-opus-5-5', 'bedrock:us.anthropic.claude-sonnet-4-20250514-v1:0', 'bedrock:us.anthropic.claude-sonnet-4-5-20250929-v1:0', 'bedrock:us.anthropic.claude-sonnet-4-6', 'bedrock:us.anthropic.claude-sonnet-5', 'bedrock:us.meta.llama3-1-70b-instruct-v1:0', 'bedrock:us.meta.llama3-1-8b-instruct-v1:0', 'bedrock:us.meta.llama3-2-11b-instruct-v1:0', 'bedrock:us.meta.llama3-2-1b-instruct-v1:0', 'bedrock:us.meta.llama3-2-3b-instruct-v1:0', 'bedrock:us.meta.llama3-2-90b-instruct-v1:0', 'bedrock:us.meta.llama3-3-70b-instruct-v1:0', 'bedrock:us.meta.llama4-maverick-17b-instruct-v1:0', 'bedrock:us.meta.llama4-scout-17b-instruct-v1:0', 'bedrock:us.mistral.pixtral-large-2502-v1:0', 'bedrock:us.openai.gpt-5.6-luna', 'bedrock:us.openai.gpt-5.6-sol', 'bedrock:us.openai.gpt-5.6-terra', 'bedrock:us.writer.palmyra-x4-v1:0', 'bedrock:us.writer.palmyra-x5-v1:0', 'bedrock:us.xai.grok-4.6', 'bedrock:us.xai.grok-4.7', 'bedrock:zai.glm-4.7', 'bedrock:zai.glm-4.7-flash', 'bedrock:zai.glm-5', 'cerebras:gemma-4-31b', 'cerebras:gpt-oss-120b', 'cerebras:zai-glm-4.7', 'cohere:c4ai-aya-expanse-32b', 'cohere:c4ai-aya-expanse-8b', 'cohere:command-nightly', 'cohere:command-r-08-2024', 'cohere:command-r-plus-08-2024', 'cohere:command-r7b-12-2024', 'crusoe:Qwen/Qwen3-235B-A22B-Instruct-2507', 'crusoe:deepseek-ai/DeepSeek-V3-0324', 'crusoe:deepseek-ai/DeepSeek-V4-Pro', 'crusoe:deepseek-ai/Deepseek-V4-Flash', 'crusoe:google/gemma-4-31b-it', 'crusoe:meta-llama/Llama-3.3-70B-Instruct', 'crusoe:moonshotai/Kimi-K2.6', 'crusoe:nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B', 'crusoe:nvidia/NVIDIA-Nemotron-3-Super-120B-A12B', 'crusoe:nvidia/Nemotron-3-Nano-Omni-Reasoning-30B-A3B', 'crusoe:nvidia/Nemotron-3.5-Lightning-30B-A3B', 'crusoe:openai/gpt-oss-120b', 'crusoe:yutori/n1.5', 'crusoe:zai/GLM-5.1', 'crusoe:zai/GLM-5.2', 'deepseek:deepseek-chat', 'deepseek:deepseek-reasoner', 'deepseek:deepseek-v4-flash', 'deepseek:deepseek-v4-pro', 'gateway/anthropic:claude-fable-5', 'gateway/anthropic:claude-fable-5-1', 'gateway/anthropic:claude-haiku-4-5', 'gateway/anthropic:claude-haiku-4-5-20251001', 'gateway/anthropic:claude-haiku-5-5', 'gateway/anthropic:claude-opus-4-5', 'gateway/anthropic:claude-opus-4-5-20251101', 'gateway/anthropic:claude-opus-4-6', 'gateway/anthropic:claude-opus-4-7', 'gateway/anthropic:claude-opus-4-8', 'gateway/anthropic:claude-opus-5', 'gateway/anthropic:claude-opus-5-5', 'gateway/anthropic:claude-sonnet-4-5', 'gateway/anthropic:claude-sonnet-4-5-20250929', 'gateway/anthropic:claude-sonnet-4-6', 'gateway/anthropic:claude-sonnet-5', 'gateway/anthropic:claude-sonnet-5-5', 'gateway/bedrock:anthropic.claude-3-haiku-20240307-v1:0', 'gateway/bedrock:deepseek.r1-v1:0', 'gateway/bedrock:deepseek.v3.2', 'gateway/bedrock:eu.anthropic.claude-haiku-4-5-20251001-v1:0', 'gateway/bedrock:eu.anthropic.claude-haiku-5-5', 'gateway/bedrock:eu.anthropic.claude-sonnet-4-20250514-v1:0', 'gateway/bedrock:eu.anthropic.claude-sonnet-4-5-20250929-v1:0', 'gateway/bedrock:eu.anthropic.claude-sonnet-4-6', 'gateway/bedrock:global.amazon.nova-2-lite-v1:0', 'gateway/bedrock:global.anthropic.claude-fable-5', 'gateway/bedrock:global.anthropic.claude-fable-5-1', 'gateway/bedrock:global.anthropic.claude-haiku-5-5', 'gateway/bedrock:global.anthropic.claude-opus-4-5-20251101-v1:0', 'gateway/bedrock:global.anthropic.claude-opus-4-6-v1', 'gateway/bedrock:global.anthropic.claude-opus-4-7', 'gateway/bedrock:global.anthropic.claude-opus-4-8', 'gateway/bedrock:global.anthropic.claude-opus-5', 'gateway/bedrock:global.anthropic.claude-opus-5-5', 'gateway/bedrock:global.anthropic.claude-sonnet-5', 'gateway/bedrock:global.anthropic.claude-sonnet-5-5', 'gateway/bedrock:global.openai.gpt-5.6-luna', 'gateway/bedrock:global.openai.gpt-5.6-sol', 'gateway/bedrock:global.openai.gpt-5.6-terra', 'gateway/bedrock:global.xai.grok-4.6', 'gateway/bedrock:global.xai.grok-4.7', 'gateway/bedrock:google.gemma-3-12b-it', 'gateway/bedrock:google.gemma-3-27b-it', 'gateway/bedrock:google.gemma-3-4b-it', 'gateway/bedrock:minimax.minimax-m2', 'gateway/bedrock:minimax.minimax-m2.1', 'gateway/bedrock:minimax.minimax-m2.5', 'gateway/bedrock:mistral.devstral-2-123b', 'gateway/bedrock:mistral.magistral-small-2509', 'gateway/bedrock:mistral.ministral-3-14b-instruct', 'gateway/bedrock:mistral.ministral-3-3b-instruct', 'gateway/bedrock:mistral.ministral-3-8b-instruct', 'gateway/bedrock:mistral.mistral-large-3-675b-instruct', 'gateway/bedrock:mistral.mistral-small-2402-v1:0', 'gateway/bedrock:mistral.pixtral-large-2502-v1:0', 'gateway/bedrock:moonshot.kimi-k2-thinking', 'gateway/bedrock:moonshotai.kimi-k2.5', 'gateway/bedrock:nvidia.nemotron-nano-12b-v2', 'gateway/bedrock:nvidia.nemotron-nano-3-30b', 'gateway/bedrock:nvidia.nemotron-nano-9b-v2', 'gateway/bedrock:nvidia.nemotron-super-3-120b', 'gateway/bedrock:qwen.qwen3-32b-v1:0', 'gateway/bedrock:qwen.qwen3-coder-30b-a3b-v1:0', 'gateway/bedrock:qwen.qwen3-coder-next', 'gateway/bedrock:qwen.qwen3-next-80b-a3b', 'gateway/bedrock:qwen.qwen3-vl-235b-a22b', 'gateway/bedrock:us.amazon.nova-premier-v1:0', 'gateway/bedrock:us.anthropic.claude-fable-5', 'gateway/bedrock:us.anthropic.claude-fable-5-1', 'gateway/bedrock:us.anthropic.claude-opus-4-1-20250805-v1:0', 'gateway/bedrock:us.anthropic.claude-opus-4-5-20251101-v1:0', 'gateway/bedrock:us.anthropic.claude-opus-4-6-v1', 'gateway/bedrock:us.anthropic.claude-opus-4-7', 'gateway/bedrock:us.anthropic.claude-opus-4-8', 'gateway/bedrock:us.anthropic.claude-opus-5', 'gateway/bedrock:us.anthropic.claude-opus-5-5', 'gateway/bedrock:us.anthropic.claude-sonnet-5', 'gateway/bedrock:us.meta.llama4-maverick-17b-instruct-v1:0', 'gateway/bedrock:us.meta.llama4-scout-17b-instruct-v1:0', 'gateway/bedrock:us.mistral.pixtral-large-2502-v1:0', 'gateway/bedrock:us.writer.palmyra-x4-v1:0', 'gateway/bedrock:us.writer.palmyra-x5-v1:0', 'gateway/bedrock:us.xai.grok-4.6', 'gateway/bedrock:us.xai.grok-4.7', 'gateway/bedrock:zai.glm-4.7', 'gateway/bedrock:zai.glm-4.7-flash', 'gateway/bedrock:zai.glm-5', 'gateway/google-cloud:gemini-2.5-flash', 'gateway/google-cloud:gemini-2.5-flash-image', 'gateway/google-cloud:gemini-2.5-flash-lite', 'gateway/google-cloud:gemini-2.5-pro', 'gateway/google-cloud:gemini-3-flash-preview', 'gateway/google-cloud:gemini-3-pro-image', 'gateway/google-cloud:gemini-3.1-flash-image', 'gateway/google-cloud:gemini-3.1-flash-lite', 'gateway/google-cloud:gemini-3.1-flash-lite-image', 'gateway/google-cloud:gemini-3.1-pro-preview', 'gateway/google-cloud:gemini-3.5-flash', 'gateway/google-cloud:gemini-3.5-flash-lite', 'gateway/google-cloud:gemini-3.6-flash', 'gateway/google-cloud:gemini-3.7-flash', 'gateway/google-cloud:gemini-3.8-flash', 'gateway/google-cloud:gemini-4-argon', 'gateway/google:gemini-2.5-flash', 'gateway/google:gemini-2.5-flash-image', 'gateway/google:gemini-2.5-flash-lite', 'gateway/google:gemini-2.5-pro', 'gateway/google:gemini-3-flash-preview', 'gateway/google:gemini-3-pro-image', 'gateway/google:gemini-3.1-flash-image', 'gateway/google:gemini-3.1-flash-lite', 'gateway/google:gemini-3.1-flash-lite-image', 'gateway/google:gemini-3.1-pro-preview', 'gateway/google:gemini-3.5-flash', 'gateway/google:gemini-3.5-flash-lite', 'gateway/google:gemini-3.6-flash', 'gateway/google:gemini-3.7-flash', 'gateway/google:gemini-3.8-flash', 'gateway/google:gemini-4-argon', 'gateway/groq:llama-3.1-8b-instant', 'gateway/groq:llama-3.3-70b-versatile', 'gateway/groq:openai/gpt-oss-120b', 'gateway/groq:openai/gpt-oss-20b', 'gateway/groq:openai/gpt-oss-safeguard-20b', 'gateway/openai:gpt-3.5-turbo', 'gateway/openai:gpt-3.5-turbo-0125', 'gateway/openai:gpt-3.5-turbo-1106', 'gateway/openai:gpt-audio-mini', 'gateway/openai:gpt-audio-mini-2025-12-15', 'gateway/openai:gpt-4', 'gateway/openai:gpt-4-0613', 'gateway/openai:gpt-4-turbo', 'gateway/openai:gpt-4-turbo-2024-04-09', 'gateway/openai:gpt-4.1', 'gateway/openai:gpt-4.1-2025-04-14', 'gateway/openai:gpt-4.1-mini', 'gateway/openai:gpt-4.1-mini-2025-04-14', 'gateway/openai:gpt-4.1-nano', 'gateway/openai:gpt-4.1-nano-2025-04-14', 'gateway/openai:gpt-4o', 'gateway/openai:gpt-4o-2024-05-13', 'gateway/openai:gpt-4o-2024-08-06', 'gateway/openai:gpt-4o-2024-11-20', 'gateway/openai:gpt-4o-mini', 'gateway/openai:gpt-4o-mini-2024-07-18', 'gateway/openai:gpt-5', 'gateway/openai:gpt-5-2025-08-07', 'gateway/openai:gpt-5-mini', 'gateway/openai:gpt-5-mini-2025-08-07', 'gateway/openai:gpt-5-nano', 'gateway/openai:gpt-5-nano-2025-08-07', 'gateway/openai:gpt-5-pro', 'gateway/openai:gpt-5-pro-2025-10-06', 'gateway/openai:gpt-5.1', 'gateway/openai:gpt-5.1-2025-11-13', 'gateway/openai:gpt-5.2', 'gateway/openai:gpt-5.2-2025-12-11', 'gateway/openai:gpt-5.2-chat-latest', 'gateway/openai:gpt-5.2-pro', 'gateway/openai:gpt-5.2-pro-2025-12-11', 'gateway/openai:gpt-5.3-chat-latest', 'gateway/openai:gpt-5.4', 'gateway/openai:gpt-5.4-mini', 'gateway/openai:gpt-5.4-mini-2026-03-17', 'gateway/openai:gpt-5.4-nano', 'gateway/openai:gpt-5.4-nano-2026-03-17', 'gateway/openai:gpt-5.5', 'gateway/openai:gpt-5.5-2026-04-23', 'gateway/openai:gpt-5.5-pro', 'gateway/openai:gpt-5.5-pro-2026-04-23', 'gateway/openai:gpt-5.6-cyber', 'gateway/openai:gpt-5.6-luna', 'gateway/openai:gpt-5.6-sol', 'gateway/openai:gpt-5.6-terra', 'gateway/openai:gpt-6-astra', 'gateway/openai:gpt-6-luna', 'gateway/openai:gpt-6-sol', 'gateway/openai:gpt-6.1-sol', 'gateway/openai:gpt-daybreak-blue-latest', 'gateway/openai:gpt-daybreak-red-latest', 'gateway/openai:gpt-rosalind-research', 'gateway/openai:o1', 'gateway/openai:o1-2024-12-17', 'gateway/openai:o1-pro', 'gateway/openai:o1-pro-2025-03-19', 'gateway/openai:o3', 'gateway/openai:o3-2025-04-16', 'gateway/openai:o3-mini', 'gateway/openai:o3-mini-2025-01-31', 'gateway/openai:o3-pro', 'gateway/openai:o3-pro-2025-06-10', 'gateway/openai:o4-mini', 'gateway/openai:o4-mini-2025-04-16', 'google-cloud:gemini-2.0-flash', 'google-cloud:gemini-2.0-flash-lite', 'google-cloud:gemini-2.5-flash', 'google-cloud:gemini-2.5-flash-image', 'google-cloud:gemini-2.5-flash-lite', 'google-cloud:gemini-2.5-flash-preview-09-2025', 'google-cloud:gemini-2.5-pro', 'google-cloud:gemini-3-flash-preview', 'google-cloud:gemini-3-pro-image', 'google-cloud:gemini-3-pro-image-preview', 'google-cloud:gemini-3-pro-preview', 'google-cloud:gemini-3.1-flash-image', 'google-cloud:gemini-3.1-flash-image-preview', 'google-cloud:gemini-3.1-flash-lite', 'google-cloud:gemini-3.1-flash-lite-image', 'google-cloud:gemini-3.1-pro-preview', 'google-cloud:gemini-3.5-flash', 'google-cloud:gemini-3.5-flash-lite', 'google-cloud:gemini-3.6-flash', 'google-cloud:gemini-3.7-flash', 'google-cloud:gemini-3.8-flash', 'google-cloud:gemini-4-argon', 'google-cloud:gemini-flash-latest', 'google-cloud:gemini-flash-lite-latest', 'google:gemini-2.0-flash', 'google:gemini-2.0-flash-lite', 'google:gemini-2.5-flash', 'google:gemini-2.5-flash-image', 'google:gemini-2.5-flash-lite', 'google:gemini-2.5-flash-preview-09-2025', 'google:gemini-2.5-pro', 'google:gemini-3-flash-preview', 'google:gemini-3-pro-image', 'google:gemini-3-pro-image-preview', 'google:gemini-3-pro-preview', 'google:gemini-3.1-flash-image', 'google:gemini-3.1-flash-image-preview', 'google:gemini-3.1-flash-lite', 'google:gemini-3.1-flash-lite-image', 'google:gemini-3.1-pro-preview', 'google:gemini-3.5-flash', 'google:gemini-3.5-flash-lite', 'google:gemini-3.6-flash', 'google:gemini-3.7-flash', 'google:gemini-3.8-flash', 'google:gemini-4-argon', 'google:gemini-flash-latest', 'google:gemini-flash-lite-latest', 'groq:llama-3.1-8b-instant', 'groq:llama-3.3-70b-versatile', 'groq:meta-llama/llama-prompt-guard-2-22m', 'groq:meta-llama/llama-prompt-guard-2-86m', 'groq:openai/gpt-oss-120b', 'groq:openai/gpt-oss-20b', 'groq:openai/gpt-oss-safeguard-20b', 'groq:whisper-large-v3', 'groq:whisper-large-v3-turbo', 'heroku:claude-3-5-haiku', 'heroku:claude-3-5-sonnet-latest', 'heroku:claude-3-7-sonnet', 'heroku:claude-3-haiku', 'heroku:claude-4-5-haiku', 'heroku:claude-4-5-sonnet', 'heroku:claude-4-6-sonnet', 'heroku:claude-4-sonnet', 'heroku:claude-opus-4-5', 'heroku:claude-opus-4-6', 'heroku:deepseek-v3-2', 'heroku:glm-4-7', 'heroku:glm-4-7-flash', 'heroku:gpt-oss-120b', 'heroku:kimi-k2-5', 'heroku:kimi-k2-thinking', 'heroku:minimax-m2', 'heroku:minimax-m2-1', 'heroku:nova-2-lite', 'heroku:nova-lite', 'heroku:nova-pro', 'heroku:qwen3-235b', 'heroku:qwen3-coder-480b', 'huggingface:Qwen/QwQ-32B', 'huggingface:Qwen/Qwen2.5-72B-Instruct', 'huggingface:Qwen/Qwen3-235B-A22B', 'huggingface:Qwen/Qwen3-32B', 'huggingface:deepseek-ai/DeepSeek-R1', 'huggingface:meta-llama/Llama-3.3-70B-Instruct', 'huggingface:meta-llama/Llama-4-Maverick-17B-128E-Instruct', 'huggingface:meta-llama/Llama-4-Scout-17B-16E-Instruct', 'meta:muse-spark-1.1', 'meta:muse-spark-1.2', 'meta:muse-spark-1.2-contributor', 'meta:muse-spark-1.3', 'meta:muse-spark-1.3-contributor', 'meta:muse-wonder-1.0', 'mistral:codestral-latest', 'mistral:mistral-large-latest', 'mistral:mistral-moderation-latest', 'mistral:mistral-small-latest', 'moonshotai:kimi-k2-0711-preview', 'moonshotai:kimi-k2.5', 'moonshotai:kimi-k2.6', 'moonshotai:kimi-k2.7-code', 'moonshotai:kimi-k2.7-code-highspeed', 'moonshotai:kimi-k3', 'moonshotai:kimi-latest', 'moonshotai:kimi-thinking-preview', 'moonshotai:moonshot-v1-128k', 'moonshotai:moonshot-v1-128k-vision-preview', 'moonshotai:moonshot-v1-32k', 'moonshotai:moonshot-v1-32k-vision-preview', 'moonshotai:moonshot-v1-8k', 'moonshotai:moonshot-v1-8k-vision-preview', 'moonshotai:moonshot-v1-auto', 'openai-chat:computer-use-preview', 'openai-chat:computer-use-preview-2025-03-11', 'openai-chat:gpt-3.5-turbo', 'openai-chat:gpt-3.5-turbo-0125', 'openai-chat:gpt-3.5-turbo-0301', 'openai-chat:gpt-3.5-turbo-1106', 'openai-chat:gpt-3.5-turbo-16k', 'openai-chat:gpt-audio-mini', 'openai-chat:gpt-audio-mini-2025-12-15', 'openai-chat:gpt-4', 'openai-chat:gpt-4-0314', 'openai-chat:gpt-4-0613', 'openai-chat:gpt-4-turbo', 'openai-chat:gpt-4-turbo-2024-04-09', 'openai-chat:gpt-4.1', 'openai-chat:gpt-4.1-2025-04-14', 'openai-chat:gpt-4.1-mini', 'openai-chat:gpt-4.1-mini-2025-04-14', 'openai-chat:gpt-4.1-nano', 'openai-chat:gpt-4.1-nano-2025-04-14', 'openai-chat:gpt-4o', 'openai-chat:gpt-4o-2024-05-13', 'openai-chat:gpt-4o-2024-08-06', 'openai-chat:gpt-4o-2024-11-20', 'openai-chat:gpt-4o-audio-preview', 'openai-chat:gpt-4o-audio-preview-2024-12-17', 'openai-chat:gpt-4o-audio-preview-2025-06-03', 'openai-chat:gpt-4o-mini', 'openai-chat:gpt-4o-mini-2024-07-18', 'openai-chat:gpt-4o-mini-audio-preview', 'openai-chat:gpt-4o-mini-audio-preview-2024-12-17', 'openai-chat:gpt-4o-mini-search-preview', 'openai-chat:gpt-4o-mini-search-preview-2025-03-11', 'openai-chat:gpt-4o-search-preview', 'openai-chat:gpt-4o-search-preview-2025-03-11', 'openai-chat:gpt-5', 'openai-chat:gpt-5-2025-08-07', 'openai-chat:gpt-5-chat-latest', 'openai-chat:gpt-5-codex', 'openai-chat:gpt-5-mini', 'openai-chat:gpt-5-mini-2025-08-07', 'openai-chat:gpt-5-nano', 'openai-chat:gpt-5-nano-2025-08-07', 'openai-chat:gpt-5-pro', 'openai-chat:gpt-5-pro-2025-10-06', 'openai-chat:gpt-5.1', 'openai-chat:gpt-5.1-2025-11-13', 'openai-chat:gpt-5.1-chat-latest', 'openai-chat:gpt-5.1-codex', 'openai-chat:gpt-5.1-codex-max', 'openai-chat:gpt-5.2', 'openai-chat:gpt-5.2-2025-12-11', 'openai-chat:gpt-5.2-chat-latest', 'openai-chat:gpt-5.2-pro', 'openai-chat:gpt-5.2-pro-2025-12-11', 'openai-chat:gpt-5.3-chat-latest', 'openai-chat:gpt-5.4', 'openai-chat:gpt-5.4-mini', 'openai-chat:gpt-5.4-mini-2026-03-17', 'openai-chat:gpt-5.4-nano', 'openai-chat:gpt-5.4-nano-2026-03-17', 'openai-chat:gpt-5.5', 'openai-chat:gpt-5.5-2026-04-23', 'openai-chat:gpt-5.5-pro', 'openai-chat:gpt-5.5-pro-2026-04-23', 'openai-chat:gpt-5.6-cyber', 'openai-chat:gpt-5.6-luna', 'openai-chat:gpt-5.6-sol', 'openai-chat:gpt-5.6-terra', 'openai-chat:gpt-6-astra', 'openai-chat:gpt-6-luna', 'openai-chat:gpt-6-sol', 'openai-chat:gpt-6.1-sol', 'openai-chat:gpt-daybreak-blue-latest', 'openai-chat:gpt-daybreak-red-latest', 'openai-chat:gpt-rosalind-research', 'openai-chat:o1', 'openai-chat:o1-2024-12-17', 'openai-chat:o1-pro', 'openai-chat:o1-pro-2025-03-19', 'openai-chat:o3', 'openai-chat:o3-2025-04-16', 'openai-chat:o3-deep-research', 'openai-chat:o3-deep-research-2025-06-26', 'openai-chat:o3-mini', 'openai-chat:o3-mini-2025-01-31', 'openai-chat:o3-pro', 'openai-chat:o3-pro-2025-06-10', 'openai-chat:o4-mini', 'openai-chat:o4-mini-2025-04-16', 'openai-chat:o4-mini-deep-research', 'openai-chat:o4-mini-deep-research-2025-06-26', 'openai-decisions:gpt-6-luna', 'openai:computer-use-preview', 'openai:computer-use-preview-2025-03-11', 'openai:gpt-3.5-turbo', 'openai:gpt-3.5-turbo-0125', 'openai:gpt-3.5-turbo-0301', 'openai:gpt-3.5-turbo-1106', 'openai:gpt-audio-mini', 'openai:gpt-audio-mini-2025-12-15', 'openai:gpt-4', 'openai:gpt-4-0314', 'openai:gpt-4-0613', 'openai:gpt-4-turbo', 'openai:gpt-4-turbo-2024-04-09', 'openai:gpt-4.1', 'openai:gpt-4.1-2025-04-14', 'openai:gpt-4.1-mini', 'openai:gpt-4.1-mini-2025-04-14', 'openai:gpt-4.1-nano', 'openai:gpt-4.1-nano-2025-04-14', 'openai:gpt-4o', 'openai:gpt-4o-2024-05-13', 'openai:gpt-4o-2024-08-06', 'openai:gpt-4o-2024-11-20', 'openai:gpt-4o-audio-preview', 'openai:gpt-4o-audio-preview-2024-12-17', 'openai:gpt-4o-audio-preview-2025-06-03', 'openai:gpt-4o-mini', 'openai:gpt-4o-mini-2024-07-18', 'openai:gpt-4o-mini-audio-preview', 'openai:gpt-4o-mini-audio-preview-2024-12-17', 'openai:gpt-5', 'openai:gpt-5-2025-08-07', 'openai:gpt-5-chat-latest', 'openai:gpt-5-codex', 'openai:gpt-5-mini', 'openai:gpt-5-mini-2025-08-07', 'openai:gpt-5-nano', 'openai:gpt-5-nano-2025-08-07', 'openai:gpt-5-pro', 'openai:gpt-5-pro-2025-10-06', 'openai:gpt-5.1', 'openai:gpt-5.1-2025-11-13', 'openai:gpt-5.1-chat-latest', 'openai:gpt-5.1-codex', 'openai:gpt-5.1-codex-max', 'openai:gpt-5.2', 'openai:gpt-5.2-2025-12-11', 'openai:gpt-5.2-chat-latest', 'openai:gpt-5.2-pro', 'openai:gpt-5.2-pro-2025-12-11', 'openai:gpt-5.3-chat-latest', 'openai:gpt-5.4', 'openai:gpt-5.4-mini', 'openai:gpt-5.4-mini-2026-03-17', 'openai:gpt-5.4-nano', 'openai:gpt-5.4-nano-2026-03-17', 'openai:gpt-5.5', 'openai:gpt-5.5-2026-04-23', 'openai:gpt-5.5-pro', 'openai:gpt-5.5-pro-2026-04-23', 'openai:gpt-5.6-cyber', 'openai:gpt-5.6-luna', 'openai:gpt-5.6-sol', 'openai:gpt-5.6-terra', 'openai:gpt-6-astra', 'openai:gpt-6-luna', 'openai:gpt-6-sol', 'openai:gpt-6.1-sol', 'openai:gpt-daybreak-blue-latest', 'openai:gpt-daybreak-red-latest', 'openai:gpt-rosalind-research', 'openai:o1', 'openai:o1-2024-12-17', 'openai:o1-pro', 'openai:o1-pro-2025-03-19', 'openai:o3', 'openai:o3-2025-04-16', 'openai:o3-deep-research', 'openai:o3-deep-research-2025-06-26', 'openai:o3-mini', 'openai:o3-mini-2025-01-31', 'openai:o3-pro', 'openai:o3-pro-2025-06-10', 'openai:o4-mini', 'openai:o4-mini-2025-04-16', 'openai:o4-mini-deep-research', 'openai:o4-mini-deep-research-2025-06-26', 'test', 'snowflake:claude-4-sonnet', 'snowflake:claude-fable-5', 'snowflake:claude-haiku-4-5', 'snowflake:claude-opus-4-5', 'snowflake:claude-opus-4-6', 'snowflake:claude-opus-4-7', 'snowflake:claude-opus-4-8', 'snowflake:claude-opus-5', 'snowflake:claude-sonnet-4-5', 'snowflake:claude-sonnet-4-6', 'snowflake:claude-sonnet-5', 'snowflake:deepseek-r1', 'snowflake:llama3.1-405b', 'snowflake:llama3.1-70b', 'snowflake:llama3.1-8b', 'snowflake:llama4-maverick', 'snowflake:mistral-7b', 'snowflake:mistral-large', 'snowflake:mistral-large2', 'snowflake:openai-gpt-4.1', 'snowflake:openai-gpt-5', 'snowflake:openai-gpt-5-6-luna', 'snowflake:openai-gpt-5-6-sol', 'snowflake:openai-gpt-5-6-terra', 'snowflake:openai-gpt-5-chat', 'snowflake:openai-gpt-5-mini', 'snowflake:openai-gpt-5-nano', 'snowflake:openai-gpt-5.1', 'snowflake:openai-gpt-5.2', 'snowflake:openai-gpt-5.4', 'snowflake:openai-gpt-5.5', 'snowflake:snowflake-llama-3.3-70b', 'typesafe:jev-latest', 'typesafe:jev-preview', 'xai:grok-3', 'xai:grok-3-fast', 'xai:grok-3-fast-latest', 'xai:grok-3-latest', 'xai:grok-3-mini', 'xai:grok-3-mini-fast', 'xai:grok-3-mini-fast-latest', 'xai:grok-4', 'xai:grok-4-0709', 'xai:grok-4-1-fast', 'xai:grok-4-1-fast-non-reasoning', 'xai:grok-4-1-fast-non-reasoning-latest', 'xai:grok-4-1-fast-reasoning', 'xai:grok-4-1-fast-reasoning-latest', 'xai:grok-4-fast', 'xai:grok-4-fast-non-reasoning', 'xai:grok-4-fast-non-reasoning-latest', 'xai:grok-4-fast-reasoning', 'xai:grok-4-fast-reasoning-latest', 'xai:grok-4-latest', 'xai:grok-4.20', 'xai:grok-4.20-0309', 'xai:grok-4.20-0309-non-reasoning', 'xai:grok-4.20-0309-reasoning', 'xai:grok-4.20-multi-agent', 'xai:grok-4.20-multi-agent-0309', 'xai:grok-4.20-multi-agent-latest', 'xai:grok-4.20-non-reasoning', 'xai:grok-4.20-non-reasoning-latest', 'xai:grok-4.20-reasoning-latest', 'xai:grok-4.3', 'xai:grok-4.3-latest', 'xai:grok-4.5', 'xai:grok-4.5-latest', 'xai:grok-4.6', 'xai:grok-4.7', 'xai:grok-build-0.1', 'xai:grok-code-fast-1', 'zai:autoglm-phone-multilingual', 'zai:glm-4-32b-0414-128k', 'zai:glm-4.5', 'zai:glm-4.5-air', 'zai:glm-4.5-airx', 'zai:glm-4.5-flash', 'zai:glm-4.5-x', 'zai:glm-4.5v', 'zai:glm-4.6', 'zai:glm-4.6v', 'zai:glm-4.6v-flash', 'zai:glm-4.6v-flashx', 'zai:glm-4.7', 'zai:glm-4.7-flash', 'zai:glm-4.7-flashx', 'zai:glm-5', 'zai:glm-5-turbo', 'zai:glm-5.1', 'zai:glm-5.2', 'zai:glm-5.3', 'zai:glm-5.3-flash', 'zai:glm-5v-turbo'])
How a function tool is represented on the request a provider actually receives.
'visible': an ordinary entry in the provider’stoolscollection, schema included.'deferred': a declaredtoolsentry whose schema is withheld behind the provider’s schema-deferral flag until something reveals it.'withheld': absent from the request entirely.'via_history': absent from thetoolscollection; the full definition travels on the provider’s mid-conversation tool-addition channel instead.
Resolved per tool name into ModelRequestParameters.tool_visibility.
Default: Literal['visible', 'deferred', 'withheld', 'via_history']
Whether to allow requests to models.
This global setting allows you to disable request to most models, e.g. to make sure you don’t accidentally make costly requests to a model during tests.
The testing models TestModel,
FunctionModel,
TestEmbeddingModel and
TestImageGenerationModel are not affected by this setting, nor is
SentenceTransformerEmbeddingModel,
which runs inference locally and so has no per-call provider cost.
Default: True