pydantic_ai.models
Logic related to making requests to an LLM.
The aim here is to make a common interface for different LLMs, so that the rest of the code can be agnostic to the specific LLM being used.
Configuration for an agent’s request to a model, specifically related to tools and output handling.
Structured instruction parts with metadata about their origin (static vs dynamic).
Static instructions (dynamic=False) come from literal strings passed to Agent(instructions=...).
Dynamic instructions (dynamic=True) come from @agent.instructions functions, TemplateStr,
or toolset get_instructions() methods.
Models that support granular caching (e.g. Anthropic, Bedrock) use this to place cache boundaries at the static/dynamic instruction boundary.
Type: list[InstructionPart] | None Default: None
Resolved thinking/reasoning configuration for this request.
None means the model should use its default behavior. Set by the base
Model.prepare_request() from the unified thinking field in ModelSettings,
after checking that the model’s profile supports thinking.
Type: ThinkingLevel | None Default: None
Bases: ABC
Abstract class for a model.
Get the model settings.
Type: ModelSettings | None
The model name.
Type: str
The fully qualified model name in 'provider:model_name' format.
Type: str
Human-friendly display label for the model.
Handles common patterns:
- gpt-5 -> GPT 5
- claude-sonnet-4-5 -> Claude Sonnet 4.5
- gemini-2.5-pro -> Gemini 2.5 Pro
- meta-llama/llama-3-70b -> Llama 3 70b (OpenRouter style)
Type: str
The model profile.
We use this to compute the intersection of the profile’s supported_builtin_tools and the model’s implemented tools, ensuring model.profile.supported_builtin_tools is the single source of truth for what builtin tools are actually usable.
Type: ModelProfile
The model provider, ex: openai.
Use to populate the gen_ai.system OpenTelemetry semantic convention attribute,
so should use well-known values listed in
https://opentelemetry.io/docs/specs/semconv/attributes-registry/gen-ai/#gen-ai-system
when applicable.
Type: str
The base URL for the provider API, if available.
def __init__(
settings: ModelSettings | None = None,
profile: ModelProfileSpec | None = None,
) -> None
Initialize the model with optional settings and profile.
settings : ModelSettings | None Default: None
Model-specific settings that will be used as defaults for this model.
profile : ModelProfileSpec | None Default: None
The model profile to use.
@abstractmethod
@async
def request(
messages: list[ModelMessage],
model_settings: ModelSettings | None,
model_request_parameters: ModelRequestParameters,
) -> ModelResponse
Make a request to the model.
This is ultimately called by pydantic_ai._agent_graph.ModelRequestNode._make_request(...).
@async
def count_tokens(
messages: list[ModelMessage],
model_settings: ModelSettings | None,
model_request_parameters: ModelRequestParameters,
) -> RequestUsage
Make a request to the model for counting tokens.
@async
def request_stream(
messages: list[ModelMessage],
model_settings: ModelSettings | None,
model_request_parameters: ModelRequestParameters,
run_context: RunContext[Any] | None = None,
) -> AsyncIterator[StreamedResponse]
Make a request to the model and return a streaming response.
AsyncIterator[StreamedResponse]
def customize_request_parameters(
model_request_parameters: ModelRequestParameters,
) -> ModelRequestParameters
Customize the request parameters for the model.
This method can be overridden by subclasses to modify the request parameters before sending them to the model. In particular, this method can be used to make modifications to the generated tool JSON schemas if necessary for vendor/model-specific reasons.
ModelRequestParameters
def prepare_request(
model_settings: ModelSettings | None,
model_request_parameters: ModelRequestParameters,
) -> tuple[ModelSettings | None, ModelRequestParameters]
Prepare request inputs before they are passed to the provider.
This merges the given model_settings with the model’s own settings attribute and ensures
customize_request_parameters is applied to the resolved
ModelRequestParameters. Subclasses can override this method if
they need to customize the preparation flow further, but most implementations should simply call
self.prepare_request(...) at the start of their request (and related) methods.
tuple[ModelSettings | None, ModelRequestParameters]
@classmethod
def supported_builtin_tools(cls) -> frozenset[type[AbstractBuiltinTool]]
Return the set of builtin tool types this model class can handle.
Subclasses should override this to reflect their actual capabilities. Default is empty set - subclasses must explicitly declare support.
frozenset[type[AbstractBuiltinTool]]
Bases: ABC
Streamed response from an LLM when calling a tool.
Get the model name of the response.
Type: str
Get the provider name.
Get the provider base URL.
Get the timestamp of the response.
Type: datetime
def __aiter__() -> AsyncIterator[ModelResponseStreamEvent]
Stream the response as an async iterable of ModelResponseStreamEvents.
This proxies the _event_iterator() and emits all events, while also checking for matches
on the result schema and emitting a FinalResultEvent if/when the
first match is found.
AsyncIterator[ModelResponseStreamEvent]
def get() -> ModelResponse
Build a ModelResponse from the data received from the stream so far.
def usage() -> RequestUsage
Get the usage of the response so far. This will not be the final usage until the stream is exhausted.
def check_allow_model_requests() -> None
Check if model requests are allowed.
If you’re defining your own models that have costs or latency associated with their use, you should call this in
Model.request and Model.request_stream.
RuntimeError— If model requests are not allowed.
def override_allow_model_requests(allow_model_requests: bool) -> Iterator[None]
Context manager to temporarily override ALLOW_MODEL_REQUESTS.
allow_model_requests : bool
Whether to allow model requests within the context.
Known model names that can be used with the model parameter of Agent.
KnownModelName is provided as a concise way to specify a model.
Default: TypeAliasType('KnownModelName', Literal['anthropic:claude-3-5-haiku-20241022', 'anthropic:claude-3-5-haiku-latest', 'anthropic:claude-3-7-sonnet-20250219', 'anthropic:claude-3-7-sonnet-latest', 'anthropic:claude-3-haiku-20240307', 'anthropic:claude-3-opus-20240229', 'anthropic:claude-3-opus-latest', 'anthropic:claude-4-opus-20250514', 'anthropic:claude-4-sonnet-20250514', 'anthropic:claude-haiku-4-5-20251001', 'anthropic:claude-haiku-4-5', 'anthropic:claude-opus-4-0', 'anthropic:claude-opus-4-1-20250805', 'anthropic:claude-opus-4-20250514', 'anthropic:claude-opus-4-5-20251101', 'anthropic:claude-opus-4-5', 'anthropic:claude-opus-4-6', 'anthropic:claude-sonnet-4-0', 'anthropic:claude-sonnet-4-20250514', 'anthropic:claude-sonnet-4-5-20250929', 'anthropic:claude-sonnet-4-5', 'anthropic:claude-sonnet-4-6', 'bedrock:amazon.titan-text-express-v1', 'bedrock:amazon.titan-text-lite-v1', 'bedrock:amazon.titan-tg1-large', 'bedrock:anthropic.claude-3-5-haiku-20241022-v1:0', 'bedrock:anthropic.claude-3-5-sonnet-20240620-v1:0', 'bedrock:anthropic.claude-3-5-sonnet-20241022-v2:0', 'bedrock:anthropic.claude-3-7-sonnet-20250219-v1:0', 'bedrock:anthropic.claude-3-haiku-20240307-v1:0', 'bedrock:anthropic.claude-3-opus-20240229-v1:0', 'bedrock:anthropic.claude-3-sonnet-20240229-v1:0', 'bedrock:anthropic.claude-haiku-4-5-20251001-v1:0', 'bedrock:anthropic.claude-instant-v1', 'bedrock:anthropic.claude-opus-4-20250514-v1:0', 'bedrock:anthropic.claude-sonnet-4-20250514-v1:0', 'bedrock:anthropic.claude-sonnet-4-5-20250929-v1:0', 'bedrock:anthropic.claude-sonnet-4-6', 'bedrock:anthropic.claude-v2:1', 'bedrock:anthropic.claude-v2', 'bedrock:cohere.command-light-text-v14', 'bedrock:cohere.command-r-plus-v1:0', 'bedrock:cohere.command-r-v1:0', 'bedrock:cohere.command-text-v14', 'bedrock:eu.anthropic.claude-haiku-4-5-20251001-v1:0', 'bedrock:eu.anthropic.claude-sonnet-4-20250514-v1:0', 'bedrock:eu.anthropic.claude-sonnet-4-5-20250929-v1:0', 'bedrock:eu.anthropic.claude-sonnet-4-6', 'bedrock:global.anthropic.claude-opus-4-5-20251101-v1:0', 'bedrock:meta.llama3-1-405b-instruct-v1:0', 'bedrock:meta.llama3-1-70b-instruct-v1:0', 'bedrock:meta.llama3-1-8b-instruct-v1:0', 'bedrock:meta.llama3-70b-instruct-v1:0', 'bedrock:meta.llama3-8b-instruct-v1:0', 'bedrock:mistral.mistral-7b-instruct-v0:2', 'bedrock:mistral.mistral-large-2402-v1:0', 'bedrock:mistral.mistral-large-2407-v1:0', 'bedrock:mistral.mixtral-8x7b-instruct-v0:1', 'bedrock:us.amazon.nova-2-lite-v1:0', 'bedrock:us.amazon.nova-lite-v1:0', 'bedrock:us.amazon.nova-micro-v1:0', 'bedrock:us.amazon.nova-pro-v1:0', 'bedrock:us.anthropic.claude-3-5-haiku-20241022-v1:0', 'bedrock:us.anthropic.claude-3-5-sonnet-20240620-v1:0', 'bedrock:us.anthropic.claude-3-5-sonnet-20241022-v2:0', 'bedrock:us.anthropic.claude-3-7-sonnet-20250219-v1:0', 'bedrock:us.anthropic.claude-3-haiku-20240307-v1:0', 'bedrock:us.anthropic.claude-3-opus-20240229-v1:0', 'bedrock:us.anthropic.claude-3-sonnet-20240229-v1:0', 'bedrock:us.anthropic.claude-haiku-4-5-20251001-v1:0', 'bedrock:us.anthropic.claude-opus-4-20250514-v1:0', 'bedrock:us.anthropic.claude-sonnet-4-20250514-v1:0', 'bedrock:us.anthropic.claude-sonnet-4-5-20250929-v1:0', 'bedrock:us.anthropic.claude-sonnet-4-6', 'bedrock:us.meta.llama3-1-70b-instruct-v1:0', 'bedrock:us.meta.llama3-1-8b-instruct-v1:0', 'bedrock:us.meta.llama3-2-11b-instruct-v1:0', 'bedrock:us.meta.llama3-2-1b-instruct-v1:0', 'bedrock:us.meta.llama3-2-3b-instruct-v1:0', 'bedrock:us.meta.llama3-2-90b-instruct-v1:0', 'bedrock:us.meta.llama3-3-70b-instruct-v1:0', 'cerebras:gpt-oss-120b', 'cerebras:llama-3.3-70b', 'cerebras:llama3.1-8b', 'cerebras:qwen-3-235b-a22b-instruct-2507', 'cerebras:qwen-3-32b', 'cerebras:qwen-3-coder-480b', 'cerebras:zai-glm-4.6', 'cerebras:zai-glm-4.7', 'cohere:c4ai-aya-expanse-32b', 'cohere:c4ai-aya-expanse-8b', 'cohere:command-nightly', 'cohere:command-r-08-2024', 'cohere:command-r-plus-08-2024', 'cohere:command-r7b-12-2024', 'deepseek:deepseek-chat', 'deepseek:deepseek-reasoner', 'gateway/anthropic:claude-3-5-haiku-20241022', 'gateway/anthropic:claude-3-5-haiku-latest', 'gateway/anthropic:claude-3-7-sonnet-20250219', 'gateway/anthropic:claude-3-7-sonnet-latest', 'gateway/anthropic:claude-3-haiku-20240307', 'gateway/anthropic:claude-3-opus-20240229', 'gateway/anthropic:claude-3-opus-latest', 'gateway/anthropic:claude-4-opus-20250514', 'gateway/anthropic:claude-4-sonnet-20250514', 'gateway/anthropic:claude-haiku-4-5-20251001', 'gateway/anthropic:claude-haiku-4-5', 'gateway/anthropic:claude-opus-4-0', 'gateway/anthropic:claude-opus-4-1-20250805', 'gateway/anthropic:claude-opus-4-20250514', 'gateway/anthropic:claude-opus-4-5-20251101', 'gateway/anthropic:claude-opus-4-5', 'gateway/anthropic:claude-opus-4-6', 'gateway/anthropic:claude-sonnet-4-0', 'gateway/anthropic:claude-sonnet-4-20250514', 'gateway/anthropic:claude-sonnet-4-5-20250929', 'gateway/anthropic:claude-sonnet-4-5', 'gateway/anthropic:claude-sonnet-4-6', 'gateway/bedrock:amazon.titan-text-express-v1', 'gateway/bedrock:amazon.titan-text-lite-v1', 'gateway/bedrock:amazon.titan-tg1-large', 'gateway/bedrock:anthropic.claude-3-5-haiku-20241022-v1:0', 'gateway/bedrock:anthropic.claude-3-5-sonnet-20240620-v1:0', 'gateway/bedrock:anthropic.claude-3-5-sonnet-20241022-v2:0', 'gateway/bedrock:anthropic.claude-3-7-sonnet-20250219-v1:0', 'gateway/bedrock:anthropic.claude-3-haiku-20240307-v1:0', 'gateway/bedrock:anthropic.claude-3-opus-20240229-v1:0', 'gateway/bedrock:anthropic.claude-3-sonnet-20240229-v1:0', 'gateway/bedrock:anthropic.claude-haiku-4-5-20251001-v1:0', 'gateway/bedrock:anthropic.claude-instant-v1', 'gateway/bedrock:anthropic.claude-opus-4-20250514-v1:0', 'gateway/bedrock:anthropic.claude-sonnet-4-20250514-v1:0', 'gateway/bedrock:anthropic.claude-sonnet-4-5-20250929-v1:0', 'gateway/bedrock:anthropic.claude-sonnet-4-6', 'gateway/bedrock:anthropic.claude-v2:1', 'gateway/bedrock:anthropic.claude-v2', 'gateway/bedrock:cohere.command-light-text-v14', 'gateway/bedrock:cohere.command-r-plus-v1:0', 'gateway/bedrock:cohere.command-r-v1:0', 'gateway/bedrock:cohere.command-text-v14', 'gateway/bedrock:eu.anthropic.claude-haiku-4-5-20251001-v1:0', 'gateway/bedrock:eu.anthropic.claude-sonnet-4-20250514-v1:0', 'gateway/bedrock:eu.anthropic.claude-sonnet-4-5-20250929-v1:0', 'gateway/bedrock:eu.anthropic.claude-sonnet-4-6', 'gateway/bedrock:global.anthropic.claude-opus-4-5-20251101-v1:0', 'gateway/bedrock:meta.llama3-1-405b-instruct-v1:0', 'gateway/bedrock:meta.llama3-1-70b-instruct-v1:0', 'gateway/bedrock:meta.llama3-1-8b-instruct-v1:0', 'gateway/bedrock:meta.llama3-70b-instruct-v1:0', 'gateway/bedrock:meta.llama3-8b-instruct-v1:0', 'gateway/bedrock:mistral.mistral-7b-instruct-v0:2', 'gateway/bedrock:mistral.mistral-large-2402-v1:0', 'gateway/bedrock:mistral.mistral-large-2407-v1:0', 'gateway/bedrock:mistral.mixtral-8x7b-instruct-v0:1', 'gateway/bedrock:us.amazon.nova-2-lite-v1:0', 'gateway/bedrock:us.amazon.nova-lite-v1:0', 'gateway/bedrock:us.amazon.nova-micro-v1:0', 'gateway/bedrock:us.amazon.nova-pro-v1:0', 'gateway/bedrock:us.anthropic.claude-3-5-haiku-20241022-v1:0', 'gateway/bedrock:us.anthropic.claude-3-5-sonnet-20240620-v1:0', 'gateway/bedrock:us.anthropic.claude-3-5-sonnet-20241022-v2:0', 'gateway/bedrock:us.anthropic.claude-3-7-sonnet-20250219-v1:0', 'gateway/bedrock:us.anthropic.claude-3-haiku-20240307-v1:0', 'gateway/bedrock:us.anthropic.claude-3-opus-20240229-v1:0', 'gateway/bedrock:us.anthropic.claude-3-sonnet-20240229-v1:0', 'gateway/bedrock:us.anthropic.claude-haiku-4-5-20251001-v1:0', 'gateway/bedrock:us.anthropic.claude-opus-4-20250514-v1:0', 'gateway/bedrock:us.anthropic.claude-sonnet-4-20250514-v1:0', 'gateway/bedrock:us.anthropic.claude-sonnet-4-5-20250929-v1:0', 'gateway/bedrock:us.anthropic.claude-sonnet-4-6', 'gateway/bedrock:us.meta.llama3-1-70b-instruct-v1:0', 'gateway/bedrock:us.meta.llama3-1-8b-instruct-v1:0', 'gateway/bedrock:us.meta.llama3-2-11b-instruct-v1:0', 'gateway/bedrock:us.meta.llama3-2-1b-instruct-v1:0', 'gateway/bedrock:us.meta.llama3-2-3b-instruct-v1:0', 'gateway/bedrock:us.meta.llama3-2-90b-instruct-v1:0', 'gateway/bedrock:us.meta.llama3-3-70b-instruct-v1:0', 'gateway/google-vertex:gemini-2.0-flash-lite', 'gateway/google-vertex:gemini-2.0-flash', 'gateway/google-vertex:gemini-2.5-flash-image', 'gateway/google-vertex:gemini-2.5-flash-lite-preview-09-2025', 'gateway/google-vertex:gemini-2.5-flash-lite', 'gateway/google-vertex:gemini-2.5-flash-preview-09-2025', 'gateway/google-vertex:gemini-2.5-flash', 'gateway/google-vertex:gemini-2.5-pro', 'gateway/google-vertex:gemini-3-flash-preview', 'gateway/google-vertex:gemini-3-pro-image-preview', 'gateway/google-vertex:gemini-3-pro-preview', 'gateway/google-vertex:gemini-3.1-flash-image-preview', 'gateway/google-vertex:gemini-3.1-flash-lite-preview', 'gateway/google-vertex:gemini-3.1-pro-preview', 'gateway/google-vertex:gemini-flash-latest', 'gateway/google-vertex:gemini-flash-lite-latest', 'gateway/groq:llama-3.1-8b-instant', 'gateway/groq:llama-3.3-70b-versatile', 'gateway/groq:meta-llama/llama-guard-4-12b', 'gateway/groq:openai/gpt-oss-120b', 'gateway/groq:openai/gpt-oss-20b', 'gateway/groq:whisper-large-v3', 'gateway/groq:whisper-large-v3-turbo', 'gateway/groq:meta-llama/llama-4-maverick-17b-128e-instruct', 'gateway/groq:meta-llama/llama-4-scout-17b-16e-instruct', 'gateway/groq:meta-llama/llama-prompt-guard-2-22m', 'gateway/groq:meta-llama/llama-prompt-guard-2-86m', 'gateway/groq:moonshotai/kimi-k2-instruct-0905', 'gateway/groq:openai/gpt-oss-safeguard-20b', 'gateway/groq:playai-tts', 'gateway/groq:playai-tts-arabic', 'gateway/groq:qwen/qwen-3-32b', 'gateway/openai:computer-use-preview-2025-03-11', 'gateway/openai:computer-use-preview', 'gateway/openai:gpt-3.5-turbo-0125', 'gateway/openai:gpt-3.5-turbo-0301', 'gateway/openai:gpt-3.5-turbo-0613', 'gateway/openai:gpt-3.5-turbo-1106', 'gateway/openai:gpt-3.5-turbo-16k-0613', 'gateway/openai:gpt-3.5-turbo-16k', 'gateway/openai:gpt-3.5-turbo', 'gateway/openai:gpt-4-0314', 'gateway/openai:gpt-4-0613', 'gateway/openai:gpt-4-turbo-2024-04-09', 'gateway/openai:gpt-4-turbo', 'gateway/openai:gpt-4.1-2025-04-14', 'gateway/openai:gpt-4.1-mini-2025-04-14', 'gateway/openai:gpt-4.1-mini', 'gateway/openai:gpt-4.1-nano-2025-04-14', 'gateway/openai:gpt-4.1-nano', 'gateway/openai:gpt-4.1', 'gateway/openai:gpt-4', 'gateway/openai:gpt-4o-2024-05-13', 'gateway/openai:gpt-4o-2024-08-06', 'gateway/openai:gpt-4o-2024-11-20', 'gateway/openai:gpt-4o-audio-preview-2024-12-17', 'gateway/openai:gpt-4o-audio-preview-2025-06-03', 'gateway/openai:gpt-4o-audio-preview', 'gateway/openai:gpt-4o-mini-2024-07-18', 'gateway/openai:gpt-4o-mini-audio-preview-2024-12-17', 'gateway/openai:gpt-4o-mini-audio-preview', 'gateway/openai:gpt-4o-mini-search-preview-2025-03-11', 'gateway/openai:gpt-4o-mini-search-preview', 'gateway/openai:gpt-4o-mini', 'gateway/openai:gpt-4o-search-preview-2025-03-11', 'gateway/openai:gpt-4o-search-preview', 'gateway/openai:gpt-4o', 'gateway/openai:gpt-5-2025-08-07', 'gateway/openai:gpt-5-chat-latest', 'gateway/openai:gpt-5-codex', 'gateway/openai:gpt-5-mini-2025-08-07', 'gateway/openai:gpt-5-mini', 'gateway/openai:gpt-5-nano-2025-08-07', 'gateway/openai:gpt-5-nano', 'gateway/openai:gpt-5-pro-2025-10-06', 'gateway/openai:gpt-5-pro', 'gateway/openai:gpt-5.1-2025-11-13', 'gateway/openai:gpt-5.1-chat-latest', 'gateway/openai:gpt-5.1-codex-max', 'gateway/openai:gpt-5.1-codex', 'gateway/openai:gpt-5.1', 'gateway/openai:gpt-5.2-2025-12-11', 'gateway/openai:gpt-5.2-chat-latest', 'gateway/openai:gpt-5.2-pro-2025-12-11', 'gateway/openai:gpt-5.2-pro', 'gateway/openai:gpt-5.2', 'gateway/openai:gpt-5.3-chat-latest', 'gateway/openai:gpt-5.4-mini-2026-03-17', 'gateway/openai:gpt-5.4-mini', 'gateway/openai:gpt-5.4-nano-2026-03-17', 'gateway/openai:gpt-5.4-nano', 'gateway/openai:gpt-5.4', 'gateway/openai:gpt-5', 'gateway/openai:o1-2024-12-17', 'gateway/openai:o1-pro-2025-03-19', 'gateway/openai:o1-pro', 'gateway/openai:o1', 'gateway/openai:o3-2025-04-16', 'gateway/openai:o3-deep-research-2025-06-26', 'gateway/openai:o3-deep-research', 'gateway/openai:o3-mini-2025-01-31', 'gateway/openai:o3-mini', 'gateway/openai:o3-pro-2025-06-10', 'gateway/openai:o3-pro', 'gateway/openai:o3', 'gateway/openai:o4-mini-2025-04-16', 'gateway/openai:o4-mini-deep-research-2025-06-26', 'gateway/openai:o4-mini-deep-research', 'gateway/openai:o4-mini', 'google-gla:gemini-2.0-flash-lite', 'google-gla:gemini-2.0-flash', 'google-gla:gemini-2.5-flash-image', 'google-gla:gemini-2.5-flash-lite-preview-09-2025', 'google-gla:gemini-2.5-flash-lite', 'google-gla:gemini-2.5-flash-preview-09-2025', 'google-gla:gemini-2.5-flash', 'google-gla:gemini-2.5-pro', 'google-gla:gemini-3-flash-preview', 'google-gla:gemini-3-pro-image-preview', 'google-gla:gemini-3-pro-preview', 'google-gla:gemini-3.1-flash-image-preview', 'google-gla:gemini-3.1-flash-lite-preview', 'google-gla:gemini-3.1-pro-preview', 'google-gla:gemini-flash-latest', 'google-gla:gemini-flash-lite-latest', 'google-vertex:gemini-2.0-flash-lite', 'google-vertex:gemini-2.0-flash', 'google-vertex:gemini-2.5-flash-image', 'google-vertex:gemini-2.5-flash-lite-preview-09-2025', 'google-vertex:gemini-2.5-flash-lite', 'google-vertex:gemini-2.5-flash-preview-09-2025', 'google-vertex:gemini-2.5-flash', 'google-vertex:gemini-2.5-pro', 'google-vertex:gemini-3-flash-preview', 'google-vertex:gemini-3-pro-image-preview', 'google-vertex:gemini-3-pro-preview', 'google-vertex:gemini-3.1-flash-image-preview', 'google-vertex:gemini-3.1-flash-lite-preview', 'google-vertex:gemini-3.1-pro-preview', 'google-vertex:gemini-flash-latest', 'google-vertex:gemini-flash-lite-latest', 'grok:grok-2-image-1212', 'grok:grok-2-vision-1212', 'grok:grok-3-fast', 'grok:grok-3-mini-fast', 'grok:grok-3-mini', 'grok:grok-3', 'grok:grok-4-0709', 'grok:grok-4-latest', 'grok:grok-4-1-fast-non-reasoning', 'grok:grok-4-1-fast-reasoning', 'grok:grok-4-1-fast', 'grok:grok-4-fast-non-reasoning', 'grok:grok-4-fast-reasoning', 'grok:grok-4-fast', 'grok:grok-4', 'grok:grok-code-fast-1', 'xai:grok-3', 'xai:grok-3-fast', 'xai:grok-3-fast-latest', 'xai:grok-3-latest', 'xai:grok-3-mini', 'xai:grok-3-mini-fast', 'xai:grok-3-mini-fast-latest', 'xai:grok-4', 'xai:grok-4-0709', 'xai:grok-4-1-fast', 'xai:grok-4-1-fast-non-reasoning', 'xai:grok-4-1-fast-non-reasoning-latest', 'xai:grok-4-1-fast-reasoning', 'xai:grok-4-1-fast-reasoning-latest', 'xai:grok-4-fast', 'xai:grok-4-fast-non-reasoning', 'xai:grok-4-fast-non-reasoning-latest', 'xai:grok-4-fast-reasoning', 'xai:grok-4-fast-reasoning-latest', 'xai:grok-4-latest', 'xai:grok-code-fast-1', 'groq:llama-3.1-8b-instant', 'groq:llama-3.3-70b-versatile', 'groq:meta-llama/llama-guard-4-12b', 'groq:openai/gpt-oss-120b', 'groq:openai/gpt-oss-20b', 'groq:whisper-large-v3', 'groq:whisper-large-v3-turbo', 'groq:meta-llama/llama-4-maverick-17b-128e-instruct', 'groq:meta-llama/llama-4-scout-17b-16e-instruct', 'groq:meta-llama/llama-prompt-guard-2-22m', 'groq:meta-llama/llama-prompt-guard-2-86m', 'groq:moonshotai/kimi-k2-instruct-0905', 'groq:openai/gpt-oss-safeguard-20b', 'groq:playai-tts', 'groq:playai-tts-arabic', 'groq:qwen/qwen-3-32b', 'heroku:claude-3-5-haiku', 'heroku:claude-3-5-sonnet-latest', 'heroku:claude-3-7-sonnet', 'heroku:claude-3-haiku', 'heroku:claude-4-5-haiku', 'heroku:claude-4-5-sonnet', 'heroku:claude-4-sonnet', 'heroku:claude-opus-4-5', 'heroku:gpt-oss-120b', 'heroku:kimi-k2-thinking', 'heroku:minimax-m2', 'heroku:qwen3-235b', 'heroku:qwen3-coder-480b', 'heroku:nova-2-lite', 'heroku:nova-lite', 'heroku:nova-pro', 'huggingface:deepseek-ai/DeepSeek-R1', 'huggingface:meta-llama/Llama-3.3-70B-Instruct', 'huggingface:meta-llama/Llama-4-Maverick-17B-128E-Instruct', 'huggingface:meta-llama/Llama-4-Scout-17B-16E-Instruct', 'huggingface:Qwen/Qwen2.5-72B-Instruct', 'huggingface:Qwen/Qwen3-235B-A22B', 'huggingface:Qwen/Qwen3-32B', 'huggingface:Qwen/QwQ-32B', 'mistral:codestral-latest', 'mistral:mistral-large-latest', 'mistral:mistral-moderation-latest', 'mistral:mistral-small-latest', 'moonshotai:kimi-k2-0711-preview', 'moonshotai:kimi-latest', 'moonshotai:kimi-thinking-preview', 'moonshotai:moonshot-v1-128k-vision-preview', 'moonshotai:moonshot-v1-128k', 'moonshotai:moonshot-v1-32k-vision-preview', 'moonshotai:moonshot-v1-32k', 'moonshotai:moonshot-v1-8k-vision-preview', 'moonshotai:moonshot-v1-8k', 'openai:computer-use-preview-2025-03-11', 'openai:computer-use-preview', 'openai:gpt-3.5-turbo-0125', 'openai:gpt-3.5-turbo-0301', 'openai:gpt-3.5-turbo-0613', 'openai:gpt-3.5-turbo-1106', 'openai:gpt-3.5-turbo-16k-0613', 'openai:gpt-3.5-turbo-16k', 'openai:gpt-3.5-turbo', 'openai:gpt-4-0314', 'openai:gpt-4-0613', 'openai:gpt-4-turbo-2024-04-09', 'openai:gpt-4-turbo', 'openai:gpt-4.1-2025-04-14', 'openai:gpt-4.1-mini-2025-04-14', 'openai:gpt-4.1-mini', 'openai:gpt-4.1-nano-2025-04-14', 'openai:gpt-4.1-nano', 'openai:gpt-4.1', 'openai:gpt-4', 'openai:gpt-4o-2024-05-13', 'openai:gpt-4o-2024-08-06', 'openai:gpt-4o-2024-11-20', 'openai:gpt-4o-audio-preview-2024-12-17', 'openai:gpt-4o-audio-preview-2025-06-03', 'openai:gpt-4o-audio-preview', 'openai:gpt-4o-mini-2024-07-18', 'openai:gpt-4o-mini-audio-preview-2024-12-17', 'openai:gpt-4o-mini-audio-preview', 'openai:gpt-4o-mini-search-preview-2025-03-11', 'openai:gpt-4o-mini-search-preview', 'openai:gpt-4o-mini', 'openai:gpt-4o-search-preview-2025-03-11', 'openai:gpt-4o-search-preview', 'openai:gpt-4o', 'openai:gpt-5-2025-08-07', 'openai:gpt-5-chat-latest', 'openai:gpt-5-codex', 'openai:gpt-5-mini-2025-08-07', 'openai:gpt-5-mini', 'openai:gpt-5-nano-2025-08-07', 'openai:gpt-5-nano', 'openai:gpt-5-pro-2025-10-06', 'openai:gpt-5-pro', 'openai:gpt-5.1-2025-11-13', 'openai:gpt-5.1-chat-latest', 'openai:gpt-5.1-codex-max', 'openai:gpt-5.1-codex', 'openai:gpt-5.1', 'openai:gpt-5.2-2025-12-11', 'openai:gpt-5.2-chat-latest', 'openai:gpt-5.2-pro-2025-12-11', 'openai:gpt-5.2-pro', 'openai:gpt-5.2', 'openai:gpt-5.3-chat-latest', 'openai:gpt-5.4-mini-2026-03-17', 'openai:gpt-5.4-mini', 'openai:gpt-5.4-nano-2026-03-17', 'openai:gpt-5.4-nano', 'openai:gpt-5.4', 'openai:gpt-5', 'openai:o1-2024-12-17', 'openai:o1-pro-2025-03-19', 'openai:o1-pro', 'openai:o1', 'openai:o3-2025-04-16', 'openai:o3-deep-research-2025-06-26', 'openai:o3-deep-research', 'openai:o3-mini-2025-01-31', 'openai:o3-mini', 'openai:o3-pro-2025-06-10', 'openai:o3-pro', 'openai:o3', 'openai:o4-mini-2025-04-16', 'openai:o4-mini-deep-research-2025-06-26', 'openai:o4-mini-deep-research', 'openai:o4-mini', 'test'])
Whether to allow requests to models.
This global setting allows you to disable request to most models, e.g. to make sure you don’t accidentally make costly requests to a model during tests.
The testing models TestModel and
FunctionModel are no affected by this setting.
Default: True