pydantic_ai.models.bedrock
For details on how to set up authentication with this model, see model configuration for Bedrock.
Bases: ModelSettings
Settings for Bedrock models.
See the Bedrock Converse API docs for a full list. See the boto3 implementation of the Bedrock Converse API.
Content moderation and safety settings for Bedrock API requests.
See more about it on https://docs.aws.amazon.com/bedrock/latest/APIReference/API_runtime_GuardrailConfiguration.html.
Type: GuardrailConfigurationTypeDef
Performance optimization settings for model inference.
See more about it on https://docs.aws.amazon.com/bedrock/latest/APIReference/API_runtime_PerformanceConfiguration.html.
Type: PerformanceConfigurationTypeDef
Additional metadata to attach to Bedrock API requests.
See more about it on https://docs.aws.amazon.com/bedrock/latest/APIReference/API_runtime_Converse.html#API_runtime_Converse_RequestSyntax.
JSON paths to extract additional fields from model responses.
See more about it on https://docs.aws.amazon.com/bedrock/latest/userguide/model-parameters.html.
Variables for substitution into prompt templates.
See more about it on https://docs.aws.amazon.com/bedrock/latest/APIReference/API_runtime_PromptVariableValues.html.
Type: Mapping[str, PromptVariableValuesTypeDef]
Additional model-specific parameters to include in requests.
See more about it on https://docs.aws.amazon.com/bedrock/latest/userguide/model-parameters.html.
Whether to add a cache point after the last tool definition.
When enabled, the last tool in the tools array will include a cachePoint, allowing Bedrock to cache tool
definitions and reduce costs for compatible models.
Set to True or '5m' for a 5-minute TTL (the default), or '1h' for a 1-hour TTL.
See https://docs.aws.amazon.com/bedrock/latest/userguide/prompt-caching.html for more information.
Type: bool | Literal[‘5m’, ‘1h’]
Whether to add a cache point after the system prompt blocks.
When enabled, an extra cachePoint is appended to the system prompt so Bedrock can cache system instructions.
Set to True or '5m' for a 5-minute TTL (the default), or '1h' for a 1-hour TTL.
See https://docs.aws.amazon.com/bedrock/latest/userguide/prompt-caching.html for more information.
Type: bool | Literal[‘5m’, ‘1h’]
Convenience setting to enable caching for the last user message.
When enabled, this automatically adds a cache point to the last content block in the final user message, which is useful for caching conversation history or context in multi-turn conversations.
Set to True or '5m' for a 5-minute TTL (the default), or '1h' for a 1-hour TTL.
Note: Uses 1 of Bedrock’s 4 available cache points per request. Any additional CachePoint markers in messages will be automatically limited to respect the 4-cache-point maximum. See https://docs.aws.amazon.com/bedrock/latest/userguide/prompt-caching.html for more information.
Type: bool | Literal[‘5m’, ‘1h’]
Setting for optimizing performance and cost.
Accepts {'type': 'default' | 'flex' | 'priority' | 'reserved'}. Takes precedence over the
top-level service_tier, and is the only
way to request 'reserved' (which requires a pre-purchased capacity reservation).
See more about it on https://docs.aws.amazon.com/bedrock/latest/userguide/service-tiers-inference.html.
Type: ServiceTierTypeDef
An inference profile ARN to use as the modelId in API requests.
When set, this value is used as the modelId in converse and converse_stream API calls instead of the
base model_name. This allows you to pass the base model name (e.g. 'anthropic.claude-sonnet-4-5-20250929-v1:0')
as model_name for detecting model capabilities and token counting, while routing requests through an inference profile
for cost tracking or cross-region inference.
Type: str
Bases: Model[BaseClient]
A model that uses the Bedrock Converse API.
The boto3 client used to make requests to the Bedrock Converse API.
Defaults to the client from the Provider. It can be reassigned, e.g. to
rotate short-lived credentials in a long-running service, but prefer assigning to
BedrockProvider.client so all models sharing the
provider pick up the new client. Once you’ve assigned a client here, you’re responsible for keeping it valid;
the provider’s client is no longer consulted.
Type: BedrockRuntimeClient
The model name.
Type: str
The model provider.
Type: str
def __init__(
model_name: BedrockModelName,
provider: Literal['bedrock', 'gateway'] | Provider[BaseClient] = 'bedrock',
profile: ModelProfileSpec | None = None,
settings: ModelSettings | None = None,
)
Initialize a Bedrock model.
The name of the model to use.
The name of the Bedrock model to use. List of model names available here.
provider : Literal[‘bedrock’, ‘gateway’] | Provider[BaseClient] Default: 'bedrock'
The provider to use for authentication and API access. Can be either the string
‘bedrock’ or an instance of Provider[BaseClient]. If not provided, a new provider will be
created using the other parameters.
profile : ModelProfileSpec | None Default: None
The model profile to use. Defaults to a profile picked by the provider based on the model name.
settings : ModelSettings | None Default: None
Model-specific settings that will be used as defaults for this model.
@classmethod
def supported_native_tools(cls) -> frozenset[type[AbstractNativeTool]]
The set of builtin tool types this model can handle.
frozenset[type[AbstractNativeTool]]
@async
def count_tokens(
messages: list[ModelMessage],
model_settings: ModelSettings | None,
model_request_parameters: ModelRequestParameters,
) -> usage.RequestUsage
Count the number of tokens, works with limited models.
Check the actual supported models on https://docs.aws.amazon.com/bedrock/latest/userguide/count-tokens.html
Bases: StreamedResponse
Implementation of StreamedResponse for Bedrock models.
Get the model name of the response.
Type: str
Get the provider name.
Type: str
Get the provider base URL.
Type: str
Latest Bedrock models.
Default: Literal['amazon.titan-tg1-large', 'amazon.titan-text-lite-v1', 'amazon.titan-text-express-v1', 'us.amazon.nova-2-lite-v1:0', 'us.amazon.nova-pro-v1:0', 'us.amazon.nova-lite-v1:0', 'us.amazon.nova-micro-v1:0', 'anthropic.claude-3-5-sonnet-20241022-v2:0', 'us.anthropic.claude-3-5-sonnet-20241022-v2:0', 'anthropic.claude-3-5-haiku-20241022-v1:0', 'us.anthropic.claude-3-5-haiku-20241022-v1:0', 'anthropic.claude-instant-v1', 'anthropic.claude-v2:1', 'anthropic.claude-v2', 'anthropic.claude-3-sonnet-20240229-v1:0', 'us.anthropic.claude-3-sonnet-20240229-v1:0', 'anthropic.claude-3-haiku-20240307-v1:0', 'us.anthropic.claude-3-haiku-20240307-v1:0', 'anthropic.claude-3-opus-20240229-v1:0', 'us.anthropic.claude-3-opus-20240229-v1:0', 'anthropic.claude-3-5-sonnet-20240620-v1:0', 'us.anthropic.claude-3-5-sonnet-20240620-v1:0', 'anthropic.claude-3-7-sonnet-20250219-v1:0', 'us.anthropic.claude-3-7-sonnet-20250219-v1:0', 'anthropic.claude-opus-4-20250514-v1:0', 'us.anthropic.claude-opus-4-20250514-v1:0', 'global.anthropic.claude-opus-4-5-20251101-v1:0', 'anthropic.claude-sonnet-4-20250514-v1:0', 'us.anthropic.claude-sonnet-4-20250514-v1:0', 'eu.anthropic.claude-sonnet-4-20250514-v1:0', 'anthropic.claude-sonnet-4-5-20250929-v1:0', 'us.anthropic.claude-sonnet-4-5-20250929-v1:0', 'eu.anthropic.claude-sonnet-4-5-20250929-v1:0', 'anthropic.claude-sonnet-4-6', 'us.anthropic.claude-sonnet-4-6', 'eu.anthropic.claude-sonnet-4-6', 'anthropic.claude-haiku-4-5-20251001-v1:0', 'us.anthropic.claude-haiku-4-5-20251001-v1:0', 'eu.anthropic.claude-haiku-4-5-20251001-v1:0', 'cohere.command-text-v14', 'cohere.command-r-v1:0', 'cohere.command-r-plus-v1:0', 'cohere.command-light-text-v14', 'meta.llama3-8b-instruct-v1:0', 'meta.llama3-70b-instruct-v1:0', 'meta.llama3-1-8b-instruct-v1:0', 'us.meta.llama3-1-8b-instruct-v1:0', 'meta.llama3-1-70b-instruct-v1:0', 'us.meta.llama3-1-70b-instruct-v1:0', 'meta.llama3-1-405b-instruct-v1:0', 'us.meta.llama3-2-11b-instruct-v1:0', 'us.meta.llama3-2-90b-instruct-v1:0', 'us.meta.llama3-2-1b-instruct-v1:0', 'us.meta.llama3-2-3b-instruct-v1:0', 'us.meta.llama3-3-70b-instruct-v1:0', 'mistral.mistral-7b-instruct-v0:2', 'mistral.mixtral-8x7b-instruct-v0:1', 'mistral.mistral-large-2402-v1:0', 'mistral.mistral-large-2407-v1:0']
Possible Bedrock model names.
Since Bedrock supports a variety of date-stamped models, we explicitly list the latest models but allow any name in the type hints. See the Bedrock docs for a full list.
Default: str | LatestBedrockModelNames