# pydantic\_ai.providers

### Provider

**Bases:** `ABC`, `Generic[InterfaceClient]`

Abstract class for a provider.

The provider is in charge of providing an authenticated client to the API.

Each provider only supports a specific interface. An interface can be supported by multiple providers.

For example, the `OpenAIChatModel` interface can be supported by the `OpenAIProvider` and the `DeepSeekProvider`.

When used as an async context manager, providers that create their own HTTP client will close it on exit. This is handled automatically when using [`Agent`](/docs/ai/api/pydantic-ai/agent/#pydantic_ai.agent.Agent) as a context manager.

#### Attributes

##### name

The provider name.

The returned value flows into [`ModelMessage.provider_name`](/docs/ai/api/pydantic-ai/messages/#pydantic_ai.messages.ModelMessage) on every part. Thinking-tag detection and native-tool detection check this value when the model class loads history, so silently renaming a concrete `name` value breaks replay of any message history captured against the old name.

**Type:** [`str`](https://docs.python.org/3/library/stdtypes.html#str)

##### base\_url

The base URL for the provider API.

**Type:** [`str`](https://docs.python.org/3/library/stdtypes.html#str)

##### client

The client for the provider.

**Type:** `InterfaceClient`

#### Methods

##### model\_profile

`@staticmethod`

```python
def model_profile(model_name: str) -> ModelProfile | None
```

The model profile for the named model, if available.

###### Returns

[`ModelProfile`](/docs/ai/api/pydantic-ai/profiles/#pydantic_ai.profiles.ModelProfile) | [`None`](https://docs.python.org/3/library/constants.html#None)

### gateway\_provider

```python
def gateway_provider(
    upstream_provider: Literal['openai', 'openai-chat', 'openai-responses', 'chat', 'responses'],
    route: str | None = None,
    api_key: str | None = None,
    base_url: str | None = None,
    http_client: httpx.AsyncClient | None = None,
) -> Provider[AsyncOpenAI]
def gateway_provider(
    upstream_provider: Literal['groq'],
    route: str | None = None,
    api_key: str | None = None,
    base_url: str | None = None,
    http_client: httpx.AsyncClient | None = None,
) -> Provider[AsyncGroq]
def gateway_provider(
    upstream_provider: Literal['anthropic'],
    route: str | None = None,
    api_key: str | None = None,
    base_url: str | None = None,
    http_client: httpx.AsyncClient | None = None,
) -> Provider[AsyncAnthropicClient]
def gateway_provider(
    upstream_provider: Literal['bedrock', 'converse'],
    route: str | None = None,
    api_key: str | None = None,
    base_url: str | None = None,
) -> Provider[BaseClient]
def gateway_provider(
    upstream_provider: Literal['gemini', 'google-cloud', 'google-vertex'],
    route: str | None = None,
    api_key: str | None = None,
    base_url: str | None = None,
    http_client: httpx.AsyncClient | None = None,
) -> Provider[GoogleClient]
def gateway_provider(
    upstream_provider: str,
    route: str | None = None,
    api_key: str | None = None,
    base_url: str | None = None,
) -> Provider[Any]
```

Create a new Gateway provider.

#### Returns

`Provider`\[[`Any`](https://docs.python.org/3/library/typing.html#typing.Any)\]

#### Parameters

**`upstream_provider`** : `UpstreamProvider` | [`str`](https://docs.python.org/3/library/stdtypes.html#str)

The upstream provider to use.

**`route`** : [`str`](https://docs.python.org/3/library/stdtypes.html#str) | [`None`](https://docs.python.org/3/library/constants.html#None) _Default:_ `None`

The name of the provider or routing group to use to handle the request. If not provided, the default routing group for the API format will be used.

**`api_key`** : [`str`](https://docs.python.org/3/library/stdtypes.html#str) | [`None`](https://docs.python.org/3/library/constants.html#None) _Default:_ `None`

The API key to use for authentication. If not provided, the `PYDANTIC_AI_GATEWAY_API_KEY` environment variable will be used if available.

**`base_url`** : [`str`](https://docs.python.org/3/library/stdtypes.html#str) | [`None`](https://docs.python.org/3/library/constants.html#None) _Default:_ `None`

The base URL to use for the Gateway. If not provided, the `PYDANTIC_AI_GATEWAY_BASE_URL` environment variable will be used if available. Otherwise, defaults to `https://gateway.pydantic.dev/proxy`.

**`http_client`** : `httpx.AsyncClient` | [`None`](https://docs.python.org/3/library/constants.html#None) _Default:_ `None`

The HTTP client to use for the Gateway.

### AnthropicProvider

**Bases:** `Provider[AsyncAnthropicClient]`

Provider for Anthropic API.

#### Methods

##### \_\_init\_\_

```python
def __init__(anthropic_client: AsyncAnthropicClient | None = None) -> None
def __init__(
    api_key: str | None = None,
    base_url: str | None = None,
    http_client: httpx.AsyncClient | None = None,
) -> None
```

Create a new Anthropic provider.

###### Returns

[`None`](https://docs.python.org/3/library/constants.html#None)

###### Parameters

**`api_key`** : [`str`](https://docs.python.org/3/library/stdtypes.html#str) | [`None`](https://docs.python.org/3/library/constants.html#None) _Default:_ `None`

The API key to use for authentication, if not provided, the `ANTHROPIC_API_KEY` environment variable will be used if available.

**`base_url`** : [`str`](https://docs.python.org/3/library/stdtypes.html#str) | [`None`](https://docs.python.org/3/library/constants.html#None) _Default:_ `None`

The base URL to use for the Anthropic API.

**`anthropic_client`** : `AsyncAnthropicClient` | [`None`](https://docs.python.org/3/library/constants.html#None) _Default:_ `None`

An existing Anthropic client to use. Accepts [`AsyncAnthropic`](https://github.com/anthropics/anthropic-sdk-python), [`AsyncAnthropicBedrock`](https://platform.claude.com/docs/en/build-with-claude/claude-on-amazon-bedrock-legacy), [`AsyncAnthropicBedrockMantle`](https://platform.claude.com/docs/en/build-with-claude/claude-in-amazon-bedrock), [`AsyncAnthropicFoundry`](https://platform.claude.com/docs/en/build-with-claude/claude-in-microsoft-foundry), or [`AsyncAnthropicVertex`](https://docs.anthropic.com/en/api/claude-on-vertex-ai). If provided, the `api_key` and `http_client` arguments will be ignored.

**`http_client`** : `httpx.AsyncClient` | [`None`](https://docs.python.org/3/library/constants.html#None) _Default:_ `None`

An existing `httpx.AsyncClient` to use for making HTTP requests.

### BaseGoogleProvider

**Bases:** `Provider[Client]`, `ABC`

Common base for the Gemini API and Google Cloud providers.

Abstract -- instantiate [`GoogleProvider`](/docs/ai/api/pydantic-ai/providers/#pydantic_ai.providers.google.GoogleProvider) for the Gemini API or `GoogleCloudProvider` for Google Cloud. Subclasses share `base_url`, `client`, `_set_http_client`, and model-profile lookup; each subclass owns its own `Client` construction.

### GoogleProvider

**Bases:** `BaseGoogleProvider`

Provider for the Gemini API (formerly Google AI Studio / Google GLA).

#### Methods

##### \_\_init\_\_

```python
def __init__(
    api_key: str,
    http_client: httpx.AsyncClient | None = None,
    base_url: str | None = None,
) -> None
def __init__(
    credentials: Credentials | None = None,
    project: str | None = None,
    location: GoogleCloudLocation | Literal['global'] | str | None = None,
    http_client: httpx.AsyncClient | None = None,
    base_url: str | None = None,
) -> None
def __init__(client: Client) -> None
def __init__(
    vertexai: bool = False,
    api_key: str | None = None,
    http_client: httpx.AsyncClient | None = None,
    base_url: str | None = None,
) -> None
```

Create a new Google provider for the Gemini API.

###### Returns

[`None`](https://docs.python.org/3/library/constants.html#None)

###### Parameters

**`api_key`** : [`str`](https://docs.python.org/3/library/stdtypes.html#str) | [`None`](https://docs.python.org/3/library/constants.html#None) _Default:_ `None`

The [API key](https://ai.google.dev/gemini-api/docs/api-key) to use for authentication. It can also be set via the `GOOGLE_API_KEY` environment variable.

**`credentials`** : `Credentials` | [`None`](https://docs.python.org/3/library/constants.html#None) _Default:_ `None`

Deprecated. Use `GoogleCloudProvider` instead.

**`project`** : [`str`](https://docs.python.org/3/library/stdtypes.html#str) | [`None`](https://docs.python.org/3/library/constants.html#None) _Default:_ `None`

Deprecated. Use `GoogleCloudProvider` instead.

**`location`** : `GoogleCloudLocation` | [`Literal`](https://docs.python.org/3/library/typing.html#typing.Literal)\['global'\] | [`str`](https://docs.python.org/3/library/stdtypes.html#str) | [`None`](https://docs.python.org/3/library/constants.html#None) _Default:_ `None`

Deprecated. Use `GoogleCloudProvider` instead.

**`vertexai`** : [`bool`](https://docs.python.org/3/library/functions.html#bool) | [`None`](https://docs.python.org/3/library/constants.html#None) _Default:_ `None`

Deprecated. Use `GoogleCloudProvider` instead.

**`client`** : `Client` | [`None`](https://docs.python.org/3/library/constants.html#None) _Default:_ `None`

A pre-initialized client to use.

**`http_client`** : `httpx.AsyncClient` | [`None`](https://docs.python.org/3/library/constants.html#None) _Default:_ `None`

An existing `httpx.AsyncClient` to use for making HTTP requests.

**`base_url`** : [`str`](https://docs.python.org/3/library/stdtypes.html#str) | [`None`](https://docs.python.org/3/library/constants.html#None) _Default:_ `None`

The base URL for the Gemini API.

### GoogleCloudLocation

Regions available for Google Cloud. More details [here](https://cloud.google.com/vertex-ai/generative-ai/docs/learn/locations#genai-locations).

**Default:** `Literal['asia-east1', 'asia-east2', 'asia-northeast1', 'asia-northeast3', 'asia-south1', 'asia-southeast1', 'australia-southeast1', 'europe-central2', 'europe-north1', 'europe-southwest1', 'europe-west1', 'europe-west2', 'europe-west3', 'europe-west4', 'europe-west6', 'europe-west8', 'europe-west9', 'me-central1', 'me-central2', 'me-west1', 'northamerica-northeast1', 'southamerica-east1', 'us-central1', 'us-east1', 'us-east4', 'us-east5', 'us-south1', 'us-west1', 'us-west4']`

### VertexAILocation

Deprecated alias for `GoogleCloudLocation`.

**Default:** `GoogleCloudLocation`

### OpenAIProvider

**Bases:** `Provider[AsyncOpenAI]`

Provider for OpenAI API.

#### Methods

##### \_\_init\_\_

```python
def __init__(openai_client: AsyncOpenAI) -> None
def __init__(
    base_url: str | None = None,
    api_key: str | None = None,
    openai_client: None = None,
    http_client: httpx.AsyncClient | None = None,
) -> None
```

Create a new OpenAI provider.

###### Returns

[`None`](https://docs.python.org/3/library/constants.html#None)

###### Parameters

**`base_url`** : [`str`](https://docs.python.org/3/library/stdtypes.html#str) | [`None`](https://docs.python.org/3/library/constants.html#None) _Default:_ `None`

The base url for the OpenAI requests. If not provided, the `OPENAI_BASE_URL` environment variable will be used if available. Otherwise, defaults to OpenAI's base url.

**`api_key`** : [`str`](https://docs.python.org/3/library/stdtypes.html#str) | [`None`](https://docs.python.org/3/library/constants.html#None) _Default:_ `None`

The API key to use for authentication, if not provided, the `OPENAI_API_KEY` environment variable will be used if available.

**`openai_client`** : `AsyncOpenAI` | [`None`](https://docs.python.org/3/library/constants.html#None) _Default:_ `None`

An existing [`AsyncOpenAI`](https://github.com/openai/openai-python?tab=readme-ov-file#async-usage) client to use. If provided, `base_url`, `api_key`, and `http_client` must be `None`.

**`http_client`** : `httpx.AsyncClient` | [`None`](https://docs.python.org/3/library/constants.html#None) _Default:_ `None`

An existing `httpx.AsyncClient` to use for making HTTP requests.

### XaiProvider

**Bases:** `Provider[AsyncClient]`

Provider for xAI API (native xAI SDK).

#### Methods

##### \_\_init\_\_

```python
def __init__(
    api_key: str | None = None,
    api_host: str | None = None,
    timeout: float | None = None,
) -> None
def __init__(xai_client: AsyncClient) -> None
```

Create a new xAI provider.

###### Returns

[`None`](https://docs.python.org/3/library/constants.html#None)

###### Parameters

**`api_key`** : [`str`](https://docs.python.org/3/library/stdtypes.html#str) | [`None`](https://docs.python.org/3/library/constants.html#None) _Default:_ `None`

The API key to use for authentication, if not provided, the `XAI_API_KEY` environment variable will be used if available.

**`api_host`** : [`str`](https://docs.python.org/3/library/stdtypes.html#str) | [`None`](https://docs.python.org/3/library/constants.html#None) _Default:_ `None`

The API host to use for the xAI SDK client.

**`timeout`** : [`float`](https://docs.python.org/3/library/functions.html#float) | [`None`](https://docs.python.org/3/library/constants.html#None) _Default:_ `None`

The client-level default timeout for the xAI SDK client, in seconds, applied to all requests made through it. This is distinct from `ModelSettings.timeout`, which overrides the timeout for an individual request.

**`xai_client`** : `AsyncClient` | [`None`](https://docs.python.org/3/library/constants.html#None) _Default:_ `None`

An existing `xai_sdk.AsyncClient` to use. This takes precedence over `api_key`, `api_host`, and `timeout`.

### DeepSeekProvider

**Bases:** `Provider[AsyncOpenAI]`

Provider for DeepSeek API.

### BedrockJsonSchemaTransformer

**Bases:** `JsonSchemaTransformer`

Transforms schemas to the subset supported by Bedrock structured outputs.

The transformer is applied to Bedrock tool and output schemas during request customization. Strict-mode rewrites are applied when:

-   `NativeOutput` is used as the `output_type` of the Agent. `BedrockConverseModel` forces native output schemas to `strict=True` before request customization.
-   `strict=True` is set explicitly on a Tool.

Like `AnthropicJsonSchemaTransformer`, Bedrock does not infer strict tool mode from `strict=None`. Strict tool definitions are opt-in: callers must set `strict=True` explicitly. This avoids silently changing large toolsets into strict toolsets, which can exceed Anthropic/Bedrock's 20 strict-tools-per-request limit, and avoids applying potentially lossy strict-mode schema rewrites unless requested.

When `strict=True`, `additionalProperties: false` is injected on objects and keys Bedrock rejects are removed from the schema and re-emitted into the field's `description` so the model still has the hint.

### BedrockModelProfile

**Bases:** [`ModelProfile`](/docs/ai/api/pydantic-ai/profiles/#pydantic_ai.profiles.ModelProfile)

Profile for models used with BedrockModel.

ALL FIELDS MUST BE `bedrock_` PREFIXED SO YOU CAN MERGE THEM WITH OTHER MODELS.

#### Attributes

##### bedrock\_supports\_strict\_tool\_definition

Whether this model accepts `strict: true` on `toolSpec` in Bedrock's Converse API.

Tracked separately from `supports_json_schema_output` (which gates `NativeOutput` / `outputConfig`) because AWS could in principle ship a model that supports one without the other; today both features track the same per-model allowlist per the Bedrock structured-output docs: [https://docs.aws.amazon.com/bedrock/latest/userguide/structured-output.html](https://docs.aws.amazon.com/bedrock/latest/userguide/structured-output.html).

**Type:** [`bool`](https://docs.python.org/3/library/functions.html#bool) **Default:** `False`

##### bedrock\_thinking\_variant

Which thinking API shape to use for unified thinking translation.

-   `'anthropic'`: Uses `{'thinking': {'type': 'adaptive'}}` for 4.6+ models, or `{'thinking': {'type': 'enabled', 'budget_tokens': N}}` for older models.
-   `'openai'`: Uses `{'reasoning_effort': 'low'|'medium'|'high'}`
-   `'qwen'`: Uses `{'reasoning_config': 'low'|'high'}`
-   `None`: No unified thinking support.

**Type:** [`Literal`](https://docs.python.org/3/library/typing.html#typing.Literal)\['anthropic', 'openai', 'qwen'\] | [`None`](https://docs.python.org/3/library/constants.html#None) **Default:** `None`

##### bedrock\_supports\_adaptive\_thinking

Whether this model accepts `{'thinking': {'type': 'adaptive'}}` (Sonnet 4.6+, Opus 4.6+).

Only meaningful for the `'anthropic'` variant. When False, the variant falls back to `{'type': 'enabled', 'budget_tokens': N}` for pre-4.6 models.

**Type:** [`bool`](https://docs.python.org/3/library/functions.html#bool) **Default:** `False`

##### bedrock\_supports\_effort

Whether this model emits `output_config.effort` on Bedrock Converse (Sonnet 4.6+, Opus 4.6+).

Only meaningful for the `'anthropic'` variant AND only honored alongside `bedrock_supports_adaptive_thinking=True`. Bedrock has not been verified to accept `output_config.effort` on the legacy `{'type': 'enabled', 'budget_tokens': N}` path (e.g. Opus 4.5), so the translator skips it there even though the direct Anthropic API accepts it. Effort lives at `additionalModelRequestFields.output_config.effort` (a sibling of `thinking`, not inside it).

**Type:** [`bool`](https://docs.python.org/3/library/functions.html#bool) **Default:** `False`

### BedrockProvider

**Bases:** `Provider[BaseClient]`

Provider for AWS Bedrock.

#### Methods

##### \_\_init\_\_

```python
def __init__(bedrock_client: BaseClient) -> None
def __init__(
    api_key: str,
    base_url: str | None = None,
    region_name: str | None = None,
    profile_name: str | None = None,
    aws_read_timeout: float | None = None,
    aws_connect_timeout: float | None = None,
) -> None
def __init__(
    aws_access_key_id: str | None = None,
    aws_secret_access_key: str | None = None,
    aws_session_token: str | None = None,
    base_url: str | None = None,
    region_name: str | None = None,
    profile_name: str | None = None,
    aws_read_timeout: float | None = None,
    aws_connect_timeout: float | None = None,
) -> None
```

Initialize the Bedrock provider.

###### Returns

[`None`](https://docs.python.org/3/library/constants.html#None)

###### Parameters

**`bedrock_client`** : `BaseClient` | [`None`](https://docs.python.org/3/library/constants.html#None) _Default:_ `None`

A boto3 client for Bedrock Runtime. If provided, other arguments are ignored.

**`aws_access_key_id`** : [`str`](https://docs.python.org/3/library/stdtypes.html#str) | [`None`](https://docs.python.org/3/library/constants.html#None) _Default:_ `None`

The AWS access key ID. If not set, the `AWS_ACCESS_KEY_ID` environment variable will be used if available.

**`aws_secret_access_key`** : [`str`](https://docs.python.org/3/library/stdtypes.html#str) | [`None`](https://docs.python.org/3/library/constants.html#None) _Default:_ `None`

The AWS secret access key. If not set, the `AWS_SECRET_ACCESS_KEY` environment variable will be used if available.

**`aws_session_token`** : [`str`](https://docs.python.org/3/library/stdtypes.html#str) | [`None`](https://docs.python.org/3/library/constants.html#None) _Default:_ `None`

The AWS session token. If not set, the `AWS_SESSION_TOKEN` environment variable will be used if available.

**`api_key`** : [`str`](https://docs.python.org/3/library/stdtypes.html#str) | [`None`](https://docs.python.org/3/library/constants.html#None) _Default:_ `None`

The API key for Bedrock client. Can be used instead of `aws_access_key_id`, `aws_secret_access_key`, and `aws_session_token`. If not set, the `AWS_BEARER_TOKEN_BEDROCK` environment variable will be used if available.

**`base_url`** : [`str`](https://docs.python.org/3/library/stdtypes.html#str) | [`None`](https://docs.python.org/3/library/constants.html#None) _Default:_ `None`

The base URL for the Bedrock client.

**`region_name`** : [`str`](https://docs.python.org/3/library/stdtypes.html#str) | [`None`](https://docs.python.org/3/library/constants.html#None) _Default:_ `None`

The AWS region name. If not set, the `AWS_DEFAULT_REGION` environment variable will be used if available.

**`profile_name`** : [`str`](https://docs.python.org/3/library/stdtypes.html#str) | [`None`](https://docs.python.org/3/library/constants.html#None) _Default:_ `None`

The AWS profile name.

**`aws_read_timeout`** : [`float`](https://docs.python.org/3/library/functions.html#float) | [`None`](https://docs.python.org/3/library/constants.html#None) _Default:_ `None`

The read timeout for Bedrock client.

**`aws_connect_timeout`** : [`float`](https://docs.python.org/3/library/functions.html#float) | [`None`](https://docs.python.org/3/library/constants.html#None) _Default:_ `None`

The connect timeout for Bedrock client.

### bedrock\_anthropic\_model\_profile

```python
def bedrock_anthropic_model_profile(model_name: str) -> ModelProfile | None
```

Get the model profile for an Anthropic model used via Bedrock.

#### Returns

[`ModelProfile`](/docs/ai/api/pydantic-ai/profiles/#pydantic_ai.profiles.ModelProfile) | [`None`](https://docs.python.org/3/library/constants.html#None)

### bedrock\_amazon\_model\_profile

```python
def bedrock_amazon_model_profile(model_name: str) -> ModelProfile | None
```

Get the model profile for an Amazon model used via Bedrock.

#### Returns

[`ModelProfile`](/docs/ai/api/pydantic-ai/profiles/#pydantic_ai.profiles.ModelProfile) | [`None`](https://docs.python.org/3/library/constants.html#None)

### bedrock\_deepseek\_model\_profile

```python
def bedrock_deepseek_model_profile(model_name: str) -> ModelProfile | None
```

Get the model profile for a DeepSeek model used via Bedrock.

#### Returns

[`ModelProfile`](/docs/ai/api/pydantic-ai/profiles/#pydantic_ai.profiles.ModelProfile) | [`None`](https://docs.python.org/3/library/constants.html#None)

### bedrock\_mistral\_model\_profile

```python
def bedrock_mistral_model_profile(model_name: str) -> ModelProfile | None
```

Get the model profile for a Mistral model used via Bedrock.

#### Returns

[`ModelProfile`](/docs/ai/api/pydantic-ai/profiles/#pydantic_ai.profiles.ModelProfile) | [`None`](https://docs.python.org/3/library/constants.html#None)

### bedrock\_qwen\_model\_profile

```python
def bedrock_qwen_model_profile(model_name: str) -> ModelProfile | None
```

Get the model profile for a Qwen model used via Bedrock.

#### Returns

[`ModelProfile`](/docs/ai/api/pydantic-ai/profiles/#pydantic_ai.profiles.ModelProfile) | [`None`](https://docs.python.org/3/library/constants.html#None)

### bedrock\_google\_model\_profile

```python
def bedrock_google_model_profile(model_name: str) -> ModelProfile | None
```

Get the model profile for a Google model used via Bedrock.

#### Returns

[`ModelProfile`](/docs/ai/api/pydantic-ai/profiles/#pydantic_ai.profiles.ModelProfile) | [`None`](https://docs.python.org/3/library/constants.html#None)

### bedrock\_minimax\_model\_profile

```python
def bedrock_minimax_model_profile(model_name: str) -> ModelProfile | None
```

Get the model profile for a MiniMax model used via Bedrock.

#### Returns

[`ModelProfile`](/docs/ai/api/pydantic-ai/profiles/#pydantic_ai.profiles.ModelProfile) | [`None`](https://docs.python.org/3/library/constants.html#None)

### bedrock\_nvidia\_model\_profile

```python
def bedrock_nvidia_model_profile(model_name: str) -> ModelProfile | None
```

Get the model profile for an NVIDIA model used via Bedrock.

#### Returns

[`ModelProfile`](/docs/ai/api/pydantic-ai/profiles/#pydantic_ai.profiles.ModelProfile) | [`None`](https://docs.python.org/3/library/constants.html#None)

### GroqProvider

**Bases:** `Provider[AsyncGroq]`

Provider for Groq API.

#### Methods

##### \_\_init\_\_

```python
def __init__(groq_client: AsyncGroq | None = None) -> None
def __init__(
    api_key: str | None = None,
    base_url: str | None = None,
    http_client: httpx.AsyncClient | None = None,
) -> None
```

Create a new Groq provider.

###### Returns

[`None`](https://docs.python.org/3/library/constants.html#None)

###### Parameters

**`api_key`** : [`str`](https://docs.python.org/3/library/stdtypes.html#str) | [`None`](https://docs.python.org/3/library/constants.html#None) _Default:_ `None`

The API key to use for authentication, if not provided, the `GROQ_API_KEY` environment variable will be used if available.

**`base_url`** : [`str`](https://docs.python.org/3/library/stdtypes.html#str) | [`None`](https://docs.python.org/3/library/constants.html#None) _Default:_ `None`

The base url for the Groq requests. If not provided, the `GROQ_BASE_URL` environment variable will be used if available. Otherwise, defaults to Groq's base url.

**`groq_client`** : `AsyncGroq` | [`None`](https://docs.python.org/3/library/constants.html#None) _Default:_ `None`

An existing [`AsyncGroq`](https://github.com/groq/groq-python?tab=readme-ov-file#async-usage) client to use. If provided, `api_key` and `http_client` must be `None`.

**`http_client`** : `httpx.AsyncClient` | [`None`](https://docs.python.org/3/library/constants.html#None) _Default:_ `None`

An existing `AsyncClient` to use for making HTTP requests.

### groq\_moonshotai\_model\_profile

```python
def groq_moonshotai_model_profile(model_name: str) -> ModelProfile | None
```

Get the model profile for an MoonshotAI model used with the Groq provider.

#### Returns

[`ModelProfile`](/docs/ai/api/pydantic-ai/profiles/#pydantic_ai.profiles.ModelProfile) | [`None`](https://docs.python.org/3/library/constants.html#None)

### meta\_groq\_model\_profile

```python
def meta_groq_model_profile(model_name: str) -> ModelProfile | None
```

Get the model profile for a Meta model used with the Groq provider.

#### Returns

[`ModelProfile`](/docs/ai/api/pydantic-ai/profiles/#pydantic_ai.profiles.ModelProfile) | [`None`](https://docs.python.org/3/library/constants.html#None)

### AzureProvider

**Bases:** `Provider[AsyncOpenAI]`

Provider for Azure OpenAI API.

See [https://azure.microsoft.com/en-us/products/ai-foundry](https://azure.microsoft.com/en-us/products/ai-foundry) for more information.

#### Methods

##### \_\_init\_\_

```python
def __init__(openai_client: AsyncAzureOpenAI) -> None
def __init__(
    azure_endpoint: str | None = None,
    api_version: str | None = None,
    api_key: str | None = None,
    http_client: httpx.AsyncClient | None = None,
) -> None
```

Create a new Azure provider.

###### Returns

[`None`](https://docs.python.org/3/library/constants.html#None)

###### Parameters

**`azure_endpoint`** : [`str`](https://docs.python.org/3/library/stdtypes.html#str) | [`None`](https://docs.python.org/3/library/constants.html#None) _Default:_ `None`

The Azure endpoint to use for authentication, if not provided, the `AZURE_OPENAI_ENDPOINT` environment variable will be used if available.

**`api_version`** : [`str`](https://docs.python.org/3/library/stdtypes.html#str) | [`None`](https://docs.python.org/3/library/constants.html#None) _Default:_ `None`

The API version to use for authentication, if not provided, the `OPENAI_API_VERSION` environment variable will be used if available. Not required (and not sent) when `azure_endpoint` targets the [Azure OpenAI v1 GA API](https://learn.microsoft.com/en-us/azure/ai-foundry/openai/api-version-lifecycle) (i.e. a path ending in `/v1`, such as `https://<resource>.openai.azure.com/openai/v1/`) or an Azure AI Foundry serverless model endpoint (`*.models.ai.azure.com`), both of which reject the `api-version` query parameter.

**`api_key`** : [`str`](https://docs.python.org/3/library/stdtypes.html#str) | [`None`](https://docs.python.org/3/library/constants.html#None) _Default:_ `None`

The API key to use for authentication, if not provided, the `AZURE_OPENAI_API_KEY` environment variable will be used if available.

**`openai_client`** : `AsyncAzureOpenAI` | [`None`](https://docs.python.org/3/library/constants.html#None) _Default:_ `None`

An existing [`AsyncAzureOpenAI`](https://github.com/openai/openai-python#microsoft-azure-openai) client to use. If provided, `base_url`, `api_key`, and `http_client` must be `None`.

**`http_client`** : `httpx.AsyncClient` | [`None`](https://docs.python.org/3/library/constants.html#None) _Default:_ `None`

An existing `httpx.AsyncClient` to use for making HTTP requests.

### CohereProvider

**Bases:** `Provider[AsyncClientV2]`

Provider for Cohere API.

#### Methods

##### \_\_init\_\_

```python
def __init__(
    api_key: str | None = None,
    cohere_client: AsyncClientV2 | None = None,
    http_client: httpx.AsyncClient | None = None,
) -> None
```

Create a new Cohere provider.

###### Returns

[`None`](https://docs.python.org/3/library/constants.html#None)

###### Parameters

**`api_key`** : [`str`](https://docs.python.org/3/library/stdtypes.html#str) | [`None`](https://docs.python.org/3/library/constants.html#None) _Default:_ `None`

The API key to use for authentication, if not provided, the `CO_API_KEY` environment variable will be used if available.

**`cohere_client`** : `AsyncClientV2` | [`None`](https://docs.python.org/3/library/constants.html#None) _Default:_ `None`

An existing [AsyncClientV2](https://github.com/cohere-ai/cohere-python) client to use. If provided, `api_key` and `http_client` must be `None`.

**`http_client`** : `httpx.AsyncClient` | [`None`](https://docs.python.org/3/library/constants.html#None) _Default:_ `None`

An existing `httpx.AsyncClient` to use for making HTTP requests.

### VoyageAIProvider

**Bases:** `Provider[AsyncClient]`

Provider for VoyageAI API.

#### Methods

##### \_\_init\_\_

```python
def __init__(voyageai_client: AsyncClient) -> None
def __init__(api_key: str | None = None) -> None
```

Create a new VoyageAI provider.

###### Returns

[`None`](https://docs.python.org/3/library/constants.html#None)

###### Parameters

**`api_key`** : [`str`](https://docs.python.org/3/library/stdtypes.html#str) | [`None`](https://docs.python.org/3/library/constants.html#None) _Default:_ `None`

The API key to use for authentication, if not provided, the `VOYAGE_API_KEY` environment variable will be used if available.

**`voyageai_client`** : `AsyncClient` | [`None`](https://docs.python.org/3/library/constants.html#None) _Default:_ `None`

An existing [AsyncClient](https://github.com/voyage-ai/voyageai-python) client to use. If provided, `api_key` must be `None`.

### CerebrasProvider

**Bases:** `Provider[AsyncOpenAI]`

Provider for Cerebras API.

#### Methods

##### \_\_init\_\_

```python
def __init__() -> None
def __init__(api_key: str) -> None
def __init__(api_key: str, http_client: httpx.AsyncClient) -> None
def __init__(http_client: httpx.AsyncClient) -> None
def __init__(openai_client: AsyncOpenAI | None = None) -> None
```

Create a new Cerebras provider.

###### Returns

[`None`](https://docs.python.org/3/library/constants.html#None)

###### Parameters

**`api_key`** : [`str`](https://docs.python.org/3/library/stdtypes.html#str) | [`None`](https://docs.python.org/3/library/constants.html#None) _Default:_ `None`

The API key to use for authentication, if not provided, the `CEREBRAS_API_KEY` environment variable will be used if available.

**`openai_client`** : `AsyncOpenAI` | [`None`](https://docs.python.org/3/library/constants.html#None) _Default:_ `None`

An existing `AsyncOpenAI` client to use. If provided, `api_key` and `http_client` must be `None`.

**`http_client`** : `httpx.AsyncClient` | [`None`](https://docs.python.org/3/library/constants.html#None) _Default:_ `None`

An existing `httpx.AsyncClient` to use for making HTTP requests.

### MistralProvider

**Bases:** `Provider[Mistral]`

Provider for Mistral API.

#### Methods

##### \_\_init\_\_

```python
def __init__(mistral_client: Mistral | None = None) -> None
def __init__(
    api_key: str | None = None,
    http_client: httpx.AsyncClient | None = None,
) -> None
```

Create a new Mistral provider.

###### Returns

[`None`](https://docs.python.org/3/library/constants.html#None)

###### Parameters

**`api_key`** : [`str`](https://docs.python.org/3/library/stdtypes.html#str) | [`None`](https://docs.python.org/3/library/constants.html#None) _Default:_ `None`

The API key to use for authentication, if not provided, the `MISTRAL_API_KEY` environment variable will be used if available.

**`mistral_client`** : `Mistral` | [`None`](https://docs.python.org/3/library/constants.html#None) _Default:_ `None`

An existing `Mistral` client to use, if provided, `api_key` and `http_client` must be `None`.

**`base_url`** : [`str`](https://docs.python.org/3/library/stdtypes.html#str) | [`None`](https://docs.python.org/3/library/constants.html#None) _Default:_ `None`

The base url for the Mistral requests.

**`http_client`** : `httpx.AsyncClient` | [`None`](https://docs.python.org/3/library/constants.html#None) _Default:_ `None`

An existing async client to use for making HTTP requests.

### FireworksProvider

**Bases:** `Provider[AsyncOpenAI]`

Provider for Fireworks AI API.

### GrokProvider

**Bases:** `Provider[AsyncOpenAI]`

Provider for Grok API (OpenAI-compatible interface).

### TogetherProvider

**Bases:** `Provider[AsyncOpenAI]`

Provider for Together AI API.

### HerokuProvider

**Bases:** `Provider[AsyncOpenAI]`

Provider for Heroku API.

### GitHubProvider

**Bases:** `Provider[AsyncOpenAI]`

Provider for GitHub Models API.

GitHub Models provides access to various AI models through an OpenAI-compatible API. See [https://docs.github.com/en/github-models](https://docs.github.com/en/github-models) for more information.

#### Methods

##### \_\_init\_\_

```python
def __init__() -> None
def __init__(api_key: str) -> None
def __init__(api_key: str, http_client: httpx.AsyncClient) -> None
def __init__(openai_client: AsyncOpenAI | None = None) -> None
```

Create a new GitHub Models provider.

###### Returns

[`None`](https://docs.python.org/3/library/constants.html#None)

###### Parameters

**`api_key`** : [`str`](https://docs.python.org/3/library/stdtypes.html#str) | [`None`](https://docs.python.org/3/library/constants.html#None) _Default:_ `None`

The GitHub token to use for authentication. If not provided, the `GITHUB_API_KEY` environment variable will be used if available.

**`openai_client`** : `AsyncOpenAI` | [`None`](https://docs.python.org/3/library/constants.html#None) _Default:_ `None`

An existing `AsyncOpenAI` client to use. If provided, `api_key` and `http_client` must be `None`.

**`http_client`** : `httpx.AsyncClient` | [`None`](https://docs.python.org/3/library/constants.html#None) _Default:_ `None`

An existing `httpx.AsyncClient` to use for making HTTP requests.

### OpenRouterProvider

**Bases:** `Provider[AsyncOpenAI]`

Provider for OpenRouter API.

#### Methods

##### \_\_init\_\_

```python
def __init__(openai_client: AsyncOpenAI) -> None
def __init__(
    api_key: str | None = None,
    app_url: str | None = None,
    app_title: str | None = None,
    openai_client: None = None,
    http_client: httpx.AsyncClient | None = None,
) -> None
```

Configure the provider with either an API key or prebuilt client.

###### Returns

[`None`](https://docs.python.org/3/library/constants.html#None)

###### Parameters

**`api_key`** : [`str`](https://docs.python.org/3/library/stdtypes.html#str) | [`None`](https://docs.python.org/3/library/constants.html#None) _Default:_ `None`

OpenRouter API key. Falls back to `OPENROUTER_API_KEY` when omitted and required unless `openai_client` is provided.

**`app_url`** : [`str`](https://docs.python.org/3/library/stdtypes.html#str) | [`None`](https://docs.python.org/3/library/constants.html#None) _Default:_ `None`

Optional url for app attribution. Falls back to `OPENROUTER_APP_URL` when omitted.

**`app_title`** : [`str`](https://docs.python.org/3/library/stdtypes.html#str) | [`None`](https://docs.python.org/3/library/constants.html#None) _Default:_ `None`

Optional title for app attribution. Falls back to `OPENROUTER_APP_TITLE` when omitted.

**`openai_client`** : `AsyncOpenAI` | [`None`](https://docs.python.org/3/library/constants.html#None) _Default:_ `None`

Existing `AsyncOpenAI` client to reuse instead of creating one internally.

**`http_client`** : `httpx.AsyncClient` | [`None`](https://docs.python.org/3/library/constants.html#None) _Default:_ `None`

Custom `httpx.AsyncClient` to pass into the `AsyncOpenAI` constructor when building a client.

###### Raises

-   `UserError` -- If no API key is available and no `openai_client` is provided.

### VercelProvider

**Bases:** `Provider[AsyncOpenAI]`

Provider for Vercel AI Gateway API.

### HuggingFaceProvider

**Bases:** `Provider[AsyncInferenceClient]`

Provider for Hugging Face.

#### Methods

##### \_\_init\_\_

```python
def __init__(base_url: str, api_key: str | None = None) -> None
def __init__(provider_name: str, api_key: str | None = None) -> None
def __init__(hf_client: AsyncInferenceClient, api_key: str | None = None) -> None
def __init__(
    hf_client: AsyncInferenceClient,
    base_url: str,
    api_key: str | None = None,
) -> None
def __init__(
    hf_client: AsyncInferenceClient,
    provider_name: str,
    api_key: str | None = None,
) -> None
def __init__(api_key: str | None = None) -> None
```

Create a new Hugging Face provider.

###### Returns

[`None`](https://docs.python.org/3/library/constants.html#None)

###### Parameters

**`base_url`** : [`str`](https://docs.python.org/3/library/stdtypes.html#str) | [`None`](https://docs.python.org/3/library/constants.html#None) _Default:_ `None`

The base url for the Hugging Face requests.

**`api_key`** : [`str`](https://docs.python.org/3/library/stdtypes.html#str) | [`None`](https://docs.python.org/3/library/constants.html#None) _Default:_ `None`

The API key to use for authentication, if not provided, the `HF_TOKEN` environment variable will be used if available.

**`hf_client`** : `AsyncInferenceClient` | [`None`](https://docs.python.org/3/library/constants.html#None) _Default:_ `None`

An existing [`AsyncInferenceClient`](https://huggingface.co/docs/huggingface_hub/en/package_reference/inference_client#huggingface_hub.AsyncInferenceClient) client to use. If not provided, a new instance will be created.

**`http_client`** : `AsyncClient` | [`None`](https://docs.python.org/3/library/constants.html#None) _Default:_ `None`

(currently ignored) An existing `httpx.AsyncClient` to use for making HTTP requests.

**`provider_name`** : [`str`](https://docs.python.org/3/library/stdtypes.html#str) | [`None`](https://docs.python.org/3/library/constants.html#None) _Default:_ `None`

Name of the provider to use for inference. available providers can be found in the [HF Inference Providers documentation](https://huggingface.co/docs/inference-providers/index#partners). defaults to "auto", which will select the first available provider for the model, the first of the providers available for the model, sorted by the user's order in [https://hf.co/settings/inference-providers](https://hf.co/settings/inference-providers). If `base_url` is passed, then `provider_name` is not used.

### MoonshotAIProvider

**Bases:** `Provider[AsyncOpenAI]`

Provider for MoonshotAI platform (Kimi models).

### OllamaProvider

**Bases:** `Provider[AsyncOpenAI]`

Provider for local or remote Ollama API.

#### Methods

##### \_\_init\_\_

```python
def __init__(
    base_url: str | None = None,
    api_key: str | None = None,
    openai_client: AsyncOpenAI | None = None,
    http_client: httpx.AsyncClient | None = None,
) -> None
```

Create a new Ollama provider.

###### Returns

[`None`](https://docs.python.org/3/library/constants.html#None)

###### Parameters

**`base_url`** : [`str`](https://docs.python.org/3/library/stdtypes.html#str) | [`None`](https://docs.python.org/3/library/constants.html#None) _Default:_ `None`

The base url for the Ollama requests. If not provided, the `OLLAMA_BASE_URL` environment variable will be used if available.

**`api_key`** : [`str`](https://docs.python.org/3/library/stdtypes.html#str) | [`None`](https://docs.python.org/3/library/constants.html#None) _Default:_ `None`

The API key to use for authentication, if not provided, the `OLLAMA_API_KEY` environment variable will be used if available.

**`openai_client`** : `AsyncOpenAI` | [`None`](https://docs.python.org/3/library/constants.html#None) _Default:_ `None`

An existing [`AsyncOpenAI`](https://github.com/openai/openai-python?tab=readme-ov-file#async-usage) client to use. If provided, `base_url`, `api_key`, and `http_client` must be `None`.

**`http_client`** : `httpx.AsyncClient` | [`None`](https://docs.python.org/3/library/constants.html#None) _Default:_ `None`

An existing `httpx.AsyncClient` to use for making HTTP requests.

### LiteLLMProvider

**Bases:** `Provider[AsyncOpenAI]`

Provider for LiteLLM API.

#### Methods

##### \_\_init\_\_

```python
def __init__(api_key: str | None = None, api_base: str | None = None) -> None
def __init__(
    api_key: str | None = None,
    api_base: str | None = None,
    http_client: AsyncHTTPClient,
) -> None
def __init__(openai_client: AsyncOpenAI) -> None
```

Initialize a LiteLLM provider.

###### Returns

[`None`](https://docs.python.org/3/library/constants.html#None)

###### Parameters

**`api_key`** : [`str`](https://docs.python.org/3/library/stdtypes.html#str) | [`None`](https://docs.python.org/3/library/constants.html#None) _Default:_ `None`

API key for the model provider. If None, LiteLLM will try to get it from environment variables.

**`api_base`** : [`str`](https://docs.python.org/3/library/stdtypes.html#str) | [`None`](https://docs.python.org/3/library/constants.html#None) _Default:_ `None`

Base URL for the model provider. Use this for custom endpoints or self-hosted models.

**`openai_client`** : `AsyncOpenAI` | [`None`](https://docs.python.org/3/library/constants.html#None) _Default:_ `None`

Pre-configured OpenAI client. If provided, other parameters are ignored.

**`http_client`** : `AsyncHTTPClient` | [`None`](https://docs.python.org/3/library/constants.html#None) _Default:_ `None`

Custom HTTP client to use.

### NebiusProvider

**Bases:** `Provider[AsyncOpenAI]`

Provider for Nebius AI Studio API.

### OVHcloudProvider

**Bases:** `Provider[AsyncOpenAI]`

Provider for OVHcloud AI Endpoints.

### AlibabaProvider

**Bases:** `Provider[AsyncOpenAI]`

Provider for Alibaba Cloud Model Studio (DashScope) OpenAI-compatible API.

### SambaNovaProvider

**Bases:** `Provider[AsyncOpenAI]`

Provider for SambaNova AI models.

SambaNova uses an OpenAI-compatible API.

#### Attributes

##### name

Return the provider name.

**Type:** [`str`](https://docs.python.org/3/library/stdtypes.html#str)

##### base\_url

Return the base URL.

**Type:** [`str`](https://docs.python.org/3/library/stdtypes.html#str)

##### client

Return the AsyncOpenAI client.

**Type:** `AsyncOpenAI`

#### Methods

##### model\_profile

`@staticmethod`

```python
def model_profile(model_name: str) -> ModelProfile | None
```

Get model profile for SambaNova models.

SambaNova serves models from multiple families including Meta Llama, DeepSeek, Qwen, and Mistral. Model profiles are matched based on model name prefixes.

###### Returns

[`ModelProfile`](/docs/ai/api/pydantic-ai/profiles/#pydantic_ai.profiles.ModelProfile) | [`None`](https://docs.python.org/3/library/constants.html#None)

##### \_\_init\_\_

```python
def __init__(
    api_key: str | None = None,
    base_url: str | None = None,
    openai_client: AsyncOpenAI | None = None,
    http_client: httpx.AsyncClient | None = None,
) -> None
```

Initialize SambaNova provider.

###### Returns

[`None`](https://docs.python.org/3/library/constants.html#None)

###### Parameters

**`api_key`** : [`str`](https://docs.python.org/3/library/stdtypes.html#str) | [`None`](https://docs.python.org/3/library/constants.html#None) _Default:_ `None`

SambaNova API key. If not provided, reads from SAMBANOVA\_API\_KEY env var.

**`base_url`** : [`str`](https://docs.python.org/3/library/stdtypes.html#str) | [`None`](https://docs.python.org/3/library/constants.html#None) _Default:_ `None`

Custom API base URL. Defaults to [https://api.sambanova.ai/v1](https://api.sambanova.ai/v1)

**`openai_client`** : `AsyncOpenAI` | [`None`](https://docs.python.org/3/library/constants.html#None) _Default:_ `None`

Optional pre-configured OpenAI client

**`http_client`** : `httpx.AsyncClient` | [`None`](https://docs.python.org/3/library/constants.html#None) _Default:_ `None`

Optional custom httpx.AsyncClient for making HTTP requests

###### Raises

-   `UserError` -- If API key is not provided and SAMBANOVA\_API\_KEY env var is not set