pydantic_ai.providers
Bases: ABC, Generic[InterfaceClient]
Abstract class for a provider.
The provider is in charge of providing an authenticated client to the API.
Each provider only supports a specific interface. An interface can be supported by multiple providers.
For example, the OpenAIChatModel interface can be supported by the OpenAIProvider and the DeepSeekProvider.
When used as an async context manager, providers that create their own HTTP client will close it on exit.
This is handled automatically when using Agent as a context manager.
Outside one, a run on a new event loop, while the one the client was last used on isn’t running, gets a new client.
The provider name.
The returned value flows into ModelMessage.provider_name
on every part. Thinking-tag detection and native-tool detection check this value when
the model class loads history, so silently renaming a concrete name value breaks
replay of any message history captured against the old name.
Type: str
The namespace used to fully qualify model IDs routed through this provider.
Type: str
The base URL for the provider API.
Type: str
The client for the provider.
Type: InterfaceClient
@staticmethod
def model_profile(model_name: str) -> ModelProfile | None
The model profile for the named model, if available.
@staticmethod
def realtime_model_profile(model_name: str) -> RealtimeModelProfile | None
The realtime model profile for the named model, if available.
RealtimeModelProfile | None
def infer_provider(provider: str) -> Provider[Any]
Infer the provider from the provider name.
Provider[Any]
def infer_provider_class(provider: str) -> type[Provider[Any]]
Infers the provider class from the provider name.
def gateway_provider(
upstream_provider: Literal['openai', 'openai-chat', 'openai-responses', 'chat', 'responses'],
/,
*,
route: str | None = None,
api_key: str | None = None,
base_url: str | None = None,
http_client: AsyncHTTPClient | None = None,
) -> Provider[AsyncOpenAI]
def gateway_provider(
upstream_provider: Literal['groq'],
/,
*,
route: str | None = None,
api_key: str | None = None,
base_url: str | None = None,
http_client: httpx.AsyncClient | None = None,
) -> Provider[AsyncGroq]
def gateway_provider(
upstream_provider: Literal['anthropic'],
/,
*,
route: str | None = None,
api_key: str | None = None,
base_url: str | None = None,
http_client: httpx2.AsyncClient | None = None,
) -> Provider[AsyncAnthropicClient]
def gateway_provider(
upstream_provider: Literal['bedrock', 'converse'],
/,
*,
route: str | None = None,
api_key: str | None = None,
base_url: str | None = None,
) -> Provider[BaseClient]
def gateway_provider(
upstream_provider: Literal['google', 'google-cloud'],
/,
*,
route: str | None = None,
api_key: str | None = None,
base_url: str | None = None,
http_client: AsyncHTTPClient | None = None,
) -> Provider[GoogleClient]
def gateway_provider(
upstream_provider: str,
/,
*,
route: str | None = None,
api_key: str | None = None,
base_url: str | None = None,
) -> Provider[Any]
Create a new Gateway provider.
Provider[Any]
upstream_provider : UpstreamProvider | str
The upstream provider to use.
The name of the provider or gateway endpoint to use to handle the request. If not provided, the default gateway endpoint for the API format will be used.
The API key to use for authentication. If not provided, the PYDANTIC_AI_GATEWAY_API_KEY
environment variable will be used if available.
The base URL to use for the Gateway. If not provided, the PYDANTIC_AI_GATEWAY_BASE_URL
environment variable will be used if available. Otherwise, it is inferred from the API key’s
region, e.g. https://gateway-us.pydantic.dev/proxy.
http_client : AsyncHTTPClient | None Default: None
The HTTP client to use for the Gateway.
Bases: Provider[AsyncAnthropicClient]
Provider for Anthropic API.
def __init__(*, anthropic_client: AsyncAnthropicClient | None = None) -> None
def __init__(
*,
api_key: str | None = None,
base_url: str | None = None,
http_client: httpx2.AsyncClient | None = None,
) -> None
Create a new Anthropic provider.
The API key to use for authentication, if not provided, the ANTHROPIC_API_KEY environment variable
will be used if available.
The base URL to use for the Anthropic API.
anthropic_client : AsyncAnthropicClient | None Default: None
An existing Anthropic client to use. Accepts
AsyncAnthropic,
AsyncAnthropicBedrock,
AsyncAnthropicBedrockMantle,
AsyncAnthropicFoundry, or
AsyncAnthropicVertex.
If provided, the api_key and http_client arguments will be ignored.
http_client : httpx2.AsyncClient | None Default: None
An existing httpx2.AsyncClient to use for making HTTP requests.
Bases: Provider[Client], ABC
Common base for the Gemini API and Google Cloud providers.
Abstract — instantiate GoogleProvider for the
Gemini API or GoogleCloudProvider for
Google Cloud. Subclasses share base_url, client, _set_http_client, and model-profile
lookup; each subclass owns its own Client construction.
Bases: BaseGoogleProvider
Provider for the Gemini API (formerly Google AI Studio / Google GLA).
def __init__(
*,
api_key: str | None = None,
http_client: AsyncHTTPClient | None = None,
base_url: str | None = None,
retry_options: HttpRetryOptions | None = None,
) -> None
def __init__(*, client: Client) -> None
Create a new Google provider for the Gemini API.
The API key to
use for authentication. It can also be set via the GOOGLE_API_KEY environment variable,
or the legacy GEMINI_API_KEY environment variable (GOOGLE_API_KEY takes precedence).
client : Client | None Default: None
A pre-initialized client to use. Stored as-is, so a client built for Google Cloud
(Client(vertexai=True, ...), or its current spelling Client(enterprise=True, ...))
is used on that transport even though name stays 'google'. GoogleModel reads the
transport off the client rather than the provider name, so it routes correctly either way.
http_client : AsyncHTTPClient | None Default: None
An existing httpx2.AsyncClient or legacy httpx.AsyncClient to use for making HTTP requests.
The base URL for the Gemini API.
retry_options : HttpRetryOptions | None Default: None
HTTP retry options for transient errors (429, 5xx, etc.).
See google.genai.types.HttpRetryOptions for available fields.
Regions available for Google Cloud. More details here.
This lists single-region values only. GoogleCloudProvider also accepts the 'global' location and the
'us'/'eu' multi-regions (routed to the aiplatform.{us,eu}.rep.googleapis.com data-residency endpoints)
as separate union members on its location parameter.
Default: Literal['asia-east1', 'asia-east2', 'asia-northeast1', 'asia-northeast3', 'asia-south1', 'asia-southeast1', 'australia-southeast1', 'europe-central2', 'europe-north1', 'europe-southwest1', 'europe-west1', 'europe-west2', 'europe-west3', 'europe-west4', 'europe-west6', 'europe-west8', 'europe-west9', 'me-central1', 'me-central2', 'me-west1', 'northamerica-northeast1', 'southamerica-east1', 'us-central1', 'us-east1', 'us-east4', 'us-east5', 'us-south1', 'us-west1', 'us-west4']
Bases: BaseGoogleProvider
Provider for Google Cloud (formerly known as Vertex AI).
def __init__(
*,
api_key: str | None = None,
credentials: Credentials | None = None,
project: str | None = None,
location: GoogleCloudLocation | Literal['global', 'us', 'eu'] | str | None = None,
client: Client | None = None,
http_client: AsyncHTTPClient | None = None,
base_url: str | None = None,
retry_options: HttpRetryOptions | None = None,
) -> None
Create a new Google Cloud provider.
The Vertex AI Express Mode API key
to use for authentication. It can also be set via the GOOGLE_API_KEY environment variable,
or the legacy GEMINI_API_KEY environment variable (GOOGLE_API_KEY takes precedence).
Explicit credentials use credential-based authentication instead.
Explicit project/location use Application Default Credentials.
GOOGLE_APPLICATION_CREDENTIALS takes precedence over an API key from the environment.
credentials : Credentials | None Default: None
The credentials to use for authentication when calling the Google Cloud APIs. Credentials can
be obtained from environment variables and default credentials. For more information, see
Set up Application Default Credentials.
Credentials that require scopes are automatically scoped with
https://www.googleapis.com/auth/cloud-platform.
The Google Cloud project ID to use for quota. Can be obtained from environment variables
(for example, GOOGLE_CLOUD_PROJECT).
The location to send API requests to, for example us-central1 (a single region) or
global. 'us' and 'eu' are multi-region values routed to the aiplatform.{us,eu}.rep.googleapis.com
data-residency endpoints. Model availability differs between single regions, multi-regions, and
global — see the
Vertex AI locations docs.
Can be obtained from the GOOGLE_CLOUD_LOCATION environment variable.
client : Client | None Default: None
A pre-initialized client to use. Stored as-is without forcing vertexai=True, so a
client built for the Gemini Developer API is used on that transport even though name
stays 'google-cloud'. GoogleModel reads the transport off the client rather than the
provider name, so it routes correctly either way.
http_client : AsyncHTTPClient | None Default: None
An existing httpx2.AsyncClient or legacy httpx.AsyncClient to use for making HTTP requests.
The base URL for the Google Cloud API.
retry_options : HttpRetryOptions | None Default: None
HTTP retry options for transient errors (429, 5xx, etc.).
See google.genai.types.HttpRetryOptions for available fields.
Bases: _OpenAICompatibleProvider
Provider for OpenAI API.
Changing base_url does not change the OpenAI model profile selection. For another service,
use its dedicated provider when available. For a custom gateway, subclass this provider and
override model_profile, or configure the model’s profile explicitly.
See OpenAI-compatible models.
def __init__(*, openai_client: AsyncOpenAI) -> None
def __init__(
base_url: str | None = None,
api_key: str | None = None,
openai_client: None = None,
http_client: _OpenAIHTTPClient | None = None,
) -> None
Create a new OpenAI provider.
The base url for the OpenAI requests. If not provided, the OPENAI_BASE_URL environment variable
will be used if available. Otherwise, defaults to OpenAI’s base url.
The API key to use for authentication, if not provided, the OPENAI_API_KEY environment variable
will be used if available.
openai_client : AsyncOpenAI | None Default: None
An existing
AsyncOpenAI
client to use. If provided, base_url, api_key, and http_client must be None.
http_client : _OpenAIHTTPClient | None Default: None
An existing httpx2.AsyncClient or legacy httpx.AsyncClient to use for making HTTP requests.
OpenAI Codex subscription-auth provider: OAuth flow primitives, credential refresh, wire dialect.
Core owns the protocol primitives (PKCE context, authorization URL, code exchange, refresh) and,
because the pinned redirect URI makes every login a localhost callback, the one-shot callback
catcher (exchange_code_from_callback()); browser opening and persistent credential storage
belong to applications and harnesses.
The authorization-code + PKCE redirect flow is the only login flow the public Codex client
supports: its registration pins the redirect URI to http://localhost:1455/auth/callback
(exact-match, probed live 2026-08-25), and the auth service serves no device-authorization
endpoint. exchange_code_from_callback() serves this exact redirect URI and exchanges the
authorization code.
Bases: _CredentialsError
Exchanging an authorization code or refresh token at the token endpoint failed.
When the underlying error is invalid_grant, the stored grant is no longer usable and a fresh
authorization is required (for Codex locally: rerun codex login; in an app: rerun your sign-in flow).
The error message.
Type: str Default: message
The name of the model associated with the error.
Type: str Default: model_name
The body of the provider’s error response or error event, if available.
Type: object | None Default: body
The provider’s machine-readable error code, if it sent one.
For example 'rate_limit_exceeded' or 'context_length_exceeded' (OpenAI), 'ThrottlingException' (Bedrock),
'RESOURCE_EXHAUSTED' (Google, xAI), or '3051' (Mistral).
Type: str | None Default: provider_error_code
The provider’s error type, if it sent one separately from the code.
For example 'overloaded_error' or 'invalid_request_error' (Anthropic, OpenAI).
Type: str | None Default: provider_error_type
Seconds the provider asked you to wait before retrying, if it said.
An HTTP error reads it from the retry-after-ms or Retry-After response header.
Type: float | None Default: retry_after
Whether the provider reported the error inside a response stream that had already started successfully.
Its status_code, if any, is then the status the provider
uses for the same error before a stream opens, rather than the stream’s own, and its
headers, if any, are the stream’s.
Type: bool Default: in_stream
Bases: _CredentialsError, OpenAIError
Rotated credentials were updated in memory but the persistence callback raised.
The in-memory credentials are current and were handed to the callback before it failed; the error surfaces so callers do not mistake a failed save for durability.
The error message.
Type: str Default: message
The name of the model associated with the error.
Type: str Default: model_name
The body of the provider’s error response or error event, if available.
Type: object | None Default: body
The provider’s machine-readable error code, if it sent one.
For example 'rate_limit_exceeded' or 'context_length_exceeded' (OpenAI), 'ThrottlingException' (Bedrock),
'RESOURCE_EXHAUSTED' (Google, xAI), or '3051' (Mistral).
Type: str | None Default: provider_error_code
The provider’s error type, if it sent one separately from the code.
For example 'overloaded_error' or 'invalid_request_error' (Anthropic, OpenAI).
Type: str | None Default: provider_error_type
Seconds the provider asked you to wait before retrying, if it said.
An HTTP error reads it from the retry-after-ms or Retry-After response header.
Type: float | None Default: retry_after
Whether the provider reported the error inside a response stream that had already started successfully.
Its status_code, if any, is then the status the provider
uses for the same error before a stream opens, rather than the stream’s own, and its
headers, if any, are the stream’s.
Type: bool Default: in_stream
Codex subscription credentials.
The tokens are excluded from repr so a logged or traceback-embedded instance does not leak them.
@classmethod
def from_codex_cli_auth(cls, data: Mapping[str, Any]) -> Self
Parse the Codex CLI ~/.codex/auth.json shape ({'tokens': {...}}).
Bases: OAuthCredentialSource[OpenAICodexCredentials], Protocol
Application-owned storage for the credentials, so refreshed tokens outlive the process.
The provider owns the credential lifecycle (expiry checks, refresh, single-flight) and calls
this only to read and write the stored set: load() on first use, and save() after tokens
are refreshed.
Refresh tokens are single-use, so before refreshing, the provider re-reads storage with
load() and adopts newer credentials if another process using the same store already
refreshed them. That is best-effort, not mutual exclusion: two processes can still race
between load() and save().
Conformance is structural, but implementations are encouraged to subclass the protocol explicitly so type checkers verify the method signatures.
@async
def load() -> CredentialsT
Return the currently stored credentials.
CredentialsT
@async
def save(credentials: CredentialsT) -> None
Durably replace the stored credentials with a freshly rotated set.
Bases: OAuthFlow[OpenAICodexCredentials]
Pure authorization-code + PKCE context for the OpenAI Codex public client.
This is the only login flow the public client supports (no device flow; redirect URI pinned to
localhost:1455, probed exact-match). Construction does no I/O: build the context anywhere,
send the user to authorization_url(), then let exchange_code_from_callback() receive the
redirect on localhost and exchange its code. The browser and credential storage stay
caller-owned.
The S256 PKCE challenge derived from code_verifier.
Type: str
@async
def exchange_code_from_callback() -> CredentialsT
Serve redirect_uri for one authorization callback, then exchange the received code.
Binds the host and port from redirect_uri with a one-shot local HTTP server, ignores
requests that don’t carry this flow’s state, and raises
UserError when the provider reports an authorization
error instead of a code (e.g. the user clicked Deny). Callers wanting a time limit can wrap
the call in anyio.fail_after.
CredentialsT
def __init__(*, redirect_uri: str = _REDIRECT_URI, state: str | None = None) -> None
Create a new flow context. Construction does no I/O.
redirect_uri : str Default: _REDIRECT_URI
Where the authorization code is delivered. The public client’s
registration pins this to http://localhost:1455/auth/callback (exact-match),
so leave the default unchanged when using the public client.
The CSRF token bound to the callback; auto-generated when None.
def authorization_url(
*,
scope: str | None = None,
extra_params: Mapping[str, str] | None = None,
) -> str
The URL to send the user to. Note the public client pins redirects to localhost.
The OAuth scopes to request; None means the standard Codex login scopes.
Additional query parameters, merged over the defaults (so they can also
override them), except client_id and redirect_uri: exchange_code() always
posts the public client id and the flow’s redirect_uri, so overriding either
here would make the authorization code unusable. The production Codex login’s
id_token_add_organizations=true and codex_cli_simplified_flow=true are sent by
default: without the former, the id_token can omit the account id for multi-org
accounts (live-verified 2026-08-25).
@async
def exchange_code(code: str) -> OpenAICodexCredentials
Exchange an authorization code for credentials (call this in your callback handler).
OpenAICodexCredentials
Bases: OAuthCredentialsProvider[OpenAICodexCredentials]
Provider for OpenAI Codex subscription authentication.
Wraps the standard OpenAIProvider machinery pointed at the Codex backend, injecting Codex
OAuth credentials instead of API keys. One provider instance carries one set of credentials
(there is no process-global cache). The instance binds its refresh lock to the first event
loop that awaits a request, so do not reuse it across loops.
provider = OpenAICodexProvider(credential_source=YourCredentialStore())
agent = Agent('openai-codex:gpt-5.6-luna', provider=provider)
The namespace used to fully qualify model IDs routed through this provider.
Type: str
The credentials currently held in memory, rotated in place by refreshes.
Type: CredentialsT
@staticmethod
def realtime_model_profile(model_name: str) -> RealtimeModelProfile | None
The realtime model profile for the named model, if available.
RealtimeModelProfile | None
def __init__(
credentials: OpenAICodexCredentials | None = None,
*,
credential_source: OpenAICodexCredentialSource | None = None,
openai_client: AsyncOpenAI | None = None,
http_client: _OpenAIHTTPClient | None = None,
) -> None
Create a new OpenAI Codex provider.
credentials : OpenAICodexCredentials | None Default: None
The subscription credentials to inject. If both this and
credential_source are omitted, they are loaded read-only from the Codex
CLI’s auth.json (honors CODEX_HOME), which never writes the file: refreshed
tokens then live in memory only. Pydantic AI never falls back to OPENAI_API_KEY.
credential_source : OpenAICodexCredentialSource | None Default: None
Application-owned storage for the credentials, so refreshed tokens
are persisted between runs; see
OpenAICodexCredentialSource.
Mutually exclusive with credentials.
openai_client : AsyncOpenAI | None Default: None
An existing AsyncOpenAI client to use as-is. Opts out of credential
injection entirely; credentials, credential_source, and http_client must
be None.
http_client : _OpenAIHTTPClient | None Default: None
An existing httpx2.AsyncClient to use. Must be dedicated to this
provider (no auth of its own): the provider attaches its credential-injecting auth to it, and
sharing a client between providers would mix their credentials. The auth only
injects credentials on HTTPS requests to the Codex host, so the client can safely
be reused for other destinations.
ChatGPT plan usage through Sign in with ChatGPT: OAuth flow, credential refresh, and provider.
OpenAI lets open-source and locally run apps use a signed-in user’s ChatGPT plan for Responses API
requests, as documented in ChatGPT plan usage.
The first sign-in registers an OAuth client for the user’s ChatGPT account through dynamic registration;
later sign-ins reuse the issued client_id. Every sign-in redirects to an HTTP loopback callback on
127.0.0.1, so core owns the one-shot callback catcher (exchange_code_from_callback()); browser
opening and persistent credential storage belong to applications.
Bases: _CredentialsError
Exchanging an authorization code or refresh token at the token endpoint failed.
When the underlying error is invalid_grant, the stored grant is no longer usable and a fresh
authorization is required (for Codex locally: rerun codex login; in an app: rerun your sign-in flow).
The error message.
Type: str Default: message
The name of the model associated with the error.
Type: str Default: model_name
The body of the provider’s error response or error event, if available.
Type: object | None Default: body
The provider’s machine-readable error code, if it sent one.
For example 'rate_limit_exceeded' or 'context_length_exceeded' (OpenAI), 'ThrottlingException' (Bedrock),
'RESOURCE_EXHAUSTED' (Google, xAI), or '3051' (Mistral).
Type: str | None Default: provider_error_code
The provider’s error type, if it sent one separately from the code.
For example 'overloaded_error' or 'invalid_request_error' (Anthropic, OpenAI).
Type: str | None Default: provider_error_type
Seconds the provider asked you to wait before retrying, if it said.
An HTTP error reads it from the retry-after-ms or Retry-After response header.
Type: float | None Default: retry_after
Whether the provider reported the error inside a response stream that had already started successfully.
Its status_code, if any, is then the status the provider
uses for the same error before a stream opens, rather than the stream’s own, and its
headers, if any, are the stream’s.
Type: bool Default: in_stream
Bases: _CredentialsError, OpenAIError
Rotated credentials were updated in memory but the persistence callback raised.
The in-memory credentials are current and were handed to the callback before it failed; the error surfaces so callers do not mistake a failed save for durability.
The error message.
Type: str Default: message
The name of the model associated with the error.
Type: str Default: model_name
The body of the provider’s error response or error event, if available.
Type: object | None Default: body
The provider’s machine-readable error code, if it sent one.
For example 'rate_limit_exceeded' or 'context_length_exceeded' (OpenAI), 'ThrottlingException' (Bedrock),
'RESOURCE_EXHAUSTED' (Google, xAI), or '3051' (Mistral).
Type: str | None Default: provider_error_code
The provider’s error type, if it sent one separately from the code.
For example 'overloaded_error' or 'invalid_request_error' (Anthropic, OpenAI).
Type: str | None Default: provider_error_type
Seconds the provider asked you to wait before retrying, if it said.
An HTTP error reads it from the retry-after-ms or Retry-After response header.
Type: float | None Default: retry_after
Whether the provider reported the error inside a response stream that had already started successfully.
Its status_code, if any, is then the status the provider
uses for the same error before a stream opens, rather than the stream’s own, and its
headers, if any, are the stream’s.
Type: bool Default: in_stream
Credentials for using one ChatGPT account’s plan, tied to the OAuth client registered for it.
Persist every field together: client_id identifies the registration to reuse on the next sign-in,
id_token is passed back as that sign-in’s id_token_hint, and scope records whether the user
granted plan usage. The tokens are excluded from repr so a logged or traceback-embedded instance
does not leak them.
The OAuth client id OpenAI issued when this ChatGPT account first signed in.
Type: str
The ID token from the latest sign-in or refresh, to pass as id_token_hint when signing in again.
Type: str | None Default: field(default=None, repr=False)
The space-separated scopes the user granted.
Type: str Default: ''
Whether the user granted the chatgpt.tokens.use.direct scope, which inference requires.
Type: bool
Bases: OAuthCredentialSource[OpenAIChatGPTCredentials], Protocol
Application-owned storage for the credentials, so refreshed tokens outlive the process.
The provider owns the credential lifecycle (expiry checks, refresh, single-flight) and calls
this only to read and write the stored set: load() on first use, and save() after tokens
are refreshed.
Refresh tokens are single-use, so before refreshing, the provider re-reads storage with
load() and adopts newer credentials if another process using the same store already
refreshed them. That is best-effort, not mutual exclusion: two processes can still race
between load() and save().
Conformance is structural, but implementations are encouraged to subclass the protocol explicitly so type checkers verify the method signatures.
@async
def load() -> CredentialsT
Return the currently stored credentials.
CredentialsT
@async
def save(credentials: CredentialsT) -> None
Durably replace the stored credentials with a freshly rotated set.
Bases: OAuthFlow[OpenAIChatGPTCredentials]
Authorization-code + PKCE context for Sign in with ChatGPT with plan usage.
Without a client_id, the sign-in registers a new OAuth client for the user’s ChatGPT account; the
resulting credentials carry the issued client_id, which later sign-ins to the same account pass back
so they reuse the registration. Construction does no I/O: build the context anywhere, send the user to
authorization_url(), then let exchange_code_from_callback() receive the redirect on 127.0.0.1 and
exchange its code. The browser and credential storage stay caller-owned.
The S256 PKCE challenge derived from code_verifier.
Type: str
@async
def exchange_code_from_callback() -> CredentialsT
Serve redirect_uri for one authorization callback, then exchange the received code.
Binds the host and port from redirect_uri with a one-shot local HTTP server, ignores
requests that don’t carry this flow’s state, and raises
UserError when the provider reports an authorization
error instead of a code (e.g. the user clicked Deny). Callers wanting a time limit can wrap
the call in anyio.fail_after.
CredentialsT
def __init__(
*,
app_name: str,
host_id: str | None = None,
client_id: str | None = None,
id_token_hint: str | None = None,
redirect_uri: str = _REDIRECT_URI,
state: str | None = None,
) -> None
Create a new flow context. Construction does no I/O.
app_name : str
Your app’s name, shown to the user when a new registration is approved. Use the same name across installations; it is only sent when registering.
A stable, opaque identifier for this installation of your app, created once and
persisted, such as uuid.uuid4().urn. OpenAI also accepts RFC 9278 JWK thumbprint URIs
and did:key identifiers.
The client_id of credentials from an earlier sign-in to the same ChatGPT account,
to reuse its registration. None registers a new client.
The id_token of credentials from an earlier sign-in, so the user skips
choosing the account. The new sign-in must then complete for the same account.
redirect_uri : str Default: _REDIRECT_URI
The loopback callback the authorization code is delivered to. OpenAI requires
http://127.0.0.1:PORT/auth/callback; only the port may differ from the default.
The CSRF token bound to the callback; auto-generated when None.
def authorization_url(
*,
scope: str | None = None,
extra_params: Mapping[str, str] | None = None,
) -> str
The URL to send the user to.
The URL carries id_token_hint when one was given, so keep it out of logs.
The OAuth scopes to request; None means the identity scopes plus plan usage. Keep
offline_access and chatgpt.tokens.use.direct: the credentials need a refresh token,
and the provider refuses credentials without plan usage.
Additional query parameters, merged over the defaults (so they can also
override them), except client_id, redirect_uri, state, code_challenge, nonce,
and resource, which the callback validation and code exchange rely on. Pass
{'prompt': 'consent'} to ask a user who declined plan usage again, or
{'login_hint': email} to preselect an account.
@async
def exchange_code(
code: str,
*,
client_id: str | None = None,
) -> OpenAIChatGPTCredentials
Exchange an authorization code for credentials (call this in your callback handler).
The returned credentials may lack plan usage when the user did not grant it; check
plan_usage_granted,
and persist them either way so the next sign-in reuses the registration.
OpenAIChatGPTCredentials
code : str
The callback’s code parameter.
The callback’s client_id parameter, which carries the issued client id after a
new registration. A sign-in that reused client_id may omit it. Once accepted, it is
kept as the flow’s client_id: if the exchange fails, retry with a new flow created
with client_id=flow.client_id, so the retry reuses the registration.
Bases: OAuthCredentialsProvider[OpenAIChatGPTCredentials]
Provider for ChatGPT plan usage through Sign in with ChatGPT.
Sends Responses API requests to https://api.openai.com/v1 with the signed-in user’s OAuth access
token instead of an API key, refreshing it as needed. One provider instance carries one set of
credentials (there is no process-global cache). The instance binds its refresh lock to the first
event loop that awaits a request, so do not reuse it across loops.
provider = OpenAIChatGPTProvider(credential_source=YourCredentialStore())
agent = Agent(OpenAIResponsesModel('gpt-6.1-sol', provider=provider))
The namespace used to fully qualify model IDs routed through this provider.
Type: str
The credentials currently held in memory, rotated in place by refreshes.
Type: CredentialsT
@staticmethod
def realtime_model_profile(model_name: str) -> RealtimeModelProfile | None
The realtime model profile for the named model, if available.
RealtimeModelProfile | None
def __init__(
credentials: OpenAIChatGPTCredentials | None = None,
*,
credential_source: OpenAIChatGPTCredentialSource | None = None,
openai_client: AsyncOpenAI | None = None,
http_client: _OpenAIHTTPClient | None = None,
) -> None
Create a new OpenAI ChatGPT provider.
credentials : OpenAIChatGPTCredentials | None Default: None
The credentials from
OpenAIChatGPTOAuthFlow.
Refreshed tokens then live in memory only. Pydantic AI never falls back to OPENAI_API_KEY.
credential_source : OpenAIChatGPTCredentialSource | None Default: None
Application-owned storage for the credentials, so refreshed tokens
are persisted between runs; see
OpenAIChatGPTCredentialSource.
Mutually exclusive with credentials.
openai_client : AsyncOpenAI | None Default: None
An existing AsyncOpenAI client to use as-is. Opts out of credential
injection entirely; credentials, credential_source, and http_client must
be None.
http_client : _OpenAIHTTPClient | None Default: None
An existing httpx2.AsyncClient to use. Must be dedicated to this
provider (no auth of its own): the provider attaches its credential-injecting auth to it, and
sharing a client between providers would mix their credentials. The auth only
injects credentials on HTTPS requests to api.openai.com, so the client can safely
be reused for other destinations.
Bases: OpenAIProvider
Deprecated provider for OpenAI’s Decisions API, to be removed in v3.
Use OpenAIProvider with
OpenAIDecisionsModel instead: it takes the same
arguments, and the model profiles itself and names its responses 'openai-decisions', as OpenAIChatModel and
OpenAIResponsesModel share OpenAIProvider.
The namespace used to fully qualify model IDs routed through this provider.
Type: str
def __init__(*, openai_client: AsyncOpenAI) -> None
def __init__(
base_url: str | None = None,
api_key: str | None = None,
openai_client: None = None,
http_client: _OpenAIHTTPClient | None = None,
) -> None
Create a new OpenAI provider.
The base url for the OpenAI requests. If not provided, the OPENAI_BASE_URL environment variable
will be used if available. Otherwise, defaults to OpenAI’s base url.
The API key to use for authentication, if not provided, the OPENAI_API_KEY environment variable
will be used if available.
openai_client : AsyncOpenAI | None Default: None
An existing
AsyncOpenAI
client to use. If provided, base_url, api_key, and http_client must be None.
http_client : _OpenAIHTTPClient | None Default: None
An existing httpx2.AsyncClient or legacy httpx.AsyncClient to use for making HTTP requests.
Bases: Provider[AsyncClient]
Provider for xAI API (native xAI SDK).
The resolved API key, or None when the provider was built from a pre-configured xai_client.
The gRPC AsyncClient doesn’t expose its key, so this returns the one resolved from the api_key
argument or XAI_API_KEY. Used by transports that authenticate outside the SDK, e.g.
XaiRealtimeModel’s WebSocket Authorization header.
The custom api_host this provider was configured with, or None.
Read by XaiRealtimeModel to reject a custom host
it can’t yet honor: the realtime WebSocket derives its URL from base_url, not the gRPC channel
target that api_host sets.
def __init__(
*,
api_key: str | None = None,
api_host: str | None = None,
timeout: float | None = None,
metadata: tuple[tuple[str, str], ...] | None = None,
) -> None
def __init__(*, xai_client: AsyncClient) -> None
Create a new xAI provider.
The API key to use for authentication, if not provided, the XAI_API_KEY environment variable
will be used if available.
The API host to use for the xAI SDK client.
The client-level default timeout for the xAI SDK client, in seconds, applied to all requests
made through it. The xAI SDK does not support per-request timeouts, so ModelSettings.timeout is
not supported and has no effect.
gRPC metadata to attach to every request the xAI SDK client makes, forwarded to
xai_sdk.AsyncClient. This is client-scoped, not per-request, so it applies
to every request made through the provider. The canonical use is xAI prompt-cache sticky routing via
metadata=(('x-grok-conv-id', '<conversation-id>'),); see the
xAI prompt-caching docs.
Because it is client-scoped, a provider configured with conversation-specific metadata (e.g. a fixed
x-grok-conv-id) must not be shared between unrelated conversations. Ignored when xai_client is
passed.
xai_client : AsyncClient | None Default: None
An existing xai_sdk.AsyncClient to use. This takes precedence over api_key, api_host,
timeout, and metadata.
Bases: Provider[AsyncHTTPClient]
Provider for the ElevenLabs API.
The provider’s client is a plain async HTTP client: model classes make their ElevenLabs API calls with it,
authenticated by the api_key and addressed to the base_url the provider exposes. (For
ElevenLabs Agents, these are the REST
calls around the agent WebSocket: fetching the agent configuration, minting a signed WebSocket URL, and
optionally syncing tool definitions.)
base_url doubles as the region switch: pass https://api.eu.residency.elevenlabs.io (or the
India and Singapore hosts, api.in.residency.elevenlabs.io and api.sg.residency.elevenlabs.io) to keep
both the REST calls and the derived WebSocket endpoint inside a data-residency region.
The resolved API key, sent as the xi-api-key header on REST requests.
Type: str
def __init__(
*,
api_key: str | None = None,
base_url: str = 'https://api.elevenlabs.io',
http_client: AsyncHTTPClient | None = None,
) -> None
Create a new ElevenLabs provider.
The API key to use for authentication. If not provided, the ELEVENLABS_API_KEY
environment variable will be used if available. The key is always required: even a
public agent’s conversation is preceded by REST preflight calls that authenticate
with it.
base_url : str Default: 'https://api.elevenlabs.io'
The base URL for the ElevenLabs API. Defaults to the global endpoint; pass a
data-residency host
such as https://api.eu.residency.elevenlabs.io to pin a region.
http_client : AsyncHTTPClient | None Default: None
An existing async HTTP client to use. If not provided, a new one is created (and closed with the provider).
Bases: _OpenAICompatibleProvider
Provider for DeepSeek API.
Bases: JsonSchemaTransformer
Transforms schemas to the subset supported by Bedrock structured outputs.
The transformer is applied to Bedrock tool and output schemas during request customization. Strict-mode rewrites are applied when:
NativeOutputis used as theoutput_typeof the Agent.BedrockConverseModelforces native output schemas tostrict=Truebefore request customization.strict=Trueis set explicitly on a Tool.
Like AnthropicJsonSchemaTransformer, Bedrock does not infer strict tool mode
from strict=None. Strict tool definitions are opt-in: callers must set
strict=True explicitly. This avoids silently changing large toolsets into
strict toolsets, which can exceed Anthropic/Bedrock’s 20 strict-tools-per-request
limit, and avoids applying potentially lossy strict-mode schema rewrites unless
requested.
When strict=True, additionalProperties: false is injected on objects and keys
Bedrock rejects are removed from the schema and re-emitted into the field’s
description so the model still has the hint.
Bases: ModelProfile
Profile for models used with BedrockModel.
ALL FIELDS MUST BE bedrock_ PREFIXED SO YOU CAN MERGE THEM WITH OTHER MODELS.
Default: False.
Type: bool
Default: 'text'.
Type: Literal[‘text’, ‘json’]
Default: False.
Type: bool
Default: False.
Type: bool
Default: False.
Type: bool
Default: frozenset({'image'}).
Type: frozenset[Literal[‘image’, ‘document’, ‘video’]]
Content-block kinds that this model accepts in the same user message as a toolResult block.
pydantic-ai merges consecutive user turns into one Bedrock message, which can place a toolResult
alongside a following turn’s text/attachment. Some models reject that: Anthropic rejects documents
and video next to a toolResult, while Llama and Mistral reject any content sharing the turn (the
toolResult must be alone). When a merge would co-locate a toolResult with a kind not listed here,
the adapter splits the turns and separates them with a synthetic assistant message (Bedrock re-merges
consecutive same-role turns, so a bare split doesn’t suffice). See https://github.com/pydantic/pydantic-ai/issues/6081.
Default: all kinds (no restriction); the model receives merged turns unchanged.
Type: frozenset[Literal[‘text’, ‘image’, ‘document’, ‘video’]]
Whether this model accepts a conversation that starts with an assistant message.
Bedrock’s Converse API requires that a conversation start with a user message for most model
families (Amazon Nova, Meta Llama, Mistral, Cohere, AI21, Writer, …), which reject a leading
assistant turn with "A conversation must start with a user message...". Anthropic and Qwen
models accept a leading assistant turn, so for them we don’t need to synthesize a placeholder
user message when message_history starts with a ModelResponse.
Verified against Bedrock us-east-1 on 2026-07-03.
Default: False (strict — synthesize a leading user message when history starts with an
assistant turn).
Type: bool
Whether this model accepts the status field on a toolResult block in Bedrock’s Converse API.
Most families accept (and pydantic-ai emits) status: 'success'/'error' on toolResult blocks, but
Writer Palmyra rejects it ("This model doesn't support the status field. Remove status and try again."),
so the field is omitted for it. Verified against Bedrock us-east-1.
Default: True.
Type: bool
Whether this model accepts strict: true on toolSpec in Bedrock’s Converse API.
Tracked separately from supports_json_schema_output (which gates NativeOutput /
outputConfig) because AWS could in principle ship a model that supports one without the
other; today both features track the same per-model allowlist per the Bedrock structured-output
docs: https://docs.aws.amazon.com/bedrock/latest/userguide/structured-output.html.
Default: False.
Type: bool
Which thinking API shape to use for unified thinking translation.
'anthropic': Uses{'thinking': {'type': 'adaptive'}}for 4.6+ models, or{'thinking': {'type': 'enabled', 'budget_tokens': N}}for older models.'openai': Uses{'reasoning_effort': 'low'|'medium'|'high'}'openai_responses': Uses{'reasoning': {'effort': 'none'|'low'|'medium'|'high'|'xhigh'}}, the Responses API shape that the OpenAI GPT-5.6 and GPT-6 models take on Converse.'qwen': Uses{'reasoning_config': 'low'|'high'}'xai': Uses{'reasoning': {'effort': 'low'|'medium'|'high'|'xhigh'}}None: No unified thinking support.
Default: None.
Type: Literal[‘anthropic’, ‘openai’, ‘openai_responses’, ‘qwen’, ‘xai’] | None
Whether this model accepts {'thinking': {'type': 'adaptive'}} (Haiku 5.5, Sonnet 4.6+, Opus 4.6+).
Only meaningful for the 'anthropic' variant. When False, the variant falls back to
{'type': 'enabled', 'budget_tokens': N} for pre-4.6 models.
Default: False.
Type: bool
Whether this model emits output_config.effort on Bedrock Converse (Haiku 5.5, Sonnet 4.6+, Opus 4.6+).
Only meaningful for the 'anthropic' variant AND only honored alongside
bedrock_supports_adaptive_thinking=True. Bedrock has not been verified to accept
output_config.effort on the legacy {'type': 'enabled', 'budget_tokens': N} path
(e.g. Opus 4.5), so the translator skips it there even though the direct Anthropic
API accepts it. Effort lives at additionalModelRequestFields.output_config.effort
(a sibling of thinking, not inside it).
Default: False.
Type: bool
How the unified top_k setting is placed in additionalModelRequestFields.
Bedrock’s Converse inferenceConfig has no topK field, so top_k must travel in the
model-specific additionalModelRequestFields blob, where the shape differs per family
(and Bedrock 400s on an unrecognized key rather than ignoring it):
'anthropic': flat{'top_k': N}'nova': nested{'inferenceConfig': {'topK': N}}None:top_kis silently dropped (Llama/Mistral/DeepSeek/Jamba don’t accept it on Converse; Cohere’skand Qwen’s key are unverified on Converse, so they stay here too).
Type: Literal[‘anthropic’, ‘nova’] | None
How tool_choice='none' is sent once the history has a tool call.
Converse’s toolChoice has no none mode, so the tools are dropped from the request instead. Converse rejects
toolUse/toolResult blocks without a toolConfig, though, so once the history has them the tools are kept:
'anthropic':toolChoiceis omitted and{'tool_choice': {'type': 'none'}}is sent inadditionalModelRequestFields. Bedrock 400s when this is combined with a ConversetoolChoice.None: no nativenoneexists, sotoolChoiceis omitted and the model can still call the tools.
Default: None.
Type: Literal[‘anthropic’] | None
Where a cache point goes when the message ends with documents.
Bedrock rejects a cachePoint directly after some document blocks, so it can’t always go at the end:
'anthropic': acachePointmay follow a PDF, so it’s appended, for aCachePointtoo. Bedrock renders documents of every other format as text, moves them to the front of the message unless acachePointprecedes them, and rejects acachePointafter them. When the message has no earliercachePoint, an automaticbedrock_cache_messagescache point goes at the end, with such trailing documents moved ahead of the block before them, unless that’s a PDF: the rendered prompt is unchanged, and matches the next request’s, which no longer has thiscachePoint, so that request reads the cache.None: thecachePointgoes before the trailing documents and videos.
Default: None.
Type: Literal[‘anthropic’] | None
Whether Converse rejects temperature, top_p and top_k for this model. Default: False.
When set, BedrockConverseModel drops these settings with a warning instead of sending them.
Type: bool
Whether this model is served by the Bedrock Converse API. Default: True.
Set to False for models that Bedrock serves only through the Mantle OpenAI-compatible API (today,
the proprietary OpenAI GPT models not allowlisted in bedrock_openai_model_profile);
BedrockConverseModel raises at construction so the user gets an actionable pointer to
BedrockMantleProvider instead of an opaque Converse error at request time.
Type: bool
Bases: Provider[BaseClient]
Provider for AWS Bedrock.
The boto3 client used to make requests to the Bedrock API.
Type: BaseClient
def __init__(*, bedrock_client: BaseClient) -> None
def __init__(
*,
api_key: str,
base_url: str | None = None,
region_name: str | None = None,
profile_name: str | None = None,
aws_read_timeout: float | None = None,
aws_connect_timeout: float | None = None,
) -> None
def __init__(
*,
aws_access_key_id: str | None = None,
aws_secret_access_key: str | None = None,
aws_session_token: str | None = None,
base_url: str | None = None,
region_name: str | None = None,
profile_name: str | None = None,
aws_read_timeout: float | None = None,
aws_connect_timeout: float | None = None,
) -> None
Initialize the Bedrock provider.
bedrock_client : BaseClient | None Default: None
A boto3 client for Bedrock Runtime. If provided, other arguments are ignored.
The AWS access key ID. If not set, the AWS_ACCESS_KEY_ID environment variable will be used if available.
The AWS secret access key. If not set, the AWS_SECRET_ACCESS_KEY environment variable will be used if available.
The AWS session token. If not set, the AWS_SESSION_TOKEN environment variable will be used if available.
The API key for Bedrock client. Can be used instead of aws_access_key_id, aws_secret_access_key, and aws_session_token. If not set, the AWS_BEARER_TOKEN_BEDROCK environment variable will be used if available.
The base URL for the Bedrock client.
The AWS region name. If not set, the AWS_DEFAULT_REGION environment variable will be used if available.
The AWS profile name.
The read timeout for Bedrock client.
The connect timeout for Bedrock client.
def bedrock_anthropic_model_profile(model_name: str) -> ModelProfile | None
Get the model profile for an Anthropic model used via Bedrock.
def bedrock_amazon_model_profile(model_name: str) -> ModelProfile | None
Get the model profile for an Amazon model used via Bedrock.
def bedrock_deepseek_model_profile(model_name: str) -> ModelProfile | None
Get the model profile for a DeepSeek model used via Bedrock.
def bedrock_meta_model_profile(model_name: str) -> ModelProfile | None
Get the model profile for a Meta Llama model used via Bedrock.
def bedrock_mistral_model_profile(model_name: str) -> ModelProfile | None
Get the model profile for a Mistral model used via Bedrock.
def bedrock_qwen_model_profile(model_name: str) -> ModelProfile | None
Get the model profile for a Qwen model used via Bedrock.
def bedrock_google_model_profile(model_name: str) -> ModelProfile | None
Get the model profile for a Google model used via Bedrock.
def bedrock_zai_model_profile(model_name: str) -> ModelProfile | None
Get the model profile for a Z.AI (Zhipu) GLM model used via Bedrock.
def bedrock_moonshotai_model_profile(model_name: str) -> ModelProfile | None
Get the model profile for a Moonshot AI Kimi model used via Bedrock.
Registered for both the moonshot. and moonshotai. Bedrock provider prefixes.
def bedrock_xai_model_profile(model_name: str) -> ModelProfile | None
Get the model profile for an xAI Grok model used via Bedrock Converse.
def bedrock_writer_model_profile(model_name: str) -> ModelProfile | None
Get the model profile for a Writer Palmyra model used via Bedrock.
def bedrock_minimax_model_profile(model_name: str) -> ModelProfile | None
Get the model profile for a MiniMax model used via Bedrock.
def bedrock_nvidia_model_profile(model_name: str) -> ModelProfile | None
Get the model profile for an NVIDIA model used via Bedrock.
def bedrock_openai_model_profile(model_name: str) -> ModelProfile | None
Get the model profile for an OpenAI model used via Bedrock Converse.
Bases: OpenAIModelProfile
Profile for OpenAI models served through Amazon Bedrock Mantle.
Which Mantle endpoint family serves this model, selecting the model class and base URL.
Type: BedrockMantleInterface
Bases: _OpenAICompatibleProvider
Provider for the Amazon Bedrock Mantle OpenAI-compatible API.
def __init__(*, openai_client: AsyncOpenAI) -> None
def __init__(
*,
region_name: str | None = None,
base_url: str | None = None,
api_key: str | None = None,
aws_access_key_id: str | None = None,
aws_secret_access_key: str | None = None,
aws_session_token: str | None = None,
profile_name: str | None = None,
http_client: _OpenAIHTTPClient | None = None,
) -> None
Create a Bedrock Mantle provider.
The AWS region used to construct the default bedrock-mantle.{region}.api.aws
origin. If not set, the AWS_DEFAULT_REGION or AWS_REGION environment variable is used.
A Mantle base URL. Its origin (scheme + host, with any /openai/v1 or /v1 suffix
stripped) is used to route between /v1 and /openai/v1 per model, the same as region_name.
A Bedrock API key. If omitted, AWS_BEARER_TOKEN_BEDROCK is used. Use this or the
aws_* credentials, not both.
The AWS access key ID for SigV4 authentication.
The AWS secret access key for SigV4 authentication.
The AWS session token for SigV4 authentication.
The AWS profile name for SigV4 authentication.
openai_client : AsyncOpenAI | None Default: None
An existing OpenAI client. If provided, no other argument may be set; its base
URL’s origin is used to derive both the /v1 and /openai/v1 endpoints (preserving its
auth and transport) so every interface routes correctly.
http_client : _OpenAIHTTPClient | None Default: None
An existing httpx2.AsyncClient or legacy httpx.AsyncClient used to make requests.
def bedrock_mantle_model_profile(model_name: str) -> ModelProfile
Resolve the profile for an OpenAI model served through Bedrock Mantle.
The OpenAI-compatible endpoint family a Bedrock Mantle model is served on.
'chat': Chat Completions at/v1/chat/completions(GPT-OSS Safeguard).'responses': Responses at/v1/responses(GPT-OSS).'openai-responses': Responses at/openai/v1/responses, the OpenAI-model-specific path (GPT-5.4+).
Default: Literal['chat', 'responses', 'openai-responses']
Bases: Provider[AsyncGroq]
Provider for Groq API.
def __init__(*, groq_client: AsyncGroq | None = None) -> None
def __init__(
*,
api_key: str | None = None,
base_url: str | None = None,
http_client: httpx.AsyncClient | None = None,
) -> None
Create a new Groq provider.
The API key to use for authentication, if not provided, the GROQ_API_KEY environment variable
will be used if available.
The base url for the Groq requests. If not provided, the GROQ_BASE_URL environment variable
will be used if available. Otherwise, defaults to Groq’s base url.
groq_client : AsyncGroq | None Default: None
An existing
AsyncGroq
client to use. If provided, api_key and http_client must be None.
http_client : httpx.AsyncClient | None Default: None
An existing AsyncClient to use for making HTTP requests.
def groq_moonshotai_model_profile(model_name: str) -> ModelProfile | None
Get the model profile for an MoonshotAI model used with the Groq provider.
def meta_groq_model_profile(model_name: str) -> ModelProfile | None
Get the model profile for a Meta model used with the Groq provider.
Bases: _OpenAICompatibleProvider
Provider for Azure OpenAI API.
See https://azure.microsoft.com/en-us/products/ai-foundry for more information.
The Azure resource endpoint used to derive service-specific URLs.
Type: str
The Azure resource key, for transports that authenticate with one.
Raises UserError when the provider has no key, i.e. it
was built from a Microsoft Entra ID client (azure_ad_token / azure_ad_token_provider).
Type: str
The Azure AI Voice Live endpoint (AZURE_VOICELIVE_ENDPOINT, else the Azure OpenAI endpoint).
Type: str
The Azure AI Voice Live key (AZURE_VOICELIVE_API_KEY, else the Azure OpenAI key).
Type: str
The Azure AI Voice Live API version (AZURE_VOICELIVE_API_VERSION, else a supported default).
Type: str
@classmethod
def for_realtime(
cls,
*,
azure_endpoint: str | None = None,
api_version: str | None = None,
api_key: str | None = None,
entra_authenticated: bool = False,
http_client: _OpenAIHTTPClient | None = None,
) -> Self
Create an Azure provider for the GA realtime API.
The realtime transport always uses Azure’s /openai/v1 protocol and does not send an
api_version. When neither api_version nor OPENAI_API_VERSION is set, a bare resource
endpoint is therefore normalized to its /openai/v1 form before constructing the provider.
Explicit arguments otherwise follow the same environment fallbacks and validation as the
standard constructor.
The Azure resource endpoint. Falls back to AZURE_OPENAI_ENDPOINT.
The API version for endpoints that require one. Falls back to
OPENAI_API_VERSION.
The Azure resource key. Falls back to AZURE_OPENAI_API_KEY.
entra_authenticated : bool Default: False
Set when every request is authenticated with a Microsoft Entra ID
credential instead of the resource key (see
AzureRealtimeModel(credential=...)).
The key is then neither required nor sent, and api_key raises its usual explanatory
error if anything asks — the same state a provider built from an Entra-authenticated
openai_client lands in.
http_client : _OpenAIHTTPClient | None Default: None
An existing httpx2.AsyncClient or legacy httpx.AsyncClient used to construct the provider client.
def __init__(*, openai_client: AsyncAzureOpenAI) -> None
def __init__(
*,
azure_endpoint: str | None = None,
api_version: str | None = None,
api_key: str | None = None,
voice_live_endpoint: str | None = None,
voice_live_api_key: str | None = None,
voice_live_api_version: str | None = None,
http_client: _OpenAIHTTPClient | None = None,
) -> None
Create a new Azure provider.
The Azure endpoint to use for authentication, if not provided, the AZURE_OPENAI_ENDPOINT
environment variable will be used if available.
The API version to use for authentication, if not provided, the OPENAI_API_VERSION
environment variable will be used if available. Not required (and not sent) when
azure_endpoint targets the Azure OpenAI v1 GA API
(i.e. a path ending in /v1, such as https://<resource>.openai.azure.com/openai/v1/)
or an Azure AI Foundry serverless model endpoint (*.models.ai.azure.com), both of
which reject the api-version query parameter.
The API key to use for authentication, if not provided, the AZURE_OPENAI_API_KEY environment variable
will be used if available.
The Azure AI Voice Live
endpoint, used only by AzureRealtimeModel
with azure_voice_live=True. Voice Live is a distinct Azure resource, so when this is
not provided the AZURE_VOICELIVE_ENDPOINT environment variable is used, and finally
azure_endpoint as a fallback.
The Voice Live API key; falls back to AZURE_VOICELIVE_API_KEY, then api_key.
The Voice Live API version; falls back to AZURE_VOICELIVE_API_VERSION,
then a supported default. Deliberately not derived from api_version, which versions
the Azure OpenAI data plane on an unrelated schedule.
openai_client : AsyncAzureOpenAI | None Default: None
An existing
AsyncAzureOpenAI
client to use. If provided, base_url, api_key, and http_client must be None.
http_client : _OpenAIHTTPClient | None Default: None
An existing httpx2.AsyncClient or legacy httpx.AsyncClient to use for making HTTP requests.
Bases: Provider[AsyncClientV2]
Provider for Cohere API.
def __init__(
*,
api_key: str | None = None,
cohere_client: AsyncClientV2 | None = None,
http_client: httpx.AsyncClient | None = None,
) -> None
Create a new Cohere provider.
The API key to use for authentication, if not provided, the CO_API_KEY environment variable
will be used if available.
cohere_client : AsyncClientV2 | None Default: None
An existing
AsyncClientV2
client to use. If provided, api_key and http_client must be None.
http_client : httpx.AsyncClient | None Default: None
An existing httpx.AsyncClient to use for making HTTP requests.
Bases: Provider[AsyncClient]
Provider for VoyageAI API.
def __init__(*, voyageai_client: AsyncClient) -> None
def __init__(*, api_key: str | None = None) -> None
Create a new VoyageAI provider.
The API key to use for authentication, if not provided, the VOYAGE_API_KEY environment variable
will be used if available.
voyageai_client : AsyncClient | None Default: None
An existing
AsyncClient
client to use. If provided, api_key must be None.
Bases: Provider[Any]
Provider for Sentence Transformers API.
The provider name.
Type: str
The base URL for the provider API.
Type: str
The client for the provider.
Type: Any
Bases: _OpenAICompatibleProvider
Provider for Cerebras API.
def __init__() -> None
def __init__(*, api_key: str) -> None
def __init__(*, api_key: str, http_client: _OpenAIHTTPClient) -> None
def __init__(*, http_client: _OpenAIHTTPClient) -> None
def __init__(*, openai_client: AsyncOpenAI | None = None) -> None
Create a new Cerebras provider.
The API key to use for authentication, if not provided, the CEREBRAS_API_KEY environment variable
will be used if available.
openai_client : AsyncOpenAI | None Default: None
An existing AsyncOpenAI client to use. If provided, api_key and http_client must be None.
http_client : _OpenAIHTTPClient | None Default: None
An existing httpx2.AsyncClient or legacy httpx.AsyncClient to use for making HTTP requests.
Bases: Provider[Mistral]
Provider for Mistral API.
def __init__(*, mistral_client: Mistral | None = None) -> None
def __init__(
*,
api_key: str | None = None,
base_url: str | None = None,
http_client: AsyncHTTPClient | None = None,
) -> None
Create a new Mistral provider.
The API key to use for authentication, if not provided, the MISTRAL_API_KEY environment variable
will be used if available.
mistral_client : Mistral | None Default: None
An existing Mistral client to use, if provided, api_key and http_client must be None.
The base url for the Mistral requests.
http_client : AsyncHTTPClient | None Default: None
An existing httpx2.AsyncClient or legacy httpx.AsyncClient to use for making HTTP requests.
Bases: _OpenAICompatibleProvider
Provider for Fireworks AI API.
Bases: _OpenAICompatibleProvider
Provider for Together AI API.
Bases: _OpenAICompatibleProvider
Provider for Heroku API.
Bases: Provider[AsyncOpenAI]
Provider for the retired GitHub Models API.
GitHub Models was retired on 2026-07-30; the inference API is no longer available, so this provider is deprecated and will be removed in v3.
def __init__() -> None
def __init__(*, api_key: str) -> None
def __init__(*, api_key: str, http_client: httpx.AsyncClient) -> None
def __init__(*, openai_client: AsyncOpenAI | None = None) -> None
Create a new GitHub Models provider.
The GitHub token to use for authentication. If not provided, the GITHUB_API_KEY
environment variable will be used if available.
openai_client : AsyncOpenAI | None Default: None
An existing AsyncOpenAI client to use. If provided, api_key and http_client must be None.
http_client : httpx.AsyncClient | None Default: None
An existing httpx.AsyncClient to use for making HTTP requests.
Bases: _OpenAICompatibleProvider
Provider for GitHub Copilot.
Routes requests through Copilot’s OpenAI-compatible Chat Completions API at
https://api.githubcopilot.com/chat/completions. Copilot serves Anthropic, OpenAI, Google, xAI and
MoonshotAI models under a subscription, but only the ids whose catalog entry lists /chat/completions
under supported_endpoints are reachable here; xAI’s Grok ids, for one, are served on the Responses
API alone. Which ids you can reach also depends on your plan; list yours with
GET https://api.githubcopilot.com/models.
This is not GitHubProvider, which served the
retired GitHub Models API.
def __init__(*, openai_client: AsyncOpenAI) -> None
def __init__(
*,
api_key: str | None = None,
base_url: str | None = None,
integration_id: str | None = None,
openai_client: None = None,
http_client: _OpenAIHTTPClient | None = None,
) -> None
Create a new GitHub Copilot provider.
The Copilot token to authenticate with. Defaults to the GITHUB_COPILOT_API_KEY
environment variable, then to GITHUB_COPILOT_API_TOKEN and COPILOT_GITHUB_TOKEN,
the names GitHub’s own tooling uses. The general-purpose GITHUB_TOKEN, GH_TOKEN
and GITHUB_API_KEY variables are deliberately not read, so a token meant for the
GitHub API is never sent to Copilot.
The base URL of the Copilot inference API, e.g. for an enterprise host or a
local proxy. Defaults to the GITHUB_COPILOT_BASE_URL, COPILOT_API_URL or
GITHUB_COPILOT_API_BASE environment variable, then to https://api.githubcopilot.com.
The copilot-integration-id header sent on every request. Copilot
attributes the traffic to it and ties its quota and model allowlist to it. Defaults
to the GITHUB_COPILOT_INTEGRATION_ID environment variable, the name the Copilot CLI
reads, then to vscode-chat.
openai_client : AsyncOpenAI | None Default: None
An existing AsyncOpenAI client to use. Its base_url must already point
at the Copilot inference API, and it is used as-is, without the Copilot client
headers. If provided, api_key, base_url, integration_id and http_client must
be None.
http_client : _OpenAIHTTPClient | None Default: None
An existing httpx2.AsyncClient or legacy httpx.AsyncClient to use for making HTTP requests.
One GitHub.com device-login flow for a caller-supplied OAuth application.
Construction does no I/O. Call start, display the returned challenge, then call
wait_for_authorization. No browser is opened and no credentials are persisted.
Use one instance per login attempt; do not call its methods concurrently.
def __init__(
*,
client_id: str,
scope: str = '',
http_client: httpx2.AsyncClient | None = None,
) -> None
Configure GitHub’s device flow without borrowing another application’s client identity.
client_id : str
Your registered OAuth application’s client ID. Device flow must be enabled in its settings. GitHub authorization alone does not guarantee Copilot API access.
scope : str Default: ''
Space-separated GitHub OAuth scopes. No scopes are requested by default.
http_client : httpx2.AsyncClient | None Default: None
Optional caller-owned client. Otherwise each request uses a temporary client that is closed before returning. Redirects are not followed.
@async
def start() -> GitHubCopilotDeviceAuthorization
Request a new device code. Replaces any previous challenge on this instance.
The local expiry deadline uses expires_in from receipt of the response.
Raises UserError for an HTTP failure, rejected client, or malformed response.
Transport errors propagate unchanged.
GitHubCopilotDeviceAuthorization
@async
def wait_for_authorization() -> GitHubCopilotCredentials
Poll until approval, rejection, or expiry; cancellation propagates to the caller.
Waits at least GitHub’s interval before each request, including the first, and increases
it on slow_down. The challenge’s original deadline also bounds in-flight requests.
A challenge can be consumed once: call start again after success, failure, or cancellation.
Raises UserError for expiry, denial, an HTTP failure, or a malformed response.
GitHubCopilotCredentials
A GitHub device authorization challenge. Display user_code at verification_uri.
device_code is used only by the token exchange and is excluded from repr.
expires_in and interval are seconds, as returned by GitHub.
GitHub OAuth credentials, with tokens excluded from repr.
Pass access_token to GitHubCopilotProvider(api_key=...). GitHub authorization does not
establish Copilot entitlement: the inference API checks the account’s subscription and policy.
Expiry durations, when present, are seconds from issuance. Applications own persistence and
renewal; the provider does not refresh a token passed through api_key.
Bases: _OpenAICompatibleProvider
Provider for OpenRouter API.
def __init__(*, openai_client: AsyncOpenAI) -> None
def __init__(
*,
api_key: str | None = None,
app_url: str | None = None,
app_title: str | None = None,
openai_client: None = None,
http_client: _OpenAIHTTPClient | None = None,
) -> None
Configure the provider with either an API key or prebuilt client.
OpenRouter API key. Falls back to OPENROUTER_API_KEY
when omitted and required unless openai_client is provided.
Optional url for app attribution. Falls back to
OPENROUTER_APP_URL when omitted.
Optional title for app attribution. Falls back to
OPENROUTER_APP_TITLE when omitted.
openai_client : AsyncOpenAI | None Default: None
Existing AsyncOpenAI client to reuse instead of
creating one internally.
http_client : _OpenAIHTTPClient | None Default: None
Custom httpx2.AsyncClient or legacy httpx.AsyncClient to pass into the
AsyncOpenAI constructor when building a client.
UserError— If no API key is available and noopenai_clientis provided.
Bases: _OpenAICompatibleProvider
Provider for Vercel AI Gateway API.
Bases: Provider[AsyncInferenceClient]
Provider for Hugging Face.
def __init__(*, base_url: str, api_key: str | None = None) -> None
def __init__(*, provider_name: str, api_key: str | None = None) -> None
def __init__(*, hf_client: AsyncInferenceClient, api_key: str | None = None) -> None
def __init__(
*,
hf_client: AsyncInferenceClient,
base_url: str,
api_key: str | None = None,
) -> None
def __init__(
*,
hf_client: AsyncInferenceClient,
provider_name: str,
api_key: str | None = None,
) -> None
def __init__(*, api_key: str | None = None) -> None
Create a new Hugging Face provider.
The base url for the Hugging Face requests.
The API key to use for authentication, if not provided, the HF_TOKEN environment variable
will be used if available.
hf_client : AsyncInferenceClient | None Default: None
An existing
AsyncInferenceClient
client to use. If not provided, a new instance will be created.
http_client : AsyncClient | None Default: None
(currently ignored) An existing httpx.AsyncClient to use for making HTTP requests.
Name of the provider to use for inference. available providers can be found in the HF Inference Providers documentation.
defaults to “auto”, which will select the first available provider for the model, the first of the providers available for the model, sorted by the user’s order in https://hf.co/settings/inference-providers.
If base_url is passed, then provider_name is not used.
Bases: _OpenAICompatibleProvider
Provider for Meta’s Model API (Muse models).
def __init__() -> None
def __init__(*, api_key: str) -> None
def __init__(*, api_key: str, http_client: _OpenAIHTTPClient) -> None
def __init__(*, openai_client: AsyncOpenAI | None = None) -> None
Create a new Meta provider.
The API key to use for authentication, if not provided, the META_API_KEY environment
variable will be used if available.
openai_client : AsyncOpenAI | None Default: None
An existing AsyncOpenAI client to use. If provided, api_key and http_client must be None.
http_client : _OpenAIHTTPClient | None Default: None
An existing httpx2.AsyncClient or legacy httpx.AsyncClient to use for making HTTP requests.
Bases: _OpenAICompatibleProvider
Provider for MoonshotAI platform (Kimi models).
def __init__(*, base_url: str | None = None) -> None
def __init__(*, api_key: str, base_url: str | None = None) -> None
def __init__(
*,
api_key: str,
http_client: _OpenAIHTTPClient,
base_url: str | None = None,
) -> None
def __init__(*, openai_client: AsyncOpenAI | None = None) -> None
Create a new MoonshotAI provider.
The API key for authentication. Defaults to the MOONSHOTAI_API_KEY
environment variable.
The base URL for requests. Defaults to the MOONSHOTAI_BASE_URL
environment variable, then https://api.moonshot.ai/v1.
Use https://api.moonshot.cn/v1 for the Chinese endpoint.
openai_client : AsyncOpenAI | None Default: None
An existing AsyncOpenAI client to use. Its base URL is used
instead of MOONSHOTAI_BASE_URL. Cannot be combined with base_url.
http_client : _OpenAIHTTPClient | None Default: None
An existing httpx2.AsyncClient or legacy httpx.AsyncClient to use for making HTTP requests.
Bases: _OpenAICompatibleProvider
Provider for local or remote Ollama API.
def __init__(
base_url: str | None = None,
api_key: str | None = None,
openai_client: AsyncOpenAI | None = None,
http_client: _OpenAIHTTPClient | None = None,
) -> None
Create a new Ollama provider.
The base url for the Ollama requests. If not provided, the OLLAMA_BASE_URL environment variable
will be used if available.
The API key to use for authentication, if not provided, the OLLAMA_API_KEY environment variable
will be used if available.
openai_client : AsyncOpenAI | None Default: None
An existing
AsyncOpenAI
client to use. If provided, base_url, api_key, and http_client must be None.
http_client : _OpenAIHTTPClient | None Default: None
An existing httpx2.AsyncClient or legacy httpx.AsyncClient to use for making HTTP requests.
Bases: _OpenAICompatibleProvider
Provider for LiteLLM API.
def __init__(*, api_key: str | None = None, api_base: str | None = None) -> None
def __init__(
*,
api_key: str | None = None,
api_base: str | None = None,
http_client: _OpenAIHTTPClient,
) -> None
def __init__(*, openai_client: AsyncOpenAI) -> None
Initialize a LiteLLM provider.
API key for the model provider. If None, LiteLLM will try to get it from environment variables.
Base URL for the model provider. Use this for custom endpoints or self-hosted models.
openai_client : AsyncOpenAI | None Default: None
Pre-configured OpenAI client. If provided, other parameters are ignored.
http_client : _OpenAIHTTPClient | None Default: None
Custom HTTP client to use.
Bases: _OpenAICompatibleProvider
Provider for local or remote vLLM API.
def __init__(*, openai_client: AsyncOpenAI) -> None
def __init__(
*,
base_url: str | None = None,
api_key: str | None = None,
openai_client: None = None,
http_client: _OpenAIHTTPClient | None = None,
) -> None
Create a new vLLM provider.
The base url for the vLLM requests. If not provided, the VLLM_BASE_URL environment variable
will be used if available.
The API key to use for authentication, if not provided, the VLLM_API_KEY environment variable
will be used if available.
openai_client : AsyncOpenAI | None Default: None
An existing
AsyncOpenAI
client to use. If provided, base_url, api_key, and http_client must be None.
http_client : _OpenAIHTTPClient | None Default: None
An existing httpx2.AsyncClient or legacy httpx.AsyncClient to use for making HTTP requests.
Bases: _OpenAICompatibleProvider
Provider for Nebius AI Studio API.
Bases: _OpenAICompatibleProvider
Provider for OVHcloud AI Endpoints.
Bases: _OpenAICompatibleProvider
Provider for Crusoe Serverless Inference API.
def __init__() -> None
def __init__(*, api_key: str) -> None
def __init__(*, api_key: str, http_client: _OpenAIHTTPClient) -> None
def __init__(*, http_client: _OpenAIHTTPClient) -> None
def __init__(*, openai_client: AsyncOpenAI | None = None) -> None
Create a new Crusoe provider.
The API key to use for authentication, if not provided, the CRUSOE_API_KEY environment
variable will be used if available.
openai_client : AsyncOpenAI | None Default: None
An existing AsyncOpenAI client to use. If provided, api_key and http_client must be None.
http_client : _OpenAIHTTPClient | None Default: None
An existing httpx2.AsyncClient or legacy httpx.AsyncClient to use for making HTTP requests.
Bases: _OpenAICompatibleProvider
Provider for Alibaba Cloud Model Studio (DashScope) OpenAI-compatible API.
Bases: _OpenAICompatibleProvider
Provider for SambaNova AI models.
SambaNova uses an OpenAI-compatible API.
Return the provider name.
Type: str
Return the base URL.
Type: str
Return the AsyncOpenAI client.
Type: AsyncOpenAI
@staticmethod
def model_profile(model_name: str) -> ModelProfile | None
Get model profile for SambaNova models.
SambaNova serves models from multiple families including Meta Llama, DeepSeek, Qwen, and Mistral. Model profiles are matched based on model name prefixes.
def __init__(
*,
api_key: str | None = None,
base_url: str | None = None,
openai_client: AsyncOpenAI | None = None,
http_client: _OpenAIHTTPClient | None = None,
) -> None
Initialize SambaNova provider.
SambaNova API key. If not provided, reads from SAMBANOVA_API_KEY env var.
Custom API base URL. Defaults to https://api.sambanova.ai/v1
openai_client : AsyncOpenAI | None Default: None
Optional pre-configured OpenAI client
http_client : _OpenAIHTTPClient | None Default: None
Optional custom httpx2.AsyncClient or legacy httpx.AsyncClient for making HTTP requests.
UserError— If API key is not provided and SAMBANOVA_API_KEY env var is not set
Bases: _OpenAICompatibleProvider
Provider for Snowflake Cortex.
Routes requests through Snowflake’s OpenAI-compatible Chat Completions API at
https://<account>.snowflakecomputing.com/api/v2/cortex/v1/chat/completions, which serves
Claude, GPT, Llama, Mistral, DeepSeek, and Snowflake’s own models. All inference runs inside
the customer’s Snowflake account, so data never leaves the Snowflake security perimeter.
def __init__(*, openai_client: AsyncOpenAI) -> None
def __init__(
*,
account: str | None = None,
token: str | None = None,
base_url: str | None = None,
openai_client: None = None,
http_client: _OpenAIHTTPClient | None = None,
) -> None
Create a new Snowflake provider.
The Snowflake account identifier,
e.g. myorg-myaccount. Defaults to the SNOWFLAKE_ACCOUNT environment variable.
A Snowflake programmatic access token,
OAuth token, or key-pair JWT, sent as Authorization: Bearer <token>.
Defaults to the SNOWFLAKE_TOKEN environment variable.
The base URL of the Cortex REST API, e.g. when connecting through
private connectivity.
Defaults to https://<account>.snowflakecomputing.com/api/v2/cortex/v1.
openai_client : AsyncOpenAI | None Default: None
An existing AsyncOpenAI client to use. Its base_url must already
point at the Cortex REST API. If provided, account, token, base_url, and
http_client must be None.
http_client : _OpenAIHTTPClient | None Default: None
An existing httpx2.AsyncClient or legacy httpx.AsyncClient to use for making HTTP requests.
Bases: Provider[AsyncTypeSafeClient]
Provider for the TypeSafe API, which serves the Jev models.
def __init__(*, typesafe_client: AsyncTypeSafeClient) -> None
def __init__(
*,
api_key: str | None = None,
base_url: str | None = None,
http_client: httpx2.AsyncClient | None = None,
) -> None
Create a new TypeSafe provider.
The API key to use for authentication, if not provided, the TYPESAFE_API_KEY environment
variable will be used if available.
The base URL for the TypeSafe API, if not provided, the TYPESAFE_BASE_URL environment
variable will be used if available, and https://api.typesafe.ai otherwise.
typesafe_client : AsyncTypeSafeClient | None Default: None
An existing AsyncTypeSafeClient to use. If provided, api_key, base_url and
http_client must be None.
http_client : httpx2.AsyncClient | None Default: None
An existing httpx2.AsyncClient to use for making HTTP requests.
Bases: AsyncClient]
Provider for the POST /v1/systemone decisions API, at a URL of your choosing.
SystemOneModel speaks the API; this provider says where it is
and how to authenticate. Decision models such as Contrastive Language Models and
Laya are available over this API, and
Ollama runs decision models locally over it at http://localhost:11434.
The key sent as a bearer token, if the API needs one.
def __init__(
*,
base_url: str | None = None,
api_key: str | None = None,
http_client: httpx2.AsyncClient | None = None,
) -> None
Create a new System One provider.
The URL of the API, with or without a trailing /v1. If not provided,
the SYSTEM_ONE_BASE_URL environment variable is used.
The API key, sent as a bearer token. If not provided, the
SYSTEM_ONE_API_KEY environment variable is used if set.
http_client : httpx2.AsyncClient | None Default: None
An existing httpx2.AsyncClient to use for making HTTP requests.
Bases: _OpenAICompatibleProvider
Provider for Z.AI (Zhipu AI) API.
Z.AI provides GLM models with support for thinking/reasoning mode and preserved thinking across turns.
def __init__(*, base_url: str | None = None) -> None
def __init__(*, api_key: str, base_url: str | None = None) -> None
def __init__(
*,
api_key: str,
base_url: str | None = None,
http_client: _OpenAIHTTPClient,
) -> None
def __init__(*, base_url: str | None = None, http_client: _OpenAIHTTPClient) -> None
def __init__(*, openai_client: AsyncOpenAI | None = None) -> None
Create a new Z.AI provider.
The API key to use for authentication, if not provided, the ZAI_API_KEY environment variable
will be used if available.
The base URL to use for requests, if not provided, the general API endpoint
https://api.z.ai/api/paas/v4 will be used. Set this to use a different Z.AI endpoint,
such as https://api.z.ai/api/coding/paas/v4 for the GLM Coding Plan.
openai_client : AsyncOpenAI | None Default: None
An existing AsyncOpenAI client to use. If provided, api_key, base_url and
http_client must be None.
http_client : _OpenAIHTTPClient | None Default: None
An existing httpx2.AsyncClient or legacy httpx.AsyncClient to use for making HTTP requests.