pydantic_ai.models.fallback
Bases: Exception
Raised within a FallbackExceptionGroup when model responses are rejected by a response handler.
Bases: Model
A model that uses one or more fallback models upon failure.
Apart from __init__, all methods are private or match those of the base class.
The model name.
Type: str
The fully qualified model identifier, combining the wrapped models’ IDs.
Type: str
def __init__(
default_model: Model | KnownModelName | str,
*fallback_models: Model | KnownModelName | str,
fallback_on: FallbackOn = (ModelAPIError,),
)
Initialize a fallback model instance.
default_model : Model | KnownModelName | str
The name or instance of the default model to use.
fallback_models : Model | KnownModelName | str Default: ()
The names or instances of the fallback models to use upon failure.
Conditions that trigger fallback to the next model. Accepts:
- A tuple of exception types:
(ModelAPIError, RateLimitError) - An exception handler (sync or async):
lambda exc: isinstance(exc, MyError) - A response handler (sync or async):
def check(r: ModelResponse) -> bool - A sequence mixing all of the above:
[ModelAPIError, exc_handler, response_handler]
Handler type is auto-detected by inspecting type hints on the first parameter.
If the first parameter is hinted as ModelResponse, it’s a response handler.
Otherwise (including untyped handlers and lambdas), it’s an exception handler.
@async
def __aenter__() -> FallbackModel
Enter all sub-models so their providers can manage HTTP client lifecycle.
FallbackModel
@async
def __aexit__(
exc_type: type[BaseException] | None,
exc_val: BaseException | None,
exc_tb: TracebackType | None,
) -> bool | None
Exit all sub-models, closing their providers’ HTTP clients.
@async
def request(
messages: list[ModelMessage],
model_settings: ModelSettings | None,
model_request_parameters: ModelRequestParameters,
) -> ModelResponse
Try each model in sequence until one succeeds.
In case of failure, raise a FallbackExceptionGroup with all exceptions.
If a previous response set state='suspended', the request is routed directly
to the pinned continuation model, bypassing the fallback chain. If the pinned model
raises a fallback-eligible error during continuation, the messages are rewound
(stripping the suspended response and trailing continuation request) and the
normal fallback chain is tried.
@async
def request_stream(
messages: list[ModelMessage],
model_settings: ModelSettings | None,
model_request_parameters: ModelRequestParameters,
run_context: RunContext[Any] | None = None,
) -> AsyncGenerator[StreamedResponse]
Try each model in sequence until one succeeds.
If a previous response set state='suspended', the request is routed directly
to the pinned continuation model, bypassing the fallback chain. If the pinned model
raises a fallback-eligible error while opening the stream, the messages are rewound
and the normal fallback chain is tried. Mid-stream failures still propagate.
AsyncGenerator[StreamedResponse]
@async
def cancel_suspended_response(response: ModelResponse) -> None
Cancel a suspended continuation on the underlying model holding the server-side job.
When the response carries a continuation pin, resolve that model and delegate to it. Resolve
the pin directly from metadata rather than via _get_continuation_model: the cancel path is
driven by _ContinuationStreamedResponse.get(), whose state is already
'interrupted'/'incomplete'/'complete' (never 'suspended') by the time cancellation
unwinds, so gating on state == 'suspended' here would never find the pin.
When no pin resolves, the response can still hold a live server-side job: the pin is only
stamped when a segment ends suspended, so a streamed background job cancelled during its
first segment (e.g. OpenAI background mode, marked by provider_details['background'] +
provider_response_id) has no pin yet. Best-effort delegate to every inner model so the job
is torn down rather than leaked. This is safe because each model’s own cancel guard is strict
(OpenAI only acts on its own background marker with a matching provider_name; others
no-op), and a raising model doesn’t stop the rest.
A sync or async callable that decides whether an exception should trigger fallback.
Default: Callable[[Exception], Awaitable[bool]] | Callable[[Exception], bool]
A sync or async callable that decides whether a model response should trigger fallback.
Default: Callable[[ModelResponse], Awaitable[bool]] | Callable[[ModelResponse], bool]
The type of the fallback_on parameter to FallbackModel.
Default: type[Exception] | tuple[type[Exception], ...] | ExceptionHandler | ResponseHandler | Sequence[type[Exception] | ExceptionHandler | ResponseHandler]