pydantic_ai.models.fallback
Bases: Exception
Raised within a FallbackExceptionGroup when model responses are rejected by a response handler.
Bases: Model
A model that uses one or more fallback models upon failure.
Apart from __init__, all methods are private or match those of the base class.
The model name.
Type: str
The fully qualified model identifier, combining the wrapped models’ IDs.
Type: str
The smallest known context window among the candidate models, or None if none is known.
Any candidate may end up answering, and history that fits the smallest window fits them all, so compacting against it errs towards compacting early rather than overflowing a fallback. Candidates with an unknown window don’t constrain the result.
def __init__(
default_model: Model | KnownModelName | str,
*fallback_models: Model | KnownModelName | str,
fallback_on: FallbackOn = (ModelAPIError,),
)
Initialize a fallback model instance.
default_model : Model | KnownModelName | str
The name or instance of the default model to use.
fallback_models : Model | KnownModelName | str Default: ()
The names or instances of the fallback models to use upon failure.
Conditions that trigger fallback to the next model. Accepts:
- A tuple of exception types:
(ModelRateLimitError, ModelUnavailableError) - An exception handler (sync or async):
lambda exc: isinstance(exc, MyError) - A response handler (sync or async):
def check(r: ModelResponse) -> bool - A sequence mixing all of the above:
[ModelAPIError, exc_handler, response_handler]
Handler type is auto-detected by inspecting type hints on the first parameter.
If the first parameter is hinted as ModelResponse, it’s a response handler.
Otherwise (including untyped handlers and lambdas), it’s an exception handler.
Response handlers only apply to non-streamed requests. A streamed request accepts
the response as is and emits a UserWarning. A request is streamed whenever
something consumes the run’s events, including run() with an
event_stream_handler or an event listener.
@async
def __aenter__() -> FallbackModel
Enter all sub-models so their providers can manage HTTP client lifecycle.
FallbackModel
@async
def __aexit__(
exc_type: type[BaseException] | None,
exc_val: BaseException | None,
exc_tb: TracebackType | None,
) -> bool | None
Exit all sub-models, closing their providers’ HTTP clients.
@async
def request(
messages: list[ModelMessage],
model_settings: ModelSettings | None,
model_request_parameters: ModelRequestParameters,
) -> ModelResponse
Try each model in sequence until one succeeds.
In case of failure, raise a FallbackExceptionGroup with all exceptions.
If a previous response set state='suspended', the request is routed directly
to the pinned continuation model, bypassing the fallback chain. If the pinned model
raises a fallback-eligible error during continuation, or its response is rejected by a
response handler, the messages are rewound (stripping the suspended response and
trailing continuation request) and the normal fallback chain is tried.
@async
def request_stream(
messages: list[ModelMessage],
model_settings: ModelSettings | None,
model_request_parameters: ModelRequestParameters,
run_context: RunContext[Any] | None = None,
) -> AsyncGenerator[StreamedResponse]
Try each model in sequence until one succeeds.
If a previous response set state='suspended', the request is routed directly
to the pinned continuation model, bypassing the fallback chain. If the pinned model
raises a fallback-eligible error while opening the stream, the messages are rewound
and the normal fallback chain is tried.
A stream is only handed over once it has produced its first event, so an error the stream raises before that, such as a provider reporting an overload after acknowledging the request, falls back like an error while opening it. Failures after the first event still propagate.
Response handlers are not applied: the response has already been streamed by the time
it could be judged, so a UserWarning is emitted and the response is accepted as is.
AsyncGenerator[StreamedResponse]
@async
def cancel_suspended_response(response: ModelResponse) -> None
Cancel a suspended continuation on the underlying model holding the server-side job.
When the response carries a continuation pin, resolve that model and delegate to it. Resolve
the pin directly from metadata rather than via _get_continuation_model: the cancel path is
driven by _ContinuationStreamedResponse.get(), whose state is already
'interrupted'/'incomplete'/'complete' (never 'suspended') by the time cancellation
unwinds, so gating on state == 'suspended' here would never find the pin.
When no pin resolves, the response can still hold a live server-side job: the pin is only
stamped when a segment ends suspended, so a streamed background job cancelled during its
first segment (e.g. OpenAI background mode, marked by provider_details['background'] +
provider_response_id) has no pin yet. Best-effort delegate to every inner model so the job
is torn down rather than leaked. This is safe because each model’s own cancel guard is strict
(OpenAI only acts on its own background marker with a matching provider_name; others
no-op), and a raising model doesn’t stop the rest.
def resolve_cache_retention(model_settings: ModelSettings | None) -> timedelta | None
A fallback model can’t know which model will serve the request, so no retention is claimed.
timedelta | None
A sync or async callable that decides whether an exception should trigger fallback.
Default: Callable[[Exception], Awaitable[bool]] | Callable[[Exception], bool]
A sync or async callable that decides whether a model response should trigger fallback.
Default: Callable[[ModelResponse], Awaitable[bool]] | Callable[[ModelResponse], bool]
The type of the fallback_on parameter to FallbackModel.
Default: type[Exception] | tuple[type[Exception], ...] | ExceptionHandler | ResponseHandler | Sequence[type[Exception] | ExceptionHandler | ResponseHandler]