pydantic_ai.usage
Number of input/prompt tokens.
Type: Annotated[int, Field(validation_alias=(AliasChoices(input_tokens, request_tokens)))] Default: 0
Number of tokens written to the cache.
Type: int Default: 0
Number of tokens read from the cache.
Type: int Default: 0
Number of output/completion tokens.
Type: Annotated[int, Field(validation_alias=(AliasChoices(output_tokens, response_tokens)))] Default: 0
Number of audio input tokens.
Type: int Default: 0
Number of audio tokens read from the cache.
Type: int Default: 0
Number of audio output tokens.
Type: int Default: 0
Any extra details returned by the model.
Type: Annotated[dict[str, int], BeforeValidator(lambda d: d or {})] Default: dataclasses.field(default_factory=(dict[str, int]))
Sum of input_tokens + output_tokens.
Type: int
def __copy__() -> UsageBase
Shallow copy that also copies mutable fields like details.
UsageBase
def opentelemetry_attributes() -> dict[str, int]
Get the token usage values as OpenTelemetry attributes.
def has_values() -> bool
Whether any values are set and non-zero.
Bases: UsageBase
LLM usage associated with a single request.
This is an implementation of genai_prices.types.AbstractUsage so it can be used to calculate the price of the
request using genai-prices.
def incr(incr_usage: RequestUsage) -> None
Increment the usage in place.
incr_usage : RequestUsage
The usage to increment by.
def __add__(other: RequestUsage) -> RequestUsage
Add two RequestUsages together.
This is provided so it’s trivial to sum usage information from multiple parts of a response.
WARNING: this CANNOT be used to sum multiple requests without breaking some pricing calculations.
@classmethod
def extract(
cls,
data: Any,
provider: str,
provider_url: str,
provider_fallback: str,
api_flavor: str = 'default',
details: dict[str, Any] | None = None,
) -> RequestUsage
Extract usage information from the response data using genai-prices.
data : Any
The response data from the model API.
provider : str
The actual provider ID
provider_url : str
The provider base_url
provider_fallback : str
The fallback provider ID to use if the actual provider is not found in genai-prices. For example, an OpenAI model should set this to “openai” in case it has an obscure provider ID.
api_flavor : str Default: 'default'
The API flavor to use when extracting usage information, e.g. ‘chat’ or ‘responses’ for OpenAI.
Becomes the details field on the returned RequestUsage for convenience.
Bases: UsageBase
LLM usage associated with an agent run.
Responsibility for calculating request usage is on the model; Pydantic AI simply sums the usage information across requests.
Number of requests made to the LLM API.
Type: int Default: 0
Number of successful tool calls executed during the run.
Type: int Default: 0
Total number of input/prompt tokens.
Type: int Default: 0
Total number of tokens written to the cache.
Type: int Default: 0
Total number of tokens read from the cache.
Type: int Default: 0
Total number of audio input tokens.
Type: int Default: 0
Total number of audio tokens read from the cache.
Type: int Default: 0
Total number of output/completion tokens.
Type: int Default: 0
Any extra details returned by the model.
Type: dict[str, int] Default: dataclasses.field(default_factory=(dict[str, int]))
def incr(incr_usage: RunUsage | RequestUsage) -> None
Increment the usage in place.
incr_usage : RunUsage | RequestUsage
The usage to increment by.
def __add__(other: RunUsage | RequestUsage) -> RunUsage
Add two RunUsages together.
This is provided so it’s trivial to sum usage information from multiple runs.
Bases: RunUsage
Deprecated alias for RunUsage.
Limits on model usage.
The request count is tracked by pydantic_ai, and the request limit is checked before each request to the model. Token counts are provided in responses from the model, and the token limits are checked after each response.
Each of the limits can be set to None to disable that limit.
The maximum number of requests allowed to the model.
Type: int | None Default: request_limit
The maximum number of successful tool calls allowed to be executed.
Type: int | None Default: tool_calls_limit
The maximum number of input/prompt tokens allowed.
Type: int | None Default: input_tokens_limit if input_tokens_limit is not None else request_tokens_limit
The maximum number of output/response tokens allowed.
Type: int | None Default: output_tokens_limit if output_tokens_limit is not None else response_tokens_limit
The maximum number of tokens allowed in requests and responses combined.
Type: int | None Default: total_tokens_limit
If True, perform a token counting pass before sending the request to the model,
to enforce request_tokens_limit ahead of time.
This may incur additional overhead (from calling the model’s count_tokens API before making the actual request) and is disabled by default.
Supported by:
- Anthropic
- Bedrock Converse
Support for OpenAI is in development: https://github.com/pydantic/pydantic-ai/issues/3430
Type: bool Default: count_tokens_before_request
def has_token_limits() -> bool
Returns True if this instance places any limits on token counts.
If this returns False, the check_tokens method will never raise an error.
This is useful because if we have token limits, we need to check them after receiving each streamed message. If there are no limits, we can skip that processing in the streaming response iterator.
def check_before_request(usage: RunUsage) -> None
Raises a UsageLimitExceeded exception if the next request would exceed any of the limits.
def check_tokens(usage: RunUsage) -> None
Raises a UsageLimitExceeded exception if the usage exceeds any of the token limits.
def check_before_tool_call(projected_usage: RunUsage) -> None
Raises a UsageLimitExceeded exception if the next tool call(s) would exceed the tool call limit.