Tool Output Limits
ToolOutputLimits reduces a tool return that is large enough to dominate the context
window. Tool returns persist in history as ToolReturnParts, so an oversized one is re-sent
on every later model request, paying its token cost for the rest of the run. This capability
intercepts a return when it is produced, reduces it once, and lets the reduced form persist —
the reduction is not recomputed per request.
While Pydantic AI Harness is on 0.x releases, the API may change between minor releases; when it does, deprecation warnings and release-note migration guidance tell you (or your agent) exactly how to upgrade. See the version policy.
A tool can return a payload large enough to dominate the context window: a big file read, a verbose log, a large JSON document. Because tool returns persist in history, an oversized one is re-sent on every later model request, paying its token cost for the rest of the run.
This is the overflow-to-file follow-up the compaction capability names as out of scope: it moves large tool outputs out of the window at production time, rather than compressing or dropping context already inside it.
| Mode | Cost | Lossy? | What the model gets |
|---|---|---|---|
Truncate | zero-LLM | yes | A head / tail / head+tail clamp of the text |
Spill | zero-LLM | no | A handle + preview + shape sketch; full payload read back on demand |
Summarize | one LLM call | yes | A size-gated summary (inherits the run’s model by default) |
Spill is lossless: the full payload is persisted and the model reads slices of it through
the registered read_tool_result(handle, offset, limit, from_end, pattern) tool (the Claude
Code pattern, the core #4352 design).
That tool is bounded: offset >= 0, limit clamped to a built-in line cap, the joined output
capped, and pattern is a literal substring (not a regex), so a model-supplied value cannot
hang the host with catastrophic backtracking.
Construct an Agent with ToolOutputLimits() in its capabilities. With no arguments it
uses the default band: spill returns of 10,000 characters or more, with a bounded truncation
fallback if the store cannot accept the write.
from pydantic_ai import Agent
from pydantic_ai_harness import ToolOutputLimits
agent = Agent('openai:gpt-4o', capabilities=[ToolOutputLimits()])
The capability registers a single read_tool_result tool so the model can page back into any
spilled payload. Its own returns are exempt from reduction.
Configure an ordered list of size bands. Each band is a (over, action) pair: when a
return’s measured size reaches over, its action runs. The band with the largest threshold
that fits wins; anything below the smallest threshold passes through.
from pydantic_ai import Agent
from pydantic_ai_harness import ToolOutputLimits
from pydantic_ai_harness.tool_output_limits import Band, Spill, Summarize, Truncate
agent = Agent(
'openai:gpt-4o',
capabilities=[
ToolOutputLimits(
bands=[
Band(over=100_000, action=Spill()), # huge: keep losslessly, read back on demand
Band(over=20_000, action=Summarize()), # large: compress with the run's model
Band(over=5_000, action=Truncate()), # medium: cheap clamp
],
# below 5,000: passthrough
)
],
)
The default band, when you pass no bands, is Spill(then=Truncate()) at a 10,000-character
threshold: lossless when a store accepts the write, a bounded truncation otherwise — zero LLM
cost and no silent drop.
Passthrough() is an explicit no-op action for bands or per_tool lists, leaving matching
returns untouched.
Every action takes an optional then, applied when the action cannot run: a Spill whose
store errors, a Truncate / Summarize on a binary payload, a Summarize whose model call
raises. then chains, so Summarize(then=Spill(then=Truncate())) degrades summarize ->
spill -> truncate.
per_tool replaces the global band list for named tools (file reads to head, logs to
tail); tool_filter (a ToolSelector) scopes which tools the capability touches at all.
from pydantic_ai import Agent
from pydantic_ai_harness import ToolOutputLimits
from pydantic_ai_harness.tool_output_limits import Band, Truncate, TruncationStrategy
agent = Agent(
'openai:gpt-4o',
capabilities=[
ToolOutputLimits(
per_tool={
'read_file': [Band(over=8_000, action=Truncate(strategy=TruncationStrategy.head))],
'run_shell': [Band(over=8_000, action=Truncate(strategy=TruncationStrategy.tail))],
},
tool_filter=['read_file', 'run_shell', 'search'],
)
],
)
TruncationStrategy has three members: head (keep the first characters, good for headers and
schemas), tail (keep the last characters, good for build and test output where errors land
last), and head_tail (keep both ends, elide the middle — the default).
Set Truncate(keep_tail_lines=N) to reserve the final N lines before allocating the rest of
the character budget. The default is zero, which leaves existing truncation behavior unchanged.
from pydantic_ai_harness.tool_output_limits import Band, ToolOutputLimits, Truncate, TruncationStrategy
truncate = Truncate(max_chars=4_000, strategy=TruncationStrategy.head, keep_tail_lines=2)
limits = ToolOutputLimits(bands=[], per_tool={'run_command': [Band(over=4_000, action=truncate)]})
With head, the remaining content budget keeps the beginning of the output. With
head_tail, it is split 2:3 between the beginning and the text immediately before the
reserved lines. tail continues to keep the end. Markers and all retained characters count
toward max_chars.
The cap takes priority. If the requested lines exceed it, truncation falls back to the
usual tail strategy; if they fit but a full marker would displace them, the result is a
bare tail slice within the cap. This does not invoke then or guarantee that an oversized
control trailer remains intact.
Lines are separated by LF or CRLF, and their existing endings are retained. A terminating
newline does not add a line, but a blank final line counts. Requesting more lines than exist
selects the whole text, subject to the same cap. Negative keep_tail_lines values are rejected.
This applies to the text after serialization and optional ANSI stripping, independently
for ToolReturn.return_value and textual content. Binary fallbacks are unchanged.
Tail-line selection adds no telemetry spans: it is a slicing choice within the existing
tool-result reduction, rather than a separate operation.
Keep Shell’s native max_output_chars above the ToolOutputLimits thresholds. Use
tail truncation for moderate command output and Spill for large output:
from pydantic_ai import Agent
from pydantic_ai_harness.shell import Shell
from pydantic_ai_harness.tool_output_limits import (
Band,
Spill,
ToolOutputLimits,
Truncate,
TruncationStrategy,
)
tail = Truncate(max_chars=4_000, strategy=TruncationStrategy.tail)
agent = Agent(
'openai:gpt-5.6-sol',
capabilities=[
Shell(allowed_commands=['git', 'rg', 'pytest'], max_output_chars=100_000),
ToolOutputLimits(
bands=[],
per_tool={
'run_command': [
Band(over=20_000, action=Spill(then=tail)),
Band(over=4_000, action=tail),
],
},
),
],
)
Only run_command uses these bands; bands=[] leaves other tools to their native limits.
A tail slice retains an exit-code trailer when it fits in the retained suffix. It does not
guarantee that an entire line survives a small budget; head can remove the trailer.
Shell applies its native cap before ToolOutputLimits sees the result. With the spill
threshold below that cap, a natively truncated result is stored instead of being truncated
again, unless the store fails. The spill preview shows both ends and the model can use
read_tool_result to inspect the stored text.
Spilling preserves only the result received from Shell. It cannot recover content already
removed by the native cap. A preview can contain both spill and native truncation notices;
a native notice describes the stored result, not the shorter preview. A positive
Spill.preview_chars value controls the preview’s content budget; its header and omission
marker add to that length.
A ToolReturn carries a return_value and an optional content that core renders as a
separate, model-visible part which also persists in history. This capability measures and
reduces both with the same band logic (they spill to distinct handles). Text content is
reduced in place; non-text content (multimodal parts) that overflows is left unreduced with
a warnings.warn, since it cannot be safely truncated.
Spill keys (overflow_handle, overflow_bytes, overflow_content_handle) live in
ToolReturn.metadata alongside whatever the tool put there. A pre-existing mapping is copied
in with stringified keys; a pre-existing non-mapping value (a string, a dataclass, anything
that is not a Mapping) is kept under original_metadata rather than dropped. Either way the
caller’s own metadata survives a spill and is readable from metadata on the resulting
ToolReturnPart after Agent.run.
Thresholds are measured in characters by default. Set over_tokens=True to measure in
estimated tokens (the same ~4-chars-per-token heuristic as compaction); pass a
tokenizer callable for accuracy. Truncate.max_chars is always characters — truncation is a
character operation regardless of the threshold unit. The cap includes the truncation marker
and applies separately to each reduced text value. If the budget cannot fit both retained
content and a complete marker, truncation keeps the selected slice without a marker. A
non-positive cap returns an empty string. Set strip_ansi=True to strip ANSI escape sequences
from text returns before measuring and reducing.
A spilled structured return is stored as compact JSON: one long line. read_tool_result
pages by line, so page 1 returns the whole payload and page 2 is empty. Setting serializer
stores the value in a layout with real lines instead:
from pydantic_ai_harness.tool_output_limits import ToolOutputLimits, indented_json, json_lines
ToolOutputLimits(serializer=indented_json) # one field per line
ToolOutputLimits(serializer=json_lines) # one record per line
Use json_lines for tools that return lists of records: line N is record N, so page offsets
and pattern matches line up with whole records. Anything that is not a list-like sequence
falls back to indented_json — including a list wrapped in a dict, so return the list
directly for per-record paging. Use indented_json for everything else.
Any (value) -> str callable works too, but prefer the presets: they escape the Unicode
line separators (U+0085/U+2028/U+2029) that would otherwise knock read-back offsets off the
line grid. The serialized text is also what gets measured, so an indented layout can cross
a size band that compact JSON would not. Strings and binary returns are never serialized,
returns below the smallest band pass through untouched, and a serializer that raises or
returns non-text warns and falls back to compact JSON rather than losing the tool output.
Spilled payloads go through the narrow OverflowStore protocol. The default LocalFileStore
writes one file per (run_id, tool_call_id, retry) under a stable root directory and keeps it
after the run, so a later read_tool_result — in this run or a subsequent agent/run — can
still reach it. The handle is backend-addressable (a relative key), not an absolute local
path, so a durable backend (Temporal, a blob store, or the core ExecutionEnvironment
workspace once #4352 lands) can resolve the same handle in another process. Supply your own
backend with store=....
from typing import Protocol
class OverflowStore(Protocol):
async def write(self, key: str, data: bytes) -> str: ... # returns a handle
async def read(self, handle: str) -> bytes: ...
The store root is stable and shareable on purpose — spilled files must be readable by a later
agent or run — so security does not come from per-instance isolation. It comes from two
mechanisms: the root is created with 0700 (owner-only) permissions, and read resolves the
target (following symlinks) and rejects anything that escapes the root via symlink, .., or
an absolute path. Handle segments are also sanitized so a crafted handle cannot traverse out.
By default the store keeps spilled files forever — deleting on run end would break a later agent that still wants to read a spill. To bound disk use, opt into age-based pruning:
from datetime import timedelta
from pydantic_ai import Agent
from pydantic_ai_harness import ToolOutputLimits
from pydantic_ai_harness.tool_output_limits import LocalFileStore
store = LocalFileStore(cleanup_after=timedelta(hours=6)) # default: None = keep forever
agent = Agent('openai:gpt-4o', capabilities=[ToolOutputLimits(store=store)])
When set, a write schedules a background prune (a daemon thread, off the hot path) that
deletes files whose modification time (st_mtime) is older than cleanup_after. Pruning is
non-blocking and non-erroring: any failure is caught and surfaced via warnings.warn, never
propagated into the agent run, so cleanup can never fail a run or block the hot path.
Last-read time (st_atime) is unreliable on noatime/relatime mounts and is not used.
Prefer external cleanup (cron, a sweeper) over the in-process TTL? Point it at the store root and delete by mtime:
import time
from pathlib import Path
root = Path('/tmp/pyai_harness_overflow') # or your configured base_dir
cutoff = time.time() - 6 * 3600
for path in root.rglob('*'):
if path.is_file() and path.stat().st_mtime < cutoff:
path.unlink(missing_ok=True)
A built-in Summarize call is a real request to the model, so its full usage — tokens and the
request itself — folds into the run’s ctx.usage, exactly like SummarizingCompaction. Its nested
run receives the parent limits unchanged except that a finite request limit reserves one request for
the pending parent request, and is filed under the parent run’s conversation_id.
By default Summarize inherits the running agent’s model (ctx.model). Pass a model id or
instance to Summarize(model=...) to override, or a summarize callable to bypass the
built-in prompt entirely. The summary_prompt template on the capability must contain both
{tool_name} and {output} placeholders.
With a durable-execution capability attached, built-in model summarization is a journaled
capability operation. ToolOutputLimits carries the stable default id='tool_output_limits', so
durable recovery works without configuration.
Two details matter when choosing a band under durability. The text being summarized is part of the
journaled operation input, so prefer Spill over built-in Summarize for outputs near the
engine’s payload limit. And a custom summarize callable runs directly rather than as a durable
operation — arbitrary callables cannot be reconstructed on the worker side — so it may be called
again on replay.
- Binary returns spill verbatim and are never stringify-truncated;
Truncate/Summarizeon binary fall through tothen. - Structured / nested returns spill (or summarize) by preference — truncating JSON produces
invalid JSON.
Spillincludes a one-line shape sketch of the top level. ModelRetryand tool errors never reach this hook (they are raised, not returned), so the model always gets the full error it needs to recover.- A large
ToolReturn.contentis reduced with the same bands asreturn_value; non-text content that overflows is left unreduced with a warning. - Multiple oversized returns in one step get distinct handles (keyed per
tool_call_id); retries get distinct handles too (keyed perretry), so a retried call never clobbers the earlier attempt’s spill.
- Distinct from compaction, which compresses or drops context already inside the window; this capability moves large tool outputs out of the window at production time.
- Consumes core #4352 (the canonical
queryable-file primitive) through the
OverflowStoreseam once it lands. - Distinct from
ClampOversizedMessages, which clamps runaway model responses, not tool returns.
Bases: AbstractCapability[AgentDepsT]
Reduce oversized tool returns when they are produced, persisting the reduction.
A tool can return a payload large enough to dominate the context window. Tool returns
persist in history, so an oversized one is re-sent on every later request. This
capability intercepts a return in after_tool_execute, reduces it once, and lets the
reduced form persist — it is not recomputed per request.
Three reduction modes, freely combined through an ordered list of size bands:
Truncate: clamp to a character budget. Lossy, zero-cost.Spill: persist the full payload, hand the model aread_tool_resulthandle plus a preview. Lossless.Summarize: size-gated LLM summary. Inherits the run’s model by default.
The first band whose over threshold the measured size meets wins; smaller returns pass
through. per_tool replaces the band list for named tools; tool_filter scopes which
tools are touched at all. The default is Spill(then=Truncate()): lossless when a store
accepts the write, a bounded truncation otherwise.
ModelRetry and other errors never reach this hook (they are raised, not returned), so
error payloads the model needs to recover are never spilled or summarized.
Ordered size bands. The first band whose over threshold is met wins.
Type: Sequence[Band] Default: field(default_factory=_default_bands)
Per-tool band lists that replace bands for the named tools.
Type: Mapping[str, Sequence[Band]] Default: field(default_factory=(dict[str, Sequence[Band]]))
Which tools this capability touches. Non-matching tools always pass through.
Type: ToolSelector[AgentDepsT] Default: 'all'
Measure band thresholds in estimated tokens instead of characters.
Type: bool Default: False
Optional (str) -> int tokenizer for over_tokens. Defaults to a ~4-char heuristic.
Type: Callable[[str], int] | None Default: None
Backend for spilled payloads. Defaults to a LocalFileStore.
Type: OverflowStore | None Default: None
Strip ANSI escape sequences from text returns before measuring and reducing.
Type: bool Default: False
Prompt template for Summarize. Must contain {tool_name} and {output}.
Type: str Default: _DEFAULT_SUMMARY_PROMPT
Render a structured (non-string, non-binary) return to the text that is measured,
previewed, spilled, and read back. When unset, structured returns render as compact
JSON, which puts the whole value on one line; the indented_json and json_lines
presets make large spills pageable by line through read_tool_result. A return that
stays under every band threshold passes through as the original object, so the
serialized text is only model-visible once a band triggers.
Type: Serializer | None Default: None
def get_toolset() -> AgentToolset[AgentDepsT] | None
Register the read_tool_result tool for reading spilled payloads on demand.
AgentToolset[AgentDepsT] | None
@async
def after_tool_execute(
ctx: RunContext[AgentDepsT],
*,
call: ToolCallPart,
tool_def: ToolDefinition,
args: dict[str, Any],
result: Any,
) -> Any
Reduce the tool result — both return_value and model-visible content.