Compaction
Compaction is a menu of strategies for keeping an agent’s conversation history within a model’s context window. These Pydantic AI Capability classes edit the message history just before each request goes out. FallbackCompaction optionally triggers its chain at a token threshold and also works as a composing CompactionStrategy. The edits persist into the run’s message history, so a trim, clear, or summary carries forward to later steps — it is not recomputed from the full history every turn.
All strategies preserve tool-call / tool-return pairing. Core does not validate this, and a provider rejects an orphaned pair, so the pairing guarantee is what makes these safe to drop into an agent. The zero-LLM strategies never call a model; only SummarizingCompaction and RelevanceCompaction (and TieredCompaction when it escalates to one of them) spend tokens.
On OpenAI and Anthropic, core also ships provider-native compaction — the provider summarizes history server-side. The strategies on this page are the model-agnostic alternative: they work with every model and keep the compaction logic (and its costs) under your control.
While Pydantic AI Harness is on 0.x releases, the API may change between minor releases; when it does, deprecation warnings and release-note migration guidance tell you (or your agent) exactly how to upgrade. See the version policy.
An agent that runs for many turns accumulates history: tool outputs, file reads, model reasoning, repeated content. Left unchecked, that history outgrows the model’s context window and the next request fails. Compaction keeps the history bounded, and the right strategy depends on where the bloat lives and how much you can afford to spend reclaiming it.
| Component | Cost | What it does | Reach for it when |
|---|---|---|---|
ClampOversizedMessages | zero-LLM | Head/tail-truncates a single oversized part (response text, tool-call args) | One runaway generation blew past the context cap and no other strategy can reach it |
SlidingWindowCompaction | zero-LLM | Drops the oldest whole messages down to a tail | You only need the recent turns and can discard old context entirely |
ClearToolResults | zero-LLM | Blanks the content of old tool results in place, keeping the last keep_pairs | Tool outputs dominate context and can be re-fetched on demand (the cheap first tier) |
DeduplicateFileReads | zero-LLM | Blanks every file read superseded by a newer read of the same file | The agent re-reads files and only the latest version matters |
SummarizingCompaction | one LLM call | Summarizes older messages into a structured summary, keeping the recent tail | Old context still matters but must be compressed; use behind the cheap tiers |
RelevanceCompaction | one judge request per new step, plus an optional summary | Keeps, reduces, or drops each older step by a judge model’s answers | Some old tool results are still needed verbatim and others are dead weight |
TieredCompaction | escalates | Runs cheap passes first, summarizes only if still over target_tokens | You want a sensible default: spend the expensive summary only when needed |
FallbackCompaction | depends on chain | Tries the next strategy when one raises | Summarization can fail and deterministic truncation must keep the run alive |
WarnNearLimits | zero-LLM | Injects an URGENT/CRITICAL warning as limits approach | You want the agent to wrap up rather than have its history rewritten |
ReportContextUsage | zero-LLM | Reports context usage to your application; never edits history | You want a live context gauge in a UI |
Compaction updates persistent run history and replaces the current request view. List request-only injectors such as Memory after compaction so they apply to the compacted request. Earlier request-only edits are discarded, even when ClampOversizedMessages has nothing to shorten.
Instruction replacement and withdrawal records contribute their full rendered system text to token estimates. Superseded updates before a new instruction baseline are excluded.
Every size-based strategy triggers on max_messages, max_tokens (estimated), or max_fraction. Token counts anchor on the provider-reported usage of the most recent model response when one is available. That provider usage includes the instructions, tool definitions, and FilePart payloads sent in the anchored request; only the messages added since are estimated. The suffix after the anchor, or a history with no usage anchor, uses tokenizer or a ~4-chars-per-token heuristic and cannot see FilePart payloads. Pending tool schemas newly revealed for the request are conservatively estimated by the implementation. DeduplicateFileReads runs on every request when no trigger is set (it is cheap and near-lossless). TieredCompaction triggers and stops on a single target_tokens / target_fraction budget. ClampOversizedMessages triggers per part (max_part_tokens / max_part_chars), not on the whole history — the failure it targets is one oversized part, not a large total.
An absolute max_tokens is only correct for the model it was measured against. Configure 180_000 and a 1M-context model compacts at a fifth of its capacity, paying for summaries it did not need; a 128K model configured for 1_000_000 never compacts before the provider rejects the request.
max_fraction is resolved per request against the model’s real context window, so one configuration is correct everywhere:
from pydantic_ai import Agent
from pydantic_ai_harness import SummarizingCompaction
agent = Agent(
'anthropic:claude-sonnet-5-5',
capabilities=[SummarizingCompaction(max_fraction=0.9, keep_messages=20)],
)
That compacts at 900K on a 1M model and at 115K on a 128K one. WarnNearLimits takes the same shape as max_context_fraction, and TieredCompaction as target_fraction. max_tokens and max_fraction are mutually exclusive — a strategy taking both would have to pick one and discard the other, leaving the caller unable to tell which budget was in force.
The window is the model’s own context_window: the value its profile sets, including one you pass as profile={'context_window': ...}, filled from genai-prices when no profile does. A FallbackModel reports the smallest window among its models, since any of them may answer. resolve_context_window is exported if you want the number yourself. Nothing is cached: only a window the model or the registry states is ever treated as the real one.
The model consulted is ModelRequestContext.model, the one the request will be sent to, not the one the run started with. A capability ordered earlier may replace it, and the budget follows.
Not every model has a known window. A local endpoint, a bespoke deployment, a model the registry knows without a recorded window, and a FallbackModel none of whose models has a window all resolve to nothing. The fraction is then taken of fallback_context_window, which defaults to a conservative 200K (DEFAULT_CONTEXT_WINDOW): compacting earlier than necessary costs one summary, overestimating costs the whole request.
Every capability that takes a fraction takes the fallback too, so you are not stuck with 200K on a model you know the size of:
from pydantic_ai import Agent
from pydantic_ai_harness import SummarizingCompaction
agent = Agent(
'ollama:qwen3',
capabilities=[SummarizingCompaction(max_fraction=0.9, fallback_context_window=40_000)],
)
It is only consulted when resolution fails, so it costs nothing on a model the registry does know.
TestModel is one of the models that does not resolve: its model_id is test:test, so a fraction is taken of fallback_context_window and max_fraction=0.9 becomes a 180,000-token trigger. A compaction config exercised only against TestModel will look like it never fires; pass context_window= or fallback_context_window= in the test to put the trigger where you can reach it.
Resolution can also succeed and be wrong, which fallback_context_window cannot help with — it applies only when resolution fails. Three cases:
-
The registry entry itself is wrong. Harness reads
genai-pricesand cannot validate it. Measured againstgenai-prices0.1.3:model id registry records real window anthropic:claude-sonnet-4-51,000,000 200,000 anthropic:claude-opus-4-6200,000 1,000,000 An over-recorded window is the direction that breaks a run. On
anthropic:claude-sonnet-4-5,max_fraction=0.9resolves to a 900,000-token trigger against a 200,000-token window: compaction never fires, and the provider rejects the request instead. Passcontext_window=200_000explicitly onclaude-sonnet-4-5.claude-sonnet-5’s recorded 1,000,000 matches Anthropic’s model documentation, so it needs no override; check any other Sonnet id you use against the provider’s own documentation before relying on the resolved number. An under-recorded window is safe but wasteful — it compacts earlier than it has to. -
The registry records the maximum a model can be made to accept. Where that maximum is gated — a beta header, a pricing tier — an ordinary request gets less, and a fraction of the recorded number never triggers before the provider rejects the request.
-
A self-hosted or proxied endpoint reports a model id whose registry entry describes someone else’s deployment.
context_window overrides resolution outright, on every capability that takes a fraction:
from pydantic_ai import Agent
from pydantic_ai_harness import SummarizingCompaction
agent = Agent(
'openai:gpt-6-luna', # served by a local endpoint with a smaller window than the registry records
capabilities=[SummarizingCompaction(max_fraction=0.9, context_window=32_000)],
)
With a usage anchor, the provider-reported usage covers everything billed for the anchored request,
including its instructions, tool definitions, and FilePart payloads. A response whose billed usage
sums several model requests anchors on the input tokens of the one that measures its context, plus
its output: the first request of a Google long decode, whose
billed input re-counts the earlier requests’ output, or the last sampling pass of an
Anthropic response that reports its passes, whose
billed input re-counts the context of each pass. For the suffix after the
anchor, and for a whole history with no usage anchor, the estimator uses tokenizer when supplied
or a ~4-characters-per-token heuristic. That estimated portion cannot see FilePart payloads, and
counts a file inside a tool return as the See file <identifier>. reference that stands in for it.
When the instructions changed since the anchor, the new set replaces the old one in the estimate
rather than adding to it, since a request sends only one. Pending tool schemas newly revealed for the request are conservatively estimated by the
implementation, since they are not covered by the earlier anchor.
If you already set an absolute max_tokens, re-check it. The estimator used to count only user and system prompts, tool returns, response text, and tool calls. ThinkingPart / CompactionPart content, retry content, NativeToolCallPart / NativeToolReturnPart, and the most recent ModelRequest.instructions are now counted too, so the same history measures higher and an unchanged max_tokens compacts earlier. How much earlier depends on how much of the history is thinking blocks, retries, and instructions; on a thinking-heavy tool-calling history it can be several times the old count. What each strategy clears is unchanged — only when it runs.
The retained tail is measured per message with estimate_message_tokens, so keep_tokens means
tokens as the provider counts them. A response counts at least its reported output_tokens, which
includes thinking that some providers return as an empty part with only a signature. The messages
between two responses that carry usage share the difference between their input_tokens, which is
exactly what the provider counted for them. A response whose billed usage sums several model
requests enters that difference with the input tokens of the one that measures its context: the
first request of a Google long decode, or the last sampling pass
of an Anthropic response that reports its passes.
That last pass also read the response’s own earlier passes, so the difference credits them to the
messages before it, and the tail measures high. Before the first such response, after the last, and
where the history did not grow between two of them (as across an earlier compaction), tokenizer or the
~4-characters-per-token heuristic applies. Instructions are not part of the tail. A history with no
provider usage is measured with the heuristic as before.
If you set keep_tokens, re-check it. It used to be measured with the heuristic alone, which
on a thinking-heavy history can see a third of the real tokens or less, so the same budget kept
several times more history than it named. It now keeps the named amount, so fewer messages.
A strategy knows when to act but says nothing about how close the run is to the limit, so an application that wants to show context: 73% ends up re-counting the history and guessing the denominator. ReportContextUsage does neither — it reuses the same estimator and the same resolved window, and only observes:
from pydantic_ai import Agent
from pydantic_ai_harness import ReportContextUsage, SummarizingCompaction
from pydantic_ai_harness.compaction import ContextUsageEvent
agent = Agent(
'anthropic:claude-sonnet-5-5',
capabilities=[
SummarizingCompaction(max_fraction=0.9, keep_messages=20),
ReportContextUsage(),
],
)
@agent.on_event(ContextUsageEvent)
async def show(ctx, event):
print(f'{event.fraction:.0%}')
Each reading carries used_tokens, window_tokens, fraction, and resolved — False when the window is the fallback rather than the model’s real one, so a gauge can show that the percentage is a guess. Migration: on_usage remains supported but is deprecated. Move its callback body to a ContextUsageEvent subscription. Order matters: register the monitor after a compaction capability to observe the corrected current history after same-cycle compaction, or before it to see what triggered the compaction.
used_tokens follows the accounting above: provider usage anchors include instructions, tool
definitions, and FilePart payloads from the anchored request. The suffix after the anchor, or a
history with no anchor, uses tokenizer or a ~4-characters-per-token heuristic and cannot see
FilePart payloads. Pending newly revealed tool schemas are conservatively estimated by the
implementation. When a compaction capability runs earlier in the same cycle, a monitor registered
after it subtracts the rewrite’s heuristic reclaim while retaining the anchor’s fixed provider
overhead.
A strategy’s compact takes a RunContext, which an application holding a conversation between runs does not have — and that is exactly when a user types /compact. compact_now builds a throwaway context so the same strategy the agent uses can be driven from a command handler:
from pydantic_ai_harness import SummarizingCompaction
from pydantic_ai_harness.compaction import compact_now
strategy = SummarizingCompaction(max_fraction=0.9, keep_messages=20)
history = await compact_now(
strategy,
history,
model='anthropic:claude-sonnet-5-5',
focus='the auth refactor, not the earlier CSS work',
)
compact_now applies no trigger of its own, so a strategy whose compact is unconditional runs whatever the history size. A strategy that defines its own stop condition still honours it: TieredCompaction escalates only until the history fits its target, so a history already under target comes back unchanged. Pass the tier directly if you need it to run regardless.
focus steers strategies that write prose or judge relevance — SummarizingCompaction and RelevanceCompaction, via the exported SupportsFocus protocol’s with_focus — and is passed over by the ones that drop or blank content by rule, since they have nothing to steer. TieredCompaction is focusable when any of its tiers is, so a focus reaches the summarizing tier rather than stopping at the wrapper.
A compaction that changes the history emits the same compact_messages span the in-run path emits, so an instrumented application sees one shape however compaction was triggered. Pass tracer= to record it; without one the span goes to a no-op tracer.
Pass conversation_id= with the id of the conversation being compacted. It is set on the throwaway context, so the summary run of SummarizingCompaction is recorded under that conversation (its messages and its gen_ai.conversation.id span attribute) rather than under a fresh id of its own. Inside a run, the summary run takes the parent run’s conversation_id without this.
The field consensus (Anthropic, OpenCode, Letta) is to clear and dedupe first, and summarize only when that is not enough. Summarization turns input tokens into output tokens, which are billed at a premium and generated serially, so it is genuinely expensive. The zero-LLM strategies touch only the cheaper input side.
TieredCompaction encodes that escalation: it runs each tier in order, re-measures the token count after each, and stops as soon as the conversation fits target_tokens. Order the tiers cheap-to-expensive so the expensive summarization tier is only reached when the cheap passes cannot reclaim enough.
from pydantic_ai import Agent
from pydantic_ai_harness import ClearToolResults, DeduplicateFileReads, SummarizingCompaction, TieredCompaction
from pydantic_ai.messages import ToolCallPart
def my_file_key(call: ToolCallPart) -> str | None:
if call.tool_name != 'read_file':
return None
return call.args_as_dict().get('path')
agent = Agent(
'openai:gpt-6-luna',
capabilities=[
TieredCompaction(
tiers=[
DeduplicateFileReads(file_key=my_file_key),
ClearToolResults(max_tokens=1, keep_pairs=3),
SummarizingCompaction(max_messages=1, keep_messages=20), # model inherits the run's
],
target_tokens=120_000,
)
],
)
A tier inside TieredCompaction is driven directly by the orchestrator, which re-measures after each tier and stops once under target_tokens. A tier’s own max_* trigger is therefore irrelevant when it runs inside TieredCompaction — set it to anything valid (for example ClearToolResults(max_tokens=1)). Any object with async def compact(messages, ctx) -> list[ModelMessage] (the CompactionStrategy protocol) can be a tier, so you can plug in your own.
TieredCompaction advances when a successful tier does not reclaim enough. FallbackCompaction advances only when a strategy raises an exception selected by fallback_on, which defaults to Pydantic AI’s ModelAPIError and FallbackExceptionGroup. The latter is raised when every model in a FallbackModel fails. Each attempt receives a fresh list containing the original message objects, so list-level changes by a failed strategy do not affect its fallback. Strategies must still honor the CompactionStrategy contract and avoid mutating message objects. If every strategy fails, the last exception is re-raised. Non-matching exceptions, cancellation, and other BaseException subclasses pass through immediately; fallback_on rejects types that do not derive from Exception.
Register it directly when summarization should fall back to deterministic truncation:
from pydantic_ai_harness import FallbackCompaction, SlidingWindowCompaction, SummarizingCompaction
fallback = FallbackCompaction(
max_fraction=0.85,
fallback_chain=[
SummarizingCompaction(max_messages=1, keep_tokens=20_000),
SlidingWindowCompaction(max_messages=1, keep_tokens=20_000),
]
)
Register fallback directly with Agent(..., capabilities=[fallback]). Its optional
max_tokens or max_fraction trigger runs the chain only when estimated context tokens
exceed the threshold. Fractions resolve against the request’s model; context_window
overrides its window and fallback_context_window supplies an unknown model’s window.
tokenizer customizes token estimation. The hook preserves pinned parts and persists
compacted history, emitting the standard compact_messages span when history changes.
With neither trigger configured, the request hook does nothing. Direct compact() and
compact_now() calls run the chain regardless of its threshold, so it remains usable
inside other composing strategies and for manual compaction. Each child’s own trigger
is bypassed when the chain calls its compact() method.
A single model response of repeated whitespace, or a single tool call with a giant payload, can produce one part so large the next request exceeds the provider’s context cap. None of the other strategies can reach it: SlidingWindowCompaction drops the oldest messages but the offender is the newest; ClearToolResults only touches tool results; WarnNearLimits never edits history; and feeding the history to SummarizingCompaction hits the same cap.
ClampOversizedMessages truncates the offending part in place, keeping a head slice and a tail slice with a [clamped: removed N of M characters] marker between them. Degenerate generations are low-entropy repetition, so a head/tail slice loses little.
from pydantic_ai import Agent
from pydantic_ai_harness import ClampOversizedMessages
agent = Agent(
'openai:gpt-6.1-sol',
capabilities=[
ClampOversizedMessages(max_part_tokens=50_000, keep_head_chars=2_000, keep_tail_chars=2_000)
],
)
A part is clamped only when it is oversized and the clamp actually shrinks it, so keep keep_head_chars + keep_tail_chars well below your per-part threshold.
It clamps two kinds of part inside each ModelResponse:
- Response text (
TextPart) — the critical case, a runaway model-response text part. - Tool-call args (
ToolCallPart), whenclamp_tool_call_args=True(the default) — the same failure shape for a giant payload (for example a runawaywrite_plan). The args are replaced with a small JSON object{"_clamped": "<head>...<tail>"}so they stay valid function arguments; the original call already executed, so this only shrinks the history copy. Setclamp_tool_call_args=Falseto clamp response text only. Framework-typed call parts — core’ssearch_toolsandload_capabilitycalls — are never clamped, because their typed args are validated when persisted history is restored (for example aStepPersistenceresume) and the_clampedobject would fail that round-trip.
Request-side parts (user prompts, tool returns, system prompts) are deliberately out of scope: user input should not be silently rewritten, and oversized tool returns are the job of ToolOutputLimits, which reduces a return when the tool produces it. ClearToolResults keeps the newest keep_pairs results intact, and with keep_pairs=0 it blanks a fresh return outright rather than shrinking it.
Use it as the first tier of TieredCompaction, before ClearToolResults:
from pydantic_ai_harness import ClampOversizedMessages, ClearToolResults, TieredCompaction
TieredCompaction(
tiers=[
ClampOversizedMessages(max_part_tokens=50_000),
ClearToolResults(max_tokens=1, keep_pairs=3),
],
target_tokens=120_000,
)
Tool outputs typically dominate an agent’s context, and the agent can usually re-run a tool if it needs the data again. ClearToolResults replaces the content of the oldest tool results with a short placeholder while keeping the most recent keep_pairs tool-call / tool-return pairs intact. The tool calls stay paired with their now-blanked results, so the history stays valid. A retried return, which asks the model to make the call again, isn’t a result: it is never cleared and doesn’t count toward keep_pairs, and DeduplicateFileReads doesn’t let a retried read supersede an earlier one.
Framework-typed tool results — core’s search_tools and load_capability returns — are left intact (a small token floor), because their structured content is re-parsed on later requests and rewriting it via dataclasses.replace would bypass validation and corrupt the part. FileSystem’s typed read_file results (ReadFileReturnPart) are cleared like any other result and keep their type, unless their metadata is something other than None or a dict: a cleared result is marked in its metadata, so one whose metadata can’t hold the mark is left uncleared.
from pydantic_ai import Agent
from pydantic_ai_harness import ClearToolResults
agent = Agent(
'openai:gpt-6-luna',
capabilities=[ClearToolResults(max_tokens=100_000, keep_pairs=3)],
)
Set clear_tool_inputs=True to also blank the arguments of the cleared calls, and exclude_tools to a set of tool names whose results are never cleared.
With max_messages, the count starts after the last cleared result, because clearing blanks content but removes no messages. Clearing then happens in batches: after a clear, the next one waits until more than max_messages messages follow the last cleared result, so the requests in between extend the cached prefix instead of rewriting it. The kept pairs count toward that: right after a clear, about 2 * keep_pairs messages follow the last cleared result, and each tool step adds two more. So each clear is followed by another after about (max_messages - 2 * keep_pairs) / 2 tool steps, and at or below 2 * keep_pairs it clears on every request. Pick the value with that gap in mind.
When the same file is read more than once, only the latest read keeps its content; earlier reads are blanked with a placeholder, with pairing preserved.
There is no default file_key: identifying a file read is agent-specific, and a wrong guess would drop live data. Supply a callable mapping a ToolCallPart to a stable file key, or None when the call is not a file read:
from pydantic_ai import Agent
from pydantic_ai.messages import ToolCallPart
from pydantic_ai_harness import DeduplicateFileReads
def file_key(call: ToolCallPart) -> str | None:
if call.tool_name != 'read_file':
return None
return call.args_as_dict().get('path')
agent = Agent('openai:gpt-6.1-sol', capabilities=[DeduplicateFileReads(file_key=file_key)])
With no max_messages or max_tokens trigger set, DeduplicateFileReads runs on every request. It is cheap and near-lossless, so that default is usually what you want.
When the conversation exceeds the configured threshold, SlidingWindowCompaction discards the oldest whole messages down to a tail, preserving tool-call / tool-return pairs. Reach for it when you only need the recent turns and can discard old context entirely.
from pydantic_ai import Agent
from pydantic_ai_harness import SlidingWindowCompaction
agent = Agent(
'openai:gpt-6.1-sol',
capabilities=[SlidingWindowCompaction(max_messages=80, keep_messages=40)],
)
By default preserve_first_user_message=True keeps the first user turn (in addition to system prompts) even when it falls outside the window, so the agent does not lose the original task. keep_messages=0 still keeps the newest message, the request being answered. Pass keep_tokens instead of keep_messages to retain a minimum amount of original message text rather than a message count.
max_tokens determines when compaction runs; a positive keep_tokens determines how much original recent history survives. Both SlidingWindowCompaction and SummarizingCompaction include the whole message that crosses the retention target, extending backward for matching tool calls and tool-specific retry responses.
For example, automatic compaction with max_tokens=170_000 may encounter a 190k-token history consisting of 139k older tokens, a recent 48k-token message, and a 3k-token newest tail. With keep_tokens=50_000, it retains the 51k-token suffix, including the message that crosses the target. Summarizing compaction summarizes the older prefix; sliding-window compaction discards it.
The suffix is measured as described in What counts toward keep_tokens: provider-reported usage where the history has it, otherwise the configured tokenizer or the default character-based estimate over message-part text, including system-prompt parts. Attached ModelRequest.instructions do not satisfy the retention target. If the whole history is below the target, compaction leaves it unchanged without a summary call. keep_tokens=0 uses a zero-token target rather than a minimum-retention floor; None uses keep_messages.
Whole messages and tool dependencies can overshoot the target. Summaries, receipts, reinserted pins, and retained older user turns are additional; they do not displace the protected suffix. Existing receipts inside that suffix remain unchanged. This is not a context-window-fit guarantee: callers must check the remaining request budget and stop, switch models, or explicitly reduce the retention target if necessary. Configure the same retention target on each fallback that must preserve it.
A large message can keep the retained suffix and the summary above max_tokens. Until newer messages meet the target without it, everything before the suffix is already accounted for: an earlier summary, receipts, pins, and messages kept verbatim anyway, such as the preserved first user turn. SummarizingCompaction then has nothing new to summarize, so it leaves the history unchanged without a summary call instead of re-summarizing its own summary on every request. The request still goes out above the trigger; ClampOversizedMessages can shorten an oversized message when that matters.
When old context still matters but must be compressed, SummarizingCompaction summarizes the older messages with a dedicated model call and replaces them with a single structured summary, preserving the recent tail and tool-call integrity. It is the expensive tier, so it is best used behind the cheaper passes (see TieredCompaction).
from pydantic_ai import Agent
from pydantic_ai_harness import SummarizingCompaction
agent = Agent(
'anthropic:claude-opus-5-5',
capabilities=[
SummarizingCompaction(
model='anthropic:claude-sonnet-5-5',
max_messages=60,
keep_messages=20,
)
],
)
model accepts a model name or a Model; when left None it inherits the running agent’s model. That model has to write text, so an agent running a model that can’t, such as a decision model like TypeSafe’s Jev, needs model= set to a language model; without it, the first compaction raises a UserError saying so. A summary that is empty or only whitespace is retried within the summary run’s output retries. Exhausting those retries raises UnexpectedModelBehavior rather than replacing the history with an empty summary. Its nested summary run inherits the parent usage limits and reserves one request from a finite request limit for the pending parent request. These limits also apply to retries: if the summary run exhausts its budget first, it raises UsageLimitExceeded instead. To fall back on either failure, set FallbackCompaction’s fallback_on=(UnexpectedModelBehavior, UsageLimitExceeded), preferably with a fallback strategy that makes no model requests. This does not increase the parent run’s limits. Pass model_settings to give the dedicated summary call settings that differ from defaults carried by that model; the supplied settings merge over the model defaults without mutating the model or the settings dictionary. Pass summarization_capabilities to attach capabilities to the summary agent; capabilities on the outer agent do not run on the summary call. By default incremental=True updates the newest existing summary as an anchor. This changes the summary-call prompt from earlier releases; set incremental=False to retain the prior regeneration behavior. keep_messages=0 still keeps the newest message, the request being answered, alongside the summary.
Both prompt surfaces of the summary request are fields: summary_prompt is the user-turn template (it must contain a {messages} placeholder and no other replacement field; any other prompt emits a HarnessDeprecationWarning and will raise ValueError in a future release), and instructions sets the internal agent’s static instructions, which Pydantic AI sends in the request’s system prompt. Override instructions when the summarizer endpoint requires a fixed leading instruction.
The messages served into that template are rendered as a transcript with one <message> element per part (each user prompt, system prompt, response text, tool call and tool return), tagged with its role and, for a tool call or return, its tool name. Content that looks like the transcript’s own tags, in any case or spacing, is escaped, so a tool return can’t close its message and forge another, or end the transcript early. Each tool return is capped, after that escaping, at tool_return_max_chars (default 500) characters using the same explicit truncation marker as kept user turns. Raise it, or set it to None to render each return whole, when the summarizer’s context window is large enough to absorb the payloads. Content a harness capability added on the user channel and tagged with an origin in its TextContent.metadata starting with pydantic-ai-harness. is not a user message. Today that’s a BackgroundTools result and TrajectoryJudge feedback. Memory snapshots aren’t tagged this way yet, so they are still summarized as user messages; that gap closes once core provenance lands. Tagged content is rendered as a <harness_data origin="..."> element, capped like a tool return, rather than as role="user". When the transcript holds such content, the summary request also tells the summarizer not to treat it as the user’s requests or standing constraints; other summary requests are unchanged. max_tokens and keep_tokens control when compaction runs and which history messages are retained; they do not cap the summary-request payload.
The summary request is non-streaming unless event_stream_handler is set. Supply a handler to watch the summary as it is written, or pass drain_summary_events to take the streaming request path without handling the events — which is what a summarizer endpoint that rejects non-streaming requests needs:
from pydantic_ai import Agent
from pydantic_ai_harness.compaction import SummarizingCompaction, drain_summary_events
agent = Agent(
'openai:gpt-6.1-sol',
capabilities=[
SummarizingCompaction(max_messages=60, event_stream_handler=drain_summary_events),
],
)
Neither transport works everywhere, which is why this is a choice rather than a default: some endpoints reject non-streaming requests and others reject streaming ones. The handler receives the summary run’s own RunContext and event stream; the outer Agent.run(...) handler is not inherited and never sees the summary token deltas.
The summary call is a real request to the model, so its full usage — tokens and the request itself — is folded into the run’s ctx.usage. This is deliberate: it keeps cost honest, keeps the request count consistent (a model request that did not count as one would be the surprise), and lets a UsageLimits request limit catch a runaway compaction. The nested run receives the other parent limits unchanged; the finite request limit is reduced by one so it cannot spend the slot already approved for the parent request. The summary run is filed under the parent run’s conversation_id, so its requests, spans, and cost group with the conversation it compacts. A run-request or iteration limiter will therefore see compaction calls among its requests.
With a durable-execution capability attached, the summary call runs as a contributed durable
operation, so replay uses the recorded summary instead of calling the model again. When model is
not set, the operation uses the run’s model. The capability carries a stable default id, which
durable execution uses to recover the operation by the same identity. Overriding it with a custom
value orphans recorded operations for in-flight workflows, so keep it fixed once a workflow is live.
RelevanceCompaction asks a judge model about each older step of the history and keeps the step verbatim, reduces it, or drops it, based on the answers. Reach for it when the older context is uneven: some tool results hold code or error text the agent still needs exactly, others are dead weight, and one summary of everything would lose the exact parts.
from pydantic_ai import Agent
from pydantic_ai_harness.compaction import RelevanceCompaction, SummarizingCompaction
agent = Agent(
'anthropic:claude-opus-5-5',
capabilities=[
RelevanceCompaction(
'system-one:nimble',
summarizer=SummarizingCompaction(model='anthropic:claude-haiku-5-5', max_messages=1),
max_fraction=0.8,
)
],
)
The judge, model, is required. A decision model, such as system-one:<model> on System One (including the models Ollama runs locally) or TypeSafe’s typesafe:jev-latest, answers with a probability. Any other model that supports structured output fills the same two fields, so model also takes a FallbackModel that tries a decision model first and a language model second.
A step is a model response together with the requests that answer it: its tool results, retry feedback, and the next user prompt. A step is kept or removed whole, so a tool call never loses its result. Everything before the first response (the system prompt, the first user message, and an earlier summary) and the most recent keep_tokens (default 8,000) of history are never judged. Pinned content survives even when its step is removed.
Each remaining step is judged in one request with two questions:
needed_verbatim: “Will the agent need the exact content of this earlier step, such as code, file contents, identifiers, numbers, commands or error text, to finish the current task?”still_matters: “Does this earlier step record a fact, decision, change, result or user instruction that still matters for the current task?”
The judge is shown the current task (the first and the latest user prompts, plus any focus passed to compact_now), the most recent part of the conversation, and the step itself, including its tool results. A step longer than judge_step_tokens (default 2,000) is shown by its start and end, and the task and the recent conversation are each clamped to judge_context_tokens (default 500). The defaults keep a request within the 8,192-token input of small local decision models, since code and JSON take about twice as many tokens as the ~4-characters-per-token estimate suggests. The judge is told that all of it is untrusted data, and text that looks like the prompt’s own tags is escaped.
Steps are chosen by rank against a token budget, never by comparing a probability to a fixed threshold: a judge’s probabilities shift as a whole with the shape of the request, while the order it puts steps in holds. The history is compacted to target_tokens (or target_fraction), which defaults to half the trigger:
- Steps are kept verbatim in order of
needed_verbatimwhile they fit the target. - The rest are reduced in order of
still_matters. With asummarizer, they are written into one summary placed after the head of the history, up tosummary_input_tokens(default 30,000) of steps per compaction. Without one, each of their tool results is cut to its firsttruncated_result_chars(default 300) characters plus a note that the tool can be called again, while that still fits the target. - Whatever is left is dropped, except a step holding a user prompt: without a summarizer it is cut, and with one it is summarized while it fits
summary_input_tokensand kept verbatim otherwise.
The summarizer is a SummarizingCompaction used for its model, prompts and settings; its trigger and keep_* fields are never consulted, so any trigger such as max_messages=1 works. With the summarizer’s incremental=True (the default) the earlier summary is updated; with incremental=False it is summarized again along with the steps. receipts=True adds a receipt when steps were summarized or dropped.
Judge requests run concurrently, up to judge_concurrency (default 8) at once. Every judgement is remembered on the instance under the task and the step’s content, so a later compaction with the same task judges only the new steps. All the judge requests of one compaction count as one request against a UsageLimits request limit, checked before any of them is sent. Their tokens are added to the run’s usage and checked against its token and cost limits once they return, like the summary request’s. On a laptop running Ollama, one judge request for a step of a few thousand tokens took 5 to 10 seconds, so the first compaction of a long history can take minutes there; later ones judge only the new steps.
A judge error propagates, as a summary error does. Wrap the strategy in FallbackCompaction to fall back to one that makes no model requests:
from pydantic_ai_harness.compaction import FallbackCompaction, RelevanceCompaction, SlidingWindowCompaction
compaction = FallbackCompaction(
max_fraction=0.8,
fallback_chain=[
RelevanceCompaction('system-one:nimble', max_fraction=0.8),
SlidingWindowCompaction(max_messages=1, keep_tokens=20_000),
],
)
As a TieredCompaction tier, set its target_tokens explicitly: a tier’s compact runs with its own target, and the default of half the trigger means nothing when the trigger is only there to satisfy the constructor.
With a durable-execution capability attached, the judgements and the summary run as the durable operations judge and summarize of the capability’s stable default id, relevance_compaction, so replay reuses them instead of calling the models again.
The two-question split and concurrent per-step requests come from fast-jev-compaction (MIT), which compacts Claude Code transcripts with TypeSafe’s Jev. Its measurements also motivated showing the judge each step’s content and choosing steps by rank. No code is shared.
WarnNearLimits never edits history. As the run approaches a configured limit, it adds an URGENT (then CRITICAL) warning to each request, so the model wraps up rather than having its context rewritten under it. Each warning is a request-scoped system prompt: the model sees it for its own request only, and it stays in the message history, so earlier requests are sent unchanged and the prompt cache and the model’s earlier reasoning stay valid. Warnings that older versions sent as user prompts are left out of requests when a saved history still holds them. They are recognized by metadata on the part, not by their text, so a user prompt that quotes a warning is never removed.
from pydantic_ai import Agent
from pydantic_ai_harness import WarnNearLimits
agent = Agent(
'google:gemini-3.8-flash',
capabilities=[
WarnNearLimits(
max_iterations=40,
max_context_tokens=100_000,
)
],
)
Warnings begin at warning_threshold (default 0.7, a fraction of the limit) and become CRITICAL for iterations once the remaining request count drops to critical_remaining_iterations (default 3). It watches three kinds of limit — max_iterations, max_context_tokens (or max_context_fraction), and max_total_tokens — and by default warns on whichever are configured; narrow that with warn_on.
Clearing, deduplicating, clamping, and summarizing all rewrite message content, which invalidates the provider’s prompt cache from the edit point onward — the next request pays a cache-write. For ClearToolResults, use min_clear_tokens to skip clearing that reclaims too little to be worth busting the cache. For ClampOversizedMessages the cache bust is unavoidable, because the alternative is a failed request.
When core instrumentation is active (the Instrumentation capability, agent.instrument, or Agent.instrument_all()), each strategy emits a compact_messages span on the run’s tracer the moment it actually compacts — that is, in before_model_request, once the strategy’s threshold is exceeded (ClampOversizedMessages emits only when a part is actually clamped). TieredCompaction emits a single span for the whole escalation rather than one per tier, because it drives each tier’s compact directly. Without instrumentation the tracer is a no-op, so the span adds no overhead.
The span name is the static compact_messages; the strategy is an attribute, not part of the name, to keep span cardinality low. Attributes:
| Attribute | Type | Meaning |
|---|---|---|
gen_ai.conversation.compacted | bool | Always true; the OpenTelemetry GenAI convention’s flag for a compacted context |
compaction.strategy | str | Strategy class name (for example SlidingWindowCompaction, SummarizingCompaction) |
compaction.messages_before | int | Message count before compaction |
compaction.messages_after | int | Message count after compaction |
compaction.tokens_before | int | Estimated token count before compaction |
compaction.tokens_after | int | Estimated token count after compaction |
gen_ai.conversation.compacted is the GenAI semantic convention’s flag; the rest is harness-specific. Token counts use the strategy’s tokenizer when set, otherwise the ~4-chars-per-token heuristic. Raw message content is not recorded.
SummarizingCompaction runs its summarizer as a nested Agent named summarizing_compaction, so under Agent.instrument_all() (or logfire.instrument_pydantic_ai()) its runs carry agent_name = summarizing_compaction. Filter on that to track summarization usage and cost separately from the parent agent.
RelevanceCompaction adds a select compaction steps span inside compact_messages. It records the number of steps and how many were judged rather than remembered (relevance_compaction.steps, relevance_compaction.steps_judged), the steps and tokens of each tier (relevance_compaction.keep.steps, relevance_compaction.keep.tokens, and the same for reduce and drop), and one relevance_compaction.step event per step with its position, size, both answers, whether it was remembered, and its tier. No message content is recorded. The judge’s own runs are not instrumented, because their prompts carry the conversation.
Compaction is a memory wipe the model cannot veto and often cannot detect, which invites resumption drift — the model confabulates continuity with history it no longer has. A receipt makes the wipe legible: after a boundary-crossing strategy rewrites history it appends a short, deterministic note recording how much was compacted, warning that what survives is secondhand, and — when a handle provider is attached — an identifier for persisted run history.
from pydantic_ai_harness import SlidingWindowCompaction, SummarizingCompaction
SummarizingCompaction(max_messages=60, keep_messages=20, receipts=True)
SlidingWindowCompaction(max_messages=80, keep_messages=40, receipts=True)
The receipt text carries no timestamp, so it is a pure function of the compaction. The message part still has its ordinary request timestamp.
Wording follows what actually survived. SummarizingCompaction leaves a summary, so its receipt says the summary above is secondhand; SlidingWindowCompaction drops history outright, so its receipt says that context is gone. The blank-in-place strategies (ClearToolResults, DeduplicateFileReads, ClampOversizedMessages) keep every message and cross no boundary, so they emit no receipt.
SlidingWindowCompaction counts its receipt toward keep_messages, so keep_messages=40 keeps the receipt and 39 messages. The receipt never takes the last slot: keep_messages=1 keeps the receipt and the newest message.
Attach any capability exposing compaction_transcript_handle() -> str | None — the TranscriptHandleProvider protocol — and the receipt gains a Persisted run handle: <handle> pointer. StepPersistence implements it, returning its run_id, so attaching it is enough. Each receipt is also emitted as a compaction.receipt event on the compact_messages span, carrying compaction.receipt.strategy, compaction.receipt.messages_dropped, compaction.receipt.tokens_dropped, compaction.receipt.by, and compaction.receipt.handle when a handle was found.
The handle addresses the persisted run, not a pristine transcript. Compaction’s edits persist into the run’s message history, so the run’s latest snapshot reflects the compacted history — reading it back does not recover what the receipt says was dropped. A store keeping per-step snapshots may still hold pre-compaction steps, subject to its own retention (max_snapshots_per_run on the shipped stores). Treat the handle as a pointer to the run, and check your store’s retention before promising an agent it can read the original.
The receipt’s by attribution uses the same coarse family heuristic as the bridge prefix, with the same approximations — see the note below.
Receipts are opt-in (receipts=False by default) because the receipt text is content: the exact wording is provisional pending a benchmark eval-rig pass, while the mechanism is structural.
pin marks content that every shipped strategy must preserve:
from pydantic_ai_harness.compaction import pin
# In a ModelRequest placed in the run's message history (by a capability or the user):
pinned = pin('Durable task state the model must never lose across compaction.')
A pinned part is never summarized away or dropped; if a strategy would have discarded it, the strategy re-injects it near the top of the surviving history. is_pinned reports whether a given part carries the marker.
Pins use model-invisible TextContent.metadata, so their contents remain ordinary user context while compaction can distinguish them from user turns.
The Planning capability does not need pinning: its plan is re-injected ephemerally on every request in wrap_model_request, so it already survives compaction by construction. Pinning is for durable task state and scratchpads that live in the history.
User turns are the highest signal-per-token content in a conversation, and losing them is the main driver of resumption drift. SummarizingCompaction(keep_user_messages=True) preserves the newest user turns from the summarized prefix alongside the summary. In message-count mode, they consume the existing keep_messages tail budget, so at most that many retained user messages and tail messages survive together, except that one tail slot is reserved for the request being answered when retained user turns would otherwise consume it; compaction therefore does not grow retained copies on each cycle. With a positive keep_tokens, the original suffix takes precedence: keep_messages caps only the additional older user turns, which do not reduce the retained tail. With keep_tokens=0, retained user copies must fit a zero-token budget, and the newest tail message still survives. Each retained turn is bounded to keep_user_messages_max_chars (default 20k) with an explicit truncation marker when it overruns. The character budget applies per part and is shared across the text items of a multi-part prompt; images, audio, and cache points pass through untouched. Harness-added content on the user channel, such as a background tool’s result, isn’t a user turn: it is left out of what keep_user_messages and preserve_first_user_message carry over from the summarized prefix (including from a turn an earlier compaction kept), and summarized as data rather than as the user’s intent. The recent suffix compaction leaves unsummarized is kept as is, harness content included. This supersedes preserve_first_user_message, which keeps only the first turn.
from pydantic_ai_harness import SummarizingCompaction
SummarizingCompaction(max_tokens=120_000, keep_messages=20, keep_user_messages=True)
Retaining user turns leaves the summary, any receipt, and the retained turns as adjacent ModelRequests. Providers that require one request per turn — Bedrock Converse and Gemini among them — never see that shape: Pydantic AI normalizes the history with _merge_consecutive_messages after the before_model_request hooks run, combining adjacent requests into a single turn before dispatch. keep_user_messages therefore needs no provider-specific handling.
With incremental=True (the default), a prior summary is not re-summarized — summarizing summaries decays over successive compactions. It is fed back as an anchored <previous-summary> block with an update instruction: preserve still-true details, remove stale ones, merge in new facts. The summary becomes a living document updated in place under a fixed structure.
bridge_prefix=True prepends a one-line note to the summary only when the summarizer’s model family differs from the family that produced the history, derived from the history’s model_name and the summarizer config. It marks the summary as a cross-model handoff so the resuming model builds on it rather than confabulating that it did the work itself. It never fires in the common same-model case, so it is cheap. It defaults to False because the note is prompt content.
The family token is a coarse approximation: drop any provider: prefix, then take the leading token before the first - or /. It separates gpt from claude on ordinary references (openai:gpt-5.6-luna -> gpt, google:gemini-3.6-flash -> gemini) and misreads several real ones — us.anthropic.claude-sonnet-4-5-v1:0 reduces to 0, ollama/llama3 to ollama, and a fallback: model string to its last listed model rather than its first. A FallbackModel object is read correctly, from its first model. So bridge and receipt attribution are best-effort: a misread family can suppress a bridge note or fire one between two same-family models. Neither outcome changes what compaction keeps or drops.
As with receipts, the update instruction and the bridge-prefix wording are content, shipped minimal and neutral pending the eval-rig pass; the anchoring and family-gating mechanisms are structural.
These strategies compress or drop context inside the window. Moving large tool outputs out of the window — overflowing them to a file the agent (or a subagent) can query on demand — is a separate capability (tool output limits), not lossy truncation. Within SummarizingCompaction, tool_return_max_chars makes the summarizer’s per-return cap tunable (or None to render returns whole), but the summary request still reads a lossy rendering; prefer tool output limits when a payload must be queryable in full.
The recommended default is TieredCompaction; the other strategies below can be used standalone or plugged in as its tiers.
Bases: AbstractCapability[AgentDepsT]
Escalation orchestrator over a sequence of compaction strategies.
Runs each tier in order, re-measuring the token count after each, and stops as soon as
the conversation fits target_tokens. Order tiers cheap-to-expensive (e.g. clear
tool results, deduplicate reads, then summarize) so the expensive summarization tier is
only reached when the cheap passes cannot reclaim enough.
Each tier’s own trigger is bypassed — TieredCompaction drives the tiers directly via
their compact method and decides when to stop.
Strategies to apply in order, cheap-to-expensive. The last is typically a summarizer.
Type: Sequence[CompactionStrategy[AgentDepsT]]
Stop escalating once the estimated token count is at or below this value.
Mutually exclusive with target_fraction; exactly one of the two must be set.
Type: int | None Default: None
Target expressed as a fraction of the model’s context window, resolved per request.
Use this instead of target_tokens when the same agent runs on models with
different windows. Mutually exclusive with target_tokens.
Type: float | None Default: field(default=None, kw_only=True)
Window override in tokens. None resolves it from the request’s model.
Unlike fallback_context_window, this applies whether or not resolution succeeds. Reach
for it when the registry is confidently wrong: a beta- or tier-gated window it records as
the maximum, or a self-hosted endpoint whose model id describes someone else’s
deployment. Only consulted alongside target_fraction.
Type: int | None Default: field(default=None, kw_only=True)
Window assumed when the model is not in the pricing registry.
Only consulted alongside target_fraction. Supply the real number for a deployment the
registry cannot resolve.
Type: int Default: field(default=DEFAULT_CONTEXT_WINDOW, kw_only=True)
Optional tokenizer for accurate token counting.
A callable that returns the token count for a given string.
When None, uses a ~4 characters-per-token heuristic.
Type: Callable[[str], int] | None Default: None
def with_focus(focus: str) -> TieredCompaction[AgentDepsT]
Return a copy whose focus-capable tiers prioritize focus.
A tiered strategy is focusable when any of its tiers is: the summarizing tier writes the prose, so the hint has to reach it rather than stopping at this wrapper. Tiers that cannot honour a focus are passed through unchanged.
TieredCompaction[AgentDepsT]
@async
def compact(
messages: list[ModelMessage],
ctx: RunContext[AgentDepsT],
) -> list[ModelMessage]
Apply tiers in order until the history fits the target or tiers run out.
@async
def before_model_request(
ctx: RunContext[AgentDepsT],
request_context: ModelRequestContext,
) -> ModelRequestContext
Escalate through the tiers when the conversation exceeds the target.
Bases: AbstractCapability[AgentDepsT]
Zero-cost head/tail truncation of any single oversized message part.
A runaway generation — a model response of repeated whitespace, a giant tool-call
payload — can produce one part so large the next request exceeds the provider’s context
cap. The size-based strategies cannot help: SlidingWindowCompaction drops the oldest messages
(the offender is the newest), ClearToolResults only touches tool results, and feeding
the history to SummarizingCompaction hits the same cap. This strategy truncates the
offending part in place: it keeps a head slice and a tail slice and inserts a marker for
the removed middle. Degenerate generations are low-entropy repetition, so a head/tail
slice loses little. No LLM calls are made.
What it clamps, in each ModelResponse:
TextPartcontent (the critical case — a runaway model-response text part).ToolCallPartargs, whenclamp_tool_call_argsis set (the same failure shape for a giant tool-call payload). The args are replaced with a small JSON object so they stay valid function arguments; the original call already executed, so this only shrinks the history copy. Framework-typed subclasses (ToolSearchCallPart,LoadCapabilityCallPart) are never clamped: their typed args must survive theModelMessagesTypeAdapterround-trip that persistence relies on.
Request-side parts (user prompts, tool returns, system prompts) are out of scope: user
input should not be silently rewritten, and oversized tool returns are the job of
ToolOutputLimits, which reduces a return when the tool produces it. ClearToolResults keeps
the newest keep_pairs results intact, and with keep_pairs=0 it blanks a fresh return
outright rather than shrinking it.
Clamping rewrites message content, so it invalidates the provider’s prompt cache from the clamped message onward. That is unavoidable here — the alternative is a failed request.
A part is clamped only when it is oversized and the clamp actually shrinks it, so set
keep_head_chars + keep_tail_chars well below your per-part threshold.
Composes as the first tier of a TieredCompaction (run it before ClearToolResults):
it is the only zero-LLM way to keep a run alive after a runaway generation.
Clamp a part whose estimated token count exceeds this value. None disables this trigger.
Type: int | None Default: None
Clamp a part whose character count exceeds this value. None disables this trigger.
Type: int | None Default: None
Characters of the part’s head to retain.
Type: int Default: 2000
Characters of the part’s tail to retain.
Type: int Default: 2000
When True, also clamp oversized ToolCallPart args, not just response text.
Type: bool Default: True
Optional tokenizer for accurate token counting.
A callable that returns the token count for a given string.
When None, uses a ~4 characters-per-token heuristic.
Type: Callable[[str], int] | None Default: None
@async
def compact(
messages: list[ModelMessage],
ctx: RunContext[AgentDepsT],
) -> list[ModelMessage]
Clamp every oversized response text part (and tool-call args, if enabled).
@async
def before_model_request(
ctx: RunContext[AgentDepsT],
request_context: ModelRequestContext,
) -> ModelRequestContext
Clamp any oversized response part before the request is sent.
Bases: AbstractCapability[AgentDepsT]
Zero-cost in-place clearing of old tool results.
Replaces the content of the oldest tool results with a short placeholder while
keeping the most recent keep_pairs tool-call / tool-return pairs intact. Tool
calls remain paired with their (now-blanked) results, so the history stays valid.
No LLM calls are made.
This is the cheap first tier of compaction — tool results typically dominate context, and the agent can re-run a tool if it needs the data again.
Cache tradeoff: clearing rewrites message content, which invalidates the provider’s
prompt cache from the clear point onward (the next request pays a cache-write). Use
min_clear_tokens to skip clearing that reclaims too little to be worth busting the
cache.
Trigger clearing when the messages since the last cleared tool result exceed this value. None disables.
Clearing blanks content but removes no messages, so a plain message count would stay above
the threshold and clear one more result on every request, rewriting history (and busting the
prompt cache) each step. Counting from the last cleared result instead makes clearing happen
in batches, and the requests between two clears extend the cached prefix. Before the first clear
the count is the whole history. The kept pairs count toward it: right after a clear about
2 * keep_pairs messages follow the last cleared result, and each tool step adds two more, so
the next clear comes after about (max_messages - 2 * keep_pairs) / 2 tool steps. At or below
2 * keep_pairs it clears on every request.
Type: int | None Default: None
Trigger clearing when estimated token count exceeds this value. None disables.
Type: int | None Default: None
Trigger when estimated tokens exceed this fraction of the model’s context window.
Resolved per request from the request’s model, so one setting behaves correctly on any
model. Mutually exclusive with max_tokens.
Type: float | None Default: field(default=None, kw_only=True)
Window override in tokens. None resolves it from the request’s model.
Unlike fallback_context_window, this applies whether or not resolution succeeds. Reach
for it when the registry is confidently wrong: a beta- or tier-gated window it records as
the maximum, or a self-hosted endpoint whose model id describes someone else’s
deployment. Only consulted alongside max_fraction.
Type: int | None Default: field(default=None, kw_only=True)
Window assumed when the request’s model is not in the pricing registry.
Only consulted alongside max_fraction. Supply the real number for a deployment the
registry cannot resolve.
Type: int Default: field(default=DEFAULT_CONTEXT_WINDOW, kw_only=True)
Number of most-recent tool-call / tool-return pairs left untouched.
Type: int Default: 3
Replacement content for a cleared tool result.
Type: str Default: '[tool result cleared]'
Tool names whose results are never cleared.
Type: frozenset[str] Default: frozenset()
When True, also blank the arguments of the cleared tool calls.
Type: bool Default: False
Only clear if doing so reclaims at least this many estimated tokens.
Protects the prompt cache from being invalidated for a trivial gain. None always clears.
Type: int | None Default: None
Optional tokenizer for accurate token counting.
A callable that returns the token count for a given string.
When None, uses a ~4 characters-per-token heuristic.
Type: Callable[[str], int] | None Default: None
@async
def compact(
messages: list[ModelMessage],
ctx: RunContext[AgentDepsT],
) -> list[ModelMessage]
Blank the oldest tool results beyond the most recent keep_pairs.
@async
def before_model_request(
ctx: RunContext[AgentDepsT],
request_context: ModelRequestContext,
) -> ModelRequestContext
Clear old tool results if the conversation exceeds the configured threshold.
Bases: AbstractCapability[AgentDepsT]
Zero-cost in-place clearing of superseded file reads.
When the same file is read more than once, only the latest read keeps its content; earlier reads are blanked with a placeholder. Tool-call pairing is preserved. No LLM calls are made.
File identity is supplied by the file_key seam — given a ToolCallPart it returns
a stable key for the file being read, or None if the call is not a file read. There
is no default: file-read identification is agent-specific, and a wrong guess would drop
live data.
Map a tool call to a stable file key, or None if it is not a file read.
Type: Callable[[ToolCallPart], str | None]
Replacement content for a superseded file read.
Type: str Default: '[superseded file read]'
Optional message-count trigger. When both triggers are None, runs whenever invoked.
Type: int | None Default: None
Optional token-count trigger. When both triggers are None, runs whenever invoked.
Type: int | None Default: None
Trigger when estimated tokens exceed this fraction of the model’s context window.
Resolved per request from the request’s model, so one setting behaves correctly on any
model. Mutually exclusive with max_tokens.
Type: float | None Default: field(default=None, kw_only=True)
Window override in tokens. None resolves it from the request’s model.
Unlike fallback_context_window, this applies whether or not resolution succeeds. Reach
for it when the registry is confidently wrong: a beta- or tier-gated window it records as
the maximum, or a self-hosted endpoint whose model id describes someone else’s
deployment. Only consulted alongside max_fraction.
Type: int | None Default: field(default=None, kw_only=True)
Window assumed when the request’s model is not in the pricing registry.
Only consulted alongside max_fraction. Supply the real number for a deployment the
registry cannot resolve.
Type: int Default: field(default=DEFAULT_CONTEXT_WINDOW, kw_only=True)
Optional tokenizer for accurate token counting.
A callable that returns the token count for a given string.
When None, uses a ~4 characters-per-token heuristic.
Type: Callable[[str], int] | None Default: None
@async
def compact(
messages: list[ModelMessage],
ctx: RunContext[AgentDepsT],
) -> list[ModelMessage]
Blank every file read that is later superseded by a newer read of the same file.
@async
def before_model_request(
ctx: RunContext[AgentDepsT],
request_context: ModelRequestContext,
) -> ModelRequestContext
Deduplicate file reads, optionally gated on a size threshold.
Bases: AbstractCapability[AgentDepsT]
Zero-cost sliding-window trimmer.
When the conversation exceeds a configurable threshold (message count or estimated token count), the oldest messages are discarded while preserving tool-call / tool-return pairs. No LLM calls are made.
Trimming happens in before_model_request so it is transparent to the
rest of the agent run.
Trigger trimming when message count exceeds this value. None disables.
Type: int | None Default: None
Trigger trimming when estimated token count exceeds this value. None disables.
Type: int | None Default: None
Trigger when estimated tokens exceed this fraction of the model’s context window.
Resolved per request from the request’s model, so one setting behaves correctly on any
model. Mutually exclusive with max_tokens.
Type: float | None Default: field(default=None, kw_only=True)
Window override in tokens. None resolves it from the request’s model.
Unlike fallback_context_window, this applies whether or not resolution succeeds. Reach
for it when the registry is confidently wrong: a beta- or tier-gated window it records as
the maximum, or a self-hosted endpoint whose model id describes someone else’s
deployment. Only consulted alongside max_fraction.
Type: int | None Default: field(default=None, kw_only=True)
Window assumed when the request’s model is not in the pricing registry.
Only consulted alongside max_fraction. Supply the real number for a deployment the
registry cannot resolve.
Type: int Default: field(default=DEFAULT_CONTEXT_WINDOW, kw_only=True)
Number of tail messages to retain after trimming (message-count trigger).
The newest message is always kept, so 0 keeps one. With receipts=True, the receipt
counts as one of these, but never displaces the newest message.
Type: int Default: 40
Minimum original message-content tokens to retain in an unchanged recent suffix.
Whole messages and tool dependencies can exceed this floor. Attached instructions do
not count. When None, falls back to keep_messages; zero keeps legacy edge behavior.
Type: int | None Default: None
Optional tokenizer for accurate token counting.
A callable that returns the token count for a given string.
When None, uses a ~4 characters-per-token heuristic.
Type: Callable[[str], int] | None Default: None
When True, the first ModelRequest containing a UserPromptPart
is always kept after trimming, in addition to system prompts.
Type: bool Default: True
When True, prepend a deterministic compaction receipt recording how much history
was dropped, with a transcript handle when a TranscriptHandleProvider capability is attached.
Opt-in for now: the receipt text is content, so defaulting it on is deferred to the benchmark eval-rig pass. The mechanism itself is structural.
Type: bool Default: False
@async
def compact(
messages: list[ModelMessage],
ctx: RunContext[AgentDepsT],
) -> list[ModelMessage]
Drop the oldest messages down to the configured tail.
@async
def before_model_request(
ctx: RunContext[AgentDepsT],
request_context: ModelRequestContext,
) -> ModelRequestContext
Trim the message list if it exceeds the configured threshold.
Bases: AbstractCapability[AgentDepsT]
LLM-powered conversation compaction.
When the conversation exceeds a configurable threshold, older messages are summarized using a dedicated model call and replaced with a compact, structured summary message, preserving recent context and tool-call integrity.
This is the expensive tier — summarization turns input tokens into (pricier) output
tokens — so it is best used behind cheaper passes (see TieredCompaction).
The summary call’s usage is folded into the parent run’s usage (it counts as a real request), so cost accounting stays honest; note this also increments the run’s request count, which a request-count limiter would see.
Model used to generate summaries.
When None, inherits the model the request being compacted is going to. Core starts
that as the run’s model, so the two differ only where a capability replaced
ModelRequestContext.model; set this explicitly to pin the summarizer regardless.
Type: str | Model | None Default: None
Settings for the dedicated summary model call.
These merge over defaults carried by model, allowing the summary call to use a
policy that differs from the running agent without mutating the model.
Type: ModelSettings | None Default: field(default=None, kw_only=True)
If set, this handler is passed to the nested summary run, so the summarizer’s own model-streaming events surface to the caller.
Setting it also selects the streaming request path, which is what a summarizer endpoint
that rejects non-streaming requests needs; pass drain_summary_events to take that path without
handling the events. Left None, the summary request is non-streaming, which is what an
endpoint that rejects streaming requests needs. The handler receives the summary run’s own
RunContext, never the outer run’s, and the outer Agent.run(...) handler is not inherited.
Type: EventStreamHandler[object] | None Default: field(default=None, kw_only=True)
Capabilities attached to the agent that writes the summary.
The summary runs on its own Agent, so capabilities on the outer agent do not run on it.
Capabilities listed here do, with every hook of that run.
The summary agent has no deps, so these are AbstractCapability[None].
Type: Sequence[AbstractCapability[None]] Default: field(default=(), kw_only=True)
Trigger compaction when message count exceeds this value.
Type: int | None Default: None
Trigger compaction when estimated token count exceeds this value.
Type: int | None Default: None
Trigger when estimated tokens exceed this fraction of the model’s context window.
Resolved per request from the request’s model, so one setting behaves correctly on any
model. Mutually exclusive with max_tokens.
Type: float | None Default: field(default=None, kw_only=True)
Window override in tokens. None resolves it from the request’s model.
Unlike fallback_context_window, this applies whether or not resolution succeeds. Reach
for it when the registry is confidently wrong: a beta- or tier-gated window it records as
the maximum, or a self-hosted endpoint whose model id describes someone else’s
deployment. Only consulted alongside max_fraction.
Type: int | None Default: field(default=None, kw_only=True)
Window assumed when the request’s model is not in the pricing registry.
Only consulted alongside max_fraction. Supply the real number for a deployment the
registry cannot resolve.
Type: int Default: field(default=DEFAULT_CONTEXT_WINDOW, kw_only=True)
Number of tail messages to preserve after compaction (message-count trigger).
The newest message is always kept, so 0 keeps one.
Type: int Default: 20
Minimum original message-content tokens to retain in an unchanged recent suffix.
Whole messages and tool dependencies can exceed this floor. Attached instructions do
not count. When None, falls back to keep_messages; zero keeps legacy edge behavior.
Type: int | None Default: None
Prompt template for generating summaries.
Must contain a {messages} placeholder and no other replacement field. Write a literal
brace as {{ or }}.
Type: str Default: _DEFAULT_SUMMARY_PROMPT
Instructions for the internal agent that writes the summary.
summary_prompt shapes the user turn of the summary request; this sets the internal
agent’s static instructions, which Pydantic AI sends in the request’s system prompt.
Override it when the summarizer endpoint requires a fixed leading instruction.
Type: str Default: field(default=_DEFAULT_INSTRUCTIONS, kw_only=True)
Optional tokenizer for accurate token counting.
A callable that returns the token count for a given string.
When None, uses a ~4 characters-per-token heuristic.
Type: Callable[[str], int] | None Default: None
When True, the first ModelRequest containing a UserPromptPart
is always kept after compaction, in addition to system prompts. System prompts,
earlier summaries and receipts in it are dropped, as the new summary request
carries their current versions.
Type: bool Default: True
When True, feed any existing summary from a prior compaction back as an anchored
<previous-summary> block with an update instruction (preserve still-true, remove stale,
merge new) so it is updated in place rather than re-summarized — avoiding
summary-of-summary decay.
Type: bool Default: True
When True and the summarizer’s model family differs from the family that produced
the history, prepend a neutral one-line note marking the summary as a cross-model handoff
(Codex prior art, anti-confabulation). Only fires on a genuine family mismatch, so it is
cheap and off in the common same-model case; the note’s wording is flagged pending eval-rig.
Type: bool Default: False
When True, preserve recent summarized user messages (each truncated to
keep_user_messages_max_chars) alongside the summary. Retained messages consume the
keep_messages tail budget in message-count mode. With positive keep_tokens,
keep_messages caps only extra user copies, which do not displace the retained suffix. Supersedes
preserve_first_user_message.
Type: bool Default: False
Per-message character cap for keep_user_messages; oversized messages are truncated
with an explicit marker (the shared truncation-marker convention).
Type: int Default: 20000
Per-return character cap when rendering tool results for the summarizer. None renders
them whole.
Type: int | None Default: field(default=500, kw_only=True)
When True, append a deterministic compaction receipt after the summary noting how
much history was summarized, that the summary is secondhand, and — when a
TranscriptHandleProvider capability is attached — a persisted-run handle.
Opt-in for now: the receipt text is content, so defaulting it on is deferred to the benchmark eval-rig pass. The mechanism itself is structural.
Type: bool Default: False
def with_focus(focus: str) -> SummarizingCompaction[AgentDepsT]
Return a copy whose summary prompt prioritizes focus.
Used by compact_now so a user-invoked compaction can say what the summary must not
lose. The prompt is later run through str.format, so braces in a user- or
model-supplied focus are escaped to survive it.
SummarizingCompaction[AgentDepsT]
@async
def compact(
messages: list[ModelMessage],
ctx: RunContext[AgentDepsT],
) -> list[ModelMessage]
Summarize older messages, replacing them with a single summary message.
@async
def before_model_request(
ctx: RunContext[AgentDepsT],
request_context: ModelRequestContext,
) -> ModelRequestContext
Summarize older messages when the threshold is exceeded.
Bases: AbstractCapability[AgentDepsT]
Compact by judging each older step of the history: keep it verbatim, reduce it, or drop it.
A step is a model response together with the requests that answer it (its tool results, retry
feedback, and the next user prompt), so removing one never splits a tool call from its result. The
system prompt and first user message, and the most recent keep_tokens of history, are never
judged. Pinned content survives even when its step is removed.
For every other step, the judge (model) is shown the task (the first and latest user prompts), the
recent conversation and the step, and asked two questions in one request: will the agent need the
step’s exact content, and does it record anything that still matters. On a decision model, such
as system-one:<model> or TypeSafe’s typesafe:jev-latest, each answer is the model’s probability of
yes; any other model fills the same two fields as structured output. Steps are kept verbatim in order
of the first answer until the history fits the target, the rest are ranked by the second answer and
reduced while they fit, and whatever is left is dropped. A step holding a user prompt is never
dropped: it is reduced, or kept when it does not fit the summarizer’s input.
Reducing writes the reduced steps into one summary at the front of the history when a summarizer
is configured; without one, each tool result in a reduced step is cut to its first
truncated_result_chars characters and a note, and everything else in the step is kept.
Judge failures propagate, as a summarizer failure does: wrap this in FallbackCompaction to fall back
to another strategy. Inside a durable workflow the judgements and the summary are durable operations,
addressed by this capability’s id.
The judge: the model that judges each step, a decision model or any model that supports structured output.
Type: Model | KnownModelName | str
Writes the reduced steps into a summary, with its own model, prompt and settings.
Only its summary settings are used; its trigger and keep_* fields are not. None reduces a step by
truncating its tool results instead, which needs no language model.
Type: SummarizingCompaction[AgentDepsT] | None Default: None
Trigger when the estimated context exceeds this many tokens. Mutually exclusive with max_fraction.
Type: int | None Default: None
Trigger above this fraction of the request model’s context window. Mutually exclusive with max_tokens.
Type: float | None Default: None
Size to compact down to, in tokens. Defaults to half the trigger. Mutually exclusive with target_fraction.
Type: int | None Default: None
Size to compact down to, as a fraction of the request model’s context window.
Type: float | None Default: None
Window override in tokens, consulted only alongside a fraction.
Type: int | None Default: None
Window assumed when the request model cannot be resolved, alongside a fraction.
Type: int Default: DEFAULT_CONTEXT_WINDOW
Message tokens at the end of the history that are kept unchanged and never judged.
Type: int Default: 8000
Most tokens of reduced steps handed to the summarizer in one compaction; lower-ranked steps are dropped.
Type: int Default: 30000
Most tokens of a step the judge is shown: a longer step is shown by its start and end.
Type: int Default: 2000
Most tokens of the current task, and separately of the recent conversation, the judge is shown with a step.
A judge request holds at most judge_step_tokens plus twice this, as estimated at ~4 characters per
token. Code and JSON tokenize nearer 2 characters per token, so the defaults keep a request under
the 8,192-token input of small local decision models.
Type: int Default: 500
Most judge requests in flight at once.
Type: int Default: 8
Characters of a tool result kept when a step is reduced without a summarizer.
Type: int Default: 300
Insert a deterministic note after the head of the history saying how much was summarized or dropped.
Type: bool Default: False
Optional token counter; defaults to the characters-per-token heuristic.
Type: Callable[[str], int] | None Default: None
Stable identity for durable operations; two on one agent merge into one.
Type: str | None Default: 'relevance_compaction'
def with_focus(focus: str) -> RelevanceCompaction[AgentDepsT]
Return a copy whose judge, and summarizer if any, give particular weight to focus.
RelevanceCompaction[AgentDepsT]
@async
def before_model_request(
ctx: RunContext[AgentDepsT],
request_context: ModelRequestContext,
) -> ModelRequestContext
Compact when the estimated context exceeds the trigger.
@async
def compact(
messages: list[ModelMessage],
ctx: RunContext[AgentDepsT],
) -> list[ModelMessage]
Judge the steps between the protected head and tail, then keep, reduce or drop each one.
Bases: AbstractCapability[AgentDepsT]
Warns the model when the run approaches configured limits.
Each warning is enqueued as a request-scoped system prompt: the model sees it for the request it’s part of only, and it stays in the message history, so the cached prefix and the model’s earlier reasoning stay valid.
Older versions sent the warning as a user prompt marked with metadata instead. Those are stripped from the request when a history still holds them; a user prompt that quotes a warning is left alone, since they’re recognized by that metadata, never by their text.
Maximum allowed requests for the run.
Type: int | None Default: None
Maximum context-window size to warn against.
Type: int | None Default: None
Context limit as a fraction of the model’s real context window, resolved per request.
Use this instead of max_context_tokens when the same agent runs on models with
different windows. Mutually exclusive with max_context_tokens.
Type: float | None Default: field(default=None, kw_only=True)
Window override in tokens. None resolves it from the request’s model.
Unlike fallback_context_window, this applies whether or not resolution succeeds. Reach
for it when the registry is confidently wrong: a beta- or tier-gated window it records as
the maximum, or a self-hosted endpoint whose model id describes someone else’s
deployment. Only consulted alongside max_context_fraction.
Type: int | None Default: field(default=None, kw_only=True)
Window assumed when the request’s model is not in the pricing registry.
Only consulted alongside max_context_fraction. Supply the real number for a deployment
the registry cannot resolve.
Type: int Default: field(default=DEFAULT_CONTEXT_WINDOW, kw_only=True)
Maximum cumulative run token budget to warn against.
Type: int | None Default: None
Which limits should emit warnings. Defaults to all configured limits.
Type: list[WarningKind] | None Default: None
Fraction of a limit at which warnings begin (between 0 and 1).
Type: float Default: 0.7
Remaining request count at which iteration warnings become CRITICAL.
Type: int Default: 3
@async
def before_model_request(
ctx: RunContext[AgentDepsT],
request_context: ModelRequestContext,
) -> ModelRequestContext
Strip user-prompt warnings older versions sent, then enqueue a warning if a threshold is crossed.
Enqueued from before_model_request, the warning goes into this request and into the run’s
history, as a ModelRequest of its own that’s sent merged into the request it follows.
Bases: AbstractCapability[AgentDepsT]
Report context usage to the application before each model request.
A compaction strategy knows when to act but says nothing about how close the run is to the limit, so an application that wants to show “context: 73%” has to re-count the history itself and guess the denominator. This capability does neither: it reuses the same estimator the strategies use and the model’s real context window.
It only observes — it never edits the history.
Order matters: register it after a compaction capability to see the compacted history, or before it to see what triggered the compaction. After a preceding compactor rewrites anchored history, the reading subtracts that compactor’s heuristic reclaim while retaining the anchor’s fixed provider overhead.
Deprecated callback with a fresh reading before every model request.
Subscribe to ContextUsageEvent instead. A coroutine function is awaited and an exception
raised here propagates and fails the run.
Type: Callable[[ContextUsage], None | Awaitable[None]] | None Default: None
Window override in tokens. None resolves it from the request’s model.
Type: int | None Default: None
Window assumed when the request’s model is not in the pricing registry.
Type: int Default: DEFAULT_CONTEXT_WINDOW
Optional tokenizer, matching the one your compaction strategy uses.
Type: Callable[[str], int] | None Default: None
@async
def before_run(ctx: RunContext[AgentDepsT]) -> None
Reset the request correction even when the caller reuses a run ID.
@async
def before_model_request(
ctx: RunContext[AgentDepsT],
request_context: ModelRequestContext,
) -> ModelRequestContext
Measure the pending history, emit it, and invoke the compatibility callback.
A single reading of how full the context is.
Estimated tokens in the message history about to be sent.
Counted by estimate_context_tokens: the provider-reported usage of the most recent model
response (tool schemas included) plus an estimate of messages and newly revealed tool schemas
added since. With no reported usage, message text falls back to the character heuristic;
schemas named by availability deltas are still estimated from the pending request
parameters, but other tool schemas remain outside that heuristic.
Type: int
Context window the reading is measured against.
Type: int
Whether window_tokens is the model’s real window or the fallback.
A gauge can render an unresolved window differently — the percentage is a guess when the model is not in the pricing registry.
Type: bool
used_tokens as a fraction of the window.
Type: float
def pin(content: str) -> UserPromptPart
Mark content so every shipped compaction strategy preserves it.
The returned UserPromptPart can be placed in a ModelRequest in the run’s message
history (e.g. by a capability or by the user); compaction keeps it verbatim.
def is_pinned(part: object) -> bool
Return True if part carries the pin metadata marker.
def reinject_pinned(
original: Sequence[ModelMessage],
compacted: list[ModelMessage],
*,
protected_tail: int = 0,
insert_at: int | None = None,
) -> list[ModelMessage]
Re-inject any pinned parts from original that compacted dropped.
Pinned parts already present in compacted are left where they are; missing ones are
gathered into a single ModelRequest placed right after any leading system/summary
messages, so they sit near the top of the surviving history. A no-op when original has
no pins or all pins survived, so it is always safe to call. protected_tail bounds
insertion to the prefix before that many unchanged trailing messages. insert_at places the
recovered pins at that index instead, for a strategy whose leading messages must stay as they were.
Bases: Protocol
A capability that can hand out a handle for its persisted run history.
Any capability implementing this method is discovered from RunContext.capabilities; the
first non-None handle is used. StepPersistence implements it by returning its run_id.