Skip to content
Docs Pydantic Docs

Refine Output

Have an agent review and revise its answer before the run ends. Use it where a second pass catches mistakes that a first draft misses and the extra model request is worth it.

While Pydantic AI Harness is on 0.x releases, the API may change between minor releases; when it does, deprecation warnings and release-note migration guidance tell you (or your agent) exactly how to upgrade. See the version policy.

Usage

Terminal
pip install pydantic-ai-harness

from pydantic_ai import Agent
from pydantic_ai_harness import RefineOutput

agent = Agent('anthropic:claude-opus-5-5', capabilities=[RefineOutput()])

When the run would end with an output, RefineOutput adds a system prompt to the conversation that tells the model to review its last answer and give the complete answer again, revised. The model answers once more, with its tools available, and that answer is the run’s output. The earlier answer stays in the message history.

FieldDefaultMeaning
rounds1How many times per run the agent reviews its output
instructionsDEFAULT_REFINE_OUTPUT_INSTRUCTIONSWhat the agent is told at the start of each review round
from pydantic_ai_harness import RefineOutput

RefineOutput(rounds=2, instructions='Check every figure against the source documents, then answer again.')

Review rounds

  • A round costs at least one more model request, more when the model calls tools during it, and counts toward UsageLimits like any other request.
  • With structured output the model produces the output again. Output validators and output functions run on every answer, not only the last, so keep them free of side effects.
  • The instruction is a SystemPromptPart whose metadata holds 'origin': 'pydantic-ai-harness.refine_output.v1' and the round number. It keeps the default conversation scope, so it stays in the message history. Each run starts with no rounds used.
  • Code that streams the run sees every answer as the model writes it: the first one, then each revised one. Only the last is the run’s output.

Four kinds of run end without a review:

  • A run that pauses for deferred tools and returns DeferredToolRequests.
  • A run that ends while the provider is still processing a model request, such as a batch or background response, and returns PendingModelResponse.
  • A run whose output did not come from a model request, such as the refusal of an input guardrail that blocked the prompt. A review would send the model the prompt the guardrail kept from it. RefineOutput recognizes such an output by a response with no model_name, so a custom Model should set model_name on the responses it returns, or its runs are not reviewed.
  • Agent.run_stream(), which ends at the first output it streams. To stream a run that is reviewed, use Agent.run() with an event_stream_handler, or Agent.iter().

Durable execution

The rounds a run has used live in memory, in the instance for_run creates for that run. Under durable execution the hook that counts them (after_node_run) runs in workflow code, so when Temporal, DBOS, or Prefect replays a run, it runs again in the same order over the journaled responses and counts the same rounds.

Telemetry

RefineOutput emits no spans of its own. On an instrumented agent, each round shows as an ordinary model request in the run’s trace.

Composition

  • Several RefineOutput on one agent each count their own rounds, and each adds its instructions to every round up to its own rounds. Two with different instructions have the model review for both in the same round.
  • The count does not depend on the message history, so compaction that summarizes a review instruction away does not start the rounds again.
  • Trajectory Judge has a second model review the run while it is in progress. RefineOutput needs no second model and acts only on the output.
  • An output guardrail rejects an output that breaks a rule. RefineOutput asks for a second look at an output that was accepted.

See the source.

API reference

RefineOutput

Bases: AbstractCapability[AgentDepsT]

Send the agent back to review and revise its output before the run ends.

Each time the run would end with an output, a system prompt carrying instructions is added to the conversation and the model answers again, up to rounds times per run. The last answer is the run’s output.

from pydantic_ai import Agent
from pydantic_ai_harness import RefineOutput

agent = Agent('anthropic:claude-opus-5-5', capabilities=[RefineOutput()])

Not reviewed: a run that ends for its caller to resume, an output that did not come from a model request, and Agent.run_stream().

Attributes

rounds

How many times per run the agent reviews its output.

Type: int Default: 1

instructions

What the agent is told at the start of each review round.

Type: str Default: DEFAULT_REFINE_OUTPUT_INSTRUCTIONS

Methods

for_run

@async

def for_run(ctx: RunContext[AgentDepsT]) -> RefineOutput[AgentDepsT]

Return a fresh instance, so runs don’t share the rounds they used.

Returns

RefineOutput[AgentDepsT]

DEFAULT_REFINE_OUTPUT_INSTRUCTIONS

What RefineOutput tells the agent at the start of each review round, unless instructions is set.

Default: 'Review your last answer against the request. Look for mistakes, gaps, and anything unclear or unsupported. Fix what you find, using tools where needed, then give the complete final answer again, revised. Leave the review itself out of the answer.'