Skip to content
Docs Pydantic Docs

Work with large tool results without filling the context window

“Which recent deploys coincide with more exceptions?” To answer, an agent needs the error counts for every service over several hours, the list of deploys, and a comparison of error rates before and after each deploy. As tool results, that is hundreds of rows that the model must read and compare in its head. Every row takes space in the model’s context window, costs tokens, and gives the model another chance to misread a number.

With Agent Sandboxes, the model writes a script that fetches the rows, compares them in Python, and returns only the deploys with a higher proportion of exception-bearing spans. The rows stay in the sandbox. The model reads one line.

This page uses an on-call agent as the example. The same pattern fits any agent whose tools return more data than the model needs to read: account histories, search results, spreadsheets, or log lines.

Before you start

  • Follow Run code from Python to install logfire[agent-sandboxes] and set LOGFIRE_API_KEY to a key with permission to run code in agent sandboxes.
  • Install the query client dependency with pip install httpx.
  • To query your real data, create a read token (a credential that lets code query your project’s data) in Settings → Read tokens. See Query API.

Give the agent the data sources as functions

The first function runs SQL against your Logfire project with the query client. The second returns recent deploys. Replace it with a call to your deploy system.

tools.py
import os
from datetime import datetime, timedelta, timezone

from logfire.query_client import AsyncLogfireQueryClient


async def query_logfire(sql: str) -> list[dict]:
    """Run a SQL query against the Logfire project's last 24 hours of data and return the rows."""
    since = datetime.now(timezone.utc) - timedelta(hours=24)
    async with AsyncLogfireQueryClient(read_token=os.environ['LOGFIRE_READ_TOKEN']) as client:
        result = await client.query_json_rows(sql, min_timestamp=since)
    return result['rows']


async def list_deploys(since_hours: int) -> list[dict]:
    """List deploys from the last `since_hours` hours, oldest first."""
    now = datetime.now(timezone.utc).replace(minute=0, second=0, microsecond=0)
    return [  # replace with a call to your deploy system
        {'service': 'checkout', 'version': 'v412', 'deployed_at': (now - timedelta(hours=4)).isoformat()},
        {'service': 'auth', 'version': 'v88', 'deployed_at': (now - timedelta(hours=3)).isoformat()},
        {'service': 'search', 'version': 'v1201', 'deployed_at': (now - timedelta(hours=1)).isoformat()},
    ]

The sandbox never sees LOGFIRE_READ_TOKEN. It can only call query_logfire and list_deploys, and both run in your application.

See the script your agent writes

A model asked “which recent deploys coincide with more exceptions?” can write a script like this one:

main.py
import asyncio

from tools import list_deploys, query_logfire

import logfire

logfire.configure()

AGENT_CODE = '''
from datetime import datetime, timedelta

rows = await query_logfire("""
    SELECT service_name, time_bucket('1 minute', start_timestamp) AS bucket,
           count(*) FILTER (WHERE is_exception) AS errors, count(*) AS spans
    FROM records
    WHERE start_timestamp > now() - interval '6 hours'
      AND kind = 'span'
    GROUP BY service_name, bucket
""")
deploys = await list_deploys(since_hours=6)


def error_rate(service, start, end):
    window = [r for r in rows if r['service_name'] == service and start <= datetime.fromisoformat(r['bucket']) < end]
    spans = sum(r['spans'] for r in window)
    return sum(r['errors'] for r in window) / spans if spans else None


suspects = []
for d in deploys:
    at = datetime.fromisoformat(d['deployed_at'])
    before = error_rate(d['service'], at - timedelta(minutes=30), at)
    after = error_rate(d['service'], at, at + timedelta(minutes=30))
    if before is not None and after is not None and after > 2 * before and after > 0.005:
        suspects.append({**d, 'error_rate_before': round(before, 4), 'error_rate_after': round(after, 4)})
suspects
'''


async def main():
    async with logfire.agent_sandboxes() as pool:
        async with pool.checkout() as session:
            result = await session.feed_run(
                AGENT_CODE,
                external_lookup={'query_logfire': query_logfire, 'list_deploys': list_deploys},
            )
    print(result)


asyncio.run(main())

Set LOGFIRE_READ_TOKEN and run python main.py. The output depends on your data. In a project with six services and a bad auth deploy, the query returns 2,160 rows, and the script prints one deploy:

[{'service': 'auth', 'version': 'v88', 'deployed_at': '2026-10-23T09:00:00+00:00', 'error_rate_before': 0.0057, 'error_rate_after': 0.0397}]

The model reads the summary instead of all 2,160 rows. This ratio counts exception-bearing spans, not failed requests. A change after a deploy is a lead to investigate. It does not prove that the deploy caused the exceptions.

This example closes its session after the first answer. For a follow-up, keep the session open or save its ID and restore it. See Keep an agent’s work across conversation turns.

We asked this question of a Pydantic AI agent with gpt-6.1-sol, three times with plain tool calls and three times with Agent Sandboxes through Code Mode. Both found the auth deploy every time. With plain tool calls, the model read all 2,160 rows, and each run used about 70,500 input tokens. With Agent Sandboxes, each run used about 4,000. These three-run results use fixed sample data. They do not predict production cost or accuracy.

In separate runs with gpt-6.1-sol and 288 rows (five-minute buckets), plain tool calls used about 10,000 input tokens and were also correct every time. Agent Sandboxes used about 22,600, so a sandbox did not help. These results do not establish how token use scales with the number of rows. A smaller model, gpt-6-luna, often returned every row from its scripts instead of a summary, and used more tokens with Agent Sandboxes than without. Tell your agent to return a summary from its scripts if your model does not do it unprompted.

In your application, the agent writes the script and your code runs it. See Use Agent Sandboxes with Pydantic AI or Add a code tool to any agent.

Why a script beats SQL alone here

The query does the part that SQL is good at: counting errors per service in one-minute buckets. The comparison needs data that this query does not contain (the external deploy list). A script can join any number of sources, and the model can change the comparison for each question without you adding a new tool.

When one query can answer the whole question, let the agent run that query as a plain tool call. A separate sandbox step may be unnecessary.

A sandbox session has a 20 MiB memory limit, including interpreter state and retained variables, so a script can work with a few thousand rows at a time, not millions. Let the query aggregate first, as the one-minute buckets do here, and let the script compare and join.

Verify it worked

Open the Live view. You should see a run code span with a child span for each of the two tool calls. Open the query_logfire span to see the SQL the model wrote and the 2,160 rows it returned. You can see every row the script used, even though the model never read them.

Troubleshooting

  • MontyRuntimeError: TimeoutError: feed time limit exceeded: each run in a hosted sandbox can use only a short amount of processing time. Time spent waiting for your tools does not count. Let the query do the heavy aggregation, and keep the script to comparing and filtering. See What sandbox code can do.
  • KeyError on a column name: the script used a column the query did not select. The error message goes back to the model, which usually corrects the script on its next try.
  • 429 Too Many Requests from query_logfire: a daily query budget is exhausted, or a queued query waits too long. Wait before you retry and reduce query work. See query limits.

Next steps