How Dosu cuts agent debugging time by 90% without leaving its coding agent

#The challenge: Tracing an agent-first system

Dosu has been building agents for over three years. The team initially built its agent framework in-house because agent frameworks were still so early that committing to one risked locking into an approach that would soon be outdated. An in-house framework, traced with LangSmith, kept the team free to adapt as the space matured. The agent framework they would later adopt, Pydantic AI, did not exist yet.

However, their scaffolding around LangSmith was built for a premodern agent harness world. When Dosu decided to rebuild its agents to take advantage of the latest harness engineering approaches, the team needed a tool that gave them the flexibility to maintain pieces of their own framework while giving them better traceability at the agent level rather than the function level.

#The solution: Pydantic AI for the agents, and Logfire for the AI observability

Once they decided to adopt a framework, Dosu evaluated the Claude Agent SDK, OpenAI's Agents SDK, and LangChain's agent framework, and ultimately chose Pydantic AI. It was the easiest to extend alongside the code Dosu already had, and it fit cleanly with their existing stack including DBOS, the durable-execution layer Dosu's long-running agents run on. And it was the least opinionated, tied to no single model provider's ecosystem.

Adopting Pydantic Logfire for AI observability took a single step with Pydantic AI's built-in instrumentation, and it closed the gap LangSmith left open: Logfire treats an agent's conversation as a first-class object, so the team reads a run's chat history directly instead of reconstructing it from tool calls.

"Seeing the conversation, the chat history, as a first-class citizen in Pydantic Logfire was a big unlock for us."

— Devin Stein, Founder and CEO at Dosu

#The Logfire MCP server: Debugging from the coding agent

An unexpected benefit of migrating to Logfire was the MCP server. Most of Dosu's debugging now happens inside a coding agent rather than in the Logfire UI. The Logfire MCP server exposes trace data to their agent, so an engineer asks about production behavior in plain language and gets cited answers back without opening Logfire.

"Being able to ask questions right within our coding agent and get cited metrics back has been invaluable. It's rare that I have to really dig into Logfire unless it's a really gnarly problem."

— Taylor Dolezal, Head of OSS at Dosu

The first payoff is triage. When a trace looks wrong, the team used to click through it by hand. Some runs exceed 25 minutes and stack up a long list of tool calls. Across a 14-day period, the team worked through 193 production traces this way. Now the coding agent uses the MCP server to filter a trace down to the sections that matter.

"Some of our traces run for over 25 minutes, and that's a lot of tool calls to sift through manually. The Logfire MCP makes it a lot easier and cuts debugging time down by 90%."

— Devin Stein, Founder and CEO at Dosu

The bigger payoff is a self-improvement loop. Devin says "using coding agents and the MCP to analyze runs, suggest improvements, and implement them, has been really powerful for us." Dosu queries Logfire for its worst runs, a run that lasted 45 minutes, a tool call that took far too long, hands them to a coding agent to analyze, and lets it propose and implement the fix. Find the problem, fix it, ship it, from one session.

#One view of every agent: The Logfire agents dashboard

Dosu runs many agents, and the Logfire agents dashboard gives the team an aggregate view of all of them out of the box. Seen together, it revealed expensive cost bugs that were nearly impossible to spot one run at a time.

"Getting an aggregate view of everything out of the box was really nice. It also helped us identify some pretty big bugs around prompt caching that were really hard to track down."

— Devin Stein, Founder and CEO at Dosu

#What's next: Pydantic Evals and capabilities

Dosu is moving its in-house evals onto Pydantic Evals, which is integrated with Logfire. The custom pieces the team built on Pydantic AI, safe shared state across parallel agents, file-backed context management, subagents, and resumable runs, are next in line to move onto Pydantic AI's built-in capabilities.

#The results

The payoff shows up in speed. Time to root cause on an agent issue has dropped sharply from around an hour down to just a few minutes. Most debugging never even requires the Logfire UI at all. The MCP server surfaces all the relevant information and gives developers answers directly within their coding agent. And the agents dashboard turned a class of hard-to-find cost bugs into something the team could see and fix, saving them over $30k annually.

#Key takeaways

• Match the tool to the shape of the system. A tracing UI built for linear tool calls fights an agent-first system. Dosu needed the conversation as a first-class object, and that drove the move to Pydantic Logfire.
• Put AI observability where the work happens. The Logfire MCP server moved debugging into the coding agent, so engineers ask questions in plain language.
• Close the loop. Query the worst runs, let a coding agent analyze them, and ship the fix from the same session, so observability actively improves the system.
• Aggregate views find what per-run views hide. Dosu's cost bugs only surfaced once every agent showed up in one place.

Interested in how Pydantic AI and Pydantic Logfire work together in production? Explore the docs or get started with Logfire.