Skip to content
Docs Pydantic Docs

For AI engineers

You’re building on a large language model, and the hardest part isn’t the first demo, it’s knowing what the model actually did on a real request and whether the answers are good. This path takes you from your first instrumented call to seeing every conversation and its cost, then to measuring quality and changing prompts without shipping code.

Follow these in order. Each link says why it’s here.

Your path

  1. Send your first trace: configure Logfire and instrument the LLM library you already use:

    See the full list under LLM integrations.

  2. Understand an agent run: once data is flowing, see how Logfire records the model’s conversation, tool calls, tokens, cost, and the code around them. A trace is the full record of one agent run or request; a span is one step inside it.

  3. Read every LLM call, with tokens and cost: the LLMs and providers view breaks down which models you’re calling, how many tokens each request used, and what it’s costing you.

  4. Evaluate your AI: measure output quality: move from “it looks fine” to evidence. An evaluation runs your AI over a set of test cases. A scorer judges each output and produces results such as pass/fail assertions, numeric scores, or categorical labels.

  5. Change a prompt without shipping code: version your prompts outside your codebase so you can test a new wording, promote it to production, and roll it back, all without a redeploy.

Try the full journey

  • Instrument and evaluate an agent: a start-to-finish walkthrough: build a small agent, see its traces, then score its answers with an evaluation.
  • Debug a slow tool call: when your agent is slow, prove in one trace whether it’s the model or a tool, and find the query behind it.
  • Roll out a prompt safely: change a live prompt, watch the effect, and roll back if it gets worse.
  • Track LLM cost: turn token usage into a dashboard and an alert so a runaway bill can’t surprise you.