Comparison

Logfire vs Braintrust

Logfire is the production AI improvement system: investigate the complete agent and application trace, annotate the runs that matter, score production and offline datasets with Pydantic Evals, let the optimizer propose a trace-backed change, then ship managed prompts, agent specs, tools, and skills with targeting and rollout controls. Add infrastructure monitoring, browser session replay, feature flags, and the industry-leading agent trace investigator—all in one OpenTelemetry-native product, with $0 per 1,000 scores.

Try Logfire free

As Markdown

Switching

Keep your eval suite

Already use Braintrust's Eval API? Change two environment variables and send your next runs to Logfire's Evals workspace. Your inline data, local tasks, and scorers can stay where they are.

See compatibility details Talk to us about hosted workflows

export BRAINTRUST_APP_URL="https://logfire-us.pydantic.dev/v1/braintrust"
export BRAINTRUST_API_KEY="<your-logfire-write-token>"

Use https://logfire-eu.pydantic.dev/v1/braintrust for an EU project. The endpoint does not import existing history or provide Braintrust-hosted datasets, prompts, functions, a model proxy, server-side scoring, or public sharing.

Feature comparison

From a production run to a better agent

Feature	Logfire	Braintrust
Production context	Browser, agents, services, databases, logs, metrics, and infrastructure	AI and application traces, logs, and OpenTelemetry spans
Full-stack observability suite	Infrastructure monitoring, service maps, logs, metrics, traces, and AI	✗
Browser session replay	Browser tracing and session replay alongside the agent trace	✗
First-class feature flags	OpenFeature/OFREP flags, targeting, and controlled rollout	✗
Evaluation workflow	Pydantic Evals: one evaluator model online and offline	Experiments, playgrounds, CI/CD, and online scoring
Score pricing	$0 per 1,000 scores	$1.50 per 1,000 scores after 50K/month on Pro
Human review	Annotation queues on production runs and evals	Human-review scores and assigned trace review
From failure to change	Trace-backed optimizer and managed agent configuration	Prompt, scorer, dataset, and environment workflows
Managed agent configuration	Prompts, agent specs, tools, skills, versioning, targeting, and rollout	✗
Controlled rollout	Immutable versions, labels, targeting, weighted rollout, and feature flags	Prompt, dataset, and parameter environments
Investigation workflow	Agent trace investigator, PostgreSQL-compatible SQL, and MCP across full telemetry	Logs, trace views, SQL, and MCP for Braintrust data
Deployment options	Cloud, Dedicated, or the same product self-hosted on Kubernetes	Cloud or an Enterprise self-hosted data plane

Score economics

Score freely at production scale

Production coverage	Scores	Estimated Braintrust Pro monthly charge	Logfire score meter
10M runs × 10% sampled × 3 scores	3M scores/month	$4,674/month	$0 score charges
10M runs × 25% sampled × 5 scores	12.5M scores/month	$18,924/month	$0 score charges
100M runs × 10% sampled × 5 scores	50M scores/month	$75,174/month	$0 score charges

Illustrative coverage models assume three or five recorded scores per sampled trace. Braintrust Pro published list pricing: $249/month + max(scores − 50,000, 0) ÷ 1,000 × $1.50. Figures include the platform fee, but exclude Braintrust processed-data and retention charges, and do not model provider or model costs an LLM-as-a-judge configuration may incur. Logfire charges $0 per 1,000 scores; telemetry records are billed on the selected Logfire plan.

Why Logfire

The full production improvement loop

Investigate the system, not only the output

An agent failure is often a browser, API, database, retrieval, tool, or infrastructure failure wearing an LLM-shaped mask. Logfire keeps those signals in one nested trace, with service maps, logs, metrics, SQL, and an agent trace investigator built for the production incident behind the score.

Evaluate online and offline without rationing coverage

Use the same Pydantic Evals evaluators for fast offline feedback and online production monitoring. Cheap heuristics can run on every run; LLM judges can sample the traffic that deserves them. With $0 per 1,000 scores, coverage is a quality decision, instead of a new billing meter.

Turn a human judgment into the next improvement

Annotation queues let reviewers work through the production runs that matter, with verdicts, failure categories, expected outputs, comments, and tags. That judgment stays linked to the trace, becomes a reusable evaluation case, and gives the optimizer grounded evidence for the next change.

Change the agent safely, without a second control plane

Logfire manages prompts, agent specs, tools, and skills as versioned configuration. Review a trace-backed proposal, then target a cohort, canary a weighted rollout, watch the live result, and roll back by moving a label. The version that served every run is part of that run's trace.

Decision Guide

Which should you choose?

Choose Logfire if...

✓You already have Braintrust evals and want to try Logfire without rewriting the suite
✓You need to diagnose agents in the context of the browser, service map, database, API, logs, metrics, and infrastructure
✓You want online and offline evaluation without a per-score charge
✓You want reviewers to work from annotation queues, then export an annotated failure into a reusable evaluation case
✓You want a trace-backed prompt optimizer to propose a production-grounded change
✓You want to version, target, canary, and roll back managed prompts, agent specs, tools, and skills
✓You want your coding agent to investigate the same telemetry with MCP and PostgreSQL-compatible SQL

Choose Braintrust if...

•You depend on Braintrust-hosted datasets, prompts, functions, the model proxy, server-side scoring, or public sharing

FAQ

Common questions

Can I send Braintrust SDK evals to Logfire?

Yes. Point the Braintrust SDK at Logfire and keep your existing eval suite. Future eval runs appear in the Evals workspace alongside the traces and telemetry that explain each result. Existing Braintrust history and Braintrust-hosted services are not imported.

How do Logfire and Braintrust compare on evaluation pricing?

Logfire charges $0 per 1,000 scores, so online and offline evaluation do not add a per-score meter. Braintrust Pro lists a $249 monthly platform fee with 50,000 scores included, then $1.50 per 1,000 scores. At 3 million scores in a month, estimated platform and score charges are $4,674; at 12.5 million, $18,924; at 50 million, $75,174. Those Braintrust figures exclude processed-data and retention charges, and do not model provider or model costs that an LLM-as-a-judge configuration may incur.

Do both Logfire and Braintrust support online and offline evaluation?

Yes. Both support offline experiments and asynchronous scoring of production traffic. Logfire uses the same Pydantic Evals evaluators online and offline, keeps every result attached to its OpenTelemetry trace, and does not charge per score.

What is Logfire's advantage for production AI systems?

Logfire investigates the whole production system, not only the AI output: browser activity, agent and model calls, tool calls, APIs, databases, logs, metrics, services, and infrastructure. From a failing run, teams can annotate it, query it with SQL or MCP, export it into an evaluation case, use the prompt optimizer to propose a trace-backed change, and ship managed prompts, agent specs, tools, and skills with versioning, targeting, and rollout controls.

Can Logfire replace Braintrust for production evaluation?

Yes. Logfire combines online and offline Pydantic Evals, annotation queues, $0 per 1,000 scores, trace-backed prompt optimization, and managed configuration with the full production context around an agent: browser, services, databases, logs, metrics, and infrastructure.

Ready to switch from Braintrust?

Get started with 10 million free spans per month. No credit card required.

Start free

View pricing