/jobs

Principal Software Engineer

Tech Lead — Managed Ingest & Synthetics Monitoring

Last updated:

Everything in Logfire starts with ingest. Today we ingest OpenTelemetry — OTLP traces, logs, and metrics — over gRPC and HTTP into our Rust ingest processor at high volume.

The next chapter is bigger: a managed ingest platform. We want customers to get their data into Logfire without standing up their own infrastructure — managed OpenTelemetry collectors, and direct ingest from cloud providers (AWS CloudWatch, Google Cloud Logging, Azure Monitor). And we want to ship synthetics monitoring — scheduled probes that test endpoints from the outside — as a native part of the observability product. Much of this is greenfield.

We're looking for a Principal Engineer to be the technical lead for this area: own the architecture, set the direction, and build it with a small team.

  • Lead the technical direction of the managed ingest platform — OTLP ingest, managed collectors, and cloud-provider (CSP) ingest — and own the reliability and scale problems that come with it.
  • Design and build CSP ingest integrations: pulling logs and metrics from AWS CloudWatch, GCP Cloud Logging, and Azure Monitor (subscription management, cross-account auth, schema mapping, large-scale parsing).
  • Build synthetics monitoring from the ground up: a distributed probe scheduler and runners (HTTP/gRPC/DNS), availability and latency metrics, and integration with our alerting.
  • Harden the ingest path: durability and replay, backpressure and adaptive concurrency, multi-region, and per-tenant fairness and limits.
  • Act as tech lead — set technical direction, run design reviews, mentor, and stay hands-on in the code.
  • Instrument everything with Logfire (we run on our own product).

We expect a candidate for this position to have:

  • A track record of leading the design and delivery of high-throughput data or ingestion systems at a senior/principal level.
  • Deep knowledge of OpenTelemetry (OTLP, the collector, semantic conventions) and observability data models.
  • Strong systems engineering skills, comfortable in Rust (our ingest path is Rust) — or able to ramp quickly given strong systems experience in a comparable language.
  • Hands-on experience with at least one major cloud's telemetry/logging stack (AWS CloudWatch, GCP Cloud Logging, or Azure Monitor) and its auth model.
  • A strong distributed-systems foundation: durability, backpressure, idempotency, multi-region, and the failure modes of high-volume pipelines.
  • The instincts of a technical leader who lifts the team through design and mentorship, not just personal output.
  • At least 8 years of software engineering experience.

Nice to haves but not required:

  • Experience building or operating an observability/telemetry ingestion pipeline at scale.
  • Contributions to OpenTelemetry or related open source.
  • Experience with synthetics / uptime monitoring or distributed probe systems.
  • Apache Arrow / Parquet / object-storage data pipelines.
  • Python and/or TypeScript for the surrounding services.
  • Live and work in a timezone between PT (UTC-8) and CET (UTC+1)
  • Able to travel to EU, UK and US up to 4 times a year to join our off-sites
  • Willing to join our on-call rotation, roughly 1 week in every 10

Pydantic Validation is the data validation library that powers modern Python development - 500 million downloads per month, used by virtually every tech company you've heard of. Why? Because we obsess over developer experience and write code we'd actually want to use ourselves.

We're applying that same engineering mindset to Pydantic Logfire, our observability platform with first class support for AI engineering, built for today's development reality: AI workloads, multi-language environments, and cloud infrastructure that's designed to be straightforward to set up and maintain.

We build with technologies developers actually want to work with:

  • OpenTelemetry for standardized instrumentation
  • SQL for intuitive querying (no proprietary query language to learn)
  • Rust, Python, and TypeScript for performance and productivity
  • Postgres, DataFusion, and object storage for scalable backends

Unlike other companies that pay lip service to open source, we commit over 20% of our engineering team to maintaining and expanding our open source ecosystem. This includes the core Pydantic Validation library and Pydantic AI - our rapidly growing framework that's becoming the standard for AI application development. We're signatories of the open source pledge and build on open standards because we believe in interoperability, not lock-in. Use our OpenTelemetry-based SDK with any compatible backend - we're confident you'll choose us on merit.

We're backed by Sequoia Capital and run a fully remote team across multiple time zones (with regular in-person offsites - next one is June 2026 in London).

Join our team of exceptional engineers who value substance over hype, practical approaches over perfectionism, and meaningful progress over busyness. We've built a culture that balances technical ambition with sustainable practices—minimal meetings and respect for your expertise and time. We're creating tools that genuinely improve developers' lives, and we're looking for thoughtful contributors who share our commitment to quality and our passion for elegant solutions.

  • 💰 Compensation: Competitive salary and stock options
  • 🌍 Truly Remote: Work from anywhere within our timezone range - no office requirements
  • 🌐 Global & Diverse: Join a multi-cultural team of 8+ nationalities
  • 💪 Impact: Direct influence on tools used by millions of developers worldwide
  • 🎯 Focus on Growth: Regular opportunities for learning and professional development
  • 🤝 Team Gatherings: Connect with the team at our regular international off-sites
  • 🏥 Healthcare: Comprehensive health coverage for you and your dependents
  • 🎮 Flexible Hours: Work when you're most productive
  • 💻 Equipment: Budget for your home office setup
  • ⚖️ Work-Life Balance: flexible working hours and 33 days PTO no matter where you live (including public holidays, which you can choose to take or not)

To apply, email careers@pydantic.dev with the job title in the subject line. We'd also appreciate a few lines explaining why you think you'd be a good fit for the role and what you've done in the past that evidences that.

No recruiters or agencies please. Unsolicited recruiters will be marked as spam.

To make your application stand out, please include examples of ingestion, telemetry, or distributed-systems work you've led — including any relevant open source contributions.