Observe services without code changes using OBI

See HTTP and gRPC (a remote procedure call protocol) requests from Linux services without changing their application code. OpenTelemetry eBPF Instrumentation (OBI) observes supported processes at the operating-system level and sends the resulting data directly to Logfire. Each span is one unit of work: a single operation, with a name, a start, and a duration. A trace is the full journey of one request, made of nested spans. A metric is a number tracked over time, like requests per second or CPU load.

OBI is useful when you cannot add an SDK to a service, or when you want a quick inventory before adding deeper instrumentation. It does not collect application logs or instrument application-level work such as model calls. OBI can trace supported database protocols, but add an OpenTelemetry or Logfire SDK when you need deeper application detail.

OBI is a pre-release OpenTelemetry project. Its configuration and telemetry can change between v0 minor releases, so this guide pins the exact OBI v0.13.0 image tested with Logfire.

Before you start

You need:

• A Linux host and kernel supported by OBI.
• Permission to inspect the target processes and load extended Berkeley Packet Filter (eBPF) programs into the kernel.
• A Logfire project and a write token from Project settings > Write tokens.
• The endpoint for your Logfire data region: https://logfire-us.pydantic.dev for the US or https://logfire-eu.pydantic.dev for the EU.

OBI needs host-level access

The Docker example uses the host process namespace and privileged mode. This lets OBI inspect every process on the host and load eBPF programs. Review OBI’s security guidance and use its narrower capability set before deploying to production.

Run OBI with Docker Compose

Keep the Logfire write token outside the Compose file:

Terminal

export LOGFIRE_TOKEN='your-write-token'

Add OBI beside the service you want to observe. This example assumes the target executable is /app/checkout-api and selects it together with container port 8080. Replace the image and executable path with those for your service. It sends both traces and metrics through the OpenTelemetry Protocol (OTLP), the standard wire format Logfire uses to receive that data:

This sends data to Logfire

This configuration sends observed request and service metadata to your Logfire project, where Logfire stores it according to the project’s retention settings. Review the attributes OBI captures before enabling it for production traffic.

services:
  checkout-api:
    image: your-registry/checkout-api:latest
    environment:
      OTEL_SERVICE_NAME: checkout-api
      OTEL_RESOURCE_ATTRIBUTES: >-
        service.namespace=shop,
        service.version=1.4.0,
        deployment.environment.name=production
    ports:
      - "8080:8080"

  obi:
    image: otel/ebpf-instrument:v0.13.0@sha256:5e89d7478b5feeb8ee73881c58bfe5bb0ccb6dcd4f8cd62e30457aa6e6426adb
    pid: host
    privileged: true
    restart: unless-stopped
    environment:
      OTEL_EBPF_OPEN_PORT: "8080"
      OTEL_EBPF_AUTO_TARGET_EXE: /app/checkout-api
      OTEL_EBPF_ENFORCE_SYS_CAPS: "1"
      OTEL_EXPORTER_OTLP_ENDPOINT: https://logfire-us.pydantic.dev
      OTEL_EXPORTER_OTLP_HEADERS: Authorization=${LOGFIRE_TOKEN}
      OTEL_EXPORTER_OTLP_PROTOCOL: http/protobuf

Change the endpoint to https://logfire-eu.pydantic.dev for an EU project. Start the services, then follow the OBI logs:

Terminal

docker compose up -d
docker compose logs -f obi

Wait until the logs contain instrumenting process, then press Ctrl+C and send several requests:

Terminal

for request in 1 2 3 4 5; do
  curl --fail http://localhost:8080/health
done

OTEL_EBPF_OPEN_PORT matches the port opened by the process inside its container. If you publish container port 8080 as host port 18080, keep the selector set to 8080. OTEL_EBPF_AUTO_TARGET_EXE matches the target’s full executable path. OBI requires both selectors to match, which prevents it from instrumenting Docker’s port-forwarding process instead.

Set OTEL_SERVICE_NAME and the service metadata on each target workload. For one OBI instance that observes multiple services, configure discovery metadata for each target. Use OBI’s service discovery configuration when you need to select workloads by executable, container, namespace, or Kubernetes metadata.

Deploy OBI on Kubernetes

Use the official OBI Helm chart to run OBI as a DaemonSet, which places one OBI pod on each node. The chart’s application preset discovers workloads across the cluster and configures the required host process namespace, privileges, trace filesystem mount, service account, and role-based access control (RBAC).

Create a Secret containing only the Logfire write token:

Terminal

kubectl create namespace obi
printf '%s' "$LOGFIRE_TOKEN" | kubectl --namespace obi create secret generic logfire-otlp \
  --from-file=token=/dev/stdin

Save these chart overrides as obi-values.yaml. They replace the chart’s default exporters with OTLP over HTTP exporters that send traces and metrics directly to Logfire:

image:
  registry: docker.io
  repository: otel/ebpf-instrument
  tag: v0.13.0
  digest: sha256:5e89d7478b5feeb8ee73881c58bfe5bb0ccb6dcd4f8cd62e30457aa6e6426adb

envValueFrom:
  LOGFIRE_TOKEN:
    secretKeyRef:
      name: logfire-otlp
      key: token

config:
  data:
    file_format: "1.0"
    tracer_provider:
      processors:
        - batch:
            exporter:
              otlp_http:
                endpoint: https://logfire-us.pydantic.dev
                encoding: protobuf
                headers:
                  - name: Authorization
                    value: ${LOGFIRE_TOKEN}
    meter_provider:
      readers:
        - periodic:
            interval: 60000
            exporter:
              otlp_http:
                endpoint: https://logfire-us.pydantic.dev
                encoding: protobuf
                headers:
                  - name: Authorization
                    value: ${LOGFIRE_TOKEN}
                default_histogram_aggregation: explicit_bucket_histogram
    extensions:
      obi:
        version: "2.0"

Install the chart:

Terminal

helm repo add open-telemetry https://open-telemetry.github.io/opentelemetry-helm-charts
helm repo update
helm install obi open-telemetry/opentelemetry-ebpf-instrumentation \
  --version 0.14.0 \
  --namespace obi \
  --values obi-values.yaml

Change both endpoints to https://logfire-eu.pydantic.dev for an EU project. The application preset instruments applications cluster-wide by default. Add OBI v2 capture rules under config.data.extensions.obi.capture when you need to limit discovery to particular namespaces, labels, ports, or executables.

To observe only one workload, follow OBI’s manual Kubernetes deployment guide and add OBI as a sidecar. A sidecar needs shareProcessNamespace: true on the pod, the required security context, and the host’s /sys/kernel/tracing directory mounted at the same path. Configure the same OTLP endpoint, protocol, and authorization header on that OBI container. Create the Secret in the workload’s namespace, replacing checkout with that namespace:

Terminal

export WORKLOAD_NAMESPACE=checkout
printf 'Authorization=%s' "$LOGFIRE_TOKEN" | kubectl --namespace "$WORKLOAD_NAMESPACE" \
  create secret generic logfire-otlp --from-file=headers=/dev/stdin

Then reference that Secret from the sidecar:

env:
  - name: OTEL_EXPORTER_OTLP_ENDPOINT
    value: https://logfire-us.pydantic.dev
  - name: OTEL_EXPORTER_OTLP_PROTOCOL
    value: http/protobuf
  - name: OTEL_EXPORTER_OTLP_HEADERS
    valueFrom:
      secretKeyRef:
        name: logfire-otlp
        key: headers

Use standard workload labels so service names remain stable when pods are replaced.

Verify the telemetry

Generate several requests, then allow up to one minute for OBI v0.13’s default metrics export interval.

1. Open Live and filter by service_name = 'checkout-api'. You should see HTTP or gRPC server spans, which are units of work with a start time and duration.
2. Open Services and confirm that checkout-api appears with recent activity.
3. Open Explore > Metrics and run:

SELECT
  DISTINCT metric_name
FROM metrics
WHERE service_name = 'checkout-api'
  AND metric_name IN ('http.server.request.duration', 'target.info')
ORDER BY metric_name;

You should see:

metric_name
--------------------------------
http.server.request.duration
target.info

OBI places the discovered workload’s identity values on each target.info datapoint instead of its enclosing OpenTelemetry resource. Logfire promotes those values into its core service_name, service_namespace, service_version, service_instance_id, and deployment_environment fields so the metadata metric appears under the same service as its request traces and metrics.

Troubleshoot the setup

Symptom	Cause and fix
No traces or metrics appear	Check the OBI logs for instrumenting process. If it is missing, check for missing capabilities or an unsupported kernel and confirm the selectors match the target process. Send traffic only after OBI attaches.
OBI reports 401 or 403	Use a write token for the intended project and set OTEL_EXPORTER_OTLP_HEADERS to Authorization=your-write-token.
The service appears as unknown_service	Set OTEL_SERVICE_NAME on the target workload. Add service.namespace, service.version, and deployment.environment.name through OTEL_RESOURCE_ATTRIBUTES for clearer grouping.
OBI repeatedly reports 422 before discovering a process	Upgrade the receiving self-hosted Logfire deployment. Current managed Logfire accepts OBI v0.13’s startup-only metric envelope as an empty success.
Requests appear but application logs do not	OBI does not collect application logs. Send them through an OpenTelemetry SDK or the Logfire SDK.

For deeper traces inside each request, continue with a Logfire integration or configure an alternative OpenTelemetry client.