Skip to content
Docs Pydantic Docs

Kubernetes monitoring with the OTel Collector

See cluster health, workload state, pod logs, and Kubernetes Events in Logfire without hand-building a monitoring pipeline. This guide uses the upstream OpenTelemetry Kubernetes Helm chart with a balanced configuration that keeps the Kubernetes page working without collecting every signal enabled by the chart.

What the standard setup collects

The setup on this page collects:

  • Kubernetes cluster, workload, node, pod, and container metrics once per minute.
  • Pod stdout and stderr, plus Kubernetes Events.
  • Kubernetes resource identity for telemetry sent through the Collector.

It does not collect persistent-volume metrics, broad Prometheus scrapes, or host metrics. Those signals are not required by the Kubernetes page and can add substantial volume. You can collect more Kubernetes data after the cluster setup works.

The Collector also accepts application telemetry over the OpenTelemetry Protocol (OTLP), but does not instrument application pods automatically.

Install with Helm

Use the upstream opentelemetry-kube-stack Helm chart to populate Logfire’s Kubernetes view.

Copy a write token, the credential that lets this deployment send data to your project, from Project → Settings → Write tokens.

Add the token to a Kubernetes Secret, then provide a small values.yaml to name the cluster, select the balanced data set, and route it to Logfire:

values.yaml
# values.yaml: Logfire-shaped overrides for opentelemetry-kube-stack.
# See the chart's own values.yaml for the full schema; this is only the
# overrides on top of the defaults.

clusterName: my-cluster   # shows up as the row label in the Clusters tab

# Generate the Operator webhook certificate during installation so this works
# on a new cluster without requiring cert-manager first.
opentelemetry-operator:
  admissionWebhooks:
    certManager:
      enabled: false
    autoGenerateCert:
      enabled: true

collectors:
  daemon:
    # Disable the chart's broad cAdvisor and annotated-pod Prometheus scrapes.
    scrape_configs_file: ""
    presets:
      # The Kubernetes page uses kubelet metrics for node resource usage.
      # Enable host metrics separately if you also use the Hosts page.
      hostMetrics:
        enabled: false
    # Scope the write token to the Collector. Top-level `extraEnvs` also
    # copies values into auto-instrumented application pods.
    env:
      - name: LOGFIRE_TOKEN
        valueFrom:
          secretKeyRef:
            name: logfire-token
            key: LOGFIRE_TOKEN
    # Override must live under `collectors.daemon.config`. The chart's
    # collector-specific config wins over `defaultCRConfig.config`.
    config:
      receivers:
        kubelet_stats:
          collection_interval: 60s
          metric_groups: [node, pod, container]
        k8s_cluster:
          collection_interval: 60s
      exporters:
        otlp_http/logfire:
          endpoint: https://logfire-us.pydantic.dev   # or https://logfire-eu.pydantic.dev
          headers:
            Authorization: "Bearer ${env:LOGFIRE_TOKEN}"
      service:
        pipelines:
          traces:  {exporters: [otlp_http/logfire]}
          metrics: {exporters: [otlp_http/logfire]}
          logs:    {exporters: [otlp_http/logfire]}
Terminal
helm repo add open-telemetry https://open-telemetry.github.io/opentelemetry-helm-charts
kubectl create namespace observability --dry-run=client -o yaml | kubectl apply -f -

printf 'Paste your Logfire write token: '
read -r -s LOGFIRE_TOKEN
printf '\n'
printf '%s' "$LOGFIRE_TOKEN" | kubectl -n observability create secret generic logfire-token \
  --from-file=LOGFIRE_TOKEN=/dev/stdin --dry-run=client -o yaml | kubectl apply -f -
unset LOGFIRE_TOKEN

helm upgrade --install otel-stack open-telemetry/opentelemetry-kube-stack \
  --version 0.20.6 \
  -n observability -f values.yaml

Data usually starts flowing within two minutes of the Collector pods reaching Ready.

Verify data arrives

Confirm that the Operator and Collector pods are running:

Terminal
kubectl -n observability get opentelemetrycollectors,pods

Within two minutes:

  1. Open Kubernetes in Logfire. You should see your cluster, nodes, namespaces, workloads, and pods.
  2. Open Live and filter by k8s.cluster.name. You should see pod logs and Kubernetes Events.

Monitor hosts too

The Kubernetes page gets node CPU and memory from kubelet metrics. To populate the separate Hosts view with load, disk, filesystem, network, and paging data, set collectors.daemon.presets.hostMetrics.enabled to true and collectors.daemon.config.receivers.host_metrics.collection_interval to 60s in an additional values file.

Production considerations
  • The quickstart generates the Operator webhook certificate so cert-manager is not required. The certificate is valid for 365 days and renewed by helm upgrade. If your cluster already uses cert-manager, enable opentelemetry-operator.admissionWebhooks.certManager and remove autoGenerateCert.
  • The chart disables kubelet certificate verification for compatibility across Kubernetes distributions. When the kubelet certificate chains to your cluster certificate authority, set collectors.daemon.config.receivers.kubelet_stats.insecure_skip_verify to false. Otherwise, mount its issuer certificate and configure ca_file.

Troubleshoot setup

SymptomWhat to check
No Collector podsRun kubectl -n observability logs deployment/otel-stack-opentelemetry-operator --container manager. Admission or custom-resource errors appear there.
Collector logs show 401 or 403Confirm that the endpoint uses the same region as the Logfire project and recreate the Secret with a current write token.
Collector runs but the Kubernetes page is emptyRun kubectl -n observability logs -l app.kubernetes.io/name=opentelemetry-collector --tail=200 and look for Kubernetes API or RBAC errors.
Application traces do not appearThe standard setup does not modify application pods. Follow Collect more data from Kubernetes to instrument them.

Next steps