← Back to Changelog
NEW FEATURE

A new /v2/query endpoint

A new /v2/query endpoint is now available, and is expected to replace the existing /v1/query endpoint.

Users have historically encountered issues when using the /v1/query endpoint, in particular when a large amount of data was being queried. This was partly due to the fact that this endpoint was exposed from our Python backend, which required loading the data from our database in memory before returning it.

To mitigate this issue, the /v2/query endpoint is exposed directly from our database query API, and a new NDJSON format is now available to allow streaming responses.

This new endpoint has some differences compared to the /v1/query one:

  • The endpoint only accepts POST requests, unlike the /v1/query endpoint which accepts GET requests with query parameters. While theoretically there is no length limit for query parameters, some web services might set an arbitrary limit, and the sql query parameter might contain sensitive data.

  • The min_timestamp body parameter is now required, and it is recommended to use a small time range to reduce the query overhead.

  • The response JSON schema have changed slightly, and only provides the layout where rows are separated from columns (which could be obtained from the /v1/query endpoint using the opt-in json_rows=true query parameter):

    /v1/query

    {
  "columns": [
    {"name": "service_name", "datatype": "Utf8", "nullable": false, "values": ["backend"]}
  ]
}

    /v1/query (json_rows=true)

    {
  "columns": [
    {"name": "service_name", "datatype": "Utf8", "nullable": false}
  ],
  "rows": [
    {"service_name": "backend"}
  ]
}

    /v2/query

    {
  "schema": {
    "fields": [
      {"name": "service_name", "data_type": "Utf8", "nullable": false}
    ],
    "metadata": {}
  },
  "data": [
    {"service_name": "backend"}
  ]
}

The complete set of body parameters can be found in the documentation.

The query client available in the Python Logfire SDK was updated to automatically use the new endpoint. For backwards compatibility, the response from the /v2/query endpoint was adapted to match the structure of the /v1/query endpoint (with json_rows=True).

A couple deprecations were introduced in the latest release:

  • The query_json() method of the query client, which mapped to the /v1/query columns format was deprecated.
  • Using the query methods (query_json_rows(), query_arrow(), query_csv()) without providing a min_timestamp is deprecated.

These changes werer introduced in the v4.35.0 release.

The query endpoint can return data in a variety of formats, following the Accept request header:

  • application/json: returns JSON data (this is the default if not provided).
  • application/x-ndjson: returns NDJSON data in a streaming fashion.
  • application/vnd.apache.arrow.stream: Returns the data in Apache Arrow format, suitable for high-performance data processing.
  • text/csv: Returns the data in CSV format, which is easy to use with many data tools.

The application/x-ndjson format is a new addition to the /v2/query endpoint. See the documentation for details on the messages structure of this format.