Visualizing OTLP Histograms as heatmaps

Docs > OpenTelemetry in Datadog > OpenTelemetry Guides and Resources > Visualizing OTLP Histograms as heatmaps

Overview

The OpenTelemetry Protocol (OTLP) supports sending OTLP Histograms, a type of metric that compresses information about a set of measurements by providing aggregate statistics, like sum, count, min, and max. OTLP Histograms also count how many of these measurements fall into user-configurable buckets.

You can visualize this datatype as a heatmap in Datadog by following the steps on this page.

Note: The related OTLP Exponential Histogram type can also be displayed as a heatmap, since it is converted to a distribution. Read more about distributions on the dedicated Distributions page.

Setup

This guide assumes you already have a functioning setup for sending OpenTelemetry metrics to Datadog.

OpenTelemetry SDK configuration

If you are producing metrics from an OpenTelemetry SDK, take the following steps to configure them:

Configure the SDK you are sending OTLP Histograms with delta temporality. This makes the minimum and maximum available for the heatmap widget.
Check if you want to override the default bucket boundaries from your aggregation. Note: Each additional bucket is considered a separate custom metric.

For metrics coming from other sources, ensure if possible that these come as delta OTLP Histograms with the minimum and maximum fields set.

Datadog Exporter or Datadog Agent configuration

Set the histogram mode and enable aggregation metrics on your Datadog Exporter or Datadog Agent.

In the collector.yaml file for the Datadog Exporter, configure the histogram mode to counters and enable aggregation metrics with the send_aggregation_metrics flag.

exporters:
  datadog:
    metrics:
      histograms:
        mode: counters
        send_aggregation_metrics: true

Note: send_aggregation_metrics is available starting with Datadog Exporter v0.75.0. If you are using an earlier version, use the send_count_sum_metrics flag instead. The minimum and maximum are missing in earlier versions.

On the otlp_config section, configure the histogram mode to counters and enable aggregation metrics with the send_aggregation_metrics flag.

otlp_config:
  metrics:
    histograms:
      mode: counters
      send_aggregation_metrics: true

Note: send_aggregation_metrics is available starting on the Datadog Agent v6.45.0/v7.45.0. If you are using an earlier version, use the send_count_sum_metrics flag instead. The minimum and maximum are missing in earlier versions.

The heatmap widget uses the set of <YOUR METRIC NAME>.bucket metrics generated by the Datadog Exporter or Datadog Agent, each corresponding to a different histogram bucket. To visualize your histogram as a heatmap:

Select <YOUR METRIC NAME>.bucket as the metric to visualize.
Choose the pre-binned data option on the distributions of menu.

You can now see your OTLP Histogram as a heatmap widget.

OpenMetrics compatibility

The Datadog Agent OpenMetrics check is also compatible with the pre-binned data heatmap widget option. If you want to send metrics to the OpenMetrics check directly without converting to OpenTelemetry, enable the collect_histogram_buckets and non_cumulative_histogram_buckets flags on your instance to ensure the data is sent in a compatible way to Datadog.