Metrics Types
Datadog's Research Report: The State of Serverless Report: The State of Serverless

Metrics Types

Overview

Each metric submitted to Datadog should have a type. A metric’s type affects how the metric values are displayed when queried, as well as the associated graphing possibilities within Datadog using additional modifiers and functions. A metric’s type is displayed on the details side panel for the given metric on the Metrics Summary page.

Note: Changing the metric type in this details side panel can change metric behavior in all existing visualizations and monitors, potentially rendering historical data as nonsensical.

The following metric submission types are accepted:

These different metric submission types are mapped to four in-app metric types found within the Datadog web application:

  • COUNT
  • RATE
  • GAUGE
  • DISTRIBUTION

Note: If you submit a metric to Datadog without a type, the metric type appears as Other within Datadog. The Other metric type cannot be further changed to another in-app type until an initial metric type is submitted.

Submission vs. In-App type

Metrics are submitted to Datadog in three main ways:

The majority of data that Datadog receives is submitted by the Agent, either through an Agent check or DogStatsD. For these submission methods, a metric’s type determines how multiple values collected on an Agent in a flush time interval are aggregated. The Agent combines these values into a single representative metric value for that interval. This combined value is stored with a single timestamp in Datadog.

Data submitted directly to the Datadog API is not aggregated by Datadog, with the exception of distribution metrics. The raw values sent to Datadog are stored as-is.

Refer to the Submission and In-App Types section to see how different metric submission types are mapped to their corresponding in-app types.

Metric types

Definition

The COUNT metric submission type represents the total number of event occurrences in one time interval. A COUNT can be used to track the total number of connections made to a database or the total number of requests to an endpoint. This number of events can accumulate or decrease over time—it is not monotonically increasing.

Note: A COUNT is different from the RATE metric type, which represents the number of event occurrences normalized per second given the defined time interval.

The RATE metric submission type represents the total number of event occurrences per second in one time interval. A RATE can be used to track how often something is happening—like the frequency of connections made to a database or the flow of requests made to an endpoint.

Note: A RATE is different from the COUNT metric submission type, which represents the total number of event occurrences in the given time interval.

The GAUGE metric submission type represents a snapshot of events in one time interval. This representative snapshot value is the last value submitted to the Agent during a time interval. A GAUGE can be used to take a measure of something reporting continuously—like the available disk space or memory used.

The HISTOGRAM metric submission type represents the statistical distribution of a set of values calculated Agent-side in one time interval. Datadog’s HISTOGRAM metric type is an extension of the StatsD timing metric type: the Agent aggregates the values that are sent in a defined time interval and produces different metrics which represent the set of values.

If you send X values for a HISTOGRAM metric <METRIC_NAME> in a given time interval, the following metrics are produced by the Agent by default:

New Metric NameDescriptionDatadog In-App Type
<METRIC_NAME>.avgRepresents the average of those X values in the time interval.GAUGE
<METRIC_NAME>.countRepresents the number of values submitted during the interval, X. The Agent submits this number as a RATE so it would show in app the value of X/interval.RATE
<METRIC_NAME>.medianRepresents the median of those X values in the time interval.GAUGE
<METRIC_NAME>.95percentileRepresents the 95th percentile of those X values in the time interval.GAUGE
<METRIC_NAME>.maxRepresents the maximum value of those X values sent during the time intervalGAUGE

Note:

  • Configure which aggregations you want to send to Datadog with the histogram_aggregates parameter in your datadog.yaml configuration file. By default, only max, median, avg, and count aggregations are sent to Datadog. sum and min are also available.
  • Configure which percentile aggregation you want to send to Datadog with the histogram_percentiles parameter in your datadog.yaml configuration file. By default, only the 95percentile is sent to Datadog.

The DISTRIBUTION metric submission type represents the global statistical distribution of a set of values calculated across your entire distributed infrastructure in one time interval. A DISTRIBUTION can be used to instrument logical objects, like services, independently from the underlying hosts.

Unlike the HISTOGRAM metric type, which aggregates on the Agent during a given time interval, a DISTRIBUTION metric sends all the raw data during a time interval to Datadog, and aggregations occur server-side. Because the underlying data structure represents raw, unaggregated data, distributions provide two major features:

  • Calculation of percentile aggregations
  • Customization of tagging

If you send X values for a DISTRIBUTION metric <METRIC_NAME> in a given time interval, the following aggregations are available for query by default:

AggregationDescriptionDatadog In-App Type
avg:<METRIC_NAME>Represents the average of those X values in the time interval.GAUGE
count:<METRIC_NAME>Represents the number of points submitted in the time interval, X. The Agent then sends it as a RATE so it would show in app the value X/interval.RATE
max:<METRIC_NAME>Represents the maximum value of those X values sent in the time interval.GAUGE
min:<METRIC_NAME>Represents the minimum value of those X sent in the time interval.GAUGE
sum:<METRIC_NAME>Represents the sum of all X values sent in the time intervalGAUGE

Example

Suppose you are submitting a COUNT metric, activeusers.basket_size, from a single host running the Datadog Agent. This host emits the following values in a flush time interval: [1,1,1,2,2,2,3,3].

The Agent adds all the values received in one time interval and submits the total number, in this case 15, as the COUNT metric’s value.

Suppose you are submitting a RATE metric, queue_messages.rate, from a single host running the Datadog Agent. This host emits the following values in a flush time interval: [1,1,1,2,2,2,3,3].

The Agent adds all the values received in one time interval and submits the total number divided by the total number of seconds in this time interval.In this case, if the flush interval is 10 seconds, the value submitted would be 1.5 as the RATE metric’s value.

Suppose you are submitting a GAUGE metric, temperature, from a single host running the Datadog Agent. This host emits the following values in a flush time interval: [71,71,71,71,71,71,71.5].

The Agent submits the last reported number, in this case 71.5, as the GAUGE metric’s value.

For example, suppose you are submitting a HISTOGRAM metric, request.response_time.histogram, from a web server that reports the values [1,1,1,2,2,2,3,3] in a flush time interval. By default, the Agent submits the following metrics to Datadog which represent the statistical distribution of these values in this time interval:

Metric NameValueDatadog In-App Type
request.response_time.histogram.avg1.88GAUGE
request.response_time.histogram.count8RATE
request.response_time.histogram.median2GAUGE
request.response_time.histogram.95percentile3GAUGE
request.response_time.histogram.max3GAUGE

Suppose you are submitting a DISTRIBUTION metric, request.response_time.distribution, from two webservers: webserver:web_1 and webserver:web_2. Suppose in a given flush time interval, webserver:web_1 reports the metric with the values [1,1,1,2,2,2,3,3], and webserver:web_2 reports the same metric with the values [1,1,2]. Over this time interval, the following five aggregations will represent the global statistical distribution of all values collected from both webservers:

Metric NameValueDatadog In-App Type
avg:request.response_time.distribution1.73GAUGE
count:request.response_time.distribution11RATE
max:request.response_time.distribution3GAUGE
min:request.response_time.distribution1GAUGE
sum:request.response_time.distribution19GAUGE

Calculation of percentile aggregations

Like other metric types, such as GAUGE or HISTOGRAM, the DISTRIBUTION metric type has the following aggregations available: count, min, max, sum, and avg. Distribution metrics are initially tagged the same way as other metrics (with custom tags set in the code).

Additional percentile aggregations (p50, p75, p90, p95, p99) can be added to distribution metrics. If you were to add percentile aggregations to your distribution metric in-app, the following five additional aggregations are available for query:

Metric NameValueDatadog In-app Type
p50:request.response_time.distribution2GAUGE
p75:request.response_time.distribution2GAUGE
p90:request.response_time.distribution3GAUGE
p95:request.response_time.distribution3GAUGE
p99:request.response_time.distribution3GAUGE

That is, for a distribution metric with added percentile aggregations during a given time interval, the following 10 aggregations are available: count, sum, min, max, avg, p50, p75, p90, p95, and p99.

Customization of tagging

This functionality allows you to control tagging for metrics where host-level granularity is not necessary. See the Distribution Metric page to learn more about whitelist-based tagging control.

Note: The exclusion of tags with ! is not accepted with this feature.

Submission

Submit your COUNT type metrics from one of the following sources:

Submission SourceSubmission Method (python)Submission TypeDatadog In-App Type
Agent checkself.count(...)COUNTCOUNT
Agent checkself.monotonic_count(...)COUNTCOUNT
APIapi.Metric.send(type="count", ...)COUNTCOUNT
DogStatsDdog.count(...)COUNTRATE
DogStatsDdog.increment(...)COUNTRATE
DogStatsDdog.decrement(...)COUNTRATE

Note: When submitting a COUNT metric type through DogStatsD, the metric appears as a RATE in-app to ensure relevant comparison across different Agents. Consequently, StatsD counts may appear with a decimal value within Datadog (since they are normalized over a time interval to report units per second).

Submit your RATE type metrics from one of the following sources:

Submission SourceSubmission Method (python)Submission TypeDatadog In-App Type
Agent checkself.rate(...)RATEGAUGE
APIapi.Metric.send(type="rate", ...)RATERATE

Note: When submitting a RATE metric type through DogStatsD, the metric appears as a GAUGE in-app to ensure relevant comparison across different Agents.

Submit your GAUGE type metrics from one of the following sources:

Submission SourceSubmission Method (Python)Submission TypeDatadog In-App Type
Agent checkself.gauge(...)GAUGEGAUGE
APIapi.Metric.send(type="gauge", ...)GAUGEGAUGE
DogStatsDdog.gauge(...)GAUGEGAUGE

Submit your HISTOGRAM type metrics from one of the following sources:

Submission SourceSubmission Method (Python)Submission TypeDatadog In-App Types
Agent checkself.histogram(...)HISTOGRAMGAUGE, RATE
DogStatsDdog.histogram(...)HISTOGRAMGAUGE, RATE

Note: If you submit a TIMER metric to the Datadog Agent, it is equivalent to submitting a HISTOGRAM metric type within DogStatsD (not to be confused with timers in the standard StatsD). Timers represent duration data only; for example, the amount of time a section of code takes to execute or how long it takes to fully render a page.

Submit your DISTRIBUTION type metrics from the following source:

Submission SourceSubmission Method (Python)Submission TypeDatadog In-App Types
DogStatsDdog.distribution(...)DISTRIBUTIONGAUGE, RATE

Submission types and Datadog in-app types

Find below a summary of all available metric submission sources and methods with the mapping between the corresponding metric submission type and the in-app types:

Submission SourceSubmission Method (Python)Submission TypeDatadog In-App Types
Agent checkself.count(...)COUNTCOUNT
Agent checkself.monotonic_count(...)COUNTCOUNT
Agent checkself.gauge(...)GAUGEGAUGE
Agent checkself.histogram(...)HISTOGRAMGAUGE, RATE
Agent checkself.rate(...)RATEGAUGE
APIapi.Metric.send(type="count", ...)COUNTCOUNT
APIapi.Metric.send(type="gauge", ...)GAUGEGAUGE
APIapi.Metric.send(type="rate", ...)RATERATE
DogStatsDdog.gauge(...)GAUGEGAUGE
DogStatsDdog.distribution(...)DISTRIBUTIONGAUGE, RATE
DogStatsDdog.count(...)COUNTRATE
DogStatsDdog.increment(...)COUNTRATE
DogStatsDdog.decrement(...)COUNTRATE
DogStatsDdog.set(...)SETGAUGE
DogStatsDdog.histogram(...)HISTOGRAMGAUGE, RATE

Further reading