Change Widget

The Change graph shows you the change in a metric over a period of time. It compares the absolute or relative (%) change in value between N minutes ago and now against a given threshold. The compared datapoints aren’t single points but are computed using the parameters in the define the metric section. For more information, see the Metric Monitor documentation, and the Change Alert Monitors guide.

Example of a change widget for jvm.heap_memory metric

Setup

Configuration

  1. Choose a metric to graph.
  2. Choose an aggregation function.
  3. Optional: choose a specific context for your widget.
  4. Break down your aggregation by a tag key such as host or service.
  5. Choose a value for the “Compare to” period:
    • an hour before
    • a day before
    • a week before
    • a month before
  6. Choose between relative or absolute change.
  7. Select the field by which the metrics are ordered:
    • change
    • name
    • present value
    • past value
  8. Choose ascending or descending ordering.
  9. Choose whether to display the current value in the graph.

Options

Context links are enabled by default, and can be toggled on or off. Context links bridge dashboard widgets with other pages (in Datadog, or third-party).

API

This widget can be used with the Dashboards API. See the following table for the widget JSON schema definition:

Expand All

Field

Type

Description

custom_links

[object]

List of custom links.

is_hidden

boolean

The flag for toggling context menu link visibility.

label

string

The label for the custom link URL. Keep the label short and descriptive. Use metrics and tags as variables.

link

string

The URL of the custom link. URL must include http or https. A relative URL must start with /.

override_label

string

The label ID that refers to a context menu link. Can be logs, hosts, traces, profiles, processes, containers, or rum.

requests [required]

[object]

Array of one request object to display in the widget.

See the dedicated Request JSON schema documentation to learn how to build the REQUEST_SCHEMA.

apm_query

object

The log query.

compute

object

Define computation for a log query.

aggregation [required]

string

The aggregation method.

facet

string

Facet name.

interval

int64

Define a time interval in seconds.

group_by

[object]

List of tag prefixes to group by in the case of a cluster check.

facet [required]

string

Facet name.

limit

int64

Maximum number of items in the group.

sort

object

Define a sorting method.

aggregation [required]

string

The aggregation method.

facet

string

Facet name.

order [required]

enum

Widget sorting methods. Allowed enum values: asc,desc

index

string

A coma separated-list of index names. Use "*" query all indexes at once. Multiple Indexes

multi_compute

[object]

This field is mutually exclusive with compute.

aggregation [required]

string

The aggregation method.

facet

string

Facet name.

interval

int64

Define a time interval in seconds.

search

object

The query being made on the logs.

query [required]

string

Search value to apply.

change_type

enum

Show the absolute or the relative change. Allowed enum values: absolute,relative

compare_to

enum

Timeframe used for the change comparison. Allowed enum values: hour_before,day_before,week_before,month_before

event_query

object

The log query.

compute

object

Define computation for a log query.

aggregation [required]

string

The aggregation method.

facet

string

Facet name.

interval

int64

Define a time interval in seconds.

group_by

[object]

List of tag prefixes to group by in the case of a cluster check.

facet [required]

string

Facet name.

limit

int64

Maximum number of items in the group.

sort

object

Define a sorting method.

aggregation [required]

string

The aggregation method.

facet

string

Facet name.

order [required]

enum

Widget sorting methods. Allowed enum values: asc,desc

index

string

A coma separated-list of index names. Use "*" query all indexes at once. Multiple Indexes

multi_compute

[object]

This field is mutually exclusive with compute.

aggregation [required]

string

The aggregation method.

facet

string

Facet name.

interval

int64

Define a time interval in seconds.

search

object

The query being made on the logs.

query [required]

string

Search value to apply.

formulas

[object]

List of formulas that operate on queries.

alias

string

Expression alias.

cell_display_mode

enum

Define a display mode for the table cell. Allowed enum values: number,bar

conditional_formats

[object]

List of conditional formats.

comparator [required]

enum

Comparator to apply. Allowed enum values: =,>,>=,<,<=

custom_bg_color

string

Color palette to apply to the background, same values available as palette.

custom_fg_color

string

Color palette to apply to the foreground, same values available as palette.

hide_value

boolean

True hides values.

image_url

string

Displays an image as the background.

metric

string

Metric from the request to correlate this conditional format with.

palette [required]

enum

Color palette to apply. Allowed enum values: blue,custom_bg,custom_image,custom_text,gray_on_white,grey,green,orange,red,red_on_white,white_on_gray,white_on_green,green_on_white,white_on_red,white_on_yellow,yellow_on_white,black_on_light_yellow,black_on_light_green,black_on_light_red

timeframe

string

Defines the displayed timeframe.

value [required]

double

Value for the comparator.

formula [required]

string

String expression built from queries, formulas, and functions.

limit

object

Options for limiting results returned.

count

int64

Number of results to return.

order

enum

Direction of sort. Allowed enum values: asc,desc

default: desc

style

object

Styling options for widget formulas.

palette

string

The color palette used to display the formula. A guide to the available color palettes can be found at https://docs.datadoghq.com/dashboards/guide/widget_colors

palette_index

int64

Index specifying which color to use within the palette.

increase_good

boolean

Whether to show increase as good.

log_query

object

The log query.

compute

object

Define computation for a log query.

aggregation [required]

string

The aggregation method.

facet

string

Facet name.

interval

int64

Define a time interval in seconds.

group_by

[object]

List of tag prefixes to group by in the case of a cluster check.

facet [required]

string

Facet name.

limit

int64

Maximum number of items in the group.

sort

object

Define a sorting method.

aggregation [required]

string

The aggregation method.

facet

string

Facet name.

order [required]

enum

Widget sorting methods. Allowed enum values: asc,desc

index

string

A coma separated-list of index names. Use "*" query all indexes at once. Multiple Indexes

multi_compute

[object]

This field is mutually exclusive with compute.

aggregation [required]

string

The aggregation method.

facet

string

Facet name.

interval

int64

Define a time interval in seconds.

search

object

The query being made on the logs.

query [required]

string

Search value to apply.

network_query

object

The log query.

compute

object

Define computation for a log query.

aggregation [required]

string

The aggregation method.

facet

string

Facet name.

interval

int64

Define a time interval in seconds.

group_by

[object]

List of tag prefixes to group by in the case of a cluster check.

facet [required]

string

Facet name.

limit

int64

Maximum number of items in the group.

sort

object

Define a sorting method.

aggregation [required]

string

The aggregation method.

facet

string

Facet name.

order [required]

enum

Widget sorting methods. Allowed enum values: asc,desc

index

string

A coma separated-list of index names. Use "*" query all indexes at once. Multiple Indexes

multi_compute

[object]

This field is mutually exclusive with compute.

aggregation [required]

string

The aggregation method.

facet

string

Facet name.

interval

int64

Define a time interval in seconds.

search

object

The query being made on the logs.

query [required]

string

Search value to apply.

order_by

enum

What to order by. Allowed enum values: change,name,present,past

order_dir

enum

Widget sorting methods. Allowed enum values: asc,desc

process_query

object

The process query to use in the widget.

filter_by

[string]

List of processes.

limit

int64

Max number of items in the filter list.

metric [required]

string

Your chosen metric.

search_by

string

Your chosen search term.

profile_metrics_query

object

The log query.

compute

object

Define computation for a log query.

aggregation [required]

string

The aggregation method.

facet

string

Facet name.

interval

int64

Define a time interval in seconds.

group_by

[object]

List of tag prefixes to group by in the case of a cluster check.

facet [required]

string

Facet name.

limit

int64

Maximum number of items in the group.

sort

object

Define a sorting method.

aggregation [required]

string

The aggregation method.

facet

string

Facet name.

order [required]

enum

Widget sorting methods. Allowed enum values: asc,desc

index

string

A coma separated-list of index names. Use "*" query all indexes at once. Multiple Indexes

multi_compute

[object]

This field is mutually exclusive with compute.

aggregation [required]

string

The aggregation method.

facet

string

Facet name.

interval

int64

Define a time interval in seconds.

search

object

The query being made on the logs.

query [required]

string

Search value to apply.

q

string

Query definition.

queries

[ <oneOf>]

List of queries that can be returned directly or used in formulas.

Option 1

object

A formula and functions metrics query.

aggregator

enum

The aggregation methods available for metrics queries. Allowed enum values: avg,min,max,sum,last,area,l2norm,percentile

data_source [required]

enum

Data source for metrics queries. Allowed enum values: metrics

name [required]

string

Name of the query for use in formulas.

query [required]

string

Metrics query definition.

Option 2

object

A formula and functions events query.

compute [required]

object

Compute options.

aggregation [required]

enum

Aggregation methods for event platform queries. Allowed enum values: count,cardinality,median,pc75,pc90,pc95,pc98,pc99,sum,min,max,avg

interval

int64

A time interval in milliseconds.

metric

string

Measurable attribute to compute.

data_source [required]

enum

Data source for event platform-based queries. Allowed enum values: logs,spans,network,rum,security_signals,profiles,audit,events,ci_tests,ci_pipelines

group_by

[object]

Group by options.

facet [required]

string

Event facet.

limit

int64

Number of groups to return.

sort

object

Options for sorting group by results.

aggregation [required]

enum

Aggregation methods for event platform queries. Allowed enum values: count,cardinality,median,pc75,pc90,pc95,pc98,pc99,sum,min,max,avg

metric

string

Metric used for sorting group by results.

order

enum

Direction of sort. Allowed enum values: asc,desc

default: desc

indexes

[string]

An array of index names to query in the stream. Omit or use [] to query all indexes at once.

name [required]

string

Name of the query for use in formulas.

search

object

Search options.

query [required]

string

Events search string.

storage

string

Option for storage location. Feature in Private Beta.

Option 3

object

Process query using formulas and functions.

aggregator

enum

The aggregation methods available for metrics queries. Allowed enum values: avg,min,max,sum,last,area,l2norm,percentile

data_source [required]

enum

Data sources that rely on the process backend. Allowed enum values: process,container

is_normalized_cpu

boolean

Whether to normalize the CPU percentages.

limit

int64

Number of hits to return.

metric [required]

string

Process metric name.

name [required]

string

Name of query for use in formulas.

sort

enum

Direction of sort. Allowed enum values: asc,desc

default: desc

tag_filters

[string]

An array of tags to filter by.

text_filter

string

Text to use as filter.

Option 4

object

A formula and functions APM dependency stats query.

data_source [required]

enum

Data source for APM dependency stats queries. Allowed enum values: apm_dependency_stats

env [required]

string

APM environment.

is_upstream

boolean

Determines whether stats for upstream or downstream dependencies should be queried.

name [required]

string

Name of query to use in formulas.

operation_name [required]

string

Name of operation on service.

primary_tag_name

string

The name of the second primary tag used within APM; required when primary_tag_value is specified. See https://docs.datadoghq.com/tracing/guide/setting_primary_tags_to_scope/#add-a-second-primary-tag-in-datadog.

primary_tag_value

string

Filter APM data by the second primary tag. primary_tag_name must also be specified.

resource_name [required]

string

APM resource.

service [required]

string

APM service.

stat [required]

enum

APM statistic. Allowed enum values: avg_duration,avg_root_duration,avg_spans_per_trace,error_rate,pct_exec_time,pct_of_traces,total_traces_count

Option 5

object

APM resource stats query using formulas and functions.

data_source [required]

enum

Data source for APM resource stats queries. Allowed enum values: apm_resource_stats

env [required]

string

APM environment.

group_by

[string]

Array of fields to group results by.

name [required]

string

Name of this query to use in formulas.

operation_name

string

Name of operation on service.

primary_tag_name

string

Name of the second primary tag used within APM. Required when primary_tag_value is specified. See https://docs.datadoghq.com/tracing/guide/setting_primary_tags_to_scope/#add-a-second-primary-tag-in-datadog

primary_tag_value

string

Value of the second primary tag by which to filter APM data. primary_tag_name must also be specified.

resource_name

string

APM resource name.

service [required]

string

APM service name.

stat [required]

enum

APM resource stat name. Allowed enum values: errors,error_rate,hits,latency_avg,latency_distribution,latency_max,latency_p50,latency_p75,latency_p90,latency_p95,latency_p99

Option 6

object

A formula and functions metrics query.

additional_query_filters

string

Additional filters applied to the SLO query.

data_source [required]

enum

Data source for SLO measures queries. Allowed enum values: slo

group_mode

enum

Group mode to query measures. Allowed enum values: overall,components

measure [required]

enum

SLO measures queries. Allowed enum values: good_events,bad_events,slo_status,error_budget_remaining,burn_rate,error_budget_burndown

name

string

Name of the query for use in formulas.

slo_id [required]

string

ID of an SLO to query measures.

slo_query_type

enum

Name of the query for use in formulas. Allowed enum values: metric

Option 7

object

A formula and functions Cloud Cost query.

aggregator

enum

Aggregator used for the request. Allowed enum values: avg,last,max,min,sum,percentile

data_source [required]

enum

Data source for Cloud Cost queries. Allowed enum values: cloud_cost

name [required]

string

Name of the query for use in formulas.

query [required]

string

Query for Cloud Cost data.

response_format

enum

Timeseries, scalar, or event list response. Event list response formats are supported by Geomap widgets. Allowed enum values: timeseries,scalar,event_list

rum_query

object

The log query.

compute

object

Define computation for a log query.

aggregation [required]

string

The aggregation method.

facet

string

Facet name.

interval

int64

Define a time interval in seconds.

group_by

[object]

List of tag prefixes to group by in the case of a cluster check.

facet [required]

string

Facet name.

limit

int64

Maximum number of items in the group.

sort

object

Define a sorting method.

aggregation [required]

string

The aggregation method.

facet

string

Facet name.

order [required]

enum

Widget sorting methods. Allowed enum values: asc,desc

index

string

A coma separated-list of index names. Use "*" query all indexes at once. Multiple Indexes

multi_compute

[object]

This field is mutually exclusive with compute.

aggregation [required]

string

The aggregation method.

facet

string

Facet name.

interval

int64

Define a time interval in seconds.

search

object

The query being made on the logs.

query [required]

string

Search value to apply.

security_query

object

The log query.

compute

object

Define computation for a log query.

aggregation [required]

string

The aggregation method.

facet

string

Facet name.

interval

int64

Define a time interval in seconds.

group_by

[object]

List of tag prefixes to group by in the case of a cluster check.

facet [required]

string

Facet name.

limit

int64

Maximum number of items in the group.

sort

object

Define a sorting method.

aggregation [required]

string

The aggregation method.

facet

string

Facet name.

order [required]

enum

Widget sorting methods. Allowed enum values: asc,desc

index

string

A coma separated-list of index names. Use "*" query all indexes at once. Multiple Indexes

multi_compute

[object]

This field is mutually exclusive with compute.

aggregation [required]

string

The aggregation method.

facet

string

Facet name.

interval

int64

Define a time interval in seconds.

search

object

The query being made on the logs.

query [required]

string

Search value to apply.

show_present

boolean

Whether to show the present value.

time

object

Time setting for the widget.

live_span

enum

The available timeframes depend on the widget you are using. Allowed enum values: 1m,5m,10m,15m,30m,1h,4h,1d,2d,1w,1mo,3mo,6mo,week_to_date,month_to_date,1y,alert

title

string

Title of the widget.

title_align

enum

How to align the text on the widget. Allowed enum values: center,left,right

title_size

string

Size of the title.

type [required]

enum

Type of the change widget. Allowed enum values: change

default: change

{
  "custom_links": [
    {
      "is_hidden": false,
      "label": "Search logs for {{host}}",
      "link": "https://app.datadoghq.com/logs?query={{host}}",
      "override_label": "logs"
    }
  ],
  "requests": [
    {
      "apm_query": {
        "compute": {
          "aggregation": "avg",
          "facet": "@duration",
          "interval": 5000
        },
        "group_by": [
          {
            "facet": "resource_name",
            "limit": 50,
            "sort": {
              "aggregation": "avg",
              "facet": "@string_query.interval",
              "order": "desc"
            }
          }
        ],
        "index": "days-3,days-7",
        "multi_compute": [
          {
            "aggregation": "avg",
            "facet": "@duration",
            "interval": 5000
          }
        ],
        "search": {
          "query": ""
        }
      },
      "change_type": "string",
      "compare_to": "string",
      "event_query": {
        "compute": {
          "aggregation": "avg",
          "facet": "@duration",
          "interval": 5000
        },
        "group_by": [
          {
            "facet": "resource_name",
            "limit": 50,
            "sort": {
              "aggregation": "avg",
              "facet": "@string_query.interval",
              "order": "desc"
            }
          }
        ],
        "index": "days-3,days-7",
        "multi_compute": [
          {
            "aggregation": "avg",
            "facet": "@duration",
            "interval": 5000
          }
        ],
        "search": {
          "query": ""
        }
      },
      "formulas": [
        {
          "alias": "string",
          "cell_display_mode": "number",
          "conditional_formats": [
            {
              "comparator": ">",
              "custom_bg_color": "string",
              "custom_fg_color": "string",
              "hide_value": false,
              "image_url": "string",
              "metric": "string",
              "palette": "blue",
              "timeframe": "string",
              "value": 0
            }
          ],
          "formula": "func(a) + b",
          "limit": {
            "count": "integer",
            "order": "string"
          },
          "style": {
            "palette": "classic",
            "palette_index": 1
          }
        }
      ],
      "increase_good": false,
      "log_query": {
        "compute": {
          "aggregation": "avg",
          "facet": "@duration",
          "interval": 5000
        },
        "group_by": [
          {
            "facet": "resource_name",
            "limit": 50,
            "sort": {
              "aggregation": "avg",
              "facet": "@string_query.interval",
              "order": "desc"
            }
          }
        ],
        "index": "days-3,days-7",
        "multi_compute": [
          {
            "aggregation": "avg",
            "facet": "@duration",
            "interval": 5000
          }
        ],
        "search": {
          "query": ""
        }
      },
      "network_query": {
        "compute": {
          "aggregation": "avg",
          "facet": "@duration",
          "interval": 5000
        },
        "group_by": [
          {
            "facet": "resource_name",
            "limit": 50,
            "sort": {
              "aggregation": "avg",
              "facet": "@string_query.interval",
              "order": "desc"
            }
          }
        ],
        "index": "days-3,days-7",
        "multi_compute": [
          {
            "aggregation": "avg",
            "facet": "@duration",
            "interval": 5000
          }
        ],
        "search": {
          "query": ""
        }
      },
      "order_by": "string",
      "order_dir": "desc",
      "process_query": {
        "filter_by": [],
        "limit": "integer",
        "metric": "system.load.1",
        "search_by": "string"
      },
      "profile_metrics_query": {
        "compute": {
          "aggregation": "avg",
          "facet": "@duration",
          "interval": 5000
        },
        "group_by": [
          {
            "facet": "resource_name",
            "limit": 50,
            "sort": {
              "aggregation": "avg",
              "facet": "@string_query.interval",
              "order": "desc"
            }
          }
        ],
        "index": "days-3,days-7",
        "multi_compute": [
          {
            "aggregation": "avg",
            "facet": "@duration",
            "interval": 5000
          }
        ],
        "search": {
          "query": ""
        }
      },
      "q": "<METRIC_1>{<SCOPE_1>}",
      "queries": [],
      "response_format": "timeseries",
      "rum_query": {
        "compute": {
          "aggregation": "avg",
          "facet": "@duration",
          "interval": 5000
        },
        "group_by": [
          {
            "facet": "resource_name",
            "limit": 50,
            "sort": {
              "aggregation": "avg",
              "facet": "@string_query.interval",
              "order": "desc"
            }
          }
        ],
        "index": "days-3,days-7",
        "multi_compute": [
          {
            "aggregation": "avg",
            "facet": "@duration",
            "interval": 5000
          }
        ],
        "search": {
          "query": ""
        }
      },
      "security_query": {
        "compute": {
          "aggregation": "avg",
          "facet": "@duration",
          "interval": 5000
        },
        "group_by": [
          {
            "facet": "resource_name",
            "limit": 50,
            "sort": {
              "aggregation": "avg",
              "facet": "@string_query.interval",
              "order": "desc"
            }
          }
        ],
        "index": "days-3,days-7",
        "multi_compute": [
          {
            "aggregation": "avg",
            "facet": "@duration",
            "interval": 5000
          }
        ],
        "search": {
          "query": ""
        }
      },
      "show_present": false
    }
  ],
  "time": {
    "live_span": "5m"
  },
  "title": "string",
  "title_align": "string",
  "title_size": "string",
  "type": "change"
}

Further Reading

Additional helpful documentation, links, and articles:

Configure Change alert detection in monitorsDOCUMENTATION more more Building Dashboards using JSONDOCUMENTATION more more Widget JSON schemaDOCUMENTATION more more Request JSON schemaDOCUMENTATION more more