Graphs are the window to your monitored systems. You see graphs throughout Datadog, in email notifications, Slack, HipChat, and other chat clients. Graphs are at the center of monitoring and observability, so it is essential to understand how to define graphs.
There are two ways to interact with the Graphing Editor: using the GUI (the default method) and writing JSON (the more complete method). This page covers using the GUI. To learn more about using JSON, see Graphing Primer using JSON.
On each graph, a pencil icon opens the graph editor:
The graphing editor has the following tabs:
When you first open the graph editor, you are on the Edit tab. Here you can use the UI to choose most settings. Here is an example:
Configuring a graph is a multi-step process:
When you create a graph, you probably have a metric in mind that you want to show. You can select that in the first dropdown under step #2, Graph your data. If you don’t know which metric to use, you might want to start with the Metrics Explorer or a Notebook. You can also see a list of metrics in the Metrics Summary.
The Metrics Explorer allows you to play around with different graph settings in a more ad-hoc way. The Metrics Summary shows the type and default unit for a metric.
Once you have a metric in mind to display in your graph, select your visualization. Check the list of all visualizations (widgets).
Graphs can be displayed as Areas, Bars, or Lines. For all graph types, Datadog offers various color options to differentiate multiple metrics displayed on the same graph:
| Palette | Description |
|---|---|
| Classic | The simple colors light blue, dark blue, light purple, purple, light yellow, and yellow (colors repeat). |
| Cool | A gradient color scheme made from green and blue. |
| Warm | A gradient color scheme made from yellow and orange. |
| Purple | A gradient color scheme made from purple. |
| Orange | A gradient color scheme made from orange. |
| Gray | A gradient color scheme made from gray. |
For line graphs, different metrics can be assigned specific palettes by separating the queries in JSON.
Line graphs include two additional parameters:
| Parameter | Options |
|---|---|
| Style | Solid, Dashed, Dotted |
| Stroke | Normal, Thin, Thick |
Once the metric and a visualization are in place, you can filter the hosts to be graphed. To the right of the metric is the from dropdown which defaults to (everywhere). Click this and choose the tag(s) you want to filter by. To learn more about tags, refer to the Tagging documentation.
Next to the filter dropdown is the aggregation method. This defaults to avg by but can be changed to max by, min by, or sum by. In most cases, the metric has many values for each time interval, coming from many hosts or instances. The aggregation method chosen determines how the metrics are aggregated into a single line. So if you are graphing a metric that is from 100 hosts, sum by adds up all of those values and displays the sum.
After the aggregation method, determine what constitutes a line or grouping in a graph. If you choose host, then you have a line (on line graphs) for every host. If you choose role, then there is a line for every role. That line is made up of metrics from all the hosts in that role, aggregated using the method you chose above.
Regardless of the options chosen above, there is always some aggregation of data due to the physical size constraints of the window holding the graph. If a metric is updated every second and you are looking at 4 hours of data, you need 14,400 points to display everything. Each graph displayed has about 300 points shown at any given time.
In the example above, each point displayed on the screen represents 48 data points. In practice, metrics are collected by the Agent every 15-20 seconds. So one day’s worth of data is 4,320 data points. You might consider a rollup function that looks at 5 or 10 minutes worth of data for more control over the graph.
To use the rollup function, click the plus sign to the right of the aggregation group and choose rollup from the dropdown. Now choose how you want to aggregate the data and the interval in seconds.
To create a single line that represents the total available disk space on average across all machines rolled up in 60-second buckets, you would use a query like this:
When switching to the JSON view, the query looks like this:
"q": "avg:system.disk.free{*}.rollup(avg, 60)"
For more about using the JSON view, see Graphing Primer using JSON.
Depending on your analysis needs, you may choose to apply other mathematical functions to the query. Examples include rates and derivatives, smoothing, and more. See the list of available functions.
The Datadog UI also supports the ability to graph your metrics with various arithmetic operations. Use: +, -, /, * to modify the values displayed on your graphs. This syntax allows for both integer values and arithmetic using multiple metrics.
Modify how a metric value is displayed on a graph by performing an arithmetic operation on the metric. For example, to visualize the double of a specific metric, click the Advanced… link in the graph editor. Then enter your arithmetic in the Formula box, in this case: a * 2.
To visualize the percentage of a metric by dividing one metric over another. For example:
jvm.heap_memory / jvm.heap_memory_max
This can be done in the same manner as above, utilizing the Advanced… option in the Graph Editor. From there, select Add Query. Each query is assigned a letter: the first metric is represented by a, the second metric is represented b, and so on.
Then in the Formula box, enter the arithmetic for this example a / b:
To display only your formula, un-check your metrics a and b:
Note: Formulas are not lettered. Arithmetic cannot be done between formulas.
Each query or formula can be aliased. The alias overrides the display on the graph and legend, which is useful for long metric names. At the end of the query/formula click on as…, then enter your metric alias:
The Datadog y-axis controls are available via the UI and the JSON editor. They allow you to:
Change the Y-axis scale by expanding the Y-Axis Controls:
The following configuration options are available:
| Option | Required | Description |
|---|---|---|
Min/Max |
No | Specify the minimum and / or maximum value to show on y-axis. It takes a number or Auto as the default value. |
Scale |
No | Specifies the scale type. Possible values: - linear: A linear scale (default) - log: A logarithmic scale - pow: A Power of 2 scale (2 is default, modify in JSON) - sqrt: A square root scale |
Always include zero |
No | Always include zero or fit the axis to the data range. The default is to always include zero. |
Note: Because the mathematical log function doesn’t accept negative values, the Datadog log scale only works if values are of the same sign (everything > 0 or everything < 0). Otherwise an empty graph is returned.
Add events from related systems to add more context to your graph. For example, you can add GitHub commits, Jenkins deploys, or Docker creation events. Expand the Event Overlays section and enter a query to display those events. Use the same query format as for the Event Stream, for example:
| Query | Description |
|---|---|
sources:jenkins |
Shows all events from the Jenkins source. |
tag:role:web |
Shows all events with the tag role:web. |
tags:$<TEMPLATE_VARIABLE> |
Shows all events from the selected Template Variable. |
If you don’t enter a title, one is automatically generated based on your selections. But it may be more useful to the users of the dashboard to create a title that more aptly describes the purpose of the graph. Linking the technical purpose to the business benefits adds even more value.
Click Done to save your work and exit the editor. You can always come back to the editor to change the graph. If you make changes you don’t want to save, click Cancel.
Additional helpful documentation, links, and articles:
