AWS Integration and CloudWatch FAQ
To find out if this integration is available in your organization, see your Datadog Integrations page or ask your organization administrator.
To initiate an exception request to enable this integration for your organization, email support@ddog-gov.com.
Can I collect AWS custom metrics through the integration?
Yes. Enable Collect Custom Metrics under the Metric Collection tab on the AWS integration page.
How do I collect metrics from a service for which Datadog doesn’t have an official integration?
AWS metrics coming from an AWS/<namespace> for which there is no official integration are also brought in under the custom namespace when the Collect custom metrics option is enabled. You can filter these metrics out and only keep the metrics you want by using the filter string under custom namespace with the Set an AWS tag filter API.
How does the Datadog AWS integration use CloudWatch?
Datadog uses the CloudWatch monitoring APIs to monitor your AWS resources. Our main use of these APIs is to gather raw metrics data through the GetMetricData endpoint.
Other APIs are used to enrich metrics data. Some examples include:
Gathering custom tags to add to metrics
Gathering information about the status or health of resources, such as automuting
Gathering log streams
How many API requests are made and how can I monitor my CloudWatch usage?
Datadog gathers the available metrics every 10 minutes for each AWS sub-integration you have installed. If you have a large number of AWS resources for a particular sub-integration (SQS, ELB, DynamoDB, AWS Custom metrics), this can impact your AWS CloudWatch bill.
You can monitor your CloudWatch API usage using the AWS Billing integration.
How can I reduce the delay of receiving my CloudWatch metrics to Datadog?
By default, Datadog collects AWS metrics every 10 minutes. See Cloud Metric Delay for more information. If you need to reduce the latency, contact Datadog support for assistance. To get CloudWatch metrics into Datadog faster with a 2-3 minute latency we recommend using the Amazon CloudWatch Metric Streams and Amazon Data Firehose.
Why am I only seeing the average values of my custom AWS/Cloudwatch metrics?
By default, Datadog only collects the average values of your custom AWS/Cloudwatch metrics. However, additional values are available by contacting Datadog support. These include (where available) the min, max, sum, and sample count.
Is there a discrepancy between my data in CloudWatch and Datadog?
Some important distinctions to be aware of:
- Datadog collects a single CloudWatch statistic for the equivalent CloudWatch metric in Datadog. Comparing the
Sum in CloudWatch to the Average in Datadog results in discrepancies. For some CloudWatch metrics, multiple statistics can be useful and Datadog creates different metric names for the same CloudWatch metric with different statistics. For example, aws.elb.latency and aws.elb.latency.maximum. - In AWS for counters, a graph set to
sum 1 minute shows the total number of occurrences in one minute leading up to that point (the rate per one minute). Datadog is displaying the raw data from AWS normalized to per second values, regardless of the timeframe selected in AWS. Therefore, you might see a lower value in Datadog. - Overall,
min, max, and avg have different meanings within AWS. AWS distinctly collects average latency, minimum latency, and maximum latency. When pulling metrics from AWS CloudWatch, Datadog only receives the average latency as a single timeseries per ELB. Within Datadog, when you select min, max, or avg, you are controlling how multiple timeseries are combined. For example, requesting system.cpu.idle without any filter returns one series for each host reporting that metric. Datadog combines these timeseries using space aggregation. Otherwise, if you requested system.cpu.idle from a single host, no aggregation is necessary and switching between avg and max yields the same result.
How do I adjust my data on Datadog to match the data displayed in CloudWatch?
AWS CloudWatch reports metrics at one-minute granularity normalized to per-minute data. Datadog reports metrics at one-minute granularity normalized to per-second data. To adjust the data in Datadog, multiply by 60. Also make sure the statistic of the metric is the same. For example, the metric IntegrationLatency fetches a number of different statistics: Average, Maximum, Minimum, as well as percentiles. In Datadog, these statistics are each represented as their own metrics:
aws.apigateway.integration_latency (average)
aws.apigateway.integration_latency.maximum
aws.apigateway.integration_latency.minimum
aws.apigateway.integration_latency.p50
Will a rollup() adjust my data?
Rollups don’t display similar results. For a rollup call of rollup(sum, 60), the server groups all data points in minute bins and returns the sum of each bin as a datapoint. However, the granularity of AWS metrics is one minute, so there is only one datapoint per bin leading to no change.
Why don’t I see metrics for a new AWS service I enabled?
If you recently enabled a new AWS service integration but are not seeing metrics in Datadog, verify the following:
- IAM permissions: Confirm that the IAM role or IAM user associated with the Datadog integration includes the permissions required by the service. See the individual AWS integration pages for service-specific permission requirements.
- Region: Confirm that the AWS region where your resources are deployed is enabled in the AWS integration page.
- CloudWatch availability: Open the CloudWatch console in AWS and confirm the expected metrics exist. Some services do not emit CloudWatch metrics until specific conditions are met (for example, an ELB with no attached instances does not emit metrics).
- Polling delay: API polling collects metrics approximately every 10 minutes. If you use CloudWatch Metric Streams, expect a 2-3 minute delay. Allow at least one polling cycle before investigating further.
What is the difference between API polling and CloudWatch Metric Streams?
| API polling (default) | CloudWatch Metric Streams |
|---|
| Typical latency | ~10 minutes | 2-3 minutes |
| Setup | Included with the AWS integration | Requires separate setup with Amazon Data Firehose |
| AWS cost | CloudWatch GetMetricData API calls | CloudWatch Metric Streams and Firehose delivery charges |
| Coverage | All standard CloudWatch namespaces; custom namespaces require Collect Custom Metrics to be enabled | Most CloudWatch namespaces (some exclusions apply) |
| Custom namespaces | Supported with Collect Custom Metrics enabled | Supported by including the namespace in the stream configuration |
For more detail, see Cloud Metric Delay and the CloudWatch Metric Streams guide.
Why do my metric values look doubled after enabling Metric Streams?
When transitioning from API polling to CloudWatch Metric Streams, there is an overlap period where both collection methods send data for the same metrics. This can cause metric values to appear doubled in Datadog.
Datadog automatically detects streamed namespaces and stops polling them, so you do not need to manually disable API polling. Leave your configuration settings in the AWS integration page unchanged, as Datadog continues to use API polling to collect custom tags, metadata, and metrics that cannot be sent through Metric Streams (such as aws.s3.bucket_size_bytes and aws.billing.estimated_charges).
Detection typically takes up to five minutes, but the overlap period may last longer depending on the timing of active polling crawlers. If values still appear doubled after several minutes, see the CloudWatch Metric Streams guide for troubleshooting.
Which AWS services require additional setup beyond the core integration?
Some AWS services do not emit metrics to CloudWatch by default and require extra configuration:
