Overview

This guide walks you through integrating an Amazon Web Services (AWS) account with Datadog using Datadog’s CloudFormation template. After completing setup, you can enable individual AWS service integrations, install the Datadog Agent on EC2 instances for deeper visibility, and configure log forwarding.

Prerequisites

Before you begin, ensure that you have an AWS account. The CloudFormation template creates an IAM role and associated policy, allowing Datadog’s AWS account to make API calls to your AWS account to collect and push data. Your AWS user must have the following IAM permissions to run the template:

Setup

1. Go to the AWS integration configuration page in Datadog and click Add AWS Account.
2. Configure the integration’s settings under the Automatically using CloudFormation option.
   1. Select the AWS regions to integrate with.
   2. Add your Datadog API key.
   3. Optionally, send logs and other data to Datadog with the Datadog Forwarder Lambda.
   4. Optionally, enable Cloud Security Misconfigurations to scan your cloud environment, hosts, and containers for misconfigurations and security risks.
3. Click Launch CloudFormation Template. This opens the AWS Console and loads the CloudFormation stack. All the parameters are filled in based on your selections in the prior Datadog form, so you do not need to edit those unless desired.
   Note: The DatadogAppKey parameter enables the CloudFormation stack to make API calls to Datadog to add and edit the Datadog configuration for this AWS account. The key is automatically generated and tied to your Datadog account.
4. Check the required boxes from AWS and click Create stack. This launches the creation process for the Datadog stack along with three nested stacks. This could take several minutes. Ensure that the stack is successfully created before proceeding.
5. After the stack is created, go back to the AWS integration tile in Datadog and click Ready!
6. Wait up to 10 minutes for data to start being collected, and then view the out-of-the-box AWS overview dashboard to see metrics sent by your AWS services and infrastructure:

To set up multiple accounts at once, use the API, AWS CLI, or Terraform. For more information, see the Datadog-Amazon CloudFormation guide.

Note: Datadog’s CloudFormation template only supports creation and deletion of its defined resources. See Update your stack template for guidance on applying updates to your stack.

Configuration

Enable integrations for individual AWS services

See the Integrations page for a full listing of the available sub-integrations. Many of these integrations are installed by default when Datadog recognizes data coming in from your AWS account.

Use the Metric Collection tab on the AWS integration page to configure which services the Datadog integration collects metrics from.

Add regions

Under the General tab on the AWS integration page, you can control the AWS regions where Datadog collects metrics, CloudWatch events, and resources.

Send logs

There are two ways to send AWS service logs to Datadog:

• Amazon Data Firehose destination: Recommended for high-volume CloudWatch logs.
• Forwarder Lambda function: Required for traces, enhanced metrics, or custom metrics from Lambda functions. Also recommended for logs from S3 or other resources that cannot stream directly to Amazon Data Firehose.

See Enable logging for your AWS service for setup instructions.

Validation

Once you have enabled logs, find them in the Log Explorer using either the source or service facets from the facet panel, such as this example from S3:

Get more from the Datadog platform

Deeper visibility with the Datadog Agent on EC2

By default the Datadog AWS integration crawls the CloudWatch API for AWS-provided metrics, but you can gain even deeper visibility into your EC2 instances with the Datadog Agent. The Agent is a lightweight daemon that reports metrics and events, and can also be configured for logs and traces. The Agent Installation section of the Datadog application provides instructions for installing the Agent on a wide variety of operating systems. Many operating systems (for example, Amazon Linux) have one-step installation commands that you can run from the instance terminal to install the Agent:

Once the Agent is installed, it’s graphically represented within the Infrastructure List with a bone icon:

The screenshot above shows the host with the Datadog Agent reporting data from the System and NTP checks. The System check provides metrics around CPU, memory, filesystem, and I/O, providing additional insights into the host. You can enable additional integrations to suit the environment and use case, or additionally use DogStatsD to send custom metrics directly to Datadog.

See the FAQ on why you should install the Datadog Agent on your cloud instances for more information about the benefits of this approach.

Using the Datadog Agent with Amazon Container Services

For containerized environments, you can use the Datadog Agent, whether you’re managing your instances or using Fargate for a serverless environment.

ECS with EC2 launch type

Use the Amazon ECS documentation to run the Datadog Docker Agent on the EC2 instances in your ECS cluster. Review the Amazon ECS Data Collection documentation to see the metrics and events reported to your Datadog account.

ECS with Fargate launch type

Use the Amazon ECS on AWS Fargate documentation to run the Agent as a container in the same task definition as your application. Note: Datadog Agent version 6.1.1 or higher is needed to take full advantage of the Fargate integration.

AWS Batch with Fargate orchestration type

Use the Amazon ECS on AWS Fargate for AWS Batch documentation to run the Agent as a container in the same AWS Batch job definition as your application. Note: Datadog Agent version 6.1.1 or higher is needed to take full advantage of the Fargate integration.

EKS

You don’t need any specific configuration for Amazon Elastic Kubernetes Service (EKS), as mentioned in the Kubernetes Distributions documentation. Use the dedicated Kubernetes documentation to deploy the Agent in your EKS cluster.

EKS with Fargate

Because Fargate pods are managed by AWS, they exclude host-based system checks like CPU and memory. To collect data from your AWS Fargate pods, use the Amazon EKS on AWS Fargate documentation to run the Agent as a sidecar of your application pod with custom role-based access control (RBAC). Note: This requires Datadog Agent version 7.17 or higher.

EKS Anywhere

Use the EKS Anywhere documentation for on-premises Kubernetes clusters.

Create additional Datadog resources

In addition to using the Datadog UI or API, you can create many Datadog resources with the CloudFormation Registry. For visibility and troubleshooting, use dashboards to display key data, apply Functions, and find Metric Correlations.

To get notified of any unwanted or unexpected behavior in your account, create monitors. Monitors consistently evaluate the data reported to your account, and send Notifications to ensure that the right information gets to the right team members. Review the List of Notification Integrations for all the ways to notify your team.

Explore related products

Serverless

To monitor AWS Lambda functions with Datadog, see Serverless for instructions on instrumenting your application, installing Serverless Libraries and Integrations, implementing distributed tracing with serverless applications, or troubleshooting serverless issues.

APM

To collect distributed traces from your applications and AWS services, use the AWS X-Ray integration or the Datadog Agent with APM. See the APM documentation for details on analyzing application performance data.

You can also use Watchdog, an algorithmic feature for APM performance and infrastructure metrics, to automatically detect and be notified about potential application issues.

Security

Cloud SIEM

See Getting Started with Cloud SIEM to evaluate your logs against the out-of-the-box Log Detection Rules. These rules are customizable, and when threats are detected, they generate security signals accessible in the Security Signals Explorer. Use Notification Rules to configure notification preferences across multiple rules.

Cloud Security Misconfigurations

Use the Setting Up Cloud Security Misconfigurations guide to detect and assess misconfigurations in your cloud environment. Resource configuration data is evaluated against the out-of-the-box Cloud and Infrastructure compliance rules to flag attacker techniques and potential misconfigurations.

Troubleshooting

If you encounter the error Datadog is not authorized to perform sts:AssumeRole, see its dedicated troubleshooting page. For any other issues, see the AWS integration troubleshooting guide.

Further Reading

Additional helpful documentation, links, and articles:

Key metrics for AWS monitoringBLOG Introducing our AWS 1-click integrationBLOG Deploying and configuring Datadog with CloudFormationBLOG Implement monitoring as code with Datadog and CloudFormation RegistryBLOG Monitor your entire serverless stack in the Serverless viewBLOG Monitor ECS applications on AWS Fargate with DatadogBLOG Monitor Amazon ECS Anywhere with DatadogBLOG AWS CloudWatch Metric Streams with Amazon Data FirehoseDOCUMENTATION Monitor your Graviton3-powered EC2 instances with DatadogBLOG