Getting Started with AWS

Cette page n'est pas encore disponible en français, sa traduction est en cours.
Si vous avez des questions ou des retours sur notre projet de traduction actuel, n'hésitez pas à nous contacter.

Overview

This guide provides an overview of the process for integrating an Amazon Web Services(AWS) account with Datadog using Datadog’s CloudFormation template.

At a high level, this involves creating an IAM role and associated policy to enable Datadog’s AWS account to make API calls into your AWS account for collecting or pushing data. The template also deploys the Datadog Forwarder Lambda function for sending logs to Datadog. Using the CloudFormation template provides all the tools needed to send this data to your Datadog account, and Datadog maintains the CloudFormation template to provide the latest functionality.

After the initial connection is established, you can enable individual AWS service integrations relevant to your AWS environment. With a single click, Datadog provisions the necessary resources in your AWS account and begins querying metrics and events for the services you use. For popular AWS services you are using, Datadog provisions out-of-the-box dashboards, providing immediate and customizable visibility. This guide demonstrates setting up the integration, sending logs from CloudTrail and the Forwarder Lambda function, and installing the Datadog Agent on an Amazon Linux EC2 instance. See the Enable integrations for individual AWS service section for a list of the available sub-integrations.

This process can be repeated for as many AWS accounts as necessary, or you can also use the API, AWS CLI, or Terraform to set up multiple accounts at once. For more information, read the Datadog-Amazon CloudFormation guide.

Prerequisites

Before getting started, ensure you have the following prerequisites:

  1. An AWS account. Your AWS user needs the following IAM permissions to successfully run the CloudFormation template:

    • cloudformation:CreateStack
    • ec2:DescribeSecurityGroups
    • ec2:DescribeSubnets
    • ec2:DescribeVpcs
    • iam:AttachRolePolicy
    • iam:CreatePolicy
    • iam:CreateRole
    • iam:PassRole
    • iam:PutRolePolicy
    • iam:UpdateAssumeRolePolicy
    • kms:Decrypt
    • lambda:AddPermission
    • lambda:CreateFunction
    • lambda:GetCodeSigningConfig
    • lambda:GetFunction
    • lambda:GetFunctionCodeSigningConfig
    • lambda:InvokeFunction
    • lambda:PutFunctionConcurrency
    • logs:CreateLogGroup
    • logs:DescribeLogGroups
    • logs:PutRetentionPolicy
    • s3:CreateBucket
    • s3:GetObject
    • s3:GetObjectVersion
    • secretsmanager:CreateSecret
    • secretsmanager:GetSecretValue
    • secretsmanager:PutSecretValue
    • serverless:CreateCloudFormationTemplate

Setup

  1. From the AWS tile on the Integrations page in your Datadog account, select the Datadog products you wish to integrate with this AWS Account. This selects the correct default settings for integrating data from this AWS account for those products. These settings can be changed in the future if needed.

    The Datadog AWS integration tile showing the options for establishing the integration. The Role Delegation tab is highlighted.

  2. Select the AWS Region where the CloudFormation stack will be launched. This also sets where to create the Datadog Lambda Forwarder for sending AWS logs to Datadog (if you selected Log Management).

    Note: CloudWatch metrics are collected from ALL AWS regions you are using regardless of the region you select.

  3. Select or create the Datadog API Key used to send data from your AWS account to Datadog.

  4. Click “Launch CloudFormation Template”. This opens the AWS Console and loads the CloudFormation stack. All the parameters are filled in based on your selections in the prior Datadog form, so you do not need to edit those unless desired. Note: The DatadogAppKey parameter enables the CloudFormation stack to make API calls to Datadog to add and edit the Datadog configuration for this AWS account. The key is automatically generated and tied to your Datadog account.

    The AWS CloudFormation create-stack page showing the Stack name as datadog, IAMRoleName as DatadogIntegrationRole, ExternalId as an obfuscated value ending in be46, DdApiKey as an obfuscated value.

  5. Check the required boxes from AWS and click Create stack:

    The AWS CloudFormation Stacks page showing the four completed stacks under the 'Stacks' column along the left hand side of the page. The stacks are datadog-DatadogIntegrationRoleStack, datadog-DatadogPolicyMacroStack, datadog-ForwarderStack, and datadog. Each stack shows the timestamp of creation and a green checkmark with CREATE_COMPLETE. The 'datadog' stack and is highlighted and displaying the 'Events' tab. There are 9 events listed with their Timestamp, Logical ID, Status, and Status reason. These events reference the different stages of creation for each of the stacks.
    This launches the creation process for the Datadog stack along with three nested stacks. This could take several minutes. Ensure that the stack is successfully created before proceeding.

  6. After the Stack is created, go back to the AWS integration tile in Datadog and find the box for the new account you created. Click “Refresh to Check Status” to see a success message at the top of the page, along with the new account visible on the page with the relevant details.

    The AWS integration tile in the Datadog account showing the Account: New Account section and a message that the integration setup with CloudFormation is pending completion. There is a button for refreshing to check the status, and a warning to check the CloudFormation stack generation before checking the status.

    Depending on which AWS services you use and your use case for monitoring, there are multiple options within the integration tile to specify the data to be collected. For example, you can limit data collection based on AWS service, namespace, or tags. Additionally, you can choose to mute monitor notifications. For example, terminations triggered manually or by autoscaling with EC2 automuting enabled. If needed, enable Alarm Collection to send your CloudWatch alarms to the Datadog Event Stream and choose whether to collect custom metrics.

  7. Wait up to 10 minutes for data to start being collected, and then view the out-of-the-box AWS overview dashboard to see metrics sent by your AWS services and infrastructure:

    The AWS overview dashboard in the Datadog account. On the left is the AWS logo and an AWS events graph showing 'No matching entries found'. In the center are graphs related to EBS volumes with numerical data displayed and a heat map showing consistent data. Along the right are graphs related to ELBs showing numerical data as well as a timeseries graph showing spiky data from three sources.

Enable integrations for individual AWS service

See the Integrations page for a full listing of the available sub-integrations. Many of these integrations are installed by default when Datadog recognizes data coming in from your AWS account.

Send logs

For a full list of ways you can send your AWS logs to Datadog, see Enable logging for your AWS service.

Validation

Once you have enabled logs, find them in the Logs Explorer using either the source or service facets from the facet panel, such as this example from S3:

The Logs Explorer page of the Datadog account. Along the left the image displays the Source and Service facets, both checked with 's3'. Along the right, some log entries are displayed in a list format.

Get more from the Datadog platform

Deeper visibility with the Datadog Agent on EC2

By default the Datadog AWS integration crawls the CloudWatch API for AWS-provided metrics, but you can gain even deeper visibility into your EC2 instances with the Datadog Agent. The Agent is a lightweight daemon that reports metrics and events, and can also be configured for logs and traces. The Agent Installation section of the Datadog application provides instructions for installing the Agent on a wide variety of operating systems. Many operating systems (for example, Amazon Linux) have one-step installation commands that you can run from the instance terminal to install the Agent:

The 'Agent' section of the 'Integrations' tab in Datadog. Along the left are displayed a list of supported operating systems for the Datadog Agent. 'Amazon Linux' is highlighted from this list. On the right is displayed 'Use our easy one-step install'. The command for installing the Agent is displayed below this, with the DD_API_KEY section obfuscated.

Once the Agent is installed, it’s graphically represented within the Infrastructure List with a bone icon:

The infrastructure list showing two hosts in a list format. Both hosts show the AWS icon for the AWS integration and 'aws' shown in a blue box to show they are associated with the AWS integration. One host also shows a dog-bone icon and blue boxes for 'ntp' and 'system'.

The screen shot above shows the host with the Datadog Agent reporting data from the System and NTP checks. The System check provides metrics around CPU, memory, filesystem, and I/O, providing additional insights into the host. You can enable additional integrations to suit the environment and use case, or additionally use DogStatsD to send custom metrics directly to Datadog.

See the FAQ on why you should install the Datadog Agent on your cloud instances for more information about the benefits of this approach.

Using the Datadog Agent with Amazon Container Services

For containerized environments, you can use the Datadog Agent, whether you’re managing your instances or leveraging Fargate for a serverless environment.

ECS with EC2 launch type

Use the Amazon ECS documentation to run the Datadog Docker Agent on the EC2 instances in your ECS cluster. Review the Amazon ECS Data Collection documentation to see the metrics and events reported to your Datadog account.

ECS with Fargate launch type

Use the Amazon ECS on AWS Fargate documentation to run the Agent as a container in the same task definition as your application. Note: Datadog Agent version 6.1.1 or higher is needed to take full advantage of the Fargate integration.

EKS

You don’t need any specific configuration for Amazon Elastic Kubernetes Service (EKS), as mentioned in the Kubernetes Distributions documentation. Use the dedicated Kubernetes documentation to deploy the Agent in your EKS cluster.

EKS with Fargate

Because Fargate pods are managed by AWS, they exclude host-based system checks like CPU and memory. To collect data from your AWS Fargate pods, use the Amazon EKS on AWS Fargate documentation to run the Agent as a sidecar of your application pod with custom role-based access control (RBAC). Note: This requires Datadog Agent version 7.17 or higher.

EKS Anywhere

Use the EKS Anywhere documentation for on-premises Kubernetes clusters.

Create additional Datadog resources

In addition to using the Datadog UI or API, you can create many Datadog resources with the CloudFormation Registry. For visibility and troubleshooting, use dashboards to display key data, apply Functions, and find Metric Correlations.

To get notified of any unwanted or unexpected behavior in your account, create monitors. Monitors consistently evaluate the data reported to your account, and send Notifications to ensure that the right information gets to the right team members. Review the List of Notification Integrations for all the ways to notify your team.

Serverless

You can unify the metrics, traces, and logs from your AWS Lambda functions running serverless applications in Datadog. Check out Serverless for instructions on instrumenting your application, installing Serverless Libraries and Integrations, implementing Distributed Tracing with Serverless Applications, or Serverless Troubleshooting.

APM

To dig even deeper and gather more data from your applications and AWS services, enable collecting distributed traces from either the AWS X-Ray integration or from a host with the Datadog Agent using APM. Then, read Explore Datadog APM for a better understanding of how to use this data to gain insights into your application performance.

Additionally, you can use Watchdog, an algorithmic feature for APM performance and infrastructure metrics, to automatically detect and be notified about potential application issues.

Security

Cloud SIEM

Review Getting Started with Cloud SIEM to evaluate your logs against the out-of-the-box Log Detection Rules. These rules are customizable, and when threats are detected, they generate security signals which can be accessed on the Security Signals Explorer. To ensure that the correct team is notified, use Notification Rules to configure notification preferences across multiple rules.

CSPM

Use the Getting Started with CSPM guide to learn about detecting and assessing misconfigurations in your cloud environment. Resource configuration data is evaluated against the out-of-the-box Posture Management Cloud and Infrastructure Detection Rules to flag attacker techniques and potential misconfigurations, allowing for fast response and remediation.

Troubleshooting

If you encounter any issues, be sure to check out the Troubleshooting section.

Further Reading