Amazon Elastic Beanstalk

Crawler

Overview

AWS Elastic Beanstalk is an easy-to-use service for deploying and scaling web applications and services developed with Java, .NET, PHP, Node.js, Python, Ruby, Go, and Docker on familiar servers such as Apache, Nginx, Passenger, and IIS.

Setup

Installation

If needed, set up the Amazon Web Services integration first. To receive Elastic Beanstalk metrics, you must enable the Enhanced Health Reporting feature for your environment, and configure your environment to publish enhanced health metrics to CloudWatch.

Note: These settings increase your CloudWatch custom metric charges.

Datadog Container Agent Configuration

If you use Docker containers in your Elastic Beanstalk environment, use the containerized Datadog Agent to monitor Docker usage. Follow the steps below to configure your environment to integrate the Datadog Agent container.

Task definition

To run docker environments with multiple containers per instance, Elastic Beanstalk relies on Amazon EC2 Container Service (ECS). For this reason you need to describe the containers you want to deploy the ECS-way. In Elastic Beanstalk this is configured using a file named Dockerrun.aws.json.

A Dockerrun.aws.json file is an Elastic Beanstalk–specific JSON file that describes how to deploy a set of Docker containers as an Elastic Beanstalk application. You can use this file for a multicontainer Docker environment. Dockerrun.aws.json describes the containers to deploy to each container instance in the environment and the data volumes to create on the host instance for the containers to mount.

A Dockerrun.aws.json file can be used on its own or zipped up with additional source code in a single archive. Source code that is archived with Dockerrun.aws.json is deployed to container instances and accessible in the /var/app/current/ directory. Use the volumes section of the config to provide mount points for the containers running on the instance and the mountPoints section of the embedded container definitions to mount them from the containers.

The following code sample illustrates a Dockerrun.aws.json declaring the Datadog Agent. Update the containerDefinitions section with your Datadog API Key, tags (optional), and any additional container definitions. If your are using the Datadog EU site, set DD_SITE to datadoghq.eu. If needed, this file can be zipped with additional content as described above. For more info about the syntax of this file you can refer to the Beanstalk documentation.

Notes:

• For high resource usage, you may need a higher memory limit.
• To ensure all hosts run the same Agent version, it is recommended to change agent:7 to a specific minor version of the Docker image.

{
    "AWSEBDockerrunVersion": 2,
    "volumes": [
        {
            "name": "docker_sock",
            "host": {
                "sourcePath": "/var/run/docker.sock"
            }
        },
        {
            "name": "proc",
            "host": {
                "sourcePath": "/proc/"
            }
        },
        {
            "name": "cgroup",
            "host": {
                "sourcePath": "/cgroup/"
            }
        }
    ],
    "containerDefinitions": [
        {
            "name": "dd-agent",
            "image": "gcr.io/datadoghq/agent:7",
            "environment": [
                {
                    "name": "DD_API_KEY",
                    "value": "<YOUR_DD_API_KEY>"
                },
                {
                    "name": "DD_SITE",
                    "value": "datadoghq.com"
                },
                {
                    "name": "DD_TAGS",
                    "value": "<SIMPLE_TAG>, <KEY:VALUE_TAG>"
                }
            ],
            "memory": 256,
            "mountPoints": [
                {
                    "sourceVolume": "docker_sock",
                    "containerPath": "/var/run/docker.sock",
                    "readOnly": false
                },
                {
                    "sourceVolume": "proc",
                    "containerPath": "/host/proc",
                    "readOnly": true
                },
                {
                    "sourceVolume": "cgroup",
                    "containerPath": "/host/sys/fs/cgroup",
                    "readOnly": true
                }
            ]
        }
    ]
}

Creating the environment

Once the container definition is ready, ship it to Elastic Beanstalk. For specific instructions, refer to Multicontainer Docker Environments in the AWS Elastic Beanstalk documentation.

DogStatsD

To collect custom metrics from your application container using DogStatsD in the Multicontainer Docker Environment, add the following to your Dockerrun.aws.json:

1. Add the environment variable DD_DOGSTATSD_NON_LOCAL_TRAFFIC under the dd-agent container:

   {
     "name": "DD_DOGSTATSD_NON_LOCAL_TRAFFIC",
     "value": "true"
   }
   

2. Add a link to the dd-agent container under your application container:

   "links": [ "dd-agent:dd-agent"]
   

See DogStatsD and Docker for additional information.

Alternate Datadog Agent Configuration

Follow these steps to install the Datadog Agent in Elastic Beanstalk using Advanced Environment Customization with Configuration Files (.ebextensions).

Additional Settings

Add more settings to datadog.yaml by updating the "/configure_datadog_yaml.sh" section of 99datadog.config. The line below enables the Datadog Process Agent.

echo -e "process_config:\n  enabled: \"true\"\n" >> /etc/datadog-agent/datadog.yaml

Trace Collection

Host based

When the application isn’t containerized and the Datadog Agent is configured with 99datadog.config, tracing is enabled without any additional configuration, provided the application is instrumented with the tracing library setup.

Single Docker containers

If the application is containerized and the Datadog Agent is configured with 99datadog.config, do this additional configuration to enable tracing:

1. Update the "/configure_datadog_yaml.sh" section in 99datadog.config file with apm_non_local_traffic, formatted like this:

echo -e "apm_config:\n  enabled: \"true\"\n" >> /etc/datadog-agent/datadog.yaml

echo -e "  apm_non_local_traffic: \"true\"\n" >> /etc/datadog-agent/datadog.yaml

2. Set up the tracing libraries to direct traces to the Gateway IP of the bridge network, which defaults to 172.17.0.1 from inside the application container. (If you’re not sure this is the Gateway IP, run docker inspect <container id> to confirm.)

For all languages, you can set an environment variable called DD_AGENT_HOST to the Gateway IP.

Alternatively, for the languages below, you can programmatically set the host name:

from ddtrace import tracer

tracer.configure(hostname="172.17.0.1")

const tracer = require('dd-trace');

tracer.init({ hostname: "172.17.0.1" });

require 'ddtrace'

Datadog.configure do |c|
  c.tracer hostname: "172.17.0.1")
end

package main

import (
    "gopkg.in/DataDog/dd-trace-go.v1/ddtrace/tracer"
)

func main() {
  tracer.Start(tracer.WithAgentAddr("172.17.0.1"))
  defer tracer.Stop()
  
  // ...
}

Multiple Docker containers

1. In the same Dockerrun.aws.json as the application, add a Datadog Agent container using the gcr.io/datadoghq/agent image. Add the following:
   • Under the portMappings section, add a hostPort 8126 with containerPort 8126.
   • Under the environment section, setDD_APM_ENABLED and DD_APM_NON_LOCAL_TRAFFIC to true.
2. Under your application container, which was instrumented with the [tracing library setup][17], add the following:
   • Under the environment section, add an environment variable called DD_AGENT_HOST to the name of the Datadog Agent container.
   • Under the links section, set the Agent container to be used as an environment variable.

An example can be seen below:

 "containerDefinitions": [    {
      "name": "dd-agent",
      "image": "gcr.io/datadoghq/agent:latest",
      "environment": [
          {
              "name": "DD_API_KEY",
              "value": "<api key>"
          },
          {
              "name": "DD_APM_ENABLED",
              "value": "true"
          },
          {
             "name": "DD_APM_NON_LOCAL_TRAFFIC",
             "value": "true"
          },
         # any other environment variables needed 
      ],
      "portMappings": [
        {
          "hostPort": 8126,
          "containerPort": 8126
        }
      ],
      "memory": 256,
      "mountPoints": [
          # any mountpoints needed
         }
      ]
    },
    {
      "name": "application-container",
      "image": "<application image name>",
      "environment": [
        {
          "name": "DD_AGENT_HOST",
          "value": "dd-agent",
          # any other environment variables needed
        }
      ],
      "links": [
        "dd-agent:dd-agent"
      ],

Data Collected

Metrics

Each of the metrics retrieved from AWS will be assigned the same tags that appear in the AWS console, including but not limited to host name, security-groups, and more.

Events

The AWS Elastic Beanstalk integration does not include any events.

Service Checks

The AWS Elastic Beanstalk integration does not include any service checks.

Troubleshooting

Need help? Contact Datadog support.

Further Reading

• Blog post: Deploy Datadog on AWS Elastic Beanstalk