New announcements for Serverless, Network, RUM, and more from Dash! New announcements from Dash!

Using Autodiscovery with Kubernetes and Docker

QuickStart

This page covers Autodiscovery with Agent 6 only, refer to the dedicated documentation to setup Autodiscovery with Agent 5

Here’s a 5-minute video for a bird’s eye view of Datadog Agent v6 Autodiscovery functionality.

How it Works

In a traditional non-container environment, the Datadog Agent configuration is—like the environment in which it runs—static. The Agent reads check configurations from disk when it starts, and as long as it’s running, it continuously runs every configured check. The configuration files are static, and any network-related options configured within them serve to identify specific instances of a monitored service (e.g. a Redis instance at 10.0.0.61:6379).When an Agent Check cannot connect to such a service, metrics are missing until you troubleshoot the issue. The Agent check retries its failed connection attempts until an administrator revives the monitored service or fixes the check’s configuration.

With Autodiscovery enabled, the Agent runs checks differently.

The overall process of Datadog Agent Autodiscovery is:

  1. Create and Load Integration template: When the Agent starts with Autodiscovery enabled, it loads integration templates from all available template sources; along with the Autodiscovery container identifiers. Static configuration files aren’t suitable for checks that collect data from ever-changing network endpoints like host or ports, so Autodiscovery uses Template Variables for integration template configuration. Those integration template configurations can be loaded into the Agent in 4 main ways:

  2. Apply an integration template to a specific container: Unlike in a traditional Agent setup, the Agent doesn’t run all checks all the time; it decides which checks to enable by inspecting all containers running on the same host as the Agent and the corresponding loaded integration templates. The Agent then watches for Kubernetes/Docker events—container creation, destruction, starts, and stops—and enables, disables, and regenerates static check configurations on such events. As the Agent inspects each running container, it checks if the container matches any of the Autodiscovery container identifiers from any loaded integration templates. For each match, the Agent generates a static check configuration by substituting the Template Variables with the matching container’s specific values. Then it enables the check using the static configuration.

How to set it up

If running the Agent as a binary on a host, enable Autodiscovery with the Host Agent tab instructions. If running the Agent as a container, enable Autodiscovery with the Containerized Agent Tab instructions.

Docker Autodiscovery

To enable Autodiscovery over Docker containers, add the following configuration block in the Agent datadog.yaml configuration file:

listeners:
  - name: docker
config_providers:
  - name: docker
    polling: true

To automatically enable Autodiscovery over Docker containers, mount /var/run/docker.sock into the Containerized Agent.

Kubernetes Autodiscovery

To enable Autodiscovery over containers within Kubernetes, add the following configuration block in the Agent datadog.yaml configuration file:

listeners:
  - name: kubelet
config_providers:
  - name: kubelet
    polling: true
  # needed to support legacy docker label config templates
  - name: docker
    polling: true

To enable Autodiscovery over containers within Kubernetes, add the following environment variable when starting the containerized Agent:

KUBERNETES=true

Note: For Kubernetes users, both a CRI integration and a CRI-O integration are available.

ECS Fargate Autodiscovery

ECS Fargate can’t be monitored with the Datadog Agent running as a binary on a host, see the Containerized Agent Tab instructions.

To enable Autodiscovery over containers within ECS Fargate, add the following environment variable when starting the containerized Agent:

ECS_FARGATE=true

Further Reading