Basic Agent Usage

Agent v6 is a complete rewrite in Go of the Agent v5. V6 offers better performances, smaller footprint, and more features. It is the default Datadog Agent (v5 is no longer in active development).

Agent v6 is a composed of a main process responsible for collecting infrastructure metrics, logs, and receiving DogStatsD metrics. The main components to this process are:

• The Collector is in charge of running checks and collecting metrics.
• The Forwarder sends payloads to Datadog.

Two optional processes are spawned by the Agent if enabled in the datadog.yaml configuration file:

• The APM Agent is a process to collect traces (enabled by default).
• The Process Agent is a process to collect live process information. By default, it only collects available containers, otherwise it is disabled.

On Windows the services are listed as:

By default the Agent binds 3 ports on Linux and 4 on Windows and OSX:

The Collector

The collector gathers all standard metrics every 15 seconds. Agent v6 embed a Python2.7 interpreter to run integrations and custom checks.

The Forwarder

The Agent forwarder send metrics over HTTPS to Datadog. Buffering prevents network splits from affecting metric reporting. Metrics are buffered in memory until a limit in size or number of outstanding send requests are reached. Afterwards, the oldest metrics are discarded to keep the forwarder’s memory footprint manageable. Logs are sent over an SSL-encrypted TCP connection to Datadog.

DogStatsD

In v6, DogStatsD is a Golang implementation of Etsy’s StatsD metric aggregation daemon. It is used to receive and roll up arbitrary metrics over UDP or unix socket, thus allowing custom code to be instrumented without adding latency to the mix. Learn more about DogStatsD.

Agent v5 is composed of four major components, each written in Python running as a separate process:

• Collector (agent.py): The collector runs checks on the current machine for configured integrations, and captures system metrics, such as memory and CPU.
• DogStatsD (dogstatsd.py): This is a StatsD-compatible backend server that you can send custom metrics to from your applications.
• Forwarder (ddagent.py): The forwarder retrieves data from both DogStatsD and the collector, queues it up, and then sends it to Datadog.
• SupervisorD: This is all controlled by a single supervisor process. It is kept separate to limit the overhead of each application if you aren’t running all parts. However, it is generally recommended to run all parts.

Note: For Windows users, all four Agent processes appear as instances of ddagent.exe with the description DevOps’ best friend.

Supervision, Privileges, and Network Ports

A SupervisorD master process runs as the dd-agent user, and all forked subprocesses run as the same user. This also applies to any system call (iostat/netstat) initiated by the Datadog Agent. The Agent configuration resides at /etc/dd-agent/datadog.conf and /etc/dd-agent/conf.d. All configuration must be readable by dd-agent. The recommended permissions are 0600 since configuration files contain your API key and other credentials needed to access metrics.

The following ports are open for operations:

All listening processes are bound by default to 127.0.0.1 and/or ::1 on v3.4.1+ of the Agent. In earlier versions, they were bound to 0.0.0.0 (all interfaces). For information on running the Agent through a proxy see Agent proxy configuration. For information on IP ranges to allow, see Network Traffic.

The recommended number of open file descriptors is 1024. You can see this value with the command ulimit -a. If you have a hard limitation below the recommended value, for example Shell Fork Bomb Protection, one solution is to add the following in superisord.conf:

[supervisord]
minfds = 100  # Your hard limit

The Collector

The collector gathers all standard metrics every 15 seconds. It also supports the execution of python-based, user-provided checks, stored in /etc/dd-agent/checks.d. User-provided checks must inherit from the AgentCheck abstract class defined in checks/init.py. See Writing a custom Agent check for more details.

The Forwarder

The Agent forwarder listens for incoming requests over HTTP to send metrics over HTTPS to Datadog. Buffering prevents network splits from affecting metric reporting. Metrics are buffered in memory until a limit in size or number of outstanding send requests are reached. Afterwards, the oldest metrics are discarded to keep the forwarder’s memory footprint manageable.

DogStatsD

DogStatsD is a python implementation of Etsy’s StatsD metric aggregation daemon. It is used to receive and roll up arbitrary metrics over UDP, thus allowing custom code to be instrumented without adding latency to the mix. Learn more about DogStatsD.

Note: Source install may work on operating systems not listed here and is supported on a best effort basis.

Note: Source install may work on operating systems not listed here and is supported on a best effort basis.

• Agent Test version: 6.7.0
• CPU: ~ 0.12% of the CPU used on average
• Memory: ~ 60MB of RAM used (RSS memory)
• Network bandwidth: ~ 86 B/s ▼ | 260 B/s ▲
• Disk:
  • Linux 350MB to 400MB depending on the distribution
  • Windows: 260MB

• Agent Test version: 5.24.0
• CPU: ~ 0.35% of the CPU used on average
• Memory: ~ 115MB of RAM used.
• Network bandwidth: ~ 1900 B/s ▼ | 800 B/s ▲
• Disk:
  • Linux 312MB
  • Windows: 295MB

Note: Since v5.15 of the container Agent, it is recommended to set container resources to at least 256MB due to an added memory cache – upping the limit is not to account for baseline usage but rather to accommodate temporary spikes. Agent 6 has a much more limited memory footprint.

• Chef GitHub project
• Installing Datadog Agent with Chef

• Puppet GitHub project
• Installing Datadog Agent with Puppet

• Ansible GitHub project
• Installing Datadog Agent with Ansible

• Installing Datadog Agent with Saltstack