Basic Agent Usage

To get started using the Agent, select your platform.

integration
integration
integration
integration
integration
integration
integration
integration
integration
integration
integration
integration
integration
integration
integration
integration
integration
integration
integration
integration

Agent architecture

Agent v6 and v7 are composed of a main process responsible for collecting infrastructure metrics, logs, and receiving DogStatsD metrics. The main components to this process are:

  • The Collector is in charge of running checks and collecting metrics.
  • The Forwarder sends payloads to Datadog.

Two optional processes are spawned by the Agent if enabled in the datadog.yaml configuration file:

  • The APM Agent is a process to collect traces (enabled by default).
  • The Process Agent is a process to collect live process information. By default, it only collects available containers, otherwise it is disabled.

On Windows the services are listed as:

ServiceDescription
DatadogAgentDatadog Agent
datadog-trace-agentDatadog Trace Agent
datadog-process-agentDatadog Process Agent

By default the Agent binds 3 ports on Linux and 4 on Windows and OSX:

PortDescription
5000Exposes runtime metrics about the Agent.
5001Used by the Agent CLI and GUI to send commands and pull information from the running Agent.
5002Serves the GUI server on Windows and OSX.
8125Used for the DogStatsD server to receive external metrics.

For information on configuring the ports, see Network Traffic.

Collector

The collector gathers all standard metrics every 15 seconds. Agent v6 embeds a Python 2.7 interpreter to run integrations and custom checks.

Forwarder

The Agent forwarder send metrics over HTTPS to Datadog. Buffering prevents network splits from affecting metric reporting. Metrics are buffered in memory until a limit in size or number of outstanding send requests are reached. Afterwards, the oldest metrics are discarded to keep the forwarder’s memory footprint manageable. Logs are sent over an SSL-encrypted TCP connection to Datadog.

DogStatsD

In v6, DogStatsD is a Golang implementation of Etsy’s StatsD metric aggregation daemon. It is used to receive and roll up arbitrary metrics over UDP or Unix socket, thus allowing custom code to be instrumented without adding latency. Learn more about DogStatsD.

Agent v5 Architecture

Agent v5 is composed of four major components, each written in Python running as a separate process:

  • Collector (agent.py): The collector runs checks on the current machine for configured integrations, and captures system metrics, such as memory and CPU.
  • DogStatsD (dogstatsd.py): This is a StatsD-compatible backend server that you can send custom metrics to from your applications.
  • Forwarder (ddagent.py): The forwarder retrieves data from both DogStatsD and the collector, queues it up, and then sends it to Datadog.
  • SupervisorD: This is all controlled by a single supervisor process. It is kept separate to limit the overhead of each application if you aren’t running all parts. However, it is generally recommended to run all parts.

Note: For Windows users, all four Agent processes appear as instances of ddagent.exe with the description DevOps' best friend.

Supervision, privileges, and network ports

A SupervisorD primary process runs as the dd-agent user, and all forked subprocesses run as the same user. This also applies to any system call (iostat/netstat) initiated by the Datadog Agent. The Agent configuration resides at /etc/dd-agent/datadog.conf and /etc/dd-agent/conf.d. All configuration must be readable by dd-agent. The recommended permissions are 0600 since configuration files contain your API key and other credentials needed to access metrics.

The following ports are open for operations:

PortDescription
tcp/17123The forwarder for normal operations
tcp/17124The forwarder for graphite support
udp/8125DogStatsD

All listening processes are bound by default to 127.0.0.1 and/or ::1 on v3.4.1+ of the Agent. In earlier versions, they were bound to 0.0.0.0 (all interfaces). For information on running the Agent through a proxy see Agent proxy configuration. For information on IP ranges to allow, see Network Traffic.

The recommended number of open file descriptors is 1024. You can see this value with the command ulimit -a. If you have a hard limitation below the recommended value, for example Shell Fork Bomb Protection, one solution is to add the following in supervisord.conf:

[supervisord]
minfds = 100  # Your hard limit

GUI

You can configure the port on which the GUI runs in the datadog.yaml file. To disable the GUI, set the port’s value to -1. For Windows and macOS, the GUI is enabled by default and runs on port 5002. For Linux, the GUI is disabled by default.

When the Agent is running, use the datadog-agent launch-gui command to open the GUI in your default web browser.

Note: The Agent GUI isn’t supported on 32-bit Windows platforms.

Requirements

  1. Cookies must be enabled in your browser. The GUI generates and saves a token in your browser which is used for authenticating all communications with the GUI server.

  2. To start the GUI, the user must have the required permissions. If you are able to open datadog.yaml, you are able to use the GUI.

  3. For security reasons, the GUI can only be accessed from the local network interface (localhost/127.0.0.1), therefore you must be on the same host that the Agent is running. That is, you can’t run the Agent on a VM or a container and access it from the host machine.

CLI

With Agent v6+, the command line interface is based on subcommands. To run a subcommand, first invoke the Agent binary:

<AGENT_BIN_PATH> <SUB_COMMAND> <OPTIONS>
SubcommandNotes
checkRun the specified check.
configcheckPrint all configurations loaded & resolved of a running Agent.
diagnoseExecute connectivity diagnosis on your system.
flareCollect a flare and send it to Datadog.
healthPrint the current Agent health.
helpHelp about any command.
hostnamePrint the hostname used by the Agent.
importImport and convert configuration files from previous versions of the Agent.
launch-guiStart the Datadog Agent GUI.
restartRestart the Agent.
restart-serviceRestart the Agent within the service control manager.
startStart the Agent.
start-serviceStart the Agent within the service control manager.
statusPrint the current Agent status.
stream-logsStream the logs being processed by a running agent.
stopStop the Agent.
stopserviceStop the Agent within the service control manager.
versionPrint version info.

Note: Some options have their own set of flags and options detailed in a help message. For example, to see how to use the check subcommand, run:

<AGENT_BIN_PATH> check --help

Agent overhead

An example of the Datadog Agent resource consumption is below. Tests were made on an Amazon EC2 machine c5.xlarge instance (4 VCPU/ 8GB RAM) and comparable performance was seen for ARM64-based instances with similar resourcing. The vanilla datadog-agent was running with a process check to monitor the Agent itself. Enabling more integrations may increase Agent resource consumption. Enabling JMX Checks forces the Agent to use more memory depending on the number of beans exposed by the monitored JVMs. Enabling the trace and process Agents increases the resource consumption as well.

  • Agent Test version: 7.34.0
  • CPU: ~ 0.08% of the CPU used on average
  • Memory: ~ 130MB of RAM used (RSS memory)
  • Network bandwidth: ~ 140 B/s ▼ | 800 B/s ▲
  • Disk:
    • Linux 830MB to 880MB depending on the distribution
    • Windows: 870MB

Log Collection:

The results below are obtained from a collection of 110KB of logs per seconds from a file with the HTTP forwarder enabled. It shows the evolution of resource usage for the different compression levels available.

  • Agent Test version: 6.15.0
  • CPU: ~ 1.5% of the CPU used on average
  • Memory: ~ 95MB of RAM used.
  • Network bandwidth: ~ 14 KB/s ▲
  • Agent Test version: 6.15.0
  • CPU: ~ 1% of the CPU used on average
  • Memory: ~ 95MB of RAM used.
  • Network bandwidth: ~ 20 KB/s ▲
  • Agent Test version: 6.15.0
  • CPU: ~ 0.7% of the CPU used on average
  • Memory: ~ 90MB of RAM used (RSS memory)
  • Network bandwidth: ~ 200 KB/s ▲

Getting further with the Datadog Agent

Update the Agent

To manually update the Datadog Agent core between two minor versions on a given host, run the corresponding install command for your platform.

Note: If you want to manually update one specific Agent integration see the Integration Management guide.

Configuration files

See the Agent configuration files documentation.

Datadog site

Edit the Agent’s main configuration file, datadog.yaml, to set the site parameter (defaults to datadoghq.com).

site: 

Note: See the Getting Started with Datadog Sites documentation for further details on the site parameter.

Log location

See the Agent log files documentation.

Further Reading