To get started using the Agent, select your platform.
Agent architecture
Agent v6 and v7 are composed of a main process responsible for collecting infrastructure metrics, logs, and receiving DogStatsD metrics. The main components to this process are:
- The Collector is in charge of running checks and collecting metrics.
- The Forwarder sends payloads to Datadog.
Two optional processes are spawned by the Agent if enabled in the datadog.yaml
configuration file:
- The APM Agent is a process to collect traces (enabled by default).
- The Process Agent is a process to collect live process information. By default, it only collects available containers, otherwise it is disabled.
On Windows the services are listed as:
Service | Description |
---|
DatadogAgent | Datadog Agent |
datadog-trace-agent | Datadog Trace Agent |
datadog-process-agent | Datadog Process Agent |
By default the Agent binds 3 ports on Linux and 4 on Windows and OSX:
Port | Description |
---|
5000 | Exposes runtime metrics about the Agent. |
5001 | Used by the Agent CLI and GUI to send commands and pull information from the running Agent. |
5002 | Serves the GUI server on Windows and OSX. |
8125 | Used for the DogStatsD server to receive external metrics. |
For information on configuring the ports, see Network Traffic.
Collector
The collector gathers all standard metrics every 15 seconds. Agent v6 embeds a Python 2.7 interpreter to run integrations and custom checks.
Forwarder
The Agent forwarder send metrics over HTTPS to Datadog. Buffering prevents network splits from affecting metric reporting. Metrics are buffered in memory until a limit in size or number of outstanding send requests are reached. Afterwards, the oldest metrics are discarded to keep the forwarder’s memory footprint manageable. Logs are sent over an SSL-encrypted TCP connection to Datadog.
DogStatsD
In v6, DogStatsD is a Golang implementation of Etsy’s StatsD metric aggregation daemon. It is used to receive and roll up arbitrary metrics over UDP or Unix socket, thus allowing custom code to be instrumented without adding latency. Learn more about DogStatsD.
Agent v5 is composed of four major components, each written in Python running as a separate process:
- Collector (
agent.py
): The collector runs checks on the current machine for configured integrations, and captures system metrics, such as memory and CPU. - DogStatsD (
dogstatsd.py
): This is a StatsD-compatible backend server that you can send custom metrics to from your applications. - Forwarder (
ddagent.py
): The forwarder retrieves data from both DogStatsD and the collector, queues it up, and then sends it to Datadog. - SupervisorD: This is all controlled by a single supervisor process. It is kept separate to limit the overhead of each application if you aren’t running all parts. However, it is generally recommended to run all parts.
Note: For Windows users, all four Agent processes appear as instances of ddagent.exe
with the description DevOps' best friend
.
Supervision, privileges, and network ports
A SupervisorD primary process runs as the dd-agent
user, and all forked subprocesses run as the same user. This also applies to any system call (iostat
/netstat
) initiated by the Datadog Agent. The Agent configuration resides at /etc/dd-agent/datadog.conf
and /etc/dd-agent/conf.d
. All configuration must be readable by dd-agent
. The recommended permissions are 0600 since configuration files contain your API key and other credentials needed to access metrics.
The following ports are open for operations:
Port | Description |
---|
tcp/17123 | The forwarder for normal operations |
tcp/17124 | The forwarder for graphite support |
udp/8125 | DogStatsD |
All listening processes are bound by default to 127.0.0.1
and/or ::1
on v3.4.1+ of the Agent. In earlier versions, they were bound to 0.0.0.0
(all interfaces). For information on running the Agent through a proxy see Agent proxy configuration. For information on IP ranges to allow, see Network Traffic.
The recommended number of open file descriptors is 1024. You can see this value with the command ulimit -a
. If you have a hard limitation below the recommended value, for example Shell Fork Bomb Protection, one solution is to add the following in supervisord.conf
:
[supervisord]
minfds = 100 # Your hard limit
GUI
You can configure the port on which the GUI runs in the datadog.yaml
file. To disable the GUI, set the port’s value to -1
. For Windows and macOS, the GUI is enabled by default and runs on port 5002
. For Linux, the GUI is disabled by default.
When the Agent is running, use the datadog-agent launch-gui
command to open the GUI in your default web browser.
Note: The Agent GUI isn’t supported on 32-bit Windows platforms.
Requirements
Cookies must be enabled in your browser. The GUI generates and saves a token in your browser which is used for authenticating all communications with the GUI server.
To start the GUI, the user must have the required permissions. If you are able to open datadog.yaml
, you are able to use the GUI.
For security reasons, the GUI can only be accessed from the local network interface (localhost
/127.0.0.1
), therefore you must be on the same host that the Agent is running. That is, you can’t run the Agent on a VM or a container and access it from the host machine.
CLI
With Agent v6+, the command line interface is based on subcommands. To run a subcommand, first invoke the Agent binary:
<AGENT_BIN_PATH> <SUB_COMMAND> <OPTIONS>
Subcommand | Notes |
---|
check | Run the specified check. |
configcheck | Print all configurations loaded & resolved of a running Agent. |
diagnose | Execute connectivity diagnosis on your system. |
flare | Collect a flare and send it to Datadog. |
health | Print the current Agent health. |
help | Help about any command. |
hostname | Print the hostname used by the Agent. |
import | Import and convert configuration files from previous versions of the Agent. |
launch-gui | Start the Datadog Agent GUI. |
restart | Restart the Agent. |
restart-service | Restart the Agent within the service control manager. |
start | Start the Agent. |
start-service | Start the Agent within the service control manager. |
status | Print the current Agent status. |
stream-logs | Stream the logs being processed by a running agent. |
stop | Stop the Agent. |
stopservice | Stop the Agent within the service control manager. |
version | Print version info. |
Note: Some options have their own set of flags and options detailed in a help message. For example, to see how to use the check
subcommand, run:
<AGENT_BIN_PATH> check --help
Agent overhead
An example of the Datadog Agent resource consumption is below. Tests were made on an Amazon EC2 machine c5.xlarge
instance (4 VCPU/ 8GB RAM) and comparable performance was seen for ARM64-based instances with similar resourcing. The vanilla datadog-agent
was running with a process check to monitor the Agent itself. Enabling more integrations may increase Agent resource consumption.
Enabling JMX Checks forces the Agent to use more memory depending on the number of beans exposed by the monitored JVMs. Enabling the trace and process Agents increases the resource consumption as well.
- Agent Test version: 7.34.0
- CPU: ~ 0.08% of the CPU used on average
- Memory: ~ 130MB of RAM used (RSS memory)
- Network bandwidth: ~ 140 B/s ▼ | 800 B/s ▲
- Disk:
- Linux 830MB to 880MB depending on the distribution
- Windows: 870MB
Log Collection:
The results below are obtained from a collection of 110KB of logs per seconds from a file with the HTTP forwarder enabled. It shows the evolution of resource usage for the different compression levels available.
- Agent Test version: 6.15.0
- CPU: ~ 1.5% of the CPU used on average
- Memory: ~ 95MB of RAM used.
- Network bandwidth: ~ 14 KB/s ▲
- Agent Test version: 6.15.0
- CPU: ~ 1% of the CPU used on average
- Memory: ~ 95MB of RAM used.
- Network bandwidth: ~ 20 KB/s ▲
- Agent Test version: 6.15.0
- CPU: ~ 0.7% of the CPU used on average
- Memory: ~ 90MB of RAM used (RSS memory)
- Network bandwidth: ~ 200 KB/s ▲
Getting further with the Datadog Agent
Update the Agent
To manually update the Datadog Agent core between two minor versions on a given host, run the corresponding install command for your platform.
Note: If you want to manually update one specific Agent integration see the Integration Management guide.
Configuration files
See the Agent configuration files documentation.
Datadog site
Edit the Agent’s main configuration file, datadog.yaml
, to set the site
parameter (defaults to datadoghq.com
).
Note: See the Getting Started with Datadog Sites documentation for further details on the site
parameter.
Log location
See the Agent log files documentation.
Further Reading
Additional helpful documentation, links, and articles: