Starting with v6.0, the Agent can ingest metrics with a Unix Domain Socket (UDS) as an alternative to UDP when running on Linux systems.
While UDP works great on
localhost, it can be a challenge to set up in containerized environments. Unix Domain Sockets allow you to establish the connection with a socket file, regardless of the IP of the Datadog Agent container. It also enables the following benefits:
Instead of using an
IP:port pair to establish connections, Unix Domain Sockets use a placeholder socket file. Once the connection is open, data is transmitted in the same datagram format as UDP. When the Agent restarts, the existing socket is deleted and replaced by a new one. Client libraries detect this change and connect seamlessly to the new socket.
Note: By design, UDS traffic is local to the host, which means the Datadog Agent must run on every host you send metrics from.
To set up DogStatsD, enable the DogStatsD server through the
dogstatsd_socket parameter. Then, configure the DogStatsD client in your code.
To enable the Agent DogStatsD server:
Edit the Agent’s main configuration file to set
dogstatsd_socket to the path where DogStatsD should create its listening socket:
## @param dogstatsd_socket - string - optional - default: "" ## Listen for Dogstatsd metrics on a Unix Socket (*nix only). Set to a valid filesystem path to enable. # dogstatsd_socket: "/var/run/datadog/dsd.socket"
Note: You can also set the socket path with the
DD_DOGSTATSD_SOCKET environment variable for the container Agent.
The following official DogStatsD client libraries natively support UDS traffic. Refer to the library’s documentation on how to enable UDS traffic. Note: As with UDP, enabling client-side buffering is highly recommended to improve performance on heavy traffic:
To send metrics from shell scripts, or to test that DogStatsD is listening on the socket, you can use
netcat. Most implementations of
netcat-openbsd on Debian or
nmap-ncat on RHEL) support Unix Socket traffic via the
echo -n "custom.metric.name:1|c" | nc -U -u -w1 /var/run/datadog/dsd.socket
If an application or a client library you use does not support UDS traffic, you can run
socat to listen on UDP port
8125 and proxy the requests to the socket:
socat -s -u UDP-RECV:8125 UNIX-SENDTO:/var/run/datadog/dsd.socket
For guidelines on creating additional implementation options, refer to the datadog-agent github wiki.
When running in a containerized environment, the socket file needs to be accessible to the client containers. To achieve this, Datadog recommends mounting a host directory on both sides (read-only in your client containers, read-write in the Agent container). Mounting the parent folder instead of the individual socket enables socket communication to persist across DogStatsD restarts:
Mount the folder in your
volumeMounts: - name: dsdsocket mountPath: /var/run/datadog ##... volumes: - hostPath: path: /var/run/datadog/ name: dsdsocket
Expose the same folder in your client containers:
volumeMounts: - name: dsdsocket mountPath: /var/run/datadog readOnly: true ## ... volumes: - hostPath: path: /var/run/datadog/ name: dsdsocket
readOnly: true if your client containers need write access to the socket.
Origin detection allows DogStatsD to detect where the container metrics come from, and tag metrics automatically. When this mode is enabled, all metrics received via UDS are tagged by the same container tags as Autodiscovery metrics.
pod_name tags are not added to avoid creating too many custom metrics.
To use origin detection:
dogstatsd_origin_detection option in your Agent’s main configuration file:
## @param dogstatsd_origin_detection - boolean - optional - default: false ## When using Unix Socket, DogStatsD can tag metrics with container metadata. ## If running DogStatsD in a container, host PID mode (e.g. with --pid=host) is required. # dogstatsd_origin_detection: true
Note: For the containerized Agent, set the environment variable
DD_DOGSTATSD_ORIGIN_DETECTION to true.
When running inside a container, DogStatsd needs to run in the host’s PID namespace for origin detection to work reliably. Enable this with the Docker
Note: This is supported by ECS with the parameter
"pidMode": "host" in the task definition of the container. This option is not supported in Fargate. For more information, see the AWS documentation.
Additional helpful documentation, links, and articles: