Starting with version 6.0, the Agent can ingest metrics with a Unix Domain Socket (UDS) as an alternative to UDP transport.
While UDP works great on localhost
, it can be a challenge to set up in containerized environments. Unix Domain Sockets allow you to establish the connection with a socket file, regardless of the IP of the Datadog Agent container. It also enables the following benefits:
Instead of using an IP:port
pair to establish connections, Unix Domain Sockets use a placeholder socket file. Once the connection is open, data is transmitted in the same datagram format as for the UDP transport. When the Agent restarts, the existing socket is deleted and replaced by a new one. Client libraries detect this change and connect seamlessly to the new socket.
Notes:
To set up DogStatsD with Unix Domain Socket, enable the DogStatsD server through the dogstatsd_socket
parameter. Then, configure the DogStatsD client in your code.
To enable the Agent DogStatsD UDS:
Edit the Agent’s main configuration file to set dogstatsd_socket
to the path where DogStatsD should create its listening socket:
## @param dogstatsd_socket - string - optional - default: ""
## Listen for Dogstatsd metrics on a Unix Socket (*nix only).
## Set to a valid and existing filesystem path to enable.
#
dogstatsd_socket: '/var/run/datadog/dsd.socket'
Set the socket path with the DD_DOGSTATSD_SOCKET=<YOUR_UDS_PATH>
environment variable on the Agent container.
Make the socket file accessible to the application containers by mounting a host directory on both sides (read-only in your application containers and read-write in the Agent container). Mounting the parent folder instead of the individual socket enables socket communication to persist across DogStatsD restarts:
-v /var/run/datadog:/var/run/datadog
-v /var/run/datadog:/var/run/datadog:ro
Set the socket path with the DD_DOGSTATSD_SOCKET=<YOUR_UDS_PATH>
environment variable on the Agent container (example: /var/run/datadog/dsd.socket
).
Make the socket file accessible to the application containers by mounting a host directory on both sides (read-only in your application containers and read-write in the Agent container). Mounting the parent folder instead of the individual socket enables socket communication to persist across DogStatsD restarts.
Mount the socket folder in your datadog-agent
container:
volumeMounts:
- name: dsdsocket
mountPath: /var/run/datadog
##...
volumes:
- hostPath:
path: /var/run/datadog/
name: dsdsocket
Expose the same folder in your application containers:
volumeMounts:
- name: dsdsocket
mountPath: /var/run/datadog
readOnly: true
## ...
volumes:
- hostPath:
path: /var/run/datadog/
name: dsdsocket
Note: Remove readOnly: true
if your application containers need write access to the socket.
To send metrics from shell scripts, or to test that DogStatsD is listening on the socket, you can use netcat
. Most implementations of netcat
(ex. netcat-openbsd
on Debian or nmap-ncat
on RHEL) support Unix Socket traffic via the -U
flag:
echo -n "custom.metric.name:1|c" | nc -U -u -w1 /var/run/datadog/dsd.socket
Origin detection allows DogStatsD to detect where the container metrics come from, and tag metrics automatically. When this mode is enabled, all metrics received by UDS are tagged with the same container tags as Autodiscovery metrics.
Enable the dogstatsd_origin_detection
option in your Agent’s main configuration file:
## @param dogstatsd_origin_detection - boolean - optional - default: false
## When using Unix Socket, DogStatsD can tag metrics
## with container metadata. If running DogStatsD in a container,
## host PID mode (e.g. with --pid=host) is required.
#
dogstatsd_origin_detection: true
Optional - To configure tag cardinality for the metrics collected using origin detection, set the parameter dogstatsd_tag_cardinality
to low
(default), orchestrator
, or high
:
## @param dogstatsd_tag_cardinality - string - optional - default: low
## Configure the level of granularity of tags to send for DogStatsD
## metrics and events. Choices are:
## * low: add tags about low-cardinality objects
## (clusters, hosts, deployments, container images, ...)
## * orchestrator: add tags about pods (Kubernetes),
## or tasks (ECS or Mesos) -level of cardinality
## * high: add tags about high-cardinality objects
## (individual containers, user IDs in requests, etc.)
##
## WARNING: Sending container tags for DogStatsD metrics may create
## more metrics (one per container instead of one per host).
## This may impact your custom metrics billing.
#
dogstatsd_tag_cardinality: low
Set the DD_DOGSTATSD_ORIGIN_DETECTION=true
environement variable for the Agent container.
Optional - To configure tag cardinality for the metrics collected using origin detection, set the environment variable DD_DOGSTATSD_TAG_CARDINALITY
to low
(default), orchestrator
, or high
.
When running inside a container, DogStatsD needs to run in the host’s PID namespace for origin detection to work reliably. Enable this in Docker with --pid=host
flag. This is supported by ECS with the parameter "pidMode": "host"
in the task definition of the container. This option is not supported in Fargate. For more information, see the AWS documentation.
Set the DD_DOGSTATSD_ORIGIN_DETECTION
environment variable to true for the Agent container:
# (...)
env:
# (...)
- name: DD_DOGSTATSD_ORIGIN_DETECTION
value: 'true'
Optional - To configure tag cardinality for the metrics collected using origin detection, set the environment variable DD_DOGSTATSD_TAG_CARDINALITY
to low
(default), orchestrator
, or high
:
# (...)
env:
# (...)
- name: DD_DOGSTATSD_TAG_CARDINALITY
value: 'low'
Note: container_id
, container_name
, and pod_name
tags are not added by default to avoid creating too many custom metrics.
The following official DogStatsD client libraries natively support UDS traffic. Refer to the library’s documentation on how to enable UDS traffic. Note: As with UDP, enabling client-side buffering is highly recommended to improve performance on heavy traffic:
Language | Library |
---|---|
Golang | DataDog/datadog-go |
Java | DataDog/java-dogstatsd-client |
Python | DataDog/datadogpy |
Ruby | DataDog/dogstatsd-ruby |
PHP | DataDog/php-datadogstatsd |
C# | DataDog/dogstatsd-csharp-client |
If an application or a client library does not support UDS traffic, run socat
to listen on UDP port 8125
and proxy the requests to the socket:
socat -s -u UDP-RECV:8125 UNIX-SENDTO:/var/run/datadog/dsd.socket
For guidelines on creating additional implementation options, refer to the datadog-agent github wiki.
Additional helpful documentation, links, and articles:
On this Page