Using HAProxy as a TCP proxy for logs
This example explains how to configure the Datadog Agent to send logs in TCP to a server with HAProxy installed and listening on port 10514 to then forward the logs to Datadog.
agent ---> haproxy ---> Datadog
The encryption is disabled between the Agent and HAProxy which is then configured to encrypt the data before sending it to Datadog.
Agent configuration
Edit the datadog.yaml Agent configuration file and set logs_no_ssl to true. This is needed as HAProxy does not forward the traffic and is not the Datadog backend, so you cannot use the same certificate.
Note: logs_no_ssl might set to true because HAProxy is configured to encrypt the data. Do not set this parameter to true otherwise.
logs_config:
force_use_tcp: true
logs_dd_url: "<PROXY_SERVER_DOMAIN>:10514"
logs_no_ssl: true
HAProxy configuration
HAProxy should be installed on a host that has connectivity to Datadog. Use the following configuration file if you do not already have it configured.
# Basic configuration
global
log 127.0.0.1 local0
maxconn 4096
stats socket /tmp/haproxy
# Some sane defaults
defaults
log global
option dontlognull
retries 3
option redispatch
timeout client 5s
timeout server 5s
timeout connect 5s
# This declares a view into HAProxy statistics, on port 3833
# You do not need credentials to view this page and you can
# turn it off once you are done with setup.
listen stats
bind *:3833
mode http
stats enable
stats uri /
# This section is to reload DNS Records
# Replace <DNS_SERVER_IP> and <DNS_SECONDARY_SERVER_IP> with your DNS Server IP addresses.
# For HAProxy 1.8 and newer
resolvers my-dns
nameserver dns1 <DNS_SERVER_IP>:53
nameserver dns2 <DNS_SECONDARY_SERVER_IP>:53
resolve_retries 3
timeout resolve 2s
timeout retry 1s
accepted_payload_size 8192
hold valid 10s
hold obsolete 60s
# This declares the endpoint where your Agents connects for
# sending Logs (e.g the value of "logs.config.logs_dd_url")
frontend logs_frontend
bind *:10514
mode tcp
option tcplog
default_backend datadog-logs
# This is the Datadog server. In effect any TCP request coming
# to the forwarder frontends defined above are proxied to
# Datadog's public endpoints.
backend datadog-logs
balance roundrobin
mode tcp
option tcplog
server datadog agent-intake.logs.datadoghq.com:10516 ssl verify required ca-file /etc/ssl/certs/ca-certificates.crt check port 10516
Note: Download the certificate with the following command:
sudo apt-get install ca-certificates (Debian, Ubuntu)yum install ca-certificates (CentOS, Redhat)
If successful, the file will be located at /etc/ssl/certs/ca-bundle.crt for CentOS, Redhat.
Once the HAProxy configuration is in place, you can reload it or restart HAProxy. It is recommended to have a cron job that reloads HAProxy every 10 minutes (for example, service haproxy reload) to force a refresh of HAProxy’s DNS cache, in case app.datadoghq.com fails over to another IP.
# Basic configuration
global
log 127.0.0.1 local0
maxconn 4096
stats socket /tmp/haproxy
# Some sane defaults
defaults
log global
option dontlognull
retries 3
option redispatch
timeout client 5s
timeout server 5s
timeout connect 5s
# This declares a view into HAProxy statistics, on port 3833
# You do not need credentials to view this page and you can
# turn it off once you are done with setup.
listen stats
bind *:3833
mode http
stats enable
stats uri /
# This section is to reload DNS Records
# Replace <DNS_SERVER_IP> and <DNS_SECONDARY_SERVER_IP> with your DNS Server IP addresses.
# For HAProxy 1.8 and newer
resolvers my-dns
nameserver dns1 <DNS_SERVER_IP>:53
nameserver dns2 <DNS_SECONDARY_SERVER_IP>:53
resolve_retries 3
timeout resolve 2s
timeout retry 1s
accepted_payload_size 8192
hold valid 10s
hold obsolete 60s
# This declares the endpoint where your Agents connects for
# sending Logs (e.g the value of "logs.config.logs_dd_url")
frontend logs_frontend
bind *:10514
mode tcp
default_backend datadog-logs
# This is the Datadog server. In effect any TCP request coming
# to the forwarder frontends defined above are proxied to
# Datadog's public endpoints.
backend datadog-logs
balance roundrobin
mode tcp
option tcplog
server datadog agent-intake.logs.datadoghq.eu:443 ssl verify required ca-file /etc/ssl/certs/ca-bundle.crt check port 443
Download the certificate with the following command:
sudo apt-get install ca-certificates (Debian, Ubuntu)yum install ca-certificates (CentOS, Redhat)
If successful, the file will be located at /etc/ssl/certs/ca-bundle.crt for CentOS, Redhat.
Once the HAProxy configuration is in place, you can reload it or restart HAProxy. It is recommended to have a cron job that reloads HAProxy every 10 minutes (for example, service haproxy reload) to force a refresh of HAProxy’s DNS cache, in case app.datadoghq.eu fails over to another IP.
Using NGINX as a TCP Proxy for logs
Agent configuration
Edit the datadog.yaml Agent configuration file and set logs_config.logs_dd_url to use the newly created proxy instead of establishing a connection directly with Datadog:
logs_config:
force_use_tcp: true
logs_dd_url: myProxyServer.myDomain:10514
Note: Do not change the logs_no_ssl parameter as NGINX is forwarding the traffic to Datadog and does not decrypt or encrypt the traffic.
NGINX configuration
In this example, nginx.conf can be used to proxy Agent traffic to Datadog. The last server block in this configuration does TLS wrapping to ensure internal plaintext logs are encrypted between your proxy and Datadog’s log intake API endpoint:
user nginx;
worker_processes auto;
error_log /var/log/nginx/error.log;
pid /run/nginx.pid;
events {
worker_connections 1024;
}
# TCP Proxy for Datadog Agent
stream {
server {
listen 10514; #listen for logs
proxy_ssl on;
proxy_pass agent-intake.logs.datadoghq.com:10516;
}
}
user nginx;
worker_processes auto;
error_log /var/log/nginx/error.log;
pid /run/nginx.pid;
events {
worker_connections 1024;
}
# TCP Proxy for Datadog Agent
stream {
server {
listen 10514; #listen for logs
proxy_ssl on;
proxy_pass agent-intake.logs.datadoghq.eu:443;
}
}
