HAProxy
New announcements from Dash: Incident Management, Continuous Profiler, and more! New announcements from Dash!

HAProxy

Agent Check Agent Check

Supported OS: Linux Mac OS Windows

HAProxy Out of the box Dashboard

Overview

Capture HAProxy activity in Datadog to:

  • Visualize HAProxy load-balancing performance.
  • Know when a server goes down.
  • Correlate the performance of HAProxy with the rest of your applications.

Setup

Installation

The Haproxy check is included in the Datadog Agent package, so you don’t need to install anything else on your Haproxy server.

Prepare HAProxy

Versions < 2

The Agent collects metrics via a stats endpoint:

  1. Configure one in your haproxy.conf:

     listen stats # Define a listen section called "stats"
     bind :9000 # Listen on localhost:9000
     mode http
     stats enable  # Enable stats page
     stats hide-version  # Hide HAProxy version
     stats realm Haproxy\ Statistics  # Title text for popup window
     stats uri /haproxy_stats  # Stats URI
     stats auth Username:Password  # Authentication credentials
  2. Restart HAProxy to enable the stats endpoint.

Versions >= 2

The check supports a newer implementation starting with HAProxy version 2 that is based on a Prometheus endpoint:

  1. Configure your haproxy.conf using the official guide.

  2. Enable the setting use_prometheus in haproxy.d/conf.yaml.

  3. Restart HAProxy to enable the stats endpoint.

Configuration

Host

To configure this check for an Agent running on a host:

Edit the haproxy.d/conf.yaml file, in the conf.d/ folder at the root of your Agent’s configuration directory to start collecting your HAProxy metrics and logs. See the sample haproxy.d/conf.yaml for all available configuration options.

Metric collection
  1. Add this configuration block to your haproxy.d/conf.yaml file to start gathering your Haproxy Metrics:

    init_config:
    
    instances:
     ## @param url - string - required
     ## Haproxy URL to connect to gather metrics.
     ## Set the according <USERNAME> and <PASSWORD> or use directly a unix stats
     ## or admin socket: unix:///var/run/haproxy.sock
     #
     - url: http://localhost/admin?stats
  2. Restart the Agent.

Log collection

Available for Agent versions >6.0

By default Haproxy sends logs over UDP to port 514. The Agent can listen for these logs on this port, however, binding to a port number under 1024 requires elevated permissions. Follow the instructions below to set this up. Alternatively, you can use a different port and skip step 3.

  1. Collecting logs is disabled by default in the Datadog Agent, enable it in your datadog.yaml file:

    logs_enabled: true
  2. Add this configuration block to your haproxy.d/conf.yaml file to start collecting your Haproxy Logs:

    logs:
     - type: udp
       port: 514
       service: <SERVICE_NAME>
       source: haproxy

    Change the service parameter value and configure it for your environment. See the sample haproxy.d/conf.yaml for all available configuration options.

  3. Grant access to port 514 using the setcap command:

    sudo setcap CAP_NET_BIND_SERVICE=+ep /opt/datadog-agent/bin/agent/agent

    Verify the setup is correct by running the getcap command:

    sudo getcap /opt/datadog-agent/bin/agent/agent

    With the expected output:

    /opt/datadog-agent/bin/agent/agent = cap_net_bind_service+ep

    Note: Re-run this setcap command every time you upgrade the Agent.

  4. Restart the Agent.

Containerized

For containerized environments, see the Autodiscovery Integration Templates for guidance on applying the parameters below.

Metric collection
ParameterValue
<INTEGRATION_NAME>haproxy
<INIT_CONFIG>blank or {}
<INSTANCE_CONFIG>{"url": "https://%%host%%/admin?stats"}
Log collection

Available for Agent versions >6.0

Collecting logs is disabled by default in the Datadog Agent. To enable it, see Kubernetes log collection documentation.

ParameterValue
<LOG_CONFIG>{"source": "haproxy", "service": "<SERVICE_NAME>"}

Validation

Run the Agent’s status subcommand and look for haproxy under the Checks section.

Data Collected

Metrics

haproxy.backend_hosts
(gauge)
Number of backend hosts.
Shown as host
haproxy.backend.bytes.in_rate
(gauge)
Rate of bytes in on backend hosts.
Shown as byte
haproxy.backend.bytes.out_rate
(gauge)
Rate of bytes out on backend hosts.
Shown as byte
haproxy.backend.connect.time
(gauge)
Average connect time over the last 1024 backend requests.
Shown as millisecond
haproxy.backend.denied.req_rate
(gauge)
Number of backend requests denied due to security concerns.
Shown as request
haproxy.backend.denied.resp_rate
(gauge)
Number of backend responses denied due to security concerns.
Shown as response
haproxy.backend.errors.con_rate
(gauge)
Rate of backend requests that encountered an error trying to connect to a backend server.
Shown as error
haproxy.backend.errors.resp_rate
(gauge)
Rate of backend responses aborted due to error.
Shown as error
haproxy.backend.queue.current
(gauge)
Number of backend requests without an assigned backend.
Shown as request
haproxy.backend.queue.time
(gauge)
Average queue time over the last 1024 backend requests.
Shown as millisecond
haproxy.backend.response.1xx
(gauge)
Backend HTTP responses with 1xx code.
Shown as response
haproxy.backend.response.2xx
(gauge)
Backend HTTP responses with 2xx code.
Shown as response
haproxy.backend.response.3xx
(gauge)
Backend HTTP responses with 3xx code.
Shown as response
haproxy.backend.response.4xx
(gauge)
Backend HTTP responses with 4xx code.
Shown as response
haproxy.backend.response.5xx
(gauge)
Backend HTTP responses with 5xx code.
Shown as response
haproxy.backend.response.other
(gauge)
Backend HTTP responses with other code (protocol error).
Shown as response
haproxy.backend.response.time
(gauge)
Average response time over the last 1024 backend requests (0 for TCP).
Shown as millisecond
haproxy.backend.requests.tot_rate
(gauge)
Rate of total number of backend HTTP requests.
Shown as request
haproxy.backend.session.current
(gauge)
Number of active backend sessions.
Shown as connection
haproxy.backend.session.limit
(gauge)
Configured backend session limit.
Shown as connection
haproxy.backend.session.pct
(gauge)
Percentage of sessions in use (backend.session.current/backend.session.limit * 100).
Shown as percent
haproxy.backend.session.rate
(gauge)
Number of backend sessions created per second.
Shown as connection
haproxy.backend.session.time
(gauge)
Average total session time over the last 1024 requests.
Shown as millisecond
haproxy.backend.uptime
(gauge)
Number of seconds since the last UP<->DOWN transition
Shown as second
haproxy.backend.warnings.redis_rate
(gauge)
Number of times a request was redispatched to another server.
Shown as error
haproxy.backend.warnings.retr_rate
(gauge)
Number of times a connection to a server was retried.
Shown as error
haproxy.count_per_status
(gauge)
Number of hosts by status (UP/DOWN/NOLB/MAINT).
Shown as host
haproxy.frontend.bytes.in_rate
(gauge)
Rate of bytes in on frontend hosts.
Shown as byte
haproxy.frontend.bytes.out_rate
(gauge)
Rate of bytes out on frontend hosts.
Shown as byte
haproxy.frontend.connections.rate
(gauge)
Number of connections per second.
Shown as connection
haproxy.frontend.connections.tot_rate
(gauge)
Rate of total number of frontend connections.
Shown as connection
haproxy.frontend.denied.req_rate
(gauge)
Number of frontend requests denied due to security concerns.
Shown as request
haproxy.frontend.denied.resp_rate
(gauge)
Number of frontend responses denied due to security concerns.
Shown as response
haproxy.frontend.errors.req_rate
(gauge)
Rate of frontend request errors.
Shown as error
haproxy.frontend.requests.intercepted
(gauge)
Number of intercepted frontend requests per second.
Shown as request
haproxy.frontend.requests.rate
(gauge)
Number of frontend HTTP requests per second.
Shown as request
haproxy.frontend.requests.tot_rate
(gauge)
Rate of total number of frontend HTTP requests.
Shown as request
haproxy.frontend.response.1xx
(gauge)
Frontend HTTP responses with 1xx code.
Shown as response
haproxy.frontend.response.2xx
(gauge)
Frontend HTTP responses with 2xx code.
Shown as response
haproxy.frontend.response.3xx
(gauge)
Frontend HTTP responses with 3xx code.
Shown as response
haproxy.frontend.response.4xx
(gauge)
Frontend HTTP responses with 4xx code.
Shown as response
haproxy.frontend.response.5xx
(gauge)
Frontend HTTP responses with 5xx code.
Shown as response
haproxy.frontend.response.other
(gauge)
Frontend HTTP responses with other code (protocol error).
Shown as response
haproxy.frontend.session.current
(gauge)
Number of active frontend sessions.
Shown as connection
haproxy.frontend.session.limit
(gauge)
Configured frontend session limit.
Shown as connection
haproxy.frontend.session.pct
(gauge)
Percentage of sessions in use (frontend.session.current/frontend.session.limit * 100).
Shown as percent
haproxy.frontend.session.rate
(gauge)
Number of frontend sessions created per second.
Shown as connection
haproxy.sticktable.used
(gauge)
Number of elements in the table.
Shown as item
haproxy.sticktable.size
(gauge)
Maximum number of elements the table can hold.
Shown as item
haproxy.backend.active.servers
(gauge)
Current number of active servers.
haproxy.backend.backup.servers
(gauge)
Current number of backup servers.
haproxy.backend.bytes.in.total
(count)
Current total of incoming bytes.
haproxy.backend.bytes.out.total
(count)
Current total of outgoing bytes.
haproxy.backend.check.last.change.seconds
(gauge)
Number of seconds since the last UP<->DOWN transition.
haproxy.backend.check.up.down.total
(count)
Total number of UP->DOWN transitions.
haproxy.backend.client.aborts.total
(count)
Total number of data transfers aborted by the client.
haproxy.backend.connect.time.average.seconds
(gauge)
Avg. connect time for last 1024 successful connections.
haproxy.backend.connection.attempts.total
(count)
Total number of connection establishment attempts.
haproxy.backend.connection.errors.total
(count)
Total number of connection errors.
haproxy.backend.connection.reuses.total
(count)
Total number of connection reuses.
haproxy.backend.current.queue
(gauge)
Current number of queued requests.
haproxy.backend.current.sessions
(gauge)
Current number of active sessions.
haproxy.backend.downtime.seconds.total
(count)
Total downtime (in seconds) for the service.
haproxy.backend.failed.header.rewriting.total
(count)
Total number of failed header rewriting warnings.
haproxy.backend.http.cache.hits.total
(count)
Total number of HTTP cache hits.
haproxy.backend.http.cache.lookups.total
(count)
Total number of HTTP cache lookups.
haproxy.backend.http.comp.bytes.bypassed.total
(count)
Total number of bytes that bypassed the HTTP compressor (CPU/BW limit).
haproxy.backend.http.comp.bytes.in.total
(count)
Total number of HTTP response bytes fed to the compressor.
haproxy.backend.http.comp.bytes.out.total
(count)
Total number of HTTP response bytes emitted by the compressor.
haproxy.backend.http.comp.responses.total
(count)
Total number of HTTP responses that were compressed.
haproxy.backend.http.requests.total
(count)
Total number of HTTP requests received.
haproxy.backend.http.responses.total
(count)
Total number of HTTP responses.
haproxy.backend.last.session.seconds
(gauge)
Number of seconds since last session assigned to server/backend.
haproxy.backend.limit.sessions
(gauge)
Configured session limit.
haproxy.backend.loadbalanced.total
(count)
Total number of times a service was selected, either for new sessions, or when redispatching.
haproxy.backend.max.connect.time.seconds
(gauge)
Maximum observed time spent waiting for a connection to complete
haproxy.backend.max.queue
(gauge)
Maximum observed number of queued requests.
haproxy.backend.max.queue.time.seconds
(gauge)
Maximum observed time spent in the queue
haproxy.backend.max.response.time.seconds
(gauge)
Maximum observed time spent waiting for a server response
haproxy.backend.max.session.rate
(gauge)
Maximum observed number of sessions per second.
haproxy.backend.max.sessions
(gauge)
Maximum observed number of active sessions.
haproxy.backend.max.total.time.seconds
(gauge)
Maximum observed total request+response time (request+queue+connect+response+processing)
haproxy.backend.queue.time.average.seconds
(gauge)
Avg. queue time for last 1024 successful connections.
haproxy.backend.redispatch.warnings.total
(count)
Total number of redispatch warnings.
haproxy.backend.requests.denied.total
(count)
Total number of denied requests.
haproxy.backend.response.errors.total
(count)
Total number of response errors.
haproxy.backend.response.time.average.seconds
(gauge)
Avg. response time for last 1024 successful connections.
haproxy.backend.responses.denied.total
(count)
Total number of denied responses.
haproxy.backend.retry.warnings.total
(count)
Total number of retry warnings.
haproxy.backend.server.aborts.total
(count)
Total number of data transfers aborted by the server.
haproxy.backend.sessions.total
(count)
Total number of sessions.
haproxy.backend.status
(gauge)
Current status of the service (frontend: 0=STOP, 1=UP, 2=FULL - backend: 0=DOWN, 1=UP - server: 0=DOWN, 1=UP, 2=MAINT, 3=DRAIN, 4=NOLB).
haproxy.backend.total.time.average.seconds
(gauge)
Avg. total time for last 1024 successful connections.
haproxy.backend.weight
(gauge)
Service weight.
haproxy.frontend.bytes.in.total
(count)
Current total of incoming bytes.
haproxy.frontend.bytes.out.total
(count)
Current total of outgoing bytes.
haproxy.frontend.connections.rate.max
(gauge)
Maximum observed number of connections per second.
haproxy.frontend.connections.total
(count)
Total number of connections.
haproxy.frontend.current.sessions
(gauge)
Current number of active sessions.
haproxy.frontend.denied.connections.total
(count)
Total number of requests denied by "tcp-request connection" rules.
haproxy.frontend.denied.sessions.total
(count)
Total number of requests denied by "tcp-request session" rules.
haproxy.frontend.failed.header.rewriting.total
(count)
Total number of failed header rewriting warnings.
haproxy.frontend.http.cache.hits.total
(count)
Total number of HTTP cache hits.
haproxy.frontend.http.cache.lookups.total
(count)
Total number of HTTP cache lookups.
haproxy.frontend.http.comp.bytes.bypassed.total
(count)
Total number of bytes that bypassed the HTTP compressor (CPU/BW limit).
haproxy.frontend.http.comp.bytes.in.total
(count)
Total number of HTTP response bytes fed to the compressor.
haproxy.frontend.http.comp.bytes.out.total
(count)
Total number of HTTP response bytes emitted by the compressor.
haproxy.frontend.http.comp.responses.total
(count)
Total number of HTTP responses that were compressed.
haproxy.frontend.http.requests.rate.max
(gauge)
Maximum observed number of HTTP requests per second.
haproxy.frontend.http.requests.total
(count)
Total number of HTTP requests received.
haproxy.frontend.http.responses.total
(count)
Total number of HTTP responses.
haproxy.frontend.intercepted.requests.total
(count)
Total number of intercepted HTTP requests.
haproxy.frontend.limit.session.rate
(gauge)
Configured limit on new sessions per second.
haproxy.frontend.limit.sessions
(gauge)
Configured session limit.
haproxy.frontend.max.session.rate
(gauge)
Maximum observed number of sessions per second.
haproxy.frontend.max.sessions
(gauge)
Maximum observed number of active sessions.
haproxy.frontend.request.errors.total
(count)
Total number of request errors.
haproxy.frontend.requests.denied.total
(count)
Total number of denied requests.
haproxy.frontend.responses.denied.total
(count)
Total number of denied responses.
haproxy.frontend.sessions.total
(count)
Total number of sessions.
haproxy.frontend.status
(gauge)
Current status of the service (frontend: 0=STOP, 1=UP, 2=FULL - backend: 0=DOWN, 1=UP - server: 0=DOWN, 1=UP, 2=MAINT, 3=DRAIN, 4=NOLB).
haproxy.process.active.peers
(gauge)
Current number of active peers.
haproxy.process.busy.polling.enabled
(gauge)
Non zero if the busy polling is enabled.
haproxy.process.connected.peers
(gauge)
Current number of connected peers.
haproxy.process.connections.total
(count)
Total number of created sessions.
haproxy.process.current.backend.ssl.key.rate
(gauge)
Current backend SSL Key computation per second over last elapsed second.
haproxy.process.current.connection.rate
(gauge)
Current number of connections per second over last elapsed second.
haproxy.process.current.connections
(gauge)
Number of active sessions.
haproxy.process.current.frontend.ssl.key.rate
(gauge)
Current frontend SSL Key computation per second over last elapsed second.
haproxy.process.current.run.queue
(gauge)
Current number of tasks in the run-queue.
haproxy.process.current.session.rate
(gauge)
Current number of sessions per second over last elapsed second.
haproxy.process.current.ssl.connections
(gauge)
Number of opened SSL connections.
haproxy.process.current.ssl.rate
(gauge)
Current number of SSL sessions per second over last elapsed second.
haproxy.process.current.tasks
(gauge)
Current number of tasks.
haproxy.process.current.zlib.memory
(gauge)
Current memory used for zlib in bytes.
haproxy.process.dropped.logs.total
(count)
Total number of dropped logs.
haproxy.process.frontent.ssl.reuse
(gauge)
SSL session reuse ratio (percent).
haproxy.process.hard.max.connections
(gauge)
Initial Maximum number of concurrent connections.
haproxy.process.http.comp.bytes.in.total
(count)
Number of bytes per second over last elapsed second, before http compression.
haproxy.process.http.comp.bytes.out.total
(count)
Number of bytes per second over last elapsed second, after http compression.
haproxy.process.idle.time.percent
(gauge)
Idle to total ratio over last sample (percent).
haproxy.process.jobs
(gauge)
Current number of active jobs (listeners, sessions, open devices).
haproxy.process.limit.connection.rate
(gauge)
Configured maximum number of connections per second.
haproxy.process.limit.http.comp
(gauge)
Configured maximum input compression rate in bytes.
haproxy.process.limit.session.rate
(gauge)
Configured maximum number of sessions per second.
haproxy.process.limit.ssl.rate
(gauge)
Configured maximum number of SSL sessions per second.
haproxy.process.listeners
(gauge)
Current number of active listeners.
haproxy.process.max.backend.ssl.key.rate
(gauge)
Maximum observed backend SSL Key computation per second.
haproxy.process.max.connection.rate
(gauge)
Maximum observed number of connections per second.
haproxy.process.max.connections
(gauge)
Maximum number of concurrent connections.
haproxy.process.max.fds
(gauge)
Maximum number of open file descriptors; 0=unset.
haproxy.process.max.frontend.ssl.key.rate
(gauge)
Maximum observed frontend SSL Key computation per second.
haproxy.process.max.memory.bytes
(gauge)
Per-process memory limit (in bytes); 0=unset.
haproxy.process.max.pipes
(gauge)
Configured maximum number of pipes.
haproxy.process.max.session.rate
(gauge)
Maximum observed number of sessions per second.
haproxy.process.max.sockets
(gauge)
Maximum numer of open sockets.
haproxy.process.max.ssl.connections
(gauge)
Configured maximum number of concurrent SSL connections.
haproxy.process.max.ssl.rate
(gauge)
Maximum observed number of SSL sessions per second.
haproxy.process.max.zlib.memory
(gauge)
Configured maximum amount of memory for zlib in bytes.
haproxy.process.nbproc
(gauge)
Configured number of processes.
haproxy.process.nbthread
(gauge)
Configured number of threads.
haproxy.process.pipes.free.total
(count)
Number of pipes unused.
haproxy.process.pipes.used.total
(count)
Number of pipes in used.
haproxy.process.pool.allocated.bytes
(gauge)
Total amount of memory allocated in pools (in bytes).
haproxy.process.pool.failures.total
(count)
Total number of failed pool allocations.
haproxy.process.pool.used.bytes
(gauge)
Total amount of memory used in pools (in bytes).
haproxy.process.relative.process.id
(gauge)
Relative process id, starting at 1.
haproxy.process.requests.total
(count)
Total number of requests (TCP or HTTP).
haproxy.process.ssl.cache.lookups.total
(count)
Total number of SSL session cache lookups.
haproxy.process.ssl.cache.misses.total
(count)
Total number of SSL session cache misses.
haproxy.process.ssl.connections.total
(count)
Total number of opened SSL connections.
haproxy.process.start.time.seconds
(gauge)
Start time in seconds.
haproxy.process.stopping
(gauge)
Non zero means stopping in progress.
haproxy.process.unstoppable.jobs
(gauge)
Current number of active jobs that can't be stopped during a soft stop.
haproxy.server.bytes.in.total
(count)
Current total of incoming bytes.
haproxy.server.bytes.out.total
(count)
Current total of outgoing bytes.
haproxy.server.check.failures.total
(count)
Total number of failed check (Only counts checks failed when the server is up).
haproxy.server.check.last.change.seconds
(gauge)
Number of seconds since the last UP<->DOWN transition.
haproxy.server.check.up.down.total
(count)
Total number of UP->DOWN transitions.
haproxy.server.client.aborts.total
(count)
Total number of data transfers aborted by the client.
haproxy.server.connect.time.average.seconds
(gauge)
Avg. connect time for last 1024 successful connections.
haproxy.server.connection.attempts.total
(count)
Total number of connection establishment attempts.
haproxy.server.connection.errors.total
(count)
Total number of connection errors.
haproxy.server.connection.reuses.total
(count)
Total number of connection reuses.
haproxy.server.current.queue
(gauge)
Current number of queued requests.
haproxy.server.current.sessions
(gauge)
Current number of active sessions.
haproxy.server.current.throttle
(gauge)
Current throttle percentage for the server, when slowstart is active, or no value if not in slowstart.
haproxy.server.downtime.seconds.total
(count)
Total downtime (in seconds) for the service.
haproxy.server.failed.header.rewriting.total
(count)
Total number of failed header rewriting warnings.
haproxy.server.http.responses.total
(count)
Total number of HTTP responses.
haproxy.server.last.session.seconds
(gauge)
Number of seconds since last session assigned to server/backend.
haproxy.server.limit.sessions
(gauge)
Configured session limit.
haproxy.server.loadbalanced.total
(count)
Total number of times a service was selected, either for new sessions, or when redispatching.
haproxy.server.max.connect.time.seconds
(gauge)
Maximum observed time spent waiting for a connection to complete
haproxy.server.max.queue
(gauge)
Maximum observed number of queued requests.
haproxy.server.max.queue.time.seconds
(gauge)
Maximum observed time spent in the queue
haproxy.server.max.response.time.seconds
(gauge)
Maximum observed time spent waiting for a server response
haproxy.server.max.session.rate
(gauge)
Maximum observed number of sessions per second.
haproxy.server.max.sessions
(gauge)
Maximum observed number of active sessions.
haproxy.server.max.total.time.seconds
(gauge)
Maximum observed total request+response time (request+queue+connect+response+processing)
haproxy.server.queue.limit
(gauge)
Configured maxqueue for the server (0 meaning no limit).
haproxy.server.queue.time.average.seconds
(gauge)
Avg. queue time for last 1024 successful connections.
haproxy.server.redispatch.warnings.total
(count)
Total number of redispatch warnings.
haproxy.server.response.errors.total
(count)
Total number of response errors.
haproxy.server.response.time.average.seconds
(gauge)
Avg. response time for last 1024 successful connections.
haproxy.server.responses.denied.total
(count)
Total number of denied responses.
haproxy.server.retry.warnings.total
(count)
Total number of retry warnings.
haproxy.server.server.aborts.total
(count)
Total number of data transfers aborted by the server.
haproxy.server.server.idle.connections.current
(gauge)
Current number of idle connections available for reuse
haproxy.server.server.idle.connections.limit
(gauge)
Limit on the number of available idle connections
haproxy.server.sessions.total
(count)
Total number of sessions.
haproxy.server.status
(gauge)
Current status of the service (frontend: 0=STOP, 1=UP, 2=FULL - backend: 0=DOWN, 1=UP - server: 0=DOWN, 1=UP, 2=MAINT, 3=DRAIN, 4=NOLB).
haproxy.server.total.time.average.seconds
(gauge)
Avg. total time for last 1024 successful connections.
haproxy.server.weight
(gauge)
Service weight.

Events

The Haproxy check does not include any events.

Service Checks

haproxy.backend_up:
Converts the HAProxy status page into service checks. Returns CRITICAL for a given service if HAProxy is reporting it down. Returns OK for maint, ok and any other state.

Troubleshooting

Need help? Contact Datadog support.

Further Reading