Join us at the Dash conference! July 16-17, NYC

AWS Load Balancer

Crawler Crawler

Overview

Elastic Load Balancing (ELB) is an AWS service used to dispatch incoming web traffic from your applications across your Amazon EC2 backend instances, which may be in different availability zones. ELB helps ensure a smooth user experience and provide increased fault tolerance, handling traffic peaks and failed EC2 instances without interruption.

Datadog collects metrics and metadata from all three flavors of Elastic Load Balancers that AWS offers: Application, Classic, and Network Load Balancers.

Setup

Installation

If you haven’t already, set up the Amazon Web Services integration first.

Metric collection

  1. In the AWS integration tile, ensure that ELB is checked under metric collection. Check also ApplicationELB checkbox for Application ELB metrics, and the NetworkELB checkbox for Network ELB metrics.

  2. Add those permissions to your Datadog IAM policy in order to collect Amazon ELB metrics:

    • elasticloadbalancing:DescribeLoadBalancers: List ELBs, add additional tags and metrics.
    • elasticloadbalancing:DescribeTags: Add custom ELB tags to ELB metrics.
    • elasticloadbalancing:DescribeInstanceHealth: Add state of your instances.

    For more information on ELB policies, review the documentation on the AWS website.

  3. Install the Datadog - AWS ELB integration.

Log collection

Enable AWS ELB logging

Enable the logging on your ELB first to collect your logs. ELB logs can be written in a AWS S3 bucket and consumed by a Lambda function. For more information, refer to the AWS documentation

Set interval to 5 minutes and define your s3 buckets:

Send logs to Datadog

  1. If you haven’t already, set up the Datadog log collection AWS Lambda function.
  2. Once the lambda function is installed, manually add a trigger on the S3 bucket that contains your ELB logs in the AWS console, in your Lambda, click on S3 in the trigger list: Configure your trigger by choosing the S3 bucket that contains your ELB logs and change the event type to Object Created (All) then click on the add button.

Once done, go in your Datadog Log section to start exploring your logs!

Data Collected

Metrics are collected under the following namespaces:

Datadog Namespace AWS Service
aws.applicationelb Application Load Balancers
aws.elb Classic Elastic Load Balancing
aws.network.elb Network Load Balancers

Metrics

aws.applicationelb.active_connection_count
(count)
The total number of concurrent TCP connections active from clients to the load balancer and from the load balancer to targets.
shown as connection
aws.applicationelb.client_tlsnegotiation_error_count
(count)
Number of TLS negotiation errors
shown as error
aws.applicationelb.consumed_lcus
(gauge)
The number of load balancer capacity units (LCU) used by your load balancer.
shown as unit
aws.applicationelb.elb_auth_error
(count)
The number of user authentications that could not be completed because an authenticate action was misconfigured, the load balancer couldn't establish a connection with the IdP, or the load balancer couldn't complete the authentication flow due to an internal error.
shown as error
aws.applicationelb.elb_auth_failure
(count)
The number of user authentications that could not be completed because the IdP denied access to the user or an authorization code was used more than once.
shown as error
aws.applicationelb.elb_auth_latency
(gauge)
The time elapsed, in milliseconds, to query the IdP for the ID token and user info. If one or more of these operations fail, this is the time to failure.
shown as millisecond
aws.applicationelb.elb_auth_refresh_token_success
(count)
The number of times the load balancer successfully refreshed user claims using a refresh token provided by the IdP.
shown as success
aws.applicationelb.elb_auth_success
(count)
The number of authenticate actions that were successful.
shown as success
aws.applicationelb.elb_authuser_claims_size_exceeded
(count)
The number of times that a configured IdP returned user claims that exceeded 11K bytes in size.
aws.applicationelb.healthy_host_count
(gauge)
Average number of healthy instances in each Availability Zone.
shown as host
aws.applicationelb.healthy_host_count_deduped
(count)
The number of healthy instances per Availability Zone, regardless of if the Cross-Zone Load Balancing option is enabled or not.
shown as host
aws.applicationelb.healthy_host_count.maximum
(gauge)
Maximum number of healthy instances in each Availability Zone.
shown as host
aws.applicationelb.healthy_host_count.minimum
(gauge)
Minimum number of healthy instances in each Availability Zone.
shown as host
aws.applicationelb.httpcode_elb_3xx
(count)
The number of HTTP 3XX redirection codes that originate from the load balancer.
shown as response
aws.applicationelb.httpcode_elb_4xx
(count)
Number of HTTP 4XX client error codes generated by the load balancer.
shown as response
aws.applicationelb.httpcode_elb_5_0_0
(count)
The number of HTTP 500 error codes that originate from the load balancer.
shown as response
aws.applicationelb.httpcode_elb_5_0_2
(count)
The number of HTTP 502 error codes that originate from the load balancer.
shown as response
aws.applicationelb.httpcode_elb_5_0_3
(count)
The number of HTTP 503 error codes that originate from the load balancer.
shown as response
aws.applicationelb.httpcode_elb_5_0_4
(count)
The number of HTTP 504 error codes that originate from the load balancer.
shown as response
aws.applicationelb.httpcode_elb_5xx
(count)
Number of HTTP 5XX client error codes generated by the load balancer.
shown as response
aws.applicationelb.httpcode_target_2xx
(count)
Number of HTTP 2XX response codes generated by registered instances.
shown as response
aws.applicationelb.httpcode_target_3xx
(count)
Number of HTTP 3XX response codes generated by registered instances.
shown as response
aws.applicationelb.httpcode_target_4xx
(count)
Number of HTTP 4XX response codes generated by registered instances.
shown as response
aws.applicationelb.httpcode_target_5xx
(count)
Number of HTTP 5XX response codes generated by registered instances.
shown as response
aws.applicationelb.httpfixed_response
(count)
The number of fixed-response actions that were successful.
shown as response
aws.applicationelb.httpredirect
(count)
The number of redirect actions that were successful.
aws.applicationelb.httpredirect_url_limit_exceeded
(count)
The number of redirect actions that couldn't be completed because the URL in the response location header is larger than 8K.
aws.applicationelb.ipv_6processed_bytes
(count)
The total number of bytes processed by the load balancer over IPv6.
shown as byte
aws.applicationelb.ipv_6request_count
(count)
The number of IPv6 requests received by the load balancer.
shown as request
aws.applicationelb.lambda_internal_error
(count)
The number of requests to a Lambda function that failed because of an issue internal to the load balancer or AWS Lambda.
shown as request
aws.applicationelb.lambda_target_processed_bytes
(gauge)
The total number of bytes processed by the load balancer for requests to and responses from a Lambda function.
shown as byte
aws.applicationelb.lambda_user_error
(count)
The number of requests to a Lambda function that failed because of an issue with the Lambda function.
shown as request
aws.applicationelb.new_connection_count
(count)
The total number of new TCP connections established from clients to the load balancer and from the load balancer to targets.
shown as connection
aws.applicationelb.non_sticky_request_count
(count)
The number of requests where the load balancer chose a new target because it couldn't use an existing sticky session.
shown as request
aws.applicationelb.processed_bytes
(count)
The total number of bytes processed by the load balancer over IPv4 and IPv6.
shown as byte
aws.applicationelb.rejected_connection_count
(count)
The number of connections that were rejected because the load balancer had reached its maximum number of connections.
shown as connection
aws.applicationelb.request_count
(count)
Total number of completed requests that were received and routed to the registered instances.
shown as request
aws.applicationelb.request_count_per_target
(count)
The average number of requests received by each target in a target group.
shown as request
aws.applicationelb.rule_evaluations
(count)
The number of rules processed by the load balancer given a request rate averaged over an hour.
aws.applicationelb.target_connection_error_count
(count)
Number of connections that were not successfully established between the load balancer and the registered instances.
shown as error
aws.applicationelb.target_response_time.average
(gauge)
Average time elapsed after the request leaves the load balancer until a response is received.
shown as second
aws.applicationelb.target_response_time.maximum
(gauge)
Maximum time elapsed after the request leaves the load balancer until a response is received.
shown as second
aws.applicationelb.target_response_time.p95
(gauge)
95th percentile of the time elapsed after the request leaves the load balancer until a response is received.
shown as second
aws.applicationelb.target_response_time.p99
(gauge)
99th percentile of the time elapsed after the request leaves the load balancer until a response is received.
shown as second
aws.applicationelb.target_tlsnegotiation_error_count
(count)
The number of TLS connections initiated by the load balancer that did not establish a session with the target.
shown as connection
aws.applicationelb.un_healthy_host_count
(gauge)
Average number of unhealthy instances in each Availability Zone.
shown as host
aws.applicationelb.un_healthy_host_count_deduped
(count)
The number of unhealthy instances per Availability Zone, regardless of if the Cross-Zone Load Balancing option is enabled or not.
shown as host
aws.applicationelb.un_healthy_host_count.maximum
(gauge)
Maximum number of unhealthy instances in each Availability Zone.
shown as host
aws.applicationelb.un_healthy_host_count.minimum
(gauge)
Minimium number of unhealthy instances in each Availability Zone.
shown as host
aws.elb.active_connection_count
(count)
The total number of concurrent TCP connections active from clients to the load balancer and from the load balancer to targets.
shown as connection
aws.elb.backend_connection_errors
(rate)
Number of connections that were not successfully established between the load balancer and the registered instances.
shown as error
aws.elb.client_tlsnegotiation_error_count
(count)
Number of TLS negotiation errors
shown as error
aws.elb.consumed_lbcapacity_units
(gauge)
Number of ELB capacity units consumed
shown as unit
aws.elb.consumed_lcus
(gauge)
The number of load balancer capacity units (LCU) used by your load balancer.
shown as unit
aws.elb.estimated_albactive_connection_count
(count)
The estimated total number of concurrent TCP connections active from clients to the load balancer and from the load balancer to targets.
shown as connection
aws.elb.estimated_albconsumed_lcus
(gauge)
The estimated total number of load balancer capacity units (LCU) used by an Application Load Balancer.
shown as unit
aws.elb.estimated_albnew_connection_count
(count)
The estimated total number of new TCP connections established from clients to the load balancer and from the load balancer to targets
shown as connection
aws.elb.estimated_processed_bytes
(count)
The estimated total number of bytes processed by an Application Load Balancer.
shown as byte
aws.elb.healthy_host_count
(gauge)
Average number of healthy instances in each Availability Zone.
shown as host
aws.elb.healthy_host_count_deduped
(count)
The number of healthy instances per Availability Zone, regardless of if the Cross-Zone Load Balancing option is enabled or not.
shown as host
aws.elb.healthy_host_count.maximum
(gauge)
Maximum number of healthy instances in each Availability Zone.
shown as host
aws.elb.healthy_host_count.minimum
(gauge)
Minimum number of healthy instances in each Availability Zone.
shown as host
aws.elb.httpcode_backend_2xx
(rate)
Number of HTTP 2XX response codes generated by registered instances.
shown as response
aws.elb.httpcode_backend_3xx
(rate)
Number of HTTP 3XX response codes generated by registered instances.
shown as response
aws.elb.httpcode_backend_4xx
(rate)
Number of HTTP 4XX response codes generated by registered instances.
shown as response
aws.elb.httpcode_backend_5xx
(rate)
Number of HTTP 5XX response codes generated by registered instances.
shown as response
aws.elb.httpcode_elb_4xx
(rate)
Number of HTTP 4XX client error codes generated by the load balancer.
shown as response
aws.elb.httpcode_elb_5_0_0
(count)
The number of HTTP 500 error codes that originate from the load balancer.
shown as response
aws.elb.httpcode_elb_5_0_2
(count)
The number of HTTP 502 error codes that originate from the load balancer.
shown as response
aws.elb.httpcode_elb_5_0_3
(count)
The number of HTTP 503 error codes that originate from the load balancer.
shown as response
aws.elb.httpcode_elb_5_0_4
(count)
The number of HTTP 504 error codes that originate from the load balancer.
shown as response
aws.elb.httpcode_elb_5xx
(rate)
Number of HTTP 5XX client error codes generated by the load balancer.
shown as response
aws.elb.httpcode_target_2xx
(count)
Number of HTTP 2XX response codes generated by registered instances.
shown as response
aws.elb.httpcode_target_3xx
(count)
Number of HTTP 3XX response codes generated by registered instances.
shown as response
aws.elb.httpcode_target_4xx
(count)
Number of HTTP 4XX response codes generated by registered instances.
shown as response
aws.elb.httpcode_target_5xx
(count)
Number of HTTP 5XX response codes generated by registered instances.
shown as response
aws.elb.ipv_6processed_bytes
(count)
The total number of bytes processed by the load balancer over IPv6.
shown as byte
aws.elb.ipv_6request_count
(count)
The number of IPv6 requests received by the load balancer.
shown as request
aws.elb.latency
(gauge)
Average time elapsed after the request leaves the load balancer until a response is received. (ELB v1)
shown as second
aws.elb.latency.maximum
(gauge)
Maximum time elapsed after the request leaves the load balancer until a response is received. (ELB v1)
shown as second
aws.elb.latency.minimum
(gauge)
Minimum time elapsed after the request leaves the load balancer until a response is received. (ELB v1)
shown as second
aws.elb.latency.p95
(gauge)
95th percentile of the time elapsed after the request leaves the load balancer until a response is received. (ELB v1)
shown as second
aws.elb.latency.p99
(gauge)
99th percentile of the time elapsed after the request leaves the load balancer until a response is received. (ELB v1)
shown as second
aws.elb.new_connection_count
(count)
The total number of new TCP connections established from clients to the load balancer and from the load balancer to targets.
shown as connection
aws.elb.processed_bytes
(count)
The total number of bytes processed by the load balancer over IPv4 and IPv6.
shown as byte
aws.elb.request_count
(rate)
Total number of completed requests that were received and routed to the registered instances.
shown as request
aws.elb.request_count_per_target
(count)
The average number of requests received by each target in a target group.
shown as request
aws.elb.rule_evaluations
(count)
The number of rules processed by the load balancer given a request rate averaged over an hour.
aws.elb.spillover_count
(rate)
Total number of requests that were rejected because the queue was full.
shown as request
aws.elb.spillover_count.maximum
(rate)
Maximum number of requests that were rejected because the queue was full per load balancer node.
shown as request
aws.elb.surge_queue_length
(gauge)
Maximum number of requests that are pending submission to a registered instance.
shown as request
aws.elb.target_connection_error_count
(count)
Number of connections that were not successfully established between the load balancer and the registered instances.
shown as error
aws.elb.target_response_time.average
(gauge)
Average time elapsed after the request leaves the load balancer until a response is received. (ELB v2)
shown as second
aws.elb.target_response_time.maximum
(gauge)
Maximum time elapsed after the request leaves the load balancer until a response is received. (ELB v2)
shown as second
aws.elb.target_response_time.p95
(gauge)
95th percentile of the time elapsed after the request leaves the load balancer until a response is received. (ELB v2)
shown as second
aws.elb.target_response_time.p99
(gauge)
99th percentile of the time elapsed after the request leaves the load balancer until a response is received. (ELB v2)
shown as second
aws.elb.un_healthy_host_count
(gauge)
Average number of unhealthy instances in each Availability Zone.
shown as host
aws.elb.un_healthy_host_count_deduped
(count)
The number of unhealthy instances per Availability Zone, regardless of if the Cross-Zone Load Balancing option is enabled or not.
shown as host
aws.elb.un_healthy_host_count.maximum
(gauge)
Maximum number of unhealthy instances in each Availability Zone.
shown as host
aws.elb.un_healthy_host_count.minimum
(gauge)
Minimium number of unhealthy instances in each Availability Zone.
shown as host
aws.networkelb.active_flow_count
(gauge)
The average number of active established connections from clients to targets
shown as connection
aws.networkelb.active_flow_count_tls
(count)
The total number of concurrent TLS flows (or connections) from clients to targets.
shown as connection
aws.networkelb.active_flow_count.maximum
(gauge)
The maximum number of active established connections from clients to targets
shown as connection
aws.networkelb.active_flow_count.minimum
(gauge)
The minimum number of active established connections from clients to targets
shown as connection
aws.networkelb.client_tls_negotiation_error
(count)
The total number of TLS handshakes that failed during negotiation between a client and a TLS listener.
shown as error
aws.networkelb.consumed_lcus
(count)
The number of LCUs used by the load balancer
shown as unit
aws.networkelb.healthy_host_count
(gauge)
Average number of healthy targets
shown as host
aws.networkelb.healthy_host_count.maximum
(gauge)
Maximum number of healthy targets
shown as host
aws.networkelb.healthy_host_count.minimum
(gauge)
Minimum number of healthy targets
shown as host
aws.networkelb.new_flow_count
(count)
The number of new TCP connections from clients to targets
shown as connection
aws.networkelb.new_flow_count_tls
(count)
The total number of new TLS flows (or connections) established from clients to targets in the time period.
shown as connection
aws.networkelb.processed_bytes
(count)
The number of LCUs used by the load balancer
shown as byte
aws.networkelb.processed_bytes_tls
(gauge)
The total number of bytes processed by TLS listeners.
shown as byte
aws.networkelb.target_tls_negotiation_error
(count)
The total number of TLS handshakes that failed during negotiation between a TLS listener and a target.
shown as error
aws.networkelb.tcpclient_reset_count
(count)
The number of reset (RST) packets created by a client and sent to a target
shown as packet
aws.networkelb.tcpelbreset_count
(count)
The number of reset (RST) packets created by a load balancer
shown as packet
aws.networkelb.tcptarget_reset_count
(count)
The number of reset (RST) packets created by a target and sent to a client
shown as packet
aws.networkelb.un_healthy_host_count
(gauge)
Average number of unhealthy targets
shown as host
aws.networkelb.un_healthy_host_count.maximum
(gauge)
Maximum number of unhealthy targets
shown as host
aws.networkelb.un_healthy_host_count.minimum
(gauge)
Minimum number of unhealthy targets
shown as host

Each of the metrics retrieved from AWS is assigned the same tags that appear in the AWS console, including but not limited to host name, security-groups, and more.

Events

The AWS Elastic Load Balancing integration does not include any events.

Service Checks

The AWS Elastic Load Balancing integration does not include any service checks.

Troubleshooting

Need help? Contact Datadog support.

Further Reading

Learn more about how to monitor ELB performance metrics thanks to our series of posts. We detail the key performance metrics, how to collect them, and how to use Datadog to monitor ELB.