AWS Lambda

This page is limited to documentation for ingesting AWS Lambda metrics from Amazon CloudWatch. See the Datadog Serverless documentation for collecting telemetry directly from your Lambda functions in real time.

Overview

AWS Lambda is a compute service that runs code in response to events and automatically manages the compute resources required by that code.

Enable this integration to begin collecting CloudWatch metrics. This page also describes how to set up custom metrics, logging, and tracing for your Lambda functions.

Setup

Installation

If you haven’t already, set up the Amazon Web Services integration first.

Metric collection

AWS Lambda metrics

  1. In the AWS integration page, ensure that Lambda is enabled under the Metric Collection tab.

  2. Add the following permissions to your Datadog IAM policy to collect AWS Lambda metrics. For more information, see the Lambda policies on the AWS website.

    AWS PermissionDescription
    lambda:List*List Lambda functions, metadata, and tags.
    tag:GetResourcesGet custom tags applied to Lambda functions.
    cloudtrail:LookupEventsUse CloudTrail History to detect changes to lambda functions
  3. Install the Datadog - AWS Lambda integration.

Once this is completed, view all of your Lambda Functions in the Datadog Serverless view. This page brings together metrics, traces, and logs from your AWS Lambda functions running serverless applications into one view. Detailed documentation on this feature can be found in the Datadog Serverless documentation.

Data Collected

When using AWS Lambda extensions, the duration metric reported by AWS includes the post_runtime_extensions_duration consumed by Lambda extensions performing activities after the function response is returned. To monitor the actual performance of the function, use duration - post_runtime_extensions_duration or the Datadog enhanced metric aws.lambda.enhanced.runtime_duration.

Each of the metrics retrieved from AWS is assigned the same tags that appear in the AWS console, including but not limited to function name, security-groups, and more.

Metrics

aws.lambda.async_event_age
(gauge)
Measures the average age of the difference between the time that an event is first enqueued in the internal queue and the time the Lambda service invokes the function.
Shown as millisecond
aws.lambda.async_event_age.maximum
(gauge)
Measures the maximum ageof the difference between the time that an event is first enqueued in the internal queue and the time the Lambda service invokes the function.
Shown as millisecond
aws.lambda.async_event_age.minimum
(gauge)
Measures the minimum age of the difference between the time that an event is first enqueued in the internal queue and the time the Lambda service invokes the function.
Shown as millisecond
aws.lambda.async_event_age.sum
(gauge)
Measures the sum of the ages of the differences between the time that an event is first enqueued in the internal queue and the time the Lambda service invokes the function.
Shown as millisecond
aws.lambda.async_events_dropped
(count)
Measures the number of events dropped without successfully executing the function.
Shown as event
aws.lambda.async_events_received
(count)
Measures the number of events that Lambda successfully queues for processing.
Shown as event
aws.lambda.claimed_account_concurrency
(count)
Measures the amount of concurrency that is unavailable for on-demand invocations for a region.
Shown as invocation
aws.lambda.concurrent_executions
(gauge)
Measures the average of concurrent executions for a given function at a given point in time.
Shown as execution
aws.lambda.concurrent_executions.maximum
(gauge)
Measures the maximum of concurrent executions for a given function at a given point in time.
Shown as execution
aws.lambda.concurrent_executions.minimum
(gauge)
Measures the minimum of concurrent executions for a given function at a given point in time.
Shown as execution
aws.lambda.concurrent_executions.sum
(gauge)
Measures the sum of concurrent executions for a given function at a given point in time.
Shown as execution
aws.lambda.concurrent_executions_global
(gauge)
Measures the average of concurrent executions for all functions in an account at a given point in time.
Shown as execution
aws.lambda.concurrent_executions_global.maximum
(gauge)
Measures the maximum of concurrent executions for all functions in an account at a given point in time.
Shown as execution
aws.lambda.concurrent_executions_global.minimum
(gauge)
Measures the minimum of concurrent executions for all functions in an account at a given point in time.
Shown as execution
aws.lambda.concurrent_executions_global.sum
(gauge)
Measures the sum of concurrent executions for all functions in an account at a given point in time.
Shown as execution
aws.lambda.dead_letter_errors
(count)
Measures the sum of times Lambda is unable to write the failed event payload to your configured Dead Letter Queues.
Shown as error
aws.lambda.deleted_event_count
(count)
Count of events deleted by Lambda on successful processing.
Shown as event
aws.lambda.destination_delivery_failures
(count)
Measures the number of times that Lambda attempts to send an event to a destination but fails.
Shown as failure
aws.lambda.dropped_event_count
(count)
Count of events dropped by Lambda due to event expiry.
Shown as event
aws.lambda.duration
(gauge)
Measures the average elapsed wall clock time from when the function code starts executing as a result of an invocation to when it stops executing - including post_runtime_extensions_duration consumed by Lambda extensions performing activities after the function response is returned.
Shown as millisecond
aws.lambda.duration.maximum
(gauge)
Measures the maximum elapsed wall clock time from when the function code starts executing as a result of an invocation to when it stops executing - including post_runtime_extensions_duration consumed by Lambda extensions performing activities after the function response is returned.
Shown as millisecond
aws.lambda.duration.minimum
(gauge)
Measures the minimum elapsed wall clock time from when the function code starts executing as a result of an invocation to when it stops executing - including post_runtime_extensions_duration consumed by Lambda extensions performing activities after the function response is returned.
Shown as millisecond
aws.lambda.duration.p50
(gauge)
Measures the p50 elapsed wall clock time from when the function code starts executing as a result of an invocation to when it stops executing - including post_runtime_extensions_duration consumed by Lambda extensions performing activities after the function response is returned.
Shown as millisecond
aws.lambda.duration.p80
(gauge)
Measures the p80 elapsed wall clock time from when the function code starts executing as a result of an invocation to when it stops executing - including post_runtime_extensions_duration consumed by Lambda extensions performing activities after the function response is returned.
Shown as millisecond
aws.lambda.duration.p95
(gauge)
Measures the p95 elapsed wall clock time from when the function code starts executing as a result of an invocation to when it stops executing - including post_runtime_extensions_duration consumed by Lambda extensions performing activities after the function response is returned.
Shown as millisecond
aws.lambda.duration.p99
(gauge)
Measures the p99 elapsed wall clock time from when the function code starts executing as a result of an invocation to when it stops executing - including post_runtime_extensions_duration consumed by Lambda extensions performing activities after the function response is returned.
Shown as millisecond
aws.lambda.duration.p99.9
(gauge)
Measures the p99.9 elapsed wall clock time from when the function code starts executing as a result of an invocation to when it stops executing - including post_runtime_extensions_duration consumed by Lambda extensions performing activities after the function response is returned.
Shown as millisecond
aws.lambda.duration.sum
(gauge)
Measures the total execution time of the lambda function executing - including post_runtime_extensions_duration consumed by Lambda extensions performing activities after the function response is returned.
Shown as millisecond
aws.lambda.enhanced.billed_duration
(gauge)
Measures the billed amount of time the function ran for.
Shown as second
aws.lambda.enhanced.cpu_max_utilization
(gauge)
Measures CPU utilization on the maximally utilized core.
Shown as percent
aws.lambda.enhanced.cpu_min_utilization
(gauge)
Measures CPU utilization on the minimally utilized core.
Shown as percent
aws.lambda.enhanced.cpu_system_time
(gauge)
Measures the amount of time the CPU spent running in kernel mode.
Shown as millisecond
aws.lambda.enhanced.cpu_total_time
(gauge)
Measures the total amount of time the CPU spent running.
Shown as millisecond
aws.lambda.enhanced.cpu_total_utilization
(gauge)
Measures the total CPU utilization of the function as a number of cores.
Shown as core
aws.lambda.enhanced.cpu_total_utilization_pct
(gauge)
Measures the total CPU utilization of the function as a percent.
Shown as percent
aws.lambda.enhanced.cpu_user_time
(gauge)
Measures the amount of time the CPU spent running in user mode.
Shown as millisecond
aws.lambda.enhanced.duration
(gauge)
Measures the elapsed seconds from when the function code starts executing as a result of an invocation to when it stops executing - including post_runtime_duration consumed by Lambda extensions performing activities after the function response is returned.
Shown as second
aws.lambda.enhanced.errors
(count)
Measures the number of invocations that failed due to errors in the function.
Shown as error
aws.lambda.enhanced.estimated_cost
(gauge)
Measures the total estimated cost of the function invocation (US dollars).
Shown as dollar
aws.lambda.enhanced.init_duration
(gauge)
Measures the initialization time (second) of a function during a cold start.
Shown as second
aws.lambda.enhanced.invocations
(count)
Measures the number of times a function is invoked in response to an event or invocation API call.
Shown as invocation
aws.lambda.enhanced.max_memory_used
(gauge)
Measures the maximum amount of memory (mb) used by the function.
Shown as mebibyte
aws.lambda.enhanced.memorysize
(gauge)
Measures the amount of allocated memory (mb) available to the function during execution.
Shown as mebibyte
aws.lambda.enhanced.num_cores
(gauge)
Measures the number of cores available.
Shown as core
aws.lambda.enhanced.out_of_memory
(count)
Measures the number of times a function runs out of memory.
Shown as error
aws.lambda.enhanced.post_runtime_duration
(gauge)
Measures the elapsed milliseconds from when the function returns the response to when the extensions finish performing activities like sending telemetry data to a preferred destination after the function’s response is returned.
Shown as millisecond
aws.lambda.enhanced.produced_bytes
(gauge)
Measures the number of bytes returned by a function.
Shown as byte
aws.lambda.enhanced.response_duration
(gauge)
Measures the elapsed time in milliseconds from when the first byte of response to the last byte of response is sent to the client.
Shown as millisecond
aws.lambda.enhanced.response_latency
(gauge)
Measures the elapsed time in milliseconds from when the invocation request is received to when the first byte of response is sent to the client.
Shown as millisecond
aws.lambda.enhanced.runtime_duration
(gauge)
Measures the elapsed milliseconds from when the function code starts to when it returns the response.
Shown as millisecond
aws.lambda.enhanced.rx_bytes
(gauge)
Measures the bytes received by the function.
Shown as byte
aws.lambda.enhanced.time_to_first_byte
(gauge)
Measures the elapsed time in milliseconds from when the function code starts to when it returns the first byte.
Shown as millisecond
aws.lambda.enhanced.timeouts
(count)
Measures the number of times a function times out.
Shown as timeout
aws.lambda.enhanced.tmp_max
(gauge)
Measures the total available space in the /tmp directory.
Shown as byte
aws.lambda.enhanced.tmp_used
(gauge)
Measures the space used in the /tmp directory.
Shown as byte
aws.lambda.enhanced.total_network
(gauge)
Measures the bytes sent and received by the function.
Shown as byte
aws.lambda.enhanced.tx_bytes
(gauge)
Measures the bytes sent by the function.
Shown as byte
aws.lambda.errors
(count)
Measures the number of invocations that failed due to errors in the function.
Shown as error
aws.lambda.failed_invoke_event_count
(count)
Count of events sent as invoke payload to target Lambda function that resulted in complete or partial failure.
Shown as event
aws.lambda.filtered_out_event_count
(count)
For event source mappings with FilterCriteria, count of events filtered out by Lambda.
Shown as event
aws.lambda.invocations
(count)
Measures the number of times a function is invoked in response to an event or invocation API call.
Shown as invocation
aws.lambda.invoked_event_count
(count)
Count of events Lambda sent as invoke payload to the target Lambda function.
Shown as event
aws.lambda.iterator_age
(gauge)
Measures the age of the last record for each batch of records processed
Shown as millisecond
aws.lambda.iterator_age.maximum
(gauge)
Measures the maximum age of the last record for each batch of records processed
Shown as millisecond
aws.lambda.iterator_age.minimum
(gauge)
Measures the minimum age of the last record for each batch of records processed
Shown as millisecond
aws.lambda.iterator_age.sum
(gauge)
Measures the sum of the ages of the last record for each batch of records processed
Shown as millisecond
aws.lambda.memorysize
(gauge)
Measures the amount of allocated memory available to the function during execution.
Shown as mebibyte
aws.lambda.on_failure_destination_delivered_event_count
(count)
For event source mappings with DestinationConfig, count of events that were successfully sent to the destination after failure.
Shown as event
aws.lambda.polled_event_count
(count)
Count of events Lambda polled from the event source successfully.
Shown as event
aws.lambda.post_runtime_extensions_duration
(gauge)
Measures the average amount of time that the runtime spends running code for extensions after the function code has completed.
Shown as millisecond
aws.lambda.post_runtime_extensions_duration.maximum
(gauge)
Measures the maximum amount of time that the runtime spends running code for extensions after the function code has completed.
Shown as millisecond
aws.lambda.post_runtime_extensions_duration.minimum
(gauge)
Measures the minimum amount of time that the runtime spends running code for extensions after the function code has completed.
Shown as millisecond
aws.lambda.post_runtime_extensions_duration.p50
(gauge)
Measures the p50 elapsed wall time that the runtime spends running code for extensions after the function code has completed.
Shown as millisecond
aws.lambda.post_runtime_extensions_duration.p99
(gauge)
Measures the p90 elapsed wall time that the runtime spends running code for extensions after the function code has completed.
Shown as millisecond
aws.lambda.post_runtime_extensions_duration.sum
(gauge)
Measures the cumulative amount of time that the runtime spends running code for extensions after the function code has completed.
Shown as millisecond
aws.lambda.provisioned_concurrency_invocations
(count)
Measures the number of invocations that are run on provisioned concurrency
Shown as invocation
aws.lambda.provisioned_concurrency_spillover_invocations
(count)
Measures the number of invocations that are run on non-provisioned concurrency when all provisioned concurrency is in use
Shown as invocation
aws.lambda.provisioned_concurrency_utilization
(gauge)
Measures the average fraction of provisioned concurrency in use for a given function at a given point in time
Shown as percent
aws.lambda.provisioned_concurrency_utilization.maximum
(gauge)
Measures the maximum fraction of provisioned concurrency in use for a given function at a given point in time
Shown as percent
aws.lambda.provisioned_concurrency_utilization.minimum
(gauge)
Measures the minimum fraction of provisioned concurrency in use for a given function at a given point in time
Shown as percent
aws.lambda.provisioned_concurrent_executions
(gauge)
Measures the average number of events that are being processed on provisioned concurrency
Shown as execution
aws.lambda.provisioned_concurrent_executions.maximum
(gauge)
Measures the maximum number of events that are being processed on provisioned concurrency
Shown as execution
aws.lambda.provisioned_concurrent_executions.minimum
(gauge)
Measures the minimum number of events that are being processed on provisioned concurrency
Shown as execution
aws.lambda.recursive_invocations_dropped
(count)
Measures the number of times that Lambda has stopped invocation of your function because it's detected that your function is part of an infinite recursive loop.
Shown as invocation
aws.lambda.signature_validation_errors
(count)
Measures the number of times a function is successfully deployed but fails a signature check
Shown as error
aws.lambda.streamed_outbound_bytes
(gauge)
Measures the number of outbound bytes returned by a streaming response function.
Shown as byte
aws.lambda.streamed_outbound_throughput
(rate)
Measures the number of bytes returned per second by a streaming response function.
Shown as byte
aws.lambda.throttles
(count)
Measures the number of Lambda function invocation attempts that were throttled due to invocation rates exceeding the customer's concurrent limits (error code 429). Failed invocations may trigger a retry attempt that succeeds.
Shown as throttle
aws.lambda.timeout
(gauge)
Measures the amount of allowed execution time for the function before the Lambda runtime stops it.
Shown as second
aws.lambda.unreserved_concurrent_executions
(gauge)
Measures the sum of the concurrency of the functions that don't have a custom concurrency limit specified.
Shown as execution
aws.lambda.url_4xx_count
(count)
The number of requests that returned a 4xx HTTP status code.
Shown as error
aws.lambda.url_5xx_count
(count)
The number of requests that returned a 5xx HTTP status code.
Shown as error
aws.lambda.url_request_count
(count)
The number of requests hit this Function URL.
Shown as request
aws.lambda.url_request_latency
(gauge)
The time between when a request was received by the Function URL and when it receives a response from the function
Shown as millisecond
aws.lambda.url_request_latency.maximum
(gauge)
The maximum time between when a request was received by the Function URL and when it receives a response from the function
Shown as millisecond

Events

The AWS Lambda integration collects Lambda deployment events from AWS CloudTrail if Datadog serverless deployment tracking is enabled.

Service Checks

The AWS Lambda integration does not include any service checks.

Real-time enhanced Lambda metrics

See the Serverless documentation to learn more.

Custom metrics

See the Serverless documentation to learn more.

Log collection

See the Serverless documentation to learn more.

Trace collection

See the Serverless documentation to learn more.

Lambda@Edge

Datadog automatically adds tags at_edge, edge_master_name, and edge_master_arn tags on your Lambda metrics to get an aggregated view of your Lambda function metrics and logs as they run in Edge locations.

Distributed tracing is not supported for Lambda@Edge functions.

Out-of-the-box monitoring

The AWS Lambda integration provides ready-to-use monitoring capabilities to monitor and optimize performance.

Troubleshooting

Need help? Contact Datadog support.

Further Reading