Troubleshoot Serverless Monitoring

Check out the additional troubleshooting guides for common issues such as the function code size is too large or webpack compatibility. This guide helps you troubleshoot general telemetry collection issues.

Understand the basics

To better understand the instructions from this guide, familiarize yourself with the key concepts first. A better understanding on how things work helps you identify the missing pieces and narrow down probable causes.

Use the Datadog Lambda Extension instead of the Forwarder

If you are still using the Datadog Forwarder Lambda function for data collection, consider adopting the Datadog Lambda Extension instead. Because the technical limitations of the Forwarder Lambda cause a number of known issues, migrating to the Lambda Extension can resolve them automatically.

  • If you are unsure whether your Lambda is using the Extension, check your Lambda function’s layer configurations and see if any Lambda layer is named Datadog-Extension.

  • If you are unsure whether your Lambda is using the Forwarder, check the subscription filters of your Lambda function’s log group, and see if the log group is subscribed by a Lambda named Datadog Forwarder or something similar.

See this comparison guide to understand the benefits of using the Extension and this migration guide for the general migration steps. Test your changes in a dev or staging environment first!

To continue using the Forwarder, see this Forwarder troubleshooting guide for more help.

Ensure the configurations are up-to-date and expected

Check out the installation guides with up-to-date instructions about applications you may have configured for Datadog monitoring in the past.

To ensure the actual changes made to your Lambda functions are expected, try setting up another testing function and configuring it following the instructions for Datadog CLI or Custom. Compare the changes (such as handler, layers, environment variables, and tags) made on your real Lambda functions against the testing one.

Collect debugging logs

Enable verbose debugging logs by setting the environment variable DD_LOG_LEVEL to debug on your Lambda functions. If you are using the Datadog Forwarder Lambda function for data forwarding from logs, also set DD_LOG_LEVEL to debug on the Forwarder Lambda function.

If you have issues with tracing, set the environment variable DD_TRACE_DEBUG to true for extra debugging logs from the Datadog Tracer.

To avoid unnecessary costs, disable the debugging logs after you collect enough data.

Check the AWS integration

Datadog can collect metrics and resource tags from AWS through an integration with AWS (optional). If you are missing certain metrics from CloudWatch and Lambda resource tags, ensure that the AWS integration is properly configured.

Ensure the tags are configured

If you have trouble applying the Datadog standard service, env, and version tags to the collected data, make sure the environment variables DD_SERVICE, DD_ENV, and DD_VERSION are configured on your Lambda functions. For custom tags, ensure DD_TAGS is configured.

If you want to enrich the data collected with your AWS Lambda resource tags, make sure the Datadog integration for AWS is properly configured.

Get help

For quick questions, post in the #serverless channel of the Datadog Slack community.

If you have followed all the troubleshooting steps above and want help from Datadog Support, include the following information in your ticket:

  1. Basic information about your Lambda function:ARN, runtime, handler, layers, environment variables, and tags. Focus on one function first if you have the same issue with many.
  2. If the Lambda function is configured to send data through logs using the Datadog Forwarder Lambda function, include basic information about the Forwarder Lambda function, as well as the subscription filters configured on your Lambda function’s log group.
  3. The installation method you followed, such as Serverless Framework or AWS CDK.
  4. The alternative installation method you attempted, such as Datadog CLI or Custom.
  5. Debugging logs from your own Lambda function.
  6. Debugging logs from the Datadog Forwarder Lambda function (if used).
  7. The project configuration files, with redacted hardcoded secrets, such as serverless.yaml, package.json, package-lock.json, yarn.lock, tsconfig.json, and webpack.config.json.

Further Reading