Distributed Tracing

By connecting your serverless traces to metrics, Datadog provides a context-rich picture of your application’s performance, allowing you to better troubleshoot performance issues given the distributed nature of serverless applications.

The Datadog Python, Node.js, Ruby, Go and Java tracing libraries support distributed tracing for AWS Lambda. The easiest way to add tracing to your application is with the Datadog Lambda Library, which includes the Datadog tracing library as a dependency.

Choose your tracing library

To start using Datadog APM with your serverless application, you can choose between installing Datadog’s tracing libraries (dd-trace) or AWS X-Ray tracing libraries based on your Lambda runtime and individual requirements. To see all of your traces in Datadog in real-time in the Live Search view, you need to use Datadog’s tracing libraries.

Runtime recommendations

Python and Node.js

The Datadog Lambda Library and tracing libraries for Python and Node.js support:

• Automatic correlation of Lambda logs and traces with trace ID and tag injection.
• Installation without any code changes using Serverless Framework, AWS SAM and AWS CDK integrations.
• Tracing HTTP requests invoking downstream Lambda functions or containers.
• Tracing consecutive Lambda invocations made via the AWS SDK.
• Tracing asynchronous Lambda invocations through AWS SQS.
• Tracing dozens of additional out-of-the-box Python and Node.js libraries.
• Tracing non-HTTP requests made through SNS, Kinesis, EventBridge, MQTT and more (requires additional instrumention outlined here).

For Python and Node.js serverless applications, Datadog recommends you install Datadog’s tracing libraries. If your application requires AWS X-Ray active tracing in AWS managed services such as API Gateway or Step Functions, we recommend you augment AWS X-Ray traces with Datadog APM by configuring both AWS X-Ray and Datadog APM tracing libraries as described here.

If you are already tracing your serverless functions with X-Ray and want to continue using X-Ray, you can install our AWS X-Ray integration.

Looking to trace through serverless resources not listed above? Open a feature request here.

Ruby

The Datadog Lambda Library and tracing libraries for Ruby support:

• Automatic correlation of Lambda logs and traces with trace ID and tag injection.
• Tracing HTTP requests invoking downstream Lambda functions or containers.
• Tracing dozens of additional out-of-the-box Ruby libraries.

You can trace your serverless functions in Datadog with Datadog’s tracing libraries or by installing our AWS X-Ray integration. If you are using Datadog’s tracing libraries, and need to connect Lambda function traces across AWS managed services, we recommend you augment your traces by configuring both AWS X-Ray and Datadog APM tracing libraries as described here.

Looking to trace through serverless resources not listed above? Open a feature request here.

Go

The Datadog Lambda Library and tracing libraries for Go support:

• Manual correlation of Lambda logs and traces with trace ID and tag injection.
• Tracing HTTP requests invoking downstream Lambda functions or containers.
• Tracing dozens of additional out-of-the-box Go libraries.

For Go serverless applications, Datadog recommends installing Datadog’s tracing libraries. If your application requires AWS X-Ray active tracing in AWS managed services such as API Gateway or Step Functions, you may want to consider instead using Datadog APM with AWS X-Ray tracing.

Looking to trace through serverless resources not listed above? Open a feature request here.

Java

The Datadog Lambda Library and tracing libraries for Java support:

• Correlation of Lambda logs and traces with trace ID and tag injection. See Connecting Java logs and traces for more details.
• Tracing HTTP requests invoking downstream Lambda functions or containers.
• Tracing dozens of additional out-of-the-box Java libraries.

For Java serverless applications, Datadog recommends installing Datadog’s tracing libraries. If your application requires AWS X-Ray active tracing in AWS managed services such as API Gateway or Step Functions, you may want to consider instead using Datadog APM with AWS X-Ray tracing.

Have feedback on the Datadog’s tracing libraries for Java Lambda functions? Make sure to check out discussions going on in the #serverless channel in the Datadog Slack community.

.NET

Datadog recommends you configure tracing for your .NET AWS Lambda serverless applications by installing our AWS X-Ray integration.

Learn more about tracing through .NET Azure serverless applications here.

Looking to trace through .NET AWS Lambda serverless applications with Datadog tracing libraries? Open a feature request here.

Hybrid environments

If you have installed Datadog’s tracing libraries (dd-trace) on both your Lambda functions and hosts, your traces will automatically show you the complete picture of requests that cross infrastructure boundaries, whether it be AWS Lambda, containers, on-prem hosts, or managed services.

If dd-trace is installed on your hosts with the Datadog Agent, and your serverless functions are traced with AWS X-Ray, trace merging is required to see a single, connected trace across your infrastructure. Learn more about merging traces from dd-trace and AWS X-Ray here.

Datadog’s AWS X-Ray integration only provides traces for Lambda functions. Learn more about tracing in container or host-based environments here.

Enable Datadog APM

The Datadog Python, Node.js, Ruby, Go and Java tracing libraries support distributed tracing for AWS Lambda. To enable tracing on your functions, follow the installation instructions.

To enable Datadog APM without enabling logging for your functions, ensure the DdForwarderLog environment variable is set to false on your Datadog Forwarder.

Further Reading

Additional helpful documentation, links, and articles:

Explore Datadog APMDOCUMENTATION Live SearchDOCUMENTATION Real-time distributed tracing for Go and Java Lambda FunctionsBLOG