Python Custom Instrumentation using OpenTelemetry API

Overview

Datadog tracing libraries provide an implementation of the OpenTelemetry API for instrumenting your code. This means you can maintain vendor-neutral instrumentation of all your services, while still taking advantage of Datadog’s native implementation, features, and products. You can configure it to generate Datadog-style spans and traces to be processed by the Datadog tracing library for your language, and send those to Datadog.

By instrumenting your code with OpenTelemetry API:

  • Your code remains free of vendor-specific API calls.
  • Your code does not depend on Datadog tracing libraries at compile time (only runtime).
  • Your code does not use the deprecated OpenTracing API.

Replace the OpenTelemetry SDK with the Datadog tracing library in the instrumented application, and the traces produced by your running code can be processed, analyzed, and monitored alongside Datadog traces and in Datadog proprietary products.

For more information, read Interoperability of OpenTelemetry API and Datadog instrumented traces.

The Datadog tracing library, when configured as described here, accepts the spans and traces generated by OpenTelemetry-instrumented code, processes the telemetry, and sends it to Datadog. You can use this approach, for example, if your code has already been instrumented with the OpenTelemetry API, or if you want to instrument using the OpenTelemetry API, and you want to gain the benefits of using the Datadog tracing libraries without changing your code.

If you’re looking for a way to instrument your code with OpenTelemetry and then send span data to Datadog without going through the Datadog tracing library, see OpenTelemetry in Datadog.

Requirements and limitations

  • Datadog python tracing library dd-trace-py version 1.12.0 or greater.
  • Python version 3.7 or greater.

The following OpenTelemetry features implemented in the Datadog library as noted:

FeatureSupport notes
OpenTelemetry Context propagationW3C Trace Context and Datadog header formats are enabled by default.
Span processorsUnsupported
Span ExportersUnsupported
Trace/span ID generatorsID generation is performed by the tracing library, with support for 128-bit trace IDs.

Configuring OpenTelemetry to use the Datadog Tracer Provider

  1. Add your desired manual OpenTelemetry instrumentation to your Python code following the OpenTelemetry Python Manual Instrumentation documentation. Important! Where those instructions indicate that your code should call the OpenTelemetry SDK, call the Datadog tracing library instead.

  2. Install the python tracer:

    pip install "ddtrace>=1.12.0"

  3. Set DD_TRACE_OTEL_ENABLED environment variable to True.

  4. Run your application with ddtrace-run. This automatically configures the Datadog Tracer Provider. If your application cannot use ddtrace-run read the dd-trace-py OpenTelemetry API docs for additional configurations.

Datadog combines these OpenTelemetry spans with other Datadog APM spans into a single trace of your application. It supports OpenTelemetry Automatic instrumentation also.

Further Reading

Additional helpful documentation, links, and articles:

Explore your services, resources, and tracesDOCUMENTATION more more Interoperability of OpenTelemetry API and Datadog instrumented tracesDOCUMENTATION more more