Datadog's Research Report: The State of Serverless Report: The State of Serverless


OpenTracing is a vendor-neutral, cross-language standard for tracing applications. Datadog offers OpenTracing implementations for many APM tracers. For more details see

Use the OpenTracing API and the Datadog Tracer (dd-trace-ot) library to measure execution times for specific pieces of code. This lets you trace your application more precisely than you can with the Java Agent alone.


For Maven, add this to pom.xml:

<!-- OpenTracing API -->

<!-- OpenTracing Util -->

<!-- Datadog Tracer (only needed if you do not use dd-java-agent) -->

For Gradle, add:

compile group: 'io.opentracing', name: 'opentracing-api', version: "0.31.0"
compile group: 'io.opentracing', name: 'opentracing-util', version: "0.31.0"
compile group: 'com.datadoghq', name: 'dd-trace-ot', version: "${dd-trace-java.version}"

Configure your application using environment variables or system properties as discussed in the configuration section.

Manual instrumentation with OpenTracing

Use a combination of these if the automatic instrumentation isn’t providing you enough depth or detail.

Using try-finally:

import datadog.trace.api.DDTags;

import io.opentracing.Scope;
import io.opentracing.Tracer;
import io.opentracing.util.GlobalTracer;

class InstrumentedClass {

    void method0() {
         * 1. Configure your application using environment variables or system properties
         * 2. Using dd-java-agent (-javaagent:/path/to/dd-java-agent.jar),
         *    GlobalTracer is automatically instantiated.
        Tracer tracer = GlobalTracer.get();

        Scope scope = tracer.buildSpan("<OPERATION_NAME>").startActive(true);
        try {
            scope.span().setTag(DDTags.SERVICE_NAME, "<SERVICE_NAME>");

            // The code you're tracing

        // If you don't call close(), the span data will NOT make it to Datadog!
        } finally {

Alternatively, wrap the code you want to trace in a try-with-resources statement:

import datadog.trace.api.DDTags;

import io.opentracing.Scope;
import io.opentracing.Tracer;
import io.opentracing.util.GlobalTracer;

class InstrumentedClass {

    void method0() {
        Tracer tracer = GlobalTracer.get();

        try (Scope scope = tracer.buildSpan("<OPERATION_NAME>").startActive(true)) {
            scope.span().setTag(DDTags.SERVICE_NAME, "<SERVICE_NAME>");

In this case, you don’t need to call scope.close().

If you’re not using dd-java-agent.jar, you must register a configured tracer with GlobalTracer. For this, call GlobalTracer.register(new DDTracer()) early on in your application startup (e.g., main method).

import datadog.opentracing.DDTracer;
import io.opentracing.Tracer;
import io.opentracing.util.GlobalTracer;
import datadog.trace.common.sampling.AllSampler;
import datadog.trace.common.writer.DDAgentWriter;

//For the API Example
import datadog.trace.common.writer.Writer;
import datadog.trace.common.sampling.Sampler;

public class Application {

    public static void main(String[] args) {

        // Initialize the tracer from environment variables or system properties
        DDTracer tracer = new DDTracer();
        // register the same tracer with the Datadog API

        // OR from the API
        Writer writer = new DDAgentWriter();
        Sampler sampler = new AllSampler();
        String service = "Service Name";
        Tracer tracer = new DDTracer(service,writer, sampler);

        // ...

Manual instrumentation for async traces

Create asynchronous traces with manual instrumentation using the OpenTracing API.

// Step 1: start the Scope/Span on the work submission thread
try (Scope scope = tracer.buildSpan("ServiceHandlerSpan").startActive(false)) {
    final Span span = scope.span();
    doAsyncWork(new Runnable() {
        public void run() {
            // Step 2: reactivate the Span in the worker thread
            try (Scope scope = tracer.scopeManager().activate(span, false)) {
              // worker thread impl...
    // submission thread impl...

Create a distributed trace using manual instrumentation with OpenTracing

// Step 1: Inject the Datadog headers in the client code
try (Scope scope = tracer.buildSpan("httpClientSpan").startActive(true)) {
    final Span span = scope.span();
    HttpRequest request = /* your code here */;

                  new MyHttpHeadersInjectAdapter(request));

    // http request impl...

public static class MyHttpHeadersInjectAdapter implements TextMap {
  private final HttpRequest httpRequest;

  public HttpHeadersInjectAdapter(final HttpRequest httpRequest) {
    this.httpRequest = httpRequest;

  public void put(final String key, final String value) {
    httpRequest.addHeader(key, value);

  public Iterator<Map.Entry<String, String>> iterator() {
    throw new UnsupportedOperationException("This class should be used only with tracer#inject()");

// Step 2: Extract the Datadog headers in the server code
HttpRequest request = /* your code here */;

final SpanContext extractedContext =
                             new MyHttpRequestExtractAdapter(request));

try (Scope scope = tracer.buildSpan("httpServerSpan").asChildOf(extractedContext).startActive(true)) {
    final Span span = scope.span(); // is a child of http client span in step 1
    // http server impl...

public class MyHttpRequestExtractAdapter implements TextMap {
  private final HttpRequest request;

  public HttpRequestExtractAdapter(final HttpRequest request) {
    this.request = request;

  public Iterator<Map.Entry<String, String>> iterator() {
    return request.headers().iterator();

  public void put(final String key, final String value) {
    throw new UnsupportedOperationException("This class should be used only with Tracer.extract()!");

Notice the above examples only use the OpenTracing classes. Check the OpenTracing API for more details and information.

Set errors

To customize an error associated to one of your spans, import the io.opentracing.Span, io.opentracing.tag.Tags, and io.opentracing.util.GlobalTracer libraries into the method where the error occurs:

import io.opentracing.Span;
import io.opentracing.tag.Tags;
import io.opentracing.util.GlobalTracer;

Then, set the error to true and add at least the error.msg, error.type, and error.stack tags to your span.

    final Span span = GlobalTracer.get().activeSpan();
    if (span != null) {
      Tags.ERROR.set(span, true);
      span.log(Collections.singletonMap(Fields.ERROR_OBJECT, ex));

Note: Any relevant error metadata explained in the Trace View docs can also be added. If the current span isn’t the root span, mark it as an error by using the dd-trace-api library to grab the root span with MutableSpan, then use setError(true). See the source code for more details.


OpenTracing support is included in the ddtrace package. Use pip to install the required opentracing package :

$ pip install ddtrace[opentracing]


The OpenTracing convention for initializing a tracer is to define an initialization method that configures and instantiates a new tracer and overwrites the global opentracing.tracer reference:

import time
import opentracing
from ddtrace.opentracer import Tracer, set_global_tracer

def init_tracer(service_name):
    config = {
      'agent_hostname': 'localhost',
      'agent_port': 8126,
    tracer = Tracer(service_name, config=config)
    return tracer

def my_operation():
  span = opentracing.tracer.start_span('<OPERATION_NAME>')
  span.set_tag('<TAG_KEY>', '<TAG_VALUE>')


For more advanced usage and configuration information see Datadog Python Opentracing API docs and the Python OpenTracing repo.

To set up Datadog with OpenTracing, see the Ruby Quickstart for OpenTracing for details.

Configuring Datadog tracer settings

The underlying Datadog tracer can be configured by passing options (which match Datadog::Tracer) when configuring the global tracer:

# Where `options` is a Hash of options provided to Datadog::Tracer
OpenTracing.global_tracer =

It can also be configured by using Datadog.configure as described in the Ruby tracer settings section.

Activating and configuring integrations

By default, configuring OpenTracing with Datadog does not automatically activate any additional instrumentation provided by Datadog. You will only receive spans and traces from OpenTracing instrumentation you have in your application.

However, additional instrumentation provided by Datadog can be activated alongside OpenTracing using Datadog.configure, which can be used to enhance your tracing further. To enable this, see Ruby integration instrumentation for more details.

Supported serialization formats

TypeSupported?Additional information
OpenTracing::FORMAT_RACKYesBecause of the loss of resolution in the Rack format, note that baggage items with names containing either upper case characters or - are be converted to lower case and _ in a round-trip, respectively. Datadog recommends avoiding these characters or accommodating accordingly on the receiving end.

Import the opentracer package to expose the Datadog tracer as an OpenTracing compatible tracer.


A basic usage example:

package main

import (


func main() {
    // Start the regular tracer and return it as an opentracing.Tracer interface. You
    // may use the same set of options as you normally would with the Datadog tracer.
    t := opentracer.New(tracer.WithServiceName("<SERVICE_NAME>"))

    // Stop it using the regular Stop call for the tracer package.
    defer tracer.Stop()

    // Set the global OpenTracing tracer.

    // Use the OpenTracing API as usual.

Note: Using the OpenTracing API in parallel with the regular API or Datadog integrations is fully supported. Under the hood, all of them make use of the same tracer. See the API documentation for more examples and details.

This library is OpenTracing compliant. Use the (OpenTracing API and the Datadog Tracer (dd-trace) library to measure execution times for specific pieces of code. In the following example, a Datadog Tracer is initialized and used as a global tracer:

// server.js

const tracer = require('dd-trace').init()
const opentracing = require('opentracing')


const app = require('./app.js')

// app.js

const tracer = opentracing.globalTracer()

The following tags are available to override Datadog specific options:

  • The service name to be used for this span. The service name from the tracer is used if this is not provided.
  • The resource name to be used for this span. The operation name is used if this is not provided.
  • span.type: The span type to be used for this span. The span type falls back to custom if not provided.

For OpenTracing support, add the Datadog.Trace.OpenTracing NuGet package to your application. During application start-up, initialize the OpenTracing library:

using Datadog.Trace.OpenTracing;

public void ConfigureServices(IServiceCollection services)
    // create an OpenTracing ITracer with default setting
    OpenTracing.ITracer tracer = OpenTracingTracerFactory.CreateTracer();

    // to use tracer with ASP.NET Core dependency injection

    // to use tracer with OpenTracing.GlobalTracer.Instance

The PHP tracer supports OpenTracing via the opentracing/opentracing library which is installed with Composer:

$ composer require opentracing/opentracing:1.0.0-beta5

When automatic instrumentation is enabled, an OpenTracing-compatible tracer is made available as the global tracer:

  $otTracer = \OpenTracing\GlobalTracer::get();
  $span = $otTracer->startActiveSpan('web.request')->getSpan();
  $span->setTag('span.type', 'web');
  $span->setTag('http.method', $_SERVER['REQUEST_METHOD']);
  // ...Use OpenTracing as expected

The Datadog C++ tracer can only be used through the OpenTracing API. The usage instructions in this document all describe generic OpenTracing functionality.

Further Reading