Overview

There are a few reasons to manually instrument your applications with the OpenTelemetry API:

• You are not using Datadog supported library instrumentation.
• You want to extend the ddtrace library’s functionality.
• You need finer control over instrumenting your applications.

The ddtrace library provides several techniques to help you achieve these goals. The following sections demonstrate how to use the OpenTelemetry API for custom instrumentation to use with Datadog.

Imports

Import the following packages to setup the Datadog trace provider and use cases demonstrated below:

import (
	"context"
	"log"
	"os"

	"gopkg.in/DataDog/dd-trace-go.v1/ddtrace/ext"
	ddotel "gopkg.in/DataDog/dd-trace-go.v1/ddtrace/opentelemetry"
	ddtracer "gopkg.in/DataDog/dd-trace-go.v1/ddtrace/tracer"

	"go.opentelemetry.io/otel"
	"go.opentelemetry.io/otel/attribute"
)

Setup

To configure OpenTelemetry to use the Datadog trace provider:

1. Add your desired manual OpenTelemetry instrumentation to your Go code following the OpenTelemetry Go Manual Instrumentation documentation. Important! Where those instructions indicate that your code should call the OpenTelemetry SDK, call the Datadog tracing library instead.

2. Install the OpenTelemetry package go.opentelemetry.io/otel using the command:

   go get go.opentelemetry.io/otel
   

3. Install the Datadog OpenTelemetry wrapper package gopkg.in/DataDog/dd-trace-go.v1/ddtrace/opentelemetry using the command:

   go get gopkg.in/DataDog/dd-trace-go.v1/ddtrace/opentelemetry
   

4. Import packages in the code:

   import (
     "go.opentelemetry.io/otel"
     ddotel "gopkg.in/DataDog/dd-trace-go.v1/ddtrace/opentelemetry"
   )
   

5. Create a TracerProvider, optionally providing a set of options, that are specific to Datadog APM, and defer the Shutdown method, which stops the tracer:

   provider := ddotel.NewTracerProvider()
   defer provider.Shutdown()
   

6. Use the Tracer Provider instance with the OpenTelemetry API to set the global TracerProvider:

   otel.SetTracerProvider(provider)
   

7. Run your application.

Datadog combines these OpenTelemetry spans with other Datadog APM spans into a single trace of your application.

Adding span tags

Add custom tags to your spans to attach additional metadata and context to your traces.

// Can only be done after the setup steps, such as initialising the tracer.

// Start a span.
ctx, span := t.Start(ctx, "read.file")
// Set an attribute, or a tag in Datadog terminology, on a span.
span.SetAttributes(attribute.String(ext.ResourceName, "test.json"))

Adding tags globally to all spans

Add tags to all spans by configuring the tracer with the WithGlobalTag option:

// Here we can leverage the Datadog tracer options by passing them into the 
// NewTracerProvider function.
provider := ddotel.NewTracerProvider(
	ddtracer.WithGlobalTag("datacenter", "us-1"),
	ddtracer.WithGlobalTag("env", "dev"),
)
defer provider.Shutdown()

// Use it with the OpenTelemetry API to set the global TracerProvider.
otel.SetTracerProvider(provider)

// Start the Tracer with the OpenTelemetry API.
t := otel.Tracer("")

Setting errors on a span

To set an error on a span, use the setStatus method like this:

// Start a span.
ctx, span := t.Start(context.Background(), "span_name")

...
// Set an error on a span with 'span.SetAttributes'.
span.SetAttributes(attribute.String(ext.ErrorMsg, "error_message"))

// ALternatively, it is possible to set an error on a span via end span options. 
EndOptions(sp, tracer.WithError(errors.New("persisted_option")))
sp.End()

Adding spans

Unlike other Datadog tracing libraries, when tracing Go applications, Datadog recommends that you explicitly manage and pass the Go context of your spans. This approach ensures accurate span relationships and meaningful tracing. For more information, see the Go context library documentation or documentation for any third-party libraries integrated with your application.

// Can only be done after the setup steps.

// Here we can leverage context.Context to pass in Datadog-specifc start span options,
// like 'ddtracer.Measured()'
ctx, span := t.Start(
	ddotel.ContextWithStartOptions(context.Background(), ddtracer.Measured()), "span_name")

span.End()

Trace client and Agent configuration

There are additional configurations to consider for both the tracing client and Datadog Agent:

• Context propagation with B3 Headers
• Exclude specific resources from sending traces to Datadog if you do not want to include these traces in metrics calculated, such as Health Checks.

Propagating context with headers extraction and injection

You can configure the propagation of context for distributed traces by injecting and extracting headers. Read Trace Context Propagation for information.

Resource filtering

Traces can be excluded based on their resource name, to remove synthetic traffic such as health checks from reporting traces to Datadog. This and other security and fine-tuning configurations can be found on the Security page.