C++ Custom Instrumentation

If you have not yet read the setup instructions, start with the C++ Setup Instructions.

Add tags

Add custom span tags to your spans to customize your observability within Datadog. The span tags are applied to your incoming traces, allowing you to correlate observed behavior with code-level information such as merchant tier, checkout amount, or user ID.

C++ tracing uses “common tags”. These tags can be sourced from both Datadog specific tags or OpenTracing tags, and included via the below:

#include <opentracing/ext/tags.h>
#include <datadog/tags.h>

Note that the Datadog tags are necessary for unified service tagging.

Add custom span tags

Add tags directly to a span object by calling Span::SetTag. For example:

auto tracer = ...
auto span = tracer->StartSpan("operation_name");
span->SetTag("key must be string", "Values are variable types");
span->SetTag("key must be string", 1234);

Values are of variable type and can be complex objects. Values are serialized as JSON, with the exception of a string value being serialized bare (without extra quotation marks).

Adding tags globally to all spans

To set tags across all your spans, set the DD_TAGS environment variable as a list of key:value pairs separated by commas.

Set errors on a span

To associate a span with an error, set one or more error-related tags on the span. For example:

span->SetTag(opentracing::ext::error, true);

Or, alternatively:

span->SetTag("error", true);

Add more specific information about the error by setting any combination of the error.msg, error.stack, or error.type tags. See Error Tracking for more information about error tags.

An example of adding a combination of error tags:

// Associate this span with the "bad file descriptor" error from the standard
// library.
span->SetTag("error.msg", "[EBADF] invalid file");
span->SetTag("error.type", "errno");

Adding any of the error.msg, error.stack, or error.type tags sets error to the value true.

To unset an error on a span, set the error tag to value false, which removes any previously set error.msg, error.stack, or error.type tags.

// Clear any error information associated with this span.
span->SetTag("error", false);

Adding spans

Manually instrument a method

To manually instrument your code, install the tracer as in the setup examples, and then use the tracer object to create spans.

{
  // Create a root span for the current request.
  auto root_span = tracer->StartSpan("get_ingredients");
  // Set a resource name for the root span.
  root_span->SetTag(datadog::tags::resource_name, "bologna_sandwich");
  // Create a child span with the root span as its parent.
  auto child_span = tracer->StartSpan(
      "cache_lookup",
      {opentracing::ChildOf(&root_span->context())});
  // Set a resource name for the child span.
  child_span->SetTag(datadog::tags::resource_name, "ingredients.bologna_sandwich");
  // Spans can be finished at an explicit time ...
  child_span->Finish();
} // ... or implicitly when the destructor is invoked.
  // For example, root_span finishes here.

Inject and extract context for distributed tracing

Distributed tracing can be accomplished by using the Inject and Extract methods on the tracer, which accept generic Reader and Writer types. Priority sampling (enabled by default) should be on to ensure uniform delivery of spans.

// Allows writing propagation headers to a simple map<string, string>.
// Copied from https://github.com/opentracing/opentracing-cpp/blob/master/mocktracer/test/propagation_test.cpp
struct HTTPHeadersCarrier : HTTPHeadersReader, HTTPHeadersWriter {
  HTTPHeadersCarrier(std::unordered_map<std::string, std::string>& text_map_)
      : text_map(text_map_) {}

  expected<void> Set(string_view key, string_view value) const override {
    text_map[key] = value;
    return {};
  }

  expected<void> ForeachKey(
      std::function<expected<void>(string_view key, string_view value)> f)
      const override {
    for (const auto& key_value : text_map) {
      auto result = f(key_value.first, key_value.second);
      if (!result) return result;
    }
    return {};
  }

  std::unordered_map<std::string, std::string>& text_map;
};

void example() {
  auto tracer = ...
  std::unordered_map<std::string, std::string> headers;
  HTTPHeadersCarrier carrier(headers);

  auto span = tracer->StartSpan("operation_name");
  tracer->Inject(span->context(), carrier);
  // `headers` now populated with the headers needed to propagate the span.
}

Trace client and Agent configuration

There are additional configurations possible for both the tracing client and Datadog Agent for context propagation with B3 Headers, as well as to exclude specific Resources from sending traces to Datadog in the event these traces are not wanted to count in metrics calculated, such as Health Checks.

B3 headers extraction and injection

Datadog APM tracer supports B3 headers extraction and injection for distributed tracing.

Distributed headers injection and extraction is controlled by configuring injection/extraction styles. Currently two styles are supported:

Datadog: Datadog
B3: B3

Injection styles can be configured using:

Environment Variable: DD_PROPAGATION_STYLE_INJECT="Datadog B3"

The value of the environment variable is a comma (or space) separated list of header styles that are enabled for injection. By default only Datadog injection style is enabled.

Extraction styles can be configured using:

Environment Variable: DD_PROPAGATION_STYLE_EXTRACT="Datadog B3"

The value of the environment variable is a comma (or space) separated list of header styles that are enabled for extraction. By default only Datadog extraction style is enabled.

If multiple extraction styles are enabled extraction attempt is done on the order those styles are configured and first successful extracted value is used.

Resource filtering

Traces can be excluded based on their resource name, to remove synthetic traffic such as health checks from reporting traces to Datadog. This and other security and fine-tuning configurations can be found on the Security page.