.NET Custom Instrumentation

This page details common use cases for adding and customizing observability with Datadog APM. For a list of supported runtimes, see the .NET Framework Compatibility Requirements or the .NET Core Compatibility Requirements.

There are several ways to get more than the default automatic instrumentation:

  • Through configuration, which depends on automatic instrumentation and does not allow you to add specific tags.
  • Using attributes, which depends on automatic instrumentation and allows you to customize operation and resource names.
  • Using custom code, which can work alongside automatic instrumentation or without it. It gives you the most control on the spans.

You can combine these solutions with each other to achieve the instrumentation detail you want.

Instrument methods through configuration

Note: This feature requires enabling automatic instrumentation for your application. For instructions on how to setup the .NET Tracer and enable automatic instrumentation, see the .NET Framework setup instructions or the .NET Core setup instructions.

Using the DD_TRACE_METHODS environment variable, you can get visibility into unsupported frameworks without changing application code. For full details on the input format for DD_TRACE_METHODS, see the .NET Framework configuration instructions or the .NET Core configuration instructions. For example, to instrument a method called SaveSession defined on the Store.Managers.SessionManager type, set:

DD_TRACE_METHODS=Store.Managers.SessionManager[SaveSession]

The resulting span has an operationName attribute with the value trace.annotation and a resourceName attribute with the value SaveSession.

If you want to customize the span’s attributes and you have the ability to modify the source code, you can instrument methods through attributes instead.

Instrument methods through attributes

Note: This feature requires adding the Datadog.Trace.Annotations NuGet package to your application and setting up automatic instrumentation. For instructions on how to setup the .NET Tracer and enable automatic instrumentation, see the .NET Framework setup instructions or the .NET Core setup instructions.

Add [Trace] to methods for Datadog to trace them when running with automatic instrumentation. If automatic instrumentation is not enabled, this attribute has no effect on your application.

[Trace] attributes have the default operation name trace.annotation and resource name of the traced method. You can set operation name and resource name as named arguments of the [Trace] attribute to better reflect what is being instrumented. Operation name and resource name are the only possible arguments that can be set for the [Trace] attribute. For example:

using Datadog.Trace.Annotations;

namespace Store.Managers
{
    public class SessionManager
    {
        [Trace(OperationName = "database.persist", ResourceName = "SessionManager.SaveSession")]
        public static void SaveSession()
        {
            // your method implementation here
        }
    }
}

Custom instrumentation with code

Note: This feature requires adding the Datadog.Trace NuGet package. to your application. It provides an API to directly access the Tracer and the active span.
Note: When using both the Datadog.Trace NuGet package and automatic instrumentation, it is important to keep the versions in sync.

Configuring Datadog in code

There are multiple ways to configure your application: using environment variables, a web.config file, or a datadog.json file, as described in our documentation. The Datadog.Trace NuGet package also allows you to configure settings in code.

To override configuration settings, create an instance of TracerSettings, and pass it to the static Tracer.Configure() method:

using Datadog.Trace;

// Create a settings object using the existing
// environment variables and config sources
var settings = TracerSettings.FromDefaultSources();

// Override a value
settings.GlobalTags.Add("SomeKey", "SomeValue");

// Replace the tracer configuration
Tracer.Configure(settings);

Calling Tracer.Configure() replaces the settings for all subsequent traces, both for custom instrumentation and for automatic instrumentation.

Replacing the configuration should be done once, as early as possible in your application.

Create custom traces/spans

In addition to automatic instrumentation, the [Trace] attribute, and DD_TRACE_METHODS configurations, you can customize your observability by programmatically creating spans around any block of code.

To create and activate a custom span, use Tracer.Instance.StartActive(). If a trace is already active (when created by automatic instrumentation, for example), the span is part of the current trace. If there is no current trace, a new one is started.

Warning: Ensure you dispose of the scope returned from StartActive. Disposing the scope closes the span, and ensures the trace is flushed to Datadog once all its spans are closed.
using Datadog.Trace;

// Start a new span
using (var scope = Tracer.Instance.StartActive("custom-operation"))
{
    // Do something
}

Add custom span tags to your spans to customize your observability within Datadog. The span tags are applied to your incoming traces, allowing you to correlate observed behavior with code-level information such as merchant tier, checkout amount, or user ID.

Manually creating a new span

Manually created spans automatically integrate with spans from other tracing mechanisms. In other words, if a trace has already started, the manual span has its caller as its parent span. Similarly, any traced methods called from the wrapped block of code have the manual span as its parent.

using (var parentScope =
       Tracer.Instance.StartActive("manual.sortorders"))
{
    parentScope.Span.ResourceName = "<RESOURCE NAME>";
    using (var childScope =
           Tracer.Instance.StartActive("manual.sortorders.child"))
    {
        // Nest using statements around the code to trace
        childScope.Span.ResourceName = "<RESOURCE NAME>";
        SortOrders();
    }
}

Add custom span tags

Add custom tags to your spans corresponding to any dynamic value within your application code such as customer.id.

using Datadog.Trace;

public class ShoppingCartController : Controller
{
    private IShoppingCartRepository _shoppingCartRepository;

    [HttpGet]
    public IActionResult Index(int customerId)
    {
        // Access the active scope through the global tracer
        // Note: This can return null if there is no active span
        var scope = Tracer.Instance.ActiveScope;

        if (scope != null)
        {
            // Add a tag to the span for use in the Datadog web UI
            scope.Span.SetTag("customer.id", customerId.ToString());
        }

        var cart = _shoppingCartRepository.Get(customerId);

        return View(cart);
    }
}

Set errors on a span

To mark errors that occur in your code, use the Span.SetException(Exception) method. The method marks the span as an error and adds related span metadata to provide insight into the exception.

try
{
    // do work that can throw an exception
}
catch(Exception e)
{
    span.SetException(e);
}

This sets the following tags on the span:

  • "error.msg":exception.Message
  • "error.stack":exception.ToString()
  • "error.type":exception.GetType().ToString()

Headers extraction and injection

The Datadog APM Tracer supports B3 and W3C headers extraction and injection for distributed tracing. For more information, see the setup documentation.

In most cases, headers extraction and injection are transparent. There are some known cases where your distributed trace can be disconnected. For instance, when reading messages from a distributed queue, some libraries may lose the span context. It also happens if you set DD_TRACE_KAFKA_CREATE_CONSUMER_SCOPE_ENABLED to false when consuming Kafka messages. In that case, you can add a custom trace using the following code:

var spanContextExtractor = new SpanContextExtractor();
var parentContext = spanContextExtractor.Extract(headers, (headers, key) => GetHeaderValues(headers, key));
var spanCreationSettings = new SpanCreationSettings() { Parent = parentContext };
using var scope = Tracer.Instance.StartActive("operation", spanCreationSettings);

Provide the GetHeaderValues method. The way this method is implemented depends on the structure that carries SpanContext.

Here are some examples:

// Confluent.Kafka
IEnumerable<string> GetHeaderValues(Headers headers, string name)
{
    if (headers.TryGetLastBytes(name, out var bytes))
    {
        try
        {
            return new[] { Encoding.UTF8.GetString(bytes) };
        }
        catch (Exception)
        {
            // ignored
        }
    }

    return Enumerable.Empty<string>();
}

// RabbitMQ
IEnumerable<string> GetHeaderValues(IDictionary<string, object> headers, string name)
{
    if (headers.TryGetValue(name, out object value) && value is byte[] bytes)
    {
        return new[] { Encoding.UTF8.GetString(bytes) };
    }

    return Enumerable.Empty<string>();
}

When using the SpanContextExtractor API to trace Kafka consumer spans, set DD_TRACE_KAFKA_CREATE_CONSUMER_SCOPE_ENABLED to false. This ensures the consumer span is correctly closed immediately after the message is consumed from the topic, and the metadata (such as partition and offset) is recorded correctly. Spans created from Kafka messages using the SpanContextExtractor API are children of the producer span, and siblings of the consumer span.

Adding tags globally to all spans

Use the DD_TAGS environment variable to set tags across all generated spans for an application. This can be useful for grouping stats for your applications, data centers, or regions within the Datadog UI. For example:

DD_TAGS=datacenter:njc,key2:value2

Resource filtering

You can exclude traces based on the resource name to remove Synthetics traffic such as health checks. For more information about security and additional configurations, see Configure the Datadog Agent or Tracer for Data Security.

Further Reading