Compatibility

For a list of supported runtimes and platforms, see .NET Framework Compatibility and .NET/.NET Core Compatiblity.

Supported test frameworks:

Configuring reporting method

To report test results to Datadog, you need to configure the Datadog .NET library:

Installing the .NET tracer CLI

Install or update the dd-trace command using one of the following ways:

• Using the .NET SDK by running the command:

  dotnet tool update -g dd-trace
  

• By downloading the appropriate version:

  • Win-x64: https://dtdg.co/dd-trace-dotnet-win-x64
  • Linux-x64: https://dtdg.co/dd-trace-dotnet-linux-x64
  • Linux-musl-x64 (Alpine): https://dtdg.co/dd-trace-dotnet-linux-musl-x64

• Or by downloading from the GitHub release page.

Instrumenting tests

To instrument your test suite, prefix your test command with dd-trace ci run, providing the name of the service or library under test as the --dd-service parameter, and the environment where tests are being run (for example, local when running tests on a developer workstation, or ci when running them on a CI provider) as the --dd-env parameter. For example:

dd-trace ci run --dd-service=my-dotnet-app -- dotnet test

dd-trace ci run --dd-service=my-dotnet-app -- VSTest.Console.exe {test_assembly}.dll

All tests are automatically instrumented.

Compatibility with Microsoft.CodeCoverage nuget package

Since Microsoft.CodeCoverage version 17.2.0 Microsoft introduced dynamic instrumentation using the .NET CLR Profiling API enabled by default only on Windows. Datadog’s automatic instrumentation relies on the .NET CLR Profiling API. This API allows only one subscriber (for example, dd-trace). The use of CodeCoverage dynamic instrumentation breaks the automatic test instrumentation.

The solution is to switch from dynamic instrumentation to static instrumentation. Modify your .runsettings file with the following configuration knobs:

<?xml version="1.0" encoding="utf-8"?>
<RunSettings>
    <DataCollectionRunSettings>
        <DataCollectors>
            <DataCollector friendlyName="Code Coverage">
              <Configuration>
                <CodeCoverage>
                  <!-- Switching to static instrumentation (dynamic instrumentation collides with dd-trace instrumentation) -->
                  <EnableStaticManagedInstrumentation>True</EnableStaticManagedInstrumentation>
                  <EnableDynamicManagedInstrumentation>False</EnableDynamicManagedInstrumentation>
                  <UseVerifiableInstrumentation>False</UseVerifiableInstrumentation>
                  <EnableStaticNativeInstrumentation>True</EnableStaticNativeInstrumentation>
                  <EnableDynamicNativeInstrumentation>False</EnableDynamicNativeInstrumentation>
                  ...
                </CodeCoverage>
              </Configuration>
            </DataCollector>
        </DataCollectors>
    </DataCollectionRunSettings>
</RunSettings>

Configuration settings

You can change the default configuration of the CLI by using command line arguments or environment variables. For a full list of configuration settings, run:

dd-trace ci run --help

The following list shows the default values for key configuration settings:

--dd-service

Name of the service or library under test.
Environment variable: DD_SERVICE
Default: The repository name
Example: my-dotnet-app

--dd-env

Name of the environment where tests are being run.
Environment variable: DD_ENV
Default: none
Examples: local, ci

--agent-url

Datadog Agent URL for trace collection in the form http://hostname:port.
Environment variable: DD_TRACE_AGENT_URL
Default: http://localhost:8126

test_session.name (only available as an environment variable)

Identifies a group of tests, such as integration-tests, unit-tests or smoke-tests.
Environment variable: DD_TEST_SESSION_NAME
Default: (CI job name + test command)
Example: unit-tests, integration-tests, smoke-tests

For more information about service and env reserved tags, see Unified Service Tagging. All other Datadog Tracer configuration options can also be used.

Adding custom tags to tests

To add custom tags to tests, configure custom instrumentation first.

You can add custom tags to your tests by using the current active span:

// inside your test
var scope = Tracer.Instance.ActiveScope; // from Datadog.Trace;
if (scope != null) {
    scope.Span.SetTag("test_owner", "my_team");
}
// test continues normally
// ...

To create filters or group by fields for these tags, you must first create facets. For more information about adding tags, see the Adding Tags section of the .NET custom instrumentation documentation.

Adding custom measures to tests

To add custom measures to tests, configure custom instrumentation first.

Just like tags, you can add custom measures to your tests by using the current active span:

// inside your test
var scope = Tracer.Instance.ActiveScope; // from Datadog.Trace;
if (scope != null) {
    scope.Span.SetTag("memory_allocations", 16);
}
// test continues normally
// ...

To create filters or visualizations for these tags, you must first create facets. For more information about adding tags, see the Adding Tags section of the .NET custom instrumentation documentation.

Read more about custom Measures in the Add Custom Measures Guide.

Reporting code coverage

When code coverage is available, the Datadog Tracer (v2.31.0 or later) reports it under the test.code_coverage.lines_pct tag for your test sessions.

If you are using Coverlet to compute your code coverage, indicate the path to the report file in the DD_CIVISIBILITY_EXTERNAL_CODE_COVERAGE_PATH environment variable when running dd-trace. The report file must be in the OpenCover or Cobertura formats. Alternatively, you can enable the Datadog Tracer’s built-in code coverage calculation with the DD_CIVISIBILITY_CODE_COVERAGE_ENABLED=true environment variable.

Note: When using Test Impact Analysis, the tracer’s built-in code coverage is enabled by default.

You can see the evolution of the test coverage in the Coverage tab of a test session.

For more information about exclusion options, see Code Coverage.

Instrumenting BenchmarkDotNet tests

To instrument your benchmark tests, you need to:

1. Add the Datadog.Trace.BenchmarkDotNet NuGet package to your project (for example, using dotnet add package Datadog.Trace.BenchmarkDotNet).
2. Configure your project to use the Datadog.Trace.BenchmarkDotNet exporter using the DatadogDiagnoser attribute or the WithDatadog() extension method. For example:

using BenchmarkDotNet.Attributes;
using Datadog.Trace.BenchmarkDotNet;

[DatadogDiagnoser]
[MemoryDiagnoser]
public class OperationBenchmark
{
    [Benchmark]
    public void Operation()
    {
        // ...
    }
}

using BenchmarkDotNet.Configs;
using BenchmarkDotNet.Running;
using Datadog.Trace.BenchmarkDotNet;

var config = DefaultConfig.Instance
              .WithDatadog();

BenchmarkRunner.Run<OperationBenchmark>(config);

3. Configure the reporting method.
4. Run the benchmark project as you normally do, all benchmark tests will be automatically instrumented.

Datadog は、テスト結果を可視化し、リポジトリ、ブランチ、コミットごとにグループ化するために Git の情報を使用します。Git のメタデータは、CI プロバイダーの環境変数や、プロジェクトパス内のローカルな .git フォルダがあれば、そこからテストインスツルメンテーションによって自動的に収集されます。

サポートされていない CI プロバイダーでテストを実行する場合や、.git フォルダがない場合は、環境変数を使って Git の情報を手動で設定することができます。これらの環境変数は、自動検出された情報よりも優先されます。Git の情報を提供するために、以下の環境変数を設定します。

DD_GIT_REPOSITORY_URL

コードが格納されているリポジトリの URL。HTTP と SSH の両方の URL に対応しています。
例: git@github.com:MyCompany/MyApp.git、https://github.com/MyCompany/MyApp.git

DD_GIT_BRANCH

テスト中の Git ブランチ。タグ情報を指定する場合は、空のままにしておきます。
例: develop

DD_GIT_TAG

テストされる Git タグ (該当する場合)。ブランチ情報を指定する場合は、空のままにしておきます。
例: 1.0.1

DD_GIT_COMMIT_SHA

フルコミットハッシュ。
例: a18ebf361cc831f5535e58ec4fae04ffd98d8152

DD_GIT_COMMIT_MESSAGE

コミットのメッセージ。
例: Set release number

DD_GIT_COMMIT_AUTHOR_NAME

コミット作成者名。
例: John Smith

DD_GIT_COMMIT_AUTHOR_EMAIL

コミット作成者メールアドレス。
例: john@example.com

DD_GIT_COMMIT_AUTHOR_DATE

ISO 8601 形式のコミット作成者の日付。
例: 2021-03-12T16:00:28Z

DD_GIT_COMMIT_COMMITTER_NAME

コミットのコミッター名。
例: Jane Smith

DD_GIT_COMMIT_COMMITTER_EMAIL

コミットのコミッターのメールアドレス。
例: jane@example.com

DD_GIT_COMMIT_COMMITTER_DATE

ISO 8601 形式のコミットのコミッターの日付。
例: 2021-03-12T16:00:28Z

Custom instrumentation

To use the custom instrumentation in your .NET application:

1. Execute dd-trace --version to get the version of the tool.
2. Add the Datadog.Trace NuGet package with the same version to your application.
3. In your application code, access the global tracer through the Datadog.Trace.Tracer.Instance property to create new spans.

For more information about how to add spans and tags for custom instrumentation, see the .NET Custom Instrumentation documentation.

Manual testing API

If you use XUnit, NUnit, or MSTest with your .NET projects, Test Optimization automatically instruments them and sends the test results to Datadog. If you use an unsupported testing framework or if you have a different testing mechanism, you can instead use the API to report test results to Datadog.

The API is based around three concepts: test module, test suites, and tests.

Test module

A test module represents the .NET assembly that includes the tests.

To start a test module, call TestModule.Create() and pass the name of the module or .NET assembly name where the tests are located.

When all your tests have finished, call module.Close() or module.CloseAsync(), which forces the library to send all remaining test results to the backend.

Test suites

A test suite comprises a set of tests. They can have a common initialization and teardown methods and share some variables. In .NET, they are usually implemented as a Test class or fixture containing multiple test methods. A test suite can optionally have additional information like attributes or error information.

Create test suites in the test module by calling module.GetOrCreateSuite() and passing the name of the test suite.

Call suite.Close() when all the related tests in the suite have finished their execution.

Tests

Each test runs inside a suite and must end in one of these three statuses: TestStatus.Pass, TestStatus.Fail, or TestStatus.Skip.

A test can optionally have additional information like:

• Parameters
• Attributes
• Error information
• Test traits
• Benchmark data

Create tests in a suite by calling suite.CreateTest() and passing the name of the test. When a test ends, call test.Close() with one of the predefined statuses.

Code example

The following code represents a simple usage of the API:

using System.Reflection;
using Datadog.Trace.Ci;

var module = TestModule.Create(Assembly.GetExecutingAssembly().GetName().Name ?? "(dyn_module)");
module.SetTag("ModuleTag", "Value");

var suite = module.GetOrCreateSuite("MySuite");
suite.SetTag("SuiteTag", 42);

var test = suite.CreateTest("Test01");
test.SetTag("TestTag", "Value");
test.SetParameters(new TestParameters
{
    Arguments = new Dictionary<string, object>
    {
        ["a"] = 42,
        ["b"] = 0,
    }
});
test.SetTraits(new Dictionary<string, List<string>>
{
    ["Category"] = new () { "UnitTest" }
});

try
{
    var a = 42;
    var b = 0;
    var c = a / b;
}
catch (Exception ex)
{
    test.SetErrorInfo(ex);
}

test.Close(TestStatus.Fail);
suite.Close();
await module.CloseAsync();

Always call module.Close() or module.CloseAsync() at the end so that all the test data is flushed to Datadog.

Best practices

Test session name DD_TEST_SESSION_NAME

Use DD_TEST_SESSION_NAME to define the name of the test session and the related group of tests. Examples of values for this tag would be:

• unit-tests
• integration-tests
• smoke-tests
• flaky-tests
• ui-tests
• backend-tests

If DD_TEST_SESSION_NAME is not specified, the default value used is a combination of the:

• CI job name
• Command used to run the tests (such as yarn test)

The test session name needs to be unique within a repository to help you distinguish different groups of tests.

When to use DD_TEST_SESSION_NAME

There’s a set of parameters that Datadog checks to establish correspondence between test sessions. The test command used to execute the tests is one of them. If the test command contains a string that changes for every execution, such as a temporary folder, Datadog considers the sessions to be unrelated to each other. For example:

• dotnet test --temp-dir=/var/folders/t1/rs2htfh55mz9px2j4prmpg_c0000gq/T

Datadog recommends using DD_TEST_SESSION_NAME if your test commands vary between executions.

Further reading

お役に立つドキュメント、リンクや記事:

Forwarding Environment Variables for Tests in ContainersDOCUMENTATION Explore Test Results and PerformanceDOCUMENTATION Speed up your test jobs with Test Impact AnalysisDOCUMENTATION Troubleshooting Test OptimizationDOCUMENTATION