Compatibility

For a list of supported runtimes and platforms, see .NET Framework Compatibility and .NET/.NET Core Compatiblity.

Supported test frameworks:

Configuring reporting method

To report test results to Datadog, you need to configure the Datadog .NET library:

Installing the .NET tracer CLI

Install or update the dd-trace command using one of the following ways:

• Using the .NET SDK by running the command:

  dotnet tool update -g dd-trace
  

• By downloading the appropriate version:

  • Win-x64: https://dtdg.co/dd-trace-dotnet-win-x64
  • Linux-x64: https://dtdg.co/dd-trace-dotnet-linux-x64
  • Linux-musl-x64 (Alpine): https://dtdg.co/dd-trace-dotnet-linux-musl-x64

• Or by downloading from the GitHub release page.

Instrumenting tests

To instrument your test suite, prefix your test command with dd-trace ci run, providing the name of the service or library under test as the --dd-service parameter, and the environment where tests are being run (for example, local when running tests on a developer workstation, or ci when running them on a CI provider) as the --dd-env parameter. For example:

dd-trace ci run --dd-service=my-dotnet-app -- dotnet test

dd-trace ci run --dd-service=my-dotnet-app -- VSTest.Console.exe {test_assembly}.dll

All tests are automatically instrumented.

Compatibility with Microsoft.CodeCoverage nuget package

Since Microsoft.CodeCoverage version 17.2.0 Microsoft introduced dynamic instrumentation using the .NET CLR Profiling API enabled by default only on Windows. Datadog’s automatic instrumentation relies on the .NET CLR Profiling API. This API allows only one subscriber (for example, dd-trace). The use of CodeCoverage dynamic instrumentation breaks the automatic test instrumentation.

The solution is to switch from dynamic instrumentation to static instrumentation. Modify your .runsettings file with the following configuration knobs:

<?xml version="1.0" encoding="utf-8"?>
<RunSettings>
    <DataCollectionRunSettings>
        <DataCollectors>
            <DataCollector friendlyName="Code Coverage">
              <Configuration>
                <CodeCoverage>
                  <!-- Switching to static instrumentation (dynamic instrumentation collides with dd-trace instrumentation) -->
                  <EnableStaticManagedInstrumentation>True</EnableStaticManagedInstrumentation>
                  <EnableDynamicManagedInstrumentation>False</EnableDynamicManagedInstrumentation>
                  <UseVerifiableInstrumentation>False</UseVerifiableInstrumentation>
                  <EnableStaticNativeInstrumentation>True</EnableStaticNativeInstrumentation>
                  <EnableDynamicNativeInstrumentation>False</EnableDynamicNativeInstrumentation>
                  ...
                </CodeCoverage>
              </Configuration>
            </DataCollector>
        </DataCollectors>
    </DataCollectionRunSettings>
</RunSettings>

Configuration settings

You can change the default configuration of the CLI by using command line arguments or environment variables. For a full list of configuration settings, run:

dd-trace ci run --help

The following list shows the default values for key configuration settings:

--dd-service

Name of the service or library under test.
Environment variable: DD_SERVICE
Default: The repository name
Example: my-dotnet-app

--dd-env

Name of the environment where tests are being run.
Environment variable: DD_ENV
Default: none
Examples: local, ci

--agent-url

Datadog Agent URL for trace collection in the form http://hostname:port.
Environment variable: DD_TRACE_AGENT_URL
Default: http://localhost:8126

test_session.name (only available as an environment variable)

Identifies a group of tests, such as integration-tests, unit-tests or smoke-tests.
Environment variable: DD_TEST_SESSION_NAME
Default: (CI job name + test command)
Example: unit-tests, integration-tests, smoke-tests

For more information about service and env reserved tags, see Unified Service Tagging. All other Datadog Tracer configuration options can also be used.

Adding custom tags to tests

To add custom tags to tests, configure custom instrumentation first.

You can add custom tags to your tests by using the current active span:

// inside your test
var scope = Tracer.Instance.ActiveScope; // from Datadog.Trace;
if (scope != null) {
    scope.Span.SetTag("test_owner", "my_team");
}
// test continues normally
// ...

To create filters or group by fields for these tags, you must first create facets. For more information about adding tags, see the Adding Tags section of the .NET custom instrumentation documentation.

Adding custom measures to tests

To add custom measures to tests, configure custom instrumentation first.

Just like tags, you can add custom measures to your tests by using the current active span:

// inside your test
var scope = Tracer.Instance.ActiveScope; // from Datadog.Trace;
if (scope != null) {
    scope.Span.SetTag("memory_allocations", 16);
}
// test continues normally
// ...

To create filters or visualizations for these tags, you must first create facets. For more information about adding tags, see the Adding Tags section of the .NET custom instrumentation documentation.

Read more about custom Measures in the Add Custom Measures Guide.

Reporting code coverage

When code coverage is available, the Datadog Tracer (v2.31.0 or later) reports it under the test.code_coverage.lines_pct tag for your test sessions.

If you are using Coverlet to compute your code coverage, indicate the path to the report file in the DD_CIVISIBILITY_EXTERNAL_CODE_COVERAGE_PATH environment variable when running dd-trace. The report file must be in the OpenCover or Cobertura formats. Alternatively, you can enable the Datadog Tracer’s built-in code coverage calculation with the DD_CIVISIBILITY_CODE_COVERAGE_ENABLED=true environment variable.

Note: When using Test Impact Analysis, the tracer’s built-in code coverage is enabled by default.

You can see the evolution of the test coverage in the Coverage tab of a test session.

For more information about exclusion options, see Code Coverage.

Instrumenting BenchmarkDotNet tests

To instrument your benchmark tests, you need to:

1. Add the Datadog.Trace.BenchmarkDotNet NuGet package to your project (for example, using dotnet add package Datadog.Trace.BenchmarkDotNet).
2. Configure your project to use the Datadog.Trace.BenchmarkDotNet exporter using the DatadogDiagnoser attribute or the WithDatadog() extension method. For example:

using BenchmarkDotNet.Attributes;
using Datadog.Trace.BenchmarkDotNet;

[DatadogDiagnoser]
[MemoryDiagnoser]
public class OperationBenchmark
{
    [Benchmark]
    public void Operation()
    {
        // ...
    }
}

using BenchmarkDotNet.Configs;
using BenchmarkDotNet.Running;
using Datadog.Trace.BenchmarkDotNet;

var config = DefaultConfig.Instance
              .WithDatadog();

BenchmarkRunner.Run<OperationBenchmark>(config);

3. Configure the reporting method.
4. Run the benchmark project as you normally do, all benchmark tests will be automatically instrumented.

Datadog은 Git 정보를 사용하여 테스트 결과를 시각화하고 리포지토리, 브랜치, 커밋별로 그룹화합니다. Git 메타데이터는 CI 공급자 환경 변수와 프로젝트 경로의 로컬 .git 폴더(사용 가능한 경우)에서 테스트 계측으로 자동 수집합니다.

지원되지 않는 CI 공급자이거나 .git 폴더가 없는 상태에서 테스트를 실행하는 경우, 환경 변수를 사용하여 Git 정보를 수동으로 설정할 수 있습니다. 해당 환경 변수는 자동 탐지된 정보보다 우선합니다. 다음 환경 변수를 설정하여 Git 정보를 제공합니다.

DD_GIT_REPOSITORY_URL

코드가 저장된 리포지토리 URL입니다. HTTP, SSH URL이 모두 지원됩니다.
예시: git@github.com:MyCompany/MyApp.git, https://github.com/MyCompany/MyApp.git

DD_GIT_BRANCH

테스트 중인 Git 브랜치입니다. 대신 태그 정보를 제공하는 경우 비워 둡니다.
예시: develop

DD_GIT_TAG

테스트 중인 Git 태그입니다(해당되는 경우). 대신 브랜치 정보를 제공하는 경우 비워 둡니다.
예시: 1.0.1

DD_GIT_COMMIT_SHA

전체 커밋 해시입니다.
예시: a18ebf361cc831f5535e58ec4fae04ffd98d8152

DD_GIT_COMMIT_MESSAGE

커밋 메시지입니다.
예시: Set release number

DD_GIT_COMMIT_AUTHOR_NAME

커밋 작성자 이름입니다.
예시: John Smith

DD_GIT_COMMIT_AUTHOR_EMAIL

커밋 작성자 이메일입니다.
예시: john@example.com

DD_GIT_COMMIT_AUTHOR_DATE

ISO 8601 형식의 커밋 작성자 날짜입니다.
예시: 2021-03-12T16:00:28Z

DD_GIT_COMMIT_COMMITTER_NAME

커밋 커미터 이름입니다.
예시: Jane Smith

DD_GIT_COMMIT_COMMITTER_EMAIL

커밋 커미터 이메일입니다.
예시: jane@example.com

DD_GIT_COMMIT_COMMITTER_DATE

ISO 8601 형식의 커밋 커미터 날짜입니다.
예시: 2021-03-12T16:00:28Z

Custom instrumentation

To use the custom instrumentation in your .NET application:

1. Execute dd-trace --version to get the version of the tool.
2. Add the Datadog.Trace NuGet package with the same version to your application.
3. In your application code, access the global tracer through the Datadog.Trace.Tracer.Instance property to create new spans.

For more information about how to add spans and tags for custom instrumentation, see the .NET Custom Instrumentation documentation.

Manual testing API

If you use XUnit, NUnit, or MSTest with your .NET projects, Test Optimization automatically instruments them and sends the test results to Datadog. If you use an unsupported testing framework or if you have a different testing mechanism, you can instead use the API to report test results to Datadog.

The API is based around three concepts: test module, test suites, and tests.

Test module

A test module represents the .NET assembly that includes the tests.

To start a test module, call TestModule.Create() and pass the name of the module or .NET assembly name where the tests are located.

When all your tests have finished, call module.Close() or module.CloseAsync(), which forces the library to send all remaining test results to the backend.

Test suites

A test suite comprises a set of tests. They can have a common initialization and teardown methods and share some variables. In .NET, they are usually implemented as a Test class or fixture containing multiple test methods. A test suite can optionally have additional information like attributes or error information.

Create test suites in the test module by calling module.GetOrCreateSuite() and passing the name of the test suite.

Call suite.Close() when all the related tests in the suite have finished their execution.

Tests

Each test runs inside a suite and must end in one of these three statuses: TestStatus.Pass, TestStatus.Fail, or TestStatus.Skip.

A test can optionally have additional information like:

• Parameters
• Attributes
• Error information
• Test traits
• Benchmark data

Create tests in a suite by calling suite.CreateTest() and passing the name of the test. When a test ends, call test.Close() with one of the predefined statuses.

Code example

The following code represents a simple usage of the API:

using System.Reflection;
using Datadog.Trace.Ci;

var module = TestModule.Create(Assembly.GetExecutingAssembly().GetName().Name ?? "(dyn_module)");
module.SetTag("ModuleTag", "Value");

var suite = module.GetOrCreateSuite("MySuite");
suite.SetTag("SuiteTag", 42);

var test = suite.CreateTest("Test01");
test.SetTag("TestTag", "Value");
test.SetParameters(new TestParameters
{
    Arguments = new Dictionary<string, object>
    {
        ["a"] = 42,
        ["b"] = 0,
    }
});
test.SetTraits(new Dictionary<string, List<string>>
{
    ["Category"] = new () { "UnitTest" }
});

try
{
    var a = 42;
    var b = 0;
    var c = a / b;
}
catch (Exception ex)
{
    test.SetErrorInfo(ex);
}

test.Close(TestStatus.Fail);
suite.Close();
await module.CloseAsync();

Always call module.Close() or module.CloseAsync() at the end so that all the test data is flushed to Datadog.

Best practices

Test session name DD_TEST_SESSION_NAME

Use DD_TEST_SESSION_NAME to define the name of the test session and the related group of tests. Examples of values for this tag would be:

• unit-tests
• integration-tests
• smoke-tests
• flaky-tests
• ui-tests
• backend-tests

If DD_TEST_SESSION_NAME is not specified, the default value used is a combination of the:

• CI job name
• Command used to run the tests (such as yarn test)

The test session name needs to be unique within a repository to help you distinguish different groups of tests.

When to use DD_TEST_SESSION_NAME

There’s a set of parameters that Datadog checks to establish correspondence between test sessions. The test command used to execute the tests is one of them. If the test command contains a string that changes for every execution, such as a temporary folder, Datadog considers the sessions to be unrelated to each other. For example:

• dotnet test --temp-dir=/var/folders/t1/rs2htfh55mz9px2j4prmpg_c0000gq/T

Datadog recommends using DD_TEST_SESSION_NAME if your test commands vary between executions.

Further reading

추가 유용한 문서, 링크 및 기사:

Forwarding Environment Variables for Tests in ContainersDOCUMENTATION Explore Test Results and PerformanceDOCUMENTATION Speed up your test jobs with Test Impact AnalysisDOCUMENTATION Troubleshooting Test OptimizationDOCUMENTATION