이 페이지는 아직 한국어로 제공되지 않으며 번역 작업 중입니다. 번역에 관한 질문이나 의견이 있으시면 언제든지 저희에게 연락해 주십시오.

CI Visibility is not available in the selected site () at this time.

Compatibility

For a list of supported runtimes and platforms, see .NET Framework Compatibility and .NET/.NET Core Compatiblity.

Supported test frameworks:

Test FrameworkVersion
xUnit>= 2.2
NUnit>= 3.0
MsTestV2>= 14
BenchmarkDotNet>= 0.13.2

Configuring reporting method

To report test results to Datadog, you need to configure the Datadog .NET library:

We support auto-instrumentation for the following CI providers:

CI ProviderAuto-Instrumentation method
GitHub ActionsDatadog Test Visibility Github Action
JenkinsUI-based configuration with Datadog Jenkins plugin
GitLabDatadog Test Visibility GitLab Script
CircleCIDatadog Test Visibility CircleCI Orb

If you are using auto-instrumentation for one of these providers, you can skip the rest of the setup steps below.

Agentless mode is available in Datadog .NET library versions >= 2.5.1
If you are using a cloud CI provider without access to the underlying worker nodes, such as GitHub Actions or CircleCI, configure the library to use the Agentless mode. For this, set the following environment variables:
DD_CIVISIBILITY_AGENTLESS_ENABLED=true (Required)
Enables or disables Agentless mode.
Default: false
DD_API_KEY (Required)
The Datadog API key used to upload the test results.
Default: (empty)

Additionally, configure the Datadog site to which you want to send data.

DD_SITE (Required)
The Datadog site to upload results to.
Default: datadoghq.com

If you are running tests on an on-premises CI provider, such as Jenkins or self-managed GitLab CI, install the Datadog Agent on each worker node by following the Agent installation instructions. This is the recommended option as it allows you to automatically link test results to logs and underlying host metrics.

If you are using a Kubernetes executor, Datadog recommends using the Datadog Operator. The operator includes Datadog Admission Controller which can automatically inject the tracer library into the build pods. Note: If you use the Datadog Operator, there is no need to download and inject the tracer library since the Admission Controller can do this for you, so you can skip the corresponding step below. However, you still need to make sure that your pods set the environment variables or command-line parameters necessary to enable Test Visibility.

If you are not using Kubernetes or can’t use the Datadog Admission Controller and the CI provider is using a container-based executor, set the DD_TRACE_AGENT_URL environment variable (which defaults to http://localhost:8126) in the build container running the tracer to an endpoint that is accessible from within that container. Note: Using localhost inside the build references the container itself and not the underlying worker node or any container where the Agent might be running in.

DD_TRACE_AGENT_URL includes the protocol and port (for example, http://localhost:8126) and takes precedence over DD_AGENT_HOST and DD_TRACE_AGENT_PORT, and is the recommended configuration parameter to configure the Datadog Agent’s URL for CI Visibility.

If you still have issues connecting to the Datadog Agent, use the Agentless Mode. Note: When using this method, tests are not correlated with logs and infrastructure metrics.

Installing the .NET tracer CLI

Install or update the dd-trace command using one of the following ways:

Instrumenting tests

Note: For BenchmarkDotNet follow these instructions.

To instrument your test suite, prefix your test command with dd-trace ci run, providing the name of the service or library under test as the --dd-service parameter, and the environment where tests are being run (for example, local when running tests on a developer workstation, or ci when running them on a CI provider) as the --dd-env parameter. For example:

By using dotnet test:

dd-trace ci run --dd-service=my-dotnet-app --dd-env=ci -- dotnet test

By using VSTest.Console.exe:

dd-trace ci run --dd-service=my-dotnet-app --dd-env=ci -- VSTest.Console.exe {test_assembly}.dll

All tests are automatically instrumented.

Compatibility with Microsoft.CodeCoverage nuget package

Since Microsoft.CodeCoverage version 17.2.0 Microsoft introduced dynamic instrumentation using the .NET CLR Profiling API enabled by default only on Windows. Datadog’s automatic instrumentation relies on the .NET CLR Profiling API. This API allows only one subscriber (for example, dd-trace). The use of CodeCoverage dynamic instrumentation breaks the automatic test instrumentation.

The solution is to switch from dynamic instrumentation to static instrumentation. Modify your .runsettings file with the following configuration knobs:

<?xml version="1.0" encoding="utf-8"?>
<RunSettings>
    <DataCollectionRunSettings>
        <DataCollectors>
            <DataCollector friendlyName="Code Coverage">
              <Configuration>
                <CodeCoverage>
                  <!-- Switching to static instrumentation (dynamic instrumentation collides with dd-trace instrumentation) -->
                  <EnableStaticManagedInstrumentation>True</EnableStaticManagedInstrumentation>
                  <EnableDynamicManagedInstrumentation>False</EnableDynamicManagedInstrumentation>
                  <UseVerifiableInstrumentation>False</UseVerifiableInstrumentation>
                  <EnableStaticNativeInstrumentation>True</EnableStaticNativeInstrumentation>
                  <EnableDynamicNativeInstrumentation>False</EnableDynamicNativeInstrumentation>
                  ...
                </CodeCoverage>
              </Configuration>
            </DataCollector>
        </DataCollectors>
    </DataCollectionRunSettings>
</RunSettings>

Configuration settings

You can change the default configuration of the CLI by using command line arguments or environment variables. For a full list of configuration settings, run:

dd-trace ci run --help

The following list shows the default values for key configuration settings:

--dd-service
Name of the service or library under test.
Environment variable: DD_SERVICE
Default: The repository name
Example: my-dotnet-app
--dd-env
Name of the environment where tests are being run.
Environment variable: DD_ENV
Default: none
Examples: local, ci
--agent-url
Datadog Agent URL for trace collection in the form http://hostname:port.
Environment variable: DD_TRACE_AGENT_URL
Default: http://localhost:8126

For more information about service and env reserved tags, see Unified Service Tagging. All other Datadog Tracer configuration options can also be used.

Adding custom tags to tests

To add custom tags to tests, configure custom instrumentation first.

You can add custom tags to your tests by using the current active span:

// inside your test
var scope = Tracer.Instance.ActiveScope; // from Datadog.Trace;
if (scope != null) {
    scope.Span.SetTag("test_owner", "my_team");
}
// test continues normally
// ...

To create filters or group by fields for these tags, you must first create facets. For more information about adding tags, see the Adding Tags section of the .NET custom instrumentation documentation.

Adding custom measures to tests

To add custom measures to tests, configure custom instrumentation first.

Just like tags, you can add custom measures to your tests by using the current active span:

// inside your test
var scope = Tracer.Instance.ActiveScope; // from Datadog.Trace;
if (scope != null) {
    scope.Span.SetTag("memory_allocations", 16);
}
// test continues normally
// ...

To create filters or visualizations for these tags, you must first create facets. For more information about adding tags, see the Adding Tags section of the .NET custom instrumentation documentation.

Read more about custom Measures in the Add Custom Measures Guide.

Reporting code coverage

When code coverage is available, the Datadog Tracer (v2.31.0 or later) reports it under the test.code_coverage.lines_pct tag for your test sessions.

If you are using Coverlet to compute your code coverage, indicate the path to the report file in the DD_CIVISIBILITY_EXTERNAL_CODE_COVERAGE_PATH environment variable when running dd-trace. The report file must be in the OpenCover or Cobertura formats. Alternatively, you can enable the Datadog Tracer’s built-in code coverage calculation with the DD_CIVISIBILITY_CODE_COVERAGE_ENABLED=true environment variable.

Note: When using Intelligent Test Runner, the tracer’s built-in code coverage is enabled by default.

You can see the evolution of the test coverage in the Coverage tab of a test session.

For more information about exclusion options, see Code Coverage.

Instrumenting BenchmarkDotNet tests

To instrument your benchmark tests, you need to:

  1. Add the Datadog.Trace.BenchmarkDotNet NuGet package to your project (for example, using dotnet add package Datadog.Trace.BenchmarkDotNet).
  2. Configure your project to use the Datadog.Trace.BenchmarkDotNet exporter using the DatadogDiagnoser attribute or the WithDatadog() extension method. For example:
using BenchmarkDotNet.Attributes;
using Datadog.Trace.BenchmarkDotNet;

[DatadogDiagnoser]
[MemoryDiagnoser]
public class OperationBenchmark
{
    [Benchmark]
    public void Operation()
    {
        // ...
    }
}
using BenchmarkDotNet.Configs;
using BenchmarkDotNet.Running;
using Datadog.Trace.BenchmarkDotNet;

var config = DefaultConfig.Instance
              .WithDatadog();

BenchmarkRunner.Run<OperationBenchmark>(config);
  1. Configure the reporting method.
  2. Run the benchmark project as you normally do, all benchmark tests will be automatically instrumented.

Datadog uses Git information for visualizing your test results and grouping them by repository, branch, and commit. Git metadata is automatically collected by the test instrumentation from CI provider environment variables and the local .git folder in the project path, if available.

If you are running tests in non-supported CI providers or with no .git folder, you can set the Git information manually using environment variables. These environment variables take precedence over any auto-detected information. Set the following environment variables to provide Git information:

DD_GIT_REPOSITORY_URL
URL of the repository where the code is stored. Both HTTP and SSH URLs are supported.
Example: git@github.com:MyCompany/MyApp.git, https://github.com/MyCompany/MyApp.git
DD_GIT_BRANCH
Git branch being tested. Leave empty if providing tag information instead.
Example: develop
DD_GIT_TAG
Git tag being tested (if applicable). Leave empty if providing branch information instead.
Example: 1.0.1
DD_GIT_COMMIT_SHA
Full commit hash.
Example: a18ebf361cc831f5535e58ec4fae04ffd98d8152
DD_GIT_COMMIT_MESSAGE
Commit message.
Example: Set release number
DD_GIT_COMMIT_AUTHOR_NAME
Commit author name.
Example: John Smith
DD_GIT_COMMIT_AUTHOR_EMAIL
Commit author email.
Example: john@example.com
DD_GIT_COMMIT_AUTHOR_DATE
Commit author date in ISO 8601 format.
Example: 2021-03-12T16:00:28Z
DD_GIT_COMMIT_COMMITTER_NAME
Commit committer name.
Example: Jane Smith
DD_GIT_COMMIT_COMMITTER_EMAIL
Commit committer email.
Example: jane@example.com
DD_GIT_COMMIT_COMMITTER_DATE
Commit committer date in ISO 8601 format.
Example: 2021-03-12T16:00:28Z

Custom instrumentation

Note: Your custom instrumentation setup depends on the dd-trace version. To use the custom instrumentation, you must keep the package versions for dd-trace and Datadog.Trace NuGet packages in sync.

To use the custom instrumentation in your .NET application:

  1. Execute dd-trace --version to get the version of the tool.
  2. Add the Datadog.Trace NuGet package with the same version to your application.
  3. In your application code, access the global tracer through the Datadog.Trace.Tracer.Instance property to create new spans.

For more information about how to add spans and tags for custom instrumentation, see the .NET Custom Instrumentation documentation.

Manual testing API

Note: To use the manual testing API, you must add the Datadog.Trace NuGet package in the target .NET project.

If you use XUnit, NUnit, or MSTest with your .NET projects, CI Visibility automatically instruments them and sends the test results to Datadog. If you use an unsupported testing framework or if you have a different testing mechanism, you can instead use the API to report test results to Datadog.

The API is based around three concepts: test module, test suites, and tests.

Test module

A test module represents the .NET assembly that includes the tests.

To start a test module, call TestModule.Create() and pass the name of the module or .NET assembly name where the tests are located.

When all your tests have finished, call module.Close() or module.CloseAsync(), which forces the library to send all remaining test results to the backend.

Test suites

A test suite comprises a set of tests. They can have a common initialization and teardown methods and share some variables. In .NET, they are usually implemented as a Test class or fixture containing multiple test methods. A test suite can optionally have additional information like attributes or error information.

Create test suites in the test module by calling module.GetOrCreateSuite() and passing the name of the test suite.

Call suite.Close() when all the related tests in the suite have finished their execution.

Tests

Each test runs inside a suite and must end in one of these three statuses: TestStatus.Pass, TestStatus.Fail, or TestStatus.Skip.

A test can optionally have additional information like:

  • Parameters
  • Attributes
  • Error information
  • Test traits
  • Benchmark data

Create tests in a suite by calling suite.CreateTest() and passing the name of the test. When a test ends, call test.Close() with one of the predefined statuses.

Code example

The following code represents a simple usage of the API:

using System.Reflection;
using Datadog.Trace.Ci;

var module = TestModule.Create(Assembly.GetExecutingAssembly().GetName().Name ?? "(dyn_module)");
module.SetTag("ModuleTag", "Value");

var suite = module.GetOrCreateSuite("MySuite");
suite.SetTag("SuiteTag", 42);

var test = suite.CreateTest("Test01");
test.SetTag("TestTag", "Value");
test.SetParameters(new TestParameters
{
    Arguments = new Dictionary<string, object>
    {
        ["a"] = 42,
        ["b"] = 0,
    }
});
test.SetTraits(new Dictionary<string, List<string>>
{
    ["Category"] = new () { "UnitTest" }
});

try
{
    var a = 42;
    var b = 0;
    var c = a / b;
}
catch (Exception ex)
{
    test.SetErrorInfo(ex);
}

test.Close(TestStatus.Fail);
suite.Close();
await module.CloseAsync();

Always call module.Close() or module.CloseAsync() at the end so that all the test data is flushed to Datadog.

Further reading