이 페이지는 아직 한국어로 제공되지 않으며 번역 작업 중입니다. 번역에 관한 질문이나 의견이 있으시면 언제든지 저희에게 연락해 주십시오.

The selected Datadog site () is not supported.

Note: Datadog recommends the native instrumentation of tests over uploading JUnit XML files, as the native instrumentation provides more accurate time results, supports distributed traces on integration tests and other features that are not available with JUnit XML uploads. See the Supported Features table for more details.

Overview

JUnit test report files are XML files that contain test execution information, such as test and suite names, pass or fail status, duration, and sometimes error logs. Although introduced by the JUnit testing framework, many other popular frameworks are able to output results using this format.

If your testing framework can generate JUnit XML test reports, you can use these as a lightweight alternative to instrumenting your tests natively using Datadog tracers. Test results imported from JUnit XML reports appear alongside test data reported by tracers.

Compatibility

Supported Datadog tracing library:

Datadog LibraryVersion
datadog-ci>= 2.17.0

Installing the Datadog CI CLI

Install the datadog-ci CLI globally using npm:

npm install -g @datadog/datadog-ci

Standalone binary (beta)

Note: The standalone binaries are in beta and their stability is not guaranteed.

If installing Node.js in the CI is an issue, standalone binaries are provided with Datadog CI releases. Only linux-x64, darwin-x64 (MacOS) and win-x64 (Windows) are supported. To install, run the following from your terminal:

curl -L --fail "https://github.com/DataDog/datadog-ci/releases/latest/download/datadog-ci_linux-x64" --output "/usr/local/bin/datadog-ci" && chmod +x /usr/local/bin/datadog-ci

Then run any command with datadog-ci:

datadog-ci version
curl -L --fail "https://github.com/DataDog/datadog-ci/releases/latest/download/datadog-ci_darwin-x64" --output "/usr/local/bin/datadog-ci" && chmod +x /usr/local/bin/datadog-ci

Then run any command with datadog-ci:

datadog-ci version
Invoke-WebRequest -Uri "https://github.com/DataDog/datadog-ci/releases/latest/download/datadog-ci_win-x64.exe" -OutFile "datadog-ci.exe"

Then run any command with Start-Process -FilePath "datadog-ci.exe":

Start-Process -FilePath "./datadog-ci.exe" -ArgumentList version

Uploading test reports

To upload your JUnit XML test reports to Datadog, run the following command, specifying the name of the service or library that was tested using the --service parameter, and one or more file paths to either the XML report files directly or directories containing them:

datadog-ci junit upload --service <service_name> <path> [<path> ...]

Specify a valid Datadog API key in the DATADOG_API_KEY environment variable, and the environment where tests were run (for example, local when uploading results from a developer workstation, or ci when uploading them from a CI provider) in the DD_ENV environment variable. For example:


DD_ENV=ci DATADOG_API_KEY=<api_key> DATADOG_SITE= datadog-ci junit upload \
  --service my-api-service \
  unit-tests/junit-reports e2e-tests/single-report.xml

Make sure that this command runs in your CI even when your tests have failed. Usually, when tests fail, the CI job aborts execution, and the upload command does not run.

Use the Status check functions:

steps:
  - name: Run tests
    run: ./run-tests.sh
  - name: Upload test results to Datadog
    if: always()
    run: datadog-ci junit upload --service service_name ./junit.xml

Use the after_script section:

test:
  stage: test
  script:
    - ./run-tests.sh
  after_script:
    - datadog-ci junit upload --service service_name ./junit.xml

Use the post section:

pipeline {
  agent any
  stages {
    stage('Run tests') {
      steps {
        sh './run-tests.sh'
      }
      post {
        always {
          sh 'datadog-ci junit upload --service service_name ./junit.xml'
        }
      }
    }
  }
}

If your CI system allows sub-shells:

$(./run-tests.sh); export tests_exit_code=$?
datadog-ci junit upload --service service_name ./junit.xml
if [ $tests_exit_code -ne 0 ]; then exit $tests_exit_code; fi

Reports larger than 250 MiB may not be processed completely, resulting in missing tests or logs. For the best experience, ensure that the reports are under 250 MiB.

Configuration settings

This is the full list of options available when using the datadog-ci junit upload command:

--service (Required)
Name of the service or library under test.
Environment variable: DD_SERVICE
Example: my-api-service
--env
Environment where tests were run.
Environment variable: DD_ENV
Example: ci
--tags
Key-value pairs in the form key:value to be attached to all tests (the --tags parameter can be specified multiple times). When specifying tags using DD_TAGS, separate them using commas (for example, team:backend,priority:high).
Environment variable: DD_TAGS
Default: (none)
Example: team:backend
Note: Tags specified using --tags and with the DD_TAGS environment variable are merged. If the same key appears in both --tags and DD_TAGS, the value in the environment variable DD_TAGS takes precedence.
--measures
Key-value numerical pairs in the form key:number to be attached to all tests (the --measures parameter can be specified multiple times). When specifying measures using DD_MEASURES, separate them using commas (for example, memory_allocations:13,test_importance:2).
Environment variable: DD_MEASURES
Default: (none)
Example: memory_allocations:13
Note: Measures specified using --measures and with the DD_MEASURES environment variable are merged. If the same key appears in both --measures and DD_MEASURES, the value in the environment variable DD_MEASURES takes precedence.
--report-tags
Key-value pairs in the form key:value. Works like the --tags parameter but these tags are only applied at the session level and are not merged with the environment variable DD_TAGS
Default: (none)
Example: test.code_coverage.enabled:true
--report-measures
Key-value pairs in the form key:123. Works like the --measures parameter but these tags are only applied at the session level and are not merged with the environment variable DD_MEASURES
Default: (none)
Example: test.code_coverage.lines_pct:82
--xpath-tag
Key and xpath expression in the form key=expression. These provide a way to customize tags for test in the file (the --xpath-tag parameter can be specified multiple times).
See Providing metadata with XPath expressions for more details on the supported expressions.
Default: (none)
Example: test.suite=/testcase/@classname
Note: Tags specified using --xpath-tag and with --tags or DD_TAGS environment variable are merged. xpath-tag gets the highest precedence, as the value is usually different for each test.
--logs
Enable forwarding content from the XML reports as Logs. The content inside <system-out>, <system-err>, and <failure> is collected as logs. Logs from elements inside a <testcase> are automatically connected to the test.
Environment variable: DD_CIVISIBILITY_LOGS_ENABLED
Default: false
Note: Logs are billed separately from CI Visibility.
--max-concurrency
The number of concurrent uploads to the API.
Default: 20
--dry-run
Runs the command without actually uploading the file to Datadog. All other checks are performed.
Default: false
--skip-git-metadata-upload
Flag used to skip git metadata upload. If you want to upload git metadata, you may pass –skip-git-metadata-upload=0 or –skip-git-metadata-upload=false.
Default: true
--git-repository-url
The repository URL to retrieve git metadata from. If it is not passed, the URL is retrieved from the local git repository.
Default: local git repository
Example: git@github.com:DataDog/documentation.git
--verbose
Flag used to add extra verbosity to the output of the command
Default: false
Positional arguments
The file paths or directories in which the JUnit XML reports are located. If you pass a directory, the CLI looks for all .xml files in it.

For more information about service and env reserved tags, see Unified Service Tagging.

The following environment variables are supported:

DATADOG_API_KEY (Required)
Datadog API key used to authenticate the requests.
Default: (none)

Additionally, configure the Datadog site to use the selected one ():

DATADOG_SITE (Required)
The Datadog site to upload results to.
Default: datadoghq.com
Selected site:

Collecting Git metadata

Datadog uses Git information for visualizing your test results and grouping them by repository, branch, and commit. Git metadata is automatically collected by the test instrumentation from CI provider environment variables and the local .git folder in the project path, if available.

If you are running tests in non-supported CI providers or with no .git folder, you can set the Git information manually using environment variables. These environment variables take precedence over any auto-detected information. Set the following environment variables to provide Git information:

DD_GIT_REPOSITORY_URL
URL of the repository where the code is stored. Both HTTP and SSH URLs are supported.
Example: git@github.com:MyCompany/MyApp.git, https://github.com/MyCompany/MyApp.git
DD_GIT_BRANCH
Git branch being tested. Leave empty if providing tag information instead.
Example: develop
DD_GIT_TAG
Git tag being tested (if applicable). Leave empty if providing branch information instead.
Example: 1.0.1
DD_GIT_COMMIT_SHA
Full commit hash.
Example: a18ebf361cc831f5535e58ec4fae04ffd98d8152
DD_GIT_COMMIT_MESSAGE
Commit message.
Example: Set release number
DD_GIT_COMMIT_AUTHOR_NAME
Commit author name.
Example: John Smith
DD_GIT_COMMIT_AUTHOR_EMAIL
Commit author email.
Example: john@example.com
DD_GIT_COMMIT_AUTHOR_DATE
Commit author date in ISO 8601 format.
Example: 2021-03-12T16:00:28Z
DD_GIT_COMMIT_COMMITTER_NAME
Commit committer name.
Example: Jane Smith
DD_GIT_COMMIT_COMMITTER_EMAIL
Commit committer email.
Example: jane@example.com
DD_GIT_COMMIT_COMMITTER_DATE
Commit committer date in ISO 8601 format.
Example: 2021-03-12T16:00:28Z

Uploading Git metadata

From datadog-ci version 2.9.0 or later, CI Visibility automatically uploads Git metadata information (commit history). This metadata contains file names but no file contents. If you want to opt out of this behavior, pass the flag --skip-git-metadata-upload.

Collecting environment configuration metadata

Datadog uses special dedicated tags to identify the configuration of the environment in which tests run, including the operating system, runtime, and device information, if applicable. When the same test for the same commit runs in more than one configuration (for example, on Windows and on Linux), the tags are used to differentiate the test in failure and flakiness detection.

You can specify these special tags using the --tags parameter when calling datadog-ci junit upload, or by setting the DD_TAGS environment variable.

All of these tags are optional, and only the ones you specify will be used to differentiate between environment configurations.

test.bundle
Used to execute groups of test suites separately.
Examples: ApplicationUITests, ModelTests
os.platform
Name of the operating system.
Examples: windows, linux, darwin
os.version
Version of the operating system.
Examples: 10.15.4, 14.3.2, 95
os.architecture
Architecture of the operating system.
Examples: x64, x86, arm64
runtime.name
Name of the language interpreter or programming runtime.
Examples: .NET, .NET Core, OpenJDK Runtime Environment, Java(TM) SE Runtime Environment, CPython
runtime.version
Version of the runtime.
Examples: 5.0.0, 3.1.7
runtime.vendor
Name of the runtime vendor where applicable. For example, when using a Java runtime.
Examples: AdoptOpenJDK, Oracle Corporation
runtime.architecture
Architecture of the runtime.
Examples: x64, x86, arm64

For mobile apps (Swift, Android):

device.model
The model of the device being tested.
Examples: iPhone11,4, AppleTV5,3
device.name
The name of the device being tested.
Examples: iPhone 12 Pro Simulator, iPhone 13 (QA team)

Adding code owners

To add codeowners information to your JUnit XML tests, you can use the GitHub integration to read the CODEOWNERS file in your repository or provide some additional information manually.

As a result, the JUnit XML tests have a test.codeowners tag with the owner of those tests.

To automatically add the test.codeowners tag to your tests, you need to:

  1. Have a CODEOWNERS file in one of the allowed locations in your repository.

  2. Provide the tests source file in your JUnit XML report. The following plugins do this automatically and add the file attribute to the <testcase> or <testsuite> elements in the XML report:

    • phpunit
    • Most Python plugins (pytest, unittest)
    • Most Ruby plugins (ruby minitest)

    If the XML does not have the file attribute, you need to provide the source file manually. Example of a valid report:

  <?xml version="1.0" encoding="UTF-8"?>
  <testsuite name="suite">
    <testcase name="test_with_file" file="src/commands/junit" />
  </testsuite>
  
  1. Enable the GitHub app. If you do not have a GitHub app, follow the steps in the next section. If you already have a GitHub app, enable the Contents: Read permission so Datadog can read the CODEOWNERS file. Once enabled, wait a few minutes for the changes to take effect.

Note: Github is the only supported Git provider.

Configure a GitHub App

The JUnit XML uses a private GitHub App to read the CODEOWNERS file.

  1. Go to the GitHub integration tile.
  2. Click Link GitHub Account.
  3. Follow the instructions to configure the integration for a personal or organization account.
  4. In Edit Permissions, grant Contents: Read access.
  5. Click Create App in GitHub to finish the app creation process on GitHub.
  6. Give the app a name, for example, Datadog CI Visibility.
  7. Click Install GitHub App and follow the instructions on GitHub.

Manually providing the test.source.file tag

This is an alternative to using the GitHub integration.

For those plugins that do not provide the file attribute in the XML report, you can provide the test.source.file tag. There is no need to provide the exact path to a specific file, you can use any syntax you would use in the CODEOWNERS file such as src/myTeamFolder or *.md.

There are multiple ways to provide the test.source.file tag:

  • Using the --tags parameter or the DD_TAGS environment variable.

    datadog-ci junit upload --service service-name --tags test.source.file:src/myTeamFolder my_report.xml
    

    This adds the test.source.file tag to all the tests in the report. All of the tests will have the same owner(s).

  • If you want to provide different source files for the same XML report, you can use property elements or set the file attribute manually to individual <testcase> or <testsuite> elements.

Providing metadata with XPath expressions

In addition to the --tags CLI parameter and the DD_TAGS environment variable, which apply custom tags globally to all tests included the uploaded XML report, the --xpath-tag parameter provides custom rules to add tags from different attributes within the XML to each test.

The parameter provided must have the format key=expression, where key is the name of the custom tag to be added and expression is a valid XPath expression within the ones supported.

While XPath syntax is used for familiarity, only the following expressions are supported:

/testcase/@attribute-name
The XML attribute from <testcase attribute-name="value">.
/testcase/../@attribute-name
The XML attribute from the parent <testsuite attribute-name="value"> of the current <testcase>.
/testcase/..//property[@name='property-name']
The value attribute from the <property name="property-name" value="value"> inside the parent <testsuite> of the current <testcase>.
/testcase//property[@name='property-name']
The value attribute from the <property name="property-name" value="value"> inside the current <testcase>.

Examples:

By default, the test.suite tag of the tests is read from <testsuite name="suite name">. However, some plugins might report a better value in <testcase classname="TestSuite">.

To change test.suite tags from value 1, value 2 to SomeTestSuiteClass, OtherTestSuiteClass:

<?xml version="1.0" encoding="UTF-8"?>
<testsuites>
  <testsuite tests="1" failures="0" time="0.030000" name="value 1">
    <testcase classname="SomeTestSuiteClass" name="test_something" time="0.030000"></testcase>
  </testsuite>
  <testsuite tests="1" failures="0" time="0.021300" name="value 2">
    <testcase classname="OtherTestSuiteClass" name="test_something" time="0.021300"></testcase>
  </testsuite>
</testsuites>
datadog-ci junit upload --service service_name \
  --xpath-tag test.suite=/testcase/@classname ./junit.xml

To add custom_tag to each test with values value 1, value 2:

<?xml version="1.0" encoding="UTF-8"?>
<testsuites>
  <testsuite tests="1" failures="0" time="0.020000" name="SomeTestSuiteClass">
    <testcase name="test_something" time="0.010000" attr="value 1"></testcase>
    <testcase name="test_other" time="0.010000" attr="value 2"></testcase>
  </testsuite>
</testsuites>
datadog-ci junit upload --service service_name \
  --xpath-tag custom_tag=/testcase/@attr ./junit.xml

To add custom_tag to each test with values value 1, value 2:

<?xml version="1.0" encoding="UTF-8"?>
<testsuites>
  <testsuite tests="1" failures="0" time="0.030000" name="SomeTestSuiteClass">
    <properties>
      <property name="prop" value="value 1"></property>
    </properties>
    <testcase name="test_something" time="0.030000" attr="value 1"></testcase>
  </testsuite>
  <testsuite tests="1" failures="0" time="0.021300" name="OtherTestSuiteClass">
    <properties>
      <property name="prop" value="value 1"></property>
    </properties>
    <testcase name="test_something" time="0.021300" attr="value 1"></testcase>
  </testsuite>
</testsuites>
datadog-ci junit upload --service service_name \
  --xpath-tag custom_tag=/testcase/..//property[@name=\'prop\'] ./junit.xml

Note: The name must be in quotes. Bash requires quotes to be escaped using a backslash, for example [@name='prop'] must be entered as `[@name='prop'].

When using bash from Git for Windows, define the MSYS_NO_PATHCONV=1 environment variable. Otherwise, any argument starting with / will be expanded to a Windows path.

Providing metadata through property elements

Another way to provide additional tags to specific tests is including <property name="dd_tags[key]" value="value"> elements within the <testsuite> or <testcase> elements. If you add these tags to a <testcase> element, they are stored in its test span. If you add the tags to a <testsuite> element, they are stored in all of that suite’s test spans.

To be processed, the name attribute in the <property> element must have the format dd_tags[key], where key is the name of the custom tag to be added. Other properties are ignored.

Example: Adding tags to a <testcase> element.

<?xml version="1.0" encoding="UTF-8"?>
<testsuites>
  <testsuite tests="1" failures="0" time="0.030000" name="SomeTestSuiteClass">
    <testcase classname="SomeTestSuiteClass" name="test_something" time="0.010000">
      <properties>
        <property name="dd_tags[custom_tag]" value="some value"></property>
        <property name="dd_tags[runtime.name]" value="CPython"></property>
      </properties>
    </testcase>
  </testsuite>
</testsuites>

Example: Adding tags to a <testsuite> element.

<?xml version="1.0" encoding="UTF-8"?>
<testsuites>
  <testsuite tests="1" failures="0" time="0.030000" name="SomeTestSuiteClass">
    <properties>
      <property name="dd_tags[custom_tag]" value="some value"></property>
      <property name="dd_tags[runtime.name]" value="CPython"></property>
    </properties>
    <testcase classname="SomeTestSuiteClass" name="test_something" time="0.010000"></testcase>
  </testsuite>
</testsuites>

The values that you send to Datadog are strings, so the facets are displayed in lexicographical order. To send integers instead of strings, use the --measures flag and the DD_MEASURES environment variable.

Reporting code coverage

It is possible to report code coverage for a given JUnit report via the --report-measures option, by setting the test.code_coverage.lines_pct measure:

datadog-ci junit upload --service my-api-service --report-measures test.code_coverage.lines_pct:82 unit-tests/junit-reports e2e-tests/single-report.xml

For more information, see Code Coverage.

Further reading