Note: Datadog recommends the native instrumentation of tests over uploading JUnit XML files, as the native instrumentation provides more accurate time results, supports distributed traces on integration tests, and supports structured stack traces.
JUnit test report files are XML files that contain test execution information, such as test and suite names, pass or fail status, duration, and sometimes error logs. Although introduced by the JUnit testing framework, many other popular frameworks are able to output results using this format.
If your testing framework can generate JUnit XML test reports, you can use these as a lightweight alternative to instrumenting your tests natively using Datadog tracers. Test results imported from JUnit XML reports appear alongside test data reported by tracers.
Note: The standalone binaries are in beta and their stability is not guaranteed.
If installing Node.js in the CI is an issue, standalone binaries are provided with Datadog CI releases. Only linux-x64, darwin-x64 (MacOS) and win-x64 (Windows) are supported. To install, run the following from your terminal:
To upload your JUnit XML test reports to Datadog, run the following command, specifying the name of the service or library that was tested using the --service parameter, and one or more file paths to either the XML report files directly or directories containing them:
Specify a valid Datadog API key in the DATADOG_API_KEY environment variable, and the environment where tests were run (for example, local when uploading results from a developer workstation, or ci when uploading them from a CI provider) in the DD_ENV environment variable. For example:
Note: Reports larger than 250 MiB may not be processed completely, resulting in missing tests or logs. For the best experience ensure that the reports are under 250 MiB.
This is the full list of options available when using the datadog-ci junit upload command:
Name of the service or library under test. Environment variable: DD_SERVICE Example: my-api-service
Environment where tests were run. Environment variable: DD_ENV Example: ci
Key-value pairs in the form key:value to be attached to all tests (the --tags parameter can be specified multiple times). When specifying tags using DD_TAGS, separate them using commas (for example, team:backend,priority:high). Environment variable: DD_TAGS Default: (none) Example: team:backend Note: Tags specified using --tags and with the DD_TAGS environment variable are merged. If the same key appears in both --tags and DD_TAGS, the value in the environment variable DD_TAGS takes precedence.
Key and xpath expression in the form key=expression. These provide a way to customize tags for test in the file (the --xpath-tag parameter can be specified multiple times). See Providing metadata with XPath expressions for more details on the supported expressions. Default: (none) Example: test.suite=/testcase/@classname Note: Tags specified using --xpath-tag and with --tags or DD_TAGS environment variable are merged. xpath-tag gets the highest precedence, as the value is usually different for each test.
Enable forwarding content from the XML reports as Logs. The content inside <system-out>, <system-err>, and <failure> is collected as logs. Logs from elements inside a <testcase> are automatically connected to the test. Environment variable: DD_CIVISIBILITY_LOGS_ENABLED Default: false Note: Logs are billed separately from CI Visibility.
The number of concurrent uploads to the API. Default: 20
Runs the command without actually uploading the file to Datadog. All other checks are performed. Default: false
The file paths or directories in which the JUnit XML reports are located. If you pass a directory, the CLI looks for all .xml files in it.
The following environment variables are supported:
Additionally, configure the Datadog site to use the selected one ():
The Datadog site to upload results to. Default: datadoghq.com Selected site:
Collecting repository and commit metadata
Datadog uses Git information for visualizing your test results and grouping them by repository and commit. Git metadata is collected by the Datadog CI CLI from CI provider environment variables and the local .git folder in the project path, if available. To read this directory, the git binary is required.
If you are running tests in non-supported CI providers or with no .git folder, you can set the Git information manually using environment variables. These environment variables take precedence over any auto-detected information. Set the following environment variables to provide Git information:
URL of the repository where the code is stored. Both HTTP and SSH URLs are supported. Example: email@example.com:MyCompany/MyApp.git, https://github.com/MyCompany/MyApp.git
Git branch being tested. Leave empty if providing tag information instead. Example: develop
Git tag being tested (if applicable). Leave empty if providing branch information instead. Example: 1.0.1
Full commit hash. Example: a18ebf361cc831f5535e58ec4fae04ffd98d8152
Commit message. Example: Set release number
Commit author name. Example: John Smith
Commit author email. Example: firstname.lastname@example.org
Commit author date in ISO 8601 format. Example: 2021-03-12T16:00:28Z
Commit committer name. Example: Jane Smith
Commit committer email. Example: email@example.com
Commit committer date in ISO 8601 format. Example: 2021-03-12T16:00:28Z
Git metadata upload
From datadog-ci version 2.9.0 or later, CI Visibility automatically uploads Git metadata information (commit history). This metadata contains file names but no file contents. If you want to opt out of this behavior, pass the flag --skip-git-metadata-upload.
Collecting environment configuration metadata
Datadog uses special dedicated tags to identify the configuration of the environment in which tests run, including the operating system, runtime, and device information, if applicable. When the same test for the same commit runs in more than one configuration (for example, on Windows and on Linux), the tags are used to differentiate the test in failure and flakiness detection.
You can specify these special tags using the --tags parameter when calling datadog-ci junit upload, or by setting the DD_TAGS environment variable.
All of these tags are optional, and only the ones you specify will be used to differentiate between environment configurations.
Used to execute groups of test suites separately. Examples: ApplicationUITests, ModelTests
Name of the operating system. Examples: windows, linux, darwin
Version of the operating system. Examples: 10.15.4, 14.3.2, 95
Architecture of the operating system. Examples: x64, x86, arm64
Name of the language interpreter or programming runtime. Examples: .NET, .NET Core, OpenJDK Runtime Environment, Java(TM) SE Runtime Environment, CPython
Version of the runtime. Examples: 5.0.0, 3.1.7
Name of the runtime vendor where applicable. For example, when using a Java runtime. Examples: AdoptOpenJDK, Oracle Corporation
Architecture of the runtime. Examples: x64, x86, arm64
For mobile apps (Swift, Android):
The model of the device being tested. Examples: iPhone11,4, AppleTV5,3
The name of the device being tested. Examples: iPhone 12 Pro Simulator, iPhone 13 (QA team)
Providing metadata with XPath expressions
In addition to the --tags CLI parameter and the DD_TAGS environment variable, which apply custom tags globally to all tests included the uploaded XML report, the --xpath-tag parameter provides custom rules to add tags from different attributes within the XML to each test.
The parameter provided must have the format key=expression, where key is the name of the custom tag to be added and expression is a valid XPath expression within the ones supported.
While XPath syntax is used for familiarity, only the following expressions are supported:
The XML attribute from <testcase attribute-name="value">.
The XML attribute from the parent <testsuite attribute-name="value"> of the current <testcase>.
The value attribute from the <property name="property-name" value="value"> inside the parent <testsuite> of the current <testcase>.
The value attribute from the <property name="property-name" value="value"> inside the current <testcase>.
By default, the test.suite tag of the tests is read from <testsuite name="suite name">. However, some plugins might report a better value in <testcase classname="TestSuite">.
To change test.suite tags from value 1, value 2 to SomeTestSuiteClass, OtherTestSuiteClass:
Note: The name must be in quotes. Bash requires quotes to be escaped using a backslash, for example [@name='prop'] must be entered as `[@name='prop'].
Providing metadata through <property> elements
Another way to provide additional tags to specific tests is including <property name="dd_tags[key]" value="value"> elements within the <testsuite> or <testcase> elements. If you add these tags to a <testcase> element, they are stored in its test span. If you add the tags to a <testsuite> element, they are stored in all of that suite’s test spans.
To be processed, the name attribute in the <property> element must have the format dd_tags[key], where key is the name of the custom tag to be added. Other properties are ignored.
The values that you send to Datadog are strings, so the facets are displayed in lexicographical order. To send integers instead of strings, use the --metrics flag and the DD_METRICS environment variable.
Additional helpful documentation, links, and articles: