- 필수 기능
- 시작하기
- Glossary
- 표준 속성
- Guides
- Agent
- 통합
- 개방형텔레메트리
- 개발자
- API
- Datadog Mobile App
- CoScreen
- Cloudcraft
- 앱 내
- 서비스 관리
- 인프라스트럭처
- 애플리케이션 성능
- APM
- Continuous Profiler
- 스팬 시각화
- 데이터 스트림 모니터링
- 데이터 작업 모니터링
- 디지털 경험
- 소프트웨어 제공
- 보안
- AI Observability
- 로그 관리
- 관리
JUnit test report files are XML files that contain test execution information, such as test and suite names, pass or fail status, duration, and sometimes error logs. Although introduced by the JUnit testing framework, many other popular frameworks are able to output results using this format.
If your testing framework can generate JUnit XML test reports, you can use these as a lightweight alternative to instrumenting your tests natively using Datadog tracers. Test results imported from JUnit XML reports appear alongside test data reported by tracers.
Supported Datadog tracing library:
Datadog Library | Version |
---|---|
datadog-ci | >= 2.17.0 |
Install the datadog-ci
CLI globally using npm
:
npm install -g @datadog/datadog-ci
If installing Node.js in the CI is an issue, standalone binaries are provided with Datadog CI releases. Only linux-x64, linux-arm64, darwin-x64, darwin-arm64 (MacOS) and win-x64 (Windows) are supported. To install, run the following from your terminal:
curl -L --fail "https://github.com/DataDog/datadog-ci/releases/latest/download/datadog-ci_linux-x64" --output "/usr/local/bin/datadog-ci" && chmod +x /usr/local/bin/datadog-ci
Then run any command with datadog-ci
:
datadog-ci version
curl -L --fail "https://github.com/DataDog/datadog-ci/releases/latest/download/datadog-ci_darwin-x64" --output "/usr/local/bin/datadog-ci" && chmod +x /usr/local/bin/datadog-ci
Then run any command with datadog-ci
:
datadog-ci version
Invoke-WebRequest -Uri "https://github.com/DataDog/datadog-ci/releases/latest/download/datadog-ci_win-x64.exe" -OutFile "datadog-ci.exe"
Then run any command with Start-Process -FilePath "datadog-ci.exe"
:
Start-Process -FilePath "./datadog-ci.exe" -ArgumentList version
To upload your JUnit XML test reports to Datadog, run the following command, specifying the name of the service or library that was tested using the --service
parameter, and one or more file paths to either the XML report files directly or directories containing them:
datadog-ci junit upload --service <service_name> <path> [<path> ...]
Specify a valid Datadog API key in the DATADOG_API_KEY
environment variable, and the environment where tests were run (for example, local
when uploading results from a developer workstation, or ci
when uploading them from a CI provider) in the DD_ENV
environment variable. For example:
DD_ENV=ci DATADOG_API_KEY=<api_key> DATADOG_SITE= datadog-ci junit upload \
--service my-api-service \
unit-tests/junit-reports e2e-tests/single-report.xml
Use the Status check functions:
steps:
- name: Run tests
run: ./run-tests.sh
- name: Upload test results to Datadog
if: always()
run: datadog-ci junit upload --service service_name ./junit.xml
Use the after_script
section:
test:
stage: test
script:
- ./run-tests.sh
after_script:
- datadog-ci junit upload --service service_name ./junit.xml
Use the post
section:
pipeline {
agent any
stages {
stage('Run tests') {
steps {
sh './run-tests.sh'
}
post {
always {
sh 'datadog-ci junit upload --service service_name ./junit.xml'
}
}
}
}
}
If your CI system allows sub-shells:
$(./run-tests.sh); export tests_exit_code=$?
datadog-ci junit upload --service service_name ./junit.xml
if [ $tests_exit_code -ne 0 ]; then exit $tests_exit_code; fi
Reports larger than 250 MiB may not be processed completely, resulting in missing tests or logs. For the best experience, ensure that the reports are under 250 MiB.
This is the full list of options available when using the datadog-ci junit upload
command:
--service
(Required)DD_SERVICE
my-api-service
--env
DD_ENV
ci
--tags
key:value
to be attached to all tests (the --tags
parameter can be specified multiple times). When specifying tags using DD_TAGS
, separate them using commas (for example, team:backend,priority:high
).DD_TAGS
team:backend
--tags
and with the DD_TAGS
environment variable are merged. If the same key appears in both --tags
and DD_TAGS
, the value in the environment variable DD_TAGS
takes precedence.--measures
key:number
to be attached to all tests (the --measures
parameter can be specified multiple times). When specifying measures using DD_MEASURES
, separate them using commas (for example, memory_allocations:13,test_importance:2
).DD_MEASURES
memory_allocations:13
--measures
and with the DD_MEASURES
environment variable are merged. If the same key appears in both --measures
and DD_MEASURES
, the value in the environment variable DD_MEASURES
takes precedence.--report-tags
key:value
. Works like the --tags
parameter but these tags are only applied at the session level and are not merged with the environment variable DD_TAGS
test.code_coverage.enabled:true
--report-measures
key:123
. Works like the --measures
parameter but these tags are only applied at the session level and are not merged with the environment variable DD_MEASURES
test.code_coverage.lines_pct:82
--xpath-tag
key=expression
. These provide a way to customize tags for test in the file (the --xpath-tag
parameter can be specified multiple times).test.suite=/testcase/@classname
--xpath-tag
and with --tags
or DD_TAGS
environment variable are merged. xpath-tag gets the highest precedence, as the value is usually different for each test.--logs
<system-out>
, <system-err>
, and <failure>
is collected as logs. Logs from elements inside a <testcase>
are automatically connected to the test.DD_CIVISIBILITY_LOGS_ENABLED
false
--max-concurrency
20
--dry-run
false
--skip-git-metadata-upload
false
--git-repository-url
git@github.com:DataDog/documentation.git
--verbose
false
.xml
files in it.For more information about service
and env
reserved tags, see Unified Service Tagging.
The following environment variables are supported:
DATADOG_API_KEY
(Required)Additionally, configure the Datadog site to use the selected one ():
DATADOG_SITE
(Required)datadoghq.com
Datadog uses Git information for visualizing your test results and grouping them by repository, branch, and commit. Git metadata is automatically collected by the test instrumentation from CI provider environment variables and the local .git
folder in the project path, if available.
If you are running tests in non-supported CI providers or with no .git
folder, you can set the Git information manually using environment variables. These environment variables take precedence over any auto-detected information. Set the following environment variables to provide Git information:
DD_GIT_REPOSITORY_URL
git@github.com:MyCompany/MyApp.git
, https://github.com/MyCompany/MyApp.git
DD_GIT_BRANCH
develop
DD_GIT_TAG
1.0.1
DD_GIT_COMMIT_SHA
a18ebf361cc831f5535e58ec4fae04ffd98d8152
DD_GIT_COMMIT_MESSAGE
Set release number
DD_GIT_COMMIT_AUTHOR_NAME
John Smith
DD_GIT_COMMIT_AUTHOR_EMAIL
john@example.com
DD_GIT_COMMIT_AUTHOR_DATE
2021-03-12T16:00:28Z
DD_GIT_COMMIT_COMMITTER_NAME
Jane Smith
DD_GIT_COMMIT_COMMITTER_EMAIL
jane@example.com
DD_GIT_COMMIT_COMMITTER_DATE
2021-03-12T16:00:28Z
Datadog uses special dedicated tags to identify the configuration of the environment in which tests run, including the operating system, runtime, and device information, if applicable. When the same test for the same commit runs in more than one configuration (for example, on Windows and on Linux), the tags are used to differentiate the test in failure and flakiness detection.
You can specify these special tags using the --tags
parameter when calling datadog-ci junit upload
, or by setting the DD_TAGS
environment variable.
All of these tags are optional, and only the ones you specify will be used to differentiate between environment configurations.
test.bundle
ApplicationUITests
, ModelTests
os.platform
windows
, linux
, darwin
os.version
10.15.4
, 14.3.2
, 95
os.architecture
x64
, x86
, arm64
runtime.name
.NET
, .NET Core
, OpenJDK Runtime Environment
, Java(TM) SE Runtime Environment
, CPython
runtime.version
5.0.0
, 3.1.7
runtime.vendor
AdoptOpenJDK
, Oracle Corporation
runtime.architecture
x64
, x86
, arm64
For mobile apps (Swift, Android):
device.model
iPhone11,4
, AppleTV5,3
device.name
iPhone 12 Pro Simulator
, iPhone 13 (QA team)
To add codeowners information to your JUnit XML tests, you can use the GitHub integration to read the CODEOWNERS
file in your repository or provide some additional information manually.
As a result, the JUnit XML tests have a test.codeowners
tag with the owner of those tests.
To automatically add the test.codeowners
tag to your tests, you need to:
Have a CODEOWNERS
file in one of the allowed locations in your repository.
Provide the tests source file in your JUnit XML report. The following plugins do this automatically and add the file
attribute to the <testcase>
or <testsuite>
elements in the XML report:
If the XML does not have the file
attribute, you need to provide the source file manually.
Example of a valid report:
<?xml version="1.0" encoding="UTF-8"?>
<testsuite name="suite">
<testcase name="test_with_file" file="src/commands/junit" />
</testsuite>
Contents: Read
permission so Datadog can read the CODEOWNERS
file. Once enabled, wait a few minutes for the changes to take effect.Note: Github is the only supported Git provider.
The JUnit XML uses a private GitHub App to read the CODEOWNERS
file.
Contents: Read
access.Datadog CI Visibility
.test.source.file
tagThis is an alternative to using the GitHub integration.
For those plugins that do not provide the file
attribute in the XML report, you can provide the test.source.file
tag.
There is no need to provide the exact path to a specific file, you can use any syntax you would use in the CODEOWNERS file
such as src/myTeamFolder
or *.md
.
There are multiple ways to provide the test.source.file
tag:
Using the --tags
parameter or the DD_TAGS
environment variable.
datadog-ci junit upload --service service-name --tags test.source.file:src/myTeamFolder my_report.xml
This adds the test.source.file
tag to all the tests in the report. All of the tests will have the same owner(s).
If you want to provide different source files for the same XML report, you can use property elements or set the file
attribute manually to individual <testcase>
or <testsuite>
elements.
In addition to the --tags
CLI parameter and the DD_TAGS
environment variable, which apply custom tags globally to all tests included the uploaded XML report, the --xpath-tag
parameter provides custom rules to add tags from different attributes within the XML to each test.
The parameter provided must have the format key=expression
, where key
is the name of the custom tag to be added and expression
is a valid XPath expression within the ones supported.
While XPath syntax is used for familiarity, only the following expressions are supported:
/testcase/@attribute-name
<testcase attribute-name="value">
./testcase/../@attribute-name
<testsuite attribute-name="value">
of the current <testcase>
./testcase/..//property[@name='property-name']
value
attribute from the <property name="property-name" value="value">
inside the parent <testsuite>
of the current <testcase>
./testcase//property[@name='property-name']
value
attribute from the <property name="property-name" value="value">
inside the current <testcase>
.Examples:
By default, the test.suite
tag of the tests is read from <testsuite name="suite name">
. However, some plugins might report a better value in <testcase classname="TestSuite">
.
To change test.suite
tags from value 1
, value 2
to SomeTestSuiteClass
, OtherTestSuiteClass
:
<?xml version="1.0" encoding="UTF-8"?>
<testsuites>
<testsuite tests="1" failures="0" time="0.030000" name="value 1">
<testcase classname="SomeTestSuiteClass" name="test_something" time="0.030000"></testcase>
</testsuite>
<testsuite tests="1" failures="0" time="0.021300" name="value 2">
<testcase classname="OtherTestSuiteClass" name="test_something" time="0.021300"></testcase>
</testsuite>
</testsuites>
datadog-ci junit upload --service service_name \
--xpath-tag test.suite=/testcase/@classname ./junit.xml
To add custom_tag
to each test with values value 1
, value 2
:
<?xml version="1.0" encoding="UTF-8"?>
<testsuites>
<testsuite tests="1" failures="0" time="0.020000" name="SomeTestSuiteClass">
<testcase name="test_something" time="0.010000" attr="value 1"></testcase>
<testcase name="test_other" time="0.010000" attr="value 2"></testcase>
</testsuite>
</testsuites>
datadog-ci junit upload --service service_name \
--xpath-tag custom_tag=/testcase/@attr ./junit.xml
To add custom_tag
to each test with values value 1
, value 2
:
<?xml version="1.0" encoding="UTF-8"?>
<testsuites>
<testsuite tests="1" failures="0" time="0.030000" name="SomeTestSuiteClass">
<properties>
<property name="prop" value="value 1"></property>
</properties>
<testcase name="test_something" time="0.030000" attr="value 1"></testcase>
</testsuite>
<testsuite tests="1" failures="0" time="0.021300" name="OtherTestSuiteClass">
<properties>
<property name="prop" value="value 1"></property>
</properties>
<testcase name="test_something" time="0.021300" attr="value 1"></testcase>
</testsuite>
</testsuites>
datadog-ci junit upload --service service_name \
--xpath-tag custom_tag=/testcase/..//property[@name=\'prop\'] ./junit.xml
Note: The name must be in quotes. Bash requires quotes to be escaped using a backslash, for example [@name='prop']
must be entered as `[@name='prop'].
Another way to provide additional tags to specific tests is including <property name="dd_tags[key]" value="value">
elements within the <testsuite>
or <testcase>
elements. If you add these tags to a <testcase>
element, they are stored in its test span. If you add the tags to a <testsuite>
element, they are stored in all of that suite’s test spans.
To be processed, the name
attribute in the <property>
element must have the format dd_tags[key]
, where key
is the name of the custom tag to be added. Other properties are ignored.
Example: Adding tags to a <testcase>
element.
<?xml version="1.0" encoding="UTF-8"?>
<testsuites>
<testsuite tests="1" failures="0" time="0.030000" name="SomeTestSuiteClass">
<testcase classname="SomeTestSuiteClass" name="test_something" time="0.010000">
<properties>
<property name="dd_tags[custom_tag]" value="some value"></property>
<property name="dd_tags[runtime.name]" value="CPython"></property>
</properties>
</testcase>
</testsuite>
</testsuites>
Example: Adding tags to a <testsuite>
element.
<?xml version="1.0" encoding="UTF-8"?>
<testsuites>
<testsuite tests="1" failures="0" time="0.030000" name="SomeTestSuiteClass">
<properties>
<property name="dd_tags[custom_tag]" value="some value"></property>
<property name="dd_tags[runtime.name]" value="CPython"></property>
</properties>
<testcase classname="SomeTestSuiteClass" name="test_something" time="0.010000"></testcase>
</testsuite>
</testsuites>
The values that you send to Datadog are strings, so the facets are displayed in lexicographical order. To send integers instead of strings, use the --measures
flag and the DD_MEASURES
environment variable.
It is possible to report code coverage for a given JUnit report via the --report-measures
option, by setting the test.code_coverage.lines_pct
measure:
datadog-ci junit upload --service my-api-service --report-measures test.code_coverage.lines_pct:82 unit-tests/junit-reports e2e-tests/single-report.xml
For more information, see Code Coverage.