CI Visibility is not available in the selected site () at this time.
Compatibility
Supported test frameworks:
JUnit >= 4.10 and >= 5.3
Also includes any test framework based on JUnit, such as Spock Framework and Cucumber-Junit. Note: Only Cucumber v1, which uses JUnit 4, is supported.
TestNG >= 6.4
Supported build systems:
Gradle >= 2.0
Maven >= 3.2.1
Configuring reporting method
To report test results to Datadog, you need to configure the Datadog Java library:
If you are running tests on an on-premises CI provider, such as Jenkins or self-managed GitLab CI, install the Datadog Agent on each worker node by following the Agent installation instructions. This is the recommended option as test results are then automatically linked to the underlying host metrics.
If the CI provider is using a container-based executor, set the DD_AGENT_HOST environment variable on all builds (which defaults to http://localhost:8126) to an endpoint that is accessible from within build containers, as using localhost inside the build references the container itself and not the underlying worker node where the Datadog Agent is running.
If you are using a Kubernetes executor, Datadog recommends using the Datadog Admission Controller, which automatically sets the DD_AGENT_HOST environment variable in the build pods to communicate with the local Datadog Agent.
Agentless mode is available in Datadog Java library versions >= 0.101.0
If you are using a cloud CI provider without access to the underlying worker nodes, such as GitHub Actions or CircleCI, configure the library to use the Agentless mode. For this, set the following environment variables:
DD_CIVISIBILITY_AGENTLESS_ENABLED=true (Required)
Enables or disables Agentless mode. Default: false
DD_API_KEY (Required)
The Datadog API key used to upload the test results. Default: (empty)
Additionally, configure which Datadog site to which you want to send data.
DD_SITE (Required)
The Datadog site to upload results to. Default: datadoghq.com Selected site:
Downloading tracer library
You only need to download the tracer library once for each server.
If the tracer library is already available locally on the server, you can proceed directly to running the tests.
Declare DD_TRACER_VERSION variable with the latest version of the artifacts accessible from the Maven Repository (without the preceding v: ):
DD_TRACER_VERSION=... // e.g. 1.14.0
Run the command below to download the tracer JAR to your local Maven repository:
Make sure to set the DD_TRACER_VERSION environment variable to the tracer version you have previously downloaded.
Run your tests using the MAVEN_OPTS environment variable to specify the path to the Datadog Java Tracer JAR.
When specifying tracer arguments, include the following:
Enable CI visibility by setting the dd.civisibility.enabled property to true.
Define the environment where the tests are being run using the dd.env property (for example local when running tests on a developer workstation or ci when running them on a CI provider).
Define the name of the service or library being tested in the dd.service property.
Make sure to set the DD_TRACER_VERSION environment variable to the tracer version you have previously downloaded, and the DD_TRACER_FOLDER variable to the path where you have downloaded the tracer.
Run your tests using the org.gradle.jvmargs system property to specify the path to the Datadog Java Tracer JAR.
When specifying tracer arguments, include the following:
Enable CI visibility by setting the dd.civisibility.enabled property to true.
Define the environment where the tests are being run using the dd.env property (for example local when running tests on a developer workstation or ci when running them on a CI provider).
Define the name of the service or library being tested in the dd.service property.
For example:
./gradlew cleanTest test -Pdd-civisibility --rerun-tasks -Dorg.gradle.jvmargs=\
-javaagent:$DD_TRACER_FOLDER/dd-java-agent-$DD_TRACER_VERSION.jar=\
dd.civisibility.enabled=true,\
dd.env=ci,\
dd.service=my-java-app
Specifying org.gradle.jvmargs in the command line overrides the value specified elsewhere. If you have this property specified in a gradle.properties file, be sure to replicate the necessary settings in the command line invocation.
Note: CI Visibility is not compatible with Gradle Configuration Cache, so do not enable the cache when running your tests with the tracer.
Adding custom tags to tests
You can add custom tags to your tests by using the current active span:
// inside your test
finalSpanspan=GlobalTracer.get().activeSpan();if(span!=null){span.setTag("test_owner","my_team");}// test continues normally
// ...
To create filters or group by fields for these tags, you must first create facets. For more information about adding tags, see the Adding Tags section of the Java custom instrumentation documentation.
Manual testing API
If you use one of the supported testing frameworks, the Java Tracer automatically instruments your tests and sends the results to the Datadog backend.
If you are using a framework that is not supported, or an ad-hoc testing solution, you can harness the manual testing API, which also reports test results to the backend.
Adding manual API dependency
Manual API classes are available in the com.datadoghq:dd-trace-api artifact.
Add the trace API dependency to your Maven project, replacing $VERSION with the latest version of the tracer accessible from the Maven Repository (without the preceding v: ):
Add the trace API dependency to your Maven project, replacing $VERSION with the latest version of the tracer accessible from the Maven Repository (without the preceding v: ):
The API is based around four concepts: test session, test module, test suite, and test.
Test session
A test session represents a project build, which typically corresponds to execution of a test command issued by a user or by a CI script.
To start a test session, call datadog.trace.api.civisibility.CIVisibility#startSession and pass the name of the project and the name of the testing framework you used.
When all your tests have finished, call datadog.trace.api.civisibility.DDTestSession#end, which forces the library to send all remaining test results to the backend.
Test module
A test module represents a smaller unit of work within a project build, typically corresponding to a project module. For example, a Maven submodule or Gradle subproject.
To start a test mode, call datadog.trace.api.civisibility.DDTestSession#testModuleStart and pass the name of the module.
When the module has finished building and testing, call datadog.trace.api.civisibility.DDTestModule#end.
Test Suite
A test suite comprises a set of tests that share common functionality.
They can share a common initialization and teardown, and can also share some variables.
A single suite usually corresponds to a Java class that contains test cases.
Create test suites in a test module by calling datadog.trace.api.civisibility.DDTestModule#testSuiteStart and passing the name of the test suite.
Call datadog.trace.api.civisibility.DDTestSuite#end when all the related tests in the suite have finished their execution.
Test
A test represents a single test case that is executed as part of a test suite.
Usually it corresponds to a method that contains testing logic.
Create tests in a suite by calling datadog.trace.api.civisibility.DDTestSuite#testStart and passing the name of the test.
Call datadog.trace.api.civisibility.DDTest#end when a test has finished execution.
Code Example
The following code represents a simple usage of the API:
packagecom.datadog.civisibility.example;importdatadog.trace.api.civisibility.CIVisibility;importdatadog.trace.api.civisibility.DDTest;importdatadog.trace.api.civisibility.DDTestModule;importdatadog.trace.api.civisibility.DDTestSession;importdatadog.trace.api.civisibility.DDTestSuite;importjava.lang.reflect.Method;// the null arguments in the calls below are optional startTime/endTime values:
// when they are not specified, current time is used
publicclassManualTest{publicstaticvoidmain(String[]args)throwsException{DDTestSessiontestSession=CIVisibility.startSession("my-project-name","my-test-framework",null);testSession.setTag("my-tag","additional-session-metadata");try{runTestModule(testSession);}finally{testSession.end(null);}}privatestaticvoidrunTestModule(DDTestSessiontestSession)throwsException{DDTestModuletestModule=testSession.testModuleStart("my-module",null);testModule.setTag("my-module-tag","additional-module-metadata");try{runFirstTestSuite(testModule);runSecondTestSuite(testModule);}finally{testModule.end(null);}}privatestaticvoidrunFirstTestSuite(DDTestModuletestModule)throwsException{DDTestSuitetestSuite=testModule.testSuiteStart("my-suite",ManualTest.class,null);testSuite.setTag("my-suite-tag","additional-suite-metadata");try{runTestCase(testSuite);}finally{testSuite.end(null);}}privatestaticvoidrunTestCase(DDTestSuitetestSuite)throwsException{MethodmyTestCaseMethod=ManualTest.class.getDeclaredMethod("myTestCase");DDTestddTest=testSuite.testStart("myTestCase",myTestCaseMethod,null);ddTest.setTag("my-test-case-tag","additional-test-case-metadata");ddTest.setTag("my-test-case-tag","more-test-case-metadata");try{myTestCase();}catch(Exceptione){ddTest.setErrorInfo(e);// pass error info to mark test case as failed
}finally{ddTest.end(null);}}privatestaticvoidmyTestCase()throwsException{// run some test logic
}privatestaticvoidrunSecondTestSuite(DDTestModuletestModule){DDTestSuitesecondTestSuite=testModule.testSuiteStart("my-second-suite",ManualTest.class,null);secondTestSuite.setSkipReason("this test suite is skipped");// pass skip reason to mark test suite as skipped
secondTestSuite.end(null);}}
Always call datadog.trace.api.civisibility.DDTestSession#end at the end so that all the test info is flushed to Datadog.
Datadog uses Git information for visualizing your test results and grouping them by repository, branch, and commit. Git metadata is automatically collected by the test instrumentation from CI provider environment variables and the local .git folder in the project path, if available.
If you are running tests in non-supported CI providers or with no .git folder, you can set the Git information manually using environment variables. These environment variables take precedence over any auto-detected information. Set the following environment variables to provide Git information:
DD_GIT_REPOSITORY_URL
URL of the repository where the code is stored. Both HTTP and SSH URLs are supported. Example: git@github.com:MyCompany/MyApp.git, https://github.com/MyCompany/MyApp.git
DD_GIT_BRANCH
Git branch being tested. Leave empty if providing tag information instead. Example: develop
DD_GIT_TAG
Git tag being tested (if applicable). Leave empty if providing branch information instead. Example: 1.0.1
DD_GIT_COMMIT_SHA
Full commit hash. Example: a18ebf361cc831f5535e58ec4fae04ffd98d8152
DD_GIT_COMMIT_MESSAGE
Commit message. Example: Set release number
DD_GIT_COMMIT_AUTHOR_NAME
Commit author name. Example: John Smith
DD_GIT_COMMIT_AUTHOR_EMAIL
Commit author email. Example: john@example.com
DD_GIT_COMMIT_AUTHOR_DATE
Commit author date in ISO 8601 format. Example: 2021-03-12T16:00:28Z
DD_GIT_COMMIT_COMMITTER_NAME
Commit committer name. Example: Jane Smith
DD_GIT_COMMIT_COMMITTER_EMAIL
Commit committer email. Example: jane@example.com
DD_GIT_COMMIT_COMMITTER_DATE
Commit committer date in ISO 8601 format. Example: 2021-03-12T16:00:28Z
Information collected
When CI Visibility is enabled, the following data is collected from your project:
Test names and durations.
Predefined environment variables set by CI providers.
Git commit history including the hash, message, author information, and files changed (without file contents).
Source code information: relative paths to sources of test classes, line numbers of test methods.
Information from the CODEOWNERS file.
Troubleshooting
The tests are not appearing in Datadog after enabling CI Visibility in the tracer
Ensure that you are using the latest version of the tracer.
Ensure that the dd.civisibility.enabled property is set to true in the tracer arguments.
Check the build output for any errors that indicate tracer misconfiguration, such as an unset DD_API_KEY environment variable.
Tests or source code compilation fails when building a project with the tracer attached
By default, CI Visibility runs Java code compilation with a compiler plugin attached.
The plugin is optional, as it only serves to reduce the performance overhead.
Depending on the build configuration, adding the plugin can sometimes disrupt the compilation process.
If the plugin interferes with the build, disable it by adding dd.civisibility.compiler.plugin.auto.configuration.enabled=false to the list of -javaagent arguments.
Tests fail when building a project with the tracer attached
In some cases attaching the tracer can break tests, especially if they run asserts on the internal state of the JVM or instances of third-party libraries’ classes.
While the best approach is such cases is to update the tests, there is also a quicker option of disabling the tracer’s third-party library integrations.
The integrations provide additional insights into what happens in the tested code and are especially useful in integration tests, to monitor things like HTTP requests or database calls.
They are enabled by default.
To disable a specific integration, refer to the Datadog Tracer Compatibility table for the relevant configuration property names.
For example, to disable OkHttp3 client request integration, add dd.integration.okhttp-3.enabled=false to the list of -javaagent arguments.
To disable all integrations, augment the list of -javaagent arguments with dd.trace.enabled=false.
Further reading
Additional helpful documentation, links, and articles: