Navigate to Software Delivery > Test Visibility > Test Runs to see your CI test run results across the following levels: Session, Module, Suite, and Test.
The Test panel on the left lists default facets you can use to search for your tests.
| Facet | Description |
|---|
| Test Status | The test status: Passed or Failed. |
| Duration | Length of time for the test to complete. |
| Test Service | The test service instrumented with CI Visibility. |
| Test Name | The name of the test. |
| Test Suite | The name of the service. |
| Flaky | Whether or not a test is a flaky test: true or false, meaning that the test run has flaked in the commit.
Either this test run has failed and there was a previous test run for the same test and commit that passed, or this test run passed and there is a previous failure. This attribute is only present the first time a test is detected flaky in a commit. |
| Has Parameters | Whether or not a test has parameters: true or false. |
| Known Flaky | Whether or not a test is known to be flaky: true or false.
The test run has flaked in the commit (according to the “Flaky” definition) and it was already present on the “Flaky test” table of either the branch of the test run or the default branch of the repository. This means that the test was detected as flaky in the past, and that this failure may not be the result of a change in this commit. |
| Language | The language of the service. |
| New Flaky | Whether or not this flaky test has occurred before: true or false.
The test run has flaked in the commit (according to the “Flaky” description) and didn’t previously exist in the Flaky Tests table for the branch of the test run or default branch of the repository. This also means that the test has been added to the Flaky Tests table. |
| Performance Regression | Whether or not a test is a performance regression: true or false. |
| Baseline Mean | The mean duration of the same test in the default branch calculated over the last week of test runs. |
| Baseline Standard Deviation | The standard deviation of the same test in the default branch calculated over the last week of test runs durations. |
| Absolute Change | The absolute difference between the test duration and the baseline mean. |
| Relative Change | The relative difference between the test duration and the baseline mean. |
| Standard Deviation Change | The number of standard deviations above the baseline mean. |
| Test Code Owners | The team name of the service codeowners. |
| Test Fingerprint | The fingerprint of the test. |
| Test Framework | The framework of the test. |
| Test Full Name | The full name of the test. |
| Test Module | The module of the test. |
| Test Traits | The traits of the test such as category:flaky. |
| Test Type | The type of the test such as test. |
You can filter by test level: session, module, suite, and test run. Each test level represents a different level of aggregation of tests.
Sessions
Test sessions are the highest level of aggregation. They correspond one to one to a test command, such as yarn test, mvn test, or dotnet test.
For JUnit report uploads there is 1 session per report file uploaded.
Module
The definition of module changes slightly per language:
- In .NET, a test module groups every test that is run under the same unit test project.
- In Swift, a test module groups every test that is run for a given bundle.
- In JavaScript, the test modules map one-to-one to test sessions.
- In Java, a test module groups every test that is run by the same Maven Surefire/Failsafe or Gradle Test task execution.
- In JUnit report uploads, the test modules map one-to-one to test sessions.
An example of a module is SwiftLintFrameworkTests, which corresponds to a test target in SwiftLint.
Suite
A test suite is a group of tests exercising the same unit of code.
An example of a test suite is src/commands/junit/__tests__/upload.test.ts, which corresponds to a test file in datadog-ci.
Test run data is available in dashboards and notebooks, enabling build engineering teams to customize their communication about high-priority work and CI trends over time.
Navigate to Software Delivery > CI Visibility > Executions to see your CI pipeline execution results across the following levels: Pipeline, Stage, Job, Step, and Command.
The CI panel on the left lists default facets you can use to search for your tests.
| Facet | Description |
|---|
| CI Status | The pipeline status: Success or Failure. |
| Duration | Length of time for the pipeline to execute. |
| Pipeline ID | The ID of the pipeline. |
| CI Provider | The name of the CI provider. |
| Node Labels | The labels of the node. |
| Node Name | The name of the node. |
| Partial Retry | Whether or not a partial retry is enabled: true or false. |
| Pipeline Number | The number of the pipeline. |
| Pipeline URL | The URL of the pipeline. |
| Queue time | Length of time for the pipeline wait for the queue. |
Pipeline executions details and traces
You can see aggregated data about pipeline executions over the selected time frame. Use the search field and facets to scope the list down to the executions you want to investigate. Change the list to show pipelines, stages, or jobs using the buttons at the top.
Below are three graphs that visualize the durations of your most active pipelines, your failed pipelines over time, and the executions of your pipelines with an option to toggle to accumulated duration, respectively. These graphs are scoped to the level chosen at the top left (Pipeline, Stage, Job, and more.)
Each pipeline execution is reported as a trace, which includes stage and job information. Access individual pipeline, stage, and job execution traces by clicking on an execution in the list (similar to clicking into a pipeline execution from the Pipeline Details view).
CI pipeline data is available in dashboards and notebooks, enabling build engineering teams to customize their communication about high-priority work and CI trends over time.
