Continuous Integration Visibility Explorer

Cette page n'est pas encore disponible en français, sa traduction est en cours.
Si vous avez des questions ou des retours sur notre projet de traduction actuel, n'hésitez pas à nous contacter.

Overview

The CI Visibility Explorer allows you to search and filter, visualize, and export test runs or pipeline executions at multiple levels using any tag.

Navigate to CI > Test Runs to see your CI test run results across the following levels: Session, Module, Suite, and Test.

The Test panel on the left lists default facets you can use to search for your tests.

FacetDescription
Test StatusThe test status: Passed or Failed.
DurationLength of time for the test to complete.
Test ServiceThe test service instrumented with CI Visibility.
Test NameThe name of the test.
Test SuiteThe name of the service.
FlakyWhether or not a test is a flaky test: true or false, meaning that the test run has flaked in the commit.

Either this test run has failed and there was a previous test run for the same test and commit that passed, or this test run passed and there is a previous failure. This attribute is only present the first time a test is detected flaky in a commit.
Has ParametersWhether or not a test has parameters: true or false.
Known FlakyWhether or not a test is known to be flaky: true or false.

The test run has flaked in the commit (according to the “Flaky” definition) and it was already present on the “Flaky test” table of either the branch of the test run or the default branch of the repository. This means that the test was detected as flaky in the past, and that this failure may not be the result of a change in this commit.
LanguageThe language of the service.
New FlakyWhether or not this flaky test has occurred before: true or false.

The test run has flaked in the commit (according to the “Flaky” description) and didn’t previously exist in the Flaky Tests table for the branch of the test run or default branch of the repository. This also means that the test has been added to the Flaky Tests table.
Performance RegressionWhether or not a test is a performance regression: true or false.
Baseline MeanThe mean duration of the same test in the default branch calculated over the last week of test runs.
Baseline Standard DeviationThe standard deviation of the same test in the default branch calculated over the last week of test runs durations.
Absolute ChangeThe absolute difference between the test duration and the baseline mean.
Relative ChangeThe relative difference between the test duration and the baseline mean.
Standard Deviation ChangeThe number of standard deviations above the baseline mean.
Test Code OwnersThe team name of the service codeowners.
Test FingerprintThe fingerprint of the test.
Test FrameworkThe framework of the test.
Test Full NameThe full name of the test.
Test ModuleThe module of the test.
Test TraitsThe traits of the test such as category:flaky.
Test TypeThe type of the test such as test.

You can filter by test level: session, module, suite, and test run. Each test level represents a different level of aggregation of tests.

Test Suite Visibility

Sessions

Test sessions are the highest level of aggregation. They correspond one to one to a test command, such as yarn test, mvn test, or dotnet test.

For JUnit report uploads there is 1 session per report file uploaded.

Module

The definition of module changes slightly per language:

  • In .NET, a test module groups every test that is run under the same unit test project.
  • In Swift, a test module groups every test that is run for a given bundle.
  • In JavaScript, the test modules map one-to-one to test sessions.
  • In Java, a test module groups every test that is run by the same Maven Surefire/Failsafe or Gradle Test task execution.
  • In JUnit report uploads, the test modules map one-to-one to test sessions.

An example of a module is SwiftLintFrameworkTests, which corresponds to a test target in SwiftLint.

Suite

A test suite is a group of tests exercising the same unit of code.

An example of a test suite is src/commands/junit/__tests__/upload.test.ts, which corresponds to a test file in datadog-ci.

Test run data is available in dashboards and notebooks, enabling build engineering teams to customize their communication about high-priority work and CI trends over time.

Navigate to CI > Pipeline Executions to see your CI pipeline execution results across the following levels: Pipeline, Stage, Job, Step, and Command.

The CI panel on the left lists default facets you can use to search for your tests.

FacetDescription
CI StatusThe pipeline status: Success or Failure.
DurationLength of time for the pipeline to execute.
Pipeline IDThe ID of the pipeline.
CI ProviderThe name of the CI provider.
Node LabelsThe labels of the node.
Node NameThe name of the node.
Partial RetryWhether or not a partial retry is enabled: true or false.
Pipeline NumberThe number of the pipeline.
Pipeline URLThe URL of the pipeline.
Queue timeLength of time for the pipeline wait for the queue.

Pipeline executions details and traces

You can see aggregated data about pipeline executions over the selected time frame. Use the search field and facets to scope the list down to the executions you want to investigate. Change the list to show pipelines, stages, or jobs using the buttons at the top.

Below are three graphs that visualize the durations of your most active pipelines, your failed pipelines over time, and the executions of your pipelines with an option to toggle to accumulated duration, respectively. These graphs are scoped to the level chosen at the top left (Pipeline, Stage, Job, and more.)

Explorer view trend graphs for Duration, Errored, and Executions

Each pipeline execution is reported as a trace, which includes stage and job information. Access individual pipeline, stage, and job execution traces by clicking on an execution in the list (similar to clicking into a pipeline execution from the Pipeline Details page).

CI pipeline data is available in dashboards and notebooks, enabling build engineering teams to customize their communication about high-priority work and CI trends over time.

Search and filter

You can narrow down, broaden, or shift your focus on a subset of test runs or pipeline executions by clicking on the facets to the left or writing your own custom query in the search bar. When you select and deselect facets, the search bar automatically reflects your changes. Similarly, you can modify the search bar query or write a query from scratch in the search bar to select and deselect the facets on the left.

Analyze

Group your queried test runs or pipeline executions into higher-level entities such as fields, patterns, and transactions in order to derive or consolidate information. By using facets, which you do not need to create to search for attributes, you can accomplish the following actions:

  • Search and keep track of the progress of tests running in a CI/CD pipeline.
  • Investigate every CI/CD job execution to identify and troubleshoot failing test runs.
  • Identify flaky tests to fix.

Visualize

Select a visualization type to visualize the outcomes of your filters and aggregations and better understand your test runs or pipeline executions. For example, you can view your test results in a list to organize your test data into columns, or in a timeseries graph to measure your pipeline data over time.

Export

Export your view in the CI Visibility Explorer to reuse it later or in different contexts.

Further reading