Test Visibility in Datadog
CI Visibility is not available in the selected site () at this time.
Overview
Test Visibility provides a test-first view into your CI health by displaying important metrics and results from your tests. It can help you investigate performance problems and test failures that concern you the most because you work on the related code, not because you maintain the pipelines they are run in.
Supported features
| .NET | Java | Javascript | Python | Ruby | Swift | JUnit Xml |
|---|
| Accurate time/durations results | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✗ |
| Structured stack traces | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✗ |
| Distributed traces on integration tests | ✓ | ✓ | ✓ | ✗ | ✓ | ✓ | ✗ |
| Agent-based reporting | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✗ |
| Agentless reporting | ✓ | ✓ | ✓ | ✓ | ✗ | ✓ | ✓ |
| Test suite level visibility | ✓ | ✓ | ✓ | ✓ | ✗ | ✓ | ✓ |
| Manual API | ✓ | ✓ | ✓ | ✗ | ✗ | ✓ | ✗ |
| Codeowner by test | ✓ | ✓ | ✓ | ✗ | ✗ | ✓ | ✗ |
| Source code start/end reporting | ✓ | ✓ | ✓ (only start) | ✗ | ✗ | ✓ | ✓ (only start) |
| CI and Git info | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ |
| Git metadata upload | ✓ | ✓ | ✓ | ✓ | ✗ | ✓ | ✓ |
| Intelligent Test Runner | ✓ | ✓ | ✓ | ✓ | ✗ | ✓ | ✗ |
| Code coverage support | ✓ | ✓ | ✓ | ✗ | ✗ | ✓ | ✗ |
| Benchmark tests support | ✓ | ✗ | ✗ | ✓ | ✗ | ✓ | ✗ |
| Parameterized tests | ✓ | ✓ | ✓ | ✓ | ✗ | ✓ | ✗ |
Setup
Choose a language to set up Test Visibility in Datadog:
Test suite level visibility
In addition to tests, CI Visibility provides visibility over the whole testing phase of your project.
Compatibility
Not every language supported by CI Visibility has support for test suite level visibility:
- Swift has complete support since
dd-sdk-swift-testing>=2.1.0. - .NET has complete support since
dd-trace-dotnet>2.16.0. - JavaScript has limited support since
dd-trace-js>=3.3.0. - Java has complete support since
dd-trace-java>=1.12.0. - JUnit report uploads has complete support since
datadog-ci>=2.17.0. - Python has complete support since
dd-trace-py>=1.14.0
Use CI tests data
When creating a dashboard or a notebook, you can use test execution data in your search query, which updates the visualization widget options.
Alert on test data
When you evaluate failed or flaky tests, or the performance of a CI test on the Test Runs page, click Create Monitor to create a CI Test monitor.
Further reading
Additional helpful documentation, links, and articles:
