- 필수 기능
- 시작하기
- Glossary
- 표준 속성
- Guides
- Agent
- 통합
- 개방형텔레메트리
- 개발자
- Administrator's Guide
- API
- Datadog Mobile App
- CoScreen
- Cloudcraft
- 앱 내
- 서비스 관리
- 인프라스트럭처
- 애플리케이션 성능
- APM
- Continuous Profiler
- 스팬 시각화
- 데이터 스트림 모니터링
- 데이터 작업 모니터링
- 디지털 경험
- 소프트웨어 제공
- 보안
- AI Observability
- 로그 관리
- 관리
Test Optimization’s Auto Test Retries feature allows retrying failing tests up to N times to avoid failing your build due to flaky tests: a failing test case is retried either until it passes successfully or until there are no more retry attempts left (in which case the build fails).
Ensure Test Optimization is configured for your test runs.
After you have set up Test Optimization, you can configure Auto Test Retries from the Test Service Settings page.
The default behavior of the feature is to retry any failing test case up to 5 times. This behavior can be fine-tuned with the following environment variables:
DD_CIVISIBILITY_FLAKY_RETRY_ONLY_KNOWN_FLAKES - if this environment variable is set to true, only the test cases that Test Optimization considers flaky are retried.DD_CIVISIBILITY_FLAKY_RETRY_COUNT - can be set to any non-negative number to change the maximum number of retries per test case.After you have set up Test Optimization, you can configure Auto Test Retries from the Test Service Settings page.
The default behavior of the feature is to retry any failing test case up to 5 times. This behavior can be fine-tuned with the following environment variables:
DD_CIVISIBILITY_FLAKY_RETRY_ENABLED - set to 0 or false to explicitly disable retries even if the remote setting is enabled (default: true).DD_CIVISIBILITY_FLAKY_RETRY_COUNT - a non-negative number to change the maximum number of retries per test case (default: 5).jest-image-snapshot is incompatible with jest.retryTimes unless customSnapshotIdentifier is passed (see jest-image-snapshot docs) to toMatchImageSnapshot. Therefore, auto test retries do not work unless customSnapshotIdentifier is used.
After you have set up Test Optimization, you can configure Auto Test Retries from the Test Service Settings page.
The default behavior of the feature is to retry any failing test case up to 5 times. This behavior can be fine-tuned with the following environment variables:
DD_CIVISIBILITY_FLAKY_RETRY_ENABLED - set to 0 or false to explicitly disable retries even if the remote setting is enabled (default: true)DD_CIVISIBILITY_FLAKY_RETRY_COUNT - a non-negative number to change the maximum number of retries per test case (default: 5).DD_CIVISIBILITY_TOTAL_FLAKY_RETRY_COUNT - a non-negative number to set the maximum total number of failed tests to retry (default: 1000)After you set up Test Visibility, you can configure Auto Test Retries from the Test Service Settings page.
By default, the feature retries any failing test case up to 5 times. Customize the Auto Test Retries with the following environment variables:
DD_CIVISIBILITY_FLAKY_RETRY_ENABLED - set to 0 or false to explicitly disable retries even if the remote setting is enabled (default: true)DD_CIVISIBILITY_FLAKY_RETRY_COUNT - a non-negative number to change the maximum number of retries per test case (default: 5).DD_CIVISIBILITY_TOTAL_FLAKY_RETRY_COUNT - a non-negative number to set the maximum total number of failed tests to retry (default: 1000)After you set up Test Visibility, you can configure Auto Test Retries from the Test Service Settings page.
By default, the feature retries each failing test case up to 5 times. Customize the Auto Test Retries with the following environment variables:
DD_CIVISIBILITY_FLAKY_RETRY_ENABLED - set to 0 or false to explicitly disable retries even if the remote setting is enabled (default: true)DD_CIVISIBILITY_FLAKY_RETRY_COUNT - a non-negative number to change the maximum number of retries per test case (default: 5).DD_CIVISIBILITY_TOTAL_FLAKY_RETRY_COUNT - a non-negative number to set the maximum total number of failed tests to retry (default: 1000)You can query the retried tests in the Test Optimization Explorer: they have the @test.is_retry tag set to true (some of them may also have the @test.is_new set to true, which indicates they have been retried by the Early Flakiness Detection feature).
If you suspect there are any issues with Auto Test Retries, navigate to the Test Service Settings page, look for your test service, and click Configure. Disable Auto Test Retries by clicking on the toggle.
| |
