Compatibility
dd-trace-java >= 1.34.0
Configuration
After you have set up Test Optimization, you can configure Auto Test Retries from the Test Service Settings page.
The default behavior of the feature is to retry any failing test case up to 5 times.
This behavior can be fine-tuned with the following environment variables:
DD_CIVISIBILITY_FLAKY_RETRY_ONLY_KNOWN_FLAKES - if this environment variable is set to true, only the test cases that Test Optimization considers flaky are retried.DD_CIVISIBILITY_FLAKY_RETRY_COUNT - can be set to any non-negative number to change the maximum number of retries per test case.
Compatibility
dd-trace-js >= v5.19.0 and dd-trace-js >= v4.43.0
Configuration
After you have set up Test Optimization, you can configure Auto Test Retries from the Test Service Settings page.
The default behavior of the feature is to retry any failing test case up to 5 times.
This behavior can be fine-tuned with the following environment variables:
DD_CIVISIBILITY_FLAKY_RETRY_ENABLED - set to 0 or false to explicitly disable retries even if the remote setting is enabled (default: true).DD_CIVISIBILITY_FLAKY_RETRY_COUNT - a non-negative number to change the maximum number of retries per test case (default: 5).
Known limitations
jest-image-snapshot is incompatible with jest.retryTimes unless customSnapshotIdentifier is passed (see jest-image-snapshot docs) to toMatchImageSnapshot. Therefore, auto test retries do not work unless customSnapshotIdentifier is used.
Compatibility
datadog-ci-rb >= 1.4.0
Configuration
After you have set up Test Optimization, you can configure Auto Test Retries from the Test Service Settings page.
The default behavior of the feature is to retry any failing test case up to 5 times.
This behavior can be fine-tuned with the following environment variables:
DD_CIVISIBILITY_FLAKY_RETRY_ENABLED - set to 0 or false to explicitly disable retries even if the remote setting is enabled (default: true)DD_CIVISIBILITY_FLAKY_RETRY_COUNT - a non-negative number to change the maximum number of retries per test case (default: 5).DD_CIVISIBILITY_TOTAL_FLAKY_RETRY_COUNT - a non-negative number to set the maximum total number of failed tests to retry (default: 1000)
Compatibility
dd-trace-dotnet >= 3.4.0
Configuration
After you set up Test Visibility, you can configure Auto Test Retries from the Test Service Settings page.
By default, the feature retries any failing test case up to 5 times.
Customize the Auto Test Retries with the following environment variables:
DD_CIVISIBILITY_FLAKY_RETRY_ENABLED - set to 0 or false to explicitly disable retries even if the remote setting is enabled (default: true)DD_CIVISIBILITY_FLAKY_RETRY_COUNT - a non-negative number to change the maximum number of retries per test case (default: 5).DD_CIVISIBILITY_TOTAL_FLAKY_RETRY_COUNT - a non-negative number to set the maximum total number of failed tests to retry (default: 1000)
Test optimization for Go is in Preview.
Compatibility
orchestrion >= 0.9.4 + dd-trace-go >= 1.69.1
Configuration
After you set up Test Visibility, you can configure Auto Test Retries from the Test Service Settings page.
By default, the feature retries each failing test case up to 5 times.
Customize the Auto Test Retries with the following environment variables:
DD_CIVISIBILITY_FLAKY_RETRY_ENABLED - set to 0 or false to explicitly disable retries even if the remote setting is enabled (default: true)DD_CIVISIBILITY_FLAKY_RETRY_COUNT - a non-negative number to change the maximum number of retries per test case (default: 5).DD_CIVISIBILITY_TOTAL_FLAKY_RETRY_COUNT - a non-negative number to set the maximum total number of failed tests to retry (default: 1000)
Auto Test Retries is available using the beta of the new pytest plugin. Set the DD_PYTEST_USE_NEW_PLUGIN_BETA environment variable to true to enable it.
Compatibility
dd-trace-py >= 2.18.0 (pytest >= 7.2.0)
Configuration
After you have set up Test Optimization, you can configure Auto Test Retries from the Test Service Settings page.
The default behavior of the feature is to retry any failing test case up to five times. Tests that originally fail either the original setup, teardown, or fixtures in Pytest, are not retried.
You can fine tune this behavior with the following environment variables:
DD_CIVISIBILITY_FLAKY_RETRY_ENABLED - set to 0 or false to explicitly disable retries even if the remote setting is enabled (default: true)DD_CIVISIBILITY_FLAKY_RETRY_COUNT - a non-negative number to change the maximum number of retries per test case (default: 5).DD_CIVISIBILITY_TOTAL_FLAKY_RETRY_COUNT - a non-negative number to set the maximum total number of failed tests to retry (default: 1000)
Compatibility
dd-sdk-swift-testing >= 2.5.2
Configuration
After you have set up Test Optimization, you can configure Auto Test Retries from the Test Service Settings page.
The default behavior of the feature is to retry any failing test case up to 5 times.
This behavior can be fine-tuned with the following environment variables:
DD_CIVISIBILITY_FLAKY_RETRY_ENABLED - set to 0 or false to explicitly disable retries even if the remote setting is enabled (default: true)DD_CIVISIBILITY_FLAKY_RETRY_COUNT - a non-negative number to change the maximum number of retries per test case (default: 5).DD_CIVISIBILITY_TOTAL_FLAKY_RETRY_COUNT - a non-negative number to set the maximum total number of failed tests to retry (default: 1000)
