Continuous Testing and Bitrise
Cette page n'est pas encore disponible en français, sa traduction est en cours.
Si vous avez des questions ou des retours sur notre projet de traduction actuel,
n'hésitez pas à nous contacter.
Overview
With the synthetics-test-automation-bitrise-step-run-tests
step, you can run Synthetic tests during your Bitrise CI, ensuring that all your teams using Bitrise can benefit from Synthetic tests at every stage of the software lifecycle. This step uses the Datadog CI Synthetics command.
Setup
This step is not available on the official Bitrise Step Library.
To get started:
- Add the following git URL to your workflow. See the official Bitrise documentation on how to do that though the Bitrise app. You can also configure it locally by referencing the git URL in your
bitrise.yml
file.
- git::https://github.com/DataDog/synthetics-test-automation-bitrise-step-run-tests.git@1.5.0:
- Add your API and application keys to your secrets in Bitrise.
- Configure your step inputs. You can also configure them in your
bitrise.yml
file. The only required inputs are the two secrets you configured earlier. For a comprehensive list of inputs, see the Inputs section.
How to use this step locally
You can run this step directly using the Bitrise CLI.
To run this step locally:
- Open your terminal or command line.
git clone
the Bitrise repository.cd
into the directory of the step (the one you just git clone
d).- Create a
.bitrise.secrets.yml
file in the same directory of bitrise.yml
. The .bitrise.secrets.yml
file is a Git-ignored file, so you can store your secrets in it. - Check the
bitrise.yml
file for any secret you should set in .bitrise.secrets.yml
. - Once you have the required secret parameters in your
.bitrise.secrets.yml
file, run this step with the Bitrise CLI: bitrise run test
.
An example .bitrise.secrets.yml
file:
envs:
- A_SECRET_PARAM_ONE: the value for secret one
- A_SECRET_PARAM_TWO: the value for secret two
Simple usage
Example using public IDs
- git::https://github.com/DataDog/synthetics-test-automation-bitrise-step-run-tests.git@1.5.0:
inputs:
- api_key: <DATADOG_API_KEY>
- app_key: <DATADOG_APP_KEY>
- public_ids: 'abc-d3f-ghi, jkl-mn0-pqr'
Example task using existing synthetics.json
files
- git::https://github.com/DataDog/synthetics-test-automation-bitrise-step-run-tests.git@1.5.0:
inputs:
- api_key: <DATADOG_API_KEY>
- app_key: <DATADOG_APP_KEY>
- files: 'e2e-tests/*.synthetics.json'
For an example test file, see this test.synthetics.json
file.
Complex usage
Example task using the testSearchQuery
- git::https://github.com/DataDog/synthetics-test-automation-bitrise-step-run-tests.git@1.5.0:
inputs:
- api_key: <DATADOG_API_KEY>
- app_key: <DATADOG_APP_KEY>
- test_search_query: 'tag:e2e-tests'
Example task using the testSearchQuery
and variable overrides
- git::https://github.com/DataDog/synthetics-test-automation-bitrise-step-run-tests.git@1.5.0:
inputs:
- api_key: <DATADOG_API_KEY>
- app_key: <DATADOG_APP_KEY>
- test_search_query: 'tag:e2e-tests'
- variables: |
START_URL=https://staging.website.com
PASSWORD=$STAGING_PASSWORD
Example task using a global configuration override with configPath
This task overrides the path to the global datadog-ci.config.json
file.
- git::https://github.com/DataDog/synthetics-test-automation-bitrise-step-run-tests.git@1.5.0:
inputs:
- api_key: <DATADOG_API_KEY>
- app_key: <DATADOG_APP_KEY>
- config_path: './synthetics-config.json'
For an example configuration file, see the global.config.json
file.
Example including all possible configurations
For reference, this is an example of a complete configuration:
- git::https://github.com/DataDog/synthetics-test-automation-bitrise-step-run-tests.git@1.5.0:
inputs:
- api_key: <DATADOG_API_KEY>
- app_key: <DATADOG_APP_KEY>
- batch_timeout: 4200000
- config_path: './global.config.json'
- device_ids: 'apple iphone se (2022),15.4.1, apple iphone 14 pro,16.1'
- fail_on_critical_errors: true
- fail_on_missing_tests: true
- fail_on_timeout: true
- files: 'e2e-tests/*.synthetics.json'
- junit_report: 'e2e-test-junit'
- locations: 'aws:us-west-1'
- mobile_application_version: '01234567-8888-9999-abcd-efffffffffff'
- mobile_application_version_file_path: 'path/to/application.apk'
- public_ids: 'abc-d3f-ghi, jkl-mn0-pqr'
- site: 'datadoghq.com'
- subdomain: 'myorg'
- test_search_query: 'tag:e2e-tests'
- tunnel: true
- variables: |
START_URL=https://staging.website.com
PASSWORD=$STAGING_PASSWORD
Name | Requirement | Description |
---|
apiKey | required | Your Datadog API key. This key is created by your Datadog organization and will be accessed as an environment variable. |
appKey | required | Your Datadog application key. This key is created by your Datadog organization and will be accessed as an environment variable. |
batchTimeout | optional | The duration (in milliseconds) after which the batch fails as timed out. The default is 30 minutes. |
configPath | optional | The global JSON configuration is used when launching tests. See the example configuration for more details. |
deviceIds | optional | Override the mobile device(s) to run your mobile test. |
locations | optional | String of locations separated by semicolons to override the locations where your tests run. |
failOnCriticalErrors | optional | A boolean flag that fails the CI job if no tests were triggered, or results could not be fetched from Datadog. The default is set to false . |
failOnMissingTests | optional | Fail the CI job if at least one specified test with a public ID (using publicIds or listed in a test file) is missing in a run (for example, if it has been deleted programmatically or on the Datadog site). |
failOnTimeout | optional | A boolean flag that fails the CI job if at least one test exceeds the default test timeout. The default is set to true . |
files | optional | Glob patterns to detect Synthetic test configuration files. |
jUnitReport | optional | The filename for a JUnit report if you want to generate one. |
mobileApplicationVersion | optional | Override the default mobile application version for a Synthetic mobile application test. The version must be uploaded and available within Datadog. This version is also outputted by the datadog-mobile-app-upload step. |
mobileApplicationVersionFilePath | optional | Override the application version for Synthetic mobile application tests. |
pollingTimeout | optional | DEPRECATED The duration (in milliseconds) after which datadog-ci stops polling for test results. Please use --batchTimeout instead. |
publicIds | optional | String of public IDs separated by commas for Synthetic tests you want to trigger. |
site | optional | The Datadog site to send data to. If the DD_SITE environment variable is set, it takes precedence. Your Datadog site is . . |
subdomain | optional | The name of the custom subdomain set to access your Datadog application. If the URL used to access Datadog is myorg.datadoghq.com , the subdomain value needs to be set to myorg . |
testSearchQuery | optional | Trigger tests corresponding to a search query. This can be useful if you are tagging your test configurations. See best practices for more information on tagging. |
tunnel | optional | Enable Local and Staging Environments to interact with the Datadog API. |
variables | optional | Key-value pairs for injecting variables into tests. Must be formatted using KEY=VALUE . |
Further reading
Additional helpful documentation, links, and articles: