Continuous Testing and CircleCI Orb
Overview

Run Synthetic tests in your CircleCI pipelines using the Datadog CircleCI orb.
The CircleCI command orb installs datadog-ci and uses the datadog-ci synthetics run-tests
command to execute Datadog Synthetic tests.
Setup
To get started:
- Add your Datadog API and application keys as environment variables to your CircleCI project. For more information, see API and Application Keys.
- Ensure the image running the orb is a Linux x64 base image with cURL installed.
- Customize your workflow by creating a
run-tests.yml
file and following the naming conventions to specify inputs for your workflow.
Your workflow can be simple or complex.
Simple usage
Example orb usage using public IDs
version: 2.1
orbs:
synthetics-ci: datadog/synthetics-ci-orb@2.2.0
jobs:
e2e-tests:
docker:
- image: cimg/base:stable
steps:
- synthetics-ci/run-tests:
public_ids: 'abc-d3f-ghi, jkl-mn0-pqr'
workflows:
run-tests:
jobs:
- e2e-tests
Example orb usage using a global configuration override
This orb overrides the path to the pattern for test files.
version: 2.1
orbs:
synthetics-ci: datadog/synthetics-ci-orb@2.2.0
jobs:
e2e-tests:
docker:
- image: cimg/base:stable
steps:
- synthetics-ci/run-tests:
files: e2e-tests/*.synthetics.json
workflows:
run-tests:
jobs:
- e2e-tests
For another example pipeline that triggers Synthetic tests, see the simple-example.yml
file.
Complex usage
Example orb usage using the test_search_query
version: 2.1
orbs:
synthetics-ci: datadog/synthetics-ci-orb@2.2.0
jobs:
e2e-tests:
docker:
- image: cimg/base:stable
steps:
- synthetics-ci/run-tests:
test_search_query: 'tag:e2e-tests'
workflows:
run-tests:
jobs:
- e2e-tests
version: 2.1
orbs:
synthetics-ci: datadog/synthetics-ci-orb@2.2.0
jobs:
e2e-tests:
docker:
- image: your-image
steps:
- checkout
- run:
name: Running server in background
command: npm start
background: true
- synthetics-ci/run-tests:
config_path: tests/tunnel-config.json
files: tests/*.synthetics.json
test_search_query: 'tag:e2e-tests'
tunnel: true
workflows:
test-server:
jobs:
- build-image
- integration-tests:
requires:
- build-image
For additional options such as customizing the pollingTimeout
for your CircleCI pipelines, see CI/CD Integrations Configuration. For another example pipeline that starts a local server and triggers Synthetic tests using the Continuous Testing Tunnel, see the advanced-example.yml
file.
To customize your workflow, you can set the following parameters in a run-tests.yml
file:
Name | Type | Default | Description |
---|
api_key | env var name | DATADOG_API_KEY | The name of the environment variable containing the API key. |
api_key | env var name | DATADOG_APP_KEY | The name of the environment variable containing the app key. |
config_path | string | datadog-ci.json | The global JSON configuration used when launching tests. |
fail_on_critical_errors | boolean | false | Fail if tests were not triggered or results could not be fetched. |
fail_on_timeout | boolean | true | Force the CI to fail (or pass) if one of the results exceeds its test timeout. |
files | string | {,!(node_modules)/**/}*.synthetics.json | Glob pattern to detect Synthetic tests config files. |
junit_report | string | none | The filename for a JUnit report if you want to generate one. |
locations | string | values in test config files | String of locations separated by semicolons to override the locations where your tests run. |
public_ids | string | values in test config files | String of public IDs separated by commas for Synthetic tests you want to trigger. |
site | string | datadoghq.com | The Datadog site to send data to. If the DD_SITE environment variable is set, it takes preference. |
subdomain | string | app | The name of the custom subdomain set to access your Datadog application. |
test_search_query | string | none | Trigger tests corresponding to a search query. |
tunnel | boolean | false | Use the Continuous Testing Tunnel to trigger tests. |
variables | string | none | Key-value pairs for injecting variables into tests. Must be formatted using KEY=VALUE . |
To customize parameters such as pollingTimeout
and learn about additional options for your CircleCI pipelines, see Continuous Testing & CI/CD Integrations Configuration.
Further reading
Additional helpful documentation, links, and articles: