Continuous Testing and CircleCI Orb

Docs > Tests continuos > Continuous Testing y CI/CD > Continuous Testing and CircleCI Orb

This page is not yet available in Spanish. We are working on its translation.
If you have any questions or feedback about our current translation project, feel free to reach out to us!

Overview

Run Synthetic tests in your CircleCI pipelines using the Datadog CircleCI orb.

The CircleCI command orb installs datadog-ci and uses the datadog-ci synthetics run-tests command to execute Datadog Synthetic tests.

Setup

To get started:

Add your Datadog API and application keys as environment variables to your CircleCI project. For more information, see API and Application Keys.
Ensure the image running the orb is a Linux x64 base image with cURL installed.
Customize your workflow by creating a run-tests.yml file and following the naming conventions to specify inputs for your workflow.

Your workflow can be simple or complex.

Simple usage

Example orb usage using public IDs

version: 2.1

orbs:
  synthetics-ci: datadog/synthetics-ci-orb@3.20.0

jobs:
  e2e-tests:
    docker:
      - image: cimg/base:stable
    steps:
      - synthetics-ci/run-tests:
          public_ids: 'abc-d3f-ghi, jkl-mn0-pqr'

workflows:
  run-tests:
    jobs:
      - e2e-tests

Example orb usage using a global configuration override

This orb overrides the path to the pattern for test files.

version: 2.1

orbs:
  synthetics-ci: datadog/synthetics-ci-orb@3.20.0

jobs:
  e2e-tests:
    docker:
      - image: cimg/base:stable
    steps:
      - synthetics-ci/run-tests:
          files: e2e-tests/*.synthetics.json

workflows:
  run-tests:
    jobs:
      - e2e-tests

For another example pipeline that triggers Synthetic tests, see the simple-example.yml file.

Complex usage

Example orb usage using the `test_search_query`

version: 2.1

orbs:
  synthetics-ci: datadog/synthetics-ci-orb@3.20.0

jobs:
  e2e-tests:
    docker:
      - image: cimg/base:stable
    steps:
      - synthetics-ci/run-tests:
          test_search_query: 'tag:e2e-tests'

workflows:
  run-tests:
    jobs:
      - e2e-tests

Example orb usage using the Continuous Testing Tunnel

version: 2.1

orbs:
  synthetics-ci: datadog/synthetics-ci-orb@3.20.0

jobs:
  e2e-tests:
    docker:
      - image: your-image
    steps:
      - checkout
      - run:
          name: Running server in background
          command: npm start
          background: true
      - synthetics-ci/run-tests:
          config_path: tests/tunnel-config.json
          files: tests/*.synthetics.json
          test_search_query: 'tag:e2e-tests'
          tunnel: true

workflows:
  test-server:
    jobs:
      - build-image
      - integration-tests:
          requires:
            - build-image

For additional options such as customizing the pollingTimeout for your CircleCI pipelines, see CI/CD Integrations Configuration. For another example pipeline that starts a local server and triggers Synthetic tests using the Continuous Testing Tunnel, see the advanced-example.yml file.

Inputs

To customize your workflow, you can set the following parameters in a run-tests.yml file:

Name	Type	Default	Description
`api_key`	env var name	`DATADOG_API_KEY`	The name of the environment variable containing the API key.
`app_key`	env var name	`DATADOG_APP_KEY`	The name of the environment variable containing the APP key.
`background`	boolean	`false`	Whether or not this step should run in the background. See official CircleCI documentation.
`batch_timeout`	number	30 minutes	The duration (in milliseconds) after which the batch fails as timed out. The default is 30 minutes.
`config_path`	string	`datadog-ci.json`	The global JSON configuration used when launching tests.
`fail_on_critical_errors`	boolean	`false`	Fail if tests were not triggered or results could not be fetched.
`fail_on_missing_tests`	boolean	`false`	Fail if at least one specified test with a public ID (using `public_ids` or listed in a test file) is missing in a run (for example, if it has been deleted programmatically or on the Datadog site).
`fail_on_timeout`	boolean	`true`	Force the CI to fail (or pass) if one of the results exceeds its test timeout.
`files`	string	`{,!(node_modules)/*/}.synthetics.json`	A list of glob patterns to detect Synthetic tests config files, separated by new lines.
`junit_report`	string	none	The filename for a JUnit report if you want to generate one.
`locations`	string	values in test files	String of locations separated by semicolons to override the locations where your tests run.
`no_output_timeout`	string	30 minutes	Elapsed time the command can run without output. The string is a decimal with unit suffix, such as “20m”, “1.25h”, “5s”. See official CircleCI documentation.
`polling_timeout`	number	30 minutes	DEPRECATED in favor of batch_timeout. The duration (in milliseconds) after which the action stops polling for test results. At the CI level, test results completed after this duration are considered failed.
`public_ids`	string	values in test files	A list of test IDs for Synthetic tests you want to trigger, separated by new lines or commas.
`site`	string	`datadoghq.com`	The Datadog site to send data to. If the `DD_SITE` environment variable is set, it takes preference.
`subdomain`	string	`app`	The name of the custom subdomain set to access your Datadog application.
`test_search_query`	string	none	Trigger tests corresponding to a search query.
`tunnel`	boolean	`false`	Use the Continuous Testing Tunnel to trigger tests.
`variables`	string	none	A list of global variables to use for Synthetic tests, separated by new lines or commas. For example: `START_URL=https://example.org,MY_VARIABLE="My title"`.

To learn about additional options for your CircleCI pipelines, see Continuous Testing & CI/CD Integrations Configuration.