--- title: Set Up Test Parallelization description: >- Set up Test Parallelization with ddtest, configure CI providers, and distribute test execution across CI nodes. breadcrumbs: >- Docs > Test Optimization in Datadog > Test Parallelization > Set Up Test Parallelization --- # Set Up Test Parallelization {% callout %} # Important note for users on the following Datadog sites: app.ddog-gov.com, us2.ddog-gov.com {% alert level="danger" %} This product is not supported for your selected [Datadog site](https://docs.datadoghq.com/getting_started/site.md). ({% placeholder "user-datadog-site-name" /%}). {% /alert %} {% /callout %} {% callout %} # Important note for users on the following Datadog sites: app.datadoghq.com, us3.datadoghq.com, us5.datadoghq.com, app.datadoghq.eu, ap1.datadoghq.com, ap2.datadoghq.com {% callout %} ##### Join the Preview! Test Parallelization is in Preview. Complete the form to request access. [Request Access](https://www.datadoghq.com/product-preview/test-parallelization/) {% /callout %} {% /callout %} ## Prerequisites{% #prerequisites %} Before setting up Test Parallelization: - Set up [Test Optimization](https://docs.datadoghq.com/tests/setup.md). - Use the `datadog-ci` gem version `1.31.0` or later. - Enable [Test Impact Analysis](https://docs.datadoghq.com/tests/test_impact_analysis.md) for the test service when you want Test Parallelization to split only the tests affected by a code change. ## Concepts{% #concepts %} {% dl %} {% dt %} Runner {% /dt %} {% dd %} A program that runs tests. `ddtest` can run tests directly or write file lists for another runner. {% /dd %} {% dt %} CI node {% /dt %} {% dd %} One CI execution environment, such as a GitHub Actions job, CircleCI parallel container, Kubernetes pod, VM, or local machine. {% /dd %} {% dt %} Worker {% /dt %} {% dd %} A process started by `ddtest` to execute tests. One CI node can run one worker or multiple workers. {% /dd %} {% dt %} Plan {% /dt %} {% dd %} The generated `.testoptimization/` directory. It contains the runnable test files, the selected parallelism, and per-node file lists used by `ddtest run` or another runner. {% /dd %} {% dt %} Selected parallelism {% /dt %} {% dd %} The CI node count or local worker count that `ddtest` chooses after estimating test file durations. {% /dd %} {% /dl %} ## Install ddtest{% #install-ddtest %} Install the `ddtest` CLI in your CI job. Datadog publishes precompiled binaries in [GitHub Releases](https://github.com/DataDog/ddtest/releases/latest). {% tab title="GitHub CLI" %} ```yaml - name: Download ddtest binary run: | mkdir -p bin gh release download --repo DataDog/ddtest --pattern "ddtest-linux-amd64" --dir bin mv bin/ddtest-linux-amd64 bin/ddtest chmod +x bin/ddtest env: GH_TOKEN: ${{ github.token }} ``` {% /tab %} {% tab title="curl" %} ```bash mkdir -p bin curl -fsSL https://github.com/DataDog/ddtest/releases/latest/download/ddtest-linux-amd64 -o bin/ddtest chmod +x bin/ddtest ``` {% /tab %} ## Run on a single CI node{% #run-on-a-single-ci-node %} If you run your tests on a single CI node, run `ddtest run`: ```bash bin/ddtest run --platform ruby --framework rspec ``` By default, `ddtest` can start one worker for each physical CPU core available on the node. ## Run across multiple CI nodes{% #run-across-multiple-ci-nodes %} For multiple CI nodes, run `ddtest plan` once, share the `.testoptimization/` directory with every CI node, and pass each node its zero-indexed CI node number: ```bash bin/ddtest plan \ --platform ruby \ --framework rspec \ --min-parallelism 1 \ --max-parallelism 8 bin/ddtest run \ --platform ruby \ --framework rspec \ --ci-node ``` In CI-node mode, `ddtest` uses one local worker by default. To start multiple workers in each CI node, set `--ci-node-workers` to a positive integer or `ncpu`. For a list of available environment variables, defaults, and examples, see [Configuration](https://docs.datadoghq.com/tests/test_parallelization/configuration.md). ## CI examples{% #ci-examples %} Use the following examples as starting points for GitHub Actions and CircleCI. {% tab title="GitHub Actions" %} The plan job chooses the CI node count and emits a matrix. The test job downloads the `.testoptimization/` artifact and runs only the files assigned to its matrix node. ```yaml name: CI with Test Parallelization on: [push] env: DD_TEST_OPTIMIZATION_RUNNER_PLATFORM: ruby DD_TEST_OPTIMIZATION_RUNNER_FRAMEWORK: rspec DD_TEST_OPTIMIZATION_RUNNER_MIN_PARALLELISM: 1 DD_TEST_OPTIMIZATION_RUNNER_MAX_PARALLELISM: 8 jobs: dd_plan: runs-on: ubuntu-latest outputs: matrix: ${{ steps.dd_plan.outputs.matrix }} steps: - uses: actions/checkout@v4 - name: Download ddtest binary run: | mkdir -p bin gh release download --repo DataDog/ddtest --pattern "ddtest-linux-amd64" --dir bin mv bin/ddtest-linux-amd64 bin/ddtest chmod +x bin/ddtest env: GH_TOKEN: ${{ github.token }} - name: Setup Ruby uses: ruby/setup-ruby@v1 with: bundler-cache: true - name: Configure Datadog Test Optimization uses: datadog/test-visibility-github-action@v2 with: languages: ruby api_key: ${{ secrets.DD_API_KEY }} site: datadoghq.com - id: dd_plan name: Plan test execution run: bin/ddtest plan - uses: actions/upload-artifact@v4 with: name: dd-artifacts path: .testoptimization include-hidden-files: true dd_test: runs-on: ubuntu-latest needs: [dd_plan] strategy: fail-fast: false matrix: ${{ fromJson(needs.dd_plan.outputs.matrix) }} steps: - uses: actions/checkout@v4 - name: Download ddtest binary run: | mkdir -p bin gh release download --repo DataDog/ddtest --pattern "ddtest-linux-amd64" --dir bin mv bin/ddtest-linux-amd64 bin/ddtest chmod +x bin/ddtest env: GH_TOKEN: ${{ github.token }} - uses: actions/download-artifact@v4 with: name: dd-artifacts path: .testoptimization - name: Setup Ruby uses: ruby/setup-ruby@v1 with: bundler-cache: true - name: Configure Datadog Test Optimization uses: datadog/test-visibility-github-action@v2 with: languages: ruby api_key: ${{ secrets.DD_API_KEY }} site: datadoghq.com - name: Run tests run: bin/ddtest run --ci-node ${{ matrix.ci_node_index }} ``` {% /tab %} {% tab title="CircleCI" %} The setup workflow runs `ddtest plan`, stores `.testoptimization/`, and continues into a test workflow with the selected CI node count. In `.circleci/config.yml`: ```yaml version: "2.1" setup: true orbs: ruby: circleci/ruby@2 test-optimization-circleci-orb: datadog/test-optimization-circleci-orb@1 continuation: circleci/continuation@0.2.0 jobs: plan: docker: - image: cimg/ruby:3.4.1 steps: - checkout - ruby/install-deps - test-optimization-circleci-orb/autoinstrument: languages: ruby site: datadoghq.com - run: name: Download ddtest command: | mkdir -p bin curl -fsSL https://github.com/DataDog/ddtest/releases/latest/download/ddtest-linux-amd64 -o bin/ddtest chmod +x bin/ddtest - run: name: Plan tests command: bin/ddtest plan --platform ruby --framework rspec environment: DD_TEST_OPTIMIZATION_RUNNER_MIN_PARALLELISM: 1 DD_TEST_OPTIMIZATION_RUNNER_MAX_PARALLELISM: 8 - save_cache: key: ddtest-plan-{{ .Revision }} paths: - .testoptimization - bin/ddtest - run: name: Continue with selected parallelism command: | desired=$(cat .testoptimization/runner/parallel-runners.txt 2>/dev/null || echo 1) printf '{"parallelism": %s}\n' "${desired}" > pipeline-parameters.json - continuation/continue: configuration_path: .circleci/test.yml parameters: pipeline-parameters.json workflows: plan: jobs: - plan ``` In `.circleci/test.yml`: ```yaml version: "2.1" parameters: parallelism: type: integer default: 1 orbs: ruby: circleci/ruby@2 test-optimization-circleci-orb: datadog/test-optimization-circleci-orb@1 jobs: test: parallelism: << pipeline.parameters.parallelism >> docker: - image: cimg/ruby:3.4.1 steps: - checkout - restore_cache: keys: - ddtest-plan-{{ .Revision }} - ruby/install-deps - test-optimization-circleci-orb/autoinstrument: languages: ruby site: datadoghq.com - run: name: Run tests command: | export DD_TEST_SESSION_NAME="ruby-tests-${CIRCLE_NODE_INDEX:-0}" bin/ddtest run --platform ruby --framework rspec --ci-node "${CIRCLE_NODE_INDEX:-0}" workflows: test: jobs: - test ``` {% /tab %} ## Use third-party test runners{% #use-third-party-test-runners %} Use `ddtest` plan files when you want `ddtest` to choose which files should run, but another runner should execute them. To learn about the full contents of the plan directory, see [Plan artifacts](https://docs.datadoghq.com/tests/test_parallelization/configuration.md#plan-artifacts). | File | Use | | ----------------------------------------------- | --------------------------------------------------------------------- | | `.testoptimization/runner/test-files.txt` | All runnable test files after Test Impact Analysis skips are applied. | | `.testoptimization/runner/tests-split/runner-N` | Files assigned to CI node or worker `N`. | For example, use `.testoptimization/runner/test-files.txt` with Knapsack Pro: ```bash KNAPSACK_PRO_TEST_FILE_LIST_SOURCE_FILE=.testoptimization/runner/test-files.txt bundle exec rake knapsack_pro:queue:rspec ``` ## Further reading{% #further-reading %} - [Configure Test Parallelization](https://docs.datadoghq.com/tests/test_parallelization/configuration.md) - [Troubleshooting Test Parallelization](https://docs.datadoghq.com/tests/test_parallelization/troubleshooting.md) - [Test Parallelization Best Practices](https://docs.datadoghq.com/tests/test_parallelization/best_practices.md) - [Set up Test Optimization](https://docs.datadoghq.com/tests/setup.md)