---
title: Ruby Tests
description: Datadog, the leading service for cloud-scale monitoring.
breadcrumbs: Docs > Test Optimization in Datadog > Configure Test Optimization > Ruby Tests
---

# Ruby Tests

{% callout %}
# Important note for users on the following Datadog sites: app.ddog-gov.com

{% alert level="danger" %}
This product is not supported for your selected [Datadog site](https://docs.datadoghq.com/getting_started/site). ().
{% /alert %}

{% /callout %}

## Compatibility{% #compatibility %}

Supported languages:

| Language | Version |
| -------- | ------- |
| Ruby     | \>= 2.7 |
| JRuby    | \>= 9.4 |

Supported test frameworks:

| Test Framework | Version   |
| -------------- | --------- |
| RSpec          | \>= 3.0.0 |
| Minitest       | \>= 5.0.0 |
| Cucumber       | \>= 3.0   |

Supported test runners:

| Test runner    | Version    |
| -------------- | ---------- |
| Knapsack Pro   | \>= 7.2.0  |
| parallel_tests | \>= 4.0.0  |
| ci-queue       | \>= 0.53.0 |

## Configuring reporting method{% #configuring-reporting-method %}

To report test results to Datadog, you need to configure the `datadog-ci` gem:

{% tab title="CI Provider with Auto-Instrumentation Support" %}
We support auto-instrumentation for the following CI providers:

| CI Provider    | Auto-Instrumentation method                                                                                                                         |
| -------------- | --------------------------------------------------------------------------------------------------------------------------------------------------- |
| GitHub Actions | [Datadog Test Visibility Github Action](https://github.com/marketplace/actions/configure-datadog-test-visibility)                                   |
| Jenkins        | [UI-based configuration](https://docs.datadoghq.com/continuous_integration/pipelines/jenkins/#enable-test-optimization) with Datadog Jenkins plugin |
| GitLab         | [Datadog Test Visibility GitLab Script](https://github.com/DataDog/test-visibility-gitlab-script)                                                   |
| CircleCI       | [Datadog Test Visibility CircleCI Orb](https://circleci.com/orbs/registry/orb/datadog/test-visibility-circleci-orb)                                 |

If you are using auto-instrumentation for one of these providers, you can skip the rest of the setup steps below.
{% /tab %}

{% tab title="Cloud CI provider (Agentless)" %}
If you are using a cloud CI provider without access to the underlying worker nodes, such as GitHub Actions or CircleCI, configure the library to use the Agentless mode. For this, set the following environment variables:

{% dl %}

{% dt %}
`DD_CIVISIBILITY_AGENTLESS_ENABLED=true` (Required)
{% /dt %}

{% dd %}
Enables or disables Agentless mode.**Default**: `false`
{% /dd %}

{% dt %}
`DD_API_KEY` (Required)
{% /dt %}

{% dd %}
The [Datadog API key](https://app.datadoghq.com/organization-settings/api-keys) used to upload the test results.**Default**: `(empty)`
{% /dd %}

{% /dl %}

Additionally, configure the [Datadog site](https://docs.datadoghq.com/getting_started/site/) to which you want to send data.

{% dl %}

{% dt %}
`DD_SITE` (Required)
{% /dt %}

{% dd %}
The [Datadog site](https://docs.datadoghq.com/getting_started/site/) to upload results to.**Default**: `datadoghq.com`
{% /dd %}

{% /dl %}

{% /tab %}

{% tab title="On-Premises CI Provider (Datadog Agent)" %}
If you are running tests on an on-premises CI provider, such as Jenkins or self-managed GitLab CI, install the Datadog Agent on each worker node by following the [Agent installation instructions](https://docs.datadoghq.com/agent/). This is the recommended option as it allows you to automatically link test results to [logs](https://docs.datadoghq.com/tracing/other_telemetry/connect_logs_and_traces/) and [underlying host metrics](https://docs.datadoghq.com/infrastructure/).

If you are using a Kubernetes executor, Datadog recommends using the [Datadog Operator](https://docs.datadoghq.com/containers/datadog_operator/). The operator includes [Datadog Admission Controller](https://docs.datadoghq.com/agent/cluster_agent/admission_controller/) which can automatically [inject the tracer library](https://docs.datadoghq.com/tracing/trace_collection/library_injection_local/?tab=kubernetes) into the build pods. **Note:** If you use the Datadog Operator, there is no need to download and inject the tracer library since the Admission Controller can do this for you, so you can skip the corresponding step below. However, you still need to make sure that your pods set the environment variables or command-line parameters necessary to enable Test Visibility.

If you are not using Kubernetes or can't use the Datadog Admission Controller and the CI provider is using a container-based executor, set the `DD_TRACE_AGENT_URL` environment variable (which defaults to `http://localhost:8126`) in the build container running the tracer to an endpoint that is accessible from within that container. **Note:** Using `localhost` inside the build references the container itself and not the underlying worker node or any container where the Agent might be running in.

`DD_TRACE_AGENT_URL` includes the protocol and port (for example, `http://localhost:8126`) and takes precedence over `DD_AGENT_HOST` and `DD_TRACE_AGENT_PORT`, and is the recommended configuration parameter to configure the Datadog Agent's URL for CI Visibility.

If you still have issues connecting to the Datadog Agent, use the Agentless Mode. **Note:** When using this method, tests are not correlated with [logs](https://docs.datadoghq.com/tracing/other_telemetry/connect_logs_and_traces/) and [infrastructure metrics](https://docs.datadoghq.com/infrastructure/).
{% /tab %}

## Manual instrumentation{% #manual-instrumentation %}

{% alert level="info" %}
This section is **only required** if your CI provider does not support auto-instrumentation. If you selected **CI Provider with Auto-Instrumentation Support** in the Configuring reporting method section above, skip this section and proceed to Configuration settings.
{% /alert %}

If your CI provider does not support auto-instrumentation (for example, if you selected **Cloud CI provider (Agentless)** or **On-Premises CI Provider (Datadog Agent)**), follow these steps to install the library and instrument your tests manually.

1. Add the [Ruby test optimization gem](https://github.com/DataDog/datadog-ci-rb) to your Gemfile:

In the `Gemfile` file:

```ruby
gem "datadog-ci", "~> 1.0", group: :test
```

Configure the reporting method

Set `RUBYOPT` environment variable:

`RUBYOPT="-rbundler/setup -rdatadog/ci/auto_instrument"`

Run your tests as you normally do.

4a. (Optional) If you would prefer not to set `RUBYOPT` environment variable, prepend `bundle exec ddcirb exec` to your test command:

```bash
bundle exec ddcirb exec rspec
```

## Configuration settings{% #configuration-settings %}

The following is a list of the most important configuration settings that can be used with the test optimization library:

{% dl %}

{% dt %}
`DD_CIVISIBILITY_ENABLED=true` (Required)
{% /dt %}

{% dd %}
Enables the Test Optimization product. **Default**: `false`
{% /dd %}

{% dt %}
`DD_ENV` (Required)
{% /dt %}

{% dd %}
Environment where the tests are being run (`ci` when running them on a CI provider). **Default**: `none` **Example**: `ci`
{% /dd %}

{% dt %}
`DD_SERVICE` (Optional)
{% /dt %}

{% dd %}
Name of the service or library under test. **Default**: `$PROGRAM_NAME`**Example**: `my-ruby-app`
{% /dd %}

{% dt %}
`DD_TEST_SESSION_NAME` (Optional)
{% /dt %}

{% dd %}
Use this to identify a group of tests (see "Test session name") **Example**: `integration-tests`
{% /dd %}

{% /dl %}

The following environment variables can be used to configure tests reporting:

{% dl %}

{% dt %}
`DD_TRACE_AGENT_URL`
{% /dt %}

{% dd %}
Datadog Agent URL for trace collection in the form `http://hostname:port`.**Default**: `http://localhost:8126`
{% /dd %}

{% dt %}
`DD_CIVISIBILITY_AGENTLESS_ENABLED`
{% /dt %}

{% dd %}
Enables agentless mode to send data directly to Datadog without a Datadog agent. Requires `DD_API_KEY` to be set. **Default**: `false` **Example**: `true`
{% /dd %}

{% /dl %}

All other [Datadog Tracer configuration](https://docs.datadoghq.com/tracing/trace_collection/library_config/ruby/?tab=containers#configuration) options can also be used.

Additional Test Optimization features have their own configuration options documented on their respective pages.

## Adding custom tags to tests{% #adding-custom-tags-to-tests %}

You can add custom tags to your tests by using the current active test:

```ruby
require "datadog/ci"

# inside your test
Datadog::CI.active_test&.set_tag("test_owner", "my_team")
# test continues normally
# ...
```

To create filters or `group by` fields for these tags, you must first create facets. For more information about adding tags, see the [Adding Tags](https://docs.datadoghq.com/tracing/trace_collection/custom_instrumentation/ruby?tab=locally#adding-tags) section of the Ruby custom instrumentation documentation.

## Adding custom measures to tests{% #adding-custom-measures-to-tests %}

Like tags, you can add custom measures to your tests by using the current active test:

```ruby
require "datadog/ci"

# inside your test
Datadog::CI.active_test&.set_metric("memory_allocations", 16)
# test continues normally
# ...
```

For more information on custom measures, see the [Add Custom Measures Guide](https://docs.datadoghq.com/tests/guides/add_custom_measures/?tab=ruby).

## Using additional instrumentation{% #using-additional-instrumentation %}

It can be useful to have rich tracing information about your tests that includes time spent performing database operations or other external calls, as seen in the following flame graph:

{% image
   source="https://datadog-docs.imgix.net/images/continuous_integration/tests/setup/ci-ruby-test-trace-with-redis.bf37ee71356328751dadc6fa8ab1dfcc.png?auto=format"
   alt="Test trace with Redis instrumented" /%}

You can enable automatic APM instrumentation by adding the following line in your `test_helper/spec_helper`:

```ruby
require "datadog/auto_instrument" if ENV["DD_ENV"] == "ci"
```

**Note**: In CI mode, these traces are submitted to Test Optimization, and they do **not** show up in Datadog APM.

For the full list of available instrumentation methods, see the [tracing documentation](https://docs.datadoghq.com/tracing/trace_collection/dd_libraries/ruby/#integration-instrumentation)

## Collecting Git metadata{% #collecting-git-metadata %}

Datadog uses Git information for visualizing your test results and grouping them by repository, branch, and commit. Git metadata is automatically collected by the test instrumentation from CI provider environment variables and the local `.git` folder in the project path, if available.

If you are running tests in non-supported CI providers or with no `.git` folder, you can set the Git information manually using environment variables. These environment variables take precedence over any auto-detected information. Set the following environment variables to provide Git information:

{% dl %}

{% dt %}
`DD_GIT_REPOSITORY_URL`
{% /dt %}

{% dd %}
URL of the repository where the code is stored. Both HTTP and SSH URLs are supported.**Example**: `git@github.com:MyCompany/MyApp.git`, `https://github.com/MyCompany/MyApp.git`
{% /dd %}

{% dt %}
`DD_GIT_BRANCH`
{% /dt %}

{% dd %}
Git branch being tested. Leave empty if providing tag information instead.**Example**: `develop`
{% /dd %}

{% dt %}
`DD_GIT_TAG`
{% /dt %}

{% dd %}
Git tag being tested (if applicable). Leave empty if providing branch information instead.**Example**: `1.0.1`
{% /dd %}

{% dt %}
`DD_GIT_COMMIT_SHA`
{% /dt %}

{% dd %}
Full commit hash.**Example**: `a18ebf361cc831f5535e58ec4fae04ffd98d8152`
{% /dd %}

{% dt %}
`DD_GIT_COMMIT_MESSAGE`
{% /dt %}

{% dd %}
Commit message.**Example**: `Set release number`
{% /dd %}

{% dt %}
`DD_GIT_COMMIT_AUTHOR_NAME`
{% /dt %}

{% dd %}
Commit author name.**Example**: `John Smith`
{% /dd %}

{% dt %}
`DD_GIT_COMMIT_AUTHOR_EMAIL`
{% /dt %}

{% dd %}
Commit author email.**Example**: `john@example.com`
{% /dd %}

{% dt %}
`DD_GIT_COMMIT_AUTHOR_DATE`
{% /dt %}

{% dd %}
Commit author date in ISO 8601 format.**Example**: `2021-03-12T16:00:28Z`
{% /dd %}

{% dt %}
`DD_GIT_COMMIT_COMMITTER_NAME`
{% /dt %}

{% dd %}
Commit committer name.**Example**: `Jane Smith`
{% /dd %}

{% dt %}
`DD_GIT_COMMIT_COMMITTER_EMAIL`
{% /dt %}

{% dd %}
Commit committer email.**Example**: `jane@example.com`
{% /dd %}

{% dt %}
`DD_GIT_COMMIT_COMMITTER_DATE`
{% /dt %}

{% dd %}
Commit committer date in ISO 8601 format.**Example**: `2021-03-12T16:00:28Z`
{% /dd %}

{% /dl %}

## Using library's public API for unsupported test frameworks{% #using-librarys-public-api-for-unsupported-test-frameworks %}

If you use RSpec, Minitest, or Cucumber, **do not use the manual testing API**, as Test Optimization automatically instruments them and sends the test results to Datadog. The manual testing API is **incompatible** with already supported testing frameworks.

Use the manual testing API only if you use an unsupported testing framework or have a different testing mechanism. Full public API documentation is available on [YARD site](https://datadoghq.dev/datadog-ci-rb/Datadog/CI.html).

### Domain model{% #domain-model %}

The API is based around four concepts: test session, test module, test suite, and test.

#### Test session{% #test-session %}

A test session represents a test command run.

To start a test session, call `Datadog::CI.start_test_session` and pass the Datadog service and tags (such as the test framework you are using).

When all your tests have finished, call `Datadog::CI::TestSession#finish`, which closes the session and sends the session trace to the backend.

#### Test module{% #test-module %}

A test module represents a smaller unit of work within a session. For supported test frameworks, test module is always same as test session. For your use case, this could be a package in your componentized application.

To start a test module, call `Datadog::CI.start_test_module` and pass the name of the module.

When the module run has finished, call `Datadog::CI::TestModule#finish`.

#### Test suite{% #test-suite %}

A test suite comprises a set of tests that test similar functionality. A single suite usually corresponds to a single file where tests are defined.

Create test suites by calling `Datadog::CI#start_test_suite` and passing the name of the test suite.

Call `Datadog::CI::TestSuite#finish` when all the related tests in the suite have finished their execution.

#### Test{% #test %}

A test represents a single test case that is executed as part of a test suite. Usually it corresponds to a method that contains testing logic.

Create tests in a suite by calling `Datadog::CI#start_test` or `Datadog::CI.trace_test` and passing the name of the test and name of the test suite. Test suite name must be the same as name of the test suite started in previous step.

Call `Datadog::CI::Test#finish` when a test has finished execution.

### Code example{% #code-example %}

The following code represents example usage of the API:

```ruby
require "datadog/ci"

Datadog.configure do |c|
  c.service = "my-test-service"
  c.ci.enabled = true
end

def run_test_suite(tests, test_suite_name)
  test_suite = Datadog::CI.start_test_suite(test_suite_name)

  run_tests(tests, test_suite_name)

  test_suite.passed!
  test_suite.finish
end

def run_tests(tests, test_suite_name)
  tests.each do |test_name|
    Datadog::CI.trace_test(test_name, test_suite_name) do |test|
      test.passed!
    end
  end
end

Datadog::CI.start_test_session(
  tags: {
    Datadog::CI::Ext::Test::TAG_FRAMEWORK => "my-framework",
    Datadog::CI::Ext::Test::TAG_FRAMEWORK_VERSION => "0.0.1",
  }
)
Datadog::CI.start_test_module("my-test-module")

run_test_suite(["test1", "test2", "test3"], "test-suite-name")

Datadog::CI.active_test_module&.passed!
Datadog::CI.active_test_module&.finish

Datadog::CI.active_test_session&.passed!
Datadog::CI.active_test_session&.finish
```

## Best practices{% #best-practices %}

### Test session name `DD_TEST_SESSION_NAME`{% #test-session-name-dd_test_session_name %}

Use `DD_TEST_SESSION_NAME` to define the name of the test session and the related group of tests. Examples of values for this tag would be:

- `unit-tests`
- `integration-tests`
- `smoke-tests`
- `flaky-tests`
- `ui-tests`
- `backend-tests`

If `DD_TEST_SESSION_NAME` is not specified, the default value used is a combination of the:

- CI job name
- Command used to run the tests (such as `yarn test`)

The test session name needs to be unique within a repository to help you distinguish different groups of tests.

#### When to use `DD_TEST_SESSION_NAME`{% #when-to-use-dd_test_session_name %}

There's a set of parameters that Datadog checks to establish correspondence between test sessions. The test command used to execute the tests is one of them. If the test command contains a string that changes for every execution, such as a list of files to execute, Datadog considers the sessions to be unrelated to each other. For example:

- `bundle exec rspec my_spec.rb my_other_spec.rb`

Datadog recommends using `DD_TEST_SESSION_NAME` if your test commands vary between executions.

## Further reading{% #further-reading %}

- [Forwarding Environment Variables for Tests in Containers](https://docs.datadoghq.com/continuous_integration/tests/containers/)
- [Explore Test Results and Performance](https://docs.datadoghq.com/continuous_integration/tests)
- [Troubleshooting Test Optimization](https://docs.datadoghq.com/tests/troubleshooting/)