Python Tests

Compatibility

Supported languages:

Supported test frameworks:

Configuring reporting method

To report test results to Datadog, you need to configure the Datadog Python library:

Installing the Python tracer

Install the Python tracer by running:

pip install -U ddtrace

For more information, see the Python tracer installation documentation.

Instrumenting your tests

Using pytest

To enable instrumentation of pytest tests, add the --ddtrace option when running pytest, specifying the name of the service or library under test in the DD_SERVICE environment variable, and the environment where tests are being run (for example, local when running tests on a developer workstation, or ci when running them on a CI provider) in the DD_ENV environment variable:

DD_SERVICE=my-python-app DD_ENV=ci pytest --ddtrace

If you also want to enable the rest of the APM integrations to get more information in your flamegraph, add the --ddtrace-patch-all option:

DD_SERVICE=my-python-app DD_ENV=ci pytest --ddtrace --ddtrace-patch-all

Using pytest-benchmark

To instrument your benchmark tests with pytest-benchmark, run your benchmark tests with the --ddtrace option when running pytest, and Datadog detects metrics from pytest-benchmark automatically:

def square_value(value):
    return value * value


def test_square_value(benchmark):
    result = benchmark(square_value, 5)
    assert result == 25

Using unittest

To enable instrumentation of unittest tests, run your tests by appending ddtrace-run to the beginning of your unittest command.

Make sure to specify the name of the service or library under test in the DD_SERVICE environment variable. Additionally, you may declare the environment where tests are being run in the DD_ENV environment variable:

DD_SERVICE=my-python-app DD_ENV=ci ddtrace-run python -m unittest

Alternatively, if you wish to enable unittest instrumentation manually, use patch() to enable the integration:

from ddtrace import patch
import unittest
patch(unittest=True)

class MyTest(unittest.TestCase):
    def test_will_pass(self):
        assert True

Known limitations

In some cases, if your unittest test execution is run in a parallel manner, this may break the instrumentation and affect test visibility.

Datadog recommends you use up to one process at a time to prevent affecting test visibility.

Adding custom tags to tests

You can add custom tags to your tests by using the declaring ddspan as argument to your test:

from ddtrace import tracer

# Declare `ddspan` as argument to your test
def test_simple_case(ddspan):
    # Set your tags
    ddspan.set_tag("test_owner", "my_team")
    # test continues normally
    # ...

To create filters or group by fields for these tags, you must first create facets. For more information about adding tags, see the Adding Tags section of the Python custom instrumentation documentation.

Adding custom metrics to tests

Just like tags, you can add custom metrics to your tests by using the current active span:

from ddtrace import tracer

# Declare `ddspan` as an argument to your test
def test_simple_case(ddspan):
    # Set your tags
    ddspan.set_tag("memory_allocations", 16)
    # test continues normally
    # ...

Read more about custom metrics in the Add Custom Metrics Guide.

Configuration settings

The following is a list of the most important configuration settings that can be used with the tracer, either in code or using environment variables:

ddtrace.config.service

Name of the service or library under test.
Environment variable: DD_SERVICE
Default: pytest
Example: my-python-app

ddtrace.config.env

Name of the environment where tests are being run.
Environment variable: DD_ENV
Default: none
Examples: local, ci

The following environment variable can be used to configure the location of the Datadog Agent:

DD_TRACE_AGENT_URL

Datadog Agent URL for trace collection in the form http://hostname:port.
Default: http://localhost:8126

All other Datadog Tracer configuration options can also be used.

Collecting Git metadata

Datadog uses Git information for visualizing your test results and grouping them by repository, branch, and commit. Git metadata is automatically collected by the test instrumentation from CI provider environment variables and the local .git folder in the project path, if available.

If you are running tests in non-supported CI providers or with no .git folder, you can set the Git information manually using environment variables. These environment variables take precedence over any auto-detected information. Set the following environment variables to provide Git information:

DD_GIT_REPOSITORY_URL

URL of the repository where the code is stored. Both HTTP and SSH URLs are supported.
Example: git@github.com:MyCompany/MyApp.git, https://github.com/MyCompany/MyApp.git

DD_GIT_BRANCH

Git branch being tested. Leave empty if providing tag information instead.
Example: develop

DD_GIT_TAG

Git tag being tested (if applicable). Leave empty if providing branch information instead.
Example: 1.0.1

DD_GIT_COMMIT_SHA

Full commit hash.
Example: a18ebf361cc831f5535e58ec4fae04ffd98d8152

DD_GIT_COMMIT_MESSAGE

Commit message.
Example: Set release number

DD_GIT_COMMIT_AUTHOR_NAME

Commit author name.
Example: John Smith

DD_GIT_COMMIT_AUTHOR_EMAIL

Commit author email.
Example: john@example.com

DD_GIT_COMMIT_AUTHOR_DATE

Commit author date in ISO 8601 format.
Example: 2021-03-12T16:00:28Z

DD_GIT_COMMIT_COMMITTER_NAME

Commit committer name.
Example: Jane Smith

DD_GIT_COMMIT_COMMITTER_EMAIL

Commit committer email.
Example: jane@example.com

DD_GIT_COMMIT_COMMITTER_DATE

Commit committer date in ISO 8601 format.
Example: 2021-03-12T16:00:28Z

Further reading

Additional helpful documentation, links, and articles:

Forwarding Environment Variables for Tests in ContainersDOCUMENTATION Explore Test Results and PerformanceDOCUMENTATION Troubleshooting CI VisibilityDOCUMENTATION