CI Visibility is not available in the selected site () at this time.
Compatibility
Supported languages:
Language
Version
Python 2
>= 2.7
Python 3
>= 3.6
Supported test frameworks:
Test Framework
Version
pytest
>= 3.0.0
pytest-benchmark
>= 3.1.0
unittest
>= 3.7
Configuring reporting method
To report test results to Datadog, you need to configure the Datadog Python library:
If you are running tests on an on-premises CI provider, such as Jenkins or self-managed GitLab CI, install the Datadog Agent on each worker node by following the Agent installation instructions. This is the recommended option as test results are then automatically linked to the underlying host metrics.
If you are using a Kubernetes executor, Datadog recommends using the Datadog Admission Controller, which automatically sets the environment variables in the build pods to communicate with the local Datadog Agent.
If you are not using Kubernetes or can’t use Datadog Admission Controller and the CI provider is using a container-based executor, set the DD_TRACE_AGENT_URL environment variable (which defaults to http://localhost:8126) in the build container running the tracer to an endpoint that is accessible from within that container. Note that using localhost inside the build references the container itself and not the underlying worker node or any container where the Agent might be running.
DD_TRACE_AGENT_URL includes the protocol and port (for example, http://localhost:8126) and takes precedence over DD_AGENT_HOST and DD_TRACE_AGENT_PORT, and is the recommended configuration parameter to configure the Datadog Agent’s URL for CI Visibility.
If you still have issues connecting to the Datadog Agent, use the Agentless Mode. Note: By using this method, there will be no correlation between tests and infrastructure metrics.
If you are using a cloud CI provider without access to the underlying worker nodes, such as GitHub Actions or CircleCI, configure the library to use the Agentless mode. For this, set the following environment variables:
DD_CIVISIBILITY_AGENTLESS_ENABLED=true (Required)
Enables or disables Agentless mode. Default: false
DD_API_KEY (Required)
The Datadog API key used to upload the test results. Default: (empty)
Additionally, configure which Datadog site to which you want to send data.
DD_SITE (Required)
The Datadog site to upload results to. Default: datadoghq.com
To enable instrumentation of pytest tests, add the --ddtrace option when running pytest, specifying the name of the service or library under test in the DD_SERVICE environment variable, and the environment where tests are being run (for example, local when running tests on a developer workstation, or ci when running them on a CI provider) in the DD_ENV environment variable:
To instrument your benchmark tests with pytest-benchmark, run your benchmark tests with the --ddtrace option when running pytest, and Datadog detects metrics from pytest-benchmark automatically:
To enable instrumentation of unittest tests, run your tests by appending ddtrace-run to the beginning of your unittest command.
Make sure to specify the name of the service or library under test in the DD_SERVICE environment variable.
Additionally, you may declare the environment where tests are being run in the DD_ENV environment variable:
In some cases, if your unittest test execution is run in a parallel manner, this may break the instrumentation and affect test visibility.
Datadog recommends you use up to one process at a time to prevent affecting test visibility.
Adding custom tags to tests
You can add custom tags to your tests by using the declaring ddspan as argument to your test:
fromddtraceimporttracer# Declare `ddspan` as argument to your testdeftest_simple_case(ddspan):# Set your tagsddspan.set_tag("test_owner","my_team")# test continues normally# ...
To create filters or group by fields for these tags, you must first create facets. For more information about adding tags, see the Adding Tags section of the Python custom instrumentation documentation.
Adding custom metrics to tests
Just like tags, you can add custom metrics to your tests by using the current active span:
fromddtraceimporttracer# Declare `ddspan` as an argument to your testdeftest_simple_case(ddspan):# Set your tagsddspan.set_tag("memory_allocations",16)# test continues normally# ...
Datadog uses Git information for visualizing your test results and grouping them by repository, branch, and commit. Git metadata is automatically collected by the test instrumentation from CI provider environment variables and the local .git folder in the project path, if available.
If you are running tests in non-supported CI providers or with no .git folder, you can set the Git information manually using environment variables. These environment variables take precedence over any auto-detected information. Set the following environment variables to provide Git information:
DD_GIT_REPOSITORY_URL
URL of the repository where the code is stored. Both HTTP and SSH URLs are supported. Example: git@github.com:MyCompany/MyApp.git, https://github.com/MyCompany/MyApp.git
DD_GIT_BRANCH
Git branch being tested. Leave empty if providing tag information instead. Example: develop
DD_GIT_TAG
Git tag being tested (if applicable). Leave empty if providing branch information instead. Example: 1.0.1
DD_GIT_COMMIT_SHA
Full commit hash. Example: a18ebf361cc831f5535e58ec4fae04ffd98d8152
DD_GIT_COMMIT_MESSAGE
Commit message. Example: Set release number
DD_GIT_COMMIT_AUTHOR_NAME
Commit author name. Example: John Smith
DD_GIT_COMMIT_AUTHOR_EMAIL
Commit author email. Example: john@example.com
DD_GIT_COMMIT_AUTHOR_DATE
Commit author date in ISO 8601 format. Example: 2021-03-12T16:00:28Z
DD_GIT_COMMIT_COMMITTER_NAME
Commit committer name. Example: Jane Smith
DD_GIT_COMMIT_COMMITTER_EMAIL
Commit committer email. Example: jane@example.com
DD_GIT_COMMIT_COMMITTER_DATE
Commit committer date in ISO 8601 format. Example: 2021-03-12T16:00:28Z
Further reading
Additional helpful documentation, links, and articles: