Python Tests

Compatibility

Supported Python interpreters:

• Python >= 2.7 and >= 3.5

Supported test frameworks:

• pytest >= 3.0.0
  • pytest < 5 when using Python 2

Installing the Datadog Agent

To report test results to Datadog, you need to install the Datadog Agent.

Using an on-premises CI provider

If you are running tests on an on-premises CI provider, such as Jenkins or self-managed GitLab CI, install the Datadog Agent on each worker node by following the Agent installation instructions.

If the CI provider is using a container-based executor, set the DD_AGENT_HOST environment variable on all builds (which defaults to http://localhost:8126) to an endpoint that is accessible from within build containers, as using localhost inside the build references the container itself and not the underlying worker node where the Datadog Agent is running.

If you are using a Kubernetes executor, Datadog recommends using the Datadog Admission Controller, which automatically sets the DD_AGENT_HOST environment variable in the build pods to communicate with the local Datadog Agent.

Using a cloud CI provider

If you are using a cloud CI provider with no access to the underlying worker nodes, such as GitHub Actions or CircleCI, run the Datadog Agent in a container as a build service. This method is also available for an on-premises CI provider that uses a container-based executor if installing the Datadog Agent on each worker node is not an option.

To run the Datadog Agent as a container acting as a simple results forwarder, use the Docker image gcr.io/datadoghq/agent:latest and the following environment variables:

DD_API_KEY (Required)

The Datadog API key used to upload the test results.
Default: (none)

DD_INSIDE_CI (Required)

Disables the monitoring of the Datadog Agent container, as the underlying host is not accessible.
Default: false
Required value: true

DD_HOSTNAME (Required)

Disables the reporting of hostnames associated with tests, as the underlying host cannot be monitored.
Default: (autodetected)
Required value: none

CI provider configuration examples

The following sections provide CI provider-specific instructions to run and configure the Agent to report test information.

variables:
  ddApiKey: $(DD_API_KEY)

resources:
  containers:
    - container: dd_agent
      image: gcr.io/datadoghq/agent:latest
      ports:
        - 8126:8126
      env:
        DD_API_KEY: $(ddApiKey)
        DD_INSIDE_CI: "true"
        DD_HOSTNAME: "none"

jobs:
  - job: test
    services:
      dd_agent: dd_agent
    steps:
      - script: make test

variables:
  ddApiKey: $(DD_API_KEY)

resources:
  containers:
    - container: dd_agent
      image: gcr.io/datadoghq/agent:latest
      ports:
        - 8126:8126
      env:
        DD_API_KEY: $(ddApiKey)
        DD_INSIDE_CI: "true"
        DD_HOSTNAME: "none"
        DD_SITE: "<DD_SITE>"

jobs:
  - job: test
    services:
      dd_agent: dd_agent
    steps:
      - script: make test

variables:
  DD_API_KEY: $DD_API_KEY
  DD_INSIDE_CI: "true"
  DD_HOSTNAME: "none"
  DD_AGENT_HOST: "datadog-agent"

test:
  services:
    - name: gcr.io/datadoghq/agent:latest
  script:
    - make test

variables:
  DD_API_KEY: $DD_API_KEY
  DD_INSIDE_CI: "true"
  DD_HOSTNAME: "none"
  DD_AGENT_HOST: "datadog-agent"
  DD_SITE: "<DD_SITE>"

test:
  services:
    - name: gcr.io/datadoghq/agent:latest
  script:
    - make test

jobs:
  test:
    steps:
      - name: Start the Datadog Agent locally
        uses: datadog/agent-github-action@v1
        with:
          api_key: ${{ secrets.DD_API_KEY }}
      - run: make test

jobs:
  test:
    steps:
      - name: Start the Datadog Agent locally
        uses: datadog/agent-github-action@v1
        with:
          api_key: ${{ secrets.DD_API_KEY }}
          datadog_site: <datadog_site>
      - run: make test

version: 2.1

orbs:
  datadog-agent: datadog/agent@0

jobs:
  test:
    docker:
      - image: circleci/<language>:<version_tag>
    steps:
      - checkout
      - datadog-agent/setup
      - run: make test
      - datadog-agent/stop

workflows:
  test:
    jobs:
      - test

version: 2.1

orbs:
  datadog-agent: datadog/agent@0

jobs:
  test:
    docker:
      - image: circleci/<language>:<version_tag>
    environment:
      DD_SITE: "<DD_SITE>"
    steps:
      - checkout
      - datadog-agent/setup
      - run: make test
      - datadog-agent/stop

workflows:
  test:
    jobs:
      - test

Installing the Python tracer

Install the Python tracer by running:

pip install -U ddtrace

For more information, see the Python tracer installation documentation.

Instrumenting your tests

To enable instrumentation of pytest tests, add the --ddtrace option when running pytest, specifying the name of the service or library under test in the DD_SERVICE environment variable, and the environment where tests are being run (for example, local when running tests on a developer workstation, or ci when running them on a CI provider) in the DD_ENV environment variable:

DD_SERVICE=my-python-app DD_ENV=ci pytest --ddtrace

If you also want to enable the rest of the APM integrations to get more information in your flamegraph, add the --ddtrace-patch-all option:

DD_SERVICE=my-python-app DD_ENV=ci pytest --ddtrace --ddtrace-patch-all

Adding custom tags to tests

You can add custom tags to your tests by using the declaring ddspan as argument to your test:

from ddtrace import tracer

# Declare `ddspan` as argument to your test
def test_simple_case(ddspan):
    # Set your tags
    ddspan.set_tag("test_owner", "my_team")
    # test continues normally
    # ...

To create filters or group by fields for these tags, you must first create facets. For more information about adding tags, see the Adding Tags section of the Python custom instrumentation documentation.

Configuration settings

The following is a list of the most important configuration settings that can be used with the tracer, either in code or using environment variables:

ddtrace.config.service

Name of the service or library under test.
Environment variable: DD_SERVICE
Default: pytest
Example: my-python-app

ddtrace.config.env

Name of the environment where tests are being run.
Environment variable: DD_ENV
Default: none
Examples: local, ci

The following environment variable can be used to configure the location of the Datadog Agent:

DD_TRACE_AGENT_URL

Datadog Agent URL for trace collection in the form http://hostname:port.
Default: http://localhost:8126

All other Datadog Tracer configuration options can also be used.

Collecting Git metadata

Datadog uses Git information for visualizing your test results and grouping them by repository, branch, and commit. Git metadata is automatically collected by the test instrumentation from CI provider environment variables and the local .git folder in the project path, if available.

If you are running tests in non-supported CI providers or with no .git folder, you can set the Git information manually using environment variables. These environment variables take precedence over any auto-detected information. Set the following environment variables to provide Git information:

DD_GIT_REPOSITORY_URL

URL of the repository where the code is stored. Both HTTP and SSH URLs are supported.
Example: git@github.com:MyCompany/MyApp.git, https://github.com/MyCompany/MyApp.git

DD_GIT_BRANCH

Git branch being tested. Leave empty if providing tag information instead.
Example: develop

DD_GIT_TAG

Git tag being tested (if applicable). Leave empty if providing branch information instead.
Example: 1.0.1

DD_GIT_COMMIT_SHA

Full commit hash.
Example: a18ebf361cc831f5535e58ec4fae04ffd98d8152

DD_GIT_COMMIT_MESSAGE

Commit message.
Example: Set release number

DD_GIT_COMMIT_AUTHOR_NAME

Commit author name.
Example: John Smith

DD_GIT_COMMIT_AUTHOR_EMAIL

Commit author email.
Example: john@example.com

DD_GIT_COMMIT_AUTHOR_DATE

Commit author date in ISO 8601 format.
Example: 2021-03-12T16:00:28Z

DD_GIT_COMMIT_COMMITTER_NAME

Commit committer name.
Example: Jane Smith

DD_GIT_COMMIT_COMMITTER_EMAIL

Commit committer email.
Example: jane@example.com

DD_GIT_COMMIT_COMMITTER_DATE

Commit committer date in ISO 8601 format.
Example: 2021-03-12T16:00:28Z

Information collected

When CI Visibility is enabled, the following data is collected from your project:

• Test names and durations.
• Predefined environment variables set by CI providers.
• Git commit history including the hash, message, author information, and files changed (without file contents).
• Information from the CODEOWNERS file.

Further reading

Additional helpful documentation, links, and articles:

Forwarding Environment Variables for Tests in ContainersDOCUMENTATION Explore Test Results and PerformanceDOCUMENTATION Troubleshooting CIDOCUMENTATION