Python Tests

このページは日本語には対応しておりません。随時翻訳に取り組んでいます。翻訳に関してご質問やご意見ございましたら、お気軽にご連絡ください。

Python test instrumentation is in beta.

Compatibility

Supported Python interpreters:

Python >= 2.7 and >= 3.5

Supported test frameworks:

pytest >= 3.0.0
- pytest < 5 when using Python 2

Installing the Datadog Agent

To report test results to Datadog, you need to install the Datadog Agent.

Using an on-premises CI provider

If you are running tests on an on-premises CI provider, such as Jenkins or self-managed GitLab CI, install the Datadog Agent on each worker node by following the Agent installation instructions.

If the CI provider is using a container-based executor, set the DD_AGENT_HOST environment variable on all builds (which defaults to http://localhost:8126) to an endpoint that is accessible from within build containers, as using localhost inside the build references the container itself and not the underlying worker node where the Datadog Agent is running.

If you are using a Kubernetes executor, Datadog recommends using the Datadog Admission Controller, which automatically sets the DD_AGENT_HOST environment variable in the build pods to communicate with the local Datadog Agent.

Using a cloud CI provider

If you are using a cloud CI provider with no access to the underlying worker nodes, such as GitHub Actions or CircleCI, run the Datadog Agent in a container as a build service. This method is also available for an on-premises CI provider that uses a container-based executor if installing the Datadog Agent on each worker node is not an option.

To run the Datadog Agent as a container acting as a simple results forwarder, use the Docker image gcr.io/datadoghq/agent:latest and the following environment variables:

DD_API_KEY (Required): The Datadog API key used to upload the test results.
Default: (none)
DD_INSIDE_CI (Required): Disables the monitoring of the Datadog Agent container, as the underlying host is not accessible.
Default: false
Required value: true
DD_HOSTNAME (Required): Disables the reporting of hostnames associated with tests, as the underlying host cannot be monitored.
Default: (autodetected)
Required value: none

CI provider configuration examples

The following sections provide CI provider-specific instructions to run and configure the Agent to report test information.

To run the Datadog Agent in Azure Pipelines, define a new container in the resources section and link it with the job declaring it as a service container.

azure-pipeline.yml

variables:
  ddApiKey: $(DD_API_KEY)

resources:
  containers:
    - container: dd_agent
      image: gcr.io/datadoghq/agent:latest
      ports:
        - 8126:8126
      env:
        DD_API_KEY: $(ddApiKey)
        DD_INSIDE_CI: "true"
        DD_HOSTNAME: "none"

jobs:
  - job: test
    services:
      dd_agent: dd_agent
    steps:
      - script: make test

Replace <DD_SITE> with the selected site: .

azure-pipeline.yml

variables:
  ddApiKey: $(DD_API_KEY)

resources:
  containers:
    - container: dd_agent
      image: gcr.io/datadoghq/agent:latest
      ports:
        - 8126:8126
      env:
        DD_API_KEY: $(ddApiKey)
        DD_INSIDE_CI: "true"
        DD_HOSTNAME: "none"
        DD_SITE: "<DD_SITE>"

jobs:
  - job: test
    services:
      dd_agent: dd_agent
    steps:
      - script: make test

Add your Datadog API key to your project environment variables with the key DD_API_KEY.

To run the Agent in GitLab, define the Agent container under services.

.gitlab-ci.yml

variables:
  DD_API_KEY: $DD_API_KEY
  DD_INSIDE_CI: "true"
  DD_HOSTNAME: "none"
  DD_AGENT_HOST: "datadog-agent"

test:
  services:
    - name: gcr.io/datadoghq/agent:latest
  script:
    - make test

Replace <DD_SITE> with the selected site: .

.gitlab-ci.yml

variables:
  DD_API_KEY: $DD_API_KEY
  DD_INSIDE_CI: "true"
  DD_HOSTNAME: "none"
  DD_AGENT_HOST: "datadog-agent"
  DD_SITE: "<DD_SITE>"

test:
  services:
    - name: gcr.io/datadoghq/agent:latest
  script:
    - make test

Add your Datadog API key to your project environment variables with the key DD_API_KEY.

To run the Agent in GitHub Actions, use the Datadog Agent GitHub Action datadog/agent-github-action.

jobs:
  test:
    steps:
      - name: Start the Datadog Agent locally
        uses: datadog/agent-github-action@v1
        with:
          api_key: ${{ secrets.DD_API_KEY }}
      - run: make test

Replace <datadog_site> with the selected site: .

jobs:
  test:
    steps:
      - name: Start the Datadog Agent locally
        uses: datadog/agent-github-action@v1
        with:
          api_key: ${{ secrets.DD_API_KEY }}
          datadog_site: <datadog_site>
      - run: make test

Add your Datadog API key to your project secrets with the key DD_API_KEY.

To run the Agent in CircleCI, launch the Agent container before running tests by using the datadog/agent CircleCI orb, and stop it after to ensure results are sent to Datadog.

.circleci/config.yml

version: 2.1

orbs:
  datadog-agent: datadog/agent@0

jobs:
  test:
    docker:
      - image: circleci/<language>:<version_tag>
    steps:
      - checkout
      - datadog-agent/setup
      - run: make test
      - datadog-agent/stop

workflows:
  test:
    jobs:
      - test

Replace <DD_SITE> with the selected site: .

.circleci/config.yml

version: 2.1

orbs:
  datadog-agent: datadog/agent@0

jobs:
  test:
    docker:
      - image: circleci/<language>:<version_tag>
    environment:
      DD_SITE: "<DD_SITE>"
    steps:
      - checkout
      - datadog-agent/setup
      - run: make test
      - datadog-agent/stop

workflows:
  test:
    jobs:
      - test

Add your Datadog API key to your project environment variables with the key DD_API_KEY.

Installing the Python tracer

Install the Python tracer by running:

pip install -U ddtrace

For more information, see the Python tracer installation documentation.

Instrumenting your tests

To enable instrumentation of pytest tests, add the --ddtrace option when running pytest, specifying the name of the service or library under test in the DD_SERVICE environment variable, and the environment where tests are being run (for example, local when running tests on a developer workstation, or ci when running them on a CI provider) in the DD_ENV environment variable:

DD_SERVICE=my-python-app DD_ENV=ci pytest --ddtrace

If you also want to enable the rest of the APM integrations to get more information in your flamegraph, add the --ddtrace-patch-all option:

DD_SERVICE=my-python-app DD_ENV=ci pytest --ddtrace --ddtrace-patch-all

Adding custom tags to tests

You can add custom tags to your tests by using the declaring ddspan as argument to your test:

from ddtrace import tracer

# Declare `ddspan` as argument to your test
def test_simple_case(ddspan):
    # Set your tags
    ddspan.set_tag("test_owner", "my_team")
    # test continues normally
    # ...

To create filters or group by fields for these tags, you must first create facets. For more information about adding tags, see the Adding Tags section of the Python custom instrumentation documentation.

Configuration settings

The following is a list of the most important configuration settings that can be used with the tracer, either in code or using environment variables:

ddtrace.config.service: Name of the service or library under test.
Environment variable: DD_SERVICE
Default: pytest
Example: my-python-app
ddtrace.config.env: Name of the environment where tests are being run.
Environment variable: DD_ENV
Default: none
Examples: local, ci

The following environment variable can be used to configure the location of the Datadog Agent:

DD_TRACE_AGENT_URL: Datadog Agent URL for trace collection in the form http://hostname:port.
Default: http://localhost:8126

All other Datadog Tracer configuration options can also be used.

Collecting Git metadata

Datadog uses Git information for visualizing your test results and grouping them by repository, branch, and commit. Git metadata is automatically collected by the test instrumentation from CI provider environment variables and the local .git folder in the project path, if available.

If you are running tests in non-supported CI providers or with no .git folder, you can set the Git information manually using environment variables. These environment variables take precedence over any auto-detected information. Set the following environment variables to provide Git information:

DD_GIT_REPOSITORY_URL: URL of the repository where the code is stored. Both HTTP and SSH URLs are supported.
Example: git@github.com:MyCompany/MyApp.git, https://github.com/MyCompany/MyApp.git
DD_GIT_BRANCH: Git branch being tested. Leave empty if providing tag information instead.
Example: develop
DD_GIT_TAG: Git tag being tested (if applicable). Leave empty if providing branch information instead.
Example: 1.0.1
DD_GIT_COMMIT_SHA: Full commit hash.
Example: a18ebf361cc831f5535e58ec4fae04ffd98d8152
DD_GIT_COMMIT_MESSAGE: Commit message.
Example: Set release number
DD_GIT_COMMIT_AUTHOR_NAME: Commit author name.
Example: John Smith
DD_GIT_COMMIT_AUTHOR_EMAIL: Commit author email.
Example: john@example.com
DD_GIT_COMMIT_AUTHOR_DATE: Commit author date in ISO 8601 format.
Example: 2021-03-12T16:00:28Z
DD_GIT_COMMIT_COMMITTER_NAME: Commit committer name.
Example: Jane Smith
DD_GIT_COMMIT_COMMITTER_EMAIL: Commit committer email.
Example: jane@example.com
DD_GIT_COMMIT_COMMITTER_DATE: Commit committer date in ISO 8601 format.
Example: 2021-03-12T16:00:28Z

Information collected

When CI Visibility is enabled, the following data is collected from your project:

Test names and durations.
Predefined environment variables set by CI providers.
Git commit history including the hash, message, author information, and files changed (without file contents).
Information from the CODEOWNERS file.