How to Set Up Deployment Data for DORA Metrics

Docs > DORA Metrics > How to Set Up Deployment Data for DORA Metrics

DORA Metrics is in public beta.

Overview

Deployment events are used to compute deployment frequency, change lead time, and change failure rate.

Selecting a deployment data source

DORA Metrics supports the following data sources for deployment events. See the respective documentation to set up the data source for your deployment events:

APM Deployment Tracking

Deployment Event API or datadog-ci CLI

Calculating deployment frequency

Deployment frequency is calculated based on the dora.deployments.count metric that is generated and increased with each deployment detected from your selected deployment data source. Frequency is calculated by dividing dora.deployments.count over a specific time frame.

Calculating change lead time

For a single Git commit, change lead time (CLT) is calculated as time from the creation of the commit to when the deployment including that commit was executed.

To calculate change lead time for a deployment, Datadog runs git log between the deployment commit SHA and the previous deployment commit SHA to find all the commits being deployed. Then, it computes the average of the related change lead time values. Datadog doesn’t store the actual content of files in your repository, only Git commit and tree objects.

For more information about the breakdown of change lead time metrics, see Data Collected.

Synchronize repository metadata to Datadog

GitHub workflows running on pull_request trigger are not currently supported by the GitHub integration. If you are using the pull_request trigger, use the alternative method.

If the GitHub integration is not already installed, install it on the GitHub integration tile.

When configuring the GitHub application:

Select at least Read repository permissions for Contents and Pull Requests.
Subscribe at least to Push, PullRequest and PullRequestReview events.

To confirm that the setup is valid, select your GitHub application in the GitHub integration tile and verify that, under the Features tab, the DORA Metrics: Collect Change Lead Time metric feature is enabled.

You can upload your Git repository metadata with the datadog-ci git-metadata upload command. When this command is executed, Datadog receives the repository URL, the commit SHA of the current branch, and a list of tracked file paths.

Run this command in CI for every new commit. If a deployment is executed for a specific commit SHA, ensure that the datadog-ci git-metadata upload command is run for that commit before the deployment event is sent.

Do not provide the --no-gitsync option to the datadog-ci git-metadata upload command. When that option is included, the commit information is not sent to Datadog and the change lead time metric is not calculated.

You can validate the correct setup of the command by checking the command output. An example of a correct output is:

Reporting commit 007f7f466e035b052415134600ea899693e7bb34 from repository git@github.com:organization/example-repository.git.
180 tracked file paths will be reported.
✅  Handled in 0.077 seconds.

Handling multiple services in the same repository

If the source code of multiple services is present in the same repository, further actions are needed to ensure that the change lead time is calculated by taking into account only the commits affecting the specific service being deployed.

To filter the commits measured to only the ones that affect the service, specify the source code glob file path patterns in the service definition.

If the service definition contains a full GitHub URL to the application folder, a single path pattern is automatically used.

Example (schema version v2.2):

links:
  - name: shopist
    type: repo
    provider: github
    url: https://github.com/organization/example-repository/tree/main/src/apps/shopist

DORA Metrics for the shopist service only consider the Git commits that include changes within src/apps/shopist/**. You can configure more granular control of the filtering with extensions[datadoghq.com/dora-metrics].

Example (schema version v2.2):

extensions:
  datadoghq.com/dora-metrics:
    source_patterns:
      - src/apps/shopist/**
      - src/libs/utils/**

DORA Metrics for the service shopist only consider the Git commits that include changes within src/apps/shopist/** or src/libs/utils/**.

Limitations

Change lead time stage breakdown metrics are only available for GitHub.
Change lead time is not available for the first deployment of a service that includes Git information.
If commits on a feature branch are squashed into a single commit prior to being merged into the default branch, the commit history for that pull request is not included in the change lead time calculation. The first commit included in the calculation is the newly created single commit.

Calculating change failure rate

Change failure rate is calculated by dividing dora.incidents.count over dora.deployments.count for the same services and/or teams associated to both an incident and a deployment event.