--- title: Set Up Code Coverage description: >- Configure Code Coverage by integrating with GitHub or GitLab, setting permissions, creating PR Gates, and uploading coverage reports. breadcrumbs: Docs > Code Coverage > Set Up Code Coverage --- # Set Up Code Coverage {% callout %} # Important note for users on the following Datadog sites: app.ddog-gov.com {% alert level="danger" %} This product is not supported for your selected [Datadog site](https://docs.datadoghq.com/getting_started/site). (). {% /alert %} {% /callout %} Setting up Code Coverage involves the following steps: 1. Configure the integration with your source code provider in the Datadog UI. 1. Configure code coverage data access permissions in Datadog. 1. Optionally, configure a PR Gate to block pull requests based on coverage thresholds. 1. Update your CI pipeline to upload code coverage reports to Datadog. ## Integrate with source code provider{% #integrate-with-source-code-provider %} Code Coverage supports the following: {% tab title="GitHub" %} Follow instructions in the [GitHub integration documentation](https://docs.datadoghq.com/integrations/github/#github-apps-1) on how to connect your GitHub repositories to Datadog. Code Coverage requires the following GitHub App permissions: | Permission | Access Level | Purpose | | ------------- | ------------ | -------------------------------------------------- | | Contents | Read | Show source code in the detailed coverage UI. | | Pull Requests | Write | Show PR data in coverage UI and write PR comments. | | Checks | Write | Create coverage PR Gates. | The following webhooks are required: | Webhook | Purpose | | --------------------------- | ---------------------------- | | Pull request | Receive PR data updates. | | Pull request review | Receive PR data updates. | | Pull request review comment | Receive PR data updates. | | Push | Receive Git commit metadata. | If everything is configured correctly, a green check mark is displayed in Datadog's [GitHub Integration](https://app.datadoghq.com/integrations/github/configuration) page: {% image source="https://datadog-docs.imgix.net/images/code_coverage/github_app_success.468049b86ba93e3d51a776abc47017e1.png?auto=format" alt="GitHub App integration success check" /%} {% alert level="info" %} If you have a Datadog-managed Marketplace App or a custom app with default settings, the required permissions and webhooks are included. {% /alert %} {% /tab %} {% tab title="Gitlab" %} Follow instructions in the [Gitlab Source Code integration documentation](https://docs.datadoghq.com/integrations/gitlab-source-code/) on how to connect your Gitlab repositories to Datadog. See [Datadog Source Code Integration Guide](https://docs.datadoghq.com/integrations/guide/source-code-integration/?tab=gitlabsaasonprem#connect-your-git-repositories-to-datadog) for additional context. {% /tab %} {% tab title="Azure DevOps" %} Follow instructions in the [Datadog Source Code Integration Guide](https://docs.datadoghq.com/integrations/guide/source-code-integration/?tab=azuredevopssaasonly#connect-your-git-repositories-to-datadog) on how to connect your Azure DevOps repositories to Datadog using [Azure DevOps Source Code integration](https://app.datadoghq.com/integrations/azure-devops-source-code/). {% /tab %} See [Data Collected](https://docs.datadoghq.com/code_coverage/data_collected/#source-code-provider-integration) for details on what data is collected from your source code provider. ## Data access permissions{% #data-access-permissions %} If you are using [custom roles](https://docs.datadoghq.com/account_management/rbac/permissions/#custom-roles) rather than [Datadog-managed roles](https://docs.datadoghq.com/account_management/rbac/permissions/#managed-roles), be sure to enable the `Code Coverage Read` permission for the roles that need to view code coverage data. Navigate to [Roles settings](https://app.datadoghq.com/organization-settings/roles), click `Edit` on the role you need, add the `Code Coverage Read` permission to the role, and save the changes. ## PR Gates{% #pr-gates %} If you wish to gate on PR coverage, you can configure PR Gates rules in one of two ways: - **Datadog UI**: Navigate to [PR Gates rule creation](https://app.datadoghq.com/ci/pr-gates/rule/create?dataSource=code_coverage) and configure a rule to gate on total or patch coverage. - **YAML configuration file**: Define gates in your [`code-coverage.datadog.yml`](https://docs.datadoghq.com/code_coverage/configuration#pr-gates) file. This allows you to manage gates as code alongside your repository. Rules from both sources are evaluated when a pull request is opened or updated. See [Configuration](https://docs.datadoghq.com/code_coverage/configuration#pr-gates) for YAML gate syntax and examples. ## Upload code coverage reports{% #upload-code-coverage-reports %} Update your CI pipeline to upload code coverage report files to Datadog. This involves installing and running the `datadog-ci` CLI in your CI environment. See [Data Collected](https://docs.datadoghq.com/code_coverage/data_collected/#code-coverage-report-upload) for details on what data is collected during code coverage report upload. ### Supported coverage report formats{% #supported-coverage-report-formats %} Datadog supports the following coverage data formats—expand for examples: {% collapsible-section #lcov %} #### LCOV ```text TN: SF:src/example.c FN:3,add FNDA:5,add FNF:1 FNH:1 DA:3,5 DA:4,5 DA:5,5 DA:8,0 DA:9,0 LF:5 LH:3 BRDA:4,0,0,5 BRDA:4,0,1,0 BRF:2 BRH:1 end_of_record ``` {% /collapsible-section %} {% collapsible-section #go-coverprofile %} #### Go Coverprofile ```text mode: atomic example/calculator.go:51.148,53.2 1 0 example/calculator.go:55.190,61.15 3 0 example/calculator.go:61.15,64.3 2 0 example/calculator.go:66.2,67.16 2 0 example/calculator.go:67.16,69.3 1 0 example/clients/api_client.go:27.87,31.2 3 2 example/clients/api_client.go:34.85,36.2 1 3 example/clients/api_client.go:39.126,44.2 4 3 example/clients/api_client.go:47.106,50.2 2 3 example/notifications/notifier.go:49.79,51.2 1 3 example/notifications/notifier.go:60.33,69.2 1 0 example/notifications/notifier.go:79.131,86.15 3 2 example/notifications/notifier.go:104.3,104.10 1 3 ``` {% /collapsible-section %} {% collapsible-section #cobertura-xml %} #### Cobertura XML ```xml src ``` {% /collapsible-section %} {% collapsible-section #jacoco-xml %} #### Jacoco XML ```xml ``` {% /collapsible-section %} {% collapsible-section #clover-xml %} #### Clover XML ```xml ``` {% /collapsible-section %} {% collapsible-section #opencover-xml %} #### OpenCover XML ```xml Example.dll ``` {% /collapsible-section %} {% collapsible-section #simplecov-json %} #### Simplecov JSON ```json { "meta": { "simplecov_version": "0.21.2" }, "coverage": { "/path/to/file1.rb": { "lines": [ null, 1, 2, 0, null, 1, null, null, null, "ignored", "ignored", "ignored", null ], "branches": [] }, "/path/to/file2.rb": { "lines": [1, 1, null, 0, 1], "branches": [] } } } ``` {% /collapsible-section %} ### Install the datadog-ci CLI{% #install-the-datadog-ci-cli %} {% alert level="info" %} If you use GitHub Actions, you can skip this installation step. The GitHub Actions upload method below uses a dedicated action that handles `datadog-ci` installation automatically. {% /alert %} Standalone binaries are provided with [Datadog CI releases](https://github.com/DataDog/datadog-ci/releases). The *linux-x64*, *linux-arm64*, *darwin-x64*, *darwin-arm64* (MacOS), and *win-x64* (Windows) architectures are supported. To install, run the following from your terminal: {% tab title="Linux" %} ```shell curl -L --fail "https://github.com/DataDog/datadog-ci/releases/latest/download/datadog-ci_linux-x64" --output "/usr/local/bin/datadog-ci" && chmod +x /usr/local/bin/datadog-ci ``` Then run any command with `datadog-ci`: ```shell datadog-ci version ``` {% /tab %} {% tab title="MacOS" %} ```shell curl -L --fail "https://github.com/DataDog/datadog-ci/releases/latest/download/datadog-ci_darwin-x64" --output "/usr/local/bin/datadog-ci" && chmod +x /usr/local/bin/datadog-ci ``` Then run any command with `datadog-ci`: ```shell datadog-ci version ``` {% /tab %} {% tab title="Windows" %} ```powershell Invoke-WebRequest -Uri "https://github.com/DataDog/datadog-ci/releases/latest/download/datadog-ci_win-x64" -OutFile "datadog-ci.exe" ``` Then run any command with `Start-Process -FilePath "datadog-ci.exe"`: ```powershell Start-Process -FilePath "./datadog-ci.exe" -ArgumentList version ``` {% /tab %} #### npm{% #npm %} Alternatively, if Node.js is available in your CI environment, install the [`datadog-ci`](https://www.npmjs.com/package/@datadog/datadog-ci) CLI globally using `npm`: ```shell npm install -g @datadog/datadog-ci ``` #### Docker image{% #docker-image %} Alternatively, you can update your CI job to run in a container based on the [Datadog CI Docker image](https://hub.docker.com/r/datadog/ci). The image comes with `datadog-ci` preinstalled and ready to use. ### Uploading coverage reports{% #uploading-coverage-reports %} {% alert level="info" %} Datadog automatically aggregates all reports for the same commit on the backend. You don't need to merge coverage reports before uploading them. {% /alert %} To upload your code coverage reports to Datadog, run the following command. Provide a valid [Datadog API key](https://app.datadoghq.com/organization-settings/api-keys) (`DD_API_KEY`), and one or more file paths to either the coverage report files directly or directories containing them: {% tab title="GitHub Actions" %} Use the [Datadog Code Coverage Upload](https://github.com/marketplace/actions/datadog-code-coverage-upload) GitHub Action. This action automatically installs and runs `datadog-ci`, so no additional setup is required: ```yaml steps: - name: Upload coverage reports to Datadog uses: DataDog/coverage-upload-github-action@v1 with: api_key: ${{ secrets.DD_API_KEY }} site: ``` Alternatively, if you have `datadog-ci` installed, you can run it directly: ```yaml steps: - name: Upload coverage reports to Datadog run: datadog-ci coverage upload . env: DD_API_KEY: ${{ secrets.DD_API_KEY }} DD_SITE: ``` {% /tab %} {% tab title="Gitlab" %} ```yaml test: stage: test script: - ... # run your tests and generate coverage reports - datadog-ci coverage upload . # make sure to add the DD_API_KEY CI/CD variable ``` {% /tab %} {% tab title="Azure Pipelines" %} - script: datadog-ci coverage upload --format=clover coverage/clover.xml displayName: 'Upload coverage to Datadog' env: DD_API_KEY: $(DD_API_KEY) DD_SITE: 'datadoghq.com' {% /tab %} The command recursively searches the specified directories for supported coverage report files, so specifying the current directory (`.`) is usually sufficient. See the [`datadog-ci` documentation](https://github.com/DataDog/datadog-ci/tree/master/packages/plugin-coverage) for more details on the `datadog-ci coverage upload` command. Shortly after the code coverage report upload is finished, Datadog adds a PR comment with code coverage percentage values. You can also view your coverage data aggregated by pull request in the [Code Coverage page](https://app.datadoghq.com/ci/code-coverage) in Datadog, with the ability to examine individual files and lines of code. {% image source="https://datadog-docs.imgix.net/images/code_coverage/pr_details.350c4987583ad19fe7f4617d3998bfb3.png?auto=format" alt="" /%} ## Troubleshooting{% #troubleshooting %} ### Coverage upload command does not detect coverage report files{% #coverage-upload-command-does-not-detect-coverage-report-files %} The `datadog-ci coverage upload` command automatically detects supported coverage report files in the specified directories using heuristics, such as file names and extensions. If your coverage report files do not match expected patterns, the command might not detect them automatically. In this case, specify the report format and provide the file paths as positional arguments. For example: ```shell datadog-ci coverage upload --format=lcov \ src/coverage-reports/unit-tests/coverage.info \ src/coverage-reports/e2e-tests/coverage.info ``` ### Coverage upload fails with "Format could not be detected" error{% #coverage-upload-fails-with-format-could-not-be-detected-error %} The `datadog-ci coverage upload` command automatically detects the format of the coverage report files based on their content and file extension. If the command fails with the following error: ``` Invalid coverage report file [...]: format could not be detected ``` specify the format explicitly using the `--format` option, like this: ```shell datadog-ci coverage upload --format=cobertura reports/cobertura.xml ``` ### Coverage upload outputs "Could not sync git metadata" error{% #coverage-upload-outputs-could-not-sync-git-metadata-error %} Git metadata upload is only required if you can't integrate your CI provider directly with Datadog. If you are using a source code provider integration, such as Datadog GitHub app or Gitlab integration, you can disable the git metadata upload by passing the `--skip-git-metadata-upload=1` flag to the `datadog-ci coverage upload` command, like this: ```shell datadog-ci coverage upload --skip-git-metadata-upload=1 . ``` ### Datadog UI does not show changed files in the PR view{% #datadog-ui-does-not-show-changed-files-in-the-pr-view %} By default, the "Changed files" table only contains executable source code files that are present in the uploaded coverage reports. Select **Non-executable files** or **All** in the table header to display all files that were changed in the PR, regardless of whether they are executable or not. {% image source="https://datadog-docs.imgix.net/images/code_coverage/non_executable_files.38188539f95c21e770a18e7f97253bad.png?auto=format" alt="" /%} If a source code file is mistakenly marked as non-executable, it is probably missing from your uploaded coverage reports. Make sure that you are uploading all of your relevant reports, and double-check your coverage tool configuration to verify that coverage data is collected for all applicable files. Test sources are not considered executable files as they are not part of the production codebase being measured for coverage. ### Datadog UI shows incorrect file paths{% #datadog-ui-shows-incorrect-file-paths %} Code Coverage relies on the file paths in coverage reports to be either absolute or relative to the repository root. If the paths in your report are relative to a different directory in your repository, specify the correct base path (relative to the repo root) with the `--base-path` option when running the `datadog-ci coverage upload` command, like this: ```shell datadog-ci coverage upload --base-path=frontend/src . ``` ### Inaccurate coverage from non-executable lines{% #inaccurate-coverage-from-non-executable-lines %} Some coverage tools include non-executable lines (such as comments, blank lines, and closing brackets) in their reports, counting them as uncovered. This can lower your coverage percentages and produce false negatives for lines that can never be executed. During upload, the CLI automatically scans your source files to identify these non-executable lines so they can be excluded from coverage calculations. File fixes support the following languages: Go, Kotlin, C/C++, Swift, Objective-C, and PHP. You can control this behavior with the following options: - `--disable-file-fixes`: Disable file fixes generation entirely. - `--file-fixes-search-path `: Override the root directory used to scan source files. By default, the repository root is used. This is useful in monorepos or when your coverage reports only cover a subset of the codebase, as it speeds up the scan by limiting the directory tree traversed. ### Discrepancy between Datadog UI and coverage report values{% #discrepancy-between-datadog-ui-and-coverage-report-values %} Datadog automatically merges coverage reports for the same commit. As a result, the coverage percentage displayed in the Datadog UI may differ from the values in your individual coverage reports, especially if those reports contain overlapping or duplicate source code file entries. If you use an external tool (such as [ReportGenerator](https://reportgenerator.io/)) to merge coverage reports before uploading to Datadog, ensure your merged reports do not contain duplicate source code file entries. Datadog deduplicates overlapping files across reports, which can result in differences between your original coverage values and the merged values displayed in the Datadog UI. ## Further reading{% #further-reading %} - [Code Coverage](https://docs.datadoghq.com/code_coverage) - [Configure Code Coverage](https://docs.datadoghq.com/code_coverage/configuration) - [Organize coverage data with flags](https://docs.datadoghq.com/code_coverage/flags) - [Learn what data is collected for Code Coverage](https://docs.datadoghq.com/code_coverage/data_collected) - [Learn how Code Coverage supports large monorepos](https://docs.datadoghq.com/code_coverage/monorepo_support)