Set Up Code Coverage

문서 > Code Coverage > Set Up Code Coverage

이 페이지는 아직 영어로 제공되지 않습니다. 번역 작업 중입니다.
현재 번역 프로젝트에 대한 질문이나 피드백이 있으신 경우 언제든지 연락주시기 바랍니다.

Join the Preview!

Code Coverage is in Preview. This product replaces Test Optimization's code coverage feature, which is being deprecated. Complete the form to request access for the new Code Coverage product.

Request Access

Setting up Code Coverage involves the following steps:

Configure the integration with your source code provider in the Datadog UI.
Configure code coverage data access permissions in Datadog.
Optionally, configure a PR Gate to block pull requests based on coverage thresholds.
Update your CI pipeline to upload code coverage reports to Datadog.

Integrate with source code provider

Code Coverage supports the following:

See the GitHub integration documentation for detailed instructions for integrating with GitHub.

Code Coverage requires the following GitHub App permissions:

Permission	Access Level	Purpose
Contents	Read	Show source code in the detailed coverage UI.
Pull Requests	Write	Show PR data in coverage UI and write PR comments.
Checks	Write	Create coverage PR Gates.

The following webhooks are required:

Webhook	Purpose
Pull request	Receive PR data updates.
Pull request review	Receive PR data updates.
Pull request review comment	Receive PR data updates.
Push	Receive Git commit metadata.

If everything is configured correctly, a green check mark is displayed in Datadog’s GitHub Integration page:

If you have a Datadog-managed Marketplace App or a custom app with default settings, the required permissions and webhooks are included.

See Data Collected for details on what data is collected from your source code provider.

Data access permissions

If you are using custom roles rather than Datadog-managed roles, be sure to enable the Code Coverage Read permission for the roles that need to view code coverage data.

Navigate to Roles settings, click Edit on the role you need, add the Code Coverage Read permission to the role, and save the changes.

PR Gates

If you wish to gate on PR coverage, configure PR Gates rules in Datadog.

Navigate to PR Gates rule creation and configure a rule to gate on total or patch coverage.

Upload code coverage reports

Update your CI pipeline to upload code coverage report files to Datadog. This involves installing and running the datadog-ci CLI in your CI environment.

See Data Collected for details on what data is collected during code coverage report upload.

Supported coverage report formats

Datadog supports the following coverage data formats—expand for examples:

LCOV

TN:
SF:src/example.c
FN:3,add
FNDA:5,add
FNF:1
FNH:1
DA:3,5
DA:4,5
DA:5,5
DA:8,0
DA:9,0
LF:5
LH:3
BRDA:4,0,0,5
BRDA:4,0,1,0
BRF:2
BRH:1
end_of_record

Cobertura XML

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE coverage SYSTEM "http://cobertura.sourceforge.net/xml/coverage-04.dtd">
<coverage lines-valid="5" lines-covered="3" line-rate="0.6" branches-valid="2" branches-covered="1" branch-rate="0.5" timestamp="1690658886" version="1.9">
  <sources>
    <source>src</source>
  </sources>
  <packages>
    <package name="example" line-rate="0.6" branch-rate="0.5">
      <classes>
        <class name="Example" filename="example/Example.java" line-rate="0.6" branch-rate="0.5">
          <methods>
            <method name="add" signature="(II)I" line-rate="1.0" branch-rate="1.0">
              <lines>
                <line number="3" hits="5"/>
                <line number="4" hits="5" branch="true" condition-coverage="50% (1/2)"/>
                <line number="5" hits="5"/>
              </lines>
            </method>
          </methods>
          <lines>
            <line number="3" hits="5"/>
            <line number="4" hits="5" branch="true" condition-coverage="50% (1/2)"/>
            <line number="5" hits="5"/>
            <line number="8" hits="0"/>
            <line number="9" hits="0"/>
          </lines>
        </class>
      </classes>
    </package>
  </packages>
</coverage>

Jacoco XML

<?xml version="1.0" encoding="UTF-8"?>
<report name="Example">
  <sessioninfo id="SessionId" start="1690658886000" dump="1690658887000"/>
  <package name="example">
    <sourcefile name="Example.java">
      <line nr="3" mi="0" ci="5"/>
      <line nr="4" mi="0" ci="5" mb="1" cb="1"/>
      <line nr="5" mi="0" ci="5"/>
      <line nr="8" mi="1" ci="0"/>
      <line nr="9" mi="1" ci="0"/>
    </sourcefile>
  </package>
</report>

Clover XML

<coverage generated="1661852015">
    <project timestamp="1661852015">
        <file name="/var/www/html/src/App/Console/CronjobRunnerCommand.php">
            <class name="App\Console\CronjobRunnerCommand" namespace="global">
                <metrics complexity="3" methods="3" coveredmethods="0" conditionals="0" coveredconditionals="0" statements="4" coveredstatements="0" elements="7" coveredelements="0"/>
            </class>
            <line num="18" type="method" name="__construct" visibility="public" complexity="1" crap="2" count="0"/>
            <line num="20" type="stmt" count="1"/>
            <line num="27" type="stmt" count="0"/>
            <line num="30" type="method" name="execute" visibility="protected" complexity="1" crap="2" count="0"/>
            <line num="32" type="stmt" count="0"/>
            <metrics loc="35" ncloc="35" classes="1" methods="3" coveredmethods="0" conditionals="0" coveredconditionals="0" statements="4" coveredstatements="0" elements="7" coveredelements="0"/>
        </file>
        <file name="/var/www/html/src/App/Console/CronjobRunnerCommand2.php">
            <line num="42" type="stmt" count="1"/>
        </file>
    </project>
</coverage>

OpenCover XML

<?xml version="1.0" encoding="utf-8"?>
<CoverageSession xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
  <Modules>
    <Module hash="ABC123">
      <ModulePath>Example.dll</ModulePath>
      <Files>
        <File uid="1" fullPath="src\example\Example.cs" />
      </Files>
      <Classes>
        <Class>
          <Methods>
            <Method visited="true" cyclomaticComplexity="1" sequenceCoverage="100">
              <FileRef uid="1"/>
              <SequencePoints>
                <SequencePoint vc="5" sl="3" />
                <SequencePoint vc="5" sl="4" />
                <SequencePoint vc="5" sl="5" />
                <SequencePoint vc="0" sl="9" />
              </SequencePoints>
              <BranchPoints>
                <BranchPoint vc="5" sl="4" path="0"/>
                <BranchPoint vc="0" sl="4" path="1"/>
              </BranchPoints>
            </Method>
          </Methods>
        </Class>
      </Classes>
    </Module>
  </Modules>
</CoverageSession>

Simplecov JSON

{
  "meta": {
    "simplecov_version": "0.21.2"
  },
  "coverage": {
    "/path/to/file1.rb": {
      "lines": [
        null,
        1,
        2,
        0,
        null,
        1,
        null,
        null,
        null,
        "ignored",
        "ignored",
        "ignored",
        null
      ],
      "branches": []
    },
    "/path/to/file2.rb": {
      "lines": [1, 1, null, 0, 1],
      "branches": []
    }
  }
}

Install the datadog-ci CLI

Install the datadog-ci CLI globally using npm:

npm install -g @datadog/datadog-ci

Standalone binary

If installing Node.js in the CI is an issue, standalone binaries are provided with Datadog CI releases. Only linux-x64, linux-arm64, darwin-x64, darwin-arm64 (MacOS) and win-x64 (Windows) are supported. To install, run the following from your terminal:

curl -L --fail "https://github.com/DataDog/datadog-ci/releases/latest/download/datadog-ci_linux-x64" --output "/usr/local/bin/datadog-ci" && chmod +x /usr/local/bin/datadog-ci

Then run any command with datadog-ci:

datadog-ci version

curl -L --fail "https://github.com/DataDog/datadog-ci/releases/latest/download/datadog-ci_darwin-x64" --output "/usr/local/bin/datadog-ci" && chmod +x /usr/local/bin/datadog-ci

Then run any command with datadog-ci:

datadog-ci version

Invoke-WebRequest -Uri "https://github.com/DataDog/datadog-ci/releases/latest/download/datadog-ci_win-x64" -OutFile "datadog-ci.exe"

Then run any command with Start-Process -FilePath "datadog-ci.exe":

Start-Process -FilePath "./datadog-ci.exe" -ArgumentList version

Uploading coverage reports

Datadog automatically aggregates all reports for the same commit on the backend. You don't need to merge coverage reports before uploading them.

To upload your code coverage reports to Datadog, run the following command. Provide a valid Datadog API key (DD_API_KEY), and one or more file paths to either the coverage report files directly or directories containing them:


steps:
- name: Upload coverage reports to Datadog
  run: datadog-ci coverage upload .
  env:
    DD_API_KEY: ${{ secrets.DD_API_KEY }}
    DD_SITE:

The command recursively searches the specified directories for supported coverage report files, so specifying the current directory (.) is usually sufficient. See the datadog-ci documentation for more details on the datadog-ci coverage upload command.

Shortly after the code coverage report upload is finished, Datadog adds a PR comment with code coverage percentage values. You can also view your coverage data aggregated by pull request in the Code Coverage page in Datadog, with the ability to examine individual files and lines of code.

Troubleshooting

Coverage upload command does not detect your coverage report files

The datadog-ci coverage upload command automatically detects supported coverage report files in the specified directories using heuristics, such as file names and extensions. If your coverage report files do not match expected patterns, the command might not detect them automatically. In this case, specify the report format and provide the file paths as positional arguments. For example:

datadog-ci coverage upload --format=lcov \
  src/coverage-reports/unit-tests/coverage.info \
  src/coverage-reports/e2e-tests/coverage.info

Coverage upload fails with “Format could not be detected” error

The datadog-ci coverage upload command automatically detects the format of the coverage report files based on their content and file extension. If the command fails with the following error:

Invalid coverage report file [...]: format could not be detected

specify the format explicitly using the --format option, like this:

datadog-ci coverage upload --format=cobertura reports/cobertura.xml

Coverage upload outputs “Could not sync git metadata” error

Git metadata upload is only required if you can’t integrate your CI provider directly with Datadog. If you are using a source code provider integration, such as Datadog GitHub app or Gitlab integration, you can disable the git metadata upload by passing the --skip-git-metadata-upload=1 flag to the datadog-ci coverage upload command, like this:

datadog-ci coverage upload --skip-git-metadata-upload=1 .

Discrepancy between Datadog UI and coverage report values

Datadog automatically merges coverage reports for the same commit. As a result, the coverage percentage displayed in the Datadog UI may differ from the values in your individual coverage reports, especially if those reports contain overlapping or duplicate source code file entries.

If you use an external tool (such as ReportGenerator) to merge coverage reports before uploading to Datadog, ensure your merged reports do not contain duplicate source code file entries. Datadog deduplicates overlapping files across reports, which can result in differences between your original coverage values and the merged values displayed in the Datadog UI.