Azure Pipelines Setup for CI Visibility

Docs > Continuous Integration Visibility > CI Pipeline Visibility in Datadog > Azure Pipelines Setup for CI Visibility

Azure DevOps Server is not officially supported.

Overview

Azure Pipelines is a continuous integration and delivery service that supports any language, platform, or cloud.

Set up CI Visibility for Azure Pipelines to gain real time insights into your CI/CD workflows, track pipeline performance, analyze inefficiencies, and manage your deployment operations.

Compatibility

Pipeline Visibility	Platform	Definition
CI jobs failure analysis	CI jobs failure analysis	Use LLM models on relevant logs to analyze the root cause of failed CI jobs.
Logs correlation	Logs correlation	Correlate pipeline and job spans to logs. Requires job log collection.
Custom tags and measures at runtime	Custom tags and measures at runtime	Configure custom tags and measures at runtime.
Custom spans	Custom spans	Configure custom spans for your pipelines.
Filter CI Jobs on the critical path	Filter CI Jobs on the critical path	Filter by jobs on the critical path.
Execution time	Execution time	View the amount of time pipelines have been running jobs.

Terminology

This table shows the mapping of concepts between Datadog CI Visibility and Azure Pipelines:

Datadog	Azure Pipelines
Pipeline	Pipeline
Stage	Stage
Job	Job
Not available in Datadog	Step

Configure the Datadog integration

Enable CI Visibility in Datadog

After the Azure App is created and installed, enable CI Visibility for the organizations and projects you want Datadog to monitor.

Verify that your Azure DevOps organization is linked to a Microsoft Entra tenant. See the Azure source code setup instructions for guidance on connecting Azure DevOps projects to Datadog.
In Datadog, navigate to Software Delivery → CI Visibility → Add a Pipeline Provider → Azure Pipelines.
Click Configure next to the Azure DevOps organization you want to enable.
To enable CI Visibility for the entire organization, toggle Enable CI Visibility. Future projects detected by the Azure app will automatically be enabled.
To enable CI Visibility for individual projects:
- Scroll through the projects list.
- Toggle Enable CI Visibility for each project you want to monitor.

Pipelines appear in Datadog immediately after CI Visibility is enabled for an organization or project.

If you previously configured Azure Pipelines using Service Hook Subscriptions, disable that integration before enabling CI Visibility for the same projects. Running both integrations simultaneously does not incur additional costs, but it creates duplicate data.

The Datadog integration for Azure Pipelines works by using service hooks to send data to Datadog.

Install the Datadog CI Visibility extension from the Azure Marketplace. There are several extensions starting with Datadog, make sure that you are installing the Datadog CI Visibility extension.
For each project, go to Project settings > Service hooks in Azure DevOps and select the green plus (+) icon to create a subscription.
Create a subscription to the Datadog CI Visibility service for each of the following webhook types. These event types are required and must be enabled individually.
- Run state changed
- Run stage state changed
- Run job state changed
- Run stage approval completed
- Run stage waiting for approval
- Build completed
Click Next to continue to the next step and set the following:
- Datadog Site: <span class="js-region-param region-param" data-region-param="dd_site"></span>
- Datadog API Key: Your Datadog API key.
Click Finish.

Configuring multiple projects in bulk

Datadog offers a script to help you enable service hooks across multiple or all of your Azure projects using the Azure API. The script requires Python 3 and the requests package.

To run the script, you need:

An Azure DevOps username
An Azure DevOps API token
An Azure DevOps organization name

The script supports environment variables DD_API_KEY and DD_SITE, and flags parameters --dd-api-key and --dd-site.

For more information, you can run the following command:

./service_hooks.py --help

All Azure projects

Example for enabling the hooks in all projects:

./service_hooks.py \
    --dd-api-key ******************** \
    --az-user "John Doe" \
    --az-token ********************** \
    --az-org datadoghq \
    --threads 4

Specific Azure projects

Example for enabling the hooks in specified projects:

./service_hooks.py \
    --dd-api-key ******************** \
    --az-user "John Doe" \
    --az-token ********************** \
    --az-org datadoghq \
    projectName1 projectName2

Advanced configuration

Set custom tags

You can set custom tags for all pipeline and job spans from your Azure projects to improve traceability. For more information, see Custom Tags and Measures.

Collect job logs

To enable log collection for Azure DevOps pipelines:

Set up the Datadog Azure DevOps integration by following the steps in the Azure integration tile.
In Datadog, go to CI Visibility Settings.
Enable log pulling for Azure DevOps.

Logs are billed separately from CI Visibility. Log retention, exclusion, and indexes are configured in Log Management. Logs for Azure jobs can be identified by the datadog.product:cipipeline and source:azurepipelines tags.

CI jobs failure analysis

If job logs collection is enabled, CI Visibility uses LLM models to compute the analysis for failed CI jobs based on relevant logs coming from Azure Pipelines.

You can also add job failure analysis to a PR comment. See the guide on using PR comments.

For a full explanation, see the guide on using CI jobs failure analysis.