Continuous Testing and Datadog CI Azure DevOps Extension

Visual Studio Marketplace Version Build Status License

Overview

With the Datadog CI Azure DevOps Extension, you can run Synthetic tests within your Azure Pipeline configuration and ensure all your teams using Azure DevOps can benefit from Synthetic tests at every stage of the software lifecycle. You can run SyntheticsRunTests as a task.

Authentication

Service Connection

To connect to your Datadog site, Datadog recommends setting up a custom service connection when configuring the Synthetics Run Test task.

You need to provide the following inputs:

  • Datadog site: Which Datadog site to connect to.
  • Custom subdomain (default: app): The name of the custom subdomain set to access your Datadog application. If the URL used to access Datadog is myorg.datadoghq.com, this value needs to be set to myorg.
  • API Key: Your Datadog API key. This key is created by your Datadog organization.
  • Application key: Your Datadog application key. This key is created by your Datadog organization.

API and Application keys

  • API Key: Your Datadog API key. This key is created by your Datadog organization and is accessed as an environment variable.
  • Application key: Your Datadog application key. This key is created by your Datadog organization and is accessed as an environment variable.
  • Datadog site: The Datadog site.
  • Custom subdomain (optional): The name of the custom subdomain set to access your Datadog application. If the URL used to access Datadog is myorg.datadoghq.com, this value needs to be set to myorg.

Setup

To connect to your Datadog account, create a Datadog CI service connection in your Azure pipelines project. Once created, all you need is the name of the service connection in the tasks.

  1. Install the Datadog CI Extension from the Visual Studio marketplace in your Azure Organization.
  2. Add your Datadog API and application keys in the Datadog CI Service Connection, or as secrets to your Azure Pipelines project.
  3. In your Azure DevOps pipeline, use the SyntheticsRunTests task.

Your task can be simple or complex.

Simple usage

Example task using public IDs

- task: SyntheticsRunTests@0
  displayName: Run Datadog Synthetics tests
  inputs:
    authenticationType: 'connectedService'
    connectedService: 'my-datadog-ci-connected-service'
    publicIds: |
      abc-d3f-ghi
      jkl-mn0-pqr      

Example task using existing synthetics.json files

- task: SyntheticsRunTests@0
  displayName: Run Datadog Synthetics tests
  inputs:
    authenticationType: 'connectedService'
    connectedService: 'my-datadog-ci-connected-service'
    files: 'e2e-tests/*.synthetics.json'

Example task using pipeline secrets for authentication

- task: SyntheticsRunTests@0
  inputs:
    authenticationType: 'apiAppKeys'
    apiKey: '$(DatadogApiKey)'
    appKey: '$(DatadogAppKey)'
    subdomain: 'myorg'
    datadogSite: 'datadoghq.eu'

Complex usage

Example task using the testSearchQuery

- task: SyntheticsRunTests@0
  displayName: Run Datadog Synthetics tests
  inputs:
    authenticationType: 'connectedService'
    connectedService: 'my-datadog-ci-connected-service'
    testSearchQuery: 'tag:e2e-tests'
    variables: |
      START_URL=https://staging.website.com
      PASSWORD=$(StagingPassword)      

Example task using the testSearchQuery and variable overrides

- task: SyntheticsRunTests@0
  displayName: Run Datadog Synthetics tests
  inputs:
    authenticationType: 'connectedService'
    connectedService: 'my-datadog-ci-connected-service'
    testSearchQuery: 'tag:e2e-tests'

Example task using a global configuration override with configPath

- task: SyntheticsRunTests@0
  displayName: Run Datadog Synthetics tests
  inputs:
    authenticationType: 'connectedService'
    connectedService: 'my-datadog-ci-connected-service'
    configPath: './synthetics-config.json'

Inputs

NameRequirementDescription
authenticationTyperequiredThe type of authentication you want Datadog to use, either connectedService or apiAppKeys.
connectedServiceoptionalThe name of the Datadog CI service connection to use when using the connectedService authentication type.
apiKeyoptionalYour Datadog API key when using the apiAppKeys authentication type. This key is created by your Datadog organization and should be stored as a secret.
appKeyoptionalYour Datadog application key when using the apiAppKeys authentication type. This key is created by your Datadog organization and should be stored as a secret.
subdomainoptionalThe name of the custom subdomain set to access your Datadog application when using the apiAppKeys authentication type. If the URL used to access Datadog is myorg.datadoghq.com, this value needs to be set to myorg. Default: app.
datadogSiteoptionalThe Datadog site when using the apiAppKeys authentication type. Default: datadoghq.com.
publicIdsoptionalA list of tests IDs for Synthetic tests you want to trigger, separated by new lines or commas. If no value is provided, the task looks for files named synthetics.json.
testSearchQueryoptionalTrigger tests corresponding to a search query. This can be useful if you are tagging your test configurations. For more information, see rules and best practices for naming tags.
filesoptionalGlob pattern to detect Synthetic tests’ config files. Default: {,!(node_modules)/**/}*.synthetics.json.
configPathoptionalThe global JSON configuration used when launching tests. For more information, see the example configuration. Default: datadog-ci.json.
variablesoptionalA list of global variables to use for Synthetic tests, separated by new lines or commas. For example: START_URL=https://example.org,MY_VARIABLE=My title. Default: [].
jUnitReportoptionalThe filename for a JUnit report if you want to generate one.

Further reading

Additional helpful documentation, links, and articles: