Overview

AWS CodePipeline is a fully managed continuous delivery service that helps you automate your release pipelines for fast and reliable application and infrastructure updates.

Set up tracing on AWS CodePipeline to collect data about pipeline executions, analyze performance bottlenecks or operational issues, and monitor your deployment workflows.

Compatibility

*AWS CodePipeline running pipelines don’t have Git information until they have finished.
**AWS CodePipeline logs correlation is only available for AWS CodeBuild actions.

Configure the Datadog integration

To set up the integration between AWS CodePipeline and Pipeline Visibility, create two AWS resources.

API Destination

An HTTP endpoint pointing to Datadog’s intake.

AWS EventBridge Rule

A rule that forwards CodePipeline events to the API destination.

You can create these resources separately, or at the same time, during the EventBridge Rule creation process. For more information about monitoring pipeline events, see the official AWS guide.

Create the API destination

1. In the AWS Console, navigate to EventBridge > API destinations and click Create API destination.
2. Choose a name for the API Destination (for example, datadog-ci-visibility-api) and optionally add a description.
3. Under API destination endpoint, input https://webhook-intake./api/v2/webhook.
4. Under HTTP method, select POST.
5. Under Connection type, select Create a new connection:
   1. Choose a name for the connection (for example, datadog-ci-visibility-connection) and optionally add a description.
   2. Under Destination type, select Other.
   3. Under Authorization type, select API key. Input DD-API-KEY as the API key name and add your Datadog API key in the Value field.
6. Click Create.

Create the EventBridge rule

1. In the AWS Console, navigate to EventBridge > Rules and click Create Rule.

2. Choose a name for the rule (for example, datadog-ci-visibility-integration) and optionally add a description.

3. Leave the event bus as default, and under Rule Type, select Rule with an event pattern. Click Next.

4. Under Event Source, select AWS events or EventBridge partner events.

5. Under Creation Method, select Custom pattern (JSON editor). Then, under Event Pattern, input the following:

   {
     "source": ["aws.codepipeline"],
     "detail-type": ["CodePipeline Pipeline Execution State Change", "CodePipeline Action Execution State Change", "CodePipeline Stage Execution State Change"]
   }
   

   The JSON above sets up the integration for all of your pipelines. To restrict the set of pipelines, follow the Only monitor specific pipelines section below.

6. Click Next.

7. Under Target Types, select EventBridge API destination. Then, choose Use an existing API Destination and select the API destination that you created in the previous step. Alternatively, you can also create the API destination by following the steps outlined in the Create the API Destination section.

8. Under Headers Parameters, click Add header parameter. Input DD-CI-PROVIDER-AWSCODEPIPELINE as the key and true as the value.

9. Choose Create a new role for this specific resource (or use an existing one).

10. Review that the information is correct and create the rule.

Once you’ve created the rule, you can monitor your pipelines in Datadog.

Advanced configuration

Only monitor specific pipelines

You can optionally restrict the pipelines that are monitored by Pipeline Visibility. To do this, add the detail.pipeline filter in the rule event pattern defined when creating the EventBridge Rule. For example:

 {
   "source": ["aws.codepipeline"],
   "detail-type": ["CodePipeline Pipeline Execution State Change", "CodePipeline Action Execution State Change", "CodePipeline Stage Execution State Change"],
   "detail": {
     "pipeline": ["first-pipeline", "second-pipeline"]
   }
 }

The event pattern sets up the integration only for the first-pipeline and second-pipeline pipelines.

Correlate pipelines with tests

If you are using Test Optimization and your pipeline contains one or more AWS CodeBuild actions to execute tests, you can correlate your tests with the related pipeline inside Datadog Pipeline Visibility. For instructions, refer to Add the pipeline execution ID.

Collect job logs

The AWS CodePipeline integration supports correlating CodeBuild actions with their respective job and pipeline spans. To enable log collection for your CodeBuild actions, see the AWS log forwarding guide.

Logs are billed separately from CI Visibility. Log retention, exclusion, and indexes are configured in Logs Settings. Logs for AWS CodeBuild can be identified by the source:codebuild and sourcecategory:aws tags.

Add the pipeline execution ID as an environment variable

The pipeline execution ID is an identifier Datadog needs to uniquely identify a pipeline execution. Perform the following steps to assign a pipeline execution ID to correlate pipelines with tests and custom commands:

1. In the AWS Console, go to your pipeline configuration and click Edit
2. Go to the stage containing the AWS CodeBuild action, click Edit Stage, and then edit the relevant action.
3. Under Environment variables, add an environment variable. Name the variable DD_PIPELINE_EXECUTION_ID, and the value #{codepipeline.PipelineExecutionId}. Leave the type as Plaintext.
4. Click Done to save your changes.

The steps above allow you to add the pipeline execution ID to your CodeBuild action environment variables. For more information on working with variables, see the official AWS guide.

Visualize pipeline data in Datadog

View your data on the CI Pipeline List and Executions pages after the pipelines finish.

The CI Pipeline List page shows data for only the default branch of each repository. For more information, see Search and Manage CI Pipelines.

Further reading

Additional helpful documentation, links, and articles:

Explore Pipeline Execution Results and PerformanceDOCUMENTATION Troubleshooting CI VisibilityDOCUMENTATION Search and Manage CI PipelinesDOCUMENTATION Monitor and improve your CI/CD on AWS CodePipeline with Datadog CI VisibilityBLOG