Set up Tracing on a AWS CodePipeline Pipeline

Documentos > Continuous Integration Visibility > CI Pipeline Visibility in Datadog > Set up Tracing on a AWS CodePipeline Pipeline

This page is not yet available in Spanish. We are working on its translation.
If you have any questions or feedback about our current translation project, feel free to reach out to us!

Overview

AWS CodePipeline is a fully managed continuous delivery service that helps you automate your release pipelines for fast and reliable application and infrastructure updates.

Set up tracing on AWS CodePipeline to collect data about pipeline executions, analyze performance bottlenecks or operational issues, and monitor your deployment workflows.

Compatibility

Pipeline Visibility	Platform	Definition
Partial retries	Partial pipelines	View partially retried pipeline executions.
*Running pipelines	Running pipelines	View pipeline executions that are running. Queued or waiting pipelines show with status “Running” on Datadog.
**Logs correlation	Logs correlation	Correlate pipeline and job spans to logs and enable job log correlation.
Approval wait time	Approval wait time	View the amount of time jobs and pipelines wait for manual approvals.
Custom spans	Custom spans	Configure custom spans for your pipelines.

*AWS CodePipeline running pipelines don’t have Git information until they have finished.
**AWS CodePipeline logs correlation is only available for AWS CodeBuild actions.

Configure the Datadog integration

To set up the integration between AWS CodePipeline and Pipeline Visibility, create two AWS resources.

API Destination: An HTTP endpoint pointing to Datadog’s intake.
AWS EventBridge Rule: A rule that forwards CodePipeline events to the API destination.

You can create these resources separately, or at the same time, during the EventBridge Rule creation process. For more information about monitoring pipeline events, see the official AWS guide.

Create the API destination

In the AWS Console, navigate to EventBridge > API destinations and click Create API destination.
Choose a name for the API Destination (for example, datadog-ci-visibility-api) and optionally add a description.
Under API destination endpoint, input https://webhook-intake./api/v2/webhook.
Under HTTP method, select POST.
Under Connection type, select Create a new connection:
1. Choose a name for the connection (for example, datadog-ci-visibility-connection) and optionally add a description.
2. Under Destination type, select Other.
3. Under Authorization type, select API key. Input DD-API-KEY as the API key name and add your Datadog API key in the Value field.
Click Create.

Create the EventBridge rule

In the AWS Console, navigate to EventBridge > Rules and click Create Rule.
Choose a name for the rule (for example, datadog-ci-visibility-integration) and optionally add a description.
Leave the event bus as default, and under Rule Type, select Rule with an event pattern. Click Next.
Under Event Source, select AWS events or EventBridge partner events.
Under Creation Method, select Custom pattern (JSON editor). Then, under Event Pattern, input the following:
```
{
  "source": ["aws.codepipeline"],
  "detail-type": ["CodePipeline Pipeline Execution State Change", "CodePipeline Action Execution State Change", "CodePipeline Stage Execution State Change"]
}
```
The JSON above sets up the integration for all of your pipelines. To restrict the set of pipelines, follow the Only monitor specific pipelines section below.
Click Next.
Under Target Types, select EventBridge API destination. Then, choose Use an existing API Destination and select the API destination that you created in the previous step. Alternatively, you can also create the API destination by following the steps outlined in the Create the API Destination section.
Under Headers Parameters, click Add header parameter. Input DD-CI-PROVIDER-AWSCODEPIPELINE as the key and true as the value.
Choose Create a new role for this specific resource (or use an existing one).
Review that the information is correct and create the rule.

Once you’ve created the rule, you can monitor your pipelines in Datadog.

Advanced configuration

Only monitor specific pipelines

You can optionally restrict the pipelines that are monitored by Pipeline Visibility. To do this, add the detail.pipeline filter in the rule event pattern defined when creating the EventBridge Rule. For example:

 {
   "source": ["aws.codepipeline"],
   "detail-type": ["CodePipeline Pipeline Execution State Change", "CodePipeline Action Execution State Change", "CodePipeline Stage Execution State Change"],
   "detail": {
     "pipeline": ["first-pipeline", "second-pipeline"]
   }
 }

The event pattern sets up the integration only for the first-pipeline and second-pipeline pipelines.

Correlate pipelines with tests

If you are using Test Optimization and your pipeline contains one or more AWS CodeBuild actions to execute tests, you can correlate your tests with the related pipeline inside Datadog Pipeline Visibility. For instructions, refer to Add the pipeline execution ID.

Collect job logs

The AWS CodePipeline integration supports correlating CodeBuild actions with their respective job and pipeline spans. To enable log collection for your CodeBuild actions, see the AWS log forwarding guide.

Note: Log correlation for CodeBuild actions requires the CodeBuild project to have the default CloudWatch log group and log stream names.

Logs are billed separately from CI Visibility. Log retention, exclusion, and indexes are configured in Logs Settings. Logs for AWS CodeBuild can be identified by the source:codebuild and sourcecategory:aws tags.

Add the pipeline execution ID as an environment variable

The pipeline execution ID is an identifier Datadog needs to uniquely identify a pipeline execution. Perform the following steps to assign a pipeline execution ID to correlate pipelines with tests and custom commands:

In the AWS Console, go to your pipeline configuration and click Edit
Go to the stage containing the AWS CodeBuild action, click Edit Stage, and then edit the relevant action.
Under Environment variables, add an environment variable. Name the variable DD_PIPELINE_EXECUTION_ID, and the value #{codepipeline.PipelineExecutionId}. Leave the type as Plaintext.
Click Done to save your changes.

The steps above allow you to add the pipeline execution ID to your CodeBuild action environment variables. For more information on working with variables, see the official AWS guide.