Set up Tracing on a Buildkite Pipeline
CI Visibility is not available in the selected site () at this time.
Compatibility
Partial pipelines: View partially retried and downstream pipeline executions
Manual steps: View manually triggered pipelines
Queue time: View amount of time pipeline jobs sit in the queue before processing
Custom tags and metrics at runtime: Configure custom tags and metrics at runtime
The steps to activate the Datadog integration for Buildkite are:
- Go to Settings > Notification Services in Buildkite and click add a Datadog Pipeline Visibility integration.
- Fill in the form with the following information:
- Description: A description to help identify the integration in the future, such as Datadog CI Visibility integration.
- API key: your Datadog API Key.
- Datadog site:
- Pipelines: Select all pipelines or the subset of pipelines you want to trace.
- Branch filtering: Leave empty to trace all branches or select the subset of branches you want to trace.
- Click Add Datadog Pipeline Visibility Notification to save the integration.
Custom tags can be added to Buildkite traces by using the buildkite-agent meta-data set command.
Any metadata tags with a key starting with dd_tags. are added to the job and pipeline spans. These
tags can be used to create string facets to search and organize the pipelines.
The YAML below illustrates a simple pipeline where tags for the team name and the Go version have
been set.
steps:
- command: buildkite-agent meta-data set "dd_tags.team" "backend"
- command: go version | buildkite-agent meta-data set "dd_tags.go.version"
label: Go version
- commands: go test ./...
label: Run tests
The following tags are shown in the root span as well as the relevant job span in Datadog.
team: backendgo.version: go version go1.17 darwin/amd64 (output depends on the runner)
The resulting pipeline looks as follows:
Any metadata with a key starting with dd-metrics. and containing a numerical value will be set as
a metric tag that can be used to create numerical measures. You can use the buildkite-agent meta-data set
command to create such tags. This can be used for example to measure the binary size in a pipeline:
steps:
- commands:
- go build -o dst/binary .
- ls -l dst/binary | awk '{print \$5}' | tr -d '\n' | buildkite-agent meta-data set "dd_metrics.binary_size"
label: Go build
The resulting pipeline will have the tags shown below in the pipeline span:
binary_size: 502 (output depends on the file size)
In this example, you can use the value of binary_size to plot the change in the binary size over time.
Visualize pipeline data in Datadog
The Pipelines and Pipeline Executions pages populate with data after the pipelines finish.
Note: The Pipelines page shows data for only the default branch of each repository.
View partial and downstream pipelines
In the Pipeline Executions page, you can use the filters below in the search bar:
Downstream Pipeline- Possible values:
true, false Manually Triggered- Possible values:
true, false Partial Pipeline- Possible values:
retry, paused, resumed
These filters can also be applied through the facet panel on the left hand side of the page.
Further reading
Additional helpful documentation, links, and articles:
