Overview

Setting up and updating a pipeline with Terraform or the API can be time-consuming, manual, and prone to syntax errors. Instead, you can create a pipeline or update an existing pipeline in the UI, and then export the configuration to JSON or Terraform to programmatically create or update the pipeline. Additionally, you can import a pipeline’s configuration in JSON to the UI.

Export the configuration of a pipeline in draft mode

When building a pipeline that has not been deployed yet, you can set up the pipeline in the UI, and then export the pipeline configuration to JSON or Terraform to programmatically create it.

1. Navigate to Observability Pipelines.
2. Set up a pipeline by adding your source, processors, and destinations.
3. After your pipeline has been configured, click Export Pipeline.

4. Click the copy or download button to export the configuration as JSON or Terraform.

5. Deploy the pipeline configuration using the API or Terraform.
6. Install the Worker to start sending and processing data through the pipeline.

Export the configuration of a deployed pipeline

A deployed pipeline has a Worker installed to send data through the pipeline and can be either active or inactive.

Export a pipeline’s configuration

When viewing a deployed pipeline in the UI, you can export its configuration to JSON or Terraform to programmatically create it.

1. Navigate to Observability Pipelines.
2. Select a deployed pipeline (status can be active or inactive) to view the configuration.
3. Click Export Pipeline.

4. Click the copy or download button to export the configuration as JSON or Terraform.
5. Update the JSON or Terraform configuration for any additional changes you want to make and then deploy the configuration programmatically.
   • API documentation
   • Terraform documentation

Export a pipeline’s configuration after making edits to the pipeline

When editing a deployed pipeline, you can add new components such as Packs or update volume control rules in the UI and export the configuration to JSON or Terraform to deploy the changes programmatically.

1. Navigate to Observability Pipelines.
2. Select a deployed pipeline (status can be either active or inactive) to view the configuration.
3. Make changes to your source, destinations, or processors.
4. Before exiting the pipeline, click Export Pipeline.

5. Click the copy or download button to export the configuration as JSON or Terraform.
   • Note: After making updates to your pipeline in the UI, the exported configuration includes the latest changes, even if you haven’t saved or deployed the changes.

6. Click the Observability Pipelines logo on the top left side of the page to exit the pipeline without saving the changes made in the UI.

7. Use the copied or downloaded configuration to programmatically update your pipeline.
   • API documentation
   • Terraform documentation

Import a pipeline’s JSON configuration to the UI

To help build your pipelines and transfer pipeline configurations across Datadog orgs, you can also import a pipeline’s JSON to the UI.

1. Navigate to Observability Pipelines.
2. Select a draft pipeline to view the configuration.
3. Click on the settings and select Import Pipeline

4. Choose a pipeline configuration as a JSON file from your computer.

5. Click Import. This replaces your current pipeline configuration with the imported JSON’s pipeline configuration.
6. Install the Observability Pipelines Worker and deploy your pipeline using the UI.

Further reading

Additional helpful documentation, links, and articles:

Set up a pipelineDOCUMENTATION Update an existing pipelineDOCUMENTATION Observability Pipelines APIDOCUMENTATION Observability Pipelines Terraform resourceDOCUMENTATION