Export a Pipeline Configuration to JSON or Terraform

This product is not supported for your selected Datadog site. ().

Overview

Setting up and updating a pipeline with Terraform or the API can be time-consuming, manual, and prone to syntax errors. Instead, you can create a pipeline or update an existing pipeline in the UI, and then export the configuration to JSON or Terraform to programmatically create or update the pipeline.

Export the configuration of a pipeline in draft mode

When building a pipeline that has not been deployed yet, you can set up the pipeline in the UI, and then export the pipeline configuration to JSON or Terraform to programmatically create it.

  1. Navigate to Observability Pipelines.
  2. Set up a pipeline by adding your source, processors, and destinations.
  3. After your pipeline has been configured, click Export Pipeline.
    The export button highlighted in the pipeline UI
  4. Click the copy or download button to export the configuration as JSON or Terraform.
    The export modal showing the pipeline configuration in JSON
  5. Deploy the pipeline configuration using the API or Terraform.
  6. Install the Worker to start sending and processing data through the pipeline.

Export the configuration of a deployed pipeline

A deployed pipeline has a Worker installed to send data through the pipeline and can be either active or inactive.

Export a pipeline’s configuration

When viewing a deployed pipeline in the UI, you can export its configuration to JSON or Terraform to programmatically create it.

  1. Navigate to Observability Pipelines.
  2. Select a deployed pipeline (status can be active or inactive) to view the configuration.
  3. Click Export Pipeline.
    The export button on a deployed pipeline page
  4. Click the copy or download button to export the configuration as JSON or Terraform.
  5. Update the JSON or Terraform configuration for any additional changes you want to make and then deploy the configuration programmatically.

Export a pipeline’s configuration after making edits to the pipeline

When editing a deployed pipeline, you can add new components such as Packs or update volume control rules in the UI and export the configuration to JSON or Terraform to deploy the changes programmatically.

  1. Navigate to Observability Pipelines.
  2. Select a deployed pipeline (status can be either active or inactive) to view the configuration.
  3. Make changes to your source, destinations, or processors.
  4. Before exiting the pipeline, click Export Pipeline.
    The export button highlighted on the pipeline edit page
  5. Click the copy or download button to export the configuration as JSON or Terraform.
    • Note: After making updates to your pipeline in the UI, the exported configuration includes the latest changes, even if you haven’t saved or deployed the changes.
      The export modal showing the pipeline configuration in Terraform
  6. Click the Observability Pipelines logo on the top left side of the page to exit the pipeline without saving the changes made in the UI.
    The Observability Pipelines log highlighted on the pipeline page
  7. Use the copied or downloaded configuration to programmatically update your pipeline.

Further reading

Additional helpful documentation, links, and articles:

Set up a pipelineDOCUMENTATION more more Update an existing pipelineDOCUMENTATION more more Observability Pipelines APIDOCUMENTATION more more Observability Pipelines Terraform resourceDOCUMENTATION more more