---
isPrivate: true
title: (LEGACY) Set Up the Observability Pipelines Worker
description: Datadog, the leading service for cloud-scale monitoring.
breadcrumbs: >-
  Docs > Observability Pipelines > (LEGACY) Observability Pipelines
  Documentation > (LEGACY) Set Up the Observability Pipelines Worker
---

# (LEGACY) Set Up the Observability Pipelines Worker

{% callout %}
# Important note for users on the following Datadog sites: app.ddog-gov.com

{% alert level="danger" %}
This product is not supported for your selected [Datadog site](https://docs.datadoghq.com/getting_started/site). ().
{% /alert %}

{% /callout %}

{% alert level="warning" %}
If you upgrade your OP Workers version 1.8 or below to version 2.0 or above, your existing pipelines will break. Do not upgrade your OP Workers if you want to continue using OP Workers version 1.8 or below. If you want to use OP Worker 2.0 or above, you must migrate your OP Worker 1.8 or earlier pipelines to OP Worker 2.x.Datadog recommends that you update to OP Worker versions 2.0 or above. Upgrading to a major OP Worker version and keeping it updated is the only supported way to get the latest OP Worker functionality, fixes, and security updates.
{% /alert %}

## Overview{% #overview %}

The [Observability Pipelines Worker](https://docs.datadoghq.com/observability_pipelines/legacy/#what-is-observability-pipelines-and-the-observability-pipelines-worker) can collect, process, and route logs from any source to any destination. Using Datadog, you can build and manage all of your Observability Pipelines Worker deployments at scale.

There are several ways to get started with the Observability Pipelines Worker.

- Quickstart: Install the Worker with a simple pipeline that emits demo data to get started quickly.
- [Datadog setup guide](https://docs.datadoghq.com/observability_pipelines/legacy/setup/datadog/): Install the Worker with an out-of-the-box pipeline for receiving and routing data from your Datadog Agents to Datadog.
- [Datadog archiving setup guide](https://docs.datadoghq.com/observability_pipelines/legacy/setup/datadog_with_archiving/): Install the Worker with an out-of-the-box pipeline for receiving and routing data from your Datadog Agents to Datadog and S3.
- [Splunk setup guide](https://docs.datadoghq.com/observability_pipelines/legacy/setup/splunk/): Install the Worker with an out-of-the-box pipeline for receiving and routing data from Splunk HEC to both Splunk and Datadog.

This document walks you through the quickstart installation steps and then provides resources for next steps. Use and operation of this software is governed by the [End User License Agreement](https://www.datadoghq.com/legal/eula/).

## Deployment Modes{% #deployment-modes %}

{% alert level="danger" %}
Remote configuration for Observability Pipelines is in private beta. Contact [Datadog support](https://docs.datadoghq.com/help/) or your Customer Success Manager for access.
{% /alert %}

If you are enrolled in the private beta of [Remote Configuration](https://docs.datadoghq.com/agent/remote_config), you can remotely roll out changes to your Workers from the Datadog UI, rather than make updates to your pipeline configuration in a text editor and then manually rolling out your changes. Choose your deployment method when you create a pipeline and install your Workers.

See Updating deployment modes on how to change the deployment mode after a pipeline is deployed.

## Prerequisites{% #prerequisites %}

To install the Observability Pipelines Worker, you need the following:

- A valid [Datadog API key](https://docs.datadoghq.com/account_management/api-app-keys/#api-keys).
- A pipeline ID.

To generate a new API key and pipeline:

1. Navigate to [Observability Pipelines](https://app.datadoghq.com/observability-pipelines).
1. Click **New Pipeline**.
1. Enter a name for your pipeline.
1. Click **Next**.
1. Select the template you want and follow the instructions.

## Quickstart{% #quickstart %}

Follow the below instructions to install the Worker and deploy a sample pipeline configuration that uses demo data.

### Install the Observability Pipelines Worker{% #install-the-observability-pipelines-worker %}

{% tab title="Docker" %}
The Observability Pipelines Worker Docker image is published to Docker Hub [here](https://hub.docker.com/r/datadog/observability-pipelines-worker).

1. Download the [sample pipeline configuration file](https://docs.datadoghq.com/resources/yaml/observability_pipelines/quickstart/pipeline.yaml). This configuration emits demo data, parses and structures the data, and then sends them to the console and Datadog. See [Configurations](https://docs.datadoghq.com/observability_pipelines/legacy/configurations/) for more information about the source, transform, and sink used in the sample configuration.

1. Run the following command to start the Observability Pipelines Worker with Docker:

   ```shell
   docker run -i -e DD_API_KEY=<API_KEY> \
     -e DD_OP_PIPELINE_ID=<PIPELINE_ID> \
     -e DD_SITE=<SITE> \
     -p 8282:8282 \
     -v ./pipeline.yaml:/etc/observability-pipelines-worker/pipeline.yaml:ro \
     datadog/observability-pipelines-worker run
   ```

Replace `<API_KEY>` with your Datadog API key, `<PIPELINES_ID>` with your Observability Pipelines configuration ID, and `<SITE>` with . **Note**: `./pipeline.yaml` must be the relative or absolute path to the configuration you downloaded in step 1.

{% /tab %}

{% tab title="AWS EKS" %}

1. Download the [Helm chart values file](https://docs.datadoghq.com/resources/yaml/observability_pipelines/quickstart/aws_eks.yaml) for AWS EKS. See [Configurations](https://docs.datadoghq.com/observability_pipelines/legacy/configurations/) for more information about the source, transform, and sink used in the sample configuration.

1. In the Helm chart, replace the `datadog.apiKey` and `datadog.pipelineId` values to match your pipeline and use  for the `site` value. Then, install it in your cluster with the following commands:

   ```shell
   helm repo add datadog https://helm.datadoghq.com
   ```

   ```shell
   helm repo update
   ```

   ```shell
   helm upgrade --install \
       opw datadog/observability-pipelines-worker \
       -f aws_eks.yaml
   ```

{% /tab %}

{% tab title="Azure AKS" %}

1. Download the [Helm chart values file](https://docs.datadoghq.com/resources/yaml/observability_pipelines/quickstart/azure_aks.yaml) for Azure AKS. See [Configurations](https://docs.datadoghq.com/observability_pipelines/legacy/configurations/) for more information about the source, transform, and sink used in the sample configuration.

1. In the Helm chart, replace the `datadog.apiKey` and `datadog.pipelineId` values to match your pipeline and use  for the `site` value. Then, install it in your cluster with the following commands:

   ```shell
   helm repo add datadog https://helm.datadoghq.com
   ```

   ```shell
   helm repo update
   ```

   ```shell
   helm upgrade --install \
     opw datadog/observability-pipelines-worker \
     -f azure_aks.yaml
   ```

{% /tab %}

{% tab title="Google GKE" %}

1. Download the [Helm chart values file](https://docs.datadoghq.com/resources/yaml/observability_pipelines/quickstart/google_gke.yaml) for Google GKE. See [Configurations](https://docs.datadoghq.com/observability_pipelines/legacy/configurations/) for more information about the source, transform, and sink used in the sample configuration.

1. In the Helm chart, replace the `datadog.apiKey` and `datadog.pipelineId` values to match your pipeline and use  for the `site` value. Then, install it in your cluster with the following commands:

   ```shell
   helm repo add datadog https://helm.datadoghq.com
   ```

   ```shell
   helm repo update
   ```

   ```shell
   helm upgrade --install \
     opw datadog/observability-pipelines-worker \
     -f google_gke.yaml
   ```

{% /tab %}

{% tab title="APT-based Linux" %}
Install the Worker with the one-line install script or manually.

#### One-line installation script{% #one-line-installation-script %}

1. Run the one-line install command to install the Worker. Replace `<DD_API_KEY>` with your Datadog API key, `<PIPELINES_ID>` with your Observability Pipelines ID, and `<SITE>` with .

   ```
   DD_API_KEY=<DD_API_KEY> DD_OP_PIPELINE_ID=<PIPELINES_ID> DD_SITE=<SITE> bash -c "$(curl -L https://install.datadoghq.com/scripts/install_script_op_worker1.sh)"
   ```

1. Download the [sample configuration file](https://docs.datadoghq.com/resources/yaml/observability_pipelines/datadog/pipeline.yaml) to `/etc/observability-pipelines-worker/pipeline.yaml` on the host. See [Configurations](https://docs.datadoghq.com/observability_pipelines/legacy/configurations/) for more information about the source, transform, and sink used in the sample configuration.

1. Start the worker:

   ```
   sudo systemctl restart observability-pipelines-worker
   ```

#### Manual installation{% #manual-installation %}

1. Run the following commands to set up APT to download through HTTPS:

   ```
   sudo apt-get update
   sudo apt-get install apt-transport-https curl gnupg
   ```

1. Run the following commands to set up the Datadog `deb` repo on your system and create a Datadog archive keyring:

   ```
   sudo sh -c "echo 'deb [signed-by=/usr/share/keyrings/datadog-archive-keyring.gpg] https://apt.datadoghq.com/ stable observability-pipelines-worker-1' > /etc/apt/sources.list.d/datadog-observability-pipelines-worker.list"
   sudo touch /usr/share/keyrings/datadog-archive-keyring.gpg
   sudo chmod a+r /usr/share/keyrings/datadog-archive-keyring.gpg
   curl https://keys.datadoghq.com/DATADOG_APT_KEY_CURRENT.public | sudo gpg --no-default-keyring --keyring /usr/share/keyrings/datadog-archive-keyring.gpg --import --batch
   curl https://keys.datadoghq.com/DATADOG_APT_KEY_06462314.public | sudo gpg --no-default-keyring --keyring /usr/share/keyrings/datadog-archive-keyring.gpg --import --batch
   curl https://keys.datadoghq.com/DATADOG_APT_KEY_F14F620E.public | sudo gpg --no-default-keyring --keyring /usr/share/keyrings/datadog-archive-keyring.gpg --import --batch
   curl https://keys.datadoghq.com/DATADOG_APT_KEY_C0962C7D.public | sudo gpg --no-default-keyring --keyring /usr/share/keyrings/datadog-archive-keyring.gpg --import --batch
   ```

1. Run the following commands to update your local `apt` repo and install the Worker:

   ```
   sudo apt-get update
   sudo apt-get install observability-pipelines-worker datadog-signing-keys
   ```

1. Add your keys and the site () to the Worker's environment variables:

   ```
   sudo cat <<-EOF > /etc/default/observability-pipelines-worker
   DD_API_KEY=<API_KEY>
   DD_OP_PIPELINE_ID=<PIPELINE_ID>
   DD_SITE=<SITE>
   EOF
   ```

1. Download the [sample configuration file](https://docs.datadoghq.com/resources/yaml/observability_pipelines/datadog/pipeline.yaml) to `/etc/observability-pipelines-worker/pipeline.yaml` on the host.

1. Start the Worker:

   ```
   sudo systemctl restart observability-pipelines-worker
   ```

{% /tab %}

{% tab title="RPM-based Linux" %}
Install the Worker with the one-line install script or manually.

#### One-line installation script{% #one-line-installation-script %}

1. Run the one-line install command to install the Worker. Replace `<DD_API_KEY>` with your Datadog API key, `<PIPELINES_ID>` with your Observability Pipelines ID, and `<SITE>` with .

   ```
   DD_API_KEY=<DD_API_KEY> DD_OP_PIPELINE_ID=<PIPELINES_ID> DD_SITE=<SITE> bash -c "$(curl -L https://install.datadoghq.com/scripts/install_script_op_worker1.sh)"
   ```

1. Download the [sample configuration file](https://docs.datadoghq.com/resources/yaml/observability_pipelines/quickstart/pipeline.yaml) to `/etc/observability-pipelines-worker/pipeline.yaml` on the host. See [Configurations](https://docs.datadoghq.com/observability_pipelines/legacy/configurations/) for more information about the source, transform, and sink used in the sample configuration.

1. Run the following command to start the Worker:

   ```
   sudo systemctl restart observability-pipelines-worker
   ```

#### Manual installation{% #manual-installation %}

1. Run the following commands to set up the Datadog `rpm` repo on your system:

   ```
   cat <<EOF > /etc/yum.repos.d/datadog-observability-pipelines-worker.repo
   [observability-pipelines-worker]
   name = Observability Pipelines Worker
   baseurl = https://yum.datadoghq.com/stable/observability-pipelines-worker-1/\$basearch/
   enabled=1
   gpgcheck=1
   repo_gpgcheck=1
   gpgkey=https://keys.datadoghq.com/DATADOG_RPM_KEY_CURRENT.public
          https://keys.datadoghq.com/DATADOG_RPM_KEY_4F09D16B.public
          https://keys.datadoghq.com/DATADOG_RPM_KEY_B01082D3.public
          https://keys.datadoghq.com/DATADOG_RPM_KEY_FD4BF915.public
          https://keys.datadoghq.com/DATADOG_RPM_KEY_E09422B3.public
   EOF
   ```

**Note:** If you are running RHEL 8.1 or CentOS 8.1, use `repo_gpgcheck=0` instead of `repo_gpgcheck=1` in the configuration above.

1. Update your packages and install the Worker:

   ```
   sudo yum makecache
   sudo yum install observability-pipelines-worker
   ```

1. Add your keys and the site () to the Worker's environment variables:

   ```
   sudo cat <<-EOF > /etc/default/observability-pipelines-worker
   DD_API_KEY=<API_KEY>
   DD_OP_PIPELINE_ID=<PIPELINE_ID>
   DD_SITE=<SITE>
   EOF
   ```

1. Download the [sample configuration file](https://docs.datadoghq.com/resources/yaml/observability_pipelines/quickstart/pipeline.yaml) to `/etc/observability-pipelines-worker/pipeline.yaml` on the host. See [Configurations](https://docs.datadoghq.com/observability_pipelines/legacy/configurations/) for more information about the source, transform, and sink used in the sample configuration.

1. Run the following command to start the Worker:

   ```
   sudo systemctl restart observability-pipelines-worker
   ```

{% /tab %}

{% tab title="Terraform (AWS)" %}

1. Download the [sample configuration](https://docs.datadoghq.com/resources/yaml/observability_pipelines/quickstart/terraform_opw.tf).
1. Set up the Worker module in your existing Terraform using the sample configuration. Make sure to update the values in `vpc-id`, `subnet-ids`, and `region` to match your AWS deployment in the configuration. Also,update the values in `datadog-api-key` and `pipeline-id` to match your pipeline.

See [Configurations](https://docs.datadoghq.com/observability_pipelines/legacy/configurations/) for more information about the source, transform, and sink used in the sample configuration.
{% /tab %}

See [Working with Data](https://docs.datadoghq.com/observability_pipelines/legacy/working_with_data/) for more information on transforming your data.

## Updating deployment modes{% #updating-deployment-modes %}

After deploying a pipeline, you can also switch deployment methods, such as going from a manually managed pipeline to a remote configuration enabled pipeline or vice versa.

If you want to switch from a remote configuration deployment to a manually managed deployment:

1. Navigate to Observability Pipelines and select the pipeline.
1. Click the settings cog.
1. In **Deployment Mode**, select **manual** to enable it.
1. Set the `DD_OP_REMOTE_CONFIGURATION_ENABLED` flag to `false` and restart the Worker. Workers that are not restarted with this flag continue to be remote configuration enabled, which means that the Workers are not updated manually through a local configuration file.

If you want to switch from manually managed deployment to a remote configuration deployment:

1. Navigate to Observability Pipelines and select the pipeline.
1. Click the settings cog.
1. In **Deployment Mode**, select **Remote Configuration** to enable it.
1. Set the `DD_OP_REMOTE_CONFIGURATION_ENABLED` flag to `true` and restart the Worker. Workers that are not restarted with this flag are not polled for configurations deployed in the UI.
1. Deploy a version in your version history, so that the Workers receive the new version configuration. Click on a version. Click **Edit as Draft** and then click **Deploy**.

## Next steps{% #next-steps %}

The quickstart walked you through installing the Worker and deploying a sample pipeline configuration. For instructions on how to install the Worker to receive and route data from your Datadog Agents to Datadog or to receive and route data from your Splunk HEC to Splunk and Datadog, select your specific use case:

- [Datadog](datadog)
- [Splunk](splunk)

For recommendations on deploying and scaling multiple Workers:

- See [Deployment Design and Principles](https://docs.datadoghq.com/observability_pipelines/legacy/production_deployment_overview/) for information on what to consider when designing your Observability Pipelines architecture.
- See [Best Practices for OP Worker Aggregator Architecture](https://docs.datadoghq.com/observability_pipelines/legacy/architecture/).

## Further reading{% #further-reading %}

- [Working with data in Observability Pipelines](https://docs.datadoghq.com/observability_pipelines/legacy/working_with_data/)
- [Deployment Design and Principles for the Observability Pipelines Worker](https://docs.datadoghq.com/observability_pipelines/legacy/production_deployment_overview/)
- [Production deployment design and principles for the Observability Pipelines Worker](https://docs.datadoghq.com/observability_pipelines/legacy/architecture/)
- [Safe and Secure Local Processing with Observability Pipelines](https://dtdg.co/d22op)