1. Click Select API key to choose the Datadog API key you want to use.
2. Run the command provided in the UI to install the Worker. The command is automatically populated with the environment variables you entered earlier.

   docker run -i -e DD_API_KEY=<DATADOG_API_KEY> \
       -e DD_OP_PIPELINE_ID=<PIPELINE_ID> \
       -e DD_SITE=<DATADOG_SITE> \
       -e <SOURCE_ENV_VARIABLE> \
       -e <DESTINATION_ENV_VARIABLE> \
       -p 8088:8088 \
       datadog/observability-pipelines-worker run
   

   Note: By default, the docker run command exposes the same port the Worker is listening on. If you want to map the Worker’s container port to a different port on the Docker host, use the -p | --publish option in the command:

   -p 8282:8088 datadog/observability-pipelines-worker run
   

3. Navigate back to the Observability Pipelines installation page and click Deploy.

See Update Existing Pipelines if you want to make changes to your pipeline’s configuration.

1. Download the Helm chart values file. If you are not using a managed service such as Amazon EKS, Google GKE, or Azure AKS, see Self-hosted and self-managed Kubernetes clusters before continuing to the next step.
2. Click Select API key to choose the Datadog API key you want to use.
3. Add the Datadog chart repository to Helm:

   helm repo add datadog https://helm.datadoghq.com
   

   If you already have the Datadog chart repository, run the following command to make sure it is up to date:

   helm repo update
   

4. Run the command provided in the UI to install the Worker. The command is automatically populated with the environment variables you entered earlier.

   helm upgrade --install opw \
   -f values.yaml \
   --set datadog.apiKey=<DATADOG_API_KEY> \
   --set datadog.pipelineId=<PIPELINE_ID> \
   --set <SOURCE_ENV_VARIABLES> \
   --set <DESTINATION_ENV_VARIABLES> \
   --set service.ports[0].protocol=TCP,service.ports[0].port=<SERVICE_PORT>,service.ports[0].targetPort=<TARGET_PORT> \
   datadog/observability-pipelines-worker
   

   Note: By default, the Kubernetes Service maps incoming port <SERVICE_PORT> to the port the Worker is listening on (<TARGET_PORT>). If you want to map the Worker’s pod port to a different incoming port of the Kubernetes Service, use the following service.ports[0].port and service.ports[0].targetPort values in the command:

   --set service.ports[0].protocol=TCP,service.ports[0].port=8088,service.ports[0].targetPort=8282
   

5. Navigate back to the Observability Pipelines installation page and click Deploy.

See Update Existing Pipelines if you want to make changes to your pipeline’s configuration.

Self-hosted and self-managed Kubernetes clusters

If you are running a self-hosted and self-managed Kubernetes cluster, and defined zones with node labels using topology.kubernetes.io/zone, then you can use the Helm chart values file as is. However, if you are not using the label topology.kubernetes.io/zone, you need to update the topologyKey in the values.yaml file to match the key you are using. Or if you run your Kubernetes install without zones, remove the entire topology.kubernetes.io/zone section.

Follow the steps below if you want to use the one-line installation script to install the Worker. Otherwise, see Manually install the Worker on Linux.

1. Click Select API key to choose the Datadog API key you want to use.

2. Run the one-step command provided in the UI to install the Worker.

   Note: The environment variables used by the Worker in /etc/default/observability-pipelines-worker are not updated on subsequent runs of the install script. If changes are needed, update the file manually and restart the Worker.

3. Navigate back to the Observability Pipelines installation page and click Deploy.

See Update Existing Pipelines if you want to make changes to your pipeline’s configuration.

1. Select one of the options in the dropdown to provide the expected log volume for the pipeline:

   Option	Description
   Unsure	Use this option if you are not able to project the log volume or you want to test the Worker. This option provisions the EC2 Auto Scaling group with a maximum of 2 general purpose t4g.large instances.
   1-5 TB/day	This option provisions the EC2 Auto Scaling group with a maximum of 2 compute optimized instances c6g.large.
   5-10 TB/day	This option provisions the EC2 Auto Scaling group with a minimum of 2 and a maximum of 5 compute optimized c6g.large instances.
   >10 TB/day	Datadog recommends this option for large-scale production deployments. It provisions the EC2 Auto Scaling group with a minimum of 2 and a maximum of 10 compute optimized c6g.xlarge instances.

   Note: All other parameters are set to reasonable defaults for a Worker deployment, but you can adjust them for your use case as needed in the AWS Console before creating the stack.

2. Select the AWS region you want to use to install the Worker.

3. Click Select API key to choose the Datadog API key you want to use.

4. Click Launch CloudFormation Template to navigate to the AWS Console to review the stack configuration and then launch it. Make sure the CloudFormation parameters are as expected.

5. Select the VPC and subnet you want to use to install the Worker.

6. Review and check the necessary permissions checkboxes for IAM. Click Submit to create the stack. CloudFormation handles the installation at this point; the Worker instances are launched, the necessary software is downloaded, and the Worker starts automatically.

7. Navigate back to the Observability Pipelines installation page and click Deploy.

See Update Existing Pipelines if you want to make changes to your pipeline’s configuration.

1. Set up APT transport for downloading using HTTPS:

   sudo apt-get update
   sudo apt-get install apt-transport-https curl gnupg
   

2. Run the following commands to set up the Datadog deb repo on your system and create a Datadog archive keyring:

   sudo sh -c "echo 'deb [signed-by=/usr/share/keyrings/datadog-archive-keyring.gpg] https://apt.datadoghq.com/ stable observability-pipelines-worker-2' > /etc/apt/sources.list.d/datadog-observability-pipelines-worker.list"
   sudo touch /usr/share/keyrings/datadog-archive-keyring.gpg
   sudo chmod a+r /usr/share/keyrings/datadog-archive-keyring.gpg
   curl https://keys.datadoghq.com/DATADOG_APT_KEY_CURRENT.public | sudo gpg --no-default-keyring --keyring /usr/share/keyrings/datadog-archive-keyring.gpg --import --batch
   curl https://keys.datadoghq.com/DATADOG_APT_KEY_06462314.public | sudo gpg --no-default-keyring --keyring /usr/share/keyrings/datadog-archive-keyring.gpg --import --batch
   curl https://keys.datadoghq.com/DATADOG_APT_KEY_F14F620E.public | sudo gpg --no-default-keyring --keyring /usr/share/keyrings/datadog-archive-keyring.gpg --import --batch
   curl https://keys.datadoghq.com/DATADOG_APT_KEY_C0962C7D.public | sudo gpg --no-default-keyring --keyring /usr/share/keyrings/datadog-archive-keyring.gpg --import --batch
   

3. Run the following commands to update your local apt repo and install the Worker:

   sudo apt-get update
   sudo apt-get install observability-pipelines-worker datadog-signing-keys
   

4. Add your keys, site (for example, datadoghq.com for US1), source, and destination environment variables to the Worker’s environment file:

   sudo cat <<EOF > /etc/default/observability-pipelines-worker
   DD_API_KEY=<DATADOG_API_KEY>
   DD_OP_PIPELINE_ID=<PIPELINE_ID>
   DD_SITE=<DATADOG_SITE>
   <SOURCE_ENV_VARIABLES>
   <DESTINATION_ENV_VARIABLES>
   EOF
   

5. Start the worker:

   sudo systemctl restart observability-pipelines-worker
   

Note: The environment variables used by the Worker in /etc/default/observability-pipelines-worker are not updated on subsequent runs of the install script. If changes are needed, update the file manually and restart the Worker.

See Update Existing Pipelines if you want to make changes to your pipeline’s configuration.

1. Set up the Datadog rpm repo on your system with the below command.
   Note: If you are running RHEL 8.1 or CentOS 8.1, use repo_gpgcheck=0 instead of repo_gpgcheck=1 in the configuration below.

   cat <<EOF > /etc/yum.repos.d/datadog-observability-pipelines-worker.repo
   [observability-pipelines-worker]
   name = Observability Pipelines Worker
   baseurl = https://yum.datadoghq.com/stable/observability-pipelines-worker-2/\$basearch/
   enabled=1
   gpgcheck=1
   repo_gpgcheck=1
   gpgkey=https://keys.datadoghq.com/DATADOG_RPM_KEY_CURRENT.public
       https://keys.datadoghq.com/DATADOG_RPM_KEY_B01082D3.public
   EOF
   

2. Update your packages and install the Worker:

   sudo yum makecache
   sudo yum install observability-pipelines-worker
   

3. Add your keys, site (for example, datadoghq.com for US1), source, and destination environment variables to the Worker’s environment file:

   sudo cat <<-EOF > /etc/default/observability-pipelines-worker
   DD_API_KEY=<API_KEY>
   DD_OP_PIPELINE_ID=<PIPELINE_ID>
   DD_SITE=<SITE>
   <SOURCE_ENV_VARIABLES>
   <DESTINATION_ENV_VARIABLES>
   EOF
   

4. Start the worker:

   sudo systemctl restart observability-pipelines-worker
   

5. Navigate back to the Observability Pipelines installation page and click Deploy.

Note: The environment variables used by the Worker in /etc/default/observability-pipelines-worker are not updated on subsequent runs of the install script. If changes are needed, update the file manually and restart the Worker.

See Update Existing Pipelines if you want to make changes to your pipeline’s configuration.