Best Practices for OPW Aggregator Architecture

Observability Pipelines is not available on the US1-FED Datadog site.

This guide is for large-scale production-level deployments.

Overview

The Observability Pipelines Worker’s (OPW) aggregator architecture deploys the Observability Pipelines Worker as a standalone service for centralized data processing and routing:

A diagram showing the network load balancer receiving data from various sources and sending the data to the Observability Pipelines Worker aggregator, which has multiple Workers in different availability zones and sends data to various sinks

Deploy Observability Pipelines Worker into your infrastructure, like any other service to intercept and manipulate data, and then forward it to your destinations. Each Observability Pipelines Worker instance operates independently, so that you can scale the architecture with a simple load balancer.

This guide walks you through the recommended aggregator architecture for new Observability Pipelines Worker users. Specifically, these topics include: