Use Observability Pipelines’ Amazon Data Firehose source to receive logs from Amazon Data Firehose. Select and set up this source when you set up a pipeline.

Prerequisites

To use Observability Pipelines’ Amazon Data Firehose:

• Since Amazon Data Firehose can only deliver data over HTTP to an HTTPS URL, when you deploy the Observability Pipelines Worker, you need to deploy it with a publicly exposed endpoint and solve TLS termination. To solve TLS termination, you can front OPW with a load balancer or configure TLS options. See Understand HTTP endpoint delivery request and response specifications for more information.
• If your forwarders are globally configured to enable SSL, you need the appropriate TLS certificates and the password you used to create your private key.

Set up the source in the pipeline UI

Select and set up this source when you set up a pipeline. The information below is for the source settings in the pipeline UI.

Optionally, toggle the switch to enable TLS. If you enable TLS, the following certificate and key files are required:

• Server Certificate Path: The path to the certificate file that has been signed by your Certificate Authority (CA) Root File in DER or PEM (X.509).
• CA Certificate Path: The path to the certificate file that is your Certificate Authority (CA) Root File in DER or PEM (X.509).
• Private Key Path: The path to the .key private key file that belongs to your Server Certificate Path in DER or PEM (PKCS#8) format.

Send logs to the Observability Pipelines Worker over Amazon Data Firehose

Since Amazon Data Firehose can only deliver data over HTTP to an HTTPS URL, when you deploy the Observability Pipelines Worker, you need to deploy it with a publicly exposed endpoint and solve TLS termination. To solve TLS termination, you can front OPW with a load balancer or configure TLS options. See Understand HTTP endpoint delivery request and response specifications for more information.

To send logs to the Observability Pipelines Worker, set up an Amazon Data Firehose stream with an HTTP endpoint destination in the region where your logs are. Configure the endpoint URL to the endpoint where OPW is deployed.