Use Observability Pipelines’ CloudPrem destination to send logs to Datadog CloudPrem.

Prerequisites

Before configuring the destination, you need to deploy a CloudPrem cluster. Learn how to install it in the CloudPrem installation section.

Setup

Set up the CloudPrem destination and its environment variables when you set up a pipeline.

Set up the destination

Optional settings

Toggle the switch to enable Buffering Options. Enable a configurable buffer on your destination to ensure intermittent latency or an outage at the destination doesn’t create immediate backpressure, and allow events to continue to be ingested from your source. Disk buffers can also increase pipeline durability by writing logs to disk, ensuring buffered logs persist through a Worker restart. See Configurable buffers for destinations for more information.

• If left unconfigured, your destination uses a memory buffer with a capacity of 500 events.
• To configure a buffer on your destination:
  1. Select the buffer type you want to set (Memory or Disk).
  2. Enter the buffer size and select the unit.
     1. Maximum memory buffer size is 128 GB.
     2. Maximum disk buffer size is 500 GB.

Set secrets

These are the defaults used for secret identifiers and environment variables.

Note: If you enter identifiers for your secrets and then choose to use environment variables, the environment variable is the identifier entered and prepended with DD_OP. For example, if you entered PASSWORD_1 for a password identifier, the environment variable for that password is DD_OP_PASSWORD_1.

• CloudPrem endpoint URL identifier:
  • References the intake endpoint to which Observability Pipelines sends logs.
  • In your secrets manager:
    • Define the cluster URL, such as http://cloudprem.acme.internal:7280. Note: The URL must include the port.
    • The Worker appends /api/v2/logs and /api/v1/validate to the endpoint URL, so these endpoints must be allowed if you are using forwarding or firewall rules.
  • The default identifier is DESTINATION_CLOUDPREM_ENDPOINT_URL.

• CloudPrem endpoint URL
  • Observability Pipelines sends logs to the CloudPrem intake endpoint. Define the cluster URL, such as http://cloudprem.acme.internal:7280. Note: The URL must include the port.
  • The Worker appends /api/v2/logs and /api/v1/validate to the endpoint URL, so these endpoints must be allowed if you are using forwarding or firewall rules.
  • Stored as the environment variable: DD_OP_DESTINATION_CLOUDPREM_ENDPOINT_URL.

How the destination works

Event batching

A batch of events is flushed when one of these parameters is met. See event batching for more information.