Use Observability Pipelines’ Google Chronicle destination to send logs to Google Chronicle.

The Observability Pipelines Worker uses standard Google authentication methods. See Authentication methods at Google for more information about choosing the authentication method for your use case.

Setup

Set up the Google Chronicle destination and its environment variables when you set up a pipeline. The information below is configured in the pipelines UI.

Set up the destination

To set up the Worker’s Google Chronicle destination:

1. Enter the customer ID for your Google Chronicle instance.
2. If you have a credentials JSON file, enter the path to your credentials JSON file. The credentials file must be placed under DD_OP_DATA_DIR/config. Alternatively, you can use the GOOGLE_APPLICATION_CREDENTIALS environment variable to provide the credential path.
   • If you’re using workload identity on Google Kubernetes Engine (GKE), the GOOGLE_APPLICATION_CREDENTIALS is provided for you.
   • The Worker uses standard Google authentication methods.
3. Select JSON or Raw encoding in the dropdown menu.
4. Enter the log type. See template syntax if you want to route logs to different log types based on specific fields in your logs.
5. Optionally, toggle the switch to enable Buffering Options.
   Note: Buffering options is in Preview. Contact your account manager to request access.
   • If left disabled, the maximum size for buffering is 500 events.
   • If enabled:
     1. Select the buffer type you want to set (Memory or Disk).
     2. Enter the buffer size and select the unit.

Note: Logs sent to the Google Chronicle destination must have ingestion labels. For example, if the logs are from a A10 load balancer, it must have the ingestion label A10_LOAD_BALANCER. See Google Cloud’s Support log types with a default parser for a list of available log types and their respective ingestion labels.

Set the environment variables

• Google Chronicle endpoint URL:
  • Stored in the environment variable: DD_OP_DESTINATION_GOOGLE_CHRONICLE_UNSTRUCTURED_ENDPOINT_URL.

How the destination works

Event batching

A batch of events is flushed when one of these parameters is met. See event batching for more information.