Syslog Source

Use Observability Pipelines’ rsyslog or syslog-ng to receive logs sent to rsyslog or syslog-ng. Select and set up this source when you set up a pipeline.

Prerequisites

To use Observability Pipelines’s Syslog source, your applications must be sending data in one of the following formats: RFC 6587, RFC 5424, RFC 3164. You also need to have the following information available:

  1. The bind address that your Observability Pipelines Worker (OPW) will listen on to receive logs from your applications. For example, 0.0.0.0:8088. Later on, you configure your applications to send logs to this address.
  2. The appropriate TLS certificates and the password you used to create your private key if your forwarders are globally configured to enable SSL.

Set up the source in the pipeline UI

Select and set up this source when you set up a pipeline. The information below is for the source settings in the pipeline UI.

To configure your Syslog source:

  1. In the Socket Type dropdown menu, select the communication protocol you want to use: TCP or UDP.
  2. Optionally, toggle the switch to enable TLS. If you enable TLS, the following certificate and key files are required:
    • Server Certificate Path: The path to the certificate file that has been signed by your Certificate Authority (CA) Root File in DER or PEM (X.509) format.
    • CA Certificate Path: The path to the certificate file that is your Certificate Authority (CA) Root File in DER or PEM (X.509) format.
    • Private Key Path: The path to the .key private key file that belongs to your Server Certificate Path in DER or PEM (PKCS#8) format.

Send logs to the Observability Pipelines Worker over Syslog

rsyslog

To send rsyslog logs to the Observability Pipelines Worker, update your rsyslog config file:

ruleset(name="infiles") {
action(type="omfwd" protocol="tcp" target="<OPW_HOST>" port="<OPW_PORT>")
}

<OPW_HOST> is the IP/URL of the host (or load balancer) associated with the Observability Pipelines Worker.

  • For CloudFormation installs, the LoadBalancerDNS CloudFormation output has the correct URL to use.
  • For Kubernetes installs, the internal DNS record of the Observability Pipelines Worker service can be used, for example opw-observability-pipelines-worker.default.svc.cluster.local.

syslog-ng

To send syslog-ng logs to the Observability Pipelines Worker, update your syslog-ng config file:

destination obs_pipelines {
  http(
      url("<OPW_HOST>")
      method("POST")
      body("<${PRI}>1 ${ISODATE} ${HOST:--} ${PROGRAM:--} ${PID:--} ${MSGID:--} ${SDATA:--} $MSG\n")
  );
};

<OPW_HOST> is the IP/URL of the host (or load balancer) associated with the Observability Pipelines Worker.

  • For CloudFormation installs, the LoadBalancerDNS CloudFormation output has the correct URL to use.
  • For Kubernetes installs, the internal DNS record of the Observability Pipelines Worker service can be used, for example opw-observability-pipelines-worker.default.svc.cluster.local.