Log forwarding is not available for the Government site. Contact your account representative for more information.
Overview
Log Forwarding allows you to send logs from Datadog to custom destinations like Splunk, Elasticsearch, and HTTP endpoints. This means that you can use Log Pipelines to centrally collect, process, and standardize your logs in Datadog. Then, send the logs from Datadog to other tools to support individual teams’ workflows. You can choose to forward any of the ingested logs, whether or not they are indexed, to custom destinations. Logs are forwarded in JSON format and compressed with GZIP.
If a forwarding attempt fails (for example: if your destination temporarily becomes unavailable), Datadog retries periodically for 2 hours using an exponential backoff strategy. The first attempt is made following a 1-minute delay. For subsequent retries, the delay increases progressively to a maximum of 8-12 minutes (10 minutes with 20% variance).
The following metrics report on logs that have been forwarded successfully, including logs that were sent successfully after retries, as well as logs that were dropped.
datadog.forwarding.logs.bytes
datadog.forwarding.logs.count
Set up log forwarding to custom destinations
Add webhook IPs from the IP ranges list to the allowlist.
Enter the query to filter your logs for forwarding. See Search Syntax for more information.
Select the Destination Type.
Enter a name for the destination.
In the Define endpoint field, enter the endpoint to which you want to send the logs. The endpoint must start with https://.
For example, if you want to send logs to Sumo Logic, follow their Configure HTTP Source for Logs and Metrics documentation to get the HTTP Source Address URL to send data to their collector. Enter the HTTP Source Address URL in the Define endpoint field.
In the Configure Authentication section, select one of the following authentication types and provide the relevant details:
Authentication Type
Description
Example
Basic Authentication
Provide the username and password for the account to which you want to send logs.
Username: myaccount Password: mypassword
Request Header
Provide the header name and value. Example for Authorization: - Enter Authorization for Header Name. - Use a header value formatted as Basic username:password, encoded in base64.
In the Configure Destination section, enter the endpoint to which you want to send the logs. The endpoint must start with https://. For example, enter https://<your_account>.splunkcloud.com:8088. Note: /services/collector/event is automatically appended to the endpoint.
Specify the immutable ID of the Data Collection Rule (DCR) where logging routes are defined, as found on the DCR Overview page as “Immutable Id”. Note: Ensure the Monitoring Metrics Publisher role is assigned in the DCR IAM settings.
dcr-000a00a000a00000a000000aa000a0aa
Stream Declaration Name
Provide the name of the target Stream Declaration found in the Resource JSON of the DCR under streamDeclarations.
Custom-MyTable
In the Select Tags to Forward section:
a. Select whether you want All tags, No tags, or Specific Tags to be included.
b. Select whether you want to Include or Exclude specific tags, and specify which tags to include or exclude.
Click Save.
On the Log Forwarding page, hover over the status for a destination to see the percentage of logs that matched the filter criteria and have been forwarded in the past hour.
Select Custom Destinations to view a list of all existing destinations.
Click the Delete button for the destination that you want to delete, and click Confirm. This removes the destination from the configured list of destinations and logs are no longer forwarded to it.
Further reading
Additional helpful documentation, links, and articles:
