Log collection requires the Datadog Agent v6.0+. Older versions of the Agent do not include the log collection interface. If you are not using the Agent already, follow the Agent installation instructions.
See Observability Pipelines if you want to send logs using another vendor’s collector or forwarder, or you want to preprocess your log data within your environment before shipping.
Activate log collection
Collecting logs is not enabled by default in the Datadog Agent. If you are running the Agent in a Kubernetes or Docker environment, see the dedicated Kubernetes Log Collection or Docker Log Collection documentation.
To enable log collection with an Agent running on your host, change logs_enabled: false to logs_enabled: true in the Agent’s main configuration file (datadog.yaml).
Starting with Agent v6.19+/v7.19+, HTTPS transport is the default transport used. For more details, see Agent transport.
To send logs with environment variables, configure the following:
DD_LOGS_ENABLED=true
After activating log collection, the Agent is ready to forward logs to Datadog. Next, configure the Agent on where to collect logs from.
Custom log collection
Datadog Agent v6 can collect logs and forward them to Datadog from files, the network (TCP or UDP), journald, and Windows channels:
In the conf.d/ directory at the root of your Agent’s configuration directory, create a new <CUSTOM_LOG_SOURCE>.d/ folder that is accessible by the Datadog user.
Create a new conf.yaml file in this new folder.
Add a custom log collection configuration group with the parameters below.
Below are examples of custom log collection setup:
To gather logs from your <APP_NAME> application stored in <PATH_LOG_FILE>/<LOG_FILE_NAME>.log create a <APP_NAME>.d/conf.yaml file at the root of your Agent’s configuration directory with the following content:
On Windows, use the path <DRIVE_LETTER>:\\<PATH_LOG_FILE>\\<LOG_FILE_NAME>.log, and verify that the user ddagentuser has read and write access to the log file.
Note: A log line needs to be terminated with a newline character, \n or \r\n, otherwise the Agent waits indefinitely and does not send the log line.
To capture the sender IP address and include it in the log message payload, add the following configuration to your datadog.yaml file:
logs_config:use_sourcehost_tag:true
To gather logs from your <APP_NAME> application that forwards its logs to TCP port 10518, create a <APP_NAME>.d/conf.yaml file at the root of your Agent’s configuration directory with the following content:
Edit the <CHANNEL_X> parameters with the Windows channel name you want to collect events from.
Set the corresponding source parameter to the same channel name to benefit from the integration automatic processing pipeline setup.
Follow the steps in these sections to send Windows Private Location logs to Datadog:
Configure the Agent
Enable Agent log collection by setting logs_enabled: true in the Agent configuration file.
Navigate to C:\ProgramData\Datadog\conf.d and create a folder named synthetics_worker.d.
Inside the synthetics_worker.d folder, create a file named conf.yaml using the following example as a template:
logs:- type:filepath:"C:\\Program Files\\Datadog-Synthetics\\Synthetics\\private-location-service.out.log"service:<YOUR_SERVICE>source:syntheticstags:# Defined per user preference- env:<YOUR_ENV>- private_location:<YOUR_PRIVATE_LOCATION>
Verify the user running the Agent
Since the Private Location installation folder is restricted to admin access, the Datadog Agent needs permission to access the log file. Follow these steps to verify the user running the Datadog Agent:
Press the Windows key and R, and search for Run.
Find the Datadog Agent, right-click it, and select Properties.
In the Log On tab, verify the account (the default is ddagentuser).
Close the window.
Grant permission to the user running the Agent
Go to C:\Program Files and find the synthetics_worker.d folder.
Right-click the synthetics_worker.d folder and select Properties.
Go to the Security tab.
Click Edit and add ddagentuser.
Grant the necessary permissions.
Restart the Datadog Agent through the Services screen or command line to apply the changes and begin sending logs to Datadog.
List of all available parameters for log collection:
Parameter
Required
Description
type
Yes
The type of log input source. Valid values are: tcp, udp, file, windows_event, docker, or journald.
port
Yes
If type is tcp or udp, set the port for listening to logs.
path
Yes
If type is file or journald, set the file path for gathering logs.
channel_path
Yes
If type is windows_event, list the Windows event channels for collecting logs.
service
Yes
The name of the service owning the log. If you instrumented your service with Datadog APM, this must be the same service name. Check the unified service tagging instructions when configuring service across multiple data types.
source
Yes
The attribute that defines which integration is sending the logs. If the logs do not come from an existing integration, then this field may include a custom source name. However, it is recommended that you match this value to the namespace of any related custom metrics you are collecting, for example: myapp from myapp.request.count.
include_units
No
If type is journald, list of the specific journald units to include.
exclude_paths
No
If type is file, and path contains a wildcard character, list the matching file or files to exclude from log collection. This is available for Agent version >= 6.18.
exclude_units
No
If type is journald, list of the specific journald units to exclude.
sourcecategory
No
The attribute used to define the category a source attribute belongs to, for example: source:postgres, sourcecategory:database or source: apache, sourcecategory: http_web_access.
If type is file, set the encoding for the Agent to read the file. Set it to utf-16-le for UTF-16 little-endian, utf-16-be for UTF-16 big-endian, or shift-jis for Shift JIS. If set to any other value, the Agent reads the file as UTF-8. Added utf-16-le and utf-16be in Agent v6.23/v7.23, shift-jis in Agent v6.34/v7.34
The start_position parameter is supported by file and journald tailer types. The start_position is always beginning when tailing a container.
Support:
File: Agent 6.19+/7.19+
Journald: Agent 6.38+/7.38+
If type is file:
Set the position for the Agent to start reading the file.
Valid values are beginning, end, forceBeginning, and forceEnd (default: end).
The beginning position does not support paths with wildcards.
If type is journald:
Set the position for the Agent to start reading the journal.
Valid values are beginning, end, forceBeginning, and forceEnd (default: end).
Precedence
For both file and journald tailer types, if an end or beginning position is specified, but an offset is stored, the offset takes precedence. Using forceBeginning or forceEnd forces the Agent to use the specified value even if there is a stored offset.
Further Reading
Additional helpful documentation, links, and articles: