--- title: Host Agent Log collection description: Use the Datadog Agent to collect your logs and send them to Datadog breadcrumbs: Docs > Agent > Host Agent Log collection --- # Host Agent Log collection Log collection requires the Datadog Agent v6.0+. Older versions of the Agent do not include the `log collection` interface. If you are not using the Agent already, follow the [Agent installation instructions](https://app.datadoghq.com/account/settings/agent/latest). See [Observability Pipelines](https://docs.datadoghq.com/observability_pipelines.md) if you want to send logs using another vendor's collector or forwarder, or you want to preprocess your log data within your environment before shipping. ## Activate log collection{% #activate-log-collection %} Collecting logs is **not enabled** by default in the Datadog Agent. If you are running the Agent in a Kubernetes or Docker environment, see the dedicated [Kubernetes Log Collection](https://docs.datadoghq.com/containers/kubernetes/log.md) or [Docker Log Collection](https://docs.datadoghq.com/containers/docker/log.md) documentation. To enable log collection with an Agent running on your host, change `logs_enabled: false` to `logs_enabled: true` in the Agent's [main configuration file](https://docs.datadoghq.com/agent/configuration/agent-configuration-files.md) (`datadog.yaml`). In the `datadog.yaml` file: ```yaml logs_enabled: true logs_config: auto_multi_line_detection: true force_use_http: true ``` See the [sample config_template.yaml file](https://github.com/DataDog/datadog-agent/blob/master/pkg/config/config_template.yaml) for all available configuration options. {% alert level="info" %} Starting with Agent v6.19+/v7.19+, HTTPS transport is the default transport used. For more details, see [Agent transport](https://docs.datadoghq.com/agent/logs/log_transport.md). {% /alert %} To send logs with **environment variables**, configure the following: ``` DD_LOGS_ENABLED=true ``` After activating log collection, the Agent is ready to forward logs to Datadog. Next, configure the Agent on where to collect logs from. ## Custom log collection{% #custom-log-collection %} Datadog Agent v6 can collect logs and forward them to Datadog from files, the network (TCP or UDP), journald, and Windows channels: 1. In the `conf.d/` directory at the root of your [Agent's configuration directory](https://docs.datadoghq.com/agent/configuration/agent-configuration-files.md), create a new `.d/` folder that is accessible by the Datadog user. 1. Create a new `conf.yaml` file in this new folder. 1. Add a custom log collection configuration group with the parameters below. 1. [Restart your Agent](https://docs.datadoghq.com/agent/configuration/agent-commands.md#restart-the-agent) to take into account this new configuration. 1. Run the [Agent's status subcommand](https://docs.datadoghq.com/agent/configuration/agent-commands.md#agent-status-and-information) and look for `` under the Checks section. If there are permission errors, see [Permission issues tailing log files](https://docs.datadoghq.com/logs/guide/log-collection-troubleshooting-guide.md#permission-issues-tailing-log-files) to troubleshoot. Below are examples of custom log collection setup: {% tab title="Tail files" %} To gather logs from your `` application stored in `/.log` create a `.d/conf.yaml` file at the root of your [Agent's configuration directory](https://docs.datadoghq.com/agent/configuration/agent-configuration-files.md) with the following content: ```yaml logs: - type: file path: "/.log" service: "" source: "" ``` On **Windows**, use the path `:\\\\.log`, and verify that the user `ddagentuser` has read access to the log file. **Note**: A log line needs to be terminated with a newline character, `\n` or `\r\n`, otherwise the Agent waits indefinitely and does not send the log line. {% /tab %} {% tab title="TCP/UDP" %} To capture the sender IP address and include it in the log message payload, add the following configuration to your `datadog.yaml` file: ```yaml logs_config: use_sourcehost_tag: true ``` To gather logs from your `` application that forwards its logs to TCP port **10518**, create a `.d/conf.yaml` file at the root of your [Agent's configuration directory](https://docs.datadoghq.com/agent/configuration/agent-configuration-files.md) with the following content: ```yaml logs: - type: tcp port: 10518 service: "" source: "" ``` If you are using Serilog, `Serilog.Sinks.Network` is an option for connecting with UDP. In the Agent version 7.31.0+, the TCP connection stays open indefinitely even when idle. **Notes**: - The Agent supports raw string, JSON, and Syslog formatted logs. If you are sending logs in batch, use line break characters to separate your logs. - A log line needs to be terminated with a newline character, `\n` or `\r\n`, otherwise the Agent waits indefinitely and does not send the log line. {% /tab %} {% tab title="journald" %} To gather logs from journald, create a `journald.d/conf.yaml` file at the root of your [Agent's configuration directory](https://docs.datadoghq.com/agent/configuration/agent-configuration-files.md) with the following content: ```yaml logs: - type: journald path: /var/log/journal/ ``` Refer to the [journald integration](https://docs.datadoghq.com/integrations/journald.md) documentation for more details regarding the setup for containerized environments and units filtering. {% /tab %} {% tab title="Windows Events" %} To send Windows events as logs to Datadog, add the channels to `conf.d/win32_event_log.d/conf.yaml` manually or use the Datadog Agent Manager. To see your channel list, run the following command in a PowerShell: ```text Get-WinEvent -ListLog * ``` To see the most active channels, run the following command in a PowerShell: ```text Get-WinEvent -ListLog * | sort RecordCount -Descending ``` Then add the channels to your `win32_event_log.d/conf.yaml` configuration file: ```yaml logs: - type: windows_event channel_path: "" source: "" service: "" sourcecategory: windowsevent - type: windows_event channel_path: "" source: "" service: "" sourcecategory: windowsevent ``` Edit the `` parameters with the Windows channel name you want to collect events from. Set the corresponding `source` parameter to the same channel name to benefit from the [integration automatic processing pipeline setup](https://docs.datadoghq.com/logs/log_configuration/pipelines.md#integration-pipelines). Finally, [restart the Agent](https://docs.datadoghq.com/agent/basic_agent_usage/windows.md). {% /tab %} {% tab title="Windows Private Location" %} Follow the steps in these sections to send Windows Private Location logs to Datadog: ### Configure the Agent{% #configure-the-agent %} 1. Enable Agent log collection by setting `logs_enabled: true` in the Agent configuration file. 1. Navigate to `C:\ProgramData\Datadog\conf.d` and create a folder named `synthetics_worker.d`. 1. Inside the `synthetics_worker.d` folder, create a file named `conf.yaml` using the following example as a template: ```yaml logs: - type: file path: "C:\\Program Files\\Datadog-Synthetics\\Synthetics\\private-location-service.out.log" service: source: synthetics tags: # Defined per user preference - env: - private_location: ``` ### Verify the user running the Agent{% #verify-the-user-running-the-agent %} Since the Private Location installation folder is restricted to admin access, the Datadog Agent needs permission to access the log file. Follow these steps to verify the user running the Datadog Agent: 1. Press the Windows key and `R`, and search for Run. 1. Find the Datadog Agent, right-click it, and select Properties. 1. In the Log On tab, verify the account (the default is `ddagentuser`). 1. Close the window. ### Grant permission to the user running the Agent{% #grant-permission-to-the-user-running-the-agent %} 1. Go to `C:\Program Files` and find the `synthetics_worker.d` folder. 1. Right-click the `synthetics_worker.d` folder and select Properties. 1. Go to the Security tab. 1. Click Edit and add `ddagentuser`. 1. Grant the necessary permissions. 1. Restart the Datadog Agent through the Services screen or command line to apply the changes and begin sending logs to Datadog. {% /tab %} List of all available parameters for log collection: | Parameter | Required | Description | | ---------------- | -------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `type` | Yes | The type of log input source. Valid values are: `tcp`, `udp`, `file`, `windows_event`, `docker`, or `journald`. | | `port` | Yes | If `type` is **tcp** or **udp**, set the port for listening to logs. | | `path` | Yes | If `type` is **file** or **journald**, set the file path for gathering logs. | | `channel_path` | Yes | If `type` is **windows\_event**, list the Windows event channels for collecting logs. | | `service` | Yes | The name of the service owning the log. If you instrumented your service with [Datadog APM](https://docs.datadoghq.com/tracing.md), this must be the same service name. Check the [unified service tagging](https://docs.datadoghq.com/getting_started/tagging/unified_service_tagging.md) instructions when configuring `service` across multiple data types. | | `source` | Yes | The attribute that defines which integration is sending the logs. If the logs do not come from an existing integration, then this field may include a custom source name. However, it is recommended that you match this value to the namespace of any related [custom metrics](https://docs.datadoghq.com/metrics/custom_metrics.md#overview) you are collecting, for example: `myapp` from `myapp.request.count`. | | `include_units` | No | If `type` is **journald**, list of the specific journald units to include. | | `exclude_paths` | No | If `type` is **file**, and `path` contains a wildcard character, list the matching file or files to exclude from log collection. This is available for Agent version >= 6.18. | | `exclude_units` | No | If `type` is **journald**, list of the specific journald units to exclude. | | `sourcecategory` | No | The attribute used to define the category a source attribute belongs to, for example: `source:postgres, sourcecategory:database` or `source: apache, sourcecategory: http_web_access`. | | `start_position` | No | See Start position for more information. | | `encoding` | No | If `type` is **file**, set the encoding for the Agent to read the file. Set it to `utf-16-le` for UTF-16 little-endian, `utf-16-be` for UTF-16 big-endian, or `shift-jis` for Shift JIS. If set to any other value, the Agent reads the file as UTF-8. *Added `utf-16-le` and `utf-16be` in Agent v6.23/v7.23, `shift-jis` in Agent v6.34/v7.34* | | `tags` | No | A list of tags added to each log collected ([learn more about tagging](https://docs.datadoghq.com/getting_started/tagging.md)). | ### Start position{% #start-position %} The `start_position` parameter is supported by **file** and **journald** tailer types. The `start_position` is always `beginning` when tailing a container. Support: - **File**: Agent 6.19+/7.19+ - **Journald**: Agent 6.38+/7.38+ If `type` is **file**: - Set the position for the Agent to start reading the file. - Valid values are `beginning`, `end`, `forceBeginning`, and `forceEnd` (default: `end`). - The `beginning` position does not support paths with wildcards. If `type` is **journald**: - Set the position for the Agent to start reading the journal. - Valid values are `beginning`, `end`, `forceBeginning`, and `forceEnd` (default: `end`). #### Precedence{% #precedence %} For both file and journald tailer types, if an `end` or `beginning` position is specified, but an offset is stored, the offset takes precedence. Using `forceBeginning` or `forceEnd` forces the Agent to use the specified value even if there is a stored offset. ## Further Reading{% #further-reading %} - [Agent tags automatically added to logs](https://docs.datadoghq.com/agent/logs/agent_tags.md) - [Filter logs sent to Datadog](https://docs.datadoghq.com/agent/logs/advanced_log_collection.md#filter-logs) - [Scrub sensitive data from your logs](https://docs.datadoghq.com/agent/logs/advanced_log_collection.md#scrub-sensitive-data-from-your-logs) - [Multi-line log aggregation](https://docs.datadoghq.com/agent/logs/advanced_log_collection.md#multi-line-aggregation) - [Tail directories by using wildcards](https://docs.datadoghq.com/agent/logs/advanced_log_collection.md#tail-directories-using-wildcards) - [Global processing rules](https://docs.datadoghq.com/agent/logs/advanced_log_collection.md#global-processing-rules)