Log collection

Available for Agent versions 6.0 or later

Log collection is disabled by default in the Datadog Agent. To collect Windows Event Logs as Datadog logs, activate log collection by setting logs_enabled: true in your datadog.yaml file.

To collect Windows Event Logs as Datadog logs, configure channels under the logs: section of your win32_event_log.d/conf.yaml configuration file. This example shows entries for the Security and <CHANNEL_2> channels:

logs:
  - type: windows_event
    channel_path: Security
    source: windows.events
    service: Windows

  - type: windows_event
    channel_path: "<CHANNEL_2>"
    source: windows.events
    service: myservice

Set the corresponding source parameter to windows.events to benefit from the integration automatic processing pipeline.

Event collection using the Event Log API (Recommended)

The Datadog Agent can be configured to collect Windows Event Logs as Datadog events using the Event Log API. Datadog recommends using the Event Log API because it has better performance than the legacy method below. Note, each method has its own configuration syntax for channels and for filters. For more information, see Filtering Events.

To collect Windows Event Logs as Datadog events, configure channels under the instances: section of your win32_event_log.d/conf.yaml configuration file.

Set legacy_mode: false in each instance. If legacy_mode: false is set, the path is required to be set in the \win32_event_log.d\conf.yaml file.

This example shows entries for the Security and <CHANNEL_2> channels:

init_config:
instances:
  - # Event Log API 
    path: Security
    legacy_mode: false
    filters: {}

  - path: "<CHANNEL_2>" 
    legacy_mode: false
    filters: {}

Agent versions 7.49 and later support setting legacy_mode in the shared init_config section. This sets the default for all instances and no longer requires you to set legacy_mode individually for each instance. However, the option can still be set on a per-instance basis.

init_config:
    legacy_mode: false
instances:
  - # Event Log API
    path: Security
    filters: {}

  - path: "<CHANNEL_2>"
    filters: {}

Event collection using Legacy Mode (Deprecated)

The legacy method uses WMI (Windows Management Instrumentation) and was deprecated in Agent version 7.20.

To collect Windows Event Logs as Datadog events, configure channels under the instances: section of your win32_event_log.d/conf.yaml configuration file.

To use Legacy Mode, set legacy_mode to true. Then, set at least one of the following filters: source_name, event_id, message_filters, log_file, or type.

This example shows entries for the Security and <CHANNEL_2> channels:

init_config:
instances:
  - # WMI (default)
    legacy_mode: true
    log_file:
      - Security
      
  - legacy_mode: true
    log_file:
      - "<CHANNEL_2>"

For more information, see Add event log files to the Win32_NTLogEvent WMI class.

You can use the query, as well as the log_processing_rules regex option, to filter event logs. Datadog recommends using the query option which is faster at high rates of Windows Event Log generation and requires less CPU and memory than the log_processing_rules filters. When using the log_processing_rules filters, the Agent is forced to process and format each event, even if it will be excluded by log_processing_rules regex. With the query option, these events are not reported to the Agent.

You can use the query option to filter events with an XPATH or structured XML query. The query option can reduce the number of events that are processed by log_processing_rules and improve performance. There is an expression limit on the syntax of XPath and XML queries. For additional filtering, use log_processing_rules filters.

Datadog recommends creating and testing the query in Event Viewer’s filter editor until the events shown in Event Viewer match what you want the Agent to collect.

Then, copy and paste the query into the Agent configuration.

  # collect Critical, Warning, and Error events
  - type: windows_event
    channel_path: Application
    source: windows.events
    service: Windows       
    query: '*[System[(Level=1 or Level=2 or Level=3)]]'
      
  - type: windows_event
    channel_path: Application
    source: windows.events
    service: Windows       
    query: |
      <QueryList>
        <Query Id="0" Path="Application">
          <Select Path="Application">*[System[(Level=1 or Level=2 or Level=3)]]</Select>
        </Query>
      </QueryList>      

In addition to the query option, events can be further filtered with log processing rules.

Some example filters include the following:

  - type: windows_event
    channel_path: Security
    source: windows.events
    service: Windows       
    log_processing_rules:
    - type: include_at_match
      name: relevant_security_events
      pattern: '"EventID":(?:{"value":)?"(1102|4624|4625|4634|4648|4728|4732|4735|4737|4740|4755|4756)"'

  - type: windows_event
    channel_path: Security
    source: windows.events
    service: Windows       
    log_processing_rules:
    - type: exclude_at_match
      name: relevant_security_events
      pattern: '"EventID":(?:{"value":)?"(1102|4624)"'

  - type: windows_event
    channel_path: System
    source: windows.events
    service: Windows       
    log_processing_rules:
    - type: include_at_match
      name: system_errors_and_warnings
      pattern: '"level":"((?i)warning|error)"'

  - type: windows_event
    channel_path: Application
    source: windows.events
    service: Windows       
    log_processing_rules:
    - type: include_at_match
      name: application_errors_and_warnings
      pattern: '"level":"((?i)warning|error)"'

Here is an example regex pattern to only collect Windows Events Logs from a certain EventID:

logs:
  - type: windows_event
    channel_path: Security
    source: windows.event
    service: Windows
    log_processing_rules:
      - type: include_at_match
        name: include_x01
        pattern: '"EventID":(?:{"value":)?"(101|201|301)"'

Note: The pattern may vary based on the format of the logs. The Agent stream-logs subcommand can be used to view this format.

For more examples of filtering logs, see the Advanced Log Collection documentation.

Legacy events

Applies to Agent versions < 7.41

Legacy Provider EventIDs have a Qualifiers attribute that changes the format of the log, as seen in the Windows Event Schema. These events have the following XML format, visible in Windows Event Viewer:

<EventID Qualifiers="16384">3</EventID>

The following regex must be used to match these EventIDs:

logs:
  - type: windows_event
    channel_path: Security
    source: windows.event
    service: Windows
    log_processing_rules:
      - type: include_at_match
        name: include_legacy_x01
        pattern: '"EventID":(?:{"value":)?"(101|201|301)"'

Agent versions 7.41 or later normalize the EventID field. This removes the need for the substring, (?:{"value":)?, from legacy pattern as it is no longer applicable. A shorter regex pattern can be used from versions 7.41 or later as seen below:

logs:
  - type: windows_event
    channel_path: Security
    source: windows.event
    service: Windows
    log_processing_rules:
      - type: include_at_match
        name: include_x01
        pattern: '"EventID":"(101|201|301)"'

Use the Windows Event Viewer GUI to list all the event logs available for capture with this integration.

To determine the exact values, set your filters to use the following PowerShell command:

Get-WmiObject -Class Win32_NTLogEvent

For example, to see the latest event logged in the Security log file, use the following:

Get-WmiObject -Class Win32_NTLogEvent -Filter "LogFile='Security'" | select -First 1

The values listed in the output of the command can be set in win32_event_log.d/conf.yaml to capture the same kind of events.

Filtering events using the Event Log API (Recommended)

The configuration option using the Event Log API includes the following filters:

• path: Application, System, Setup, Security
• type: Critical, Error, Warning, Information, Success Audit, Failure Audit
• source: Any available source name
• id: event_id: Windows EventLog ID

See the sample win32_event_log.d/conf.yaml for all available filter options.

This example filter uses Event Log API method.

instances:
  - legacy_mode: false
    path: System
    filters:
      source:
      - Microsoft-Windows-Ntfs
      - Service Control Manager
      type:
      - Error
      - Warning
      - Information
      - Success Audit
      - Failure Audit
      id:
      - 7036

You can use the query option to filter events with an XPATH or structured XML query. Datadog recommends creating the query in Event Viewer’s filter editor until the events shown in Event Viewer match what you want the Datadog Agent to collect. The filters option is ignored when the query option is used.

init_config:
instances:
  # collect Critical, Warning, and Error events
  - path: Application
    legacy_mode: false
    query: '*[System[(Level=1 or Level=2 or Level=3)]]'
    
  - path: Application
    legacy_mode: false
    query: |
      <QueryList>
        <Query Id="0" Path="Application">
          <Select Path="Application">*[System[(Level=1 or Level=2 or Level=3)]]</Select>
        </Query>
      </QueryList>      

Filtering events using Legacy Mode (Deprecated)

The configuration option using the Legacy Mode includes the following filters:

• log_file: Application, System, Setup, Security
• type: Critical, Error, Warning, Information, Audit Success, Audit Failure
• source_name: Any available source name
• event_id: Windows EventLog ID

This example filter uses the Legacy Mode method.

instances:
  # Legacy
  # The following captures errors and warnings from SQL Server which
  # puts all events under the MSSQLSERVER source and tag them with #sqlserver.
  - tags:
      - sqlserver
    type:
      - Warning
      - Error
    log_file:
      - Application
    source_name:
      - MSSQLSERVER

  # This instance captures all system errors and tags them with #system.
  - tags:
      - system
    type:
      - Error
    log_file:
      - System

The legacy method does not support the query option. Only the Event Log API method (setting legacy_mode: false) and the Logs Tailer supports the query option.

Check the information page in the Datadog Agent Manager or run the Agent’s status subcommand and look for win32_event_log under the Logs Agent section.

It should display a section similar to the following:

Logs Agent
==========

  [...]

  win32_event_log
  ---------------
    - Type: windows_event
      ChannelPath: System
      Status: OK

Check the information page in the Datadog Agent Manager or run the Agent’s status subcommand and look for win32_event_log under the Checks section.

It should display a section similar to the following:

Checks
======

  [...]

  win32_event_log
  ---------------
      - instance #0 [OK]
      - Collected 0 metrics, 2 events & 1 service check