Join us at the Dash conference! July 16-17, NYC


original log

A Processor executes within a pipeline a data-structuring action (Remapping an attribute, Grok parsing…) on a log.

The different kinds of Processors are explained below.

Grok Parser

Create custom grok rules to parse the full message or a specific attribute of your raw event:


Read more about this in the parsing section

Log Date Remapper

As Datadog receives logs, it timestamps them using the value(s) from any of these default attributes:

  • timestamp
  • date
  • _timestamp
  • Timestamp
  • eventTime
  • published_date

If your logs put their dates in an attribute not in this list, use the log date Remapper Processor to define their date attribute as the official log timestamp:

Log date Remapper

If your logs don’t contain any of the default attributes and you haven’t defined your own date attribute, Datadog timestamps the logs with the date it received them.

The recognized date formats are: ISO8601, UNIX (the milliseconds EPOCH format), and RFC3164.

Log Status Remapper

Use this Processor if you want to assign some attributes as the official status. Just enter the attribute path in the Processor tile as follows:

Severity Remapper Processor tile

It transforms this log:

 Log pre severity

Into this log:

 Log post severity bis

However, be aware that each incoming status value is mapped as follows:

  • Integers from 0 to 7 map to the Syslog severity standards
  • Strings beginning with emerg or f (case-insensitive) map to emerg (0)
  • Strings beginning with a (case-insensitive) map to alert (1)
  • Strings beginning with c (case-insensitive) map to critical (2)
  • Strings beginning with err (case-insensitive) map to error (3)
  • Strings beginning with w (case-insensitive) map to warning (4)
  • Strings beginning with n (case-insensitive) map to notice (5)
  • Strings beginning with i (case-insensitive) map to info (6)
  • Strings beginning with d, trace or verbose (case-insensitive) map to debug (7)
  • Strings matching OK or Success (case-insensitive) map to OK
  • All others map to info (6)

Service Remapper

Use this Processor if you want to assign one or more attributes as the official service. Define the attribute(s) in the Processor tile as follows:

Service Remapper Processor tile

Log Message Remapper

The message is a key attribute in Datadog. It is displayed in the message column of the Log Explorer and you can do full string search on it. Use this Processor to define one or more attributes as the official log message. Define the attribute(s) in the Processor tile as follows:

Message Processor


This Processor remaps any source attribute(s) or tag to another target attribute or tag. For instance, here, it remaps user to user.firstname

Attribute Remapper Processor tile

It transforms this log:

attribute pre remapping

Into this log:

attribute post remapping

Constraints on the tag/attribute name are explained in the Tag Best Practice documentation. Some additional constraints are applied as : or , are not allowed in the target tag/attribute name.

URL Parser

This Processor extracts query parameters and other important parameters from a URL. To use it, just enter the source attribute of your url:

These settings:

Url Processor Tile

Give the following results:

Url Processor

User-Agent parser

The User-Agent parser takes a User-Agent attribute and does its best to extract the OS, browser, device, etc… It recognizes major bots like the Google Bot, Yahoo Slurp, Bing, and others.

If your logs contain encoded User-Agents (as, for example, IIS logs do), configure this Processor to decode the URL before parsing it.

These settings:

Useragent Processor tile

Give the following results:

Useragent Processor

Category Processor

Use the Category Processor to add a new attribute (without spaces or special characters in the new attribute name) to a log matching a provided search query. Categories are very useful to create meaningful groups which can be used in any analytical view (e.g. URL groups, Machine groups, environments, response time buckets, etc….).

For example to categorize your web access logs depending of the status code range value (2xx for a response code between 200 and 299, 3xx for a response code between 300 and 399, …) add this Processor:

Category Processor

It produces the following result:

Category Processor result

Important Note: The query can be done on any log attribute or tag, whether it is a facet or not. Wildcards can also be used inside your query. Once the log has matched one of the Processor queries, it stops. Make sure they are properly ordered in case a log could match several queries.

Arithmetic processor

Use the Arithmetic Processor to add a new attribute (without spaces or special characters in the new attribute name) to a log with the result of the provided formula. This enables you to remap different time attributes with different units into a single attribute, or to compute operations on attributes within the same log.

The formula can use parentheses and the basic arithmetic operators: -, +, *, /.


Arithmetic Processor

By default, the calculation is skipped if an attribute is missing. Select “Replace missing attribute by 0” to automatically populate missing attribute values with 0 to ensure that the calculation is done. An attribute is missing if it is not found in the log attributes, or if it cannot be converted to a number.


  • The operator - needs to be space split in the formula as it can also be contained in attribute names.
  • If the target attribute already exists, it is overwritten by the result of the formula.
  • Results are rounded up to the 9th decimal. For example, if the result of the formula is 0.1234567891, the actual value stored for the attribute is 0.123456789.

Trace Remapper

There are two ways to improve correlation between application traces and logs:

  1. Follow the documentation on how to inject a trace id in the application logs and by default log integrations take care of all the rest of the setup.

  2. Use the Trace Remapper processor to define a log attribute as its associated trace id by entering the attribute path in the Processor tile as follows:

Trace Id Processor

Further Reading