Logging is here!

Java log collection

Java logs are quite complex to handle, mainly because of stack traces. These stack traces are split into multiple lines which makes them difficult to associate to the original log event:

//4 events generated when only one is expected!
Exception in thread "main" java.lang.NullPointerException
        at com.example.myproject.Book.getTitle(Book.java:16)
        at com.example.myproject.Author.getBookTitles(Author.java:25)
        at com.example.myproject.Bootstrap.main(Bootstrap.java:14)

By asking your logging library to log into JSON, you will:

  • Ensure to have a stack_trace properly wrapped into the proper LogEvent
  • Ensure that all the attributes of a log event are properly extracted (severity, logger name, thread name, etc…)
  • You’ll have access to MDC, which are attributes you can attach to any log events

To send your logs to Datadog, we recommend logging to a file and then tailing that file with your Datadog agent.

We also strongly encourage you to setup your logging libraries to produce your logs in JSON format to avoid sustaning custom parsing rules.

Here are setup examples for the log4j, slf4j and log4j2 logging libraries:

Configure your logger


Add a new file appender to log4j.xml:

<appender name="fileAppender" class="org.apache.log4j.FileAppender">
  <param name="File" value="/logs/log4j.log" />
  <param name="Append" value="true" />
  <layout class="org.apache.log4j.PatternLayout">
      <param name="ConversionPattern" value="%d{yyyy-MM-dd HH:mm:ss} %-5p %c{1}:%L - %m%n" />


Edit your log4j2.xml file:

 <File name="MyFile" fileName="logs/app.log" immediateFlush="true">
        <PatternLayout pattern="%d{yyy-MM-dd HH:mm:ss.SSS} [%t] %-5level %logger{36} - %msg%n"/>
        <Root level="debug">
        <AppenderRef ref="MyFile" />


Edit your logback.xml file:

   <timestamp key="byDay" datePattern="yyyyMMdd'T'HHmmss"/>

   <appender name="FILE" class="ch.qos.logback.core.FileAppender">
      <file> ~/logs/log-${byDay}.log </file>
          <Pattern>%d{yyyy-MM-dd HH:mm:ss.SSS} [%thread] %-5level %logger{36} - %msg%n</Pattern>
    <root level="debug">
        <appender-ref ref="FILE" />

Configure the Datadog agent

Create a file java.yaml in the Agent’s conf.d/ directory with the following content:

#Log section

    ## - type : file (mandatory) type of log input source (tcp / udp / file)
    ##   port / path : (mandatory) Set port if type is tcp or udp. Set path if type is file
    ##   service : (mandatory) name of the service owning the log
    ##   source : (mandatory) attribute that defines which integration is sending the logs
    ##   sourcecategory : (optional) Multiple value attribute. Can be used to refine the source attribtue
    ##   tags: (optional) add tags to each logs collected

  - type: file
    path: /path/to/your/java/log.log
    service: java
    source: java
    sourcecategory: sourcecode
    # For multiline logs, if they start by the date with the format yyyy-mm-dd uncomment the following processing rule
    #  - type: multi_line
    #    name: new_log_start_with_date
    #    pattern: \d{4}\-(0?[1-9]|1[012])\-(0?[1-9]|[12][0-9]|3[01])

Getting further

Enrich your log events with valuable attributes!

Logging is great- It tells developers and administrators what is happening at specific moments in time. However, always remember to decorate them with contextual attributes.

Using the Key/Value parser

The Key/Value parser extracts any <key>=<value> pattern recognized in any log event.

To enrich your log events in Java, you can re-write messages in your code and introduce <key>=<value> sequences.

For instance if you have:

logger.info("Emitted 1001 messages during the last 93 seconds for customer scope prod30");

You can easily change it to:

logger.info("Emitted quantity=1001 messages during the last durationInMs=93180 ms for customer scope=prod30");

With the Key/Value parser enabled, Datadog automatically extracts each pair from your final JSON document:

    "message" : "Emitted quantity=1001 messages during the last durationInMs=93180 ms for customer scope=prod30",
    "scope" : "prod30",
    "durationInMs" : 93180,
    "quantity" : 1001

So you can exploit scope as a field, and durationInMs & quantity as metrics.

MDC (Mapped Diagnostic Context)

Another option to enrich your logs is to use Java’s MDC (Mapped Diagnostic Contexts).

If you use the logback technologie. It would give us the following Java code:

MDC.put("scope", "prod30");
logger.info("Emitted 1001 messages during the last 93 seconds");

MDC are great but for some reason only string types are allowed. Therefore, providing numerical values for metrics with MDCs would be a bad idea

Further Reading