Log Explorer

Log Explorer

The Log Explorer is your home base for log troubleshooting and exploration. Whether you start from scratch, from a Saved View, or land here from any other context like monitor notifications or dashboard widgets, the Log Explorer is designed to iteratively:

  1. Filter logs; to narrow down, broaden, or shift your focus on the subset of logs of current interest.
  2. Aggregate queried logs into higher-level entities in order to derive or consolidate information.
  3. Visualize the outcome of filters and aggregations to put your logs into the right perspective and bubble up decisive information.

At any moment, Export your Log Explorer view to reuse it later or in different contexts.

Filters logs

The search filter consists of a timerange and a search query mixing key:value and full-text search. Refer to our log search syntax and timerange documentation for details on advanced use cases. For example, the search query service:payment status:error rejected over a Past 5 minutes timerange:

Indexed Logs support both full-text search and key:value search queries.

Note: key:value queries require that you declare a facet beforehand.

Note: key:value queries do not require that you declare a facet beforehand.

Aggregate and measure

Logs can be valuable as individual events, but sometimes valuable information lives in a subset of events. In order to expose this information, aggregate your logs.

Note: Aggregations are supported for indexed logs only. If you need to perform aggregation on non-indexed logs, consider temporary disabling exclusion filters, using logs to metrics and/or running a rehydration on your archives.

Fields

With fields aggregation, all logs matching the query filter are aggregated into groups based on the value of one or multiple log facets. On top of these aggregates, you can extract the following measures:

  • count of logs per group
  • unique count of coded values for a facet per group
  • statistical operations (min, max, avg, and percentiles) on numerical values of a facet per group

Note: Individual logs having multiple values for a single facet belong to that many aggregates. For instance, a log having the team:sre and the team:marketplace tags are counted once in the team:sre aggregate and once in the team:marketplace aggregate.

Fields aggregation supports one dimension for the Toplist visualization, and up to three dimensions for the Timeseries and Table visualizations. When there are multiple dimensions, the top values are determined according to the first dimension, then according to the second dimension within the top values of the first dimension, then according to the third dimension within the top values of the second dimension.

Patterns

With pattern aggregation, logs that have a message with similar structures, belong to the same service and have the same status are grouped altogether. The patterns view is helpful for detecting and filtering noisy error patterns that could cause you to miss other issues:

Note: The pattern detection is based on 10,000 log samples. Refine the search to see patterns limited to a specific subset of logs.

Patterns support the List Aggregates visualization. Clicking a pattern in the list opens the pattern side panel from which you can:

  • Access a sample of logs from that pattern
  • Append the search filter to scope it down to logs from this pattern only
  • Get a kickstart for a grok parsing rule to extract structured information logs of that pattern

Transactions

Transactions aggregate indexed logs according to instances of a sequence of events, such as a user session or a request processed across multiple micro-services. For example, an e-commerce website groups log events across various user actions, such as catalog search, add to cart, and checkout, to build a transaction view using a common attribute such as requestId or orderId.

Note: The transaction aggregation differs from the natural group aggregation, in the sense that resulting aggregates not only include logs matching the query, but also all logs belonging to the related transactions.

  • Duration: The difference of timestamps for the last and first log in the transaction. This measure is automatically added.
  • Maximum Severity found in logs in the transaction. This measure is automatically added.
  • Finding key items: For any facet with string values, calculate specific log event information using the operations count unique, latest, earliest and most frequent.
  • Getting Statistics: For any measure, calculate statistical information using the operations min, max, avg, sum, median, pc75, pc90, pc95, and pc99.

Transactions support the List Aggregates visualization. Clicking a pattern in the list opens the pattern side panel from which you can:

  • Access all logs within that transaction
  • Search specific logs within that transaction

Visualize

Visualizations define how the outcome of filter and aggregates are displayed.

Lists

Lists are paginated results of logs or aggregates. They are valuable when individual results matter, but you have no prior or clear knowledge on what defines a matching result. Lists allow you examine a group of results.

Lists displaying individual logs and lists displaying aggregates of logs have slightly different capabilities.

List of logs

For a list of individual logs, choose which information of interest to display as columns. Manage the columns of the table using either:

  • The table, with interactions available in the first row. This is the preferred method to sort, rearrange, or remove columns.
  • The facet panel on the left, or the log side panel on the right. This is the preferred option to add a column for a field.

With the Options button, control the number of lines displayed in the table per log event.

configure display table

The default sort for logs in the list visualization is by timestamp, with the most recent logs on top. This is the fastest and therefore recommended sorting method for general purposes. Surface logs with lowest or highest value for a measure first, or sort your logs lexicographically for the unique value of facet, ordering a column according to that facet. Note that sorting your table according to a specific field requires that you [declare a facet][5] beforehand.

The default sort for logs in the list visualization is by timestamp, with the most recent logs on top. This is the fastest and therefore recommended sorting method for general purposes. Surface logs with lowest or highest value for a measure first, or sort your logs lexicographically for the unique value of facet, ordering a column according to that facet. Note that, although any attributes or tags can be added as a column, sorting your table according to a specific field requires that you [declare a facet][5] beforehand.

The configuration of the log table is stored alongside other elements of your troubleshooting context in Saved Views

List aggregates of logs

The columns displayed in list of aggregates are columns derived from the aggregation.

Results are sorted according to:

  • Number of matching events per aggregate for pattern aggregation (default to descending: more to less)
  • Lexicographic order of the transaction id for transaction aggregation (default to ascending: A to Z)

Timeseries

Visualize the evolution of a single measure (or a facet unique count of values) over a selected time frame, and (optionally) split by up to three available facets.

The following Timeseries log analytics shows the evolution of the top 50 URL Paths according to the 95th percentile of duration over the last 15 minutes.

Choose additional display options for timeseries: the roll-up interval, whether you display results as bars (recommended for counts and unique counts), lines (recommended for statistical aggregations) or areas, and the colorset.

Toplist

Visualize the top values from a facet according to the chosen measure.

For example, the following Toplist shows the top 15 Customers on a merchant website according to the number of unique sessions they had over the last day.

Nested tables

Visualize the top values from up to three facets according to a chosen measure (the first measure you choose in the list), and display the value of additional measures for elements appearing in this table. Update a search query or drill through logs corresponding to either dimension.

  • When there are multiple measures, the top or bottom list is determined according to the first measure.
  • The subtotal may differ from the actual sum of values in a group, since only a subset (top or bottom) is displayed. Events with a null or empty value for this dimension are not displayed as a sub-group.

Note: A table visualization used for one single measure and one single dimension is the same as a Toplist, just with a different display.

The following table log analytics show the evolution of the Top 10 Availability zones, and for each Availability Zone the Top 10 Versions according to their number or error logs, along with the number of unique count of Hosts and Container ID for each.

Export

At any moment, and depending on your current aggregation, export your exploration as a:

  • Saved View to use as an investigation starting point for future-yourself or your teammates
  • Dashboard widget for reporting or consolidation purpose
  • Monitor to trigger alerts on predefined thresholds
  • Metric to aggregate your logs into long term KPIs, as they are ingested in Datadog
  • CSV (for individual logs and transactions). You can export up to 5,000 logs at once for individual logs, 500 for Transactions.
  • Share View: Share a link to the current view with your teammates through email, Slack, and more. See all of the Datadog notification integrations available for this feature.

Further Reading