Group Logs


Logs can be valuable as individual events, but sometimes valuable information lives in a subset of events. In order to expose this information, group your logs by fields, identify patterns, or aggregate your logs into transactions.

Switch between different aggregations of your queried logs with the logs query editor. The fields you select to group, aggregate, and measure your logs are saved as you switch between different visualizations and aggregation types.

A bar graph displaying logs and the option to group into fields, patterns, and transactions

Add multiple queries to simultaneously analyze different sets of logs, and apply formulas and functions to your queries for in-depth analysis.

Note: Aggregations are supported for indexed logs only. If you need to perform aggregation on non-indexed logs, consider temporarily disabling exclusion filters, using logs to metrics and/or running a rehydration on your archives.


When aggregating by Fields, all logs matching your query filter are aggregated into groups based on the value of one or multiple log facets. On top of these aggregates, you can extract the following measures:

  • count of logs per group
  • unique count of coded values for a facet per group
  • statistical operations (min, max, avg, and percentiles) on numerical values of a facet per group

Note: Individual logs with multiple values for a single facet belong to that many aggregates. For instance, a log having the team:sre and the team:marketplace tags are counted once in the team:sre aggregate and once in the team:marketplace aggregate.

The Fields aggregation supports one dimension for the Top list visualization, and up to four dimensions for the Timeseries and Table visualizations. When there are multiple dimensions, the top values are determined according to the first dimension, then according to the second dimension within the top values of the first dimension, then according to the third dimension within the top values of the second dimension.

Multiple queries

Multiple queries are supported in Timeseries and Top list visualizations. Add multiple queries by clicking on the + Add button next to the query editor. When you add a new query, it is a copy of the last query and its grouping options:

Select or deselect queries to display in the current visualization by clicking on their letters in the query editor:

The query editor with two queries, one is labeled A and the other is labeled B

By default, when a new query is added, it is automatically selected to be displayed in the chosen visualization.

Display the timeline for one of your queries by selecting that query in the Timeline for dropdown. Scope one of your search queries by selecting that query in the Use facets with dropdown and clicking on values in the Facet Panel. Only the selected query is updated with the chosen facets.

The query editor showing the timeline for selector with dropdown options for query A and query B


Note: Functions are only supported in Timeseries and Top list visualizations.

Apply functions to your logs by clicking on the Fields aggregation in the query editor. Optionally select a faceted field to apply the function to, then click on the Σ icon next to that measure. Select or search for a function to apply to the selected log field.

All functions available for logs in the graphing editor in Dashboards can be applied to logs in the Log Explorer:

Here is an example of how to apply an Exclusion function to exclude certain values of your logs:

A query with the cutoff min exclusion filter set to 100


Apply a formula on one or multiple queries by clicking on the + Add button next to the query editor. In the following example, the formula is used to calculate the ratio of the unique number of Cart Id in logs for Merchant Tier: Enterprise / Merchant Tier: Premium customers:

The query editor with a formula dividing query A by query B

Note: To apply formulas with multiple queries, all queries must be grouped by the same facet. In the example above, both queries are grouped by Webstore Store Name.

You can apply a function to a formula by clicking on the Σ icon. Here is an example of how to apply a Timeshift function on the proportion of error logs in all logs to compare current data with data from one week before:

The query editor showing a formula with the week before timeshift function applied to it


With pattern aggregation, logs that have a message with similar structures are grouped altogether. Optionally select one to three faceted fields to pre-aggregate your logs into groups before patterns are detected within these groupings.

The patterns view is helpful for detecting and filtering noisy error patterns that could cause you to miss other issues:

The logs explorer showing logs grouped by patterns

Note: The pattern detection is based on 10,000 log samples. Refine the search to see patterns limited to a specific subset of logs.

Patterns support the List Aggregates visualization. Clicking a pattern in the list opens the pattern side panel from which you can:

  • Access a sample of logs from that pattern
  • Append the search filter to scope it down to logs from this pattern only
  • Get a kickstart for a grok parsing rule to extract structured information logs of that pattern
The log side panel with the view all button and the parsing rule highlighted

Pattern Inspector

Use Pattern Inspector to get a visual breakdown of the underlying values of a log pattern’s aggregation based on your search query. For example, if you are investigating an issue, you could see how many hosts are involved or what regions or data centers are impacted.

The distribution of values graph showing a bar graph of the values

To use Pattern Inspector:

  1. Go to Log Explorer.
  2. Click Patterns in the Group into section. In the list of patterns, the aggregate values in the message section are highlighted in yellow. Hover over an aggregate value to get a preview of the visual distribution of its values.
  3. Click on an aggregate value to open the log pattern’s side panel and see more details in the Pattern Inspector tab.
The pattern panel showing the Pattern Inspector tab


Transactions aggregate indexed logs according to instances of a sequence of events, such as a user session or a request processed across multiple micro-services. For example, an e-commerce website groups logs across various user actions, such as catalog search, add to cart, and checkout, to build a transaction view using a common attribute such as requestId or orderId.

The logs explorer showing logs grouped by transactions

Note: The transaction aggregation differs from the natural group aggregation, in the sense that resulting aggregates not only include logs matching the query, but also all logs belonging to the related transactions.

  • Duration: The difference of timestamps for the last and first log in the transaction. This measure is automatically added.
  • Maximum Severity found in logs in the transaction. This measure is automatically added.
  • Finding key items: For any facet with string values, calculate specific log information using the operations count unique, latest, earliest and most frequent.
  • Getting Statistics: For any measure, calculate statistical information using the operations min, max, avg, sum, median, pc75, pc90, pc95, and pc99.
  • Set Start and End Conditions: Customize transaction boundaries by specifying the start and end of the transaction using distinct queries.

Transactions support the List Aggregates visualization. Clicking a transaction in the list opens the transaction side panel from which you can:

  • Access all logs within that transaction
  • Search specific logs within that transaction
The transaction log panel showing logs within the selected transaction

When a start or end condition is used to define a transaction, click on a transaction group in the list to open the transaction group side panel, from which you can:

  • Access the transactions within that transaction group in sequence
  • Access all logs within each transaction
  • View statistics for each transaction and summary statistics for the entire transaction group
The transaction group panel showing transactions within the selected group in sequence

Further reading