Log monitor

Overview

Log monitors alert when a specified type of log exceeds a user-defined threshold over a given period of time.

Setup

Construct a query to control what is monitored:

1. Define the search query:

   The search query has the same behavior as the log explorer search

2. Choose a Measure or Facet to monitor. Measure lets you choose the aggregation function whereas Facet displays a count.

3. Select the aggregation function for the Measure you want to monitor:

4. (Optional) Define the alert grouping:

   With or without alert grouping defined, you get one alert when the aggregated value meets the conditions set below. Even if you split the query by host, a single notification is sent if several hosts meet the conditions set below. This is done to reduce notification noise.

5. Set alert conditions. The following options can be used:

1. Configure your notification options:
   

Refer to the notifications dedicated documentation page for detailed options.

Notifications and log samples

It is possible to add up to 10 samples of logs that triggered the monitor in the notification message. This is available for Slack, Jira, Webhook, Microsoft Teams, and email notifications.

• Samples are not displayed for recovery notifications.

Enabling log samples in notifications:

Example for a Slack notification

Notifications for groups

Notifications from monitors split by group may include the list of the top 10 of breaching values instead of 10 log samples.

Enabling top 10 breaching values in notifications

Example for a Slack notification

No Data alerts and Below Conditions

To be notified if a specific set of logs are not received anymore, set the condition below 1. This notifies when no logs match the monitor query on the given timeframe.

However, note that when splitting the monitor by any dimension (tag or facet) and using a below condition, the alert is triggered if and only if there are logs for a given group, and the count is below the threshold—or if there are no logs for all of the groups.

Examples:

1. The following monitor triggers if and only if there are no logs for all of the services:
   

2. The following monitor triggers if there are no logs for the service backend:
   

Further Reading