Log Detection Rules

Search query

Construct a search query using the same logic as a log explorer search.

Optionally, define a unique count and signal grouping. Count the number of unique values observed for an attribute in a given timeframe. The defined group-by generates a signal for each group by value. Typically, the group by is an entity (like user, or IP). The group-by is also used to join the queries together.

Click Add Query to add additional queries.

Note: The query applies to all ingested logs.

Exclude benign activity with suppression queries

In the Only generate a signal if there is a match field, you have the option to enter a query so that a trigger is only generated when a value is met.

In the This rule will not generate a signal if there is a match field, you have the option to enter suppression queries so that a trigger is not generated when the values are met. For example, if a user called john.doe is triggering a signal, but their actions are benign and you no longer want signals triggered from this user, input a logs query that excludes @user.username: john.doe.

Joining queries

Joining together logs that span a timeframe can increase the confidence or severity of the Security Signal. For example, to detect a successful brute force attack, both successful and unsuccessful authentication logs must be correlated for a user.

The Detection Rules join the logs together using a group by value. The group by values are typically entities (for example, IP address or user), but can be any attribute.

The Detection Rule cases join these queries together based on their group by value. The group by attribute is typically the same attribute because the value must be the same for the case to be met. If a group by value doesn’t exist, the case will never be met. A Security Signal is generated for each unique group by value when a case is matched.

In this example, when greater than five failed logins and a successful login exist for the same @usr.name, the first case is matched, and a Security Signal is generated.

Search query

Construct a search query using the same logic as a log explorer search. Each query has a label, which is a lowercase ASCII letter. The query name can be changed from an ASCII letter by clicking the pencil icon.

Note: The query applies to all ingested logs.

Learned value

Select the value or values to detect, the learning duration, and, optionally, define a signal grouping. The defined group-by generates a signal for each group-by value. Typically, the group-by is an entity (like user or IP).

For example, create a query for successful user authentication and set Detect new value to country and group by to user. Set a learning duration of 7 days. Once configured, logs coming in over the next 7 days are evaluated with the set values. If a log comes in with a new value after the learning duration, a signal is generated, and the new value is learned to prevent future signals with this value.

You can also identify users and entities using multiple values in a single query. For example, if you want to detect when a user signs in from a new device and from a country that they’ve never signed in from before, add device_id and country_name to Detect new value.

Exclude benign activity with suppression queries

In the Only generate a signal if there is a match field, you have the option to enter a query so that a trigger is only generated when a value is met.

In the This rule will not generate a signal if there is a match field, you have the option to enter suppression queries so that a trigger is not generated when the values are met. For example, if a user called john.doe is triggering a signal, but their actions are benign and you no longer want signals triggered from this user, input a logs query that excludes @user.username: john.doe.

Search query

Construct a search query using the same logic as a log explorer search.

Optionally, define a unique count and signal grouping. Count the number of unique values observed for an attribute in a given timeframe. The defined group-by generates a signal for each group by value. Typically, the group by is an entity (like user, or IP).

Anomaly detection inspects how the group by attribute has behaved in the past. If a group by attribute is seen for the first time (for example, the first time an IP is communicating with your system) and is anomalous, it will not generate a security signal because the anomaly detection algorithm has no historical data to base its decision on.

Note: The query applies to all ingested logs.

Search query

Construct a search query using the same logic as a log explorer search. All logs matching this query are analyzed for a potential impossible travel. You can use the preview section to see which logs are matched by the current query.

User attribute

For the user attribute, select the field in the analyzed log that contains the user ID. This can be an identifier like an email address, user name, or account identifier.

Location attribute

The location attribute specifies which field holds the geographic information for a log. The only supported value is @network.client.geoip, which is enriched by the GeoIP parser to give a log location information based on the client’s IP address.

Baseline user locations

Click the checkbox if you’d like Datadog to learn regular access locations before triggering a signal.

When selected, signals are suppressed for the first 24 hours. In that time, Datadog learns the user’s regular access locations. This can be helpful to reduce noise and infer VPN usage or credentialed API access.

Do not click the checkbox if you want Datadog to detect all impossible travel behavior.

Exclude benign activity with suppression queries

In the Only generate a signal if there is a match field, you have the option to enter a query so that a trigger is only generated when a value is met.

In the This rule will not generate a signal if there is a match field, you have the option to enter suppression queries so that a trigger is not generated when the values are met. For example, if a user called john.doe is triggering a signal, but their actions are benign and you no longer want signals triggered from this user, input a logs query that excludes @user.username: john.doe.

Root query

Construct a search query using the same logic as a log explorer search. The trigger defined for each new attribute generates a signal for each new value of that attribute over a 24-hour roll-up period.

Click Add Query to add additional queries.

Note: The query applies to all ingested logs.

Exclude benign activity with suppression queries

In the Only generate a signal if there is a match field, you have the option to enter a query so that a trigger is only generated when a value is met.

In the This rule will not generate a signal if there is a match field, you have the option to enter suppression queries so that a trigger is not generated when the values are met. For example, if a user called john.doe is triggering a signal, but their actions are benign and you no longer want signals triggered from this user, input a logs query that excludes @user.username: john.doe.

Trigger

Enable Create rules cases with the Then operator if you want to trigger a signal for the example: If query A occurs and then query B occurs. The then operator can only be used on a single rule case.

All rule cases are evaluated as case statements. Thus, the order of the cases affects which notifications are sent because the first case to match generates the signal. Click and drag your rule cases to change their ordering.

A rule case contains logical operations (>, >=, &&, ||) to determine if a signal should be generated based on the event counts in the previously defined queries. The ASCII lowercase query labels are referenced in this section. An example rule case for query a is a > 3.

Note: The query label must precede the operator. For example, a > 3 is allowed; 3 < a is not allowed.

Provide a name, for example “Case 1”, for each rule case. This name is appended to the rule name when a signal is generated.

Severity and notification

In the Set severity to dropdown menu, select the appropriate severity level (INFO, LOW, MEDIUM, HIGH, CRITICAL).

In the Notify section, optionally, configure notification targets for each rule case.

You can also create notification rules to avoid manual edits to notification preferences for individual detection rules.

Time windows

An evaluation window is specified to match when at least one of the cases matches true. This is a sliding window and evaluates cases in real time.

After a signal is generated, the signal remains “open” if a case is matched at least once within the keep alive window. Each time a new event matches any of the cases, the last updated timestamp is updated for the signal.

A signal closes once the time exceeds the maximum signal duration, regardless of the query being matched. This time is calculated from the first seen timestamp.

Click Add Case to add additional cases.

Note: The evaluation window must be less than or equal to the keep alive and maximum signal duration.

Severity and notification

In the Set severity to dropdown menu, select the appropriate severity level (INFO, LOW, MEDIUM, HIGH, CRITICAL).

In the Notify section, optionally, configure notification targets for each rule case.

You can also create notification rules to avoid manual edits to notification preferences for individual detection rules.

Forget value

To forget a value if it is not seen over a period of time, select an option from the dropdown menu.

Update the same signal

Set a maximum duration to keep updating a signal if new values are detected within a set time frame. For example, the same signal will update if any new value is detected within 1 hour, for a maximum duration of 24 hours.

Note: If a unique signal is required for every new value, configure this value to 0 minutes.

Severity and notification

In the Set severity to dropdown menu, select the appropriate severity level (INFO, LOW, MEDIUM, HIGH, CRITICAL).

In the Notify section, optionally, configure notification targets for each rule case.

You can also create notification rules to avoid manual edits to notification preferences for individual detection rules.

Time windows

Datadog automatically detects the seasonality of the data and generates a security signal when the data is determined to be anomalous.

Once a signal is generated, the signal remains “open” if the data remains anomalous and the last updated timestamp is updated for the anomalous duration.

A signal “closes” once the time exceeds the maximum signal duration, regardless of whether or not the anomaly is still anomalous. This time is calculated from the first seen timestamp.

The impossible travel detection method does not require setting a rule case.

Severity and notification

In the Set severity to dropdown menu, select the appropriate severity level (INFO, LOW, MEDIUM, HIGH, CRITICAL).

In the Notify section, optionally, configure notification targets for each rule case.

You can also create notification rules to avoid manual edits to notification preferences for individual detection rules.

Time windows

An evaluation window is specified to match when at least one of the cases matches true. This is a sliding window and evaluates cases in real time.

After a signal is generated, the signal remains “open” if a case is matched at least once within the keep alive window. Each time a new event matches any of the cases, the last updated timestamp is updated for the signal.

A signal closes once the time exceeds the maximum signal duration, regardless of the query being matched. This time is calculated from the first seen timestamp.

Trigger

All rule cases are evaluated as case statements. Thus, the order of the cases affects which notifications are sent because the first case to match generates the signal. Click and drag your rule cases to change their ordering.

A rule case contains logical operations (>, >=, &&, ||) to determine if a signal should be generated based on the event counts in the previously defined queries. The ASCII lowercase query labels are referenced in this section. An example rule case for query a is a > 3.

Note: The query label must precede the operator. For example, a > 3 is allowed; 3 < a is not allowed.

Severity and notification

In the Set severity to dropdown menu, select the appropriate severity level (INFO, LOW, MEDIUM, HIGH, CRITICAL).

In the Notify section, optionally, configure notification targets for each rule case.

You can also create notification rules to avoid manual edits to notification preferences for individual detection rules.

Click Add Case to add additional cases.