OOM Kill

OOM Kill

Agent Check Agent Check

Linux OS Supported

Overview

This check monitors the kernel OOM (out of memory) kill process through the Datadog Agent and the System Probe.

Setup

Installation

The OOM Kill check is included in the Datadog Agent package. It relies on an eBPF program implemented in the System Probe.

The eBPF program used by the System Probe is compiled at runtime and requires you to have access to the proper kernel headers.

On Debian-like distributions, install the kernel headers like this:

apt install -y linux-headers-$(uname -r)

On RHEL-like distributions, install the kernel headers like this:

yum install -y kernel-headers-$(uname -r)
yum install -y kernel-devel-$(uname -r)

Note: Kernel version 4.11 or later is required for the OOM Kill check to work. In addition, Windows, Container-Optimized OS, and CentOS/RHEL versions earlier than 8 are not supported.

Configuration

  1. In the system-probe.yaml file at the root of your Agent’s configuration directory, add the following configuration:

    system_probe_config:
        enable_oom_kill: true
    
  2. Ensure that the oom_kill.d/conf.yaml file is present in the conf.d/ folder at the root of your Agent’s configuration directory to start collecting your OOM Kill metrics. See the example oom_kill.d/conf.yaml for all available configuration options.

  3. Restart the Agent.

Configuration with Helm

With the Datadog Helm chart, ensure that the datadog.systemProbe and datadog.systemProbe.enableOOMKill parameters are enabled in the values.yaml file.

Validation

Run the Agent’s status subcommand and look for oom_kill under the Checks section.

Data Collected

Metrics

oom_kill.oom_process.count
(count)
The count of processes OOM killed
Shown as process

Service Checks

The OOM Kill check does not include any service checks.

Events

The OOM Kill check submits an event for each OOM Kill that includes the killed process ID and name, as well as the triggering process ID and name.

Troubleshooting

Need help? Contact Datadog support.