The Datadog Agent evaluates system activity on the Agent host. When activity matches an Agent rule expression, the Agent generates an event and passes it to the Datadog backend.
With the Agent Events Explorer, you can investigate Agent events separately from signals. Review what happened, where it occurred, and which Agent rule matched using the event side panel. You can also explore the investigation graph, event tree, remote access sessions, and raw JSON payload, and view triage and response instructions for the matching rule.
Investigate Agent events
To investigate an Agent event:
- Go to the Agent Events Explorer. Agent events are queried and displayed using the standard explorer controls in the Datadog Events Explorer.
- Select an Agent event. The side panel opens with tabs that help you investigate the event.
Overview
The Overview tab summarizes the event and is often the best place to start your investigation.
The Overview tab includes the following sections:
- What: A human-readable description of the detected activity. For example, A user executed the clang command on host i-0d85f97942d947ca9.
- Where: The infrastructure context where the event occurred, including cloud provider, account, region, host, Kubernetes cluster, namespace, pod, container, and image.
- Agent rule: The Agent rule that matched the event, including the rule name, event name, deployment policies, policy version, and rule expression.
- Investigation graph: A preview of the investigation graph at the bottom of the Overview tab.
- Event Tree: The execution lineage, affected infrastructure, and process, file, network, or kernel activity associated with the event.
Investigation graph
The Investigation graph is an interactive visualization that maps the infrastructure and processes involved in the event. It provides a compact overview of the attack chain by highlighting the most relevant entities and processes.
The graph traces the event from the host through the surrounding infrastructure—such as the Kubernetes pod, replica set, container, and container image—and into the process execution path. Main processes involved in the event are displayed individually, while less relevant processes are aggregated into grouped nodes (for example, +7 processes) to keep the view focused on the suspicious activity.
Use the investigation graph to understand how the detected activity fits into the broader runtime context without reviewing every process on the host.
Event tree
The Event Tree traces execution from the system init process through intermediate processes to the matching process, file, network, or kernel activity. Use it to reconstruct the execution path that led to the event and identify the affected infrastructure.
Each process entry displays:
- Path: The executable path and command-line arguments.
- PID: The process ID.
- User: The user context under which the process ran.
Expand a process entry to view its command, credentials, and executable metadata. Expand an infrastructure entry to view resource-specific information, such as status, tags, security details, or related actions.
Use the Show infrastructure entries toggle to show or hide the affected host, pod, and container.
Use attributes such as executable path, arguments, PID, and user to filter for related Agent events. Select View in JSON on a process or activity entry to open the corresponding location in the raw event JSON.
Remote access
When an Agent event occurs inside a remote access session, the Remote Access tab shows the full session the event belongs to. Use it to scope the surrounding activity instead of examining one event in isolation. A remote access session is an SSH session or a Kubernetes kubectl exec session, or both.
The tab supports the following session types:
- SSH: Interactive SSH sessions. Session details include the session ID, client IP, client port, authentication method, and authentication key.
- Kubernetes:
kubectl exec sessions. Session details include the username, groups, session ID, and UID.
For each session, the tab also shows the host it ran on, a timeline of its Agent events, and a table of those events. The timeline charts when the related events occurred, so you can spot bursts of activity across the session. Drag to select a range on the timeline to narrow the table to that window. Select an event in the table to open it in its own side panel and continue your investigation without leaving the session view. To continue in the full Agent Events Explorer, select Investigate all events in this session. This opens the explorer filtered to the session, so you can query and group every event in it using the standard explorer controls.
JSON
The JSON tab displays the raw event payload with the complete set of event attributes collected by the Agent. Use JSON when you need the most detailed view of the event data, for example, to write advanced queries in the Agent Events Explorer, or share the full event payload during an investigation. To filter in and out any field, you can click on it from the JSON.
Further reading
Additional helpful documentation, links, and articles: