---
title: Monitor Inference Workloads on Kubernetes
description: >-
  Use Datadog's Kubernetes Explorer page to monitor AI inference workloads
  running on Kubernetes.
breadcrumbs: >-
  Docs > Containers > Container Monitoring > Monitor Inference Workloads on
  Kubernetes
---

> For the complete documentation index, see [llms.txt](https://docs.datadoghq.com/llms.txt).

# Monitor Inference Workloads on Kubernetes

{% callout %}
##### Join the Preview!

Inference Workloads is in Preview. To sign up, click **Request Access** and complete the form.

[Request Access](https://www.datadoghq.com/product-preview/kubernetes-inference/)
{% /callout %}

Use the [Inference Workloads view](https://app.datadoghq.com/orchestration/explorer/inferenceWorkload) in the Kubernetes Explorer to monitor AI inference workloads running on Kubernetes. The AI inference view provides insight from endpoint to compute, simplifying troubleshooting, investigation, and debugging. Use it to monitor relevant pods, deployments, and other Kubernetes resources. You can also view resource specifications for failed pods within a deployment and correlate node activity with related metrics, logs, and traces.

The Inference Workloads view is available for workloads deployed with:

- Dynamo
- KubeAI
- KServe
- Ray

Inference Workloads is included with Infrastructure Monitoring.

## Prerequisites{% #prerequisites %}

- Datadog Agent v7.86.0 or later with the [orchestratorExplorer feature](https://docs.datadoghq.com/containers/monitoring/kubernetes_explorer_configuration.md?tab=datadogoperator) enabled
- Cluster Agent v7.86.0 or later
- Both the Datadog Agent and the Cluster Agent must be running to collect Kubernetes resources

## View inference workloads{% #view-inference-workloads %}

In the Kubernetes Explorer, select AI > Inference Workloads.

The workload list view shows running inference workloads, with the following resource information for each:

- Cluster
- Namespace
- Model
- GPU Device Types

Click a value in any of these columns to access filter options, view related information, or view the resource's detail panel.

The list also shows health and performance metrics for each workload:

- Ready: The number of pods ready to serve traffic and the total number of pods
- Latency: Average end-to-end latency within vLLM engines, excluding frontend processes
- Requests: The number of running requests and the total number of running and waiting requests
- Error Rate: Percentage of Dynamo frontend requests with an error status (only available for Dynamo)

Anomalous values in these columns are highlighted to show potential areas for investigation. Detected anomalies include:

- Pods that are not ready to serve traffic
- Requests that are waiting, indicating engine saturation
- Error rates above 1% (yellow) and above 5% (red)

### Workload details{% #workload-details %}

Select a workload from the list to view its detail panel. The detail panel shows more information about the workload, including related resources in a list or map view and workload components. It also shows related metrics, logs, and traces. If you have GPU Monitoring enabled, click View in GPU Monitoring to view more performance and health metrics about the workload. To set up GPU Monitoring for inference workloads, see [Monitor inference workloads](https://docs.datadoghq.com/gpu_monitoring/inference.md?tab=standalonevllm).
