Overview

Datadog’s GPU Monitoring provides a centralized view into GPU fleet health, cost, and performance. GPU Monitoring enables teams to make better provisioning decisions, troubleshoot failed workloads, and eliminate idle GPU costs without having to manually set up individual vendor tools (like NVIDIA’s DCGM). You can access insights into your GPU fleet by deploying the Datadog Agent.

For setup instructions, see Set up GPU Monitoring.

Make data-driven GPU allocation and provisioning decisions

With visibility into GPU utilization by host, node, or pod, you can identify hotspots or underutilization of expensive GPU infrastructure.

Troubleshoot failed workloads due to resource contention

Understand your current device availability and forecast how many devices are needed for certain teams or workloads to avoid failed workloads from resource contention.

Identify and eliminate wasted, idle GPU costs

Identify total spend on GPU infrastructure and attribute those costs to specific workloads and instances. Directly correlate GPU usage to related pods or processes.

Maximize model and application performance

With GPU Monitoring’s resource telemetry, you can analyze trends in GPU resources and metrics (including GPU utilization, power, and memory) over time, helping you understand their effects on your model and application performance.

Ready to start?

See Set up GPU Monitoring for instructions on how to set up Datadog’s GPU Monitoring.

Further Reading

Additional helpful documentation, links, and articles:

Set up GPU MonitoringDOCUMENTATION Optimize and troubleshoot AI infrastructure with Datadog GPU MonitoringBLOG