Overview

This check monitors Nvidia Triton through the Datadog Agent.

Setup

Follow the instructions below to install and configure this check for an Agent running on a host. For containerized environments, see the Autodiscovery Integration Templates for guidance on applying these instructions.

Installation

The Nvidia Triton check is included in the Datadog Agent package. No additional installation is needed on your server.

OpenMetrics endpoint

By default, the Nvidia Triton server exposes all metrics through the Prometheus endpoint. To enable all metrics reportings:

tritonserver --allow-metrics=true

To change the metric endpoint, use the --metrics-address option.

Example:

tritonserver --metrics-address=http://0.0.0.0:8002

In this case, the OpenMetrics endpoint is exposed at this URL: http://<NVIDIA_TRITON_ADDRESS>:8002/metrics.

The latency summary metrics are disabled by default. To enable summary metrics for latencies, use the command below:

tritonserver --metrics-config summary_latencies=true

The response cache metrics are not reported by default. You need to enable a cache implementation on the server side by specifying a <cache_implementation> and corresponding configuration.

For instance:

tritonserver --cache-config local,size=1048576

Nvidia Triton also offers the possibility to expose custom metrics through their Openemtrics endpoint. Datadog can also collect these custom metrics using the extra_metrics option.

Configuration

1. Edit the nvidia_triton.d/conf.yaml file, in the conf.d/ folder at the root of your Agent’s configuration directory to start collecting your nvidia_triton performance data. See the sample nvidia_triton.d/conf.yaml for all available configuration options.

2. Restart the Agent.

Validation

Run the Agent’s status subcommand and look for nvidia_triton under the Checks section.

Data Collected

Metrics

Events

The Nvidia Triton integration does not include any events.

Service Checks

nvidia_triton.openmetrics.health
Returns CRITICAL if the Agent is unable to connect to the Nvidia Triton OpenMetrics endpoint, otherwise returns OK.
Statuses: ok, critical

nvidia_triton.health.status
Returns CRITICAL if the Server is having a 4xx or 5xx response, OK if the response is 200, and unknown for everything else.
Statuses: ok, warning, critical

Logs

The Nvidia Triton integration can collect logs from the Nvidia Triton server and forward them to Datadog.

apiVersion: v1
kind: Pod
metadata:
  name: nvidia_triton
  annotations:
    ad.datadoghq.com/apache.logs: '[{"source":"nvidia_triton","service":"nvidia_triton"}]'
spec:
  containers:
    - name: ray

Troubleshooting

Need help? Contact Datadog support.