Quickstart

This product is not supported for your selected Datadog site. ().

This page demonstrates using Datadog’s LLM Observability SDK to instrument a Python, Node.js, or Java LLM application.

Prerequisites

LLM Observability requires a Datadog API key if you don’t have a Datadog Agent running. Find your API key in Datadog.

Setup

    1. Install the SDK:

      pip install ddtrace

    2. Prefix your Python start command with ddtrace-run:

      DD_LLMOBS_ENABLED=1 \
DD_LLMOBS_ML_APP=quickstart-app \
DD_API_KEY=<YOUR_DATADOG_API_KEY> \
ddtrace-run <your application command>

      Replace <YOUR_DATADOG_API_KEY> with your Datadog API key.

    1. Install the SDK:

      npm install dd-trace

    2. Add NODE_OPTIONS to your Node.js start command:

      DD_LLMOBS_ENABLED=1 \
DD_LLMOBS_ML_APP=quickstart-app \
DD_API_KEY=<YOUR_DATADOG_API_KEY> \
NODE_OPTIONS="--import dd-trace/initialize.mjs" <your application command>

      Replace <YOUR_DATADOG_API_KEY> with your Datadog API key.

    1. Install the SDK:

      wget -O dd-java-agent.jar 'https://dtdg.co/latest-java-tracer'

    2. Add the -javaagent JVM argument to your Java start command:

      java -javaagent:/path/to/dd-java-agent.jar \
-Ddd.llmobs.enabled=true \
-Ddd.llmobs.ml.app=quickstart-app \
-Ddd.api.key=<YOUR_DATADOG_API_KEY> \
-jar path/to/your/app.jar

      Replace <YOUR_DATADOG_API_KEY> with your Datadog API key.

    View traces

    Make requests to your application triggering LLM calls and then view traces in the Traces tab of the LLM Observability page in Datadog. If you don’t see any traces, make sure you are using a supported library. Otherwise, you may need to instrument your application’s LLM calls manually.

    Next steps

    After traces are being submitted from your application, you can:

    Example “Hello World” application

    See below for a simple application that can be used to begin exploring the LLM Observability product.

      1. Install OpenAI with pip install openai.

      2. Save example script app.py:

        import os
from openai import OpenAI

oai_client = OpenAI(api_key=os.environ.get("OPENAI_API_KEY"))
completion = oai_client.chat.completions.create(
    model="gpt-4o-mini",
    messages=[
     {"role": "system", "content": "You are a helpful customer assistant for a furniture store."},
     {"role": "user", "content": "I'd like to buy a chair for my living room."},
 ],
)

      3. Run the application:

        DD_LLMOBS_ENABLED=1 \
DD_LLMOBS_ML_APP=quickstart-app \
DD_API_KEY=<YOUR_DATADOG_API_KEY> \
ddtrace-run app.py

      1. Install OpenAI with npm install openai.

      2. Save example script app.js:

        const { OpenAI } = require('openai');
const oaiClient = new OpenAI(process.env.OPENAI_API_KEY);

async function main () {
    const completion = await oaiClient.chat.completions.create({
       model: 'gpt-4o-mini',
       messages: [
          { role: 'system', content: 'You are a helpful customer assistant for a furniture store.' },
          { role: 'user', content: 'I\'d like to buy a chair for my living room.' },
       ]
    });
    return completion;
}

main().then(console.log)

      3. Run the application:

        DD_LLMOBS_ENABLED=1 \
DD_LLMOBS_ML_APP=quickstart-app \
DD_API_KEY=<YOUR_DATADOG_API_KEY> \
NODE_OPTIONS="--import dd-trace/initialize.mjs" node app.js

      1. Install OpenAI:

        • Maven: Add this dependency to your pom.xml:

          <dependency>
  <groupId>com.openai</groupId>
  <artifactId>openai</artifactId>
  <version>1.0.0</version>
</dependency>

        • Gradle: Add this line to your build.gradle:

          implementation 'com.openai:openai:1.0.0'

      2. Save example script app.java:

        import com.openai.OpenAI;
import com.openai.api.models.ChatCompletionRequest;
import com.openai.api.models.ChatCompletionResponse;
import com.openai.api.models.ChatMessage;

import java.util.Arrays;

public class App {
    public static void main(String[] args) {
        String apiKey = System.getenv("OPENAI_API_KEY");
        OpenAI openAI = new OpenAI(apiKey);

        ChatCompletionRequest request = ChatCompletionRequest.builder()
            .model("gpt-4o-mini")
            .messages(Arrays.asList(
                ChatMessage.builder()
                    .role("system")
                    .content("You are a helpful customer assistant for a furniture store.")
                    .build(),
                ChatMessage.builder()
                    .role("user")
                    .content("I'd like to buy a chair for my living room.")
                    .build()
            ))
            .build();

        try {
            ChatCompletionResponse response = openAI.chat().completions().create(request);
            System.out.println(response);
        } catch (Exception e) {
            e.printStackTrace();
        }
    }
}

      3. Run the application:

        java -javaagent:/path/to/dd-java-agent.jar \
-Ddd.llmobs.enabled=true \
-Ddd.llmobs.ml.app=quickstart-app \
-Ddd.api.key=<YOUR_DATADOG_API_KEY> \
-jar path/to/your/app.jar

      Further Reading

      Additional helpful documentation, links, and articles:

      Configure Evaluations on your applicationEVALUATIONS more more Instrument your application with custom spansCUSTOM INSTRUMENTATION more more