Quickstart

This page demonstrates using Datadog’s LLM Observability SDK to instrument a Python or Node.js LLM application.

Prerequisites

LLM Observability requires a Datadog API key if you don’t have a Datadog Agent running. Find your API key in Datadog.

Setup

Install the SDK:
```
pip install ddtrace
```

Prefix your Python start command with ddtrace-run:

DD_LLMOBS_ENABLED=1 \
DD_LLMOBS_ML_APP=quickstart-app \
DD_API_KEY=<YOUR_DATADOG_API_KEY> \
ddtrace-run <your application command>

Replace <YOUR_DATADOG_API_KEY> with your Datadog API key.

Install the SDK:
```
npm install dd-trace
```

Add NODE_OPTIONS to your Node.js start command:

DD_LLMOBS_ENABLED=1 \
DD_LLMOBS_ML_APP=quickstart-app \
DD_API_KEY=<YOUR_DATADOG_API_KEY> \
NODE_OPTIONS="--import dd-trace/initialize.mjs" <your application command>

Replace <YOUR_DATADOG_API_KEY> with your Datadog API key.

View traces

Make requests to your application triggering LLM calls and then view traces in the Traces tab of the LLM Observability page in Datadog. If you don’t see any traces, make sure you are using a supported library. Otherwise, you may need to instrument your application’s LLM calls manually.

Next steps

After traces are being submitted from your application, you can:

Configure evaluations that you can use to assess the effectiveness of your LLM application.
Add custom instrumentation to your application and extract data that automatic instrumentation cannot.

Example “Hello World” application

See below for a simple application that can be used to begin exploring the LLM Observability product.

Install OpenAI with pip install openai.

Save example script app.py.

import os
from openai import OpenAI

oai_client = OpenAI(api_key=os.environ.get("OPENAI_API_KEY"))
completion = oai_client.chat.completions.create(
    model="gpt-4o-mini",
    messages=[
     {"role": "system", "content": "You are a helpful customer assistant for a furniture store."},
     {"role": "user", "content": "I'd like to buy a chair for my living room."},
 ],
)

Run the application:

# Make sure you have the required environment variables listed above
DD_...= \
ddtrace-run app.py

Install OpenAI npm install openai.

Save example script app.js

const { OpenAI } = require('openai');
const oaiClient = new OpenAI(process.env.OPENAI_API_KEY);

async function main () {
    const completion = await oaiClient.chat.completions.create({
       model: 'gpt-4o-mini',
       messages: [
          { role: 'system', content: 'You are a helpful customer assistant for a furniture store.' },
          { role: 'user', content: 'I\'d like to buy a chair for my living room.' },
       ]
    });
    return completion;
}

main().then(console.log)