List LLM Observability prompts

Note: This endpoint is in preview and is subject to change. If you have any feedback, contact Datadog support.

GET https://api.ap1.datadoghq.com/api/v2/llm-obs/v1/promptshttps://api.ap2.datadoghq.com/api/v2/llm-obs/v1/promptshttps://api.datadoghq.eu/api/v2/llm-obs/v1/promptshttps://api.ddog-gov.com/api/v2/llm-obs/v1/promptshttps://api.us2.ddog-gov.com/api/v2/llm-obs/v1/promptshttps://api.uk1.datadoghq.com/api/v2/llm-obs/v1/promptshttps://api.datadoghq.com/api/v2/llm-obs/v1/promptshttps://api.us3.datadoghq.com/api/v2/llm-obs/v1/promptshttps://api.us5.datadoghq.com/api/v2/llm-obs/v1/prompts

Overview

List all LLM Observability prompts in the prompt registry for the organization. This endpoint requires the llm_observability_read permission.

Arguments

Query Strings

Name

Type

Description

filter[prompt_id]

string

Optional filter for prompts by prompt ID.

Response

OK

Response containing a list of LLM Observability prompts.

Expand All

Field

Type

Description

data [required]

[object]

List of LLM Observability prompts.

attributes [required]

object

Attributes of an LLM Observability prompt registry entry.

author

string

UUID of the user who authored the prompt.

created_at

date-time

Timestamp when the prompt was created.

created_from [required]

string

Source that created the prompt, such as ui-registry, sdk-registry, or sdk-instrumentation.

datasets

[object]

Datasets observed in runs associated with this prompt.

id [required]

string

Unique identifier of the dataset.

name

string

Name of the dataset.

description

string

Description of the prompt.

extracted_from

string

Source prompt from which this prompt was extracted, when applicable.

in_registry [required]

boolean

Whether the prompt is a registry entry (as opposed to a code-discovered prompt).

last_seen_at

date-time

Timestamp of the most recent observed run of this prompt.

last_version_created_at

date-time

Timestamp when the most recent version of the prompt was created.

ml_app

string

The ML application this prompt is associated with.

ml_apps

[string]

ML applications observed running this prompt.

num_versions [required]

int64

Number of versions of the prompt.

prompt_id [required]

string

Customer-provided identifier of the prompt.

source [required]

enum

Whether the prompt was created from the registry or discovered from observed LLM calls. Allowed enum values: registry,code

tags

[string]

Tags observed on runs of this prompt.

title

string

Title of the prompt.

id [required]

string

Unique identifier of the prompt.

type [required]

enum

Resource type of an LLM Observability prompt. Allowed enum values: prompt-templates

{
  "data": [
    {
      "attributes": {
        "author": "3b12f1df-14fd-4e12-bd6f-4a2f5c8b3d1e",
        "created_at": "2025-01-15T10:00:00Z",
        "created_from": "sdk-registry",
        "description": "Answers customer questions using the company knowledge base.",
        "in_registry": true,
        "last_version_created_at": "2025-02-01T14:30:00Z",
        "num_versions": 2,
        "prompt_id": "customer-support-assistant",
        "source": "registry",
        "title": "Customer Support Assistant"
      },
      "id": "4a1a28ff-8a25-5f0f-946f-f48264d772eb",
      "type": "prompt-templates"
    }
  ]
}

Unauthorized

API error response.

Expand All

Field

Type

Description

errors [required]

[object]

A list of errors.

detail

string

A human-readable explanation specific to this occurrence of the error.

meta

object

Non-standard meta-information about the error

source

object

References to the source of the error.

header

string

A string indicating the name of a single request header which caused the error.

parameter

string

A string indicating which URI query parameter caused the error.

pointer

string

A JSON pointer to the value in the request document that caused the error.

status

string

Status code of the response.

title

string

Short human-readable summary of the error.

{
  "errors": [
    {
      "detail": "Missing required attribute in body",
      "meta": {},
      "source": {
        "header": "Authorization",
        "parameter": "limit",
        "pointer": "/data/attributes/title"
      },
      "status": "400",
      "title": "Bad Request"
    }
  ]
}

Forbidden

API error response.

Expand All

Field

Type

Description

errors [required]

[object]

A list of errors.

detail

string

A human-readable explanation specific to this occurrence of the error.

meta

object

Non-standard meta-information about the error

source

object

References to the source of the error.

header

string

A string indicating the name of a single request header which caused the error.

parameter

string

A string indicating which URI query parameter caused the error.

pointer

string

A JSON pointer to the value in the request document that caused the error.

status

string

Status code of the response.

title

string

Short human-readable summary of the error.

{
  "errors": [
    {
      "detail": "Missing required attribute in body",
      "meta": {},
      "source": {
        "header": "Authorization",
        "parameter": "limit",
        "pointer": "/data/attributes/title"
      },
      "status": "400",
      "title": "Bad Request"
    }
  ]
}

Too many requests

API error response.

Expand All

Field

Type

Description

errors [required]

[string]

A list of errors.

{
  "errors": [
    "Bad Request"
  ]
}

Code Example

                  # Curl command
curl -X GET "https://api.ap1.datadoghq.com"https://api.ap2.datadoghq.com"https://api.datadoghq.eu"https://api.ddog-gov.com"https://api.us2.ddog-gov.com"https://api.uk1.datadoghq.com"https://api.datadoghq.com"https://api.us3.datadoghq.com"https://api.us5.datadoghq.com/api/v2/llm-obs/v1/prompts" \ -H "Accept: application/json" \ -H "DD-API-KEY: ${DD_API_KEY}" \ -H "DD-APPLICATION-KEY: ${DD_APP_KEY}"
"""
List LLM Observability prompts returns "OK" response
"""

from os import environ
from datadog_api_client import ApiClient, Configuration
from datadog_api_client.v2.api.llm_observability_api import LLMObservabilityApi

# there is a valid "prompt" in the system
PROMPT_DATA_ATTRIBUTES_PROMPT_ID = environ["PROMPT_DATA_ATTRIBUTES_PROMPT_ID"]

configuration = Configuration()
configuration.unstable_operations["list_llm_obs_prompts"] = True
with ApiClient(configuration) as api_client:
    api_instance = LLMObservabilityApi(api_client)
    response = api_instance.list_llm_obs_prompts(
        filter_prompt_id=PROMPT_DATA_ATTRIBUTES_PROMPT_ID,
    )

    print(response)

Instructions

First install the library and its dependencies and then save the example to example.py and run following commands:

    
DD_SITE="datadoghq.comus3.datadoghq.comus5.datadoghq.comdatadoghq.euap1.datadoghq.comap2.datadoghq.comuk1.datadoghq.comddog-gov.comus2.ddog-gov.com" DD_API_KEY="<DD_API_KEY>" DD_APP_KEY="<DD_APP_KEY>" python3 "example.py"
# List LLM Observability prompts returns "OK" response

require "datadog_api_client"
DatadogAPIClient.configure do |config|
  config.unstable_operations["v2.list_llm_obs_prompts".to_sym] = true
end
api_instance = DatadogAPIClient::V2::LLMObservabilityAPI.new

# there is a valid "prompt" in the system
PROMPT_DATA_ATTRIBUTES_PROMPT_ID = ENV["PROMPT_DATA_ATTRIBUTES_PROMPT_ID"]
opts = {
  filter_prompt_id: PROMPT_DATA_ATTRIBUTES_PROMPT_ID,
}
p api_instance.list_llm_obs_prompts(opts)

Instructions

First install the library and its dependencies and then save the example to example.rb and run following commands:

    
DD_SITE="datadoghq.comus3.datadoghq.comus5.datadoghq.comdatadoghq.euap1.datadoghq.comap2.datadoghq.comuk1.datadoghq.comddog-gov.comus2.ddog-gov.com" DD_API_KEY="<DD_API_KEY>" DD_APP_KEY="<DD_APP_KEY>" rb "example.rb"
// List LLM Observability prompts returns "OK" response

package main

import (
	"context"
	"encoding/json"
	"fmt"
	"os"

	"github.com/DataDog/datadog-api-client-go/v2/api/datadog"
	"github.com/DataDog/datadog-api-client-go/v2/api/datadogV2"
)

func main() {
	// there is a valid "prompt" in the system
	PromptDataAttributesPromptID := os.Getenv("PROMPT_DATA_ATTRIBUTES_PROMPT_ID")

	ctx := datadog.NewDefaultContext(context.Background())
	configuration := datadog.NewConfiguration()
	configuration.SetUnstableOperationEnabled("v2.ListLLMObsPrompts", true)
	apiClient := datadog.NewAPIClient(configuration)
	api := datadogV2.NewLLMObservabilityApi(apiClient)
	resp, r, err := api.ListLLMObsPrompts(ctx, *datadogV2.NewListLLMObsPromptsOptionalParameters().WithFilterPromptId(PromptDataAttributesPromptID))

	if err != nil {
		fmt.Fprintf(os.Stderr, "Error when calling `LLMObservabilityApi.ListLLMObsPrompts`: %v\n", err)
		fmt.Fprintf(os.Stderr, "Full HTTP response: %v\n", r)
	}

	responseContent, _ := json.MarshalIndent(resp, "", "  ")
	fmt.Fprintf(os.Stdout, "Response from `LLMObservabilityApi.ListLLMObsPrompts`:\n%s\n", responseContent)
}

Instructions

First install the library and its dependencies and then save the example to main.go and run following commands:

    
DD_SITE="datadoghq.comus3.datadoghq.comus5.datadoghq.comdatadoghq.euap1.datadoghq.comap2.datadoghq.comuk1.datadoghq.comddog-gov.comus2.ddog-gov.com" DD_API_KEY="<DD_API_KEY>" DD_APP_KEY="<DD_APP_KEY>" go run "main.go"
// List LLM Observability prompts returns "OK" response

import com.datadog.api.client.ApiClient;
import com.datadog.api.client.ApiException;
import com.datadog.api.client.v2.api.LlmObservabilityApi;
import com.datadog.api.client.v2.api.LlmObservabilityApi.ListLLMObsPromptsOptionalParameters;
import com.datadog.api.client.v2.model.LLMObsPromptsResponse;

public class Example {
  public static void main(String[] args) {
    ApiClient defaultClient = ApiClient.getDefaultApiClient();
    defaultClient.setUnstableOperationEnabled("v2.listLLMObsPrompts", true);
    LlmObservabilityApi apiInstance = new LlmObservabilityApi(defaultClient);

    // there is a valid "prompt" in the system
    String PROMPT_DATA_ATTRIBUTES_PROMPT_ID = System.getenv("PROMPT_DATA_ATTRIBUTES_PROMPT_ID");

    try {
      LLMObsPromptsResponse result =
          apiInstance.listLLMObsPrompts(
              new ListLLMObsPromptsOptionalParameters()
                  .filterPromptId(PROMPT_DATA_ATTRIBUTES_PROMPT_ID));
      System.out.println(result);
    } catch (ApiException e) {
      System.err.println("Exception when calling LlmObservabilityApi#listLLMObsPrompts");
      System.err.println("Status code: " + e.getCode());
      System.err.println("Reason: " + e.getResponseBody());
      System.err.println("Response headers: " + e.getResponseHeaders());
      e.printStackTrace();
    }
  }
}

Instructions

First install the library and its dependencies and then save the example to Example.java and run following commands:

    
DD_SITE="datadoghq.comus3.datadoghq.comus5.datadoghq.comdatadoghq.euap1.datadoghq.comap2.datadoghq.comuk1.datadoghq.comddog-gov.comus2.ddog-gov.com" DD_API_KEY="<DD_API_KEY>" DD_APP_KEY="<DD_APP_KEY>" java "Example.java"
// List LLM Observability prompts returns "OK" response
use datadog_api_client::datadog;
use datadog_api_client::datadogV2::api_llm_observability::LLMObservabilityAPI;
use datadog_api_client::datadogV2::api_llm_observability::ListLLMObsPromptsOptionalParams;

#[tokio::main]
async fn main() {
    // there is a valid "prompt" in the system
    let prompt_data_attributes_prompt_id =
        std::env::var("PROMPT_DATA_ATTRIBUTES_PROMPT_ID").unwrap();
    let mut configuration = datadog::Configuration::new();
    configuration.set_unstable_operation_enabled("v2.ListLLMObsPrompts", true);
    let api = LLMObservabilityAPI::with_config(configuration);
    let resp = api
        .list_llm_obs_prompts(
            ListLLMObsPromptsOptionalParams::default()
                .filter_prompt_id(prompt_data_attributes_prompt_id.clone()),
        )
        .await;
    if let Ok(value) = resp {
        println!("{:#?}", value);
    } else {
        println!("{:#?}", resp.unwrap_err());
    }
}

Instructions

First install the library and its dependencies and then save the example to src/main.rs and run following commands:

    
DD_SITE="datadoghq.comus3.datadoghq.comus5.datadoghq.comdatadoghq.euap1.datadoghq.comap2.datadoghq.comuk1.datadoghq.comddog-gov.comus2.ddog-gov.com" DD_API_KEY="<DD_API_KEY>" DD_APP_KEY="<DD_APP_KEY>" cargo run
/**
 * List LLM Observability prompts returns "OK" response
 */

import { client, v2 } from "@datadog/datadog-api-client";

const configuration = client.createConfiguration();
configuration.unstableOperations["v2.listLLMObsPrompts"] = true;
const apiInstance = new v2.LLMObservabilityApi(configuration);

// there is a valid "prompt" in the system
const PROMPT_DATA_ATTRIBUTES_PROMPT_ID = process.env
  .PROMPT_DATA_ATTRIBUTES_PROMPT_ID as string;

const params: v2.LLMObservabilityApiListLLMObsPromptsRequest = {
  filterPromptId: PROMPT_DATA_ATTRIBUTES_PROMPT_ID,
};

apiInstance
  .listLLMObsPrompts(params)
  .then((data: v2.LLMObsPromptsResponse) => {
    console.log(
      "API called successfully. Returned data: " + JSON.stringify(data)
    );
  })
  .catch((error: any) => console.error(error));

Instructions

First install the library and its dependencies and then save the example to example.ts and run following commands:

    
DD_SITE="datadoghq.comus3.datadoghq.comus5.datadoghq.comdatadoghq.euap1.datadoghq.comap2.datadoghq.comuk1.datadoghq.comddog-gov.comus2.ddog-gov.com" DD_API_KEY="<DD_API_KEY>" DD_APP_KEY="<DD_APP_KEY>" tsc "example.ts"