---
title: LLM Observability
description: Datadog, the leading service for cloud-scale monitoring.
breadcrumbs: Docs > API Reference > LLM Observability
---

# LLM Observability

Manage LLM Observability spans, data, projects, datasets, dataset records, experiments, and annotations.

## List LLM Observability projects → preview{% #list-llm-observability-projects %}

| Datadog site      | API endpoint                                                 |
| ----------------- | ------------------------------------------------------------ |
| ap1.datadoghq.com | GET https://api.ap1.datadoghq.com/api/v2/llm-obs/v1/projects |
| ap2.datadoghq.com | GET https://api.ap2.datadoghq.com/api/v2/llm-obs/v1/projects |
| app.datadoghq.eu  | GET https://api.datadoghq.eu/api/v2/llm-obs/v1/projects      |
| app.ddog-gov.com  | GET https://api.ddog-gov.com/api/v2/llm-obs/v1/projects      |
| us2.ddog-gov.com  | GET https://api.us2.ddog-gov.com/api/v2/llm-obs/v1/projects  |
| app.datadoghq.com | GET https://api.datadoghq.com/api/v2/llm-obs/v1/projects     |
| us3.datadoghq.com | GET https://api.us3.datadoghq.com/api/v2/llm-obs/v1/projects |
| us5.datadoghq.com | GET https://api.us5.datadoghq.com/api/v2/llm-obs/v1/projects |

## Create an LLM Observability project → preview{% #create-an-llm-observability-project %}

| Datadog site      | API endpoint                                                  |
| ----------------- | ------------------------------------------------------------- |
| ap1.datadoghq.com | POST https://api.ap1.datadoghq.com/api/v2/llm-obs/v1/projects |
| ap2.datadoghq.com | POST https://api.ap2.datadoghq.com/api/v2/llm-obs/v1/projects |
| app.datadoghq.eu  | POST https://api.datadoghq.eu/api/v2/llm-obs/v1/projects      |
| app.ddog-gov.com  | POST https://api.ddog-gov.com/api/v2/llm-obs/v1/projects      |
| us2.ddog-gov.com  | POST https://api.us2.ddog-gov.com/api/v2/llm-obs/v1/projects  |
| app.datadoghq.com | POST https://api.datadoghq.com/api/v2/llm-obs/v1/projects     |
| us3.datadoghq.com | POST https://api.us3.datadoghq.com/api/v2/llm-obs/v1/projects |
| us5.datadoghq.com | POST https://api.us5.datadoghq.com/api/v2/llm-obs/v1/projects |

## Update an LLM Observability project → preview{% #update-an-llm-observability-project %}

| Datadog site      | API endpoint                                                                |
| ----------------- | --------------------------------------------------------------------------- |
| ap1.datadoghq.com | PATCH https://api.ap1.datadoghq.com/api/v2/llm-obs/v1/projects/{project_id} |
| ap2.datadoghq.com | PATCH https://api.ap2.datadoghq.com/api/v2/llm-obs/v1/projects/{project_id} |
| app.datadoghq.eu  | PATCH https://api.datadoghq.eu/api/v2/llm-obs/v1/projects/{project_id}      |
| app.ddog-gov.com  | PATCH https://api.ddog-gov.com/api/v2/llm-obs/v1/projects/{project_id}      |
| us2.ddog-gov.com  | PATCH https://api.us2.ddog-gov.com/api/v2/llm-obs/v1/projects/{project_id}  |
| app.datadoghq.com | PATCH https://api.datadoghq.com/api/v2/llm-obs/v1/projects/{project_id}     |
| us3.datadoghq.com | PATCH https://api.us3.datadoghq.com/api/v2/llm-obs/v1/projects/{project_id} |
| us5.datadoghq.com | PATCH https://api.us5.datadoghq.com/api/v2/llm-obs/v1/projects/{project_id} |

## Delete LLM Observability projects → preview{% #delete-llm-observability-projects %}

| Datadog site      | API endpoint                                                         |
| ----------------- | -------------------------------------------------------------------- |
| ap1.datadoghq.com | POST https://api.ap1.datadoghq.com/api/v2/llm-obs/v1/projects/delete |
| ap2.datadoghq.com | POST https://api.ap2.datadoghq.com/api/v2/llm-obs/v1/projects/delete |
| app.datadoghq.eu  | POST https://api.datadoghq.eu/api/v2/llm-obs/v1/projects/delete      |
| app.ddog-gov.com  | POST https://api.ddog-gov.com/api/v2/llm-obs/v1/projects/delete      |
| us2.ddog-gov.com  | POST https://api.us2.ddog-gov.com/api/v2/llm-obs/v1/projects/delete  |
| app.datadoghq.com | POST https://api.datadoghq.com/api/v2/llm-obs/v1/projects/delete     |
| us3.datadoghq.com | POST https://api.us3.datadoghq.com/api/v2/llm-obs/v1/projects/delete |
| us5.datadoghq.com | POST https://api.us5.datadoghq.com/api/v2/llm-obs/v1/projects/delete |

## List LLM Observability datasets → preview{% #list-llm-observability-datasets %}

| Datadog site      | API endpoint                                                              |
| ----------------- | ------------------------------------------------------------------------- |
| ap1.datadoghq.com | GET https://api.ap1.datadoghq.com/api/v2/llm-obs/v1/{project_id}/datasets |
| ap2.datadoghq.com | GET https://api.ap2.datadoghq.com/api/v2/llm-obs/v1/{project_id}/datasets |
| app.datadoghq.eu  | GET https://api.datadoghq.eu/api/v2/llm-obs/v1/{project_id}/datasets      |
| app.ddog-gov.com  | GET https://api.ddog-gov.com/api/v2/llm-obs/v1/{project_id}/datasets      |
| us2.ddog-gov.com  | GET https://api.us2.ddog-gov.com/api/v2/llm-obs/v1/{project_id}/datasets  |
| app.datadoghq.com | GET https://api.datadoghq.com/api/v2/llm-obs/v1/{project_id}/datasets     |
| us3.datadoghq.com | GET https://api.us3.datadoghq.com/api/v2/llm-obs/v1/{project_id}/datasets |
| us5.datadoghq.com | GET https://api.us5.datadoghq.com/api/v2/llm-obs/v1/{project_id}/datasets |

## Create an LLM Observability dataset → preview{% #create-an-llm-observability-dataset %}

| Datadog site      | API endpoint                                                               |
| ----------------- | -------------------------------------------------------------------------- |
| ap1.datadoghq.com | POST https://api.ap1.datadoghq.com/api/v2/llm-obs/v1/{project_id}/datasets |
| ap2.datadoghq.com | POST https://api.ap2.datadoghq.com/api/v2/llm-obs/v1/{project_id}/datasets |
| app.datadoghq.eu  | POST https://api.datadoghq.eu/api/v2/llm-obs/v1/{project_id}/datasets      |
| app.ddog-gov.com  | POST https://api.ddog-gov.com/api/v2/llm-obs/v1/{project_id}/datasets      |
| us2.ddog-gov.com  | POST https://api.us2.ddog-gov.com/api/v2/llm-obs/v1/{project_id}/datasets  |
| app.datadoghq.com | POST https://api.datadoghq.com/api/v2/llm-obs/v1/{project_id}/datasets     |
| us3.datadoghq.com | POST https://api.us3.datadoghq.com/api/v2/llm-obs/v1/{project_id}/datasets |
| us5.datadoghq.com | POST https://api.us5.datadoghq.com/api/v2/llm-obs/v1/{project_id}/datasets |

## List LLM Observability dataset records → preview{% #list-llm-observability-dataset-records %}

| Datadog site      | API endpoint                                                                                   |
| ----------------- | ---------------------------------------------------------------------------------------------- |
| ap1.datadoghq.com | GET https://api.ap1.datadoghq.com/api/v2/llm-obs/v1/{project_id}/datasets/{dataset_id}/records |
| ap2.datadoghq.com | GET https://api.ap2.datadoghq.com/api/v2/llm-obs/v1/{project_id}/datasets/{dataset_id}/records |
| app.datadoghq.eu  | GET https://api.datadoghq.eu/api/v2/llm-obs/v1/{project_id}/datasets/{dataset_id}/records      |
| app.ddog-gov.com  | GET https://api.ddog-gov.com/api/v2/llm-obs/v1/{project_id}/datasets/{dataset_id}/records      |
| us2.ddog-gov.com  | GET https://api.us2.ddog-gov.com/api/v2/llm-obs/v1/{project_id}/datasets/{dataset_id}/records  |
| app.datadoghq.com | GET https://api.datadoghq.com/api/v2/llm-obs/v1/{project_id}/datasets/{dataset_id}/records     |
| us3.datadoghq.com | GET https://api.us3.datadoghq.com/api/v2/llm-obs/v1/{project_id}/datasets/{dataset_id}/records |
| us5.datadoghq.com | GET https://api.us5.datadoghq.com/api/v2/llm-obs/v1/{project_id}/datasets/{dataset_id}/records |

## Append records to an LLM Observability dataset → preview{% #append-records-to-an-llm-observability-dataset %}

| Datadog site      | API endpoint                                                                                    |
| ----------------- | ----------------------------------------------------------------------------------------------- |
| ap1.datadoghq.com | POST https://api.ap1.datadoghq.com/api/v2/llm-obs/v1/{project_id}/datasets/{dataset_id}/records |
| ap2.datadoghq.com | POST https://api.ap2.datadoghq.com/api/v2/llm-obs/v1/{project_id}/datasets/{dataset_id}/records |
| app.datadoghq.eu  | POST https://api.datadoghq.eu/api/v2/llm-obs/v1/{project_id}/datasets/{dataset_id}/records      |
| app.ddog-gov.com  | POST https://api.ddog-gov.com/api/v2/llm-obs/v1/{project_id}/datasets/{dataset_id}/records      |
| us2.ddog-gov.com  | POST https://api.us2.ddog-gov.com/api/v2/llm-obs/v1/{project_id}/datasets/{dataset_id}/records  |
| app.datadoghq.com | POST https://api.datadoghq.com/api/v2/llm-obs/v1/{project_id}/datasets/{dataset_id}/records     |
| us3.datadoghq.com | POST https://api.us3.datadoghq.com/api/v2/llm-obs/v1/{project_id}/datasets/{dataset_id}/records |
| us5.datadoghq.com | POST https://api.us5.datadoghq.com/api/v2/llm-obs/v1/{project_id}/datasets/{dataset_id}/records |

## Update an LLM Observability dataset → preview{% #update-an-llm-observability-dataset %}

| Datadog site      | API endpoint                                                                             |
| ----------------- | ---------------------------------------------------------------------------------------- |
| ap1.datadoghq.com | PATCH https://api.ap1.datadoghq.com/api/v2/llm-obs/v1/{project_id}/datasets/{dataset_id} |
| ap2.datadoghq.com | PATCH https://api.ap2.datadoghq.com/api/v2/llm-obs/v1/{project_id}/datasets/{dataset_id} |
| app.datadoghq.eu  | PATCH https://api.datadoghq.eu/api/v2/llm-obs/v1/{project_id}/datasets/{dataset_id}      |
| app.ddog-gov.com  | PATCH https://api.ddog-gov.com/api/v2/llm-obs/v1/{project_id}/datasets/{dataset_id}      |
| us2.ddog-gov.com  | PATCH https://api.us2.ddog-gov.com/api/v2/llm-obs/v1/{project_id}/datasets/{dataset_id}  |
| app.datadoghq.com | PATCH https://api.datadoghq.com/api/v2/llm-obs/v1/{project_id}/datasets/{dataset_id}     |
| us3.datadoghq.com | PATCH https://api.us3.datadoghq.com/api/v2/llm-obs/v1/{project_id}/datasets/{dataset_id} |
| us5.datadoghq.com | PATCH https://api.us5.datadoghq.com/api/v2/llm-obs/v1/{project_id}/datasets/{dataset_id} |

## Update LLM Observability dataset records → preview{% #update-llm-observability-dataset-records %}

| Datadog site      | API endpoint                                                                                     |
| ----------------- | ------------------------------------------------------------------------------------------------ |
| ap1.datadoghq.com | PATCH https://api.ap1.datadoghq.com/api/v2/llm-obs/v1/{project_id}/datasets/{dataset_id}/records |
| ap2.datadoghq.com | PATCH https://api.ap2.datadoghq.com/api/v2/llm-obs/v1/{project_id}/datasets/{dataset_id}/records |
| app.datadoghq.eu  | PATCH https://api.datadoghq.eu/api/v2/llm-obs/v1/{project_id}/datasets/{dataset_id}/records      |
| app.ddog-gov.com  | PATCH https://api.ddog-gov.com/api/v2/llm-obs/v1/{project_id}/datasets/{dataset_id}/records      |
| us2.ddog-gov.com  | PATCH https://api.us2.ddog-gov.com/api/v2/llm-obs/v1/{project_id}/datasets/{dataset_id}/records  |
| app.datadoghq.com | PATCH https://api.datadoghq.com/api/v2/llm-obs/v1/{project_id}/datasets/{dataset_id}/records     |
| us3.datadoghq.com | PATCH https://api.us3.datadoghq.com/api/v2/llm-obs/v1/{project_id}/datasets/{dataset_id}/records |
| us5.datadoghq.com | PATCH https://api.us5.datadoghq.com/api/v2/llm-obs/v1/{project_id}/datasets/{dataset_id}/records |

## Delete LLM Observability datasets → preview{% #delete-llm-observability-datasets %}

| Datadog site      | API endpoint                                                                      |
| ----------------- | --------------------------------------------------------------------------------- |
| ap1.datadoghq.com | POST https://api.ap1.datadoghq.com/api/v2/llm-obs/v1/{project_id}/datasets/delete |
| ap2.datadoghq.com | POST https://api.ap2.datadoghq.com/api/v2/llm-obs/v1/{project_id}/datasets/delete |
| app.datadoghq.eu  | POST https://api.datadoghq.eu/api/v2/llm-obs/v1/{project_id}/datasets/delete      |
| app.ddog-gov.com  | POST https://api.ddog-gov.com/api/v2/llm-obs/v1/{project_id}/datasets/delete      |
| us2.ddog-gov.com  | POST https://api.us2.ddog-gov.com/api/v2/llm-obs/v1/{project_id}/datasets/delete  |
| app.datadoghq.com | POST https://api.datadoghq.com/api/v2/llm-obs/v1/{project_id}/datasets/delete     |
| us3.datadoghq.com | POST https://api.us3.datadoghq.com/api/v2/llm-obs/v1/{project_id}/datasets/delete |
| us5.datadoghq.com | POST https://api.us5.datadoghq.com/api/v2/llm-obs/v1/{project_id}/datasets/delete |

## Delete LLM Observability dataset records → preview{% #delete-llm-observability-dataset-records %}

| Datadog site      | API endpoint                                                                                           |
| ----------------- | ------------------------------------------------------------------------------------------------------ |
| ap1.datadoghq.com | POST https://api.ap1.datadoghq.com/api/v2/llm-obs/v1/{project_id}/datasets/{dataset_id}/records/delete |
| ap2.datadoghq.com | POST https://api.ap2.datadoghq.com/api/v2/llm-obs/v1/{project_id}/datasets/{dataset_id}/records/delete |
| app.datadoghq.eu  | POST https://api.datadoghq.eu/api/v2/llm-obs/v1/{project_id}/datasets/{dataset_id}/records/delete      |
| app.ddog-gov.com  | POST https://api.ddog-gov.com/api/v2/llm-obs/v1/{project_id}/datasets/{dataset_id}/records/delete      |
| us2.ddog-gov.com  | POST https://api.us2.ddog-gov.com/api/v2/llm-obs/v1/{project_id}/datasets/{dataset_id}/records/delete  |
| app.datadoghq.com | POST https://api.datadoghq.com/api/v2/llm-obs/v1/{project_id}/datasets/{dataset_id}/records/delete     |
| us3.datadoghq.com | POST https://api.us3.datadoghq.com/api/v2/llm-obs/v1/{project_id}/datasets/{dataset_id}/records/delete |
| us5.datadoghq.com | POST https://api.us5.datadoghq.com/api/v2/llm-obs/v1/{project_id}/datasets/{dataset_id}/records/delete |

## List LLM Observability experiments → preview{% #list-llm-observability-experiments %}

| Datadog site      | API endpoint                                                    |
| ----------------- | --------------------------------------------------------------- |
| ap1.datadoghq.com | GET https://api.ap1.datadoghq.com/api/v2/llm-obs/v1/experiments |
| ap2.datadoghq.com | GET https://api.ap2.datadoghq.com/api/v2/llm-obs/v1/experiments |
| app.datadoghq.eu  | GET https://api.datadoghq.eu/api/v2/llm-obs/v1/experiments      |
| app.ddog-gov.com  | GET https://api.ddog-gov.com/api/v2/llm-obs/v1/experiments      |
| us2.ddog-gov.com  | GET https://api.us2.ddog-gov.com/api/v2/llm-obs/v1/experiments  |
| app.datadoghq.com | GET https://api.datadoghq.com/api/v2/llm-obs/v1/experiments     |
| us3.datadoghq.com | GET https://api.us3.datadoghq.com/api/v2/llm-obs/v1/experiments |
| us5.datadoghq.com | GET https://api.us5.datadoghq.com/api/v2/llm-obs/v1/experiments |

## Create an LLM Observability experiment → preview{% #create-an-llm-observability-experiment %}

| Datadog site      | API endpoint                                                     |
| ----------------- | ---------------------------------------------------------------- |
| ap1.datadoghq.com | POST https://api.ap1.datadoghq.com/api/v2/llm-obs/v1/experiments |
| ap2.datadoghq.com | POST https://api.ap2.datadoghq.com/api/v2/llm-obs/v1/experiments |
| app.datadoghq.eu  | POST https://api.datadoghq.eu/api/v2/llm-obs/v1/experiments      |
| app.ddog-gov.com  | POST https://api.ddog-gov.com/api/v2/llm-obs/v1/experiments      |
| us2.ddog-gov.com  | POST https://api.us2.ddog-gov.com/api/v2/llm-obs/v1/experiments  |
| app.datadoghq.com | POST https://api.datadoghq.com/api/v2/llm-obs/v1/experiments     |
| us3.datadoghq.com | POST https://api.us3.datadoghq.com/api/v2/llm-obs/v1/experiments |
| us5.datadoghq.com | POST https://api.us5.datadoghq.com/api/v2/llm-obs/v1/experiments |

## Update an LLM Observability experiment → preview{% #update-an-llm-observability-experiment %}

| Datadog site      | API endpoint                                                                      |
| ----------------- | --------------------------------------------------------------------------------- |
| ap1.datadoghq.com | PATCH https://api.ap1.datadoghq.com/api/v2/llm-obs/v1/experiments/{experiment_id} |
| ap2.datadoghq.com | PATCH https://api.ap2.datadoghq.com/api/v2/llm-obs/v1/experiments/{experiment_id} |
| app.datadoghq.eu  | PATCH https://api.datadoghq.eu/api/v2/llm-obs/v1/experiments/{experiment_id}      |
| app.ddog-gov.com  | PATCH https://api.ddog-gov.com/api/v2/llm-obs/v1/experiments/{experiment_id}      |
| us2.ddog-gov.com  | PATCH https://api.us2.ddog-gov.com/api/v2/llm-obs/v1/experiments/{experiment_id}  |
| app.datadoghq.com | PATCH https://api.datadoghq.com/api/v2/llm-obs/v1/experiments/{experiment_id}     |
| us3.datadoghq.com | PATCH https://api.us3.datadoghq.com/api/v2/llm-obs/v1/experiments/{experiment_id} |
| us5.datadoghq.com | PATCH https://api.us5.datadoghq.com/api/v2/llm-obs/v1/experiments/{experiment_id} |

## Delete LLM Observability experiments → preview{% #delete-llm-observability-experiments %}

| Datadog site      | API endpoint                                                            |
| ----------------- | ----------------------------------------------------------------------- |
| ap1.datadoghq.com | POST https://api.ap1.datadoghq.com/api/v2/llm-obs/v1/experiments/delete |
| ap2.datadoghq.com | POST https://api.ap2.datadoghq.com/api/v2/llm-obs/v1/experiments/delete |
| app.datadoghq.eu  | POST https://api.datadoghq.eu/api/v2/llm-obs/v1/experiments/delete      |
| app.ddog-gov.com  | POST https://api.ddog-gov.com/api/v2/llm-obs/v1/experiments/delete      |
| us2.ddog-gov.com  | POST https://api.us2.ddog-gov.com/api/v2/llm-obs/v1/experiments/delete  |
| app.datadoghq.com | POST https://api.datadoghq.com/api/v2/llm-obs/v1/experiments/delete     |
| us3.datadoghq.com | POST https://api.us3.datadoghq.com/api/v2/llm-obs/v1/experiments/delete |
| us5.datadoghq.com | POST https://api.us5.datadoghq.com/api/v2/llm-obs/v1/experiments/delete |

## Push events for an LLM Observability experiment → preview{% #push-events-for-an-llm-observability-experiment %}

| Datadog site      | API endpoint                                                                            |
| ----------------- | --------------------------------------------------------------------------------------- |
| ap1.datadoghq.com | POST https://api.ap1.datadoghq.com/api/v2/llm-obs/v1/experiments/{experiment_id}/events |
| ap2.datadoghq.com | POST https://api.ap2.datadoghq.com/api/v2/llm-obs/v1/experiments/{experiment_id}/events |
| app.datadoghq.eu  | POST https://api.datadoghq.eu/api/v2/llm-obs/v1/experiments/{experiment_id}/events      |
| app.ddog-gov.com  | POST https://api.ddog-gov.com/api/v2/llm-obs/v1/experiments/{experiment_id}/events      |
| us2.ddog-gov.com  | POST https://api.us2.ddog-gov.com/api/v2/llm-obs/v1/experiments/{experiment_id}/events  |
| app.datadoghq.com | POST https://api.datadoghq.com/api/v2/llm-obs/v1/experiments/{experiment_id}/events     |
| us3.datadoghq.com | POST https://api.us3.datadoghq.com/api/v2/llm-obs/v1/experiments/{experiment_id}/events |
| us5.datadoghq.com | POST https://api.us5.datadoghq.com/api/v2/llm-obs/v1/experiments/{experiment_id}/events |

## List LLM Observability annotation queues → preview{% #list-llm-observability-annotation-queues %}

| Datadog site      | API endpoint                                                          |
| ----------------- | --------------------------------------------------------------------- |
| ap1.datadoghq.com | GET https://api.ap1.datadoghq.com/api/v2/llm-obs/v1/annotation-queues |
| ap2.datadoghq.com | GET https://api.ap2.datadoghq.com/api/v2/llm-obs/v1/annotation-queues |
| app.datadoghq.eu  | GET https://api.datadoghq.eu/api/v2/llm-obs/v1/annotation-queues      |
| app.ddog-gov.com  | GET https://api.ddog-gov.com/api/v2/llm-obs/v1/annotation-queues      |
| us2.ddog-gov.com  | GET https://api.us2.ddog-gov.com/api/v2/llm-obs/v1/annotation-queues  |
| app.datadoghq.com | GET https://api.datadoghq.com/api/v2/llm-obs/v1/annotation-queues     |
| us3.datadoghq.com | GET https://api.us3.datadoghq.com/api/v2/llm-obs/v1/annotation-queues |
| us5.datadoghq.com | GET https://api.us5.datadoghq.com/api/v2/llm-obs/v1/annotation-queues |

## Create an LLM Observability annotation queue → preview{% #create-an-llm-observability-annotation-queue %}

| Datadog site      | API endpoint                                                           |
| ----------------- | ---------------------------------------------------------------------- |
| ap1.datadoghq.com | POST https://api.ap1.datadoghq.com/api/v2/llm-obs/v1/annotation-queues |
| ap2.datadoghq.com | POST https://api.ap2.datadoghq.com/api/v2/llm-obs/v1/annotation-queues |
| app.datadoghq.eu  | POST https://api.datadoghq.eu/api/v2/llm-obs/v1/annotation-queues      |
| app.ddog-gov.com  | POST https://api.ddog-gov.com/api/v2/llm-obs/v1/annotation-queues      |
| us2.ddog-gov.com  | POST https://api.us2.ddog-gov.com/api/v2/llm-obs/v1/annotation-queues  |
| app.datadoghq.com | POST https://api.datadoghq.com/api/v2/llm-obs/v1/annotation-queues     |
| us3.datadoghq.com | POST https://api.us3.datadoghq.com/api/v2/llm-obs/v1/annotation-queues |
| us5.datadoghq.com | POST https://api.us5.datadoghq.com/api/v2/llm-obs/v1/annotation-queues |

## Update an LLM Observability annotation queue → preview{% #update-an-llm-observability-annotation-queue %}

| Datadog site      | API endpoint                                                                       |
| ----------------- | ---------------------------------------------------------------------------------- |
| ap1.datadoghq.com | PATCH https://api.ap1.datadoghq.com/api/v2/llm-obs/v1/annotation-queues/{queue_id} |
| ap2.datadoghq.com | PATCH https://api.ap2.datadoghq.com/api/v2/llm-obs/v1/annotation-queues/{queue_id} |
| app.datadoghq.eu  | PATCH https://api.datadoghq.eu/api/v2/llm-obs/v1/annotation-queues/{queue_id}      |
| app.ddog-gov.com  | PATCH https://api.ddog-gov.com/api/v2/llm-obs/v1/annotation-queues/{queue_id}      |
| us2.ddog-gov.com  | PATCH https://api.us2.ddog-gov.com/api/v2/llm-obs/v1/annotation-queues/{queue_id}  |
| app.datadoghq.com | PATCH https://api.datadoghq.com/api/v2/llm-obs/v1/annotation-queues/{queue_id}     |
| us3.datadoghq.com | PATCH https://api.us3.datadoghq.com/api/v2/llm-obs/v1/annotation-queues/{queue_id} |
| us5.datadoghq.com | PATCH https://api.us5.datadoghq.com/api/v2/llm-obs/v1/annotation-queues/{queue_id} |

## Delete an LLM Observability annotation queue → preview{% #delete-an-llm-observability-annotation-queue %}

| Datadog site      | API endpoint                                                                        |
| ----------------- | ----------------------------------------------------------------------------------- |
| ap1.datadoghq.com | DELETE https://api.ap1.datadoghq.com/api/v2/llm-obs/v1/annotation-queues/{queue_id} |
| ap2.datadoghq.com | DELETE https://api.ap2.datadoghq.com/api/v2/llm-obs/v1/annotation-queues/{queue_id} |
| app.datadoghq.eu  | DELETE https://api.datadoghq.eu/api/v2/llm-obs/v1/annotation-queues/{queue_id}      |
| app.ddog-gov.com  | DELETE https://api.ddog-gov.com/api/v2/llm-obs/v1/annotation-queues/{queue_id}      |
| us2.ddog-gov.com  | DELETE https://api.us2.ddog-gov.com/api/v2/llm-obs/v1/annotation-queues/{queue_id}  |
| app.datadoghq.com | DELETE https://api.datadoghq.com/api/v2/llm-obs/v1/annotation-queues/{queue_id}     |
| us3.datadoghq.com | DELETE https://api.us3.datadoghq.com/api/v2/llm-obs/v1/annotation-queues/{queue_id} |
| us5.datadoghq.com | DELETE https://api.us5.datadoghq.com/api/v2/llm-obs/v1/annotation-queues/{queue_id} |

## Add annotation queue interactions → preview{% #add-annotation-queue-interactions %}

| Datadog site      | API endpoint                                                                                   |
| ----------------- | ---------------------------------------------------------------------------------------------- |
| ap1.datadoghq.com | POST https://api.ap1.datadoghq.com/api/v2/llm-obs/v1/annotation-queues/{queue_id}/interactions |
| ap2.datadoghq.com | POST https://api.ap2.datadoghq.com/api/v2/llm-obs/v1/annotation-queues/{queue_id}/interactions |
| app.datadoghq.eu  | POST https://api.datadoghq.eu/api/v2/llm-obs/v1/annotation-queues/{queue_id}/interactions      |
| app.ddog-gov.com  | POST https://api.ddog-gov.com/api/v2/llm-obs/v1/annotation-queues/{queue_id}/interactions      |
| us2.ddog-gov.com  | POST https://api.us2.ddog-gov.com/api/v2/llm-obs/v1/annotation-queues/{queue_id}/interactions  |
| app.datadoghq.com | POST https://api.datadoghq.com/api/v2/llm-obs/v1/annotation-queues/{queue_id}/interactions     |
| us3.datadoghq.com | POST https://api.us3.datadoghq.com/api/v2/llm-obs/v1/annotation-queues/{queue_id}/interactions |
| us5.datadoghq.com | POST https://api.us5.datadoghq.com/api/v2/llm-obs/v1/annotation-queues/{queue_id}/interactions |

## Get annotated queue interactions → preview{% #get-annotated-queue-interactions %}

| Datadog site      | API endpoint                                                                                            |
| ----------------- | ------------------------------------------------------------------------------------------------------- |
| ap1.datadoghq.com | GET https://api.ap1.datadoghq.com/api/v2/llm-obs/v1/annotation-queues/{queue_id}/annotated-interactions |
| ap2.datadoghq.com | GET https://api.ap2.datadoghq.com/api/v2/llm-obs/v1/annotation-queues/{queue_id}/annotated-interactions |
| app.datadoghq.eu  | GET https://api.datadoghq.eu/api/v2/llm-obs/v1/annotation-queues/{queue_id}/annotated-interactions      |
| app.ddog-gov.com  | GET https://api.ddog-gov.com/api/v2/llm-obs/v1/annotation-queues/{queue_id}/annotated-interactions      |
| us2.ddog-gov.com  | GET https://api.us2.ddog-gov.com/api/v2/llm-obs/v1/annotation-queues/{queue_id}/annotated-interactions  |
| app.datadoghq.com | GET https://api.datadoghq.com/api/v2/llm-obs/v1/annotation-queues/{queue_id}/annotated-interactions     |
| us3.datadoghq.com | GET https://api.us3.datadoghq.com/api/v2/llm-obs/v1/annotation-queues/{queue_id}/annotated-interactions |
| us5.datadoghq.com | GET https://api.us5.datadoghq.com/api/v2/llm-obs/v1/annotation-queues/{queue_id}/annotated-interactions |

## Delete annotation queue interactions → preview{% #delete-annotation-queue-interactions %}

| Datadog site      | API endpoint                                                                                          |
| ----------------- | ----------------------------------------------------------------------------------------------------- |
| ap1.datadoghq.com | POST https://api.ap1.datadoghq.com/api/v2/llm-obs/v1/annotation-queues/{queue_id}/interactions/delete |
| ap2.datadoghq.com | POST https://api.ap2.datadoghq.com/api/v2/llm-obs/v1/annotation-queues/{queue_id}/interactions/delete |
| app.datadoghq.eu  | POST https://api.datadoghq.eu/api/v2/llm-obs/v1/annotation-queues/{queue_id}/interactions/delete      |
| app.ddog-gov.com  | POST https://api.ddog-gov.com/api/v2/llm-obs/v1/annotation-queues/{queue_id}/interactions/delete      |
| us2.ddog-gov.com  | POST https://api.us2.ddog-gov.com/api/v2/llm-obs/v1/annotation-queues/{queue_id}/interactions/delete  |
| app.datadoghq.com | POST https://api.datadoghq.com/api/v2/llm-obs/v1/annotation-queues/{queue_id}/interactions/delete     |
| us3.datadoghq.com | POST https://api.us3.datadoghq.com/api/v2/llm-obs/v1/annotation-queues/{queue_id}/interactions/delete |
| us5.datadoghq.com | POST https://api.us5.datadoghq.com/api/v2/llm-obs/v1/annotation-queues/{queue_id}/interactions/delete |

## Get a custom evaluator configuration → preview{% #get-a-custom-evaluator-configuration %}

| Datadog site      | API endpoint                                                                                |
| ----------------- | ------------------------------------------------------------------------------------------- |
| ap1.datadoghq.com | GET https://api.ap1.datadoghq.com/api/unstable/llm-obs/config/evaluators/custom/{eval_name} |
| ap2.datadoghq.com | GET https://api.ap2.datadoghq.com/api/unstable/llm-obs/config/evaluators/custom/{eval_name} |
| app.datadoghq.eu  | GET https://api.datadoghq.eu/api/unstable/llm-obs/config/evaluators/custom/{eval_name}      |
| app.ddog-gov.com  | GET https://api.ddog-gov.com/api/unstable/llm-obs/config/evaluators/custom/{eval_name}      |
| us2.ddog-gov.com  | GET https://api.us2.ddog-gov.com/api/unstable/llm-obs/config/evaluators/custom/{eval_name}  |
| app.datadoghq.com | GET https://api.datadoghq.com/api/unstable/llm-obs/config/evaluators/custom/{eval_name}     |
| us3.datadoghq.com | GET https://api.us3.datadoghq.com/api/unstable/llm-obs/config/evaluators/custom/{eval_name} |
| us5.datadoghq.com | GET https://api.us5.datadoghq.com/api/unstable/llm-obs/config/evaluators/custom/{eval_name} |

## Create or update a custom evaluator configuration → preview{% #create-or-update-a-custom-evaluator-configuration %}

| Datadog site      | API endpoint                                                                                |
| ----------------- | ------------------------------------------------------------------------------------------- |
| ap1.datadoghq.com | PUT https://api.ap1.datadoghq.com/api/unstable/llm-obs/config/evaluators/custom/{eval_name} |
| ap2.datadoghq.com | PUT https://api.ap2.datadoghq.com/api/unstable/llm-obs/config/evaluators/custom/{eval_name} |
| app.datadoghq.eu  | PUT https://api.datadoghq.eu/api/unstable/llm-obs/config/evaluators/custom/{eval_name}      |
| app.ddog-gov.com  | PUT https://api.ddog-gov.com/api/unstable/llm-obs/config/evaluators/custom/{eval_name}      |
| us2.ddog-gov.com  | PUT https://api.us2.ddog-gov.com/api/unstable/llm-obs/config/evaluators/custom/{eval_name}  |
| app.datadoghq.com | PUT https://api.datadoghq.com/api/unstable/llm-obs/config/evaluators/custom/{eval_name}     |
| us3.datadoghq.com | PUT https://api.us3.datadoghq.com/api/unstable/llm-obs/config/evaluators/custom/{eval_name} |
| us5.datadoghq.com | PUT https://api.us5.datadoghq.com/api/unstable/llm-obs/config/evaluators/custom/{eval_name} |

## Delete a custom evaluator configuration → preview{% #delete-a-custom-evaluator-configuration %}

| Datadog site      | API endpoint                                                                                   |
| ----------------- | ---------------------------------------------------------------------------------------------- |
| ap1.datadoghq.com | DELETE https://api.ap1.datadoghq.com/api/unstable/llm-obs/config/evaluators/custom/{eval_name} |
| ap2.datadoghq.com | DELETE https://api.ap2.datadoghq.com/api/unstable/llm-obs/config/evaluators/custom/{eval_name} |
| app.datadoghq.eu  | DELETE https://api.datadoghq.eu/api/unstable/llm-obs/config/evaluators/custom/{eval_name}      |
| app.ddog-gov.com  | DELETE https://api.ddog-gov.com/api/unstable/llm-obs/config/evaluators/custom/{eval_name}      |
| us2.ddog-gov.com  | DELETE https://api.us2.ddog-gov.com/api/unstable/llm-obs/config/evaluators/custom/{eval_name}  |
| app.datadoghq.com | DELETE https://api.datadoghq.com/api/unstable/llm-obs/config/evaluators/custom/{eval_name}     |
| us3.datadoghq.com | DELETE https://api.us3.datadoghq.com/api/unstable/llm-obs/config/evaluators/custom/{eval_name} |
| us5.datadoghq.com | DELETE https://api.us5.datadoghq.com/api/unstable/llm-obs/config/evaluators/custom/{eval_name} |

## Create or update annotations → preview{% #create-or-update-annotations %}

| Datadog site      | API endpoint                                                                                  |
| ----------------- | --------------------------------------------------------------------------------------------- |
| ap1.datadoghq.com | POST https://api.ap1.datadoghq.com/api/v2/llm-obs/v1/annotation-queues/{queue_id}/annotations |
| ap2.datadoghq.com | POST https://api.ap2.datadoghq.com/api/v2/llm-obs/v1/annotation-queues/{queue_id}/annotations |
| app.datadoghq.eu  | POST https://api.datadoghq.eu/api/v2/llm-obs/v1/annotation-queues/{queue_id}/annotations      |
| app.ddog-gov.com  | POST https://api.ddog-gov.com/api/v2/llm-obs/v1/annotation-queues/{queue_id}/annotations      |
| us2.ddog-gov.com  | POST https://api.us2.ddog-gov.com/api/v2/llm-obs/v1/annotation-queues/{queue_id}/annotations  |
| app.datadoghq.com | POST https://api.datadoghq.com/api/v2/llm-obs/v1/annotation-queues/{queue_id}/annotations     |
| us3.datadoghq.com | POST https://api.us3.datadoghq.com/api/v2/llm-obs/v1/annotation-queues/{queue_id}/annotations |
| us5.datadoghq.com | POST https://api.us5.datadoghq.com/api/v2/llm-obs/v1/annotation-queues/{queue_id}/annotations |

## Delete annotations → preview{% #delete-annotations %}

| Datadog site      | API endpoint                                                                                         |
| ----------------- | ---------------------------------------------------------------------------------------------------- |
| ap1.datadoghq.com | POST https://api.ap1.datadoghq.com/api/v2/llm-obs/v1/annotation-queues/{queue_id}/annotations/delete |
| ap2.datadoghq.com | POST https://api.ap2.datadoghq.com/api/v2/llm-obs/v1/annotation-queues/{queue_id}/annotations/delete |
| app.datadoghq.eu  | POST https://api.datadoghq.eu/api/v2/llm-obs/v1/annotation-queues/{queue_id}/annotations/delete      |
| app.ddog-gov.com  | POST https://api.ddog-gov.com/api/v2/llm-obs/v1/annotation-queues/{queue_id}/annotations/delete      |
| us2.ddog-gov.com  | POST https://api.us2.ddog-gov.com/api/v2/llm-obs/v1/annotation-queues/{queue_id}/annotations/delete  |
| app.datadoghq.com | POST https://api.datadoghq.com/api/v2/llm-obs/v1/annotation-queues/{queue_id}/annotations/delete     |
| us3.datadoghq.com | POST https://api.us3.datadoghq.com/api/v2/llm-obs/v1/annotation-queues/{queue_id}/annotations/delete |
| us5.datadoghq.com | POST https://api.us5.datadoghq.com/api/v2/llm-obs/v1/annotation-queues/{queue_id}/annotations/delete |

## List events for an LLM Observability experiment → preview{% #list-events-for-an-llm-observability-experiment %}

| Datadog site      | API endpoint                                                                           |
| ----------------- | -------------------------------------------------------------------------------------- |
| ap1.datadoghq.com | GET https://api.ap1.datadoghq.com/api/v2/llm-obs/v3/experiments/{experiment_id}/events |
| ap2.datadoghq.com | GET https://api.ap2.datadoghq.com/api/v2/llm-obs/v3/experiments/{experiment_id}/events |
| app.datadoghq.eu  | GET https://api.datadoghq.eu/api/v2/llm-obs/v3/experiments/{experiment_id}/events      |
| app.ddog-gov.com  | GET https://api.ddog-gov.com/api/v2/llm-obs/v3/experiments/{experiment_id}/events      |
| us2.ddog-gov.com  | GET https://api.us2.ddog-gov.com/api/v2/llm-obs/v3/experiments/{experiment_id}/events  |
| app.datadoghq.com | GET https://api.datadoghq.com/api/v2/llm-obs/v3/experiments/{experiment_id}/events     |
| us3.datadoghq.com | GET https://api.us3.datadoghq.com/api/v2/llm-obs/v3/experiments/{experiment_id}/events |
| us5.datadoghq.com | GET https://api.us5.datadoghq.com/api/v2/llm-obs/v3/experiments/{experiment_id}/events |

## Search LLM Observability experimentation entities → preview{% #search-llm-observability-experimentation-entities %}

| Datadog site      | API endpoint                                                                |
| ----------------- | --------------------------------------------------------------------------- |
| ap1.datadoghq.com | POST https://api.ap1.datadoghq.com/api/v2/llm-obs/v1/experimentation/search |
| ap2.datadoghq.com | POST https://api.ap2.datadoghq.com/api/v2/llm-obs/v1/experimentation/search |
| app.datadoghq.eu  | POST https://api.datadoghq.eu/api/v2/llm-obs/v1/experimentation/search      |
| app.ddog-gov.com  | POST https://api.ddog-gov.com/api/v2/llm-obs/v1/experimentation/search      |
| us2.ddog-gov.com  | POST https://api.us2.ddog-gov.com/api/v2/llm-obs/v1/experimentation/search  |
| app.datadoghq.com | POST https://api.datadoghq.com/api/v2/llm-obs/v1/experimentation/search     |
| us3.datadoghq.com | POST https://api.us3.datadoghq.com/api/v2/llm-obs/v1/experimentation/search |
| us5.datadoghq.com | POST https://api.us5.datadoghq.com/api/v2/llm-obs/v1/experimentation/search |

## Simple search experimentation entities → preview{% #simple-search-experimentation-entities %}

| Datadog site      | API endpoint                                                                       |
| ----------------- | ---------------------------------------------------------------------------------- |
| ap1.datadoghq.com | POST https://api.ap1.datadoghq.com/api/v2/llm-obs/v1/experimentation/simple-search |
| ap2.datadoghq.com | POST https://api.ap2.datadoghq.com/api/v2/llm-obs/v1/experimentation/simple-search |
| app.datadoghq.eu  | POST https://api.datadoghq.eu/api/v2/llm-obs/v1/experimentation/simple-search      |
| app.ddog-gov.com  | POST https://api.ddog-gov.com/api/v2/llm-obs/v1/experimentation/simple-search      |
| us2.ddog-gov.com  | POST https://api.us2.ddog-gov.com/api/v2/llm-obs/v1/experimentation/simple-search  |
| app.datadoghq.com | POST https://api.datadoghq.com/api/v2/llm-obs/v1/experimentation/simple-search     |
| us3.datadoghq.com | POST https://api.us3.datadoghq.com/api/v2/llm-obs/v1/experimentation/simple-search |
| us5.datadoghq.com | POST https://api.us5.datadoghq.com/api/v2/llm-obs/v1/experimentation/simple-search |

## Aggregate LLM Observability experimentation → preview{% #aggregate-llm-observability-experimentation %}

| Datadog site      | API endpoint                                                                   |
| ----------------- | ------------------------------------------------------------------------------ |
| ap1.datadoghq.com | POST https://api.ap1.datadoghq.com/api/v2/llm-obs/v1/experimentation/analytics |
| ap2.datadoghq.com | POST https://api.ap2.datadoghq.com/api/v2/llm-obs/v1/experimentation/analytics |
| app.datadoghq.eu  | POST https://api.datadoghq.eu/api/v2/llm-obs/v1/experimentation/analytics      |
| app.ddog-gov.com  | POST https://api.ddog-gov.com/api/v2/llm-obs/v1/experimentation/analytics      |
| us2.ddog-gov.com  | POST https://api.us2.ddog-gov.com/api/v2/llm-obs/v1/experimentation/analytics  |
| app.datadoghq.com | POST https://api.datadoghq.com/api/v2/llm-obs/v1/experimentation/analytics     |
| us3.datadoghq.com | POST https://api.us3.datadoghq.com/api/v2/llm-obs/v1/experimentation/analytics |
| us5.datadoghq.com | POST https://api.us5.datadoghq.com/api/v2/llm-obs/v1/experimentation/analytics |

## List LLM Observability spans → preview{% #list-llm-observability-spans %}

| Datadog site      | API endpoint                                                     |
| ----------------- | ---------------------------------------------------------------- |
| ap1.datadoghq.com | GET https://api.ap1.datadoghq.com/api/v2/llm-obs/v1/spans/events |
| ap2.datadoghq.com | GET https://api.ap2.datadoghq.com/api/v2/llm-obs/v1/spans/events |
| app.datadoghq.eu  | GET https://api.datadoghq.eu/api/v2/llm-obs/v1/spans/events      |
| app.ddog-gov.com  | GET https://api.ddog-gov.com/api/v2/llm-obs/v1/spans/events      |
| us2.ddog-gov.com  | GET https://api.us2.ddog-gov.com/api/v2/llm-obs/v1/spans/events  |
| app.datadoghq.com | GET https://api.datadoghq.com/api/v2/llm-obs/v1/spans/events     |
| us3.datadoghq.com | GET https://api.us3.datadoghq.com/api/v2/llm-obs/v1/spans/events |
| us5.datadoghq.com | GET https://api.us5.datadoghq.com/api/v2/llm-obs/v1/spans/events |

## Search LLM Observability spans → preview{% #search-llm-observability-spans %}

| Datadog site      | API endpoint                                                             |
| ----------------- | ------------------------------------------------------------------------ |
| ap1.datadoghq.com | POST https://api.ap1.datadoghq.com/api/v2/llm-obs/v1/spans/events/search |
| ap2.datadoghq.com | POST https://api.ap2.datadoghq.com/api/v2/llm-obs/v1/spans/events/search |
| app.datadoghq.eu  | POST https://api.datadoghq.eu/api/v2/llm-obs/v1/spans/events/search      |
| app.ddog-gov.com  | POST https://api.ddog-gov.com/api/v2/llm-obs/v1/spans/events/search      |
| us2.ddog-gov.com  | POST https://api.us2.ddog-gov.com/api/v2/llm-obs/v1/spans/events/search  |
| app.datadoghq.com | POST https://api.datadoghq.com/api/v2/llm-obs/v1/spans/events/search     |
| us3.datadoghq.com | POST https://api.us3.datadoghq.com/api/v2/llm-obs/v1/spans/events/search |
| us5.datadoghq.com | POST https://api.us5.datadoghq.com/api/v2/llm-obs/v1/spans/events/search |

## Delete LLM Observability data → preview{% #delete-llm-observability-data %}

| Datadog site      | API endpoint                                                           |
| ----------------- | ---------------------------------------------------------------------- |
| ap1.datadoghq.com | POST https://api.ap1.datadoghq.com/api/v2/llm-obs/deletion/data/llmobs |
| ap2.datadoghq.com | POST https://api.ap2.datadoghq.com/api/v2/llm-obs/deletion/data/llmobs |
| app.datadoghq.eu  | POST https://api.datadoghq.eu/api/v2/llm-obs/deletion/data/llmobs      |
| app.ddog-gov.com  | POST https://api.ddog-gov.com/api/v2/llm-obs/deletion/data/llmobs      |
| us2.ddog-gov.com  | POST https://api.us2.ddog-gov.com/api/v2/llm-obs/deletion/data/llmobs  |
| app.datadoghq.com | POST https://api.datadoghq.com/api/v2/llm-obs/deletion/data/llmobs     |
| us3.datadoghq.com | POST https://api.us3.datadoghq.com/api/v2/llm-obs/deletion/data/llmobs |
| us5.datadoghq.com | POST https://api.us5.datadoghq.com/api/v2/llm-obs/deletion/data/llmobs |

## List LLM integration accounts → preview{% #list-llm-integration-accounts %}

| Datadog site      | API endpoint                                                                            |
| ----------------- | --------------------------------------------------------------------------------------- |
| ap1.datadoghq.com | GET https://api.ap1.datadoghq.com/api/v2/llm-obs/v1/integrations/{integration}/accounts |
| ap2.datadoghq.com | GET https://api.ap2.datadoghq.com/api/v2/llm-obs/v1/integrations/{integration}/accounts |
| app.datadoghq.eu  | GET https://api.datadoghq.eu/api/v2/llm-obs/v1/integrations/{integration}/accounts      |
| app.ddog-gov.com  | GET https://api.ddog-gov.com/api/v2/llm-obs/v1/integrations/{integration}/accounts      |
| us2.ddog-gov.com  | GET https://api.us2.ddog-gov.com/api/v2/llm-obs/v1/integrations/{integration}/accounts  |
| app.datadoghq.com | GET https://api.datadoghq.com/api/v2/llm-obs/v1/integrations/{integration}/accounts     |
| us3.datadoghq.com | GET https://api.us3.datadoghq.com/api/v2/llm-obs/v1/integrations/{integration}/accounts |
| us5.datadoghq.com | GET https://api.us5.datadoghq.com/api/v2/llm-obs/v1/integrations/{integration}/accounts |

## List LLM integration models → preview{% #list-llm-integration-models %}

| Datadog site      | API endpoint                                                                                       |
| ----------------- | -------------------------------------------------------------------------------------------------- |
| ap1.datadoghq.com | GET https://api.ap1.datadoghq.com/api/v2/llm-obs/v1/integrations/{integration}/{account_id}/models |
| ap2.datadoghq.com | GET https://api.ap2.datadoghq.com/api/v2/llm-obs/v1/integrations/{integration}/{account_id}/models |
| app.datadoghq.eu  | GET https://api.datadoghq.eu/api/v2/llm-obs/v1/integrations/{integration}/{account_id}/models      |
| app.ddog-gov.com  | GET https://api.ddog-gov.com/api/v2/llm-obs/v1/integrations/{integration}/{account_id}/models      |
| us2.ddog-gov.com  | GET https://api.us2.ddog-gov.com/api/v2/llm-obs/v1/integrations/{integration}/{account_id}/models  |
| app.datadoghq.com | GET https://api.datadoghq.com/api/v2/llm-obs/v1/integrations/{integration}/{account_id}/models     |
| us3.datadoghq.com | GET https://api.us3.datadoghq.com/api/v2/llm-obs/v1/integrations/{integration}/{account_id}/models |
| us5.datadoghq.com | GET https://api.us5.datadoghq.com/api/v2/llm-obs/v1/integrations/{integration}/{account_id}/models |

## Run an LLM inference → preview{% #run-an-llm-inference %}

| Datadog site      | API endpoint                                                                                           |
| ----------------- | ------------------------------------------------------------------------------------------------------ |
| ap1.datadoghq.com | POST https://api.ap1.datadoghq.com/api/v2/llm-obs/v1/integrations/{integration}/{account_id}/inference |
| ap2.datadoghq.com | POST https://api.ap2.datadoghq.com/api/v2/llm-obs/v1/integrations/{integration}/{account_id}/inference |
| app.datadoghq.eu  | POST https://api.datadoghq.eu/api/v2/llm-obs/v1/integrations/{integration}/{account_id}/inference      |
| app.ddog-gov.com  | POST https://api.ddog-gov.com/api/v2/llm-obs/v1/integrations/{integration}/{account_id}/inference      |
| us2.ddog-gov.com  | POST https://api.us2.ddog-gov.com/api/v2/llm-obs/v1/integrations/{integration}/{account_id}/inference  |
| app.datadoghq.com | POST https://api.datadoghq.com/api/v2/llm-obs/v1/integrations/{integration}/{account_id}/inference     |
| us3.datadoghq.com | POST https://api.us3.datadoghq.com/api/v2/llm-obs/v1/integrations/{integration}/{account_id}/inference |
| us5.datadoghq.com | POST https://api.us5.datadoghq.com/api/v2/llm-obs/v1/integrations/{integration}/{account_id}/inference |

## Get annotated interactions by content IDs → preview{% #get-annotated-interactions-by-content-ids %}

| Datadog site      | API endpoint                                                               |
| ----------------- | -------------------------------------------------------------------------- |
| ap1.datadoghq.com | GET https://api.ap1.datadoghq.com/api/v2/llm-obs/v1/annotated-interactions |
| ap2.datadoghq.com | GET https://api.ap2.datadoghq.com/api/v2/llm-obs/v1/annotated-interactions |
| app.datadoghq.eu  | GET https://api.datadoghq.eu/api/v2/llm-obs/v1/annotated-interactions      |
| app.ddog-gov.com  | GET https://api.ddog-gov.com/api/v2/llm-obs/v1/annotated-interactions      |
| us2.ddog-gov.com  | GET https://api.us2.ddog-gov.com/api/v2/llm-obs/v1/annotated-interactions  |
| app.datadoghq.com | GET https://api.datadoghq.com/api/v2/llm-obs/v1/annotated-interactions     |
| us3.datadoghq.com | GET https://api.us3.datadoghq.com/api/v2/llm-obs/v1/annotated-interactions |
| us5.datadoghq.com | GET https://api.us5.datadoghq.com/api/v2/llm-obs/v1/annotated-interactions |

## Get LLM Observability dataset draft state → preview{% #get-llm-observability-dataset-draft-state %}

| Datadog site      | API endpoint                                                                                       |
| ----------------- | -------------------------------------------------------------------------------------------------- |
| ap1.datadoghq.com | GET https://api.ap1.datadoghq.com/api/v2/llm-obs/v1/{project_id}/datasets/{dataset_id}/draft_state |
| ap2.datadoghq.com | GET https://api.ap2.datadoghq.com/api/v2/llm-obs/v1/{project_id}/datasets/{dataset_id}/draft_state |
| app.datadoghq.eu  | GET https://api.datadoghq.eu/api/v2/llm-obs/v1/{project_id}/datasets/{dataset_id}/draft_state      |
| app.ddog-gov.com  | GET https://api.ddog-gov.com/api/v2/llm-obs/v1/{project_id}/datasets/{dataset_id}/draft_state      |
| us2.ddog-gov.com  | GET https://api.us2.ddog-gov.com/api/v2/llm-obs/v1/{project_id}/datasets/{dataset_id}/draft_state  |
| app.datadoghq.com | GET https://api.datadoghq.com/api/v2/llm-obs/v1/{project_id}/datasets/{dataset_id}/draft_state     |
| us3.datadoghq.com | GET https://api.us3.datadoghq.com/api/v2/llm-obs/v1/{project_id}/datasets/{dataset_id}/draft_state |
| us5.datadoghq.com | GET https://api.us5.datadoghq.com/api/v2/llm-obs/v1/{project_id}/datasets/{dataset_id}/draft_state |

## Lock LLM Observability dataset draft state → preview{% #lock-llm-observability-dataset-draft-state %}

| Datadog site      | API endpoint                                                                                              |
| ----------------- | --------------------------------------------------------------------------------------------------------- |
| ap1.datadoghq.com | PATCH https://api.ap1.datadoghq.com/api/v2/llm-obs/v1/{project_id}/datasets/{dataset_id}/draft_state/lock |
| ap2.datadoghq.com | PATCH https://api.ap2.datadoghq.com/api/v2/llm-obs/v1/{project_id}/datasets/{dataset_id}/draft_state/lock |
| app.datadoghq.eu  | PATCH https://api.datadoghq.eu/api/v2/llm-obs/v1/{project_id}/datasets/{dataset_id}/draft_state/lock      |
| app.ddog-gov.com  | PATCH https://api.ddog-gov.com/api/v2/llm-obs/v1/{project_id}/datasets/{dataset_id}/draft_state/lock      |
| us2.ddog-gov.com  | PATCH https://api.us2.ddog-gov.com/api/v2/llm-obs/v1/{project_id}/datasets/{dataset_id}/draft_state/lock  |
| app.datadoghq.com | PATCH https://api.datadoghq.com/api/v2/llm-obs/v1/{project_id}/datasets/{dataset_id}/draft_state/lock     |
| us3.datadoghq.com | PATCH https://api.us3.datadoghq.com/api/v2/llm-obs/v1/{project_id}/datasets/{dataset_id}/draft_state/lock |
| us5.datadoghq.com | PATCH https://api.us5.datadoghq.com/api/v2/llm-obs/v1/{project_id}/datasets/{dataset_id}/draft_state/lock |

## Unlock LLM Observability dataset draft state → preview{% #unlock-llm-observability-dataset-draft-state %}

| Datadog site      | API endpoint                                                                                                |
| ----------------- | ----------------------------------------------------------------------------------------------------------- |
| ap1.datadoghq.com | PATCH https://api.ap1.datadoghq.com/api/v2/llm-obs/v1/{project_id}/datasets/{dataset_id}/draft_state/unlock |
| ap2.datadoghq.com | PATCH https://api.ap2.datadoghq.com/api/v2/llm-obs/v1/{project_id}/datasets/{dataset_id}/draft_state/unlock |
| app.datadoghq.eu  | PATCH https://api.datadoghq.eu/api/v2/llm-obs/v1/{project_id}/datasets/{dataset_id}/draft_state/unlock      |
| app.ddog-gov.com  | PATCH https://api.ddog-gov.com/api/v2/llm-obs/v1/{project_id}/datasets/{dataset_id}/draft_state/unlock      |
| us2.ddog-gov.com  | PATCH https://api.us2.ddog-gov.com/api/v2/llm-obs/v1/{project_id}/datasets/{dataset_id}/draft_state/unlock  |
| app.datadoghq.com | PATCH https://api.datadoghq.com/api/v2/llm-obs/v1/{project_id}/datasets/{dataset_id}/draft_state/unlock     |
| us3.datadoghq.com | PATCH https://api.us3.datadoghq.com/api/v2/llm-obs/v1/{project_id}/datasets/{dataset_id}/draft_state/unlock |
| us5.datadoghq.com | PATCH https://api.us5.datadoghq.com/api/v2/llm-obs/v1/{project_id}/datasets/{dataset_id}/draft_state/unlock |

## List LLM Observability dataset versions → preview{% #list-llm-observability-dataset-versions %}

| Datadog site      | API endpoint                                                                                    |
| ----------------- | ----------------------------------------------------------------------------------------------- |
| ap1.datadoghq.com | GET https://api.ap1.datadoghq.com/api/v2/llm-obs/v1/{project_id}/datasets/{dataset_id}/versions |
| ap2.datadoghq.com | GET https://api.ap2.datadoghq.com/api/v2/llm-obs/v1/{project_id}/datasets/{dataset_id}/versions |
| app.datadoghq.eu  | GET https://api.datadoghq.eu/api/v2/llm-obs/v1/{project_id}/datasets/{dataset_id}/versions      |
| app.ddog-gov.com  | GET https://api.ddog-gov.com/api/v2/llm-obs/v1/{project_id}/datasets/{dataset_id}/versions      |
| us2.ddog-gov.com  | GET https://api.us2.ddog-gov.com/api/v2/llm-obs/v1/{project_id}/datasets/{dataset_id}/versions  |
| app.datadoghq.com | GET https://api.datadoghq.com/api/v2/llm-obs/v1/{project_id}/datasets/{dataset_id}/versions     |
| us3.datadoghq.com | GET https://api.us3.datadoghq.com/api/v2/llm-obs/v1/{project_id}/datasets/{dataset_id}/versions |
| us5.datadoghq.com | GET https://api.us5.datadoghq.com/api/v2/llm-obs/v1/{project_id}/datasets/{dataset_id}/versions |

## Export an LLM Observability dataset → preview{% #export-an-llm-observability-dataset %}

| Datadog site      | API endpoint                                                                                  |
| ----------------- | --------------------------------------------------------------------------------------------- |
| ap1.datadoghq.com | GET https://api.ap1.datadoghq.com/api/v2/llm-obs/v1/{project_id}/datasets/{dataset_id}/export |
| ap2.datadoghq.com | GET https://api.ap2.datadoghq.com/api/v2/llm-obs/v1/{project_id}/datasets/{dataset_id}/export |
| app.datadoghq.eu  | GET https://api.datadoghq.eu/api/v2/llm-obs/v1/{project_id}/datasets/{dataset_id}/export      |
| app.ddog-gov.com  | GET https://api.ddog-gov.com/api/v2/llm-obs/v1/{project_id}/datasets/{dataset_id}/export      |
| us2.ddog-gov.com  | GET https://api.us2.ddog-gov.com/api/v2/llm-obs/v1/{project_id}/datasets/{dataset_id}/export  |
| app.datadoghq.com | GET https://api.datadoghq.com/api/v2/llm-obs/v1/{project_id}/datasets/{dataset_id}/export     |
| us3.datadoghq.com | GET https://api.us3.datadoghq.com/api/v2/llm-obs/v1/{project_id}/datasets/{dataset_id}/export |
| us5.datadoghq.com | GET https://api.us5.datadoghq.com/api/v2/llm-obs/v1/{project_id}/datasets/{dataset_id}/export |

## Batch update LLM Observability dataset records → preview{% #batch-update-llm-observability-dataset-records %}

| Datadog site      | API endpoint                                                                                         |
| ----------------- | ---------------------------------------------------------------------------------------------------- |
| ap1.datadoghq.com | POST https://api.ap1.datadoghq.com/api/v2/llm-obs/v1/{project_id}/datasets/{dataset_id}/batch_update |
| ap2.datadoghq.com | POST https://api.ap2.datadoghq.com/api/v2/llm-obs/v1/{project_id}/datasets/{dataset_id}/batch_update |
| app.datadoghq.eu  | POST https://api.datadoghq.eu/api/v2/llm-obs/v1/{project_id}/datasets/{dataset_id}/batch_update      |
| app.ddog-gov.com  | POST https://api.ddog-gov.com/api/v2/llm-obs/v1/{project_id}/datasets/{dataset_id}/batch_update      |
| us2.ddog-gov.com  | POST https://api.us2.ddog-gov.com/api/v2/llm-obs/v1/{project_id}/datasets/{dataset_id}/batch_update  |
| app.datadoghq.com | POST https://api.datadoghq.com/api/v2/llm-obs/v1/{project_id}/datasets/{dataset_id}/batch_update     |
| us3.datadoghq.com | POST https://api.us3.datadoghq.com/api/v2/llm-obs/v1/{project_id}/datasets/{dataset_id}/batch_update |
| us5.datadoghq.com | POST https://api.us5.datadoghq.com/api/v2/llm-obs/v1/{project_id}/datasets/{dataset_id}/batch_update |

## Clone an LLM Observability dataset → preview{% #clone-an-llm-observability-dataset %}

| Datadog site      | API endpoint                                                                                  |
| ----------------- | --------------------------------------------------------------------------------------------- |
| ap1.datadoghq.com | POST https://api.ap1.datadoghq.com/api/v2/llm-obs/v1/{project_id}/datasets/{dataset_id}/clone |
| ap2.datadoghq.com | POST https://api.ap2.datadoghq.com/api/v2/llm-obs/v1/{project_id}/datasets/{dataset_id}/clone |
| app.datadoghq.eu  | POST https://api.datadoghq.eu/api/v2/llm-obs/v1/{project_id}/datasets/{dataset_id}/clone      |
| app.ddog-gov.com  | POST https://api.ddog-gov.com/api/v2/llm-obs/v1/{project_id}/datasets/{dataset_id}/clone      |
| us2.ddog-gov.com  | POST https://api.us2.ddog-gov.com/api/v2/llm-obs/v1/{project_id}/datasets/{dataset_id}/clone  |
| app.datadoghq.com | POST https://api.datadoghq.com/api/v2/llm-obs/v1/{project_id}/datasets/{dataset_id}/clone     |
| us3.datadoghq.com | POST https://api.us3.datadoghq.com/api/v2/llm-obs/v1/{project_id}/datasets/{dataset_id}/clone |
| us5.datadoghq.com | POST https://api.us5.datadoghq.com/api/v2/llm-obs/v1/{project_id}/datasets/{dataset_id}/clone |

## Restore an LLM Observability dataset version → preview{% #restore-an-llm-observability-dataset-version %}

| Datadog site      | API endpoint                                                                                    |
| ----------------- | ----------------------------------------------------------------------------------------------- |
| ap1.datadoghq.com | POST https://api.ap1.datadoghq.com/api/v2/llm-obs/v1/{project_id}/datasets/{dataset_id}/restore |
| ap2.datadoghq.com | POST https://api.ap2.datadoghq.com/api/v2/llm-obs/v1/{project_id}/datasets/{dataset_id}/restore |
| app.datadoghq.eu  | POST https://api.datadoghq.eu/api/v2/llm-obs/v1/{project_id}/datasets/{dataset_id}/restore      |
| app.ddog-gov.com  | POST https://api.ddog-gov.com/api/v2/llm-obs/v1/{project_id}/datasets/{dataset_id}/restore      |
| us2.ddog-gov.com  | POST https://api.us2.ddog-gov.com/api/v2/llm-obs/v1/{project_id}/datasets/{dataset_id}/restore  |
| app.datadoghq.com | POST https://api.datadoghq.com/api/v2/llm-obs/v1/{project_id}/datasets/{dataset_id}/restore     |
| us3.datadoghq.com | POST https://api.us3.datadoghq.com/api/v2/llm-obs/v1/{project_id}/datasets/{dataset_id}/restore |
| us5.datadoghq.com | POST https://api.us5.datadoghq.com/api/v2/llm-obs/v1/{project_id}/datasets/{dataset_id}/restore |

## Upload records to an LLM Observability dataset → preview{% #upload-records-to-an-llm-observability-dataset %}

| Datadog site      | API endpoint                                                                                           |
| ----------------- | ------------------------------------------------------------------------------------------------------ |
| ap1.datadoghq.com | POST https://api.ap1.datadoghq.com/api/v2/llm-obs/v2/{project_id}/datasets/{dataset_id}/records/upload |
| ap2.datadoghq.com | POST https://api.ap2.datadoghq.com/api/v2/llm-obs/v2/{project_id}/datasets/{dataset_id}/records/upload |
| app.datadoghq.eu  | POST https://api.datadoghq.eu/api/v2/llm-obs/v2/{project_id}/datasets/{dataset_id}/records/upload      |
| app.ddog-gov.com  | POST https://api.ddog-gov.com/api/v2/llm-obs/v2/{project_id}/datasets/{dataset_id}/records/upload      |
| us2.ddog-gov.com  | POST https://api.us2.ddog-gov.com/api/v2/llm-obs/v2/{project_id}/datasets/{dataset_id}/records/upload  |
| app.datadoghq.com | POST https://api.datadoghq.com/api/v2/llm-obs/v2/{project_id}/datasets/{dataset_id}/records/upload     |
| us3.datadoghq.com | POST https://api.us3.datadoghq.com/api/v2/llm-obs/v2/{project_id}/datasets/{dataset_id}/records/upload |
| us5.datadoghq.com | POST https://api.us5.datadoghq.com/api/v2/llm-obs/v2/{project_id}/datasets/{dataset_id}/records/upload |

## List LLM Observability experiment spans (v1) → deprecated preview{% #list-llm-observability-experiment-spans-v1 %}

| Datadog site      | API endpoint                                                                           |
| ----------------- | -------------------------------------------------------------------------------------- |
| ap1.datadoghq.com | GET https://api.ap1.datadoghq.com/api/v2/llm-obs/v1/experiments/{experiment_id}/events |
| ap2.datadoghq.com | GET https://api.ap2.datadoghq.com/api/v2/llm-obs/v1/experiments/{experiment_id}/events |
| app.datadoghq.eu  | GET https://api.datadoghq.eu/api/v2/llm-obs/v1/experiments/{experiment_id}/events      |
| app.ddog-gov.com  | GET https://api.ddog-gov.com/api/v2/llm-obs/v1/experiments/{experiment_id}/events      |
| us2.ddog-gov.com  | GET https://api.us2.ddog-gov.com/api/v2/llm-obs/v1/experiments/{experiment_id}/events  |
| app.datadoghq.com | GET https://api.datadoghq.com/api/v2/llm-obs/v1/experiments/{experiment_id}/events     |
| us3.datadoghq.com | GET https://api.us3.datadoghq.com/api/v2/llm-obs/v1/experiments/{experiment_id}/events |
| us5.datadoghq.com | GET https://api.us5.datadoghq.com/api/v2/llm-obs/v1/experiments/{experiment_id}/events |

## List LLM Observability experiment events (v2) → deprecated preview{% #list-llm-observability-experiment-events-v2 %}

| Datadog site      | API endpoint                                                                           |
| ----------------- | -------------------------------------------------------------------------------------- |
| ap1.datadoghq.com | GET https://api.ap1.datadoghq.com/api/v2/llm-obs/v2/experiments/{experiment_id}/events |
| ap2.datadoghq.com | GET https://api.ap2.datadoghq.com/api/v2/llm-obs/v2/experiments/{experiment_id}/events |
| app.datadoghq.eu  | GET https://api.datadoghq.eu/api/v2/llm-obs/v2/experiments/{experiment_id}/events      |
| app.ddog-gov.com  | GET https://api.ddog-gov.com/api/v2/llm-obs/v2/experiments/{experiment_id}/events      |
| us2.ddog-gov.com  | GET https://api.us2.ddog-gov.com/api/v2/llm-obs/v2/experiments/{experiment_id}/events  |
| app.datadoghq.com | GET https://api.datadoghq.com/api/v2/llm-obs/v2/experiments/{experiment_id}/events     |
| us3.datadoghq.com | GET https://api.us3.datadoghq.com/api/v2/llm-obs/v2/experiments/{experiment_id}/events |
| us5.datadoghq.com | GET https://api.us5.datadoghq.com/api/v2/llm-obs/v2/experiments/{experiment_id}/events |

## Get annotation queue label schema → preview{% #get-annotation-queue-label-schema %}

| Datadog site      | API endpoint                                                                                  |
| ----------------- | --------------------------------------------------------------------------------------------- |
| ap1.datadoghq.com | GET https://api.ap1.datadoghq.com/api/v2/llm-obs/v1/annotation-queues/{queue_id}/label-schema |
| ap2.datadoghq.com | GET https://api.ap2.datadoghq.com/api/v2/llm-obs/v1/annotation-queues/{queue_id}/label-schema |
| app.datadoghq.eu  | GET https://api.datadoghq.eu/api/v2/llm-obs/v1/annotation-queues/{queue_id}/label-schema      |
| app.ddog-gov.com  | GET https://api.ddog-gov.com/api/v2/llm-obs/v1/annotation-queues/{queue_id}/label-schema      |
| us2.ddog-gov.com  | GET https://api.us2.ddog-gov.com/api/v2/llm-obs/v1/annotation-queues/{queue_id}/label-schema  |
| app.datadoghq.com | GET https://api.datadoghq.com/api/v2/llm-obs/v1/annotation-queues/{queue_id}/label-schema     |
| us3.datadoghq.com | GET https://api.us3.datadoghq.com/api/v2/llm-obs/v1/annotation-queues/{queue_id}/label-schema |
| us5.datadoghq.com | GET https://api.us5.datadoghq.com/api/v2/llm-obs/v1/annotation-queues/{queue_id}/label-schema |

## Update annotation queue label schema → preview{% #update-annotation-queue-label-schema %}

| Datadog site      | API endpoint                                                                                  |
| ----------------- | --------------------------------------------------------------------------------------------- |
| ap1.datadoghq.com | PUT https://api.ap1.datadoghq.com/api/v2/llm-obs/v1/annotation-queues/{queue_id}/label-schema |
| ap2.datadoghq.com | PUT https://api.ap2.datadoghq.com/api/v2/llm-obs/v1/annotation-queues/{queue_id}/label-schema |
| app.datadoghq.eu  | PUT https://api.datadoghq.eu/api/v2/llm-obs/v1/annotation-queues/{queue_id}/label-schema      |
| app.ddog-gov.com  | PUT https://api.ddog-gov.com/api/v2/llm-obs/v1/annotation-queues/{queue_id}/label-schema      |
| us2.ddog-gov.com  | PUT https://api.us2.ddog-gov.com/api/v2/llm-obs/v1/annotation-queues/{queue_id}/label-schema  |
| app.datadoghq.com | PUT https://api.datadoghq.com/api/v2/llm-obs/v1/annotation-queues/{queue_id}/label-schema     |
| us3.datadoghq.com | PUT https://api.us3.datadoghq.com/api/v2/llm-obs/v1/annotation-queues/{queue_id}/label-schema |
| us5.datadoghq.com | PUT https://api.us5.datadoghq.com/api/v2/llm-obs/v1/annotation-queues/{queue_id}/label-schema |