--- title: LLM Observability description: Datadog, the leading service for cloud-scale monitoring. breadcrumbs: Docs > API Reference > LLM Observability --- # LLM Observability Manage LLM Observability projects, datasets, dataset records, experiments, and annotations. ## List LLM Observability projects{% #list-llm-observability-projects %} {% tab title="v2" %} **Note**: This endpoint is in preview and is subject to change. If you have any feedback, contact [Datadog support](https://docs.datadoghq.com/help/). | Datadog site | API endpoint | | ----------------- | ------------------------------------------------------------ | | ap1.datadoghq.com | GET https://api.ap1.datadoghq.com/api/v2/llm-obs/v1/projects | | ap2.datadoghq.com | GET https://api.ap2.datadoghq.com/api/v2/llm-obs/v1/projects | | app.datadoghq.eu | GET https://api.datadoghq.eu/api/v2/llm-obs/v1/projects | | app.ddog-gov.com | GET https://api.ddog-gov.com/api/v2/llm-obs/v1/projects | | app.datadoghq.com | GET https://api.datadoghq.com/api/v2/llm-obs/v1/projects | | us3.datadoghq.com | GET https://api.us3.datadoghq.com/api/v2/llm-obs/v1/projects | | us5.datadoghq.com | GET https://api.us5.datadoghq.com/api/v2/llm-obs/v1/projects | ### Overview List all LLM Observability projects sorted by creation date, newest first. ### Arguments #### Query Strings | Name | Type | Description | | ------------ | ------- | --------------------------------------------------------------- | | filter[id] | string | Filter projects by project ID. | | filter[name] | string | Filter projects by name. | | page[cursor] | string | Use the Pagination cursor to retrieve the next page of results. | | page[limit] | integer | Maximum number of results to return per page. | ### Response {% tab title="200" %} OK {% tab title="Model" %} Response containing a list of LLM Observability projects. | Parent field | Field | Type | Description | | ------------ | ----------------------------- | --------- | ------------------------------------------------------------------------------ | | | data [*required*] | [object] | List of projects. | | data | attributes [*required*] | object | Attributes of an LLM Observability project. | | attributes | created_at [*required*] | date-time | Timestamp when the project was created. | | attributes | description [*required*] | string | Description of the project. | | attributes | name [*required*] | string | Name of the project. | | attributes | updated_at [*required*] | date-time | Timestamp when the project was last updated. | | data | id [*required*] | string | Unique identifier of the project. | | data | type [*required*] | enum | Resource type of an LLM Observability project. Allowed enum values: `projects` | | | meta | object | Pagination cursor metadata. | | meta | after | string | Cursor for the next page of results. | {% /tab %} {% tab title="Example" %} ```json { "data": [ { "attributes": { "created_at": "2024-01-15T10:30:00Z", "description": "", "name": "My LLM Project", "updated_at": "2024-01-15T10:30:00Z" }, "id": "a33671aa-24fd-4dcd-9b33-a8ec7dde7751", "type": "projects" } ], "meta": { "after": "string" } } ``` {% /tab %} {% /tab %} {% tab title="400" %} Bad Request {% tab title="Model" %} API error response. | Parent field | Field | Type | Description | | ------------ | ------------------------ | -------- | ------------------------------------------------------------------------------- | | | errors [*required*] | [object] | A list of errors. | | errors | detail | string | A human-readable explanation specific to this occurrence of the error. | | errors | meta | object | Non-standard meta-information about the error | | errors | source | object | References to the source of the error. | | source | header | string | A string indicating the name of a single request header which caused the error. | | source | parameter | string | A string indicating which URI query parameter caused the error. | | source | pointer | string | A JSON pointer to the value in the request document that caused the error. | | errors | status | string | Status code of the response. | | errors | title | string | Short human-readable summary of the error. | {% /tab %} {% tab title="Example" %} ```json { "errors": [ { "detail": "Missing required attribute in body", "meta": {}, "source": { "header": "Authorization", "parameter": "limit", "pointer": "/data/attributes/title" }, "status": "400", "title": "Bad Request" } ] } ``` {% /tab %} {% /tab %} {% tab title="401" %} Unauthorized {% tab title="Model" %} API error response. | Parent field | Field | Type | Description | | ------------ | ------------------------ | -------- | ------------------------------------------------------------------------------- | | | errors [*required*] | [object] | A list of errors. | | errors | detail | string | A human-readable explanation specific to this occurrence of the error. | | errors | meta | object | Non-standard meta-information about the error | | errors | source | object | References to the source of the error. | | source | header | string | A string indicating the name of a single request header which caused the error. | | source | parameter | string | A string indicating which URI query parameter caused the error. | | source | pointer | string | A JSON pointer to the value in the request document that caused the error. | | errors | status | string | Status code of the response. | | errors | title | string | Short human-readable summary of the error. | {% /tab %} {% tab title="Example" %} ```json { "errors": [ { "detail": "Missing required attribute in body", "meta": {}, "source": { "header": "Authorization", "parameter": "limit", "pointer": "/data/attributes/title" }, "status": "400", "title": "Bad Request" } ] } ``` {% /tab %} {% /tab %} {% tab title="403" %} Forbidden {% tab title="Model" %} API error response. | Parent field | Field | Type | Description | | ------------ | ------------------------ | -------- | ------------------------------------------------------------------------------- | | | errors [*required*] | [object] | A list of errors. | | errors | detail | string | A human-readable explanation specific to this occurrence of the error. | | errors | meta | object | Non-standard meta-information about the error | | errors | source | object | References to the source of the error. | | source | header | string | A string indicating the name of a single request header which caused the error. | | source | parameter | string | A string indicating which URI query parameter caused the error. | | source | pointer | string | A JSON pointer to the value in the request document that caused the error. | | errors | status | string | Status code of the response. | | errors | title | string | Short human-readable summary of the error. | {% /tab %} {% tab title="Example" %} ```json { "errors": [ { "detail": "Missing required attribute in body", "meta": {}, "source": { "header": "Authorization", "parameter": "limit", "pointer": "/data/attributes/title" }, "status": "400", "title": "Bad Request" } ] } ``` {% /tab %} {% /tab %} {% tab title="429" %} Too many requests {% tab title="Model" %} API error response. | Field | Type | Description | | ------------------------ | -------- | ----------------- | | errors [*required*] | [string] | A list of errors. | {% /tab %} {% tab title="Example" %} ```json { "errors": [ "Bad Request" ] } ``` {% /tab %} {% /tab %} ### Code Example ##### \# Curl commandcurl -X GET "https://api.ap1.datadoghq.com"https://api.ap2.datadoghq.com"https://api.datadoghq.eu"https://api.ddog-gov.com"https://api.datadoghq.com"https://api.us3.datadoghq.com"https://api.us5.datadoghq.com/api/v2/llm-obs/v1/projects" \ -H "Accept: application/json" \ -H "DD-API-KEY: ${DD_API_KEY}" \ -H "DD-APPLICATION-KEY: ${DD_APP_KEY}" {% /tab %} ## Create an LLM Observability project{% #create-an-llm-observability-project %} {% tab title="v2" %} **Note**: This endpoint is in preview and is subject to change. If you have any feedback, contact [Datadog support](https://docs.datadoghq.com/help/). | Datadog site | API endpoint | | ----------------- | ------------------------------------------------------------- | | ap1.datadoghq.com | POST https://api.ap1.datadoghq.com/api/v2/llm-obs/v1/projects | | ap2.datadoghq.com | POST https://api.ap2.datadoghq.com/api/v2/llm-obs/v1/projects | | app.datadoghq.eu | POST https://api.datadoghq.eu/api/v2/llm-obs/v1/projects | | app.ddog-gov.com | POST https://api.ddog-gov.com/api/v2/llm-obs/v1/projects | | app.datadoghq.com | POST https://api.datadoghq.com/api/v2/llm-obs/v1/projects | | us3.datadoghq.com | POST https://api.us3.datadoghq.com/api/v2/llm-obs/v1/projects | | us5.datadoghq.com | POST https://api.us5.datadoghq.com/api/v2/llm-obs/v1/projects | ### Overview Create a new LLM Observability project. Returns the existing project if a name conflict occurs. ### Request #### Body Data (required) Create project payload. {% tab title="Model" %} | Parent field | Field | Type | Description | | ------------ | ---------------------------- | ------ | ------------------------------------------------------------------------------ | | | data [*required*] | object | Data object for creating an LLM Observability project. | | data | attributes [*required*] | object | Attributes for creating an LLM Observability project. | | attributes | description | string | Description of the project. | | attributes | name [*required*] | string | Name of the project. | | data | type [*required*] | enum | Resource type of an LLM Observability project. Allowed enum values: `projects` | {% /tab %} {% tab title="Example" %} ```json { "data": { "attributes": { "description": "string", "name": "My LLM Project" }, "type": "projects" } } ``` {% /tab %} ### Response {% tab title="200" %} OK {% tab title="Model" %} Response containing a single LLM Observability project. | Parent field | Field | Type | Description | | ------------ | ----------------------------- | --------- | ------------------------------------------------------------------------------ | | | data [*required*] | object | Data object for an LLM Observability project. | | data | attributes [*required*] | object | Attributes of an LLM Observability project. | | attributes | created_at [*required*] | date-time | Timestamp when the project was created. | | attributes | description [*required*] | string | Description of the project. | | attributes | name [*required*] | string | Name of the project. | | attributes | updated_at [*required*] | date-time | Timestamp when the project was last updated. | | data | id [*required*] | string | Unique identifier of the project. | | data | type [*required*] | enum | Resource type of an LLM Observability project. Allowed enum values: `projects` | {% /tab %} {% tab title="Example" %} ```json { "data": { "attributes": { "created_at": "2024-01-15T10:30:00Z", "description": "", "name": "My LLM Project", "updated_at": "2024-01-15T10:30:00Z" }, "id": "a33671aa-24fd-4dcd-9b33-a8ec7dde7751", "type": "projects" } } ``` {% /tab %} {% /tab %} {% tab title="201" %} Created {% tab title="Model" %} Response containing a single LLM Observability project. | Parent field | Field | Type | Description | | ------------ | ----------------------------- | --------- | ------------------------------------------------------------------------------ | | | data [*required*] | object | Data object for an LLM Observability project. | | data | attributes [*required*] | object | Attributes of an LLM Observability project. | | attributes | created_at [*required*] | date-time | Timestamp when the project was created. | | attributes | description [*required*] | string | Description of the project. | | attributes | name [*required*] | string | Name of the project. | | attributes | updated_at [*required*] | date-time | Timestamp when the project was last updated. | | data | id [*required*] | string | Unique identifier of the project. | | data | type [*required*] | enum | Resource type of an LLM Observability project. Allowed enum values: `projects` | {% /tab %} {% tab title="Example" %} ```json { "data": { "attributes": { "created_at": "2024-01-15T10:30:00Z", "description": "", "name": "My LLM Project", "updated_at": "2024-01-15T10:30:00Z" }, "id": "a33671aa-24fd-4dcd-9b33-a8ec7dde7751", "type": "projects" } } ``` {% /tab %} {% /tab %} {% tab title="400" %} Bad Request {% tab title="Model" %} API error response. | Parent field | Field | Type | Description | | ------------ | ------------------------ | -------- | ------------------------------------------------------------------------------- | | | errors [*required*] | [object] | A list of errors. | | errors | detail | string | A human-readable explanation specific to this occurrence of the error. | | errors | meta | object | Non-standard meta-information about the error | | errors | source | object | References to the source of the error. | | source | header | string | A string indicating the name of a single request header which caused the error. | | source | parameter | string | A string indicating which URI query parameter caused the error. | | source | pointer | string | A JSON pointer to the value in the request document that caused the error. | | errors | status | string | Status code of the response. | | errors | title | string | Short human-readable summary of the error. | {% /tab %} {% tab title="Example" %} ```json { "errors": [ { "detail": "Missing required attribute in body", "meta": {}, "source": { "header": "Authorization", "parameter": "limit", "pointer": "/data/attributes/title" }, "status": "400", "title": "Bad Request" } ] } ``` {% /tab %} {% /tab %} {% tab title="401" %} Unauthorized {% tab title="Model" %} API error response. | Parent field | Field | Type | Description | | ------------ | ------------------------ | -------- | ------------------------------------------------------------------------------- | | | errors [*required*] | [object] | A list of errors. | | errors | detail | string | A human-readable explanation specific to this occurrence of the error. | | errors | meta | object | Non-standard meta-information about the error | | errors | source | object | References to the source of the error. | | source | header | string | A string indicating the name of a single request header which caused the error. | | source | parameter | string | A string indicating which URI query parameter caused the error. | | source | pointer | string | A JSON pointer to the value in the request document that caused the error. | | errors | status | string | Status code of the response. | | errors | title | string | Short human-readable summary of the error. | {% /tab %} {% tab title="Example" %} ```json { "errors": [ { "detail": "Missing required attribute in body", "meta": {}, "source": { "header": "Authorization", "parameter": "limit", "pointer": "/data/attributes/title" }, "status": "400", "title": "Bad Request" } ] } ``` {% /tab %} {% /tab %} {% tab title="403" %} Forbidden {% tab title="Model" %} API error response. | Parent field | Field | Type | Description | | ------------ | ------------------------ | -------- | ------------------------------------------------------------------------------- | | | errors [*required*] | [object] | A list of errors. | | errors | detail | string | A human-readable explanation specific to this occurrence of the error. | | errors | meta | object | Non-standard meta-information about the error | | errors | source | object | References to the source of the error. | | source | header | string | A string indicating the name of a single request header which caused the error. | | source | parameter | string | A string indicating which URI query parameter caused the error. | | source | pointer | string | A JSON pointer to the value in the request document that caused the error. | | errors | status | string | Status code of the response. | | errors | title | string | Short human-readable summary of the error. | {% /tab %} {% tab title="Example" %} ```json { "errors": [ { "detail": "Missing required attribute in body", "meta": {}, "source": { "header": "Authorization", "parameter": "limit", "pointer": "/data/attributes/title" }, "status": "400", "title": "Bad Request" } ] } ``` {% /tab %} {% /tab %} {% tab title="429" %} Too many requests {% tab title="Model" %} API error response. | Field | Type | Description | | ------------------------ | -------- | ----------------- | | errors [*required*] | [string] | A list of errors. | {% /tab %} {% tab title="Example" %} ```json { "errors": [ "Bad Request" ] } ``` {% /tab %} {% /tab %} ### Code Example ##### \# Curl commandcurl -X POST "https://api.ap1.datadoghq.com"https://api.ap2.datadoghq.com"https://api.datadoghq.eu"https://api.ddog-gov.com"https://api.datadoghq.com"https://api.us3.datadoghq.com"https://api.us5.datadoghq.com/api/v2/llm-obs/v1/projects" \ -H "Accept: application/json" \ -H "Content-Type: application/json" \ -H "DD-API-KEY: ${DD_API_KEY}" \ -H "DD-APPLICATION-KEY: ${DD_APP_KEY}" \ -d @- << EOF { "data": { "attributes": { "name": "My LLM Project" }, "type": "projects" } } EOF {% /tab %} ## Update an LLM Observability project{% #update-an-llm-observability-project %} {% tab title="v2" %} **Note**: This endpoint is in preview and is subject to change. If you have any feedback, contact [Datadog support](https://docs.datadoghq.com/help/). | Datadog site | API endpoint | | ----------------- | --------------------------------------------------------------------------- | | ap1.datadoghq.com | PATCH https://api.ap1.datadoghq.com/api/v2/llm-obs/v1/projects/{project_id} | | ap2.datadoghq.com | PATCH https://api.ap2.datadoghq.com/api/v2/llm-obs/v1/projects/{project_id} | | app.datadoghq.eu | PATCH https://api.datadoghq.eu/api/v2/llm-obs/v1/projects/{project_id} | | app.ddog-gov.com | PATCH https://api.ddog-gov.com/api/v2/llm-obs/v1/projects/{project_id} | | app.datadoghq.com | PATCH https://api.datadoghq.com/api/v2/llm-obs/v1/projects/{project_id} | | us3.datadoghq.com | PATCH https://api.us3.datadoghq.com/api/v2/llm-obs/v1/projects/{project_id} | | us5.datadoghq.com | PATCH https://api.us5.datadoghq.com/api/v2/llm-obs/v1/projects/{project_id} | ### Overview Partially update an existing LLM Observability project. ### Arguments #### Path Parameters | Name | Type | Description | | ---------------------------- | ------ | ---------------------------------------- | | project_id [*required*] | string | The ID of the LLM Observability project. | ### Request #### Body Data (required) Update project payload. {% tab title="Model" %} | Parent field | Field | Type | Description | | ------------ | ---------------------------- | ------ | ------------------------------------------------------------------------------ | | | data [*required*] | object | Data object for updating an LLM Observability project. | | data | attributes [*required*] | object | Attributes for updating an LLM Observability project. | | attributes | description | string | Updated description of the project. | | attributes | name | string | Updated name of the project. | | data | type [*required*] | enum | Resource type of an LLM Observability project. Allowed enum values: `projects` | {% /tab %} {% tab title="Example" %} ```json { "data": { "attributes": { "description": "string", "name": "string" }, "type": "projects" } } ``` {% /tab %} ### Response {% tab title="200" %} OK {% tab title="Model" %} Response containing a single LLM Observability project. | Parent field | Field | Type | Description | | ------------ | ----------------------------- | --------- | ------------------------------------------------------------------------------ | | | data [*required*] | object | Data object for an LLM Observability project. | | data | attributes [*required*] | object | Attributes of an LLM Observability project. | | attributes | created_at [*required*] | date-time | Timestamp when the project was created. | | attributes | description [*required*] | string | Description of the project. | | attributes | name [*required*] | string | Name of the project. | | attributes | updated_at [*required*] | date-time | Timestamp when the project was last updated. | | data | id [*required*] | string | Unique identifier of the project. | | data | type [*required*] | enum | Resource type of an LLM Observability project. Allowed enum values: `projects` | {% /tab %} {% tab title="Example" %} ```json { "data": { "attributes": { "created_at": "2024-01-15T10:30:00Z", "description": "", "name": "My LLM Project", "updated_at": "2024-01-15T10:30:00Z" }, "id": "a33671aa-24fd-4dcd-9b33-a8ec7dde7751", "type": "projects" } } ``` {% /tab %} {% /tab %} {% tab title="400" %} Bad Request {% tab title="Model" %} API error response. | Parent field | Field | Type | Description | | ------------ | ------------------------ | -------- | ------------------------------------------------------------------------------- | | | errors [*required*] | [object] | A list of errors. | | errors | detail | string | A human-readable explanation specific to this occurrence of the error. | | errors | meta | object | Non-standard meta-information about the error | | errors | source | object | References to the source of the error. | | source | header | string | A string indicating the name of a single request header which caused the error. | | source | parameter | string | A string indicating which URI query parameter caused the error. | | source | pointer | string | A JSON pointer to the value in the request document that caused the error. | | errors | status | string | Status code of the response. | | errors | title | string | Short human-readable summary of the error. | {% /tab %} {% tab title="Example" %} ```json { "errors": [ { "detail": "Missing required attribute in body", "meta": {}, "source": { "header": "Authorization", "parameter": "limit", "pointer": "/data/attributes/title" }, "status": "400", "title": "Bad Request" } ] } ``` {% /tab %} {% /tab %} {% tab title="401" %} Unauthorized {% tab title="Model" %} API error response. | Parent field | Field | Type | Description | | ------------ | ------------------------ | -------- | ------------------------------------------------------------------------------- | | | errors [*required*] | [object] | A list of errors. | | errors | detail | string | A human-readable explanation specific to this occurrence of the error. | | errors | meta | object | Non-standard meta-information about the error | | errors | source | object | References to the source of the error. | | source | header | string | A string indicating the name of a single request header which caused the error. | | source | parameter | string | A string indicating which URI query parameter caused the error. | | source | pointer | string | A JSON pointer to the value in the request document that caused the error. | | errors | status | string | Status code of the response. | | errors | title | string | Short human-readable summary of the error. | {% /tab %} {% tab title="Example" %} ```json { "errors": [ { "detail": "Missing required attribute in body", "meta": {}, "source": { "header": "Authorization", "parameter": "limit", "pointer": "/data/attributes/title" }, "status": "400", "title": "Bad Request" } ] } ``` {% /tab %} {% /tab %} {% tab title="403" %} Forbidden {% tab title="Model" %} API error response. | Parent field | Field | Type | Description | | ------------ | ------------------------ | -------- | ------------------------------------------------------------------------------- | | | errors [*required*] | [object] | A list of errors. | | errors | detail | string | A human-readable explanation specific to this occurrence of the error. | | errors | meta | object | Non-standard meta-information about the error | | errors | source | object | References to the source of the error. | | source | header | string | A string indicating the name of a single request header which caused the error. | | source | parameter | string | A string indicating which URI query parameter caused the error. | | source | pointer | string | A JSON pointer to the value in the request document that caused the error. | | errors | status | string | Status code of the response. | | errors | title | string | Short human-readable summary of the error. | {% /tab %} {% tab title="Example" %} ```json { "errors": [ { "detail": "Missing required attribute in body", "meta": {}, "source": { "header": "Authorization", "parameter": "limit", "pointer": "/data/attributes/title" }, "status": "400", "title": "Bad Request" } ] } ``` {% /tab %} {% /tab %} {% tab title="404" %} Not Found {% tab title="Model" %} API error response. | Parent field | Field | Type | Description | | ------------ | ------------------------ | -------- | ------------------------------------------------------------------------------- | | | errors [*required*] | [object] | A list of errors. | | errors | detail | string | A human-readable explanation specific to this occurrence of the error. | | errors | meta | object | Non-standard meta-information about the error | | errors | source | object | References to the source of the error. | | source | header | string | A string indicating the name of a single request header which caused the error. | | source | parameter | string | A string indicating which URI query parameter caused the error. | | source | pointer | string | A JSON pointer to the value in the request document that caused the error. | | errors | status | string | Status code of the response. | | errors | title | string | Short human-readable summary of the error. | {% /tab %} {% tab title="Example" %} ```json { "errors": [ { "detail": "Missing required attribute in body", "meta": {}, "source": { "header": "Authorization", "parameter": "limit", "pointer": "/data/attributes/title" }, "status": "400", "title": "Bad Request" } ] } ``` {% /tab %} {% /tab %} {% tab title="429" %} Too many requests {% tab title="Model" %} API error response. | Field | Type | Description | | ------------------------ | -------- | ----------------- | | errors [*required*] | [string] | A list of errors. | {% /tab %} {% tab title="Example" %} ```json { "errors": [ "Bad Request" ] } ``` {% /tab %} {% /tab %} ### Code Example ##### \# Path parametersexport project_id="a33671aa-24fd-4dcd-9b33-a8ec7dde7751"\# Curl commandcurl -X PATCH "https://api.ap1.datadoghq.com"https://api.ap2.datadoghq.com"https://api.datadoghq.eu"https://api.ddog-gov.com"https://api.datadoghq.com"https://api.us3.datadoghq.com"https://api.us5.datadoghq.com/api/v2/llm-obs/v1/projects/${project_id}" \ -H "Accept: application/json" \ -H "Content-Type: application/json" \ -H "DD-API-KEY: ${DD_API_KEY}" \ -H "DD-APPLICATION-KEY: ${DD_APP_KEY}" \ -d @- << EOF { "data": { "attributes": {}, "type": "projects" } } EOF {% /tab %} ## Delete LLM Observability projects{% #delete-llm-observability-projects %} {% tab title="v2" %} **Note**: This endpoint is in preview and is subject to change. If you have any feedback, contact [Datadog support](https://docs.datadoghq.com/help/). | Datadog site | API endpoint | | ----------------- | -------------------------------------------------------------------- | | ap1.datadoghq.com | POST https://api.ap1.datadoghq.com/api/v2/llm-obs/v1/projects/delete | | ap2.datadoghq.com | POST https://api.ap2.datadoghq.com/api/v2/llm-obs/v1/projects/delete | | app.datadoghq.eu | POST https://api.datadoghq.eu/api/v2/llm-obs/v1/projects/delete | | app.ddog-gov.com | POST https://api.ddog-gov.com/api/v2/llm-obs/v1/projects/delete | | app.datadoghq.com | POST https://api.datadoghq.com/api/v2/llm-obs/v1/projects/delete | | us3.datadoghq.com | POST https://api.us3.datadoghq.com/api/v2/llm-obs/v1/projects/delete | | us5.datadoghq.com | POST https://api.us5.datadoghq.com/api/v2/llm-obs/v1/projects/delete | ### Overview Delete one or more LLM Observability projects. ### Request #### Body Data (required) Delete projects payload. {% tab title="Model" %} | Parent field | Field | Type | Description | | ------------ | ----------------------------- | -------- | ------------------------------------------------------------------------------ | | | data [*required*] | object | Data object for deleting LLM Observability projects. | | data | attributes [*required*] | object | Attributes for deleting LLM Observability projects. | | attributes | project_ids [*required*] | [string] | List of project IDs to delete. | | data | type [*required*] | enum | Resource type of an LLM Observability project. Allowed enum values: `projects` | {% /tab %} {% tab title="Example" %} ```json { "data": { "attributes": { "project_ids": [ "a33671aa-24fd-4dcd-9b33-a8ec7dde7751" ] }, "type": "projects" } } ``` {% /tab %} ### Response {% tab title="204" %} No Content {% /tab %} {% tab title="400" %} Bad Request {% tab title="Model" %} API error response. | Parent field | Field | Type | Description | | ------------ | ------------------------ | -------- | ------------------------------------------------------------------------------- | | | errors [*required*] | [object] | A list of errors. | | errors | detail | string | A human-readable explanation specific to this occurrence of the error. | | errors | meta | object | Non-standard meta-information about the error | | errors | source | object | References to the source of the error. | | source | header | string | A string indicating the name of a single request header which caused the error. | | source | parameter | string | A string indicating which URI query parameter caused the error. | | source | pointer | string | A JSON pointer to the value in the request document that caused the error. | | errors | status | string | Status code of the response. | | errors | title | string | Short human-readable summary of the error. | {% /tab %} {% tab title="Example" %} ```json { "errors": [ { "detail": "Missing required attribute in body", "meta": {}, "source": { "header": "Authorization", "parameter": "limit", "pointer": "/data/attributes/title" }, "status": "400", "title": "Bad Request" } ] } ``` {% /tab %} {% /tab %} {% tab title="401" %} Unauthorized {% tab title="Model" %} API error response. | Parent field | Field | Type | Description | | ------------ | ------------------------ | -------- | ------------------------------------------------------------------------------- | | | errors [*required*] | [object] | A list of errors. | | errors | detail | string | A human-readable explanation specific to this occurrence of the error. | | errors | meta | object | Non-standard meta-information about the error | | errors | source | object | References to the source of the error. | | source | header | string | A string indicating the name of a single request header which caused the error. | | source | parameter | string | A string indicating which URI query parameter caused the error. | | source | pointer | string | A JSON pointer to the value in the request document that caused the error. | | errors | status | string | Status code of the response. | | errors | title | string | Short human-readable summary of the error. | {% /tab %} {% tab title="Example" %} ```json { "errors": [ { "detail": "Missing required attribute in body", "meta": {}, "source": { "header": "Authorization", "parameter": "limit", "pointer": "/data/attributes/title" }, "status": "400", "title": "Bad Request" } ] } ``` {% /tab %} {% /tab %} {% tab title="403" %} Forbidden {% tab title="Model" %} API error response. | Parent field | Field | Type | Description | | ------------ | ------------------------ | -------- | ------------------------------------------------------------------------------- | | | errors [*required*] | [object] | A list of errors. | | errors | detail | string | A human-readable explanation specific to this occurrence of the error. | | errors | meta | object | Non-standard meta-information about the error | | errors | source | object | References to the source of the error. | | source | header | string | A string indicating the name of a single request header which caused the error. | | source | parameter | string | A string indicating which URI query parameter caused the error. | | source | pointer | string | A JSON pointer to the value in the request document that caused the error. | | errors | status | string | Status code of the response. | | errors | title | string | Short human-readable summary of the error. | {% /tab %} {% tab title="Example" %} ```json { "errors": [ { "detail": "Missing required attribute in body", "meta": {}, "source": { "header": "Authorization", "parameter": "limit", "pointer": "/data/attributes/title" }, "status": "400", "title": "Bad Request" } ] } ``` {% /tab %} {% /tab %} {% tab title="429" %} Too many requests {% tab title="Model" %} API error response. | Field | Type | Description | | ------------------------ | -------- | ----------------- | | errors [*required*] | [string] | A list of errors. | {% /tab %} {% tab title="Example" %} ```json { "errors": [ "Bad Request" ] } ``` {% /tab %} {% /tab %} ### Code Example ##### \# Curl commandcurl -X POST "https://api.ap1.datadoghq.com"https://api.ap2.datadoghq.com"https://api.datadoghq.eu"https://api.ddog-gov.com"https://api.datadoghq.com"https://api.us3.datadoghq.com"https://api.us5.datadoghq.com/api/v2/llm-obs/v1/projects/delete" \ -H "Content-Type: application/json" \ -H "DD-API-KEY: ${DD_API_KEY}" \ -H "DD-APPLICATION-KEY: ${DD_APP_KEY}" \ -d @- << EOF { "data": { "attributes": { "project_ids": [ "a33671aa-24fd-4dcd-9b33-a8ec7dde7751" ] }, "type": "projects" } } EOF {% /tab %} ## List LLM Observability datasets{% #list-llm-observability-datasets %} {% tab title="v2" %} **Note**: This endpoint is in preview and is subject to change. If you have any feedback, contact [Datadog support](https://docs.datadoghq.com/help/). | Datadog site | API endpoint | | ----------------- | ------------------------------------------------------------------------- | | ap1.datadoghq.com | GET https://api.ap1.datadoghq.com/api/v2/llm-obs/v1/{project_id}/datasets | | ap2.datadoghq.com | GET https://api.ap2.datadoghq.com/api/v2/llm-obs/v1/{project_id}/datasets | | app.datadoghq.eu | GET https://api.datadoghq.eu/api/v2/llm-obs/v1/{project_id}/datasets | | app.ddog-gov.com | GET https://api.ddog-gov.com/api/v2/llm-obs/v1/{project_id}/datasets | | app.datadoghq.com | GET https://api.datadoghq.com/api/v2/llm-obs/v1/{project_id}/datasets | | us3.datadoghq.com | GET https://api.us3.datadoghq.com/api/v2/llm-obs/v1/{project_id}/datasets | | us5.datadoghq.com | GET https://api.us5.datadoghq.com/api/v2/llm-obs/v1/{project_id}/datasets | ### Overview List all LLM Observability datasets for a project, sorted by creation date, newest first. ### Arguments #### Path Parameters | Name | Type | Description | | ---------------------------- | ------ | ---------------------------------------- | | project_id [*required*] | string | The ID of the LLM Observability project. | #### Query Strings | Name | Type | Description | | ------------ | ------- | --------------------------------------------------------------- | | filter[name] | string | Filter datasets by name. | | filter[id] | string | Filter datasets by dataset ID. | | page[cursor] | string | Use the Pagination cursor to retrieve the next page of results. | | page[limit] | integer | Maximum number of results to return per page. | ### Response {% tab title="200" %} OK {% tab title="Model" %} Response containing a list of LLM Observability datasets. | Parent field | Field | Type | Description | | ------------ | --------------------------------- | --------- | ------------------------------------------------------------------------------ | | | data [*required*] | [object] | List of datasets. | | data | attributes [*required*] | object | Attributes of an LLM Observability dataset. | | attributes | created_at [*required*] | date-time | Timestamp when the dataset was created. | | attributes | current_version [*required*] | int64 | Current version number of the dataset. | | attributes | description [*required*] | string | Description of the dataset. | | attributes | metadata [*required*] | object | Arbitrary metadata associated with the dataset. | | attributes | name [*required*] | string | Name of the dataset. | | attributes | updated_at [*required*] | date-time | Timestamp when the dataset was last updated. | | data | id [*required*] | string | Unique identifier of the dataset. | | data | type [*required*] | enum | Resource type of an LLM Observability dataset. Allowed enum values: `datasets` | | | meta | object | Pagination cursor metadata. | | meta | after | string | Cursor for the next page of results. | {% /tab %} {% tab title="Example" %} ```json { "data": [ { "attributes": { "created_at": "2024-01-15T10:30:00Z", "current_version": 1, "description": "", "metadata": {}, "name": "My LLM Dataset", "updated_at": "2024-01-15T10:30:00Z" }, "id": "9f64e5c7-dc5a-45c8-a17c-1b85f0bec97d", "type": "datasets" } ], "meta": { "after": "string" } } ``` {% /tab %} {% /tab %} {% tab title="400" %} Bad Request {% tab title="Model" %} API error response. | Parent field | Field | Type | Description | | ------------ | ------------------------ | -------- | ------------------------------------------------------------------------------- | | | errors [*required*] | [object] | A list of errors. | | errors | detail | string | A human-readable explanation specific to this occurrence of the error. | | errors | meta | object | Non-standard meta-information about the error | | errors | source | object | References to the source of the error. | | source | header | string | A string indicating the name of a single request header which caused the error. | | source | parameter | string | A string indicating which URI query parameter caused the error. | | source | pointer | string | A JSON pointer to the value in the request document that caused the error. | | errors | status | string | Status code of the response. | | errors | title | string | Short human-readable summary of the error. | {% /tab %} {% tab title="Example" %} ```json { "errors": [ { "detail": "Missing required attribute in body", "meta": {}, "source": { "header": "Authorization", "parameter": "limit", "pointer": "/data/attributes/title" }, "status": "400", "title": "Bad Request" } ] } ``` {% /tab %} {% /tab %} {% tab title="401" %} Unauthorized {% tab title="Model" %} API error response. | Parent field | Field | Type | Description | | ------------ | ------------------------ | -------- | ------------------------------------------------------------------------------- | | | errors [*required*] | [object] | A list of errors. | | errors | detail | string | A human-readable explanation specific to this occurrence of the error. | | errors | meta | object | Non-standard meta-information about the error | | errors | source | object | References to the source of the error. | | source | header | string | A string indicating the name of a single request header which caused the error. | | source | parameter | string | A string indicating which URI query parameter caused the error. | | source | pointer | string | A JSON pointer to the value in the request document that caused the error. | | errors | status | string | Status code of the response. | | errors | title | string | Short human-readable summary of the error. | {% /tab %} {% tab title="Example" %} ```json { "errors": [ { "detail": "Missing required attribute in body", "meta": {}, "source": { "header": "Authorization", "parameter": "limit", "pointer": "/data/attributes/title" }, "status": "400", "title": "Bad Request" } ] } ``` {% /tab %} {% /tab %} {% tab title="403" %} Forbidden {% tab title="Model" %} API error response. | Parent field | Field | Type | Description | | ------------ | ------------------------ | -------- | ------------------------------------------------------------------------------- | | | errors [*required*] | [object] | A list of errors. | | errors | detail | string | A human-readable explanation specific to this occurrence of the error. | | errors | meta | object | Non-standard meta-information about the error | | errors | source | object | References to the source of the error. | | source | header | string | A string indicating the name of a single request header which caused the error. | | source | parameter | string | A string indicating which URI query parameter caused the error. | | source | pointer | string | A JSON pointer to the value in the request document that caused the error. | | errors | status | string | Status code of the response. | | errors | title | string | Short human-readable summary of the error. | {% /tab %} {% tab title="Example" %} ```json { "errors": [ { "detail": "Missing required attribute in body", "meta": {}, "source": { "header": "Authorization", "parameter": "limit", "pointer": "/data/attributes/title" }, "status": "400", "title": "Bad Request" } ] } ``` {% /tab %} {% /tab %} {% tab title="404" %} Not Found {% tab title="Model" %} API error response. | Parent field | Field | Type | Description | | ------------ | ------------------------ | -------- | ------------------------------------------------------------------------------- | | | errors [*required*] | [object] | A list of errors. | | errors | detail | string | A human-readable explanation specific to this occurrence of the error. | | errors | meta | object | Non-standard meta-information about the error | | errors | source | object | References to the source of the error. | | source | header | string | A string indicating the name of a single request header which caused the error. | | source | parameter | string | A string indicating which URI query parameter caused the error. | | source | pointer | string | A JSON pointer to the value in the request document that caused the error. | | errors | status | string | Status code of the response. | | errors | title | string | Short human-readable summary of the error. | {% /tab %} {% tab title="Example" %} ```json { "errors": [ { "detail": "Missing required attribute in body", "meta": {}, "source": { "header": "Authorization", "parameter": "limit", "pointer": "/data/attributes/title" }, "status": "400", "title": "Bad Request" } ] } ``` {% /tab %} {% /tab %} {% tab title="429" %} Too many requests {% tab title="Model" %} API error response. | Field | Type | Description | | ------------------------ | -------- | ----------------- | | errors [*required*] | [string] | A list of errors. | {% /tab %} {% tab title="Example" %} ```json { "errors": [ "Bad Request" ] } ``` {% /tab %} {% /tab %} ### Code Example ##### \# Path parametersexport project_id="a33671aa-24fd-4dcd-9b33-a8ec7dde7751"\# Curl commandcurl -X GET "https://api.ap1.datadoghq.com"https://api.ap2.datadoghq.com"https://api.datadoghq.eu"https://api.ddog-gov.com"https://api.datadoghq.com"https://api.us3.datadoghq.com"https://api.us5.datadoghq.com/api/v2/llm-obs/v1/${project_id}/datasets" \ -H "Accept: application/json" \ -H "DD-API-KEY: ${DD_API_KEY}" \ -H "DD-APPLICATION-KEY: ${DD_APP_KEY}" {% /tab %} ## Create an LLM Observability dataset{% #create-an-llm-observability-dataset %} {% tab title="v2" %} **Note**: This endpoint is in preview and is subject to change. If you have any feedback, contact [Datadog support](https://docs.datadoghq.com/help/). | Datadog site | API endpoint | | ----------------- | -------------------------------------------------------------------------- | | ap1.datadoghq.com | POST https://api.ap1.datadoghq.com/api/v2/llm-obs/v1/{project_id}/datasets | | ap2.datadoghq.com | POST https://api.ap2.datadoghq.com/api/v2/llm-obs/v1/{project_id}/datasets | | app.datadoghq.eu | POST https://api.datadoghq.eu/api/v2/llm-obs/v1/{project_id}/datasets | | app.ddog-gov.com | POST https://api.ddog-gov.com/api/v2/llm-obs/v1/{project_id}/datasets | | app.datadoghq.com | POST https://api.datadoghq.com/api/v2/llm-obs/v1/{project_id}/datasets | | us3.datadoghq.com | POST https://api.us3.datadoghq.com/api/v2/llm-obs/v1/{project_id}/datasets | | us5.datadoghq.com | POST https://api.us5.datadoghq.com/api/v2/llm-obs/v1/{project_id}/datasets | ### Overview Create a new LLM Observability dataset within the specified project. ### Arguments #### Path Parameters | Name | Type | Description | | ---------------------------- | ------ | ---------------------------------------- | | project_id [*required*] | string | The ID of the LLM Observability project. | ### Request #### Body Data (required) Create dataset payload. {% tab title="Model" %} | Parent field | Field | Type | Description | | ------------ | ---------------------------- | ------ | ------------------------------------------------------------------------------ | | | data [*required*] | object | Data object for creating an LLM Observability dataset. | | data | attributes [*required*] | object | Attributes for creating an LLM Observability dataset. | | attributes | description | string | Description of the dataset. | | attributes | metadata | object | Arbitrary metadata associated with the dataset. | | attributes | name [*required*] | string | Name of the dataset. | | data | type [*required*] | enum | Resource type of an LLM Observability dataset. Allowed enum values: `datasets` | {% /tab %} {% tab title="Example" %} ```json { "data": { "attributes": { "description": "string", "metadata": {}, "name": "My LLM Dataset" }, "type": "datasets" } } ``` {% /tab %} ### Response {% tab title="200" %} OK {% tab title="Model" %} Response containing a single LLM Observability dataset. | Parent field | Field | Type | Description | | ------------ | --------------------------------- | --------- | ------------------------------------------------------------------------------ | | | data [*required*] | object | Data object for an LLM Observability dataset. | | data | attributes [*required*] | object | Attributes of an LLM Observability dataset. | | attributes | created_at [*required*] | date-time | Timestamp when the dataset was created. | | attributes | current_version [*required*] | int64 | Current version number of the dataset. | | attributes | description [*required*] | string | Description of the dataset. | | attributes | metadata [*required*] | object | Arbitrary metadata associated with the dataset. | | attributes | name [*required*] | string | Name of the dataset. | | attributes | updated_at [*required*] | date-time | Timestamp when the dataset was last updated. | | data | id [*required*] | string | Unique identifier of the dataset. | | data | type [*required*] | enum | Resource type of an LLM Observability dataset. Allowed enum values: `datasets` | {% /tab %} {% tab title="Example" %} ```json { "data": { "attributes": { "created_at": "2024-01-15T10:30:00Z", "current_version": 1, "description": "", "metadata": {}, "name": "My LLM Dataset", "updated_at": "2024-01-15T10:30:00Z" }, "id": "9f64e5c7-dc5a-45c8-a17c-1b85f0bec97d", "type": "datasets" } } ``` {% /tab %} {% /tab %} {% tab title="201" %} Created {% tab title="Model" %} Response containing a single LLM Observability dataset. | Parent field | Field | Type | Description | | ------------ | --------------------------------- | --------- | ------------------------------------------------------------------------------ | | | data [*required*] | object | Data object for an LLM Observability dataset. | | data | attributes [*required*] | object | Attributes of an LLM Observability dataset. | | attributes | created_at [*required*] | date-time | Timestamp when the dataset was created. | | attributes | current_version [*required*] | int64 | Current version number of the dataset. | | attributes | description [*required*] | string | Description of the dataset. | | attributes | metadata [*required*] | object | Arbitrary metadata associated with the dataset. | | attributes | name [*required*] | string | Name of the dataset. | | attributes | updated_at [*required*] | date-time | Timestamp when the dataset was last updated. | | data | id [*required*] | string | Unique identifier of the dataset. | | data | type [*required*] | enum | Resource type of an LLM Observability dataset. Allowed enum values: `datasets` | {% /tab %} {% tab title="Example" %} ```json { "data": { "attributes": { "created_at": "2024-01-15T10:30:00Z", "current_version": 1, "description": "", "metadata": {}, "name": "My LLM Dataset", "updated_at": "2024-01-15T10:30:00Z" }, "id": "9f64e5c7-dc5a-45c8-a17c-1b85f0bec97d", "type": "datasets" } } ``` {% /tab %} {% /tab %} {% tab title="400" %} Bad Request {% tab title="Model" %} API error response. | Parent field | Field | Type | Description | | ------------ | ------------------------ | -------- | ------------------------------------------------------------------------------- | | | errors [*required*] | [object] | A list of errors. | | errors | detail | string | A human-readable explanation specific to this occurrence of the error. | | errors | meta | object | Non-standard meta-information about the error | | errors | source | object | References to the source of the error. | | source | header | string | A string indicating the name of a single request header which caused the error. | | source | parameter | string | A string indicating which URI query parameter caused the error. | | source | pointer | string | A JSON pointer to the value in the request document that caused the error. | | errors | status | string | Status code of the response. | | errors | title | string | Short human-readable summary of the error. | {% /tab %} {% tab title="Example" %} ```json { "errors": [ { "detail": "Missing required attribute in body", "meta": {}, "source": { "header": "Authorization", "parameter": "limit", "pointer": "/data/attributes/title" }, "status": "400", "title": "Bad Request" } ] } ``` {% /tab %} {% /tab %} {% tab title="401" %} Unauthorized {% tab title="Model" %} API error response. | Parent field | Field | Type | Description | | ------------ | ------------------------ | -------- | ------------------------------------------------------------------------------- | | | errors [*required*] | [object] | A list of errors. | | errors | detail | string | A human-readable explanation specific to this occurrence of the error. | | errors | meta | object | Non-standard meta-information about the error | | errors | source | object | References to the source of the error. | | source | header | string | A string indicating the name of a single request header which caused the error. | | source | parameter | string | A string indicating which URI query parameter caused the error. | | source | pointer | string | A JSON pointer to the value in the request document that caused the error. | | errors | status | string | Status code of the response. | | errors | title | string | Short human-readable summary of the error. | {% /tab %} {% tab title="Example" %} ```json { "errors": [ { "detail": "Missing required attribute in body", "meta": {}, "source": { "header": "Authorization", "parameter": "limit", "pointer": "/data/attributes/title" }, "status": "400", "title": "Bad Request" } ] } ``` {% /tab %} {% /tab %} {% tab title="403" %} Forbidden {% tab title="Model" %} API error response. | Parent field | Field | Type | Description | | ------------ | ------------------------ | -------- | ------------------------------------------------------------------------------- | | | errors [*required*] | [object] | A list of errors. | | errors | detail | string | A human-readable explanation specific to this occurrence of the error. | | errors | meta | object | Non-standard meta-information about the error | | errors | source | object | References to the source of the error. | | source | header | string | A string indicating the name of a single request header which caused the error. | | source | parameter | string | A string indicating which URI query parameter caused the error. | | source | pointer | string | A JSON pointer to the value in the request document that caused the error. | | errors | status | string | Status code of the response. | | errors | title | string | Short human-readable summary of the error. | {% /tab %} {% tab title="Example" %} ```json { "errors": [ { "detail": "Missing required attribute in body", "meta": {}, "source": { "header": "Authorization", "parameter": "limit", "pointer": "/data/attributes/title" }, "status": "400", "title": "Bad Request" } ] } ``` {% /tab %} {% /tab %} {% tab title="404" %} Not Found {% tab title="Model" %} API error response. | Parent field | Field | Type | Description | | ------------ | ------------------------ | -------- | ------------------------------------------------------------------------------- | | | errors [*required*] | [object] | A list of errors. | | errors | detail | string | A human-readable explanation specific to this occurrence of the error. | | errors | meta | object | Non-standard meta-information about the error | | errors | source | object | References to the source of the error. | | source | header | string | A string indicating the name of a single request header which caused the error. | | source | parameter | string | A string indicating which URI query parameter caused the error. | | source | pointer | string | A JSON pointer to the value in the request document that caused the error. | | errors | status | string | Status code of the response. | | errors | title | string | Short human-readable summary of the error. | {% /tab %} {% tab title="Example" %} ```json { "errors": [ { "detail": "Missing required attribute in body", "meta": {}, "source": { "header": "Authorization", "parameter": "limit", "pointer": "/data/attributes/title" }, "status": "400", "title": "Bad Request" } ] } ``` {% /tab %} {% /tab %} {% tab title="429" %} Too many requests {% tab title="Model" %} API error response. | Field | Type | Description | | ------------------------ | -------- | ----------------- | | errors [*required*] | [string] | A list of errors. | {% /tab %} {% tab title="Example" %} ```json { "errors": [ "Bad Request" ] } ``` {% /tab %} {% /tab %} ### Code Example ##### \# Path parametersexport project_id="a33671aa-24fd-4dcd-9b33-a8ec7dde7751"\# Curl commandcurl -X POST "https://api.ap1.datadoghq.com"https://api.ap2.datadoghq.com"https://api.datadoghq.eu"https://api.ddog-gov.com"https://api.datadoghq.com"https://api.us3.datadoghq.com"https://api.us5.datadoghq.com/api/v2/llm-obs/v1/${project_id}/datasets" \ -H "Accept: application/json" \ -H "Content-Type: application/json" \ -H "DD-API-KEY: ${DD_API_KEY}" \ -H "DD-APPLICATION-KEY: ${DD_APP_KEY}" \ -d @- << EOF { "data": { "attributes": { "name": "My LLM Dataset" }, "type": "datasets" } } EOF {% /tab %} ## List LLM Observability dataset records{% #list-llm-observability-dataset-records %} {% tab title="v2" %} **Note**: This endpoint is in preview and is subject to change. If you have any feedback, contact [Datadog support](https://docs.datadoghq.com/help/). | Datadog site | API endpoint | | ----------------- | ---------------------------------------------------------------------------------------------- | | ap1.datadoghq.com | GET https://api.ap1.datadoghq.com/api/v2/llm-obs/v1/{project_id}/datasets/{dataset_id}/records | | ap2.datadoghq.com | GET https://api.ap2.datadoghq.com/api/v2/llm-obs/v1/{project_id}/datasets/{dataset_id}/records | | app.datadoghq.eu | GET https://api.datadoghq.eu/api/v2/llm-obs/v1/{project_id}/datasets/{dataset_id}/records | | app.ddog-gov.com | GET https://api.ddog-gov.com/api/v2/llm-obs/v1/{project_id}/datasets/{dataset_id}/records | | app.datadoghq.com | GET https://api.datadoghq.com/api/v2/llm-obs/v1/{project_id}/datasets/{dataset_id}/records | | us3.datadoghq.com | GET https://api.us3.datadoghq.com/api/v2/llm-obs/v1/{project_id}/datasets/{dataset_id}/records | | us5.datadoghq.com | GET https://api.us5.datadoghq.com/api/v2/llm-obs/v1/{project_id}/datasets/{dataset_id}/records | ### Overview List all records in an LLM Observability dataset, sorted by creation date, newest first. ### Arguments #### Path Parameters | Name | Type | Description | | ---------------------------- | ------ | ---------------------------------------- | | project_id [*required*] | string | The ID of the LLM Observability project. | | dataset_id [*required*] | string | The ID of the LLM Observability dataset. | #### Query Strings | Name | Type | Description | | --------------- | ------- | ---------------------------------------------------------------------------------- | | filter[version] | integer | Retrieve records from a specific dataset version. Defaults to the current version. | | page[cursor] | string | Use the Pagination cursor to retrieve the next page of results. | | page[limit] | integer | Maximum number of results to return per page. | ### Response {% tab title="200" %} OK {% tab title="Model" %} Response containing a paginated list of LLM Observability dataset records. | Parent field | Field | Type | Description | | --------------- | --------------------------------- | ------------------- | ----------------------------------------------------- | | | data [*required*] | [object] | List of dataset records. | | data | created_at [*required*] | date-time | Timestamp when the record was created. | | data | dataset_id [*required*] | string | Identifier of the dataset this record belongs to. | | data | expected_output [*required*] | object | Represents any valid JSON value. | | expected_output | Option 1 | string | A scalar string value. | | expected_output | Option 2 | double | A scalar numeric value. | | expected_output | Option 3 | object | An arbitrary object value with additional properties. | | expected_output | Option 4 | [ ] | An array of arbitrary values. | | Option 4 | Option 1 | string | A scalar string value. | | Option 4 | Option 2 | double | A scalar numeric value. | | Option 4 | Option 3 | object | An arbitrary object value with additional properties. | | Option 4 | Option 4 | boolean | A scalar boolean value. | | expected_output | Option 5 | boolean | A scalar boolean value. | | data | id [*required*] | string | Unique identifier of the record. | | data | input [*required*] | object | Represents any valid JSON value. | | input | Option 1 | string | A scalar string value. | | input | Option 2 | double | A scalar numeric value. | | input | Option 3 | object | An arbitrary object value with additional properties. | | input | Option 4 | [ ] | An array of arbitrary values. | | Option 4 | Option 1 | string | A scalar string value. | | Option 4 | Option 2 | double | A scalar numeric value. | | Option 4 | Option 3 | object | An arbitrary object value with additional properties. | | Option 4 | Option 4 | boolean | A scalar boolean value. | | input | Option 5 | boolean | A scalar boolean value. | | data | metadata [*required*] | object | Arbitrary metadata associated with the record. | | data | updated_at [*required*] | date-time | Timestamp when the record was last updated. | | | meta | object | Pagination cursor metadata. | | meta | after | string | Cursor for the next page of results. | {% /tab %} {% tab title="Example" %} ```json { "data": [ { "created_at": "2024-01-15T10:30:00Z", "dataset_id": "9f64e5c7-dc5a-45c8-a17c-1b85f0bec97d", "expected_output": { "description": "undefined", "type": "undefined" }, "id": "rec-7c3f5a1b-9e2d-4f8a-b1c6-3d7e9f0a2b4c", "input": { "description": "undefined", "type": "undefined" }, "metadata": {}, "updated_at": "2024-01-15T10:30:00Z" } ], "meta": { "after": "string" } } ``` {% /tab %} {% /tab %} {% tab title="400" %} Bad Request {% tab title="Model" %} API error response. | Parent field | Field | Type | Description | | ------------ | ------------------------ | -------- | ------------------------------------------------------------------------------- | | | errors [*required*] | [object] | A list of errors. | | errors | detail | string | A human-readable explanation specific to this occurrence of the error. | | errors | meta | object | Non-standard meta-information about the error | | errors | source | object | References to the source of the error. | | source | header | string | A string indicating the name of a single request header which caused the error. | | source | parameter | string | A string indicating which URI query parameter caused the error. | | source | pointer | string | A JSON pointer to the value in the request document that caused the error. | | errors | status | string | Status code of the response. | | errors | title | string | Short human-readable summary of the error. | {% /tab %} {% tab title="Example" %} ```json { "errors": [ { "detail": "Missing required attribute in body", "meta": {}, "source": { "header": "Authorization", "parameter": "limit", "pointer": "/data/attributes/title" }, "status": "400", "title": "Bad Request" } ] } ``` {% /tab %} {% /tab %} {% tab title="401" %} Unauthorized {% tab title="Model" %} API error response. | Parent field | Field | Type | Description | | ------------ | ------------------------ | -------- | ------------------------------------------------------------------------------- | | | errors [*required*] | [object] | A list of errors. | | errors | detail | string | A human-readable explanation specific to this occurrence of the error. | | errors | meta | object | Non-standard meta-information about the error | | errors | source | object | References to the source of the error. | | source | header | string | A string indicating the name of a single request header which caused the error. | | source | parameter | string | A string indicating which URI query parameter caused the error. | | source | pointer | string | A JSON pointer to the value in the request document that caused the error. | | errors | status | string | Status code of the response. | | errors | title | string | Short human-readable summary of the error. | {% /tab %} {% tab title="Example" %} ```json { "errors": [ { "detail": "Missing required attribute in body", "meta": {}, "source": { "header": "Authorization", "parameter": "limit", "pointer": "/data/attributes/title" }, "status": "400", "title": "Bad Request" } ] } ``` {% /tab %} {% /tab %} {% tab title="403" %} Forbidden {% tab title="Model" %} API error response. | Parent field | Field | Type | Description | | ------------ | ------------------------ | -------- | ------------------------------------------------------------------------------- | | | errors [*required*] | [object] | A list of errors. | | errors | detail | string | A human-readable explanation specific to this occurrence of the error. | | errors | meta | object | Non-standard meta-information about the error | | errors | source | object | References to the source of the error. | | source | header | string | A string indicating the name of a single request header which caused the error. | | source | parameter | string | A string indicating which URI query parameter caused the error. | | source | pointer | string | A JSON pointer to the value in the request document that caused the error. | | errors | status | string | Status code of the response. | | errors | title | string | Short human-readable summary of the error. | {% /tab %} {% tab title="Example" %} ```json { "errors": [ { "detail": "Missing required attribute in body", "meta": {}, "source": { "header": "Authorization", "parameter": "limit", "pointer": "/data/attributes/title" }, "status": "400", "title": "Bad Request" } ] } ``` {% /tab %} {% /tab %} {% tab title="404" %} Not Found {% tab title="Model" %} API error response. | Parent field | Field | Type | Description | | ------------ | ------------------------ | -------- | ------------------------------------------------------------------------------- | | | errors [*required*] | [object] | A list of errors. | | errors | detail | string | A human-readable explanation specific to this occurrence of the error. | | errors | meta | object | Non-standard meta-information about the error | | errors | source | object | References to the source of the error. | | source | header | string | A string indicating the name of a single request header which caused the error. | | source | parameter | string | A string indicating which URI query parameter caused the error. | | source | pointer | string | A JSON pointer to the value in the request document that caused the error. | | errors | status | string | Status code of the response. | | errors | title | string | Short human-readable summary of the error. | {% /tab %} {% tab title="Example" %} ```json { "errors": [ { "detail": "Missing required attribute in body", "meta": {}, "source": { "header": "Authorization", "parameter": "limit", "pointer": "/data/attributes/title" }, "status": "400", "title": "Bad Request" } ] } ``` {% /tab %} {% /tab %} {% tab title="429" %} Too many requests {% tab title="Model" %} API error response. | Field | Type | Description | | ------------------------ | -------- | ----------------- | | errors [*required*] | [string] | A list of errors. | {% /tab %} {% tab title="Example" %} ```json { "errors": [ "Bad Request" ] } ``` {% /tab %} {% /tab %} ### Code Example ##### \# Path parametersexport project_id="a33671aa-24fd-4dcd-9b33-a8ec7dde7751"export dataset_id="9f64e5c7-dc5a-45c8-a17c-1b85f0bec97d"\# Curl commandcurl -X GET "https://api.ap1.datadoghq.com"https://api.ap2.datadoghq.com"https://api.datadoghq.eu"https://api.ddog-gov.com"https://api.datadoghq.com"https://api.us3.datadoghq.com"https://api.us5.datadoghq.com/api/v2/llm-obs/v1/${project_id}/datasets/${dataset_id}/records" \ -H "Accept: application/json" \ -H "DD-API-KEY: ${DD_API_KEY}" \ -H "DD-APPLICATION-KEY: ${DD_APP_KEY}" {% /tab %} ## Append records to an LLM Observability dataset{% #append-records-to-an-llm-observability-dataset %} {% tab title="v2" %} **Note**: This endpoint is in preview and is subject to change. If you have any feedback, contact [Datadog support](https://docs.datadoghq.com/help/). | Datadog site | API endpoint | | ----------------- | ----------------------------------------------------------------------------------------------- | | ap1.datadoghq.com | POST https://api.ap1.datadoghq.com/api/v2/llm-obs/v1/{project_id}/datasets/{dataset_id}/records | | ap2.datadoghq.com | POST https://api.ap2.datadoghq.com/api/v2/llm-obs/v1/{project_id}/datasets/{dataset_id}/records | | app.datadoghq.eu | POST https://api.datadoghq.eu/api/v2/llm-obs/v1/{project_id}/datasets/{dataset_id}/records | | app.ddog-gov.com | POST https://api.ddog-gov.com/api/v2/llm-obs/v1/{project_id}/datasets/{dataset_id}/records | | app.datadoghq.com | POST https://api.datadoghq.com/api/v2/llm-obs/v1/{project_id}/datasets/{dataset_id}/records | | us3.datadoghq.com | POST https://api.us3.datadoghq.com/api/v2/llm-obs/v1/{project_id}/datasets/{dataset_id}/records | | us5.datadoghq.com | POST https://api.us5.datadoghq.com/api/v2/llm-obs/v1/{project_id}/datasets/{dataset_id}/records | ### Overview Append one or more records to an LLM Observability dataset. ### Arguments #### Path Parameters | Name | Type | Description | | ---------------------------- | ------ | ---------------------------------------- | | project_id [*required*] | string | The ID of the LLM Observability project. | | dataset_id [*required*] | string | The ID of the LLM Observability dataset. | ### Request #### Body Data (required) Append records payload. {% tab title="Model" %} | Parent field | Field | Type | Description | | --------------- | ---------------------------- | ------------------- | ---------------------------------------------------------------------------------- | | | data [*required*] | object | Data object for appending records to an LLM Observability dataset. | | data | attributes [*required*] | object | Attributes for appending records to an LLM Observability dataset. | | attributes | deduplicate | boolean | Whether to deduplicate records before appending. Defaults to `true`. | | attributes | records [*required*] | [object] | List of records to append to the dataset. | | records | expected_output | object | Represents any valid JSON value. | | expected_output | Option 1 | string | A scalar string value. | | expected_output | Option 2 | double | A scalar numeric value. | | expected_output | Option 3 | object | An arbitrary object value with additional properties. | | expected_output | Option 4 | [ ] | An array of arbitrary values. | | Option 4 | Option 1 | string | A scalar string value. | | Option 4 | Option 2 | double | A scalar numeric value. | | Option 4 | Option 3 | object | An arbitrary object value with additional properties. | | Option 4 | Option 4 | boolean | A scalar boolean value. | | expected_output | Option 5 | boolean | A scalar boolean value. | | records | input [*required*] | object | Represents any valid JSON value. | | input | Option 1 | string | A scalar string value. | | input | Option 2 | double | A scalar numeric value. | | input | Option 3 | object | An arbitrary object value with additional properties. | | input | Option 4 | [ ] | An array of arbitrary values. | | Option 4 | Option 1 | string | A scalar string value. | | Option 4 | Option 2 | double | A scalar numeric value. | | Option 4 | Option 3 | object | An arbitrary object value with additional properties. | | Option 4 | Option 4 | boolean | A scalar boolean value. | | input | Option 5 | boolean | A scalar boolean value. | | records | metadata | object | Arbitrary metadata associated with the record. | | data | type [*required*] | enum | Resource type of LLM Observability dataset records. Allowed enum values: `records` | {% /tab %} {% tab title="Example" %} ```json { "data": { "attributes": { "deduplicate": false, "records": [ { "expected_output": { "description": "undefined", "type": "undefined" }, "input": { "description": "undefined", "type": "undefined" }, "metadata": {} } ] }, "type": "records" } } ``` {% /tab %} ### Response {% tab title="200" %} OK {% tab title="Model" %} Response containing records after a create or update operation. | Parent field | Field | Type | Description | | --------------- | --------------------------------- | ------------------- | ----------------------------------------------------- | | | data [*required*] | [object] | List of affected dataset records. | | data | records [*required*] | [object] | List of affected dataset records. | | records | created_at [*required*] | date-time | Timestamp when the record was created. | | records | dataset_id [*required*] | string | Identifier of the dataset this record belongs to. | | records | expected_output [*required*] | object | Represents any valid JSON value. | | expected_output | Option 1 | string | A scalar string value. | | expected_output | Option 2 | double | A scalar numeric value. | | expected_output | Option 3 | object | An arbitrary object value with additional properties. | | expected_output | Option 4 | [ ] | An array of arbitrary values. | | Option 4 | Option 1 | string | A scalar string value. | | Option 4 | Option 2 | double | A scalar numeric value. | | Option 4 | Option 3 | object | An arbitrary object value with additional properties. | | Option 4 | Option 4 | boolean | A scalar boolean value. | | expected_output | Option 5 | boolean | A scalar boolean value. | | records | id [*required*] | string | Unique identifier of the record. | | records | input [*required*] | object | Represents any valid JSON value. | | input | Option 1 | string | A scalar string value. | | input | Option 2 | double | A scalar numeric value. | | input | Option 3 | object | An arbitrary object value with additional properties. | | input | Option 4 | [ ] | An array of arbitrary values. | | Option 4 | Option 1 | string | A scalar string value. | | Option 4 | Option 2 | double | A scalar numeric value. | | Option 4 | Option 3 | object | An arbitrary object value with additional properties. | | Option 4 | Option 4 | boolean | A scalar boolean value. | | input | Option 5 | boolean | A scalar boolean value. | | records | metadata [*required*] | object | Arbitrary metadata associated with the record. | | records | updated_at [*required*] | date-time | Timestamp when the record was last updated. | {% /tab %} {% tab title="Example" %} ```json { "data": [ { "records": [ { "created_at": "2024-01-15T10:30:00Z", "dataset_id": "9f64e5c7-dc5a-45c8-a17c-1b85f0bec97d", "expected_output": { "description": "undefined", "type": "undefined" }, "id": "rec-7c3f5a1b-9e2d-4f8a-b1c6-3d7e9f0a2b4c", "input": { "description": "undefined", "type": "undefined" }, "metadata": {}, "updated_at": "2024-01-15T10:30:00Z" } ] } ] } ``` {% /tab %} {% /tab %} {% tab title="201" %} Created {% tab title="Model" %} Response containing records after a create or update operation. | Parent field | Field | Type | Description | | --------------- | --------------------------------- | ------------------- | ----------------------------------------------------- | | | data [*required*] | [object] | List of affected dataset records. | | data | records [*required*] | [object] | List of affected dataset records. | | records | created_at [*required*] | date-time | Timestamp when the record was created. | | records | dataset_id [*required*] | string | Identifier of the dataset this record belongs to. | | records | expected_output [*required*] | object | Represents any valid JSON value. | | expected_output | Option 1 | string | A scalar string value. | | expected_output | Option 2 | double | A scalar numeric value. | | expected_output | Option 3 | object | An arbitrary object value with additional properties. | | expected_output | Option 4 | [ ] | An array of arbitrary values. | | Option 4 | Option 1 | string | A scalar string value. | | Option 4 | Option 2 | double | A scalar numeric value. | | Option 4 | Option 3 | object | An arbitrary object value with additional properties. | | Option 4 | Option 4 | boolean | A scalar boolean value. | | expected_output | Option 5 | boolean | A scalar boolean value. | | records | id [*required*] | string | Unique identifier of the record. | | records | input [*required*] | object | Represents any valid JSON value. | | input | Option 1 | string | A scalar string value. | | input | Option 2 | double | A scalar numeric value. | | input | Option 3 | object | An arbitrary object value with additional properties. | | input | Option 4 | [ ] | An array of arbitrary values. | | Option 4 | Option 1 | string | A scalar string value. | | Option 4 | Option 2 | double | A scalar numeric value. | | Option 4 | Option 3 | object | An arbitrary object value with additional properties. | | Option 4 | Option 4 | boolean | A scalar boolean value. | | input | Option 5 | boolean | A scalar boolean value. | | records | metadata [*required*] | object | Arbitrary metadata associated with the record. | | records | updated_at [*required*] | date-time | Timestamp when the record was last updated. | {% /tab %} {% tab title="Example" %} ```json { "data": [ { "records": [ { "created_at": "2024-01-15T10:30:00Z", "dataset_id": "9f64e5c7-dc5a-45c8-a17c-1b85f0bec97d", "expected_output": { "description": "undefined", "type": "undefined" }, "id": "rec-7c3f5a1b-9e2d-4f8a-b1c6-3d7e9f0a2b4c", "input": { "description": "undefined", "type": "undefined" }, "metadata": {}, "updated_at": "2024-01-15T10:30:00Z" } ] } ] } ``` {% /tab %} {% /tab %} {% tab title="400" %} Bad Request {% tab title="Model" %} API error response. | Parent field | Field | Type | Description | | ------------ | ------------------------ | -------- | ------------------------------------------------------------------------------- | | | errors [*required*] | [object] | A list of errors. | | errors | detail | string | A human-readable explanation specific to this occurrence of the error. | | errors | meta | object | Non-standard meta-information about the error | | errors | source | object | References to the source of the error. | | source | header | string | A string indicating the name of a single request header which caused the error. | | source | parameter | string | A string indicating which URI query parameter caused the error. | | source | pointer | string | A JSON pointer to the value in the request document that caused the error. | | errors | status | string | Status code of the response. | | errors | title | string | Short human-readable summary of the error. | {% /tab %} {% tab title="Example" %} ```json { "errors": [ { "detail": "Missing required attribute in body", "meta": {}, "source": { "header": "Authorization", "parameter": "limit", "pointer": "/data/attributes/title" }, "status": "400", "title": "Bad Request" } ] } ``` {% /tab %} {% /tab %} {% tab title="401" %} Unauthorized {% tab title="Model" %} API error response. | Parent field | Field | Type | Description | | ------------ | ------------------------ | -------- | ------------------------------------------------------------------------------- | | | errors [*required*] | [object] | A list of errors. | | errors | detail | string | A human-readable explanation specific to this occurrence of the error. | | errors | meta | object | Non-standard meta-information about the error | | errors | source | object | References to the source of the error. | | source | header | string | A string indicating the name of a single request header which caused the error. | | source | parameter | string | A string indicating which URI query parameter caused the error. | | source | pointer | string | A JSON pointer to the value in the request document that caused the error. | | errors | status | string | Status code of the response. | | errors | title | string | Short human-readable summary of the error. | {% /tab %} {% tab title="Example" %} ```json { "errors": [ { "detail": "Missing required attribute in body", "meta": {}, "source": { "header": "Authorization", "parameter": "limit", "pointer": "/data/attributes/title" }, "status": "400", "title": "Bad Request" } ] } ``` {% /tab %} {% /tab %} {% tab title="403" %} Forbidden {% tab title="Model" %} API error response. | Parent field | Field | Type | Description | | ------------ | ------------------------ | -------- | ------------------------------------------------------------------------------- | | | errors [*required*] | [object] | A list of errors. | | errors | detail | string | A human-readable explanation specific to this occurrence of the error. | | errors | meta | object | Non-standard meta-information about the error | | errors | source | object | References to the source of the error. | | source | header | string | A string indicating the name of a single request header which caused the error. | | source | parameter | string | A string indicating which URI query parameter caused the error. | | source | pointer | string | A JSON pointer to the value in the request document that caused the error. | | errors | status | string | Status code of the response. | | errors | title | string | Short human-readable summary of the error. | {% /tab %} {% tab title="Example" %} ```json { "errors": [ { "detail": "Missing required attribute in body", "meta": {}, "source": { "header": "Authorization", "parameter": "limit", "pointer": "/data/attributes/title" }, "status": "400", "title": "Bad Request" } ] } ``` {% /tab %} {% /tab %} {% tab title="404" %} Not Found {% tab title="Model" %} API error response. | Parent field | Field | Type | Description | | ------------ | ------------------------ | -------- | ------------------------------------------------------------------------------- | | | errors [*required*] | [object] | A list of errors. | | errors | detail | string | A human-readable explanation specific to this occurrence of the error. | | errors | meta | object | Non-standard meta-information about the error | | errors | source | object | References to the source of the error. | | source | header | string | A string indicating the name of a single request header which caused the error. | | source | parameter | string | A string indicating which URI query parameter caused the error. | | source | pointer | string | A JSON pointer to the value in the request document that caused the error. | | errors | status | string | Status code of the response. | | errors | title | string | Short human-readable summary of the error. | {% /tab %} {% tab title="Example" %} ```json { "errors": [ { "detail": "Missing required attribute in body", "meta": {}, "source": { "header": "Authorization", "parameter": "limit", "pointer": "/data/attributes/title" }, "status": "400", "title": "Bad Request" } ] } ``` {% /tab %} {% /tab %} {% tab title="429" %} Too many requests {% tab title="Model" %} API error response. | Field | Type | Description | | ------------------------ | -------- | ----------------- | | errors [*required*] | [string] | A list of errors. | {% /tab %} {% tab title="Example" %} ```json { "errors": [ "Bad Request" ] } ``` {% /tab %} {% /tab %} ### Code Example ##### \# Path parametersexport project_id="a33671aa-24fd-4dcd-9b33-a8ec7dde7751"export dataset_id="9f64e5c7-dc5a-45c8-a17c-1b85f0bec97d"\# Curl commandcurl -X POST "https://api.ap1.datadoghq.com"https://api.ap2.datadoghq.com"https://api.datadoghq.eu"https://api.ddog-gov.com"https://api.datadoghq.com"https://api.us3.datadoghq.com"https://api.us5.datadoghq.com/api/v2/llm-obs/v1/${project_id}/datasets/${dataset_id}/records" \ -H "Accept: application/json" \ -H "Content-Type: application/json" \ -H "DD-API-KEY: ${DD_API_KEY}" \ -H "DD-APPLICATION-KEY: ${DD_APP_KEY}" \ -d @- << EOF { "data": { "attributes": { "records": [ { "input": {} } ] }, "type": "records" } } EOF {% /tab %} ## Update an LLM Observability dataset{% #update-an-llm-observability-dataset %} {% tab title="v2" %} **Note**: This endpoint is in preview and is subject to change. If you have any feedback, contact [Datadog support](https://docs.datadoghq.com/help/). | Datadog site | API endpoint | | ----------------- | ---------------------------------------------------------------------------------------- | | ap1.datadoghq.com | PATCH https://api.ap1.datadoghq.com/api/v2/llm-obs/v1/{project_id}/datasets/{dataset_id} | | ap2.datadoghq.com | PATCH https://api.ap2.datadoghq.com/api/v2/llm-obs/v1/{project_id}/datasets/{dataset_id} | | app.datadoghq.eu | PATCH https://api.datadoghq.eu/api/v2/llm-obs/v1/{project_id}/datasets/{dataset_id} | | app.ddog-gov.com | PATCH https://api.ddog-gov.com/api/v2/llm-obs/v1/{project_id}/datasets/{dataset_id} | | app.datadoghq.com | PATCH https://api.datadoghq.com/api/v2/llm-obs/v1/{project_id}/datasets/{dataset_id} | | us3.datadoghq.com | PATCH https://api.us3.datadoghq.com/api/v2/llm-obs/v1/{project_id}/datasets/{dataset_id} | | us5.datadoghq.com | PATCH https://api.us5.datadoghq.com/api/v2/llm-obs/v1/{project_id}/datasets/{dataset_id} | ### Overview Partially update an existing LLM Observability dataset within the specified project. ### Arguments #### Path Parameters | Name | Type | Description | | ---------------------------- | ------ | ---------------------------------------- | | project_id [*required*] | string | The ID of the LLM Observability project. | | dataset_id [*required*] | string | The ID of the LLM Observability dataset. | ### Request #### Body Data (required) Update dataset payload. {% tab title="Model" %} | Parent field | Field | Type | Description | | ------------ | ---------------------------- | ------ | ------------------------------------------------------------------------------ | | | data [*required*] | object | Data object for updating an LLM Observability dataset. | | data | attributes [*required*] | object | Attributes for updating an LLM Observability dataset. | | attributes | description | string | Updated description of the dataset. | | attributes | metadata | object | Updated metadata associated with the dataset. | | attributes | name | string | Updated name of the dataset. | | data | type [*required*] | enum | Resource type of an LLM Observability dataset. Allowed enum values: `datasets` | {% /tab %} {% tab title="Example" %} ```json { "data": { "attributes": { "description": "string", "metadata": {}, "name": "string" }, "type": "datasets" } } ``` {% /tab %} ### Response {% tab title="200" %} OK {% tab title="Model" %} Response containing a single LLM Observability dataset. | Parent field | Field | Type | Description | | ------------ | --------------------------------- | --------- | ------------------------------------------------------------------------------ | | | data [*required*] | object | Data object for an LLM Observability dataset. | | data | attributes [*required*] | object | Attributes of an LLM Observability dataset. | | attributes | created_at [*required*] | date-time | Timestamp when the dataset was created. | | attributes | current_version [*required*] | int64 | Current version number of the dataset. | | attributes | description [*required*] | string | Description of the dataset. | | attributes | metadata [*required*] | object | Arbitrary metadata associated with the dataset. | | attributes | name [*required*] | string | Name of the dataset. | | attributes | updated_at [*required*] | date-time | Timestamp when the dataset was last updated. | | data | id [*required*] | string | Unique identifier of the dataset. | | data | type [*required*] | enum | Resource type of an LLM Observability dataset. Allowed enum values: `datasets` | {% /tab %} {% tab title="Example" %} ```json { "data": { "attributes": { "created_at": "2024-01-15T10:30:00Z", "current_version": 1, "description": "", "metadata": {}, "name": "My LLM Dataset", "updated_at": "2024-01-15T10:30:00Z" }, "id": "9f64e5c7-dc5a-45c8-a17c-1b85f0bec97d", "type": "datasets" } } ``` {% /tab %} {% /tab %} {% tab title="400" %} Bad Request {% tab title="Model" %} API error response. | Parent field | Field | Type | Description | | ------------ | ------------------------ | -------- | ------------------------------------------------------------------------------- | | | errors [*required*] | [object] | A list of errors. | | errors | detail | string | A human-readable explanation specific to this occurrence of the error. | | errors | meta | object | Non-standard meta-information about the error | | errors | source | object | References to the source of the error. | | source | header | string | A string indicating the name of a single request header which caused the error. | | source | parameter | string | A string indicating which URI query parameter caused the error. | | source | pointer | string | A JSON pointer to the value in the request document that caused the error. | | errors | status | string | Status code of the response. | | errors | title | string | Short human-readable summary of the error. | {% /tab %} {% tab title="Example" %} ```json { "errors": [ { "detail": "Missing required attribute in body", "meta": {}, "source": { "header": "Authorization", "parameter": "limit", "pointer": "/data/attributes/title" }, "status": "400", "title": "Bad Request" } ] } ``` {% /tab %} {% /tab %} {% tab title="401" %} Unauthorized {% tab title="Model" %} API error response. | Parent field | Field | Type | Description | | ------------ | ------------------------ | -------- | ------------------------------------------------------------------------------- | | | errors [*required*] | [object] | A list of errors. | | errors | detail | string | A human-readable explanation specific to this occurrence of the error. | | errors | meta | object | Non-standard meta-information about the error | | errors | source | object | References to the source of the error. | | source | header | string | A string indicating the name of a single request header which caused the error. | | source | parameter | string | A string indicating which URI query parameter caused the error. | | source | pointer | string | A JSON pointer to the value in the request document that caused the error. | | errors | status | string | Status code of the response. | | errors | title | string | Short human-readable summary of the error. | {% /tab %} {% tab title="Example" %} ```json { "errors": [ { "detail": "Missing required attribute in body", "meta": {}, "source": { "header": "Authorization", "parameter": "limit", "pointer": "/data/attributes/title" }, "status": "400", "title": "Bad Request" } ] } ``` {% /tab %} {% /tab %} {% tab title="403" %} Forbidden {% tab title="Model" %} API error response. | Parent field | Field | Type | Description | | ------------ | ------------------------ | -------- | ------------------------------------------------------------------------------- | | | errors [*required*] | [object] | A list of errors. | | errors | detail | string | A human-readable explanation specific to this occurrence of the error. | | errors | meta | object | Non-standard meta-information about the error | | errors | source | object | References to the source of the error. | | source | header | string | A string indicating the name of a single request header which caused the error. | | source | parameter | string | A string indicating which URI query parameter caused the error. | | source | pointer | string | A JSON pointer to the value in the request document that caused the error. | | errors | status | string | Status code of the response. | | errors | title | string | Short human-readable summary of the error. | {% /tab %} {% tab title="Example" %} ```json { "errors": [ { "detail": "Missing required attribute in body", "meta": {}, "source": { "header": "Authorization", "parameter": "limit", "pointer": "/data/attributes/title" }, "status": "400", "title": "Bad Request" } ] } ``` {% /tab %} {% /tab %} {% tab title="404" %} Not Found {% tab title="Model" %} API error response. | Parent field | Field | Type | Description | | ------------ | ------------------------ | -------- | ------------------------------------------------------------------------------- | | | errors [*required*] | [object] | A list of errors. | | errors | detail | string | A human-readable explanation specific to this occurrence of the error. | | errors | meta | object | Non-standard meta-information about the error | | errors | source | object | References to the source of the error. | | source | header | string | A string indicating the name of a single request header which caused the error. | | source | parameter | string | A string indicating which URI query parameter caused the error. | | source | pointer | string | A JSON pointer to the value in the request document that caused the error. | | errors | status | string | Status code of the response. | | errors | title | string | Short human-readable summary of the error. | {% /tab %} {% tab title="Example" %} ```json { "errors": [ { "detail": "Missing required attribute in body", "meta": {}, "source": { "header": "Authorization", "parameter": "limit", "pointer": "/data/attributes/title" }, "status": "400", "title": "Bad Request" } ] } ``` {% /tab %} {% /tab %} {% tab title="429" %} Too many requests {% tab title="Model" %} API error response. | Field | Type | Description | | ------------------------ | -------- | ----------------- | | errors [*required*] | [string] | A list of errors. | {% /tab %} {% tab title="Example" %} ```json { "errors": [ "Bad Request" ] } ``` {% /tab %} {% /tab %} ### Code Example ##### \# Path parametersexport project_id="a33671aa-24fd-4dcd-9b33-a8ec7dde7751"export dataset_id="9f64e5c7-dc5a-45c8-a17c-1b85f0bec97d"\# Curl commandcurl -X PATCH "https://api.ap1.datadoghq.com"https://api.ap2.datadoghq.com"https://api.datadoghq.eu"https://api.ddog-gov.com"https://api.datadoghq.com"https://api.us3.datadoghq.com"https://api.us5.datadoghq.com/api/v2/llm-obs/v1/${project_id}/datasets/${dataset_id}" \ -H "Accept: application/json" \ -H "Content-Type: application/json" \ -H "DD-API-KEY: ${DD_API_KEY}" \ -H "DD-APPLICATION-KEY: ${DD_APP_KEY}" \ -d @- << EOF { "data": { "attributes": {}, "type": "datasets" } } EOF {% /tab %} ## Update LLM Observability dataset records{% #update-llm-observability-dataset-records %} {% tab title="v2" %} **Note**: This endpoint is in preview and is subject to change. If you have any feedback, contact [Datadog support](https://docs.datadoghq.com/help/). | Datadog site | API endpoint | | ----------------- | ------------------------------------------------------------------------------------------------ | | ap1.datadoghq.com | PATCH https://api.ap1.datadoghq.com/api/v2/llm-obs/v1/{project_id}/datasets/{dataset_id}/records | | ap2.datadoghq.com | PATCH https://api.ap2.datadoghq.com/api/v2/llm-obs/v1/{project_id}/datasets/{dataset_id}/records | | app.datadoghq.eu | PATCH https://api.datadoghq.eu/api/v2/llm-obs/v1/{project_id}/datasets/{dataset_id}/records | | app.ddog-gov.com | PATCH https://api.ddog-gov.com/api/v2/llm-obs/v1/{project_id}/datasets/{dataset_id}/records | | app.datadoghq.com | PATCH https://api.datadoghq.com/api/v2/llm-obs/v1/{project_id}/datasets/{dataset_id}/records | | us3.datadoghq.com | PATCH https://api.us3.datadoghq.com/api/v2/llm-obs/v1/{project_id}/datasets/{dataset_id}/records | | us5.datadoghq.com | PATCH https://api.us5.datadoghq.com/api/v2/llm-obs/v1/{project_id}/datasets/{dataset_id}/records | ### Overview Update one or more existing records in an LLM Observability dataset. ### Arguments #### Path Parameters | Name | Type | Description | | ---------------------------- | ------ | ---------------------------------------- | | project_id [*required*] | string | The ID of the LLM Observability project. | | dataset_id [*required*] | string | The ID of the LLM Observability dataset. | ### Request #### Body Data (required) Update records payload. {% tab title="Model" %} | Parent field | Field | Type | Description | | --------------- | ---------------------------- | ------------------- | ---------------------------------------------------------------------------------- | | | data [*required*] | object | Data object for updating records in an LLM Observability dataset. | | data | attributes [*required*] | object | Attributes for updating records in an LLM Observability dataset. | | attributes | records [*required*] | [object] | List of records to update. | | records | expected_output | object | Represents any valid JSON value. | | expected_output | Option 1 | string | A scalar string value. | | expected_output | Option 2 | double | A scalar numeric value. | | expected_output | Option 3 | object | An arbitrary object value with additional properties. | | expected_output | Option 4 | [ ] | An array of arbitrary values. | | Option 4 | Option 1 | string | A scalar string value. | | Option 4 | Option 2 | double | A scalar numeric value. | | Option 4 | Option 3 | object | An arbitrary object value with additional properties. | | Option 4 | Option 4 | boolean | A scalar boolean value. | | expected_output | Option 5 | boolean | A scalar boolean value. | | records | id [*required*] | string | Unique identifier of the record to update. | | records | input | object | Represents any valid JSON value. | | input | Option 1 | string | A scalar string value. | | input | Option 2 | double | A scalar numeric value. | | input | Option 3 | object | An arbitrary object value with additional properties. | | input | Option 4 | [ ] | An array of arbitrary values. | | Option 4 | Option 1 | string | A scalar string value. | | Option 4 | Option 2 | double | A scalar numeric value. | | Option 4 | Option 3 | object | An arbitrary object value with additional properties. | | Option 4 | Option 4 | boolean | A scalar boolean value. | | input | Option 5 | boolean | A scalar boolean value. | | records | metadata | object | Updated metadata associated with the record. | | data | type [*required*] | enum | Resource type of LLM Observability dataset records. Allowed enum values: `records` | {% /tab %} {% tab title="Example" %} ```json { "data": { "attributes": { "records": [ { "expected_output": { "description": "undefined", "type": "undefined" }, "id": "rec-7c3f5a1b-9e2d-4f8a-b1c6-3d7e9f0a2b4c", "input": { "description": "undefined", "type": "undefined" }, "metadata": {} } ] }, "type": "records" } } ``` {% /tab %} ### Response {% tab title="200" %} OK {% tab title="Model" %} Response containing records after a create or update operation. | Parent field | Field | Type | Description | | --------------- | --------------------------------- | ------------------- | ----------------------------------------------------- | | | data [*required*] | [object] | List of affected dataset records. | | data | records [*required*] | [object] | List of affected dataset records. | | records | created_at [*required*] | date-time | Timestamp when the record was created. | | records | dataset_id [*required*] | string | Identifier of the dataset this record belongs to. | | records | expected_output [*required*] | object | Represents any valid JSON value. | | expected_output | Option 1 | string | A scalar string value. | | expected_output | Option 2 | double | A scalar numeric value. | | expected_output | Option 3 | object | An arbitrary object value with additional properties. | | expected_output | Option 4 | [ ] | An array of arbitrary values. | | Option 4 | Option 1 | string | A scalar string value. | | Option 4 | Option 2 | double | A scalar numeric value. | | Option 4 | Option 3 | object | An arbitrary object value with additional properties. | | Option 4 | Option 4 | boolean | A scalar boolean value. | | expected_output | Option 5 | boolean | A scalar boolean value. | | records | id [*required*] | string | Unique identifier of the record. | | records | input [*required*] | object | Represents any valid JSON value. | | input | Option 1 | string | A scalar string value. | | input | Option 2 | double | A scalar numeric value. | | input | Option 3 | object | An arbitrary object value with additional properties. | | input | Option 4 | [ ] | An array of arbitrary values. | | Option 4 | Option 1 | string | A scalar string value. | | Option 4 | Option 2 | double | A scalar numeric value. | | Option 4 | Option 3 | object | An arbitrary object value with additional properties. | | Option 4 | Option 4 | boolean | A scalar boolean value. | | input | Option 5 | boolean | A scalar boolean value. | | records | metadata [*required*] | object | Arbitrary metadata associated with the record. | | records | updated_at [*required*] | date-time | Timestamp when the record was last updated. | {% /tab %} {% tab title="Example" %} ```json { "data": [ { "records": [ { "created_at": "2024-01-15T10:30:00Z", "dataset_id": "9f64e5c7-dc5a-45c8-a17c-1b85f0bec97d", "expected_output": { "description": "undefined", "type": "undefined" }, "id": "rec-7c3f5a1b-9e2d-4f8a-b1c6-3d7e9f0a2b4c", "input": { "description": "undefined", "type": "undefined" }, "metadata": {}, "updated_at": "2024-01-15T10:30:00Z" } ] } ] } ``` {% /tab %} {% /tab %} {% tab title="400" %} Bad Request {% tab title="Model" %} API error response. | Parent field | Field | Type | Description | | ------------ | ------------------------ | -------- | ------------------------------------------------------------------------------- | | | errors [*required*] | [object] | A list of errors. | | errors | detail | string | A human-readable explanation specific to this occurrence of the error. | | errors | meta | object | Non-standard meta-information about the error | | errors | source | object | References to the source of the error. | | source | header | string | A string indicating the name of a single request header which caused the error. | | source | parameter | string | A string indicating which URI query parameter caused the error. | | source | pointer | string | A JSON pointer to the value in the request document that caused the error. | | errors | status | string | Status code of the response. | | errors | title | string | Short human-readable summary of the error. | {% /tab %} {% tab title="Example" %} ```json { "errors": [ { "detail": "Missing required attribute in body", "meta": {}, "source": { "header": "Authorization", "parameter": "limit", "pointer": "/data/attributes/title" }, "status": "400", "title": "Bad Request" } ] } ``` {% /tab %} {% /tab %} {% tab title="401" %} Unauthorized {% tab title="Model" %} API error response. | Parent field | Field | Type | Description | | ------------ | ------------------------ | -------- | ------------------------------------------------------------------------------- | | | errors [*required*] | [object] | A list of errors. | | errors | detail | string | A human-readable explanation specific to this occurrence of the error. | | errors | meta | object | Non-standard meta-information about the error | | errors | source | object | References to the source of the error. | | source | header | string | A string indicating the name of a single request header which caused the error. | | source | parameter | string | A string indicating which URI query parameter caused the error. | | source | pointer | string | A JSON pointer to the value in the request document that caused the error. | | errors | status | string | Status code of the response. | | errors | title | string | Short human-readable summary of the error. | {% /tab %} {% tab title="Example" %} ```json { "errors": [ { "detail": "Missing required attribute in body", "meta": {}, "source": { "header": "Authorization", "parameter": "limit", "pointer": "/data/attributes/title" }, "status": "400", "title": "Bad Request" } ] } ``` {% /tab %} {% /tab %} {% tab title="403" %} Forbidden {% tab title="Model" %} API error response. | Parent field | Field | Type | Description | | ------------ | ------------------------ | -------- | ------------------------------------------------------------------------------- | | | errors [*required*] | [object] | A list of errors. | | errors | detail | string | A human-readable explanation specific to this occurrence of the error. | | errors | meta | object | Non-standard meta-information about the error | | errors | source | object | References to the source of the error. | | source | header | string | A string indicating the name of a single request header which caused the error. | | source | parameter | string | A string indicating which URI query parameter caused the error. | | source | pointer | string | A JSON pointer to the value in the request document that caused the error. | | errors | status | string | Status code of the response. | | errors | title | string | Short human-readable summary of the error. | {% /tab %} {% tab title="Example" %} ```json { "errors": [ { "detail": "Missing required attribute in body", "meta": {}, "source": { "header": "Authorization", "parameter": "limit", "pointer": "/data/attributes/title" }, "status": "400", "title": "Bad Request" } ] } ``` {% /tab %} {% /tab %} {% tab title="404" %} Not Found {% tab title="Model" %} API error response. | Parent field | Field | Type | Description | | ------------ | ------------------------ | -------- | ------------------------------------------------------------------------------- | | | errors [*required*] | [object] | A list of errors. | | errors | detail | string | A human-readable explanation specific to this occurrence of the error. | | errors | meta | object | Non-standard meta-information about the error | | errors | source | object | References to the source of the error. | | source | header | string | A string indicating the name of a single request header which caused the error. | | source | parameter | string | A string indicating which URI query parameter caused the error. | | source | pointer | string | A JSON pointer to the value in the request document that caused the error. | | errors | status | string | Status code of the response. | | errors | title | string | Short human-readable summary of the error. | {% /tab %} {% tab title="Example" %} ```json { "errors": [ { "detail": "Missing required attribute in body", "meta": {}, "source": { "header": "Authorization", "parameter": "limit", "pointer": "/data/attributes/title" }, "status": "400", "title": "Bad Request" } ] } ``` {% /tab %} {% /tab %} {% tab title="429" %} Too many requests {% tab title="Model" %} API error response. | Field | Type | Description | | ------------------------ | -------- | ----------------- | | errors [*required*] | [string] | A list of errors. | {% /tab %} {% tab title="Example" %} ```json { "errors": [ "Bad Request" ] } ``` {% /tab %} {% /tab %} ### Code Example ##### \# Path parametersexport project_id="a33671aa-24fd-4dcd-9b33-a8ec7dde7751"export dataset_id="9f64e5c7-dc5a-45c8-a17c-1b85f0bec97d"\# Curl commandcurl -X PATCH "https://api.ap1.datadoghq.com"https://api.ap2.datadoghq.com"https://api.datadoghq.eu"https://api.ddog-gov.com"https://api.datadoghq.com"https://api.us3.datadoghq.com"https://api.us5.datadoghq.com/api/v2/llm-obs/v1/${project_id}/datasets/${dataset_id}/records" \ -H "Accept: application/json" \ -H "Content-Type: application/json" \ -H "DD-API-KEY: ${DD_API_KEY}" \ -H "DD-APPLICATION-KEY: ${DD_APP_KEY}" \ -d @- << EOF { "data": { "attributes": { "records": [ { "id": "rec-7c3f5a1b-9e2d-4f8a-b1c6-3d7e9f0a2b4c" } ] }, "type": "records" } } EOF {% /tab %} ## Delete LLM Observability datasets{% #delete-llm-observability-datasets %} {% tab title="v2" %} **Note**: This endpoint is in preview and is subject to change. If you have any feedback, contact [Datadog support](https://docs.datadoghq.com/help/). | Datadog site | API endpoint | | ----------------- | --------------------------------------------------------------------------------- | | ap1.datadoghq.com | POST https://api.ap1.datadoghq.com/api/v2/llm-obs/v1/{project_id}/datasets/delete | | ap2.datadoghq.com | POST https://api.ap2.datadoghq.com/api/v2/llm-obs/v1/{project_id}/datasets/delete | | app.datadoghq.eu | POST https://api.datadoghq.eu/api/v2/llm-obs/v1/{project_id}/datasets/delete | | app.ddog-gov.com | POST https://api.ddog-gov.com/api/v2/llm-obs/v1/{project_id}/datasets/delete | | app.datadoghq.com | POST https://api.datadoghq.com/api/v2/llm-obs/v1/{project_id}/datasets/delete | | us3.datadoghq.com | POST https://api.us3.datadoghq.com/api/v2/llm-obs/v1/{project_id}/datasets/delete | | us5.datadoghq.com | POST https://api.us5.datadoghq.com/api/v2/llm-obs/v1/{project_id}/datasets/delete | ### Overview Delete one or more LLM Observability datasets within the specified project. ### Arguments #### Path Parameters | Name | Type | Description | | ---------------------------- | ------ | ---------------------------------------- | | project_id [*required*] | string | The ID of the LLM Observability project. | ### Request #### Body Data (required) Delete datasets payload. {% tab title="Model" %} | Parent field | Field | Type | Description | | ------------ | ----------------------------- | -------- | ------------------------------------------------------------------------------ | | | data [*required*] | object | Data object for deleting LLM Observability datasets. | | data | attributes [*required*] | object | Attributes for deleting LLM Observability datasets. | | attributes | dataset_ids [*required*] | [string] | List of dataset IDs to delete. | | data | type [*required*] | enum | Resource type of an LLM Observability dataset. Allowed enum values: `datasets` | {% /tab %} {% tab title="Example" %} ```json { "data": { "attributes": { "dataset_ids": [ "9f64e5c7-dc5a-45c8-a17c-1b85f0bec97d" ] }, "type": "datasets" } } ``` {% /tab %} ### Response {% tab title="204" %} No Content {% /tab %} {% tab title="400" %} Bad Request {% tab title="Model" %} API error response. | Parent field | Field | Type | Description | | ------------ | ------------------------ | -------- | ------------------------------------------------------------------------------- | | | errors [*required*] | [object] | A list of errors. | | errors | detail | string | A human-readable explanation specific to this occurrence of the error. | | errors | meta | object | Non-standard meta-information about the error | | errors | source | object | References to the source of the error. | | source | header | string | A string indicating the name of a single request header which caused the error. | | source | parameter | string | A string indicating which URI query parameter caused the error. | | source | pointer | string | A JSON pointer to the value in the request document that caused the error. | | errors | status | string | Status code of the response. | | errors | title | string | Short human-readable summary of the error. | {% /tab %} {% tab title="Example" %} ```json { "errors": [ { "detail": "Missing required attribute in body", "meta": {}, "source": { "header": "Authorization", "parameter": "limit", "pointer": "/data/attributes/title" }, "status": "400", "title": "Bad Request" } ] } ``` {% /tab %} {% /tab %} {% tab title="401" %} Unauthorized {% tab title="Model" %} API error response. | Parent field | Field | Type | Description | | ------------ | ------------------------ | -------- | ------------------------------------------------------------------------------- | | | errors [*required*] | [object] | A list of errors. | | errors | detail | string | A human-readable explanation specific to this occurrence of the error. | | errors | meta | object | Non-standard meta-information about the error | | errors | source | object | References to the source of the error. | | source | header | string | A string indicating the name of a single request header which caused the error. | | source | parameter | string | A string indicating which URI query parameter caused the error. | | source | pointer | string | A JSON pointer to the value in the request document that caused the error. | | errors | status | string | Status code of the response. | | errors | title | string | Short human-readable summary of the error. | {% /tab %} {% tab title="Example" %} ```json { "errors": [ { "detail": "Missing required attribute in body", "meta": {}, "source": { "header": "Authorization", "parameter": "limit", "pointer": "/data/attributes/title" }, "status": "400", "title": "Bad Request" } ] } ``` {% /tab %} {% /tab %} {% tab title="403" %} Forbidden {% tab title="Model" %} API error response. | Parent field | Field | Type | Description | | ------------ | ------------------------ | -------- | ------------------------------------------------------------------------------- | | | errors [*required*] | [object] | A list of errors. | | errors | detail | string | A human-readable explanation specific to this occurrence of the error. | | errors | meta | object | Non-standard meta-information about the error | | errors | source | object | References to the source of the error. | | source | header | string | A string indicating the name of a single request header which caused the error. | | source | parameter | string | A string indicating which URI query parameter caused the error. | | source | pointer | string | A JSON pointer to the value in the request document that caused the error. | | errors | status | string | Status code of the response. | | errors | title | string | Short human-readable summary of the error. | {% /tab %} {% tab title="Example" %} ```json { "errors": [ { "detail": "Missing required attribute in body", "meta": {}, "source": { "header": "Authorization", "parameter": "limit", "pointer": "/data/attributes/title" }, "status": "400", "title": "Bad Request" } ] } ``` {% /tab %} {% /tab %} {% tab title="404" %} Not Found {% tab title="Model" %} API error response. | Parent field | Field | Type | Description | | ------------ | ------------------------ | -------- | ------------------------------------------------------------------------------- | | | errors [*required*] | [object] | A list of errors. | | errors | detail | string | A human-readable explanation specific to this occurrence of the error. | | errors | meta | object | Non-standard meta-information about the error | | errors | source | object | References to the source of the error. | | source | header | string | A string indicating the name of a single request header which caused the error. | | source | parameter | string | A string indicating which URI query parameter caused the error. | | source | pointer | string | A JSON pointer to the value in the request document that caused the error. | | errors | status | string | Status code of the response. | | errors | title | string | Short human-readable summary of the error. | {% /tab %} {% tab title="Example" %} ```json { "errors": [ { "detail": "Missing required attribute in body", "meta": {}, "source": { "header": "Authorization", "parameter": "limit", "pointer": "/data/attributes/title" }, "status": "400", "title": "Bad Request" } ] } ``` {% /tab %} {% /tab %} {% tab title="429" %} Too many requests {% tab title="Model" %} API error response. | Field | Type | Description | | ------------------------ | -------- | ----------------- | | errors [*required*] | [string] | A list of errors. | {% /tab %} {% tab title="Example" %} ```json { "errors": [ "Bad Request" ] } ``` {% /tab %} {% /tab %} ### Code Example ##### \# Path parametersexport project_id="a33671aa-24fd-4dcd-9b33-a8ec7dde7751"\# Curl commandcurl -X POST "https://api.ap1.datadoghq.com"https://api.ap2.datadoghq.com"https://api.datadoghq.eu"https://api.ddog-gov.com"https://api.datadoghq.com"https://api.us3.datadoghq.com"https://api.us5.datadoghq.com/api/v2/llm-obs/v1/${project_id}/datasets/delete" \ -H "Content-Type: application/json" \ -H "DD-API-KEY: ${DD_API_KEY}" \ -H "DD-APPLICATION-KEY: ${DD_APP_KEY}" \ -d @- << EOF { "data": { "attributes": { "dataset_ids": [ "9f64e5c7-dc5a-45c8-a17c-1b85f0bec97d" ] }, "type": "datasets" } } EOF {% /tab %} ## Delete LLM Observability dataset records{% #delete-llm-observability-dataset-records %} {% tab title="v2" %} **Note**: This endpoint is in preview and is subject to change. If you have any feedback, contact [Datadog support](https://docs.datadoghq.com/help/). | Datadog site | API endpoint | | ----------------- | ------------------------------------------------------------------------------------------------------ | | ap1.datadoghq.com | POST https://api.ap1.datadoghq.com/api/v2/llm-obs/v1/{project_id}/datasets/{dataset_id}/records/delete | | ap2.datadoghq.com | POST https://api.ap2.datadoghq.com/api/v2/llm-obs/v1/{project_id}/datasets/{dataset_id}/records/delete | | app.datadoghq.eu | POST https://api.datadoghq.eu/api/v2/llm-obs/v1/{project_id}/datasets/{dataset_id}/records/delete | | app.ddog-gov.com | POST https://api.ddog-gov.com/api/v2/llm-obs/v1/{project_id}/datasets/{dataset_id}/records/delete | | app.datadoghq.com | POST https://api.datadoghq.com/api/v2/llm-obs/v1/{project_id}/datasets/{dataset_id}/records/delete | | us3.datadoghq.com | POST https://api.us3.datadoghq.com/api/v2/llm-obs/v1/{project_id}/datasets/{dataset_id}/records/delete | | us5.datadoghq.com | POST https://api.us5.datadoghq.com/api/v2/llm-obs/v1/{project_id}/datasets/{dataset_id}/records/delete | ### Overview Delete one or more records from an LLM Observability dataset. ### Arguments #### Path Parameters | Name | Type | Description | | ---------------------------- | ------ | ---------------------------------------- | | project_id [*required*] | string | The ID of the LLM Observability project. | | dataset_id [*required*] | string | The ID of the LLM Observability dataset. | ### Request #### Body Data (required) Delete records payload. {% tab title="Model" %} | Parent field | Field | Type | Description | | ------------ | ---------------------------- | -------- | ---------------------------------------------------------------------------------- | | | data [*required*] | object | Data object for deleting records from an LLM Observability dataset. | | data | attributes [*required*] | object | Attributes for deleting records from an LLM Observability dataset. | | attributes | record_ids [*required*] | [string] | List of record IDs to delete. | | data | type [*required*] | enum | Resource type of LLM Observability dataset records. Allowed enum values: `records` | {% /tab %} {% tab title="Example" %} ```json { "data": { "attributes": { "record_ids": [ "rec-7c3f5a1b-9e2d-4f8a-b1c6-3d7e9f0a2b4c" ] }, "type": "records" } } ``` {% /tab %} ### Response {% tab title="204" %} No Content {% /tab %} {% tab title="400" %} Bad Request {% tab title="Model" %} API error response. | Parent field | Field | Type | Description | | ------------ | ------------------------ | -------- | ------------------------------------------------------------------------------- | | | errors [*required*] | [object] | A list of errors. | | errors | detail | string | A human-readable explanation specific to this occurrence of the error. | | errors | meta | object | Non-standard meta-information about the error | | errors | source | object | References to the source of the error. | | source | header | string | A string indicating the name of a single request header which caused the error. | | source | parameter | string | A string indicating which URI query parameter caused the error. | | source | pointer | string | A JSON pointer to the value in the request document that caused the error. | | errors | status | string | Status code of the response. | | errors | title | string | Short human-readable summary of the error. | {% /tab %} {% tab title="Example" %} ```json { "errors": [ { "detail": "Missing required attribute in body", "meta": {}, "source": { "header": "Authorization", "parameter": "limit", "pointer": "/data/attributes/title" }, "status": "400", "title": "Bad Request" } ] } ``` {% /tab %} {% /tab %} {% tab title="401" %} Unauthorized {% tab title="Model" %} API error response. | Parent field | Field | Type | Description | | ------------ | ------------------------ | -------- | ------------------------------------------------------------------------------- | | | errors [*required*] | [object] | A list of errors. | | errors | detail | string | A human-readable explanation specific to this occurrence of the error. | | errors | meta | object | Non-standard meta-information about the error | | errors | source | object | References to the source of the error. | | source | header | string | A string indicating the name of a single request header which caused the error. | | source | parameter | string | A string indicating which URI query parameter caused the error. | | source | pointer | string | A JSON pointer to the value in the request document that caused the error. | | errors | status | string | Status code of the response. | | errors | title | string | Short human-readable summary of the error. | {% /tab %} {% tab title="Example" %} ```json { "errors": [ { "detail": "Missing required attribute in body", "meta": {}, "source": { "header": "Authorization", "parameter": "limit", "pointer": "/data/attributes/title" }, "status": "400", "title": "Bad Request" } ] } ``` {% /tab %} {% /tab %} {% tab title="403" %} Forbidden {% tab title="Model" %} API error response. | Parent field | Field | Type | Description | | ------------ | ------------------------ | -------- | ------------------------------------------------------------------------------- | | | errors [*required*] | [object] | A list of errors. | | errors | detail | string | A human-readable explanation specific to this occurrence of the error. | | errors | meta | object | Non-standard meta-information about the error | | errors | source | object | References to the source of the error. | | source | header | string | A string indicating the name of a single request header which caused the error. | | source | parameter | string | A string indicating which URI query parameter caused the error. | | source | pointer | string | A JSON pointer to the value in the request document that caused the error. | | errors | status | string | Status code of the response. | | errors | title | string | Short human-readable summary of the error. | {% /tab %} {% tab title="Example" %} ```json { "errors": [ { "detail": "Missing required attribute in body", "meta": {}, "source": { "header": "Authorization", "parameter": "limit", "pointer": "/data/attributes/title" }, "status": "400", "title": "Bad Request" } ] } ``` {% /tab %} {% /tab %} {% tab title="404" %} Not Found {% tab title="Model" %} API error response. | Parent field | Field | Type | Description | | ------------ | ------------------------ | -------- | ------------------------------------------------------------------------------- | | | errors [*required*] | [object] | A list of errors. | | errors | detail | string | A human-readable explanation specific to this occurrence of the error. | | errors | meta | object | Non-standard meta-information about the error | | errors | source | object | References to the source of the error. | | source | header | string | A string indicating the name of a single request header which caused the error. | | source | parameter | string | A string indicating which URI query parameter caused the error. | | source | pointer | string | A JSON pointer to the value in the request document that caused the error. | | errors | status | string | Status code of the response. | | errors | title | string | Short human-readable summary of the error. | {% /tab %} {% tab title="Example" %} ```json { "errors": [ { "detail": "Missing required attribute in body", "meta": {}, "source": { "header": "Authorization", "parameter": "limit", "pointer": "/data/attributes/title" }, "status": "400", "title": "Bad Request" } ] } ``` {% /tab %} {% /tab %} {% tab title="429" %} Too many requests {% tab title="Model" %} API error response. | Field | Type | Description | | ------------------------ | -------- | ----------------- | | errors [*required*] | [string] | A list of errors. | {% /tab %} {% tab title="Example" %} ```json { "errors": [ "Bad Request" ] } ``` {% /tab %} {% /tab %} ### Code Example ##### \# Path parametersexport project_id="a33671aa-24fd-4dcd-9b33-a8ec7dde7751"export dataset_id="9f64e5c7-dc5a-45c8-a17c-1b85f0bec97d"\# Curl commandcurl -X POST "https://api.ap1.datadoghq.com"https://api.ap2.datadoghq.com"https://api.datadoghq.eu"https://api.ddog-gov.com"https://api.datadoghq.com"https://api.us3.datadoghq.com"https://api.us5.datadoghq.com/api/v2/llm-obs/v1/${project_id}/datasets/${dataset_id}/records/delete" \ -H "Content-Type: application/json" \ -H "DD-API-KEY: ${DD_API_KEY}" \ -H "DD-APPLICATION-KEY: ${DD_APP_KEY}" \ -d @- << EOF { "data": { "attributes": { "record_ids": [ "rec-7c3f5a1b-9e2d-4f8a-b1c6-3d7e9f0a2b4c" ] }, "type": "records" } } EOF {% /tab %} ## List LLM Observability experiments{% #list-llm-observability-experiments %} {% tab title="v2" %} **Note**: This endpoint is in preview and is subject to change. If you have any feedback, contact [Datadog support](https://docs.datadoghq.com/help/). | Datadog site | API endpoint | | ----------------- | --------------------------------------------------------------- | | ap1.datadoghq.com | GET https://api.ap1.datadoghq.com/api/v2/llm-obs/v1/experiments | | ap2.datadoghq.com | GET https://api.ap2.datadoghq.com/api/v2/llm-obs/v1/experiments | | app.datadoghq.eu | GET https://api.datadoghq.eu/api/v2/llm-obs/v1/experiments | | app.ddog-gov.com | GET https://api.ddog-gov.com/api/v2/llm-obs/v1/experiments | | app.datadoghq.com | GET https://api.datadoghq.com/api/v2/llm-obs/v1/experiments | | us3.datadoghq.com | GET https://api.us3.datadoghq.com/api/v2/llm-obs/v1/experiments | | us5.datadoghq.com | GET https://api.us5.datadoghq.com/api/v2/llm-obs/v1/experiments | ### Overview List all LLM Observability experiments sorted by creation date, newest first. ### Arguments #### Query Strings | Name | Type | Description | | ------------------ | ------- | ----------------------------------------------------------------------------------- | | filter[project_id] | string | Filter experiments by project ID. Required if `filter[dataset_id]` is not provided. | | filter[dataset_id] | string | Filter experiments by dataset ID. | | filter[id] | string | Filter experiments by experiment ID. Can be specified multiple times. | | page[cursor] | string | Use the Pagination cursor to retrieve the next page of results. | | page[limit] | integer | Maximum number of results to return per page. | ### Response {% tab title="200" %} OK {% tab title="Model" %} Response containing a list of LLM Observability experiments. | Parent field | Field | Type | Description | | ------------ | ----------------------------- | --------- | ------------------------------------------------------------------------------------ | | | data [*required*] | [object] | List of experiments. | | data | attributes [*required*] | object | Attributes of an LLM Observability experiment. | | attributes | config [*required*] | object | Configuration parameters for the experiment. | | attributes | created_at [*required*] | date-time | Timestamp when the experiment was created. | | attributes | dataset_id [*required*] | string | Identifier of the dataset used in this experiment. | | attributes | description [*required*] | string | Description of the experiment. | | attributes | metadata [*required*] | object | Arbitrary metadata associated with the experiment. | | attributes | name [*required*] | string | Name of the experiment. | | attributes | project_id [*required*] | string | Identifier of the project this experiment belongs to. | | attributes | updated_at [*required*] | date-time | Timestamp when the experiment was last updated. | | data | id [*required*] | string | Unique identifier of the experiment. | | data | type [*required*] | enum | Resource type of an LLM Observability experiment. Allowed enum values: `experiments` | | | meta | object | Pagination cursor metadata. | | meta | after | string | Cursor for the next page of results. | {% /tab %} {% tab title="Example" %} ```json { "data": [ { "attributes": { "config": {}, "created_at": "2024-01-15T10:30:00Z", "dataset_id": "9f64e5c7-dc5a-45c8-a17c-1b85f0bec97d", "description": "", "metadata": {}, "name": "My Experiment v1", "project_id": "a33671aa-24fd-4dcd-9b33-a8ec7dde7751", "updated_at": "2024-01-15T10:30:00Z" }, "id": "3fd6b5e0-8910-4b1c-a7d0-5b84de329012", "type": "experiments" } ], "meta": { "after": "string" } } ``` {% /tab %} {% /tab %} {% tab title="400" %} Bad Request {% tab title="Model" %} API error response. | Parent field | Field | Type | Description | | ------------ | ------------------------ | -------- | ------------------------------------------------------------------------------- | | | errors [*required*] | [object] | A list of errors. | | errors | detail | string | A human-readable explanation specific to this occurrence of the error. | | errors | meta | object | Non-standard meta-information about the error | | errors | source | object | References to the source of the error. | | source | header | string | A string indicating the name of a single request header which caused the error. | | source | parameter | string | A string indicating which URI query parameter caused the error. | | source | pointer | string | A JSON pointer to the value in the request document that caused the error. | | errors | status | string | Status code of the response. | | errors | title | string | Short human-readable summary of the error. | {% /tab %} {% tab title="Example" %} ```json { "errors": [ { "detail": "Missing required attribute in body", "meta": {}, "source": { "header": "Authorization", "parameter": "limit", "pointer": "/data/attributes/title" }, "status": "400", "title": "Bad Request" } ] } ``` {% /tab %} {% /tab %} {% tab title="401" %} Unauthorized {% tab title="Model" %} API error response. | Parent field | Field | Type | Description | | ------------ | ------------------------ | -------- | ------------------------------------------------------------------------------- | | | errors [*required*] | [object] | A list of errors. | | errors | detail | string | A human-readable explanation specific to this occurrence of the error. | | errors | meta | object | Non-standard meta-information about the error | | errors | source | object | References to the source of the error. | | source | header | string | A string indicating the name of a single request header which caused the error. | | source | parameter | string | A string indicating which URI query parameter caused the error. | | source | pointer | string | A JSON pointer to the value in the request document that caused the error. | | errors | status | string | Status code of the response. | | errors | title | string | Short human-readable summary of the error. | {% /tab %} {% tab title="Example" %} ```json { "errors": [ { "detail": "Missing required attribute in body", "meta": {}, "source": { "header": "Authorization", "parameter": "limit", "pointer": "/data/attributes/title" }, "status": "400", "title": "Bad Request" } ] } ``` {% /tab %} {% /tab %} {% tab title="403" %} Forbidden {% tab title="Model" %} API error response. | Parent field | Field | Type | Description | | ------------ | ------------------------ | -------- | ------------------------------------------------------------------------------- | | | errors [*required*] | [object] | A list of errors. | | errors | detail | string | A human-readable explanation specific to this occurrence of the error. | | errors | meta | object | Non-standard meta-information about the error | | errors | source | object | References to the source of the error. | | source | header | string | A string indicating the name of a single request header which caused the error. | | source | parameter | string | A string indicating which URI query parameter caused the error. | | source | pointer | string | A JSON pointer to the value in the request document that caused the error. | | errors | status | string | Status code of the response. | | errors | title | string | Short human-readable summary of the error. | {% /tab %} {% tab title="Example" %} ```json { "errors": [ { "detail": "Missing required attribute in body", "meta": {}, "source": { "header": "Authorization", "parameter": "limit", "pointer": "/data/attributes/title" }, "status": "400", "title": "Bad Request" } ] } ``` {% /tab %} {% /tab %} {% tab title="429" %} Too many requests {% tab title="Model" %} API error response. | Field | Type | Description | | ------------------------ | -------- | ----------------- | | errors [*required*] | [string] | A list of errors. | {% /tab %} {% tab title="Example" %} ```json { "errors": [ "Bad Request" ] } ``` {% /tab %} {% /tab %} ### Code Example ##### \# Curl commandcurl -X GET "https://api.ap1.datadoghq.com"https://api.ap2.datadoghq.com"https://api.datadoghq.eu"https://api.ddog-gov.com"https://api.datadoghq.com"https://api.us3.datadoghq.com"https://api.us5.datadoghq.com/api/v2/llm-obs/v1/experiments" \ -H "Accept: application/json" \ -H "DD-API-KEY: ${DD_API_KEY}" \ -H "DD-APPLICATION-KEY: ${DD_APP_KEY}" {% /tab %} ## Create an LLM Observability experiment{% #create-an-llm-observability-experiment %} {% tab title="v2" %} **Note**: This endpoint is in preview and is subject to change. If you have any feedback, contact [Datadog support](https://docs.datadoghq.com/help/). | Datadog site | API endpoint | | ----------------- | ---------------------------------------------------------------- | | ap1.datadoghq.com | POST https://api.ap1.datadoghq.com/api/v2/llm-obs/v1/experiments | | ap2.datadoghq.com | POST https://api.ap2.datadoghq.com/api/v2/llm-obs/v1/experiments | | app.datadoghq.eu | POST https://api.datadoghq.eu/api/v2/llm-obs/v1/experiments | | app.ddog-gov.com | POST https://api.ddog-gov.com/api/v2/llm-obs/v1/experiments | | app.datadoghq.com | POST https://api.datadoghq.com/api/v2/llm-obs/v1/experiments | | us3.datadoghq.com | POST https://api.us3.datadoghq.com/api/v2/llm-obs/v1/experiments | | us5.datadoghq.com | POST https://api.us5.datadoghq.com/api/v2/llm-obs/v1/experiments | ### Overview Create a new LLM Observability experiment. ### Request #### Body Data (required) Create experiment payload. {% tab title="Model" %} | Parent field | Field | Type | Description | | ------------ | ---------------------------- | ------- | ------------------------------------------------------------------------------------ | | | data [*required*] | object | Data object for creating an LLM Observability experiment. | | data | attributes [*required*] | object | Attributes for creating an LLM Observability experiment. | | attributes | config | object | Configuration parameters for the experiment. | | attributes | dataset_id [*required*] | string | Identifier of the dataset used in this experiment. | | attributes | dataset_version | int64 | Version of the dataset to use. Defaults to the current version if not specified. | | attributes | description | string | Description of the experiment. | | attributes | ensure_unique | boolean | Whether to ensure the experiment name is unique. Defaults to `true`. | | attributes | metadata | object | Arbitrary metadata associated with the experiment. | | attributes | name [*required*] | string | Name of the experiment. | | attributes | project_id [*required*] | string | Identifier of the project this experiment belongs to. | | data | type [*required*] | enum | Resource type of an LLM Observability experiment. Allowed enum values: `experiments` | {% /tab %} {% tab title="Example" %} ```json { "data": { "attributes": { "config": {}, "dataset_id": "9f64e5c7-dc5a-45c8-a17c-1b85f0bec97d", "dataset_version": "integer", "description": "string", "ensure_unique": false, "metadata": {}, "name": "My Experiment v1", "project_id": "a33671aa-24fd-4dcd-9b33-a8ec7dde7751" }, "type": "experiments" } } ``` {% /tab %} ### Response {% tab title="200" %} OK {% tab title="Model" %} Response containing a single LLM Observability experiment. | Parent field | Field | Type | Description | | ------------ | ----------------------------- | --------- | ------------------------------------------------------------------------------------ | | | data [*required*] | object | Data object for an LLM Observability experiment. | | data | attributes [*required*] | object | Attributes of an LLM Observability experiment. | | attributes | config [*required*] | object | Configuration parameters for the experiment. | | attributes | created_at [*required*] | date-time | Timestamp when the experiment was created. | | attributes | dataset_id [*required*] | string | Identifier of the dataset used in this experiment. | | attributes | description [*required*] | string | Description of the experiment. | | attributes | metadata [*required*] | object | Arbitrary metadata associated with the experiment. | | attributes | name [*required*] | string | Name of the experiment. | | attributes | project_id [*required*] | string | Identifier of the project this experiment belongs to. | | attributes | updated_at [*required*] | date-time | Timestamp when the experiment was last updated. | | data | id [*required*] | string | Unique identifier of the experiment. | | data | type [*required*] | enum | Resource type of an LLM Observability experiment. Allowed enum values: `experiments` | {% /tab %} {% tab title="Example" %} ```json { "data": { "attributes": { "config": {}, "created_at": "2024-01-15T10:30:00Z", "dataset_id": "9f64e5c7-dc5a-45c8-a17c-1b85f0bec97d", "description": "", "metadata": {}, "name": "My Experiment v1", "project_id": "a33671aa-24fd-4dcd-9b33-a8ec7dde7751", "updated_at": "2024-01-15T10:30:00Z" }, "id": "3fd6b5e0-8910-4b1c-a7d0-5b84de329012", "type": "experiments" } } ``` {% /tab %} {% /tab %} {% tab title="201" %} Created {% tab title="Model" %} Response containing a single LLM Observability experiment. | Parent field | Field | Type | Description | | ------------ | ----------------------------- | --------- | ------------------------------------------------------------------------------------ | | | data [*required*] | object | Data object for an LLM Observability experiment. | | data | attributes [*required*] | object | Attributes of an LLM Observability experiment. | | attributes | config [*required*] | object | Configuration parameters for the experiment. | | attributes | created_at [*required*] | date-time | Timestamp when the experiment was created. | | attributes | dataset_id [*required*] | string | Identifier of the dataset used in this experiment. | | attributes | description [*required*] | string | Description of the experiment. | | attributes | metadata [*required*] | object | Arbitrary metadata associated with the experiment. | | attributes | name [*required*] | string | Name of the experiment. | | attributes | project_id [*required*] | string | Identifier of the project this experiment belongs to. | | attributes | updated_at [*required*] | date-time | Timestamp when the experiment was last updated. | | data | id [*required*] | string | Unique identifier of the experiment. | | data | type [*required*] | enum | Resource type of an LLM Observability experiment. Allowed enum values: `experiments` | {% /tab %} {% tab title="Example" %} ```json { "data": { "attributes": { "config": {}, "created_at": "2024-01-15T10:30:00Z", "dataset_id": "9f64e5c7-dc5a-45c8-a17c-1b85f0bec97d", "description": "", "metadata": {}, "name": "My Experiment v1", "project_id": "a33671aa-24fd-4dcd-9b33-a8ec7dde7751", "updated_at": "2024-01-15T10:30:00Z" }, "id": "3fd6b5e0-8910-4b1c-a7d0-5b84de329012", "type": "experiments" } } ``` {% /tab %} {% /tab %} {% tab title="400" %} Bad Request {% tab title="Model" %} API error response. | Parent field | Field | Type | Description | | ------------ | ------------------------ | -------- | ------------------------------------------------------------------------------- | | | errors [*required*] | [object] | A list of errors. | | errors | detail | string | A human-readable explanation specific to this occurrence of the error. | | errors | meta | object | Non-standard meta-information about the error | | errors | source | object | References to the source of the error. | | source | header | string | A string indicating the name of a single request header which caused the error. | | source | parameter | string | A string indicating which URI query parameter caused the error. | | source | pointer | string | A JSON pointer to the value in the request document that caused the error. | | errors | status | string | Status code of the response. | | errors | title | string | Short human-readable summary of the error. | {% /tab %} {% tab title="Example" %} ```json { "errors": [ { "detail": "Missing required attribute in body", "meta": {}, "source": { "header": "Authorization", "parameter": "limit", "pointer": "/data/attributes/title" }, "status": "400", "title": "Bad Request" } ] } ``` {% /tab %} {% /tab %} {% tab title="401" %} Unauthorized {% tab title="Model" %} API error response. | Parent field | Field | Type | Description | | ------------ | ------------------------ | -------- | ------------------------------------------------------------------------------- | | | errors [*required*] | [object] | A list of errors. | | errors | detail | string | A human-readable explanation specific to this occurrence of the error. | | errors | meta | object | Non-standard meta-information about the error | | errors | source | object | References to the source of the error. | | source | header | string | A string indicating the name of a single request header which caused the error. | | source | parameter | string | A string indicating which URI query parameter caused the error. | | source | pointer | string | A JSON pointer to the value in the request document that caused the error. | | errors | status | string | Status code of the response. | | errors | title | string | Short human-readable summary of the error. | {% /tab %} {% tab title="Example" %} ```json { "errors": [ { "detail": "Missing required attribute in body", "meta": {}, "source": { "header": "Authorization", "parameter": "limit", "pointer": "/data/attributes/title" }, "status": "400", "title": "Bad Request" } ] } ``` {% /tab %} {% /tab %} {% tab title="403" %} Forbidden {% tab title="Model" %} API error response. | Parent field | Field | Type | Description | | ------------ | ------------------------ | -------- | ------------------------------------------------------------------------------- | | | errors [*required*] | [object] | A list of errors. | | errors | detail | string | A human-readable explanation specific to this occurrence of the error. | | errors | meta | object | Non-standard meta-information about the error | | errors | source | object | References to the source of the error. | | source | header | string | A string indicating the name of a single request header which caused the error. | | source | parameter | string | A string indicating which URI query parameter caused the error. | | source | pointer | string | A JSON pointer to the value in the request document that caused the error. | | errors | status | string | Status code of the response. | | errors | title | string | Short human-readable summary of the error. | {% /tab %} {% tab title="Example" %} ```json { "errors": [ { "detail": "Missing required attribute in body", "meta": {}, "source": { "header": "Authorization", "parameter": "limit", "pointer": "/data/attributes/title" }, "status": "400", "title": "Bad Request" } ] } ``` {% /tab %} {% /tab %} {% tab title="429" %} Too many requests {% tab title="Model" %} API error response. | Field | Type | Description | | ------------------------ | -------- | ----------------- | | errors [*required*] | [string] | A list of errors. | {% /tab %} {% tab title="Example" %} ```json { "errors": [ "Bad Request" ] } ``` {% /tab %} {% /tab %} ### Code Example ##### \# Curl commandcurl -X POST "https://api.ap1.datadoghq.com"https://api.ap2.datadoghq.com"https://api.datadoghq.eu"https://api.ddog-gov.com"https://api.datadoghq.com"https://api.us3.datadoghq.com"https://api.us5.datadoghq.com/api/v2/llm-obs/v1/experiments" \ -H "Accept: application/json" \ -H "Content-Type: application/json" \ -H "DD-API-KEY: ${DD_API_KEY}" \ -H "DD-APPLICATION-KEY: ${DD_APP_KEY}" \ -d @- << EOF { "data": { "attributes": { "dataset_id": "9f64e5c7-dc5a-45c8-a17c-1b85f0bec97d", "name": "My Experiment v1", "project_id": "a33671aa-24fd-4dcd-9b33-a8ec7dde7751" }, "type": "experiments" } } EOF {% /tab %} ## Update an LLM Observability experiment{% #update-an-llm-observability-experiment %} {% tab title="v2" %} **Note**: This endpoint is in preview and is subject to change. If you have any feedback, contact [Datadog support](https://docs.datadoghq.com/help/). | Datadog site | API endpoint | | ----------------- | --------------------------------------------------------------------------------- | | ap1.datadoghq.com | PATCH https://api.ap1.datadoghq.com/api/v2/llm-obs/v1/experiments/{experiment_id} | | ap2.datadoghq.com | PATCH https://api.ap2.datadoghq.com/api/v2/llm-obs/v1/experiments/{experiment_id} | | app.datadoghq.eu | PATCH https://api.datadoghq.eu/api/v2/llm-obs/v1/experiments/{experiment_id} | | app.ddog-gov.com | PATCH https://api.ddog-gov.com/api/v2/llm-obs/v1/experiments/{experiment_id} | | app.datadoghq.com | PATCH https://api.datadoghq.com/api/v2/llm-obs/v1/experiments/{experiment_id} | | us3.datadoghq.com | PATCH https://api.us3.datadoghq.com/api/v2/llm-obs/v1/experiments/{experiment_id} | | us5.datadoghq.com | PATCH https://api.us5.datadoghq.com/api/v2/llm-obs/v1/experiments/{experiment_id} | ### Overview Partially update an existing LLM Observability experiment. ### Arguments #### Path Parameters | Name | Type | Description | | ------------------------------- | ------ | ------------------------------------------- | | experiment_id [*required*] | string | The ID of the LLM Observability experiment. | ### Request #### Body Data (required) Update experiment payload. {% tab title="Model" %} | Parent field | Field | Type | Description | | ------------ | ---------------------------- | ------ | ------------------------------------------------------------------------------------ | | | data [*required*] | object | Data object for updating an LLM Observability experiment. | | data | attributes [*required*] | object | Attributes for updating an LLM Observability experiment. | | attributes | description | string | Updated description of the experiment. | | attributes | name | string | Updated name of the experiment. | | data | type [*required*] | enum | Resource type of an LLM Observability experiment. Allowed enum values: `experiments` | {% /tab %} {% tab title="Example" %} ```json { "data": { "attributes": { "description": "string", "name": "string" }, "type": "experiments" } } ``` {% /tab %} ### Response {% tab title="200" %} OK {% tab title="Model" %} Response containing a single LLM Observability experiment. | Parent field | Field | Type | Description | | ------------ | ----------------------------- | --------- | ------------------------------------------------------------------------------------ | | | data [*required*] | object | Data object for an LLM Observability experiment. | | data | attributes [*required*] | object | Attributes of an LLM Observability experiment. | | attributes | config [*required*] | object | Configuration parameters for the experiment. | | attributes | created_at [*required*] | date-time | Timestamp when the experiment was created. | | attributes | dataset_id [*required*] | string | Identifier of the dataset used in this experiment. | | attributes | description [*required*] | string | Description of the experiment. | | attributes | metadata [*required*] | object | Arbitrary metadata associated with the experiment. | | attributes | name [*required*] | string | Name of the experiment. | | attributes | project_id [*required*] | string | Identifier of the project this experiment belongs to. | | attributes | updated_at [*required*] | date-time | Timestamp when the experiment was last updated. | | data | id [*required*] | string | Unique identifier of the experiment. | | data | type [*required*] | enum | Resource type of an LLM Observability experiment. Allowed enum values: `experiments` | {% /tab %} {% tab title="Example" %} ```json { "data": { "attributes": { "config": {}, "created_at": "2024-01-15T10:30:00Z", "dataset_id": "9f64e5c7-dc5a-45c8-a17c-1b85f0bec97d", "description": "", "metadata": {}, "name": "My Experiment v1", "project_id": "a33671aa-24fd-4dcd-9b33-a8ec7dde7751", "updated_at": "2024-01-15T10:30:00Z" }, "id": "3fd6b5e0-8910-4b1c-a7d0-5b84de329012", "type": "experiments" } } ``` {% /tab %} {% /tab %} {% tab title="400" %} Bad Request {% tab title="Model" %} API error response. | Parent field | Field | Type | Description | | ------------ | ------------------------ | -------- | ------------------------------------------------------------------------------- | | | errors [*required*] | [object] | A list of errors. | | errors | detail | string | A human-readable explanation specific to this occurrence of the error. | | errors | meta | object | Non-standard meta-information about the error | | errors | source | object | References to the source of the error. | | source | header | string | A string indicating the name of a single request header which caused the error. | | source | parameter | string | A string indicating which URI query parameter caused the error. | | source | pointer | string | A JSON pointer to the value in the request document that caused the error. | | errors | status | string | Status code of the response. | | errors | title | string | Short human-readable summary of the error. | {% /tab %} {% tab title="Example" %} ```json { "errors": [ { "detail": "Missing required attribute in body", "meta": {}, "source": { "header": "Authorization", "parameter": "limit", "pointer": "/data/attributes/title" }, "status": "400", "title": "Bad Request" } ] } ``` {% /tab %} {% /tab %} {% tab title="401" %} Unauthorized {% tab title="Model" %} API error response. | Parent field | Field | Type | Description | | ------------ | ------------------------ | -------- | ------------------------------------------------------------------------------- | | | errors [*required*] | [object] | A list of errors. | | errors | detail | string | A human-readable explanation specific to this occurrence of the error. | | errors | meta | object | Non-standard meta-information about the error | | errors | source | object | References to the source of the error. | | source | header | string | A string indicating the name of a single request header which caused the error. | | source | parameter | string | A string indicating which URI query parameter caused the error. | | source | pointer | string | A JSON pointer to the value in the request document that caused the error. | | errors | status | string | Status code of the response. | | errors | title | string | Short human-readable summary of the error. | {% /tab %} {% tab title="Example" %} ```json { "errors": [ { "detail": "Missing required attribute in body", "meta": {}, "source": { "header": "Authorization", "parameter": "limit", "pointer": "/data/attributes/title" }, "status": "400", "title": "Bad Request" } ] } ``` {% /tab %} {% /tab %} {% tab title="403" %} Forbidden {% tab title="Model" %} API error response. | Parent field | Field | Type | Description | | ------------ | ------------------------ | -------- | ------------------------------------------------------------------------------- | | | errors [*required*] | [object] | A list of errors. | | errors | detail | string | A human-readable explanation specific to this occurrence of the error. | | errors | meta | object | Non-standard meta-information about the error | | errors | source | object | References to the source of the error. | | source | header | string | A string indicating the name of a single request header which caused the error. | | source | parameter | string | A string indicating which URI query parameter caused the error. | | source | pointer | string | A JSON pointer to the value in the request document that caused the error. | | errors | status | string | Status code of the response. | | errors | title | string | Short human-readable summary of the error. | {% /tab %} {% tab title="Example" %} ```json { "errors": [ { "detail": "Missing required attribute in body", "meta": {}, "source": { "header": "Authorization", "parameter": "limit", "pointer": "/data/attributes/title" }, "status": "400", "title": "Bad Request" } ] } ``` {% /tab %} {% /tab %} {% tab title="404" %} Not Found {% tab title="Model" %} API error response. | Parent field | Field | Type | Description | | ------------ | ------------------------ | -------- | ------------------------------------------------------------------------------- | | | errors [*required*] | [object] | A list of errors. | | errors | detail | string | A human-readable explanation specific to this occurrence of the error. | | errors | meta | object | Non-standard meta-information about the error | | errors | source | object | References to the source of the error. | | source | header | string | A string indicating the name of a single request header which caused the error. | | source | parameter | string | A string indicating which URI query parameter caused the error. | | source | pointer | string | A JSON pointer to the value in the request document that caused the error. | | errors | status | string | Status code of the response. | | errors | title | string | Short human-readable summary of the error. | {% /tab %} {% tab title="Example" %} ```json { "errors": [ { "detail": "Missing required attribute in body", "meta": {}, "source": { "header": "Authorization", "parameter": "limit", "pointer": "/data/attributes/title" }, "status": "400", "title": "Bad Request" } ] } ``` {% /tab %} {% /tab %} {% tab title="429" %} Too many requests {% tab title="Model" %} API error response. | Field | Type | Description | | ------------------------ | -------- | ----------------- | | errors [*required*] | [string] | A list of errors. | {% /tab %} {% tab title="Example" %} ```json { "errors": [ "Bad Request" ] } ``` {% /tab %} {% /tab %} ### Code Example ##### \# Path parametersexport experiment_id="3fd6b5e0-8910-4b1c-a7d0-5b84de329012"\# Curl commandcurl -X PATCH "https://api.ap1.datadoghq.com"https://api.ap2.datadoghq.com"https://api.datadoghq.eu"https://api.ddog-gov.com"https://api.datadoghq.com"https://api.us3.datadoghq.com"https://api.us5.datadoghq.com/api/v2/llm-obs/v1/experiments/${experiment_id}" \ -H "Accept: application/json" \ -H "Content-Type: application/json" \ -H "DD-API-KEY: ${DD_API_KEY}" \ -H "DD-APPLICATION-KEY: ${DD_APP_KEY}" \ -d @- << EOF { "data": { "attributes": {}, "type": "experiments" } } EOF {% /tab %} ## Delete LLM Observability experiments{% #delete-llm-observability-experiments %} {% tab title="v2" %} **Note**: This endpoint is in preview and is subject to change. If you have any feedback, contact [Datadog support](https://docs.datadoghq.com/help/). | Datadog site | API endpoint | | ----------------- | ----------------------------------------------------------------------- | | ap1.datadoghq.com | POST https://api.ap1.datadoghq.com/api/v2/llm-obs/v1/experiments/delete | | ap2.datadoghq.com | POST https://api.ap2.datadoghq.com/api/v2/llm-obs/v1/experiments/delete | | app.datadoghq.eu | POST https://api.datadoghq.eu/api/v2/llm-obs/v1/experiments/delete | | app.ddog-gov.com | POST https://api.ddog-gov.com/api/v2/llm-obs/v1/experiments/delete | | app.datadoghq.com | POST https://api.datadoghq.com/api/v2/llm-obs/v1/experiments/delete | | us3.datadoghq.com | POST https://api.us3.datadoghq.com/api/v2/llm-obs/v1/experiments/delete | | us5.datadoghq.com | POST https://api.us5.datadoghq.com/api/v2/llm-obs/v1/experiments/delete | ### Overview Delete one or more LLM Observability experiments. ### Request #### Body Data (required) Delete experiments payload. {% tab title="Model" %} | Parent field | Field | Type | Description | | ------------ | -------------------------------- | -------- | ------------------------------------------------------------------------------------ | | | data [*required*] | object | Data object for deleting LLM Observability experiments. | | data | attributes [*required*] | object | Attributes for deleting LLM Observability experiments. | | attributes | experiment_ids [*required*] | [string] | List of experiment IDs to delete. | | data | type [*required*] | enum | Resource type of an LLM Observability experiment. Allowed enum values: `experiments` | {% /tab %} {% tab title="Example" %} ```json { "data": { "attributes": { "experiment_ids": [ "3fd6b5e0-8910-4b1c-a7d0-5b84de329012" ] }, "type": "experiments" } } ``` {% /tab %} ### Response {% tab title="204" %} No Content {% /tab %} {% tab title="400" %} Bad Request {% tab title="Model" %} API error response. | Parent field | Field | Type | Description | | ------------ | ------------------------ | -------- | ------------------------------------------------------------------------------- | | | errors [*required*] | [object] | A list of errors. | | errors | detail | string | A human-readable explanation specific to this occurrence of the error. | | errors | meta | object | Non-standard meta-information about the error | | errors | source | object | References to the source of the error. | | source | header | string | A string indicating the name of a single request header which caused the error. | | source | parameter | string | A string indicating which URI query parameter caused the error. | | source | pointer | string | A JSON pointer to the value in the request document that caused the error. | | errors | status | string | Status code of the response. | | errors | title | string | Short human-readable summary of the error. | {% /tab %} {% tab title="Example" %} ```json { "errors": [ { "detail": "Missing required attribute in body", "meta": {}, "source": { "header": "Authorization", "parameter": "limit", "pointer": "/data/attributes/title" }, "status": "400", "title": "Bad Request" } ] } ``` {% /tab %} {% /tab %} {% tab title="401" %} Unauthorized {% tab title="Model" %} API error response. | Parent field | Field | Type | Description | | ------------ | ------------------------ | -------- | ------------------------------------------------------------------------------- | | | errors [*required*] | [object] | A list of errors. | | errors | detail | string | A human-readable explanation specific to this occurrence of the error. | | errors | meta | object | Non-standard meta-information about the error | | errors | source | object | References to the source of the error. | | source | header | string | A string indicating the name of a single request header which caused the error. | | source | parameter | string | A string indicating which URI query parameter caused the error. | | source | pointer | string | A JSON pointer to the value in the request document that caused the error. | | errors | status | string | Status code of the response. | | errors | title | string | Short human-readable summary of the error. | {% /tab %} {% tab title="Example" %} ```json { "errors": [ { "detail": "Missing required attribute in body", "meta": {}, "source": { "header": "Authorization", "parameter": "limit", "pointer": "/data/attributes/title" }, "status": "400", "title": "Bad Request" } ] } ``` {% /tab %} {% /tab %} {% tab title="403" %} Forbidden {% tab title="Model" %} API error response. | Parent field | Field | Type | Description | | ------------ | ------------------------ | -------- | ------------------------------------------------------------------------------- | | | errors [*required*] | [object] | A list of errors. | | errors | detail | string | A human-readable explanation specific to this occurrence of the error. | | errors | meta | object | Non-standard meta-information about the error | | errors | source | object | References to the source of the error. | | source | header | string | A string indicating the name of a single request header which caused the error. | | source | parameter | string | A string indicating which URI query parameter caused the error. | | source | pointer | string | A JSON pointer to the value in the request document that caused the error. | | errors | status | string | Status code of the response. | | errors | title | string | Short human-readable summary of the error. | {% /tab %} {% tab title="Example" %} ```json { "errors": [ { "detail": "Missing required attribute in body", "meta": {}, "source": { "header": "Authorization", "parameter": "limit", "pointer": "/data/attributes/title" }, "status": "400", "title": "Bad Request" } ] } ``` {% /tab %} {% /tab %} {% tab title="429" %} Too many requests {% tab title="Model" %} API error response. | Field | Type | Description | | ------------------------ | -------- | ----------------- | | errors [*required*] | [string] | A list of errors. | {% /tab %} {% tab title="Example" %} ```json { "errors": [ "Bad Request" ] } ``` {% /tab %} {% /tab %} ### Code Example ##### \# Curl commandcurl -X POST "https://api.ap1.datadoghq.com"https://api.ap2.datadoghq.com"https://api.datadoghq.eu"https://api.ddog-gov.com"https://api.datadoghq.com"https://api.us3.datadoghq.com"https://api.us5.datadoghq.com/api/v2/llm-obs/v1/experiments/delete" \ -H "Content-Type: application/json" \ -H "DD-API-KEY: ${DD_API_KEY}" \ -H "DD-APPLICATION-KEY: ${DD_APP_KEY}" \ -d @- << EOF { "data": { "attributes": { "experiment_ids": [ "3fd6b5e0-8910-4b1c-a7d0-5b84de329012" ] }, "type": "experiments" } } EOF {% /tab %} ## Push events for an LLM Observability experiment{% #push-events-for-an-llm-observability-experiment %} {% tab title="v2" %} **Note**: This endpoint is in preview and is subject to change. If you have any feedback, contact [Datadog support](https://docs.datadoghq.com/help/). | Datadog site | API endpoint | | ----------------- | --------------------------------------------------------------------------------------- | | ap1.datadoghq.com | POST https://api.ap1.datadoghq.com/api/v2/llm-obs/v1/experiments/{experiment_id}/events | | ap2.datadoghq.com | POST https://api.ap2.datadoghq.com/api/v2/llm-obs/v1/experiments/{experiment_id}/events | | app.datadoghq.eu | POST https://api.datadoghq.eu/api/v2/llm-obs/v1/experiments/{experiment_id}/events | | app.ddog-gov.com | POST https://api.ddog-gov.com/api/v2/llm-obs/v1/experiments/{experiment_id}/events | | app.datadoghq.com | POST https://api.datadoghq.com/api/v2/llm-obs/v1/experiments/{experiment_id}/events | | us3.datadoghq.com | POST https://api.us3.datadoghq.com/api/v2/llm-obs/v1/experiments/{experiment_id}/events | | us5.datadoghq.com | POST https://api.us5.datadoghq.com/api/v2/llm-obs/v1/experiments/{experiment_id}/events | ### Overview Push spans and metrics for an LLM Observability experiment. ### Arguments #### Path Parameters | Name | Type | Description | | ------------------------------- | ------ | ------------------------------------------- | | experiment_id [*required*] | string | The ID of the LLM Observability experiment. | ### Request #### Body Data (required) Experiment events payload. {% tab title="Model" %} | Parent field | Field | Type | Description | | ------------ | ------------------------------ | ------------------- | ------------------------------------------------------------------------------------------------------------------ | | | data [*required*] | object | Data object for pushing experiment events. | | data | attributes [*required*] | object | Attributes for pushing experiment events including spans and metrics. | | attributes | metrics | [object] | List of metrics to push for the experiment. | | metrics | assessment | enum | Assessment result for an LLM Observability experiment metric. Allowed enum values: `pass,fail` | | metrics | boolean_value | boolean | Boolean value. Used when `metric_type` is `boolean`. | | metrics | categorical_value | string | Categorical value. Used when `metric_type` is `categorical`. | | metrics | error | object | Error details for an experiment metric evaluation. | | error | message | string | Error message associated with the metric evaluation. | | metrics | json_value | object | JSON value. Used when `metric_type` is `json`. | | metrics | label [*required*] | string | Label or name for the metric. | | metrics | metadata | object | Arbitrary metadata associated with the metric. | | metrics | metric_type [*required*] | enum | Type of metric recorded for an LLM Observability experiment. Allowed enum values: `score,categorical,boolean,json` | | metrics | reasoning | string | Human-readable reasoning for the metric value. | | metrics | score_value | double | Numeric score value. Used when `metric_type` is `score`. | | metrics | span_id [*required*] | string | The ID of the span this metric measures. | | metrics | tags | [string] | List of tags associated with the metric. | | metrics | timestamp_ms [*required*] | int64 | Timestamp when the metric was recorded, in milliseconds since Unix epoch. | | attributes | spans | [object] | List of spans to push for the experiment. | | spans | dataset_id [*required*] | string | Dataset ID associated with this span. | | spans | duration [*required*] | int64 | Duration of the span in nanoseconds. | | spans | meta | object | Metadata associated with an experiment span. | | meta | error | object | Error details for an experiment span. | | error | message | string | Error message. | | error | stack | string | Stack trace of the error. | | error | type | string | The error type or exception class name. | | meta | expected_output | object | Expected output for the span, used for evaluation. | | meta | input | object | Represents any valid JSON value. | | input | Option 1 | string | A scalar string value. | | input | Option 2 | double | A scalar numeric value. | | input | Option 3 | object | An arbitrary object value with additional properties. | | input | Option 4 | [ ] | An array of arbitrary values. | | Option 4 | Option 1 | string | A scalar string value. | | Option 4 | Option 2 | double | A scalar numeric value. | | Option 4 | Option 3 | object | An arbitrary object value with additional properties. | | Option 4 | Option 4 | boolean | A scalar boolean value. | | input | Option 5 | boolean | A scalar boolean value. | | meta | output | object | Represents any valid JSON value. | | output | Option 1 | string | A scalar string value. | | output | Option 2 | double | A scalar numeric value. | | output | Option 3 | object | An arbitrary object value with additional properties. | | output | Option 4 | [ ] | An array of arbitrary values. | | Option 4 | Option 1 | string | A scalar string value. | | Option 4 | Option 2 | double | A scalar numeric value. | | Option 4 | Option 3 | object | An arbitrary object value with additional properties. | | Option 4 | Option 4 | boolean | A scalar boolean value. | | output | Option 5 | boolean | A scalar boolean value. | | spans | name [*required*] | string | Name of the span. | | spans | project_id [*required*] | string | Project ID associated with this span. | | spans | span_id [*required*] | string | Unique identifier of the span. | | spans | start_ns [*required*] | int64 | Start time of the span in nanoseconds since Unix epoch. | | spans | status [*required*] | enum | Status of the span. Allowed enum values: `ok,error` | | spans | tags | [string] | List of tags associated with the span. | | spans | trace_id [*required*] | string | Trace ID for the span. | | data | type [*required*] | enum | Resource type for LLM Observability experiment events. Allowed enum values: `events` | {% /tab %} {% tab title="Example" %} ```json { "data": { "attributes": { "metrics": [ { "assessment": "pass", "boolean_value": false, "categorical_value": "string", "error": { "message": "string" }, "json_value": {}, "label": "faithfulness", "metadata": {}, "metric_type": "score", "reasoning": "string", "score_value": "number", "span_id": "span-7a1b2c3d", "tags": [], "timestamp_ms": 1705314600000 } ], "spans": [ { "dataset_id": "9f64e5c7-dc5a-45c8-a17c-1b85f0bec97d", "duration": 1500000000, "meta": { "error": { "message": "Model response timed out", "stack": "Traceback (most recent call last):\n File \"main.py\", line 10, in \n response = model.generate(input)\n File \"model.py\", line 45, in generate\n raise TimeoutError(\"Model response timed out\")\nTimeoutError: Model response timed out", "type": "TimeoutError" }, "expected_output": {}, "input": { "description": "undefined", "type": "undefined" }, "output": { "description": "undefined", "type": "undefined" } }, "name": "llm_call", "project_id": "a33671aa-24fd-4dcd-9b33-a8ec7dde7751", "span_id": "span-7a1b2c3d", "start_ns": 1705314600000000000, "status": "ok", "tags": [], "trace_id": "abc123def456" } ] }, "type": "events" } } ``` {% /tab %} ### Response {% tab title="202" %} Accepted {% /tab %} {% tab title="400" %} Bad Request {% tab title="Model" %} API error response. | Parent field | Field | Type | Description | | ------------ | ------------------------ | -------- | ------------------------------------------------------------------------------- | | | errors [*required*] | [object] | A list of errors. | | errors | detail | string | A human-readable explanation specific to this occurrence of the error. | | errors | meta | object | Non-standard meta-information about the error | | errors | source | object | References to the source of the error. | | source | header | string | A string indicating the name of a single request header which caused the error. | | source | parameter | string | A string indicating which URI query parameter caused the error. | | source | pointer | string | A JSON pointer to the value in the request document that caused the error. | | errors | status | string | Status code of the response. | | errors | title | string | Short human-readable summary of the error. | {% /tab %} {% tab title="Example" %} ```json { "errors": [ { "detail": "Missing required attribute in body", "meta": {}, "source": { "header": "Authorization", "parameter": "limit", "pointer": "/data/attributes/title" }, "status": "400", "title": "Bad Request" } ] } ``` {% /tab %} {% /tab %} {% tab title="401" %} Unauthorized {% tab title="Model" %} API error response. | Parent field | Field | Type | Description | | ------------ | ------------------------ | -------- | ------------------------------------------------------------------------------- | | | errors [*required*] | [object] | A list of errors. | | errors | detail | string | A human-readable explanation specific to this occurrence of the error. | | errors | meta | object | Non-standard meta-information about the error | | errors | source | object | References to the source of the error. | | source | header | string | A string indicating the name of a single request header which caused the error. | | source | parameter | string | A string indicating which URI query parameter caused the error. | | source | pointer | string | A JSON pointer to the value in the request document that caused the error. | | errors | status | string | Status code of the response. | | errors | title | string | Short human-readable summary of the error. | {% /tab %} {% tab title="Example" %} ```json { "errors": [ { "detail": "Missing required attribute in body", "meta": {}, "source": { "header": "Authorization", "parameter": "limit", "pointer": "/data/attributes/title" }, "status": "400", "title": "Bad Request" } ] } ``` {% /tab %} {% /tab %} {% tab title="403" %} Forbidden {% tab title="Model" %} API error response. | Parent field | Field | Type | Description | | ------------ | ------------------------ | -------- | ------------------------------------------------------------------------------- | | | errors [*required*] | [object] | A list of errors. | | errors | detail | string | A human-readable explanation specific to this occurrence of the error. | | errors | meta | object | Non-standard meta-information about the error | | errors | source | object | References to the source of the error. | | source | header | string | A string indicating the name of a single request header which caused the error. | | source | parameter | string | A string indicating which URI query parameter caused the error. | | source | pointer | string | A JSON pointer to the value in the request document that caused the error. | | errors | status | string | Status code of the response. | | errors | title | string | Short human-readable summary of the error. | {% /tab %} {% tab title="Example" %} ```json { "errors": [ { "detail": "Missing required attribute in body", "meta": {}, "source": { "header": "Authorization", "parameter": "limit", "pointer": "/data/attributes/title" }, "status": "400", "title": "Bad Request" } ] } ``` {% /tab %} {% /tab %} {% tab title="404" %} Not Found {% tab title="Model" %} API error response. | Parent field | Field | Type | Description | | ------------ | ------------------------ | -------- | ------------------------------------------------------------------------------- | | | errors [*required*] | [object] | A list of errors. | | errors | detail | string | A human-readable explanation specific to this occurrence of the error. | | errors | meta | object | Non-standard meta-information about the error | | errors | source | object | References to the source of the error. | | source | header | string | A string indicating the name of a single request header which caused the error. | | source | parameter | string | A string indicating which URI query parameter caused the error. | | source | pointer | string | A JSON pointer to the value in the request document that caused the error. | | errors | status | string | Status code of the response. | | errors | title | string | Short human-readable summary of the error. | {% /tab %} {% tab title="Example" %} ```json { "errors": [ { "detail": "Missing required attribute in body", "meta": {}, "source": { "header": "Authorization", "parameter": "limit", "pointer": "/data/attributes/title" }, "status": "400", "title": "Bad Request" } ] } ``` {% /tab %} {% /tab %} {% tab title="429" %} Too many requests {% tab title="Model" %} API error response. | Field | Type | Description | | ------------------------ | -------- | ----------------- | | errors [*required*] | [string] | A list of errors. | {% /tab %} {% tab title="Example" %} ```json { "errors": [ "Bad Request" ] } ``` {% /tab %} {% /tab %} ### Code Example ##### \# Path parametersexport experiment_id="3fd6b5e0-8910-4b1c-a7d0-5b84de329012"\# Curl commandcurl -X POST "https://api.ap1.datadoghq.com"https://api.ap2.datadoghq.com"https://api.datadoghq.eu"https://api.ddog-gov.com"https://api.datadoghq.com"https://api.us3.datadoghq.com"https://api.us5.datadoghq.com/api/v2/llm-obs/v1/experiments/${experiment_id}/events" \ -H "Accept: application/json" \ -H "Content-Type: application/json" \ -H "DD-API-KEY: ${DD_API_KEY}" \ -H "DD-APPLICATION-KEY: ${DD_APP_KEY}" \ -d @- << EOF { "data": { "attributes": { "metrics": [ { "label": "faithfulness", "metric_type": "score", "span_id": "span-7a1b2c3d", "timestamp_ms": 1705314600000 } ], "spans": [ { "dataset_id": "9f64e5c7-dc5a-45c8-a17c-1b85f0bec97d", "duration": 1500000000, "name": "llm_call", "project_id": "a33671aa-24fd-4dcd-9b33-a8ec7dde7751", "span_id": "span-7a1b2c3d", "start_ns": 1705314600000000000, "status": "ok", "trace_id": "abc123def456" } ] }, "type": "events" } } EOF {% /tab %} ## List LLM Observability annotation queues{% #list-llm-observability-annotation-queues %} {% tab title="v2" %} **Note**: This endpoint is in preview and is subject to change. If you have any feedback, contact [Datadog support](https://docs.datadoghq.com/help/). | Datadog site | API endpoint | | ----------------- | --------------------------------------------------------------------- | | ap1.datadoghq.com | GET https://api.ap1.datadoghq.com/api/v2/llm-obs/v1/annotation-queues | | ap2.datadoghq.com | GET https://api.ap2.datadoghq.com/api/v2/llm-obs/v1/annotation-queues | | app.datadoghq.eu | GET https://api.datadoghq.eu/api/v2/llm-obs/v1/annotation-queues | | app.ddog-gov.com | GET https://api.ddog-gov.com/api/v2/llm-obs/v1/annotation-queues | | app.datadoghq.com | GET https://api.datadoghq.com/api/v2/llm-obs/v1/annotation-queues | | us3.datadoghq.com | GET https://api.us3.datadoghq.com/api/v2/llm-obs/v1/annotation-queues | | us5.datadoghq.com | GET https://api.us5.datadoghq.com/api/v2/llm-obs/v1/annotation-queues | ### Overview List annotation queues. Optionally filter by project ID or queue IDs. These parameters are mutually exclusive. If neither is provided, all queues in the organization are returned. ### Arguments #### Query Strings | Name | Type | Description | | --------- | ------ | -------------------------------------------------------------------------------------------------- | | projectId | string | Filter annotation queues by project ID. Cannot be used together with `queueIds`. | | queueIds | array | Filter annotation queues by queue IDs (comma-separated). Cannot be used together with `projectId`. | ### Response {% tab title="200" %} OK {% tab title="Model" %} Response containing a list of LLM Observability annotation queues. | Parent field | Field | Type | Description | | ------------ | ----------------------------- | --------- | ------------------------------------------------------------------------------------- | | | data [*required*] | [object] | List of annotation queues. | | data | attributes [*required*] | object | Attributes of an LLM Observability annotation queue. | | attributes | created_at [*required*] | date-time | Timestamp when the queue was created. | | attributes | created_by [*required*] | string | Identifier of the user who created the queue. | | attributes | description [*required*] | string | Description of the annotation queue. | | attributes | modified_at [*required*] | date-time | Timestamp when the queue was last modified. | | attributes | modified_by [*required*] | string | Identifier of the user who last modified the queue. | | attributes | name [*required*] | string | Name of the annotation queue. | | attributes | owned_by [*required*] | string | Identifier of the user who owns the queue. | | attributes | project_id [*required*] | string | Identifier of the project this queue belongs to. | | data | id [*required*] | string | Unique identifier of the annotation queue. | | data | type [*required*] | enum | Resource type of an LLM Observability annotation queue. Allowed enum values: `queues` | {% /tab %} {% tab title="Example" %} ```json { "data": [ { "attributes": { "created_at": "2024-01-15T10:30:00Z", "created_by": "00000000-0000-0000-0000-000000000002", "description": "Queue for annotating customer support traces", "modified_at": "2024-01-15T10:30:00Z", "modified_by": "00000000-0000-0000-0000-000000000002", "name": "My annotation queue", "owned_by": "00000000-0000-0000-0000-000000000002", "project_id": "a33671aa-24fd-4dcd-9b33-a8ec7dde7751" }, "id": "b5e7f3a1-9c2d-4f8b-a1e6-3d7e9f0a2b4c", "type": "queues" } ] } ``` {% /tab %} {% /tab %} {% tab title="400" %} Bad Request {% tab title="Model" %} API error response. | Parent field | Field | Type | Description | | ------------ | ------------------------ | -------- | ------------------------------------------------------------------------------- | | | errors [*required*] | [object] | A list of errors. | | errors | detail | string | A human-readable explanation specific to this occurrence of the error. | | errors | meta | object | Non-standard meta-information about the error | | errors | source | object | References to the source of the error. | | source | header | string | A string indicating the name of a single request header which caused the error. | | source | parameter | string | A string indicating which URI query parameter caused the error. | | source | pointer | string | A JSON pointer to the value in the request document that caused the error. | | errors | status | string | Status code of the response. | | errors | title | string | Short human-readable summary of the error. | {% /tab %} {% tab title="Example" %} ```json { "errors": [ { "detail": "Missing required attribute in body", "meta": {}, "source": { "header": "Authorization", "parameter": "limit", "pointer": "/data/attributes/title" }, "status": "400", "title": "Bad Request" } ] } ``` {% /tab %} {% /tab %} {% tab title="401" %} Unauthorized {% tab title="Model" %} API error response. | Parent field | Field | Type | Description | | ------------ | ------------------------ | -------- | ------------------------------------------------------------------------------- | | | errors [*required*] | [object] | A list of errors. | | errors | detail | string | A human-readable explanation specific to this occurrence of the error. | | errors | meta | object | Non-standard meta-information about the error | | errors | source | object | References to the source of the error. | | source | header | string | A string indicating the name of a single request header which caused the error. | | source | parameter | string | A string indicating which URI query parameter caused the error. | | source | pointer | string | A JSON pointer to the value in the request document that caused the error. | | errors | status | string | Status code of the response. | | errors | title | string | Short human-readable summary of the error. | {% /tab %} {% tab title="Example" %} ```json { "errors": [ { "detail": "Missing required attribute in body", "meta": {}, "source": { "header": "Authorization", "parameter": "limit", "pointer": "/data/attributes/title" }, "status": "400", "title": "Bad Request" } ] } ``` {% /tab %} {% /tab %} {% tab title="403" %} Forbidden {% tab title="Model" %} API error response. | Parent field | Field | Type | Description | | ------------ | ------------------------ | -------- | ------------------------------------------------------------------------------- | | | errors [*required*] | [object] | A list of errors. | | errors | detail | string | A human-readable explanation specific to this occurrence of the error. | | errors | meta | object | Non-standard meta-information about the error | | errors | source | object | References to the source of the error. | | source | header | string | A string indicating the name of a single request header which caused the error. | | source | parameter | string | A string indicating which URI query parameter caused the error. | | source | pointer | string | A JSON pointer to the value in the request document that caused the error. | | errors | status | string | Status code of the response. | | errors | title | string | Short human-readable summary of the error. | {% /tab %} {% tab title="Example" %} ```json { "errors": [ { "detail": "Missing required attribute in body", "meta": {}, "source": { "header": "Authorization", "parameter": "limit", "pointer": "/data/attributes/title" }, "status": "400", "title": "Bad Request" } ] } ``` {% /tab %} {% /tab %} {% tab title="429" %} Too many requests {% tab title="Model" %} API error response. | Field | Type | Description | | ------------------------ | -------- | ----------------- | | errors [*required*] | [string] | A list of errors. | {% /tab %} {% tab title="Example" %} ```json { "errors": [ "Bad Request" ] } ``` {% /tab %} {% /tab %} ### Code Example ##### \# Curl commandcurl -X GET "https://api.ap1.datadoghq.com"https://api.ap2.datadoghq.com"https://api.datadoghq.eu"https://api.ddog-gov.com"https://api.datadoghq.com"https://api.us3.datadoghq.com"https://api.us5.datadoghq.com/api/v2/llm-obs/v1/annotation-queues" \ -H "Accept: application/json" \ -H "DD-API-KEY: ${DD_API_KEY}" \ -H "DD-APPLICATION-KEY: ${DD_APP_KEY}" {% /tab %} ## Create an LLM Observability annotation queue{% #create-an-llm-observability-annotation-queue %} {% tab title="v2" %} **Note**: This endpoint is in preview and is subject to change. If you have any feedback, contact [Datadog support](https://docs.datadoghq.com/help/). | Datadog site | API endpoint | | ----------------- | ---------------------------------------------------------------------- | | ap1.datadoghq.com | POST https://api.ap1.datadoghq.com/api/v2/llm-obs/v1/annotation-queues | | ap2.datadoghq.com | POST https://api.ap2.datadoghq.com/api/v2/llm-obs/v1/annotation-queues | | app.datadoghq.eu | POST https://api.datadoghq.eu/api/v2/llm-obs/v1/annotation-queues | | app.ddog-gov.com | POST https://api.ddog-gov.com/api/v2/llm-obs/v1/annotation-queues | | app.datadoghq.com | POST https://api.datadoghq.com/api/v2/llm-obs/v1/annotation-queues | | us3.datadoghq.com | POST https://api.us3.datadoghq.com/api/v2/llm-obs/v1/annotation-queues | | us5.datadoghq.com | POST https://api.us5.datadoghq.com/api/v2/llm-obs/v1/annotation-queues | ### Overview Create a new annotation queue. Only `name`, `project_id`, and `description` are accepted. Fields such as `created_by`, `owned_by`, `created_at`, `modified_by`, and `modified_at` are inferred by the backend. ### Request #### Body Data (required) Create annotation queue payload. {% tab title="Model" %} | Parent field | Field | Type | Description | | ------------ | ---------------------------- | ------ | ------------------------------------------------------------------------------------- | | | data [*required*] | object | Data object for creating an LLM Observability annotation queue. | | data | attributes [*required*] | object | Attributes for creating an LLM Observability annotation queue. | | attributes | description | string | Description of the annotation queue. | | attributes | name [*required*] | string | Name of the annotation queue. | | attributes | project_id [*required*] | string | Identifier of the project this queue belongs to. | | data | type [*required*] | enum | Resource type of an LLM Observability annotation queue. Allowed enum values: `queues` | {% /tab %} {% tab title="Example" %} ```json { "data": { "attributes": { "description": "Queue for annotating customer support traces", "name": "My annotation queue", "project_id": "a33671aa-24fd-4dcd-9b33-a8ec7dde7751" }, "type": "queues" } } ``` {% /tab %} ### Response {% tab title="201" %} Created {% tab title="Model" %} Response containing a single LLM Observability annotation queue. | Parent field | Field | Type | Description | | ------------ | ----------------------------- | --------- | ------------------------------------------------------------------------------------- | | | data [*required*] | object | Data object for an LLM Observability annotation queue. | | data | attributes [*required*] | object | Attributes of an LLM Observability annotation queue. | | attributes | created_at [*required*] | date-time | Timestamp when the queue was created. | | attributes | created_by [*required*] | string | Identifier of the user who created the queue. | | attributes | description [*required*] | string | Description of the annotation queue. | | attributes | modified_at [*required*] | date-time | Timestamp when the queue was last modified. | | attributes | modified_by [*required*] | string | Identifier of the user who last modified the queue. | | attributes | name [*required*] | string | Name of the annotation queue. | | attributes | owned_by [*required*] | string | Identifier of the user who owns the queue. | | attributes | project_id [*required*] | string | Identifier of the project this queue belongs to. | | data | id [*required*] | string | Unique identifier of the annotation queue. | | data | type [*required*] | enum | Resource type of an LLM Observability annotation queue. Allowed enum values: `queues` | {% /tab %} {% tab title="Example" %} ```json { "data": { "attributes": { "created_at": "2024-01-15T10:30:00Z", "created_by": "00000000-0000-0000-0000-000000000002", "description": "Queue for annotating customer support traces", "modified_at": "2024-01-15T10:30:00Z", "modified_by": "00000000-0000-0000-0000-000000000002", "name": "My annotation queue", "owned_by": "00000000-0000-0000-0000-000000000002", "project_id": "a33671aa-24fd-4dcd-9b33-a8ec7dde7751" }, "id": "b5e7f3a1-9c2d-4f8b-a1e6-3d7e9f0a2b4c", "type": "queues" } } ``` {% /tab %} {% /tab %} {% tab title="400" %} Bad Request {% tab title="Model" %} API error response. | Parent field | Field | Type | Description | | ------------ | ------------------------ | -------- | ------------------------------------------------------------------------------- | | | errors [*required*] | [object] | A list of errors. | | errors | detail | string | A human-readable explanation specific to this occurrence of the error. | | errors | meta | object | Non-standard meta-information about the error | | errors | source | object | References to the source of the error. | | source | header | string | A string indicating the name of a single request header which caused the error. | | source | parameter | string | A string indicating which URI query parameter caused the error. | | source | pointer | string | A JSON pointer to the value in the request document that caused the error. | | errors | status | string | Status code of the response. | | errors | title | string | Short human-readable summary of the error. | {% /tab %} {% tab title="Example" %} ```json { "errors": [ { "detail": "Missing required attribute in body", "meta": {}, "source": { "header": "Authorization", "parameter": "limit", "pointer": "/data/attributes/title" }, "status": "400", "title": "Bad Request" } ] } ``` {% /tab %} {% /tab %} {% tab title="401" %} Unauthorized {% tab title="Model" %} API error response. | Parent field | Field | Type | Description | | ------------ | ------------------------ | -------- | ------------------------------------------------------------------------------- | | | errors [*required*] | [object] | A list of errors. | | errors | detail | string | A human-readable explanation specific to this occurrence of the error. | | errors | meta | object | Non-standard meta-information about the error | | errors | source | object | References to the source of the error. | | source | header | string | A string indicating the name of a single request header which caused the error. | | source | parameter | string | A string indicating which URI query parameter caused the error. | | source | pointer | string | A JSON pointer to the value in the request document that caused the error. | | errors | status | string | Status code of the response. | | errors | title | string | Short human-readable summary of the error. | {% /tab %} {% tab title="Example" %} ```json { "errors": [ { "detail": "Missing required attribute in body", "meta": {}, "source": { "header": "Authorization", "parameter": "limit", "pointer": "/data/attributes/title" }, "status": "400", "title": "Bad Request" } ] } ``` {% /tab %} {% /tab %} {% tab title="403" %} Forbidden {% tab title="Model" %} API error response. | Parent field | Field | Type | Description | | ------------ | ------------------------ | -------- | ------------------------------------------------------------------------------- | | | errors [*required*] | [object] | A list of errors. | | errors | detail | string | A human-readable explanation specific to this occurrence of the error. | | errors | meta | object | Non-standard meta-information about the error | | errors | source | object | References to the source of the error. | | source | header | string | A string indicating the name of a single request header which caused the error. | | source | parameter | string | A string indicating which URI query parameter caused the error. | | source | pointer | string | A JSON pointer to the value in the request document that caused the error. | | errors | status | string | Status code of the response. | | errors | title | string | Short human-readable summary of the error. | {% /tab %} {% tab title="Example" %} ```json { "errors": [ { "detail": "Missing required attribute in body", "meta": {}, "source": { "header": "Authorization", "parameter": "limit", "pointer": "/data/attributes/title" }, "status": "400", "title": "Bad Request" } ] } ``` {% /tab %} {% /tab %} {% tab title="429" %} Too many requests {% tab title="Model" %} API error response. | Field | Type | Description | | ------------------------ | -------- | ----------------- | | errors [*required*] | [string] | A list of errors. | {% /tab %} {% tab title="Example" %} ```json { "errors": [ "Bad Request" ] } ``` {% /tab %} {% /tab %} ### Code Example ##### \# Curl commandcurl -X POST "https://api.ap1.datadoghq.com"https://api.ap2.datadoghq.com"https://api.datadoghq.eu"https://api.ddog-gov.com"https://api.datadoghq.com"https://api.us3.datadoghq.com"https://api.us5.datadoghq.com/api/v2/llm-obs/v1/annotation-queues" \ -H "Accept: application/json" \ -H "Content-Type: application/json" \ -H "DD-API-KEY: ${DD_API_KEY}" \ -H "DD-APPLICATION-KEY: ${DD_APP_KEY}" \ -d @- << EOF { "data": { "attributes": { "name": "My annotation queue", "project_id": "a33671aa-24fd-4dcd-9b33-a8ec7dde7751" }, "type": "queues" } } EOF {% /tab %} ## Add annotation queue interactions{% #add-annotation-queue-interactions %} {% tab title="v2" %} **Note**: This endpoint is in preview and is subject to change. If you have any feedback, contact [Datadog support](https://docs.datadoghq.com/help/). | Datadog site | API endpoint | | ----------------- | ---------------------------------------------------------------------------------------------- | | ap1.datadoghq.com | POST https://api.ap1.datadoghq.com/api/v2/llm-obs/v1/annotation-queues/{queue_id}/interactions | | ap2.datadoghq.com | POST https://api.ap2.datadoghq.com/api/v2/llm-obs/v1/annotation-queues/{queue_id}/interactions | | app.datadoghq.eu | POST https://api.datadoghq.eu/api/v2/llm-obs/v1/annotation-queues/{queue_id}/interactions | | app.ddog-gov.com | POST https://api.ddog-gov.com/api/v2/llm-obs/v1/annotation-queues/{queue_id}/interactions | | app.datadoghq.com | POST https://api.datadoghq.com/api/v2/llm-obs/v1/annotation-queues/{queue_id}/interactions | | us3.datadoghq.com | POST https://api.us3.datadoghq.com/api/v2/llm-obs/v1/annotation-queues/{queue_id}/interactions | | us5.datadoghq.com | POST https://api.us5.datadoghq.com/api/v2/llm-obs/v1/annotation-queues/{queue_id}/interactions | ### Overview Add one or more interactions (traces) to an annotation queue. At least one interaction must be provided. ### Arguments #### Path Parameters | Name | Type | Description | | -------------------------- | ------ | ------------------------------------------------- | | queue_id [*required*] | string | The ID of the LLM Observability annotation queue. | ### Request #### Body Data (required) Add interactions payload. {% tab title="Model" %} | Parent field | Field | Type | Description | | ------------ | ------------------------------ | -------- | ----------------------------------------------------------------------------------------- | | | data [*required*] | object | Data object for adding interactions to an annotation queue. | | data | attributes [*required*] | object | Attributes for adding interactions to an annotation queue. | | attributes | interactions [*required*] | [object] | List of interactions to add to the queue. Must contain at least one item. | | interactions | content_id [*required*] | string | Identifier of the content (such as trace ID) for this interaction. | | interactions | type [*required*] | enum | Type of interaction in an annotation queue. Allowed enum values: `trace,experiment_trace` | | data | type [*required*] | enum | Resource type for annotation queue interactions. Allowed enum values: `interactions` | {% /tab %} {% tab title="Example" %} ```json { "data": { "attributes": { "interactions": [ { "content_id": "trace-abc-123", "type": "trace" } ] }, "type": "interactions" } } ``` {% /tab %} ### Response {% tab title="201" %} Created {% tab title="Model" %} Response containing the result of adding interactions to an annotation queue. | Parent field | Field | Type | Description | | ------------ | --------------------------------- | -------- | ----------------------------------------------------------------------------------------- | | | data [*required*] | object | Data object for the interaction addition response. | | data | attributes [*required*] | object | Attributes of the interaction addition response. | | attributes | interactions [*required*] | [object] | List of interactions that were processed. | | interactions | already_existed [*required*] | boolean | Whether this interaction already existed in the queue. | | interactions | content_id [*required*] | string | Identifier of the content for this interaction. | | interactions | id [*required*] | string | Unique identifier of the interaction. | | interactions | type [*required*] | enum | Type of interaction in an annotation queue. Allowed enum values: `trace,experiment_trace` | | data | id [*required*] | string | The queue ID the interactions were added to. | | data | type [*required*] | enum | Resource type for annotation queue interactions. Allowed enum values: `interactions` | {% /tab %} {% tab title="Example" %} ```json { "data": { "attributes": { "interactions": [ { "already_existed": false, "content_id": "trace-abc-123", "id": "00000000-0000-0000-0000-000000000000", "type": "trace" } ] }, "id": "b5e7f3a1-9c2d-4f8b-a1e6-3d7e9f0a2b4c", "type": "interactions" } } ``` {% /tab %} {% /tab %} {% tab title="400" %} Bad Request {% tab title="Model" %} API error response. | Parent field | Field | Type | Description | | ------------ | ------------------------ | -------- | ------------------------------------------------------------------------------- | | | errors [*required*] | [object] | A list of errors. | | errors | detail | string | A human-readable explanation specific to this occurrence of the error. | | errors | meta | object | Non-standard meta-information about the error | | errors | source | object | References to the source of the error. | | source | header | string | A string indicating the name of a single request header which caused the error. | | source | parameter | string | A string indicating which URI query parameter caused the error. | | source | pointer | string | A JSON pointer to the value in the request document that caused the error. | | errors | status | string | Status code of the response. | | errors | title | string | Short human-readable summary of the error. | {% /tab %} {% tab title="Example" %} ```json { "errors": [ { "detail": "Missing required attribute in body", "meta": {}, "source": { "header": "Authorization", "parameter": "limit", "pointer": "/data/attributes/title" }, "status": "400", "title": "Bad Request" } ] } ``` {% /tab %} {% /tab %} {% tab title="401" %} Unauthorized {% tab title="Model" %} API error response. | Parent field | Field | Type | Description | | ------------ | ------------------------ | -------- | ------------------------------------------------------------------------------- | | | errors [*required*] | [object] | A list of errors. | | errors | detail | string | A human-readable explanation specific to this occurrence of the error. | | errors | meta | object | Non-standard meta-information about the error | | errors | source | object | References to the source of the error. | | source | header | string | A string indicating the name of a single request header which caused the error. | | source | parameter | string | A string indicating which URI query parameter caused the error. | | source | pointer | string | A JSON pointer to the value in the request document that caused the error. | | errors | status | string | Status code of the response. | | errors | title | string | Short human-readable summary of the error. | {% /tab %} {% tab title="Example" %} ```json { "errors": [ { "detail": "Missing required attribute in body", "meta": {}, "source": { "header": "Authorization", "parameter": "limit", "pointer": "/data/attributes/title" }, "status": "400", "title": "Bad Request" } ] } ``` {% /tab %} {% /tab %} {% tab title="403" %} Forbidden {% tab title="Model" %} API error response. | Parent field | Field | Type | Description | | ------------ | ------------------------ | -------- | ------------------------------------------------------------------------------- | | | errors [*required*] | [object] | A list of errors. | | errors | detail | string | A human-readable explanation specific to this occurrence of the error. | | errors | meta | object | Non-standard meta-information about the error | | errors | source | object | References to the source of the error. | | source | header | string | A string indicating the name of a single request header which caused the error. | | source | parameter | string | A string indicating which URI query parameter caused the error. | | source | pointer | string | A JSON pointer to the value in the request document that caused the error. | | errors | status | string | Status code of the response. | | errors | title | string | Short human-readable summary of the error. | {% /tab %} {% tab title="Example" %} ```json { "errors": [ { "detail": "Missing required attribute in body", "meta": {}, "source": { "header": "Authorization", "parameter": "limit", "pointer": "/data/attributes/title" }, "status": "400", "title": "Bad Request" } ] } ``` {% /tab %} {% /tab %} {% tab title="404" %} Not Found {% tab title="Model" %} API error response. | Parent field | Field | Type | Description | | ------------ | ------------------------ | -------- | ------------------------------------------------------------------------------- | | | errors [*required*] | [object] | A list of errors. | | errors | detail | string | A human-readable explanation specific to this occurrence of the error. | | errors | meta | object | Non-standard meta-information about the error | | errors | source | object | References to the source of the error. | | source | header | string | A string indicating the name of a single request header which caused the error. | | source | parameter | string | A string indicating which URI query parameter caused the error. | | source | pointer | string | A JSON pointer to the value in the request document that caused the error. | | errors | status | string | Status code of the response. | | errors | title | string | Short human-readable summary of the error. | {% /tab %} {% tab title="Example" %} ```json { "errors": [ { "detail": "Missing required attribute in body", "meta": {}, "source": { "header": "Authorization", "parameter": "limit", "pointer": "/data/attributes/title" }, "status": "400", "title": "Bad Request" } ] } ``` {% /tab %} {% /tab %} {% tab title="429" %} Too many requests {% tab title="Model" %} API error response. | Field | Type | Description | | ------------------------ | -------- | ----------------- | | errors [*required*] | [string] | A list of errors. | {% /tab %} {% tab title="Example" %} ```json { "errors": [ "Bad Request" ] } ``` {% /tab %} {% /tab %} ### Code Example ##### \# Path parametersexport queue_id="b5e7f3a1-9c2d-4f8b-a1e6-3d7e9f0a2b4c"\# Curl commandcurl -X POST "https://api.ap1.datadoghq.com"https://api.ap2.datadoghq.com"https://api.datadoghq.eu"https://api.ddog-gov.com"https://api.datadoghq.com"https://api.us3.datadoghq.com"https://api.us5.datadoghq.com/api/v2/llm-obs/v1/annotation-queues/${queue_id}/interactions" \ -H "Accept: application/json" \ -H "Content-Type: application/json" \ -H "DD-API-KEY: ${DD_API_KEY}" \ -H "DD-APPLICATION-KEY: ${DD_APP_KEY}" \ -d @- << EOF { "data": { "attributes": { "interactions": [ { "content_id": "trace-abc-123", "type": "trace" } ] }, "type": "interactions" } } EOF {% /tab %} ## Get annotated queue interactions{% #get-annotated-queue-interactions %} {% tab title="v2" %} **Note**: This endpoint is in preview and is subject to change. If you have any feedback, contact [Datadog support](https://docs.datadoghq.com/help/). | Datadog site | API endpoint | | ----------------- | ------------------------------------------------------------------------------------------------------- | | ap1.datadoghq.com | GET https://api.ap1.datadoghq.com/api/v2/llm-obs/v1/annotation-queues/{queue_id}/annotated-interactions | | ap2.datadoghq.com | GET https://api.ap2.datadoghq.com/api/v2/llm-obs/v1/annotation-queues/{queue_id}/annotated-interactions | | app.datadoghq.eu | GET https://api.datadoghq.eu/api/v2/llm-obs/v1/annotation-queues/{queue_id}/annotated-interactions | | app.ddog-gov.com | GET https://api.ddog-gov.com/api/v2/llm-obs/v1/annotation-queues/{queue_id}/annotated-interactions | | app.datadoghq.com | GET https://api.datadoghq.com/api/v2/llm-obs/v1/annotation-queues/{queue_id}/annotated-interactions | | us3.datadoghq.com | GET https://api.us3.datadoghq.com/api/v2/llm-obs/v1/annotation-queues/{queue_id}/annotated-interactions | | us5.datadoghq.com | GET https://api.us5.datadoghq.com/api/v2/llm-obs/v1/annotation-queues/{queue_id}/annotated-interactions | ### Overview Retrieve all interactions and their annotations for a given annotation queue. ### Arguments #### Path Parameters | Name | Type | Description | | -------------------------- | ------ | ------------------------------------------------- | | queue_id [*required*] | string | The ID of the LLM Observability annotation queue. | ### Response {% tab title="200" %} OK {% tab title="Model" %} Response containing the annotated interactions for an annotation queue. | Parent field | Field | Type | Description | | ---------------------- | ---------------------------------------- | --------- | ----------------------------------------------------------------------------------------- | | | data [*required*] | object | Data object for annotated interactions. | | data | attributes [*required*] | object | Attributes containing the list of annotated interactions. | | attributes | annotated_interactions [*required*] | [object] | List of interactions with their annotations. | | annotated_interactions | annotations [*required*] | [object] | List of annotations for this interaction. | | annotations | created_at [*required*] | date-time | Timestamp when the annotation was created. | | annotations | created_by [*required*] | string | Identifier of the user who created the annotation. | | annotations | id [*required*] | string | Unique identifier of the annotation. | | annotations | interaction_id [*required*] | string | Identifier of the interaction this annotation belongs to. | | annotations | label_values [*required*] | object | The label values for this annotation. | | annotations | modified_at [*required*] | date-time | Timestamp when the annotation was last modified. | | annotations | modified_by [*required*] | string | Identifier of the user who last modified the annotation. | | annotated_interactions | content_id [*required*] | string | Identifier of the content for this interaction. | | annotated_interactions | id [*required*] | string | Unique identifier of the interaction. | | annotated_interactions | type [*required*] | enum | Type of interaction in an annotation queue. Allowed enum values: `trace,experiment_trace` | | data | id [*required*] | string | The queue ID. | | data | type [*required*] | enum | Resource type for annotated interactions. Allowed enum values: `annotated_interactions` | {% /tab %} {% tab title="Example" %} ```json { "data": { "attributes": { "annotated_interactions": [ { "annotations": [ { "created_at": "2024-01-15T10:30:00Z", "created_by": "00000000-0000-0000-0000-000000000002", "id": "annotation-789", "interaction_id": "interaction-456", "label_values": { "quality": "good" }, "modified_at": "2024-01-15T10:30:00Z", "modified_by": "00000000-0000-0000-0000-000000000002" } ], "content_id": "trace-abc-123", "id": "interaction-456", "type": "trace" } ] }, "id": "b5e7f3a1-9c2d-4f8b-a1e6-3d7e9f0a2b4c", "type": "annotated_interactions" } } ``` {% /tab %} {% /tab %} {% tab title="400" %} Bad Request {% tab title="Model" %} API error response. | Parent field | Field | Type | Description | | ------------ | ------------------------ | -------- | ------------------------------------------------------------------------------- | | | errors [*required*] | [object] | A list of errors. | | errors | detail | string | A human-readable explanation specific to this occurrence of the error. | | errors | meta | object | Non-standard meta-information about the error | | errors | source | object | References to the source of the error. | | source | header | string | A string indicating the name of a single request header which caused the error. | | source | parameter | string | A string indicating which URI query parameter caused the error. | | source | pointer | string | A JSON pointer to the value in the request document that caused the error. | | errors | status | string | Status code of the response. | | errors | title | string | Short human-readable summary of the error. | {% /tab %} {% tab title="Example" %} ```json { "errors": [ { "detail": "Missing required attribute in body", "meta": {}, "source": { "header": "Authorization", "parameter": "limit", "pointer": "/data/attributes/title" }, "status": "400", "title": "Bad Request" } ] } ``` {% /tab %} {% /tab %} {% tab title="401" %} Unauthorized {% tab title="Model" %} API error response. | Parent field | Field | Type | Description | | ------------ | ------------------------ | -------- | ------------------------------------------------------------------------------- | | | errors [*required*] | [object] | A list of errors. | | errors | detail | string | A human-readable explanation specific to this occurrence of the error. | | errors | meta | object | Non-standard meta-information about the error | | errors | source | object | References to the source of the error. | | source | header | string | A string indicating the name of a single request header which caused the error. | | source | parameter | string | A string indicating which URI query parameter caused the error. | | source | pointer | string | A JSON pointer to the value in the request document that caused the error. | | errors | status | string | Status code of the response. | | errors | title | string | Short human-readable summary of the error. | {% /tab %} {% tab title="Example" %} ```json { "errors": [ { "detail": "Missing required attribute in body", "meta": {}, "source": { "header": "Authorization", "parameter": "limit", "pointer": "/data/attributes/title" }, "status": "400", "title": "Bad Request" } ] } ``` {% /tab %} {% /tab %} {% tab title="403" %} Forbidden {% tab title="Model" %} API error response. | Parent field | Field | Type | Description | | ------------ | ------------------------ | -------- | ------------------------------------------------------------------------------- | | | errors [*required*] | [object] | A list of errors. | | errors | detail | string | A human-readable explanation specific to this occurrence of the error. | | errors | meta | object | Non-standard meta-information about the error | | errors | source | object | References to the source of the error. | | source | header | string | A string indicating the name of a single request header which caused the error. | | source | parameter | string | A string indicating which URI query parameter caused the error. | | source | pointer | string | A JSON pointer to the value in the request document that caused the error. | | errors | status | string | Status code of the response. | | errors | title | string | Short human-readable summary of the error. | {% /tab %} {% tab title="Example" %} ```json { "errors": [ { "detail": "Missing required attribute in body", "meta": {}, "source": { "header": "Authorization", "parameter": "limit", "pointer": "/data/attributes/title" }, "status": "400", "title": "Bad Request" } ] } ``` {% /tab %} {% /tab %} {% tab title="404" %} Not Found {% tab title="Model" %} API error response. | Parent field | Field | Type | Description | | ------------ | ------------------------ | -------- | ------------------------------------------------------------------------------- | | | errors [*required*] | [object] | A list of errors. | | errors | detail | string | A human-readable explanation specific to this occurrence of the error. | | errors | meta | object | Non-standard meta-information about the error | | errors | source | object | References to the source of the error. | | source | header | string | A string indicating the name of a single request header which caused the error. | | source | parameter | string | A string indicating which URI query parameter caused the error. | | source | pointer | string | A JSON pointer to the value in the request document that caused the error. | | errors | status | string | Status code of the response. | | errors | title | string | Short human-readable summary of the error. | {% /tab %} {% tab title="Example" %} ```json { "errors": [ { "detail": "Missing required attribute in body", "meta": {}, "source": { "header": "Authorization", "parameter": "limit", "pointer": "/data/attributes/title" }, "status": "400", "title": "Bad Request" } ] } ``` {% /tab %} {% /tab %} {% tab title="429" %} Too many requests {% tab title="Model" %} API error response. | Field | Type | Description | | ------------------------ | -------- | ----------------- | | errors [*required*] | [string] | A list of errors. | {% /tab %} {% tab title="Example" %} ```json { "errors": [ "Bad Request" ] } ``` {% /tab %} {% /tab %} ### Code Example ##### \# Path parametersexport queue_id="b5e7f3a1-9c2d-4f8b-a1e6-3d7e9f0a2b4c"\# Curl commandcurl -X GET "https://api.ap1.datadoghq.com"https://api.ap2.datadoghq.com"https://api.datadoghq.eu"https://api.ddog-gov.com"https://api.datadoghq.com"https://api.us3.datadoghq.com"https://api.us5.datadoghq.com/api/v2/llm-obs/v1/annotation-queues/${queue_id}/annotated-interactions" \ -H "Accept: application/json" \ -H "DD-API-KEY: ${DD_API_KEY}" \ -H "DD-APPLICATION-KEY: ${DD_APP_KEY}" {% /tab %} ## Update an LLM Observability annotation queue{% #update-an-llm-observability-annotation-queue %} {% tab title="v2" %} **Note**: This endpoint is in preview and is subject to change. If you have any feedback, contact [Datadog support](https://docs.datadoghq.com/help/). | Datadog site | API endpoint | | ----------------- | ---------------------------------------------------------------------------------- | | ap1.datadoghq.com | PATCH https://api.ap1.datadoghq.com/api/v2/llm-obs/v1/annotation-queues/{queue_id} | | ap2.datadoghq.com | PATCH https://api.ap2.datadoghq.com/api/v2/llm-obs/v1/annotation-queues/{queue_id} | | app.datadoghq.eu | PATCH https://api.datadoghq.eu/api/v2/llm-obs/v1/annotation-queues/{queue_id} | | app.ddog-gov.com | PATCH https://api.ddog-gov.com/api/v2/llm-obs/v1/annotation-queues/{queue_id} | | app.datadoghq.com | PATCH https://api.datadoghq.com/api/v2/llm-obs/v1/annotation-queues/{queue_id} | | us3.datadoghq.com | PATCH https://api.us3.datadoghq.com/api/v2/llm-obs/v1/annotation-queues/{queue_id} | | us5.datadoghq.com | PATCH https://api.us5.datadoghq.com/api/v2/llm-obs/v1/annotation-queues/{queue_id} | ### Overview Partially update an annotation queue. Only `name` and `description` can be updated. ### Arguments #### Path Parameters | Name | Type | Description | | -------------------------- | ------ | ------------------------------------------------- | | queue_id [*required*] | string | The ID of the LLM Observability annotation queue. | ### Request #### Body Data (required) Update annotation queue payload. {% tab title="Model" %} | Parent field | Field | Type | Description | | ------------ | ---------------------------- | ------ | --------------------------------------------------------------------------------------- | | | data [*required*] | object | Data object for updating an LLM Observability annotation queue. | | data | attributes [*required*] | object | Attributes for updating an LLM Observability annotation queue. All fields are optional. | | attributes | description | string | Updated description of the annotation queue. | | attributes | name | string | Updated name of the annotation queue. | | data | type [*required*] | enum | Resource type of an LLM Observability annotation queue. Allowed enum values: `queues` | {% /tab %} {% tab title="Example" %} ```json { "data": { "attributes": { "description": "Updated description", "name": "Updated queue name" }, "type": "queues" } } ``` {% /tab %} ### Response {% tab title="200" %} OK {% tab title="Model" %} Response containing a single LLM Observability annotation queue. | Parent field | Field | Type | Description | | ------------ | ----------------------------- | --------- | ------------------------------------------------------------------------------------- | | | data [*required*] | object | Data object for an LLM Observability annotation queue. | | data | attributes [*required*] | object | Attributes of an LLM Observability annotation queue. | | attributes | created_at [*required*] | date-time | Timestamp when the queue was created. | | attributes | created_by [*required*] | string | Identifier of the user who created the queue. | | attributes | description [*required*] | string | Description of the annotation queue. | | attributes | modified_at [*required*] | date-time | Timestamp when the queue was last modified. | | attributes | modified_by [*required*] | string | Identifier of the user who last modified the queue. | | attributes | name [*required*] | string | Name of the annotation queue. | | attributes | owned_by [*required*] | string | Identifier of the user who owns the queue. | | attributes | project_id [*required*] | string | Identifier of the project this queue belongs to. | | data | id [*required*] | string | Unique identifier of the annotation queue. | | data | type [*required*] | enum | Resource type of an LLM Observability annotation queue. Allowed enum values: `queues` | {% /tab %} {% tab title="Example" %} ```json { "data": { "attributes": { "created_at": "2024-01-15T10:30:00Z", "created_by": "00000000-0000-0000-0000-000000000002", "description": "Queue for annotating customer support traces", "modified_at": "2024-01-15T10:30:00Z", "modified_by": "00000000-0000-0000-0000-000000000002", "name": "My annotation queue", "owned_by": "00000000-0000-0000-0000-000000000002", "project_id": "a33671aa-24fd-4dcd-9b33-a8ec7dde7751" }, "id": "b5e7f3a1-9c2d-4f8b-a1e6-3d7e9f0a2b4c", "type": "queues" } } ``` {% /tab %} {% /tab %} {% tab title="400" %} Bad Request {% tab title="Model" %} API error response. | Parent field | Field | Type | Description | | ------------ | ------------------------ | -------- | ------------------------------------------------------------------------------- | | | errors [*required*] | [object] | A list of errors. | | errors | detail | string | A human-readable explanation specific to this occurrence of the error. | | errors | meta | object | Non-standard meta-information about the error | | errors | source | object | References to the source of the error. | | source | header | string | A string indicating the name of a single request header which caused the error. | | source | parameter | string | A string indicating which URI query parameter caused the error. | | source | pointer | string | A JSON pointer to the value in the request document that caused the error. | | errors | status | string | Status code of the response. | | errors | title | string | Short human-readable summary of the error. | {% /tab %} {% tab title="Example" %} ```json { "errors": [ { "detail": "Missing required attribute in body", "meta": {}, "source": { "header": "Authorization", "parameter": "limit", "pointer": "/data/attributes/title" }, "status": "400", "title": "Bad Request" } ] } ``` {% /tab %} {% /tab %} {% tab title="401" %} Unauthorized {% tab title="Model" %} API error response. | Parent field | Field | Type | Description | | ------------ | ------------------------ | -------- | ------------------------------------------------------------------------------- | | | errors [*required*] | [object] | A list of errors. | | errors | detail | string | A human-readable explanation specific to this occurrence of the error. | | errors | meta | object | Non-standard meta-information about the error | | errors | source | object | References to the source of the error. | | source | header | string | A string indicating the name of a single request header which caused the error. | | source | parameter | string | A string indicating which URI query parameter caused the error. | | source | pointer | string | A JSON pointer to the value in the request document that caused the error. | | errors | status | string | Status code of the response. | | errors | title | string | Short human-readable summary of the error. | {% /tab %} {% tab title="Example" %} ```json { "errors": [ { "detail": "Missing required attribute in body", "meta": {}, "source": { "header": "Authorization", "parameter": "limit", "pointer": "/data/attributes/title" }, "status": "400", "title": "Bad Request" } ] } ``` {% /tab %} {% /tab %} {% tab title="403" %} Forbidden {% tab title="Model" %} API error response. | Parent field | Field | Type | Description | | ------------ | ------------------------ | -------- | ------------------------------------------------------------------------------- | | | errors [*required*] | [object] | A list of errors. | | errors | detail | string | A human-readable explanation specific to this occurrence of the error. | | errors | meta | object | Non-standard meta-information about the error | | errors | source | object | References to the source of the error. | | source | header | string | A string indicating the name of a single request header which caused the error. | | source | parameter | string | A string indicating which URI query parameter caused the error. | | source | pointer | string | A JSON pointer to the value in the request document that caused the error. | | errors | status | string | Status code of the response. | | errors | title | string | Short human-readable summary of the error. | {% /tab %} {% tab title="Example" %} ```json { "errors": [ { "detail": "Missing required attribute in body", "meta": {}, "source": { "header": "Authorization", "parameter": "limit", "pointer": "/data/attributes/title" }, "status": "400", "title": "Bad Request" } ] } ``` {% /tab %} {% /tab %} {% tab title="404" %} Not Found {% tab title="Model" %} API error response. | Parent field | Field | Type | Description | | ------------ | ------------------------ | -------- | ------------------------------------------------------------------------------- | | | errors [*required*] | [object] | A list of errors. | | errors | detail | string | A human-readable explanation specific to this occurrence of the error. | | errors | meta | object | Non-standard meta-information about the error | | errors | source | object | References to the source of the error. | | source | header | string | A string indicating the name of a single request header which caused the error. | | source | parameter | string | A string indicating which URI query parameter caused the error. | | source | pointer | string | A JSON pointer to the value in the request document that caused the error. | | errors | status | string | Status code of the response. | | errors | title | string | Short human-readable summary of the error. | {% /tab %} {% tab title="Example" %} ```json { "errors": [ { "detail": "Missing required attribute in body", "meta": {}, "source": { "header": "Authorization", "parameter": "limit", "pointer": "/data/attributes/title" }, "status": "400", "title": "Bad Request" } ] } ``` {% /tab %} {% /tab %} {% tab title="429" %} Too many requests {% tab title="Model" %} API error response. | Field | Type | Description | | ------------------------ | -------- | ----------------- | | errors [*required*] | [string] | A list of errors. | {% /tab %} {% tab title="Example" %} ```json { "errors": [ "Bad Request" ] } ``` {% /tab %} {% /tab %} ### Code Example ##### \# Path parametersexport queue_id="b5e7f3a1-9c2d-4f8b-a1e6-3d7e9f0a2b4c"\# Curl commandcurl -X PATCH "https://api.ap1.datadoghq.com"https://api.ap2.datadoghq.com"https://api.datadoghq.eu"https://api.ddog-gov.com"https://api.datadoghq.com"https://api.us3.datadoghq.com"https://api.us5.datadoghq.com/api/v2/llm-obs/v1/annotation-queues/${queue_id}" \ -H "Accept: application/json" \ -H "Content-Type: application/json" \ -H "DD-API-KEY: ${DD_API_KEY}" \ -H "DD-APPLICATION-KEY: ${DD_APP_KEY}" \ -d @- << EOF { "data": { "attributes": {}, "type": "queues" } } EOF {% /tab %} ## Delete an LLM Observability annotation queue{% #delete-an-llm-observability-annotation-queue %} {% tab title="v2" %} **Note**: This endpoint is in preview and is subject to change. If you have any feedback, contact [Datadog support](https://docs.datadoghq.com/help/). | Datadog site | API endpoint | | ----------------- | ----------------------------------------------------------------------------------- | | ap1.datadoghq.com | DELETE https://api.ap1.datadoghq.com/api/v2/llm-obs/v1/annotation-queues/{queue_id} | | ap2.datadoghq.com | DELETE https://api.ap2.datadoghq.com/api/v2/llm-obs/v1/annotation-queues/{queue_id} | | app.datadoghq.eu | DELETE https://api.datadoghq.eu/api/v2/llm-obs/v1/annotation-queues/{queue_id} | | app.ddog-gov.com | DELETE https://api.ddog-gov.com/api/v2/llm-obs/v1/annotation-queues/{queue_id} | | app.datadoghq.com | DELETE https://api.datadoghq.com/api/v2/llm-obs/v1/annotation-queues/{queue_id} | | us3.datadoghq.com | DELETE https://api.us3.datadoghq.com/api/v2/llm-obs/v1/annotation-queues/{queue_id} | | us5.datadoghq.com | DELETE https://api.us5.datadoghq.com/api/v2/llm-obs/v1/annotation-queues/{queue_id} | ### Overview Delete an annotation queue by its ID. ### Arguments #### Path Parameters | Name | Type | Description | | -------------------------- | ------ | ------------------------------------------------- | | queue_id [*required*] | string | The ID of the LLM Observability annotation queue. | ### Response {% tab title="204" %} No Content {% /tab %} {% tab title="401" %} Unauthorized {% tab title="Model" %} API error response. | Parent field | Field | Type | Description | | ------------ | ------------------------ | -------- | ------------------------------------------------------------------------------- | | | errors [*required*] | [object] | A list of errors. | | errors | detail | string | A human-readable explanation specific to this occurrence of the error. | | errors | meta | object | Non-standard meta-information about the error | | errors | source | object | References to the source of the error. | | source | header | string | A string indicating the name of a single request header which caused the error. | | source | parameter | string | A string indicating which URI query parameter caused the error. | | source | pointer | string | A JSON pointer to the value in the request document that caused the error. | | errors | status | string | Status code of the response. | | errors | title | string | Short human-readable summary of the error. | {% /tab %} {% tab title="Example" %} ```json { "errors": [ { "detail": "Missing required attribute in body", "meta": {}, "source": { "header": "Authorization", "parameter": "limit", "pointer": "/data/attributes/title" }, "status": "400", "title": "Bad Request" } ] } ``` {% /tab %} {% /tab %} {% tab title="403" %} Forbidden {% tab title="Model" %} API error response. | Parent field | Field | Type | Description | | ------------ | ------------------------ | -------- | ------------------------------------------------------------------------------- | | | errors [*required*] | [object] | A list of errors. | | errors | detail | string | A human-readable explanation specific to this occurrence of the error. | | errors | meta | object | Non-standard meta-information about the error | | errors | source | object | References to the source of the error. | | source | header | string | A string indicating the name of a single request header which caused the error. | | source | parameter | string | A string indicating which URI query parameter caused the error. | | source | pointer | string | A JSON pointer to the value in the request document that caused the error. | | errors | status | string | Status code of the response. | | errors | title | string | Short human-readable summary of the error. | {% /tab %} {% tab title="Example" %} ```json { "errors": [ { "detail": "Missing required attribute in body", "meta": {}, "source": { "header": "Authorization", "parameter": "limit", "pointer": "/data/attributes/title" }, "status": "400", "title": "Bad Request" } ] } ``` {% /tab %} {% /tab %} {% tab title="404" %} Not Found {% tab title="Model" %} API error response. | Parent field | Field | Type | Description | | ------------ | ------------------------ | -------- | ------------------------------------------------------------------------------- | | | errors [*required*] | [object] | A list of errors. | | errors | detail | string | A human-readable explanation specific to this occurrence of the error. | | errors | meta | object | Non-standard meta-information about the error | | errors | source | object | References to the source of the error. | | source | header | string | A string indicating the name of a single request header which caused the error. | | source | parameter | string | A string indicating which URI query parameter caused the error. | | source | pointer | string | A JSON pointer to the value in the request document that caused the error. | | errors | status | string | Status code of the response. | | errors | title | string | Short human-readable summary of the error. | {% /tab %} {% tab title="Example" %} ```json { "errors": [ { "detail": "Missing required attribute in body", "meta": {}, "source": { "header": "Authorization", "parameter": "limit", "pointer": "/data/attributes/title" }, "status": "400", "title": "Bad Request" } ] } ``` {% /tab %} {% /tab %} {% tab title="429" %} Too many requests {% tab title="Model" %} API error response. | Field | Type | Description | | ------------------------ | -------- | ----------------- | | errors [*required*] | [string] | A list of errors. | {% /tab %} {% tab title="Example" %} ```json { "errors": [ "Bad Request" ] } ``` {% /tab %} {% /tab %} ### Code Example ##### \# Path parametersexport queue_id="b5e7f3a1-9c2d-4f8b-a1e6-3d7e9f0a2b4c"\# Curl commandcurl -X DELETE "https://api.ap1.datadoghq.com"https://api.ap2.datadoghq.com"https://api.datadoghq.eu"https://api.ddog-gov.com"https://api.datadoghq.com"https://api.us3.datadoghq.com"https://api.us5.datadoghq.com/api/v2/llm-obs/v1/annotation-queues/${queue_id}" \ -H "DD-API-KEY: ${DD_API_KEY}" \ -H "DD-APPLICATION-KEY: ${DD_APP_KEY}" {% /tab %} ## Delete annotation queue interactions{% #delete-annotation-queue-interactions %} {% tab title="v2" %} **Note**: This endpoint is in preview and is subject to change. If you have any feedback, contact [Datadog support](https://docs.datadoghq.com/help/). | Datadog site | API endpoint | | ----------------- | ----------------------------------------------------------------------------------------------------- | | ap1.datadoghq.com | POST https://api.ap1.datadoghq.com/api/v2/llm-obs/v1/annotation-queues/{queue_id}/interactions/delete | | ap2.datadoghq.com | POST https://api.ap2.datadoghq.com/api/v2/llm-obs/v1/annotation-queues/{queue_id}/interactions/delete | | app.datadoghq.eu | POST https://api.datadoghq.eu/api/v2/llm-obs/v1/annotation-queues/{queue_id}/interactions/delete | | app.ddog-gov.com | POST https://api.ddog-gov.com/api/v2/llm-obs/v1/annotation-queues/{queue_id}/interactions/delete | | app.datadoghq.com | POST https://api.datadoghq.com/api/v2/llm-obs/v1/annotation-queues/{queue_id}/interactions/delete | | us3.datadoghq.com | POST https://api.us3.datadoghq.com/api/v2/llm-obs/v1/annotation-queues/{queue_id}/interactions/delete | | us5.datadoghq.com | POST https://api.us5.datadoghq.com/api/v2/llm-obs/v1/annotation-queues/{queue_id}/interactions/delete | ### Overview Delete one or more interactions from an annotation queue. ### Arguments #### Path Parameters | Name | Type | Description | | -------------------------- | ------ | ------------------------------------------------- | | queue_id [*required*] | string | The ID of the LLM Observability annotation queue. | ### Request #### Body Data (required) Delete interactions payload. {% tab title="Model" %} | Parent field | Field | Type | Description | | ------------ | --------------------------------- | -------- | ------------------------------------------------------------------------------------ | | | data [*required*] | object | Data object for deleting interactions from an annotation queue. | | data | attributes [*required*] | object | Attributes for deleting interactions from an annotation queue. | | attributes | interaction_ids [*required*] | [string] | List of interaction IDs to delete. Must contain at least one item. | | data | type [*required*] | enum | Resource type for annotation queue interactions. Allowed enum values: `interactions` | {% /tab %} {% tab title="Example" %} ```json { "data": { "attributes": { "interaction_ids": [ "00000000-0000-0000-0000-000000000000", "00000000-0000-0000-0000-000000000001" ] }, "type": "interactions" } } ``` {% /tab %} ### Response {% tab title="204" %} No Content {% /tab %} {% tab title="400" %} Bad Request {% tab title="Model" %} API error response. | Parent field | Field | Type | Description | | ------------ | ------------------------ | -------- | ------------------------------------------------------------------------------- | | | errors [*required*] | [object] | A list of errors. | | errors | detail | string | A human-readable explanation specific to this occurrence of the error. | | errors | meta | object | Non-standard meta-information about the error | | errors | source | object | References to the source of the error. | | source | header | string | A string indicating the name of a single request header which caused the error. | | source | parameter | string | A string indicating which URI query parameter caused the error. | | source | pointer | string | A JSON pointer to the value in the request document that caused the error. | | errors | status | string | Status code of the response. | | errors | title | string | Short human-readable summary of the error. | {% /tab %} {% tab title="Example" %} ```json { "errors": [ { "detail": "Missing required attribute in body", "meta": {}, "source": { "header": "Authorization", "parameter": "limit", "pointer": "/data/attributes/title" }, "status": "400", "title": "Bad Request" } ] } ``` {% /tab %} {% /tab %} {% tab title="401" %} Unauthorized {% tab title="Model" %} API error response. | Parent field | Field | Type | Description | | ------------ | ------------------------ | -------- | ------------------------------------------------------------------------------- | | | errors [*required*] | [object] | A list of errors. | | errors | detail | string | A human-readable explanation specific to this occurrence of the error. | | errors | meta | object | Non-standard meta-information about the error | | errors | source | object | References to the source of the error. | | source | header | string | A string indicating the name of a single request header which caused the error. | | source | parameter | string | A string indicating which URI query parameter caused the error. | | source | pointer | string | A JSON pointer to the value in the request document that caused the error. | | errors | status | string | Status code of the response. | | errors | title | string | Short human-readable summary of the error. | {% /tab %} {% tab title="Example" %} ```json { "errors": [ { "detail": "Missing required attribute in body", "meta": {}, "source": { "header": "Authorization", "parameter": "limit", "pointer": "/data/attributes/title" }, "status": "400", "title": "Bad Request" } ] } ``` {% /tab %} {% /tab %} {% tab title="403" %} Forbidden {% tab title="Model" %} API error response. | Parent field | Field | Type | Description | | ------------ | ------------------------ | -------- | ------------------------------------------------------------------------------- | | | errors [*required*] | [object] | A list of errors. | | errors | detail | string | A human-readable explanation specific to this occurrence of the error. | | errors | meta | object | Non-standard meta-information about the error | | errors | source | object | References to the source of the error. | | source | header | string | A string indicating the name of a single request header which caused the error. | | source | parameter | string | A string indicating which URI query parameter caused the error. | | source | pointer | string | A JSON pointer to the value in the request document that caused the error. | | errors | status | string | Status code of the response. | | errors | title | string | Short human-readable summary of the error. | {% /tab %} {% tab title="Example" %} ```json { "errors": [ { "detail": "Missing required attribute in body", "meta": {}, "source": { "header": "Authorization", "parameter": "limit", "pointer": "/data/attributes/title" }, "status": "400", "title": "Bad Request" } ] } ``` {% /tab %} {% /tab %} {% tab title="404" %} Not Found {% tab title="Model" %} API error response. | Parent field | Field | Type | Description | | ------------ | ------------------------ | -------- | ------------------------------------------------------------------------------- | | | errors [*required*] | [object] | A list of errors. | | errors | detail | string | A human-readable explanation specific to this occurrence of the error. | | errors | meta | object | Non-standard meta-information about the error | | errors | source | object | References to the source of the error. | | source | header | string | A string indicating the name of a single request header which caused the error. | | source | parameter | string | A string indicating which URI query parameter caused the error. | | source | pointer | string | A JSON pointer to the value in the request document that caused the error. | | errors | status | string | Status code of the response. | | errors | title | string | Short human-readable summary of the error. | {% /tab %} {% tab title="Example" %} ```json { "errors": [ { "detail": "Missing required attribute in body", "meta": {}, "source": { "header": "Authorization", "parameter": "limit", "pointer": "/data/attributes/title" }, "status": "400", "title": "Bad Request" } ] } ``` {% /tab %} {% /tab %} {% tab title="429" %} Too many requests {% tab title="Model" %} API error response. | Field | Type | Description | | ------------------------ | -------- | ----------------- | | errors [*required*] | [string] | A list of errors. | {% /tab %} {% tab title="Example" %} ```json { "errors": [ "Bad Request" ] } ``` {% /tab %} {% /tab %} ### Code Example ##### \# Path parametersexport queue_id="b5e7f3a1-9c2d-4f8b-a1e6-3d7e9f0a2b4c"\# Curl commandcurl -X POST "https://api.ap1.datadoghq.com"https://api.ap2.datadoghq.com"https://api.datadoghq.eu"https://api.ddog-gov.com"https://api.datadoghq.com"https://api.us3.datadoghq.com"https://api.us5.datadoghq.com/api/v2/llm-obs/v1/annotation-queues/${queue_id}/interactions/delete" \ -H "Content-Type: application/json" \ -H "DD-API-KEY: ${DD_API_KEY}" \ -H "DD-APPLICATION-KEY: ${DD_APP_KEY}" \ -d @- << EOF { "data": { "attributes": { "interaction_ids": [ "00000000-0000-0000-0000-000000000000", "00000000-0000-0000-0000-000000000001" ] }, "type": "interactions" } } EOF {% /tab %}