Language

English Français 日本語 한국어 Español

Datadog Site

Site help
US1 US3 US5 EU AP1 AP2 US1-FED

LLM Observability

Manage LLM Observability projects, datasets, dataset records, and experiments via the Experiments API.

List LLM Observability projects

Note: This endpoint is in preview and is subject to change. If you have any feedback, contact Datadog support.

GET https://api.ap1.datadoghq.com/api/v2/llm-obs/v1/projectshttps://api.ap2.datadoghq.com/api/v2/llm-obs/v1/projectshttps://api.datadoghq.eu/api/v2/llm-obs/v1/projectshttps://api.ddog-gov.com/api/v2/llm-obs/v1/projectshttps://api.datadoghq.com/api/v2/llm-obs/v1/projectshttps://api.us3.datadoghq.com/api/v2/llm-obs/v1/projectshttps://api.us5.datadoghq.com/api/v2/llm-obs/v1/projects

Overview

List all LLM Observability projects sorted by creation date, newest first.

Arguments

Query Strings

Name

Type

Description

filter[id]

string

Filter projects by project ID.

filter[name]

string

Filter projects by name.

page[cursor]

string

Use the Pagination cursor to retrieve the next page of results.

page[limit]

integer

Maximum number of results to return per page.

Response

OK

Response containing a list of LLM Observability projects.

Expand All

Field

Type

Description

data [required]

[object]

List of projects.

attributes [required]

object

Attributes of an LLM Observability project.

created_at [required]

date-time

Timestamp when the project was created.

description [required]

string

Description of the project.

name [required]

string

Name of the project.

updated_at [required]

date-time

Timestamp when the project was last updated.

id [required]

string

Unique identifier of the project.

type [required]

enum

Resource type of an LLM Observability project. Allowed enum values: projects

meta

object

Pagination cursor metadata.

after

string

Cursor for the next page of results.

{
  "data": [
    {
      "attributes": {
        "created_at": "2024-01-15T10:30:00Z",
        "description": "",
        "name": "My LLM Project",
        "updated_at": "2024-01-15T10:30:00Z"
      },
      "id": "a33671aa-24fd-4dcd-9b33-a8ec7dde7751",
      "type": "projects"
    }
  ],
  "meta": {
    "after": "string"
  }
}

Bad Request

API error response.

Expand All

Field

Type

Description

errors [required]

[object]

A list of errors.

detail

string

A human-readable explanation specific to this occurrence of the error.

meta

object

Non-standard meta-information about the error

source

object

References to the source of the error.

header

string

A string indicating the name of a single request header which caused the error.

parameter

string

A string indicating which URI query parameter caused the error.

pointer

string

A JSON pointer to the value in the request document that caused the error.

status

string

Status code of the response.

title

string

Short human-readable summary of the error.

{
  "errors": [
    {
      "detail": "Missing required attribute in body",
      "meta": {},
      "source": {
        "header": "Authorization",
        "parameter": "limit",
        "pointer": "/data/attributes/title"
      },
      "status": "400",
      "title": "Bad Request"
    }
  ]
}

Unauthorized

API error response.

Expand All

Field

Type

Description

errors [required]

[object]

A list of errors.

detail

string

A human-readable explanation specific to this occurrence of the error.

meta

object

Non-standard meta-information about the error

source

object

References to the source of the error.

header

string

A string indicating the name of a single request header which caused the error.

parameter

string

A string indicating which URI query parameter caused the error.

pointer

string

A JSON pointer to the value in the request document that caused the error.

status

string

Status code of the response.

title

string

Short human-readable summary of the error.

{
  "errors": [
    {
      "detail": "Missing required attribute in body",
      "meta": {},
      "source": {
        "header": "Authorization",
        "parameter": "limit",
        "pointer": "/data/attributes/title"
      },
      "status": "400",
      "title": "Bad Request"
    }
  ]
}

Forbidden

API error response.

Expand All

Field

Type

Description

errors [required]

[object]

A list of errors.

detail

string

A human-readable explanation specific to this occurrence of the error.

meta

object

Non-standard meta-information about the error

source

object

References to the source of the error.

header

string

A string indicating the name of a single request header which caused the error.

parameter

string

A string indicating which URI query parameter caused the error.

pointer

string

A JSON pointer to the value in the request document that caused the error.

status

string

Status code of the response.

title

string

Short human-readable summary of the error.

{
  "errors": [
    {
      "detail": "Missing required attribute in body",
      "meta": {},
      "source": {
        "header": "Authorization",
        "parameter": "limit",
        "pointer": "/data/attributes/title"
      },
      "status": "400",
      "title": "Bad Request"
    }
  ]
}

Too many requests

API error response.

Expand All

Field

Type

Description

errors [required]

[string]

A list of errors.

{
  "errors": [
    "Bad Request"
  ]
}

Code Example

                  # Curl command
curl -X GET "https://api.ap1.datadoghq.com"https://api.ap2.datadoghq.com"https://api.datadoghq.eu"https://api.ddog-gov.com"https://api.datadoghq.com"https://api.us3.datadoghq.com"https://api.us5.datadoghq.com/api/v2/llm-obs/v1/projects" \
-H "Accept: application/json" \
-H "DD-API-KEY: ${DD_API_KEY}" \
-H "DD-APPLICATION-KEY: ${DD_APP_KEY}"

Create an LLM Observability project

Note: This endpoint is in preview and is subject to change. If you have any feedback, contact Datadog support.

POST https://api.ap1.datadoghq.com/api/v2/llm-obs/v1/projectshttps://api.ap2.datadoghq.com/api/v2/llm-obs/v1/projectshttps://api.datadoghq.eu/api/v2/llm-obs/v1/projectshttps://api.ddog-gov.com/api/v2/llm-obs/v1/projectshttps://api.datadoghq.com/api/v2/llm-obs/v1/projectshttps://api.us3.datadoghq.com/api/v2/llm-obs/v1/projectshttps://api.us5.datadoghq.com/api/v2/llm-obs/v1/projects

Overview

Create a new LLM Observability project. Returns the existing project if a name conflict occurs.

Request

Body Data (required)

Create project payload.

Expand All

Field

Type

Description

data [required]

object

Data object for creating an LLM Observability project.

attributes [required]

object

Attributes for creating an LLM Observability project.

description

string

Description of the project.

name [required]

string

Name of the project.

type [required]

enum

Resource type of an LLM Observability project. Allowed enum values: projects

{
  "data": {
    "attributes": {
      "description": "string",
      "name": "My LLM Project"
    },
    "type": "projects"
  }
}

Response

OK

Response containing a single LLM Observability project.

Expand All

Field

Type

Description

data [required]

object

Data object for an LLM Observability project.

attributes [required]

object

Attributes of an LLM Observability project.

created_at [required]

date-time

Timestamp when the project was created.

description [required]

string

Description of the project.

name [required]

string

Name of the project.

updated_at [required]

date-time

Timestamp when the project was last updated.

id [required]

string

Unique identifier of the project.

type [required]

enum

Resource type of an LLM Observability project. Allowed enum values: projects

{
  "data": {
    "attributes": {
      "created_at": "2024-01-15T10:30:00Z",
      "description": "",
      "name": "My LLM Project",
      "updated_at": "2024-01-15T10:30:00Z"
    },
    "id": "a33671aa-24fd-4dcd-9b33-a8ec7dde7751",
    "type": "projects"
  }
}

Created

Response containing a single LLM Observability project.

Expand All

Field

Type

Description

data [required]

object

Data object for an LLM Observability project.

attributes [required]

object

Attributes of an LLM Observability project.

created_at [required]

date-time

Timestamp when the project was created.

description [required]

string

Description of the project.

name [required]

string

Name of the project.

updated_at [required]

date-time

Timestamp when the project was last updated.

id [required]

string

Unique identifier of the project.

type [required]

enum

Resource type of an LLM Observability project. Allowed enum values: projects

{
  "data": {
    "attributes": {
      "created_at": "2024-01-15T10:30:00Z",
      "description": "",
      "name": "My LLM Project",
      "updated_at": "2024-01-15T10:30:00Z"
    },
    "id": "a33671aa-24fd-4dcd-9b33-a8ec7dde7751",
    "type": "projects"
  }
}

Bad Request

API error response.

Expand All

Field

Type

Description

errors [required]

[object]

A list of errors.

detail

string

A human-readable explanation specific to this occurrence of the error.

meta

object

Non-standard meta-information about the error

source

object

References to the source of the error.

header

string

A string indicating the name of a single request header which caused the error.

parameter

string

A string indicating which URI query parameter caused the error.

pointer

string

A JSON pointer to the value in the request document that caused the error.

status

string

Status code of the response.

title

string

Short human-readable summary of the error.

{
  "errors": [
    {
      "detail": "Missing required attribute in body",
      "meta": {},
      "source": {
        "header": "Authorization",
        "parameter": "limit",
        "pointer": "/data/attributes/title"
      },
      "status": "400",
      "title": "Bad Request"
    }
  ]
}

Unauthorized

API error response.

Expand All

Field

Type

Description

errors [required]

[object]

A list of errors.

detail

string

A human-readable explanation specific to this occurrence of the error.

meta

object

Non-standard meta-information about the error

source

object

References to the source of the error.

header

string

A string indicating the name of a single request header which caused the error.

parameter

string

A string indicating which URI query parameter caused the error.

pointer

string

A JSON pointer to the value in the request document that caused the error.

status

string

Status code of the response.

title

string

Short human-readable summary of the error.

{
  "errors": [
    {
      "detail": "Missing required attribute in body",
      "meta": {},
      "source": {
        "header": "Authorization",
        "parameter": "limit",
        "pointer": "/data/attributes/title"
      },
      "status": "400",
      "title": "Bad Request"
    }
  ]
}

Forbidden

API error response.

Expand All

Field

Type

Description

errors [required]

[object]

A list of errors.

detail

string

A human-readable explanation specific to this occurrence of the error.

meta

object

Non-standard meta-information about the error

source

object

References to the source of the error.

header

string

A string indicating the name of a single request header which caused the error.

parameter

string

A string indicating which URI query parameter caused the error.

pointer

string

A JSON pointer to the value in the request document that caused the error.

status

string

Status code of the response.

title

string

Short human-readable summary of the error.

{
  "errors": [
    {
      "detail": "Missing required attribute in body",
      "meta": {},
      "source": {
        "header": "Authorization",
        "parameter": "limit",
        "pointer": "/data/attributes/title"
      },
      "status": "400",
      "title": "Bad Request"
    }
  ]
}

Too many requests

API error response.

Expand All

Field

Type

Description

errors [required]

[string]

A list of errors.

{
  "errors": [
    "Bad Request"
  ]
}

Code Example

                  # Curl command
curl -X POST "https://api.ap1.datadoghq.com"https://api.ap2.datadoghq.com"https://api.datadoghq.eu"https://api.ddog-gov.com"https://api.datadoghq.com"https://api.us3.datadoghq.com"https://api.us5.datadoghq.com/api/v2/llm-obs/v1/projects" \
-H "Accept: application/json" \
-H "Content-Type: application/json" \
-H "DD-API-KEY: ${DD_API_KEY}" \
-H "DD-APPLICATION-KEY: ${DD_APP_KEY}" \
-d @- << EOF
{
  "data": {
    "attributes": {
      "name": "My LLM Project"
    },
    "type": "projects"
  }
}
EOF

Update an LLM Observability project

Note: This endpoint is in preview and is subject to change. If you have any feedback, contact Datadog support.

PATCH https://api.ap1.datadoghq.com/api/v2/llm-obs/v1/projects/{project_id}https://api.ap2.datadoghq.com/api/v2/llm-obs/v1/projects/{project_id}https://api.datadoghq.eu/api/v2/llm-obs/v1/projects/{project_id}https://api.ddog-gov.com/api/v2/llm-obs/v1/projects/{project_id}https://api.datadoghq.com/api/v2/llm-obs/v1/projects/{project_id}https://api.us3.datadoghq.com/api/v2/llm-obs/v1/projects/{project_id}https://api.us5.datadoghq.com/api/v2/llm-obs/v1/projects/{project_id}

Overview

Partially update an existing LLM Observability project.

Arguments

Path Parameters

Name

Type

Description

project_id [required]

string

The ID of the LLM Observability project.

Request

Body Data (required)

Update project payload.

Expand All

Field

Type

Description

data [required]

object

Data object for updating an LLM Observability project.

attributes [required]

object

Attributes for updating an LLM Observability project.

description

string

Updated description of the project.

name

string

Updated name of the project.

type [required]

enum

Resource type of an LLM Observability project. Allowed enum values: projects

{
  "data": {
    "attributes": {
      "description": "string",
      "name": "string"
    },
    "type": "projects"
  }
}

Response

OK

Response containing a single LLM Observability project.

Expand All

Field

Type

Description

data [required]

object

Data object for an LLM Observability project.

attributes [required]

object

Attributes of an LLM Observability project.

created_at [required]

date-time

Timestamp when the project was created.

description [required]

string

Description of the project.

name [required]

string

Name of the project.

updated_at [required]

date-time

Timestamp when the project was last updated.

id [required]

string

Unique identifier of the project.

type [required]

enum

Resource type of an LLM Observability project. Allowed enum values: projects

{
  "data": {
    "attributes": {
      "created_at": "2024-01-15T10:30:00Z",
      "description": "",
      "name": "My LLM Project",
      "updated_at": "2024-01-15T10:30:00Z"
    },
    "id": "a33671aa-24fd-4dcd-9b33-a8ec7dde7751",
    "type": "projects"
  }
}

Bad Request

API error response.

Expand All

Field

Type

Description

errors [required]

[object]

A list of errors.

detail

string

A human-readable explanation specific to this occurrence of the error.

meta

object

Non-standard meta-information about the error

source

object

References to the source of the error.

header

string

A string indicating the name of a single request header which caused the error.

parameter

string

A string indicating which URI query parameter caused the error.

pointer

string

A JSON pointer to the value in the request document that caused the error.

status

string

Status code of the response.

title

string

Short human-readable summary of the error.

{
  "errors": [
    {
      "detail": "Missing required attribute in body",
      "meta": {},
      "source": {
        "header": "Authorization",
        "parameter": "limit",
        "pointer": "/data/attributes/title"
      },
      "status": "400",
      "title": "Bad Request"
    }
  ]
}

Unauthorized

API error response.

Expand All

Field

Type

Description

errors [required]

[object]

A list of errors.

detail

string

A human-readable explanation specific to this occurrence of the error.

meta

object

Non-standard meta-information about the error

source

object

References to the source of the error.

header

string

A string indicating the name of a single request header which caused the error.

parameter

string

A string indicating which URI query parameter caused the error.

pointer

string

A JSON pointer to the value in the request document that caused the error.

status

string

Status code of the response.

title

string

Short human-readable summary of the error.

{
  "errors": [
    {
      "detail": "Missing required attribute in body",
      "meta": {},
      "source": {
        "header": "Authorization",
        "parameter": "limit",
        "pointer": "/data/attributes/title"
      },
      "status": "400",
      "title": "Bad Request"
    }
  ]
}

Forbidden

API error response.

Expand All

Field

Type

Description

errors [required]

[object]

A list of errors.

detail

string

A human-readable explanation specific to this occurrence of the error.

meta

object

Non-standard meta-information about the error

source

object

References to the source of the error.

header

string

A string indicating the name of a single request header which caused the error.

parameter

string

A string indicating which URI query parameter caused the error.

pointer

string

A JSON pointer to the value in the request document that caused the error.

status

string

Status code of the response.

title

string

Short human-readable summary of the error.

{
  "errors": [
    {
      "detail": "Missing required attribute in body",
      "meta": {},
      "source": {
        "header": "Authorization",
        "parameter": "limit",
        "pointer": "/data/attributes/title"
      },
      "status": "400",
      "title": "Bad Request"
    }
  ]
}

Not Found

API error response.

Expand All

Field

Type

Description

errors [required]

[object]

A list of errors.

detail

string

A human-readable explanation specific to this occurrence of the error.

meta

object

Non-standard meta-information about the error

source

object

References to the source of the error.

header

string

A string indicating the name of a single request header which caused the error.

parameter

string

A string indicating which URI query parameter caused the error.

pointer

string

A JSON pointer to the value in the request document that caused the error.

status

string

Status code of the response.

title

string

Short human-readable summary of the error.

{
  "errors": [
    {
      "detail": "Missing required attribute in body",
      "meta": {},
      "source": {
        "header": "Authorization",
        "parameter": "limit",
        "pointer": "/data/attributes/title"
      },
      "status": "400",
      "title": "Bad Request"
    }
  ]
}

Too many requests

API error response.

Expand All

Field

Type

Description

errors [required]

[string]

A list of errors.

{
  "errors": [
    "Bad Request"
  ]
}

Code Example

                  # Path parameters
export project_id="a33671aa-24fd-4dcd-9b33-a8ec7dde7751"
# Curl command
curl -X PATCH "https://api.ap1.datadoghq.com"https://api.ap2.datadoghq.com"https://api.datadoghq.eu"https://api.ddog-gov.com"https://api.datadoghq.com"https://api.us3.datadoghq.com"https://api.us5.datadoghq.com/api/v2/llm-obs/v1/projects/${project_id}" \
-H "Accept: application/json" \
-H "Content-Type: application/json" \
-H "DD-API-KEY: ${DD_API_KEY}" \
-H "DD-APPLICATION-KEY: ${DD_APP_KEY}" \
-d @- << EOF
{
  "data": {
    "attributes": {},
    "type": "projects"
  }
}
EOF

Delete LLM Observability projects

Note: This endpoint is in preview and is subject to change. If you have any feedback, contact Datadog support.

POST https://api.ap1.datadoghq.com/api/v2/llm-obs/v1/projects/deletehttps://api.ap2.datadoghq.com/api/v2/llm-obs/v1/projects/deletehttps://api.datadoghq.eu/api/v2/llm-obs/v1/projects/deletehttps://api.ddog-gov.com/api/v2/llm-obs/v1/projects/deletehttps://api.datadoghq.com/api/v2/llm-obs/v1/projects/deletehttps://api.us3.datadoghq.com/api/v2/llm-obs/v1/projects/deletehttps://api.us5.datadoghq.com/api/v2/llm-obs/v1/projects/delete

Overview

Delete one or more LLM Observability projects.

Request

Body Data (required)

Delete projects payload.

Expand All

Field

Type

Description

data [required]

object

Data object for deleting LLM Observability projects.

attributes [required]

object

Attributes for deleting LLM Observability projects.

project_ids [required]

[string]

List of project IDs to delete.

type [required]

enum

Resource type of an LLM Observability project. Allowed enum values: projects

{
  "data": {
    "attributes": {
      "project_ids": [
        "a33671aa-24fd-4dcd-9b33-a8ec7dde7751"
      ]
    },
    "type": "projects"
  }
}

Response

No Content

Bad Request

API error response.

Expand All

Field

Type

Description

errors [required]

[object]

A list of errors.

detail

string

A human-readable explanation specific to this occurrence of the error.

meta

object

Non-standard meta-information about the error

source

object

References to the source of the error.

header

string

A string indicating the name of a single request header which caused the error.

parameter

string

A string indicating which URI query parameter caused the error.

pointer

string

A JSON pointer to the value in the request document that caused the error.

status

string

Status code of the response.

title

string

Short human-readable summary of the error.

{
  "errors": [
    {
      "detail": "Missing required attribute in body",
      "meta": {},
      "source": {
        "header": "Authorization",
        "parameter": "limit",
        "pointer": "/data/attributes/title"
      },
      "status": "400",
      "title": "Bad Request"
    }
  ]
}

Unauthorized

API error response.

Expand All

Field

Type

Description

errors [required]

[object]

A list of errors.

detail

string

A human-readable explanation specific to this occurrence of the error.

meta

object

Non-standard meta-information about the error

source

object

References to the source of the error.

header

string

A string indicating the name of a single request header which caused the error.

parameter

string

A string indicating which URI query parameter caused the error.

pointer

string

A JSON pointer to the value in the request document that caused the error.

status

string

Status code of the response.

title

string

Short human-readable summary of the error.

{
  "errors": [
    {
      "detail": "Missing required attribute in body",
      "meta": {},
      "source": {
        "header": "Authorization",
        "parameter": "limit",
        "pointer": "/data/attributes/title"
      },
      "status": "400",
      "title": "Bad Request"
    }
  ]
}

Forbidden

API error response.

Expand All

Field

Type

Description

errors [required]

[object]

A list of errors.

detail

string

A human-readable explanation specific to this occurrence of the error.

meta

object

Non-standard meta-information about the error

source

object

References to the source of the error.

header

string

A string indicating the name of a single request header which caused the error.

parameter

string

A string indicating which URI query parameter caused the error.

pointer

string

A JSON pointer to the value in the request document that caused the error.

status

string

Status code of the response.

title

string

Short human-readable summary of the error.

{
  "errors": [
    {
      "detail": "Missing required attribute in body",
      "meta": {},
      "source": {
        "header": "Authorization",
        "parameter": "limit",
        "pointer": "/data/attributes/title"
      },
      "status": "400",
      "title": "Bad Request"
    }
  ]
}

Too many requests

API error response.

Expand All

Field

Type

Description

errors [required]

[string]

A list of errors.

{
  "errors": [
    "Bad Request"
  ]
}

Code Example

                  # Curl command
curl -X POST "https://api.ap1.datadoghq.com"https://api.ap2.datadoghq.com"https://api.datadoghq.eu"https://api.ddog-gov.com"https://api.datadoghq.com"https://api.us3.datadoghq.com"https://api.us5.datadoghq.com/api/v2/llm-obs/v1/projects/delete" \
-H "Content-Type: application/json" \
-H "DD-API-KEY: ${DD_API_KEY}" \
-H "DD-APPLICATION-KEY: ${DD_APP_KEY}" \
-d @- << EOF
{
  "data": {
    "attributes": {
      "project_ids": [
        "a33671aa-24fd-4dcd-9b33-a8ec7dde7751"
      ]
    },
    "type": "projects"
  }
}
EOF

List LLM Observability datasets

Note: This endpoint is in preview and is subject to change. If you have any feedback, contact Datadog support.

GET https://api.ap1.datadoghq.com/api/v2/llm-obs/v1/{project_id}/datasetshttps://api.ap2.datadoghq.com/api/v2/llm-obs/v1/{project_id}/datasetshttps://api.datadoghq.eu/api/v2/llm-obs/v1/{project_id}/datasetshttps://api.ddog-gov.com/api/v2/llm-obs/v1/{project_id}/datasetshttps://api.datadoghq.com/api/v2/llm-obs/v1/{project_id}/datasetshttps://api.us3.datadoghq.com/api/v2/llm-obs/v1/{project_id}/datasetshttps://api.us5.datadoghq.com/api/v2/llm-obs/v1/{project_id}/datasets

Overview

List all LLM Observability datasets for a project, sorted by creation date, newest first.

Arguments

Path Parameters

Name

Type

Description

project_id [required]

string

The ID of the LLM Observability project.

Query Strings

Name

Type

Description

filter[name]

string

Filter datasets by name.

filter[id]

string

Filter datasets by dataset ID.

page[cursor]

string

Use the Pagination cursor to retrieve the next page of results.

page[limit]

integer

Maximum number of results to return per page.

Response

OK

Response containing a list of LLM Observability datasets.

Expand All

Field

Type

Description

data [required]

[object]

List of datasets.

attributes [required]

object

Attributes of an LLM Observability dataset.

created_at [required]

date-time

Timestamp when the dataset was created.

current_version [required]

int64

Current version number of the dataset.

description [required]

string

Description of the dataset.

metadata [required]

object

Arbitrary metadata associated with the dataset.

name [required]

string

Name of the dataset.

updated_at [required]

date-time

Timestamp when the dataset was last updated.

id [required]

string

Unique identifier of the dataset.

type [required]

enum

Resource type of an LLM Observability dataset. Allowed enum values: datasets

meta

object

Pagination cursor metadata.

after

string

Cursor for the next page of results.

{
  "data": [
    {
      "attributes": {
        "created_at": "2024-01-15T10:30:00Z",
        "current_version": 1,
        "description": "",
        "metadata": {},
        "name": "My LLM Dataset",
        "updated_at": "2024-01-15T10:30:00Z"
      },
      "id": "9f64e5c7-dc5a-45c8-a17c-1b85f0bec97d",
      "type": "datasets"
    }
  ],
  "meta": {
    "after": "string"
  }
}

Bad Request

API error response.

Expand All

Field

Type

Description

errors [required]

[object]

A list of errors.

detail

string

A human-readable explanation specific to this occurrence of the error.

meta

object

Non-standard meta-information about the error

source

object

References to the source of the error.

header

string

A string indicating the name of a single request header which caused the error.

parameter

string

A string indicating which URI query parameter caused the error.

pointer

string

A JSON pointer to the value in the request document that caused the error.

status

string

Status code of the response.

title

string

Short human-readable summary of the error.

{
  "errors": [
    {
      "detail": "Missing required attribute in body",
      "meta": {},
      "source": {
        "header": "Authorization",
        "parameter": "limit",
        "pointer": "/data/attributes/title"
      },
      "status": "400",
      "title": "Bad Request"
    }
  ]
}

Unauthorized

API error response.

Expand All

Field

Type

Description

errors [required]

[object]

A list of errors.

detail

string

A human-readable explanation specific to this occurrence of the error.

meta

object

Non-standard meta-information about the error

source

object

References to the source of the error.

header

string

A string indicating the name of a single request header which caused the error.

parameter

string

A string indicating which URI query parameter caused the error.

pointer

string

A JSON pointer to the value in the request document that caused the error.

status

string

Status code of the response.

title

string

Short human-readable summary of the error.

{
  "errors": [
    {
      "detail": "Missing required attribute in body",
      "meta": {},
      "source": {
        "header": "Authorization",
        "parameter": "limit",
        "pointer": "/data/attributes/title"
      },
      "status": "400",
      "title": "Bad Request"
    }
  ]
}

Forbidden

API error response.

Expand All

Field

Type

Description

errors [required]

[object]

A list of errors.

detail

string

A human-readable explanation specific to this occurrence of the error.

meta

object

Non-standard meta-information about the error

source

object

References to the source of the error.

header

string

A string indicating the name of a single request header which caused the error.

parameter

string

A string indicating which URI query parameter caused the error.

pointer

string

A JSON pointer to the value in the request document that caused the error.

status

string

Status code of the response.

title

string

Short human-readable summary of the error.

{
  "errors": [
    {
      "detail": "Missing required attribute in body",
      "meta": {},
      "source": {
        "header": "Authorization",
        "parameter": "limit",
        "pointer": "/data/attributes/title"
      },
      "status": "400",
      "title": "Bad Request"
    }
  ]
}

Not Found

API error response.

Expand All

Field

Type

Description

errors [required]

[object]

A list of errors.

detail

string

A human-readable explanation specific to this occurrence of the error.

meta

object

Non-standard meta-information about the error

source

object

References to the source of the error.

header

string

A string indicating the name of a single request header which caused the error.

parameter

string

A string indicating which URI query parameter caused the error.

pointer

string

A JSON pointer to the value in the request document that caused the error.

status

string

Status code of the response.

title

string

Short human-readable summary of the error.

{
  "errors": [
    {
      "detail": "Missing required attribute in body",
      "meta": {},
      "source": {
        "header": "Authorization",
        "parameter": "limit",
        "pointer": "/data/attributes/title"
      },
      "status": "400",
      "title": "Bad Request"
    }
  ]
}

Too many requests

API error response.

Expand All

Field

Type

Description

errors [required]

[string]

A list of errors.

{
  "errors": [
    "Bad Request"
  ]
}

Code Example

                  # Path parameters
export project_id="a33671aa-24fd-4dcd-9b33-a8ec7dde7751"
# Curl command
curl -X GET "https://api.ap1.datadoghq.com"https://api.ap2.datadoghq.com"https://api.datadoghq.eu"https://api.ddog-gov.com"https://api.datadoghq.com"https://api.us3.datadoghq.com"https://api.us5.datadoghq.com/api/v2/llm-obs/v1/${project_id}/datasets" \
-H "Accept: application/json" \
-H "DD-API-KEY: ${DD_API_KEY}" \
-H "DD-APPLICATION-KEY: ${DD_APP_KEY}"

Create an LLM Observability dataset

Note: This endpoint is in preview and is subject to change. If you have any feedback, contact Datadog support.

POST https://api.ap1.datadoghq.com/api/v2/llm-obs/v1/{project_id}/datasetshttps://api.ap2.datadoghq.com/api/v2/llm-obs/v1/{project_id}/datasetshttps://api.datadoghq.eu/api/v2/llm-obs/v1/{project_id}/datasetshttps://api.ddog-gov.com/api/v2/llm-obs/v1/{project_id}/datasetshttps://api.datadoghq.com/api/v2/llm-obs/v1/{project_id}/datasetshttps://api.us3.datadoghq.com/api/v2/llm-obs/v1/{project_id}/datasetshttps://api.us5.datadoghq.com/api/v2/llm-obs/v1/{project_id}/datasets

Overview

Create a new LLM Observability dataset within the specified project.

Arguments

Path Parameters

Name

Type

Description

project_id [required]

string

The ID of the LLM Observability project.

Request

Body Data (required)

Create dataset payload.

Expand All

Field

Type

Description

data [required]

object

Data object for creating an LLM Observability dataset.

attributes [required]

object

Attributes for creating an LLM Observability dataset.

description

string

Description of the dataset.

metadata

object

Arbitrary metadata associated with the dataset.

name [required]

string

Name of the dataset.

type [required]

enum

Resource type of an LLM Observability dataset. Allowed enum values: datasets

{
  "data": {
    "attributes": {
      "description": "string",
      "metadata": {},
      "name": "My LLM Dataset"
    },
    "type": "datasets"
  }
}

Response

OK

Response containing a single LLM Observability dataset.

Expand All

Field

Type

Description

data [required]

object

Data object for an LLM Observability dataset.

attributes [required]

object

Attributes of an LLM Observability dataset.

created_at [required]

date-time

Timestamp when the dataset was created.

current_version [required]

int64

Current version number of the dataset.

description [required]

string

Description of the dataset.

metadata [required]

object

Arbitrary metadata associated with the dataset.

name [required]

string

Name of the dataset.

updated_at [required]

date-time

Timestamp when the dataset was last updated.

id [required]

string

Unique identifier of the dataset.

type [required]

enum

Resource type of an LLM Observability dataset. Allowed enum values: datasets

{
  "data": {
    "attributes": {
      "created_at": "2024-01-15T10:30:00Z",
      "current_version": 1,
      "description": "",
      "metadata": {},
      "name": "My LLM Dataset",
      "updated_at": "2024-01-15T10:30:00Z"
    },
    "id": "9f64e5c7-dc5a-45c8-a17c-1b85f0bec97d",
    "type": "datasets"
  }
}

Created

Response containing a single LLM Observability dataset.

Expand All

Field

Type

Description

data [required]

object

Data object for an LLM Observability dataset.

attributes [required]

object

Attributes of an LLM Observability dataset.

created_at [required]

date-time

Timestamp when the dataset was created.

current_version [required]

int64

Current version number of the dataset.

description [required]

string

Description of the dataset.

metadata [required]

object

Arbitrary metadata associated with the dataset.

name [required]

string

Name of the dataset.

updated_at [required]

date-time

Timestamp when the dataset was last updated.

id [required]

string

Unique identifier of the dataset.

type [required]

enum

Resource type of an LLM Observability dataset. Allowed enum values: datasets

{
  "data": {
    "attributes": {
      "created_at": "2024-01-15T10:30:00Z",
      "current_version": 1,
      "description": "",
      "metadata": {},
      "name": "My LLM Dataset",
      "updated_at": "2024-01-15T10:30:00Z"
    },
    "id": "9f64e5c7-dc5a-45c8-a17c-1b85f0bec97d",
    "type": "datasets"
  }
}

Bad Request

API error response.

Expand All

Field

Type

Description

errors [required]

[object]

A list of errors.

detail

string

A human-readable explanation specific to this occurrence of the error.

meta

object

Non-standard meta-information about the error

source

object

References to the source of the error.

header

string

A string indicating the name of a single request header which caused the error.

parameter

string

A string indicating which URI query parameter caused the error.

pointer

string

A JSON pointer to the value in the request document that caused the error.

status

string

Status code of the response.

title

string

Short human-readable summary of the error.

{
  "errors": [
    {
      "detail": "Missing required attribute in body",
      "meta": {},
      "source": {
        "header": "Authorization",
        "parameter": "limit",
        "pointer": "/data/attributes/title"
      },
      "status": "400",
      "title": "Bad Request"
    }
  ]
}

Unauthorized

API error response.

Expand All

Field

Type

Description

errors [required]

[object]

A list of errors.

detail

string

A human-readable explanation specific to this occurrence of the error.

meta

object

Non-standard meta-information about the error

source

object

References to the source of the error.

header

string

A string indicating the name of a single request header which caused the error.

parameter

string

A string indicating which URI query parameter caused the error.

pointer

string

A JSON pointer to the value in the request document that caused the error.

status

string

Status code of the response.

title

string

Short human-readable summary of the error.

{
  "errors": [
    {
      "detail": "Missing required attribute in body",
      "meta": {},
      "source": {
        "header": "Authorization",
        "parameter": "limit",
        "pointer": "/data/attributes/title"
      },
      "status": "400",
      "title": "Bad Request"
    }
  ]
}

Forbidden

API error response.

Expand All

Field

Type

Description

errors [required]

[object]

A list of errors.

detail

string

A human-readable explanation specific to this occurrence of the error.

meta

object

Non-standard meta-information about the error

source

object

References to the source of the error.

header

string

A string indicating the name of a single request header which caused the error.

parameter

string

A string indicating which URI query parameter caused the error.

pointer

string

A JSON pointer to the value in the request document that caused the error.

status

string

Status code of the response.

title

string

Short human-readable summary of the error.

{
  "errors": [
    {
      "detail": "Missing required attribute in body",
      "meta": {},
      "source": {
        "header": "Authorization",
        "parameter": "limit",
        "pointer": "/data/attributes/title"
      },
      "status": "400",
      "title": "Bad Request"
    }
  ]
}

Not Found

API error response.

Expand All

Field

Type

Description

errors [required]

[object]

A list of errors.

detail

string

A human-readable explanation specific to this occurrence of the error.

meta

object

Non-standard meta-information about the error

source

object

References to the source of the error.

header

string

A string indicating the name of a single request header which caused the error.

parameter

string

A string indicating which URI query parameter caused the error.

pointer

string

A JSON pointer to the value in the request document that caused the error.

status

string

Status code of the response.

title

string

Short human-readable summary of the error.

{
  "errors": [
    {
      "detail": "Missing required attribute in body",
      "meta": {},
      "source": {
        "header": "Authorization",
        "parameter": "limit",
        "pointer": "/data/attributes/title"
      },
      "status": "400",
      "title": "Bad Request"
    }
  ]
}

Too many requests

API error response.

Expand All

Field

Type

Description

errors [required]

[string]

A list of errors.

{
  "errors": [
    "Bad Request"
  ]
}

Code Example

                  # Path parameters
export project_id="a33671aa-24fd-4dcd-9b33-a8ec7dde7751"
# Curl command
curl -X POST "https://api.ap1.datadoghq.com"https://api.ap2.datadoghq.com"https://api.datadoghq.eu"https://api.ddog-gov.com"https://api.datadoghq.com"https://api.us3.datadoghq.com"https://api.us5.datadoghq.com/api/v2/llm-obs/v1/${project_id}/datasets" \
-H "Accept: application/json" \
-H "Content-Type: application/json" \
-H "DD-API-KEY: ${DD_API_KEY}" \
-H "DD-APPLICATION-KEY: ${DD_APP_KEY}" \
-d @- << EOF
{
  "data": {
    "attributes": {
      "name": "My LLM Dataset"
    },
    "type": "datasets"
  }
}
EOF

List LLM Observability dataset records

Note: This endpoint is in preview and is subject to change. If you have any feedback, contact Datadog support.

GET https://api.ap1.datadoghq.com/api/v2/llm-obs/v1/{project_id}/datasets/{dataset_id}/recordshttps://api.ap2.datadoghq.com/api/v2/llm-obs/v1/{project_id}/datasets/{dataset_id}/recordshttps://api.datadoghq.eu/api/v2/llm-obs/v1/{project_id}/datasets/{dataset_id}/recordshttps://api.ddog-gov.com/api/v2/llm-obs/v1/{project_id}/datasets/{dataset_id}/recordshttps://api.datadoghq.com/api/v2/llm-obs/v1/{project_id}/datasets/{dataset_id}/recordshttps://api.us3.datadoghq.com/api/v2/llm-obs/v1/{project_id}/datasets/{dataset_id}/recordshttps://api.us5.datadoghq.com/api/v2/llm-obs/v1/{project_id}/datasets/{dataset_id}/records

Overview

List all records in an LLM Observability dataset, sorted by creation date, newest first.

Arguments

Path Parameters

Name

Type

Description

project_id [required]

string

The ID of the LLM Observability project.

dataset_id [required]

string

The ID of the LLM Observability dataset.

Query Strings

Name

Type

Description

filter[version]

integer

Retrieve records from a specific dataset version. Defaults to the current version.

page[cursor]

string

Use the Pagination cursor to retrieve the next page of results.

page[limit]

integer

Maximum number of results to return per page.

Response

OK

Response containing a paginated list of LLM Observability dataset records.

Expand All

Field

Type

Description

data [required]

[object]

List of dataset records.

created_at [required]

date-time

Timestamp when the record was created.

dataset_id [required]

string

Identifier of the dataset this record belongs to.

expected_output [required]

object <oneOf>

Represents any valid JSON value.

Option 1

string

Option 2

double

Option 3

object

Option 4

[ <oneOf>]

Option 1

string

Option 2

double

Option 3

object

Option 4

boolean

Option 5

boolean

id [required]

string

Unique identifier of the record.

input [required]

object <oneOf>

Represents any valid JSON value.

Option 1

string

Option 2

double

Option 3

object

Option 4

[ <oneOf>]

Option 1

string

Option 2

double

Option 3

object

Option 4

boolean

Option 5

boolean

metadata [required]

object

Arbitrary metadata associated with the record.

updated_at [required]

date-time

Timestamp when the record was last updated.

meta

object

Pagination cursor metadata.

after

string

Cursor for the next page of results.

{
  "data": [
    {
      "created_at": "2024-01-15T10:30:00Z",
      "dataset_id": "9f64e5c7-dc5a-45c8-a17c-1b85f0bec97d",
      "expected_output": {
        "type": "undefined"
      },
      "id": "rec-7c3f5a1b-9e2d-4f8a-b1c6-3d7e9f0a2b4c",
      "input": {
        "type": "undefined"
      },
      "metadata": {},
      "updated_at": "2024-01-15T10:30:00Z"
    }
  ],
  "meta": {
    "after": "string"
  }
}

Bad Request

API error response.

Expand All

Field

Type

Description

errors [required]

[object]

A list of errors.

detail

string

A human-readable explanation specific to this occurrence of the error.

meta

object

Non-standard meta-information about the error

source

object

References to the source of the error.

header

string

A string indicating the name of a single request header which caused the error.

parameter

string

A string indicating which URI query parameter caused the error.

pointer

string

A JSON pointer to the value in the request document that caused the error.

status

string

Status code of the response.

title

string

Short human-readable summary of the error.

{
  "errors": [
    {
      "detail": "Missing required attribute in body",
      "meta": {},
      "source": {
        "header": "Authorization",
        "parameter": "limit",
        "pointer": "/data/attributes/title"
      },
      "status": "400",
      "title": "Bad Request"
    }
  ]
}

Unauthorized

API error response.

Expand All

Field

Type

Description

errors [required]

[object]

A list of errors.

detail

string

A human-readable explanation specific to this occurrence of the error.

meta

object

Non-standard meta-information about the error

source

object

References to the source of the error.

header

string

A string indicating the name of a single request header which caused the error.

parameter

string

A string indicating which URI query parameter caused the error.

pointer

string

A JSON pointer to the value in the request document that caused the error.

status

string

Status code of the response.

title

string

Short human-readable summary of the error.

{
  "errors": [
    {
      "detail": "Missing required attribute in body",
      "meta": {},
      "source": {
        "header": "Authorization",
        "parameter": "limit",
        "pointer": "/data/attributes/title"
      },
      "status": "400",
      "title": "Bad Request"
    }
  ]
}

Forbidden

API error response.

Expand All

Field

Type

Description

errors [required]

[object]

A list of errors.

detail

string

A human-readable explanation specific to this occurrence of the error.

meta

object

Non-standard meta-information about the error

source

object

References to the source of the error.

header

string

A string indicating the name of a single request header which caused the error.

parameter

string

A string indicating which URI query parameter caused the error.

pointer

string

A JSON pointer to the value in the request document that caused the error.

status

string

Status code of the response.

title

string

Short human-readable summary of the error.

{
  "errors": [
    {
      "detail": "Missing required attribute in body",
      "meta": {},
      "source": {
        "header": "Authorization",
        "parameter": "limit",
        "pointer": "/data/attributes/title"
      },
      "status": "400",
      "title": "Bad Request"
    }
  ]
}

Not Found

API error response.

Expand All

Field

Type

Description

errors [required]

[object]

A list of errors.

detail

string

A human-readable explanation specific to this occurrence of the error.

meta

object

Non-standard meta-information about the error

source

object

References to the source of the error.

header

string

A string indicating the name of a single request header which caused the error.

parameter

string

A string indicating which URI query parameter caused the error.

pointer

string

A JSON pointer to the value in the request document that caused the error.

status

string

Status code of the response.

title

string

Short human-readable summary of the error.

{
  "errors": [
    {
      "detail": "Missing required attribute in body",
      "meta": {},
      "source": {
        "header": "Authorization",
        "parameter": "limit",
        "pointer": "/data/attributes/title"
      },
      "status": "400",
      "title": "Bad Request"
    }
  ]
}

Too many requests

API error response.

Expand All

Field

Type

Description

errors [required]

[string]

A list of errors.

{
  "errors": [
    "Bad Request"
  ]
}

Code Example

                  # Path parameters
export project_id="a33671aa-24fd-4dcd-9b33-a8ec7dde7751"
export dataset_id="9f64e5c7-dc5a-45c8-a17c-1b85f0bec97d"
# Curl command
curl -X GET "https://api.ap1.datadoghq.com"https://api.ap2.datadoghq.com"https://api.datadoghq.eu"https://api.ddog-gov.com"https://api.datadoghq.com"https://api.us3.datadoghq.com"https://api.us5.datadoghq.com/api/v2/llm-obs/v1/${project_id}/datasets/${dataset_id}/records" \
-H "Accept: application/json" \
-H "DD-API-KEY: ${DD_API_KEY}" \
-H "DD-APPLICATION-KEY: ${DD_APP_KEY}"

Append records to an LLM Observability dataset

Note: This endpoint is in preview and is subject to change. If you have any feedback, contact Datadog support.

POST https://api.ap1.datadoghq.com/api/v2/llm-obs/v1/{project_id}/datasets/{dataset_id}/recordshttps://api.ap2.datadoghq.com/api/v2/llm-obs/v1/{project_id}/datasets/{dataset_id}/recordshttps://api.datadoghq.eu/api/v2/llm-obs/v1/{project_id}/datasets/{dataset_id}/recordshttps://api.ddog-gov.com/api/v2/llm-obs/v1/{project_id}/datasets/{dataset_id}/recordshttps://api.datadoghq.com/api/v2/llm-obs/v1/{project_id}/datasets/{dataset_id}/recordshttps://api.us3.datadoghq.com/api/v2/llm-obs/v1/{project_id}/datasets/{dataset_id}/recordshttps://api.us5.datadoghq.com/api/v2/llm-obs/v1/{project_id}/datasets/{dataset_id}/records

Overview

Append one or more records to an LLM Observability dataset.

Arguments

Path Parameters

Name

Type

Description

project_id [required]

string

The ID of the LLM Observability project.

dataset_id [required]

string

The ID of the LLM Observability dataset.

Request

Body Data (required)

Append records payload.

Expand All

Field

Type

Description

data [required]

object

Data object for appending records to an LLM Observability dataset.

attributes [required]

object

Attributes for appending records to an LLM Observability dataset.

deduplicate

boolean

Whether to deduplicate records before appending. Defaults to true.

records [required]

[object]

List of records to append to the dataset.

expected_output

object <oneOf>

Represents any valid JSON value.

Option 1

string

Option 2

double

Option 3

object

Option 4

[ <oneOf>]

Option 1

string

Option 2

double

Option 3

object

Option 4

boolean

Option 5

boolean

input [required]

object <oneOf>

Represents any valid JSON value.

Option 1

string

Option 2

double

Option 3

object

Option 4

[ <oneOf>]

Option 1

string

Option 2

double

Option 3

object

Option 4

boolean

Option 5

boolean

metadata

object

Arbitrary metadata associated with the record.

type [required]

enum

Resource type of LLM Observability dataset records. Allowed enum values: records

{
  "data": {
    "attributes": {
      "deduplicate": false,
      "records": [
        {
          "expected_output": {
            "type": "undefined"
          },
          "input": {
            "type": "undefined"
          },
          "metadata": {}
        }
      ]
    },
    "type": "records"
  }
}

Response

OK

Response containing records after a create or update operation.

Expand All

Field

Type

Description

data [required]

[object]

List of affected dataset records.

records [required]

[object]

List of affected dataset records.

created_at [required]

date-time

Timestamp when the record was created.

dataset_id [required]

string

Identifier of the dataset this record belongs to.

expected_output [required]

object <oneOf>

Represents any valid JSON value.

Option 1

string

Option 2

double

Option 3

object

Option 4

[ <oneOf>]

Option 1

string

Option 2

double

Option 3

object

Option 4

boolean

Option 5

boolean

id [required]

string

Unique identifier of the record.

input [required]

object <oneOf>

Represents any valid JSON value.

Option 1

string

Option 2

double

Option 3

object

Option 4

[ <oneOf>]

Option 1

string

Option 2

double

Option 3

object

Option 4

boolean

Option 5

boolean

metadata [required]

object

Arbitrary metadata associated with the record.

updated_at [required]

date-time

Timestamp when the record was last updated.

{
  "data": [
    {
      "records": [
        {
          "created_at": "2024-01-15T10:30:00Z",
          "dataset_id": "9f64e5c7-dc5a-45c8-a17c-1b85f0bec97d",
          "expected_output": {
            "type": "undefined"
          },
          "id": "rec-7c3f5a1b-9e2d-4f8a-b1c6-3d7e9f0a2b4c",
          "input": {
            "type": "undefined"
          },
          "metadata": {},
          "updated_at": "2024-01-15T10:30:00Z"
        }
      ]
    }
  ]
}

Created

Response containing records after a create or update operation.

Expand All

Field

Type

Description

data [required]

[object]

List of affected dataset records.

records [required]

[object]

List of affected dataset records.

created_at [required]

date-time

Timestamp when the record was created.

dataset_id [required]

string

Identifier of the dataset this record belongs to.

expected_output [required]

object <oneOf>

Represents any valid JSON value.

Option 1

string

Option 2

double

Option 3

object

Option 4

[ <oneOf>]

Option 1

string

Option 2

double

Option 3

object

Option 4

boolean

Option 5

boolean

id [required]

string

Unique identifier of the record.

input [required]

object <oneOf>

Represents any valid JSON value.

Option 1

string

Option 2

double

Option 3

object

Option 4

[ <oneOf>]

Option 1

string

Option 2

double

Option 3

object

Option 4

boolean

Option 5

boolean

metadata [required]

object

Arbitrary metadata associated with the record.

updated_at [required]

date-time

Timestamp when the record was last updated.

{
  "data": [
    {
      "records": [
        {
          "created_at": "2024-01-15T10:30:00Z",
          "dataset_id": "9f64e5c7-dc5a-45c8-a17c-1b85f0bec97d",
          "expected_output": {
            "type": "undefined"
          },
          "id": "rec-7c3f5a1b-9e2d-4f8a-b1c6-3d7e9f0a2b4c",
          "input": {
            "type": "undefined"
          },
          "metadata": {},
          "updated_at": "2024-01-15T10:30:00Z"
        }
      ]
    }
  ]
}

Bad Request

API error response.

Expand All

Field

Type

Description

errors [required]

[object]

A list of errors.

detail

string

A human-readable explanation specific to this occurrence of the error.

meta

object

Non-standard meta-information about the error

source

object

References to the source of the error.

header

string

A string indicating the name of a single request header which caused the error.

parameter

string

A string indicating which URI query parameter caused the error.

pointer

string

A JSON pointer to the value in the request document that caused the error.

status

string

Status code of the response.

title

string

Short human-readable summary of the error.

{
  "errors": [
    {
      "detail": "Missing required attribute in body",
      "meta": {},
      "source": {
        "header": "Authorization",
        "parameter": "limit",
        "pointer": "/data/attributes/title"
      },
      "status": "400",
      "title": "Bad Request"
    }
  ]
}

Unauthorized

API error response.

Expand All

Field

Type

Description

errors [required]

[object]

A list of errors.

detail

string

A human-readable explanation specific to this occurrence of the error.

meta

object

Non-standard meta-information about the error

source

object

References to the source of the error.

header

string

A string indicating the name of a single request header which caused the error.

parameter

string

A string indicating which URI query parameter caused the error.

pointer

string

A JSON pointer to the value in the request document that caused the error.

status

string

Status code of the response.

title

string

Short human-readable summary of the error.

{
  "errors": [
    {
      "detail": "Missing required attribute in body",
      "meta": {},
      "source": {
        "header": "Authorization",
        "parameter": "limit",
        "pointer": "/data/attributes/title"
      },
      "status": "400",
      "title": "Bad Request"
    }
  ]
}

Forbidden

API error response.

Expand All

Field

Type

Description

errors [required]

[object]

A list of errors.

detail

string

A human-readable explanation specific to this occurrence of the error.

meta

object

Non-standard meta-information about the error

source

object

References to the source of the error.

header

string

A string indicating the name of a single request header which caused the error.

parameter

string

A string indicating which URI query parameter caused the error.

pointer

string

A JSON pointer to the value in the request document that caused the error.

status

string

Status code of the response.

title

string

Short human-readable summary of the error.

{
  "errors": [
    {
      "detail": "Missing required attribute in body",
      "meta": {},
      "source": {
        "header": "Authorization",
        "parameter": "limit",
        "pointer": "/data/attributes/title"
      },
      "status": "400",
      "title": "Bad Request"
    }
  ]
}

Not Found

API error response.

Expand All

Field

Type

Description

errors [required]

[object]

A list of errors.

detail

string

A human-readable explanation specific to this occurrence of the error.

meta

object

Non-standard meta-information about the error

source

object

References to the source of the error.

header

string

A string indicating the name of a single request header which caused the error.

parameter

string

A string indicating which URI query parameter caused the error.

pointer

string

A JSON pointer to the value in the request document that caused the error.

status

string

Status code of the response.

title

string

Short human-readable summary of the error.

{
  "errors": [
    {
      "detail": "Missing required attribute in body",
      "meta": {},
      "source": {
        "header": "Authorization",
        "parameter": "limit",
        "pointer": "/data/attributes/title"
      },
      "status": "400",
      "title": "Bad Request"
    }
  ]
}

Too many requests

API error response.

Expand All

Field

Type

Description

errors [required]

[string]

A list of errors.

{
  "errors": [
    "Bad Request"
  ]
}

Code Example

                  # Path parameters
export project_id="a33671aa-24fd-4dcd-9b33-a8ec7dde7751"
export dataset_id="9f64e5c7-dc5a-45c8-a17c-1b85f0bec97d"
# Curl command
curl -X POST "https://api.ap1.datadoghq.com"https://api.ap2.datadoghq.com"https://api.datadoghq.eu"https://api.ddog-gov.com"https://api.datadoghq.com"https://api.us3.datadoghq.com"https://api.us5.datadoghq.com/api/v2/llm-obs/v1/${project_id}/datasets/${dataset_id}/records" \
-H "Accept: application/json" \
-H "Content-Type: application/json" \
-H "DD-API-KEY: ${DD_API_KEY}" \
-H "DD-APPLICATION-KEY: ${DD_APP_KEY}" \
-d @- << EOF
{
  "data": {
    "attributes": {
      "records": [
        {
          "input": {}
        }
      ]
    },
    "type": "records"
  }
}
EOF

Update an LLM Observability dataset

Note: This endpoint is in preview and is subject to change. If you have any feedback, contact Datadog support.

PATCH https://api.ap1.datadoghq.com/api/v2/llm-obs/v1/{project_id}/datasets/{dataset_id}https://api.ap2.datadoghq.com/api/v2/llm-obs/v1/{project_id}/datasets/{dataset_id}https://api.datadoghq.eu/api/v2/llm-obs/v1/{project_id}/datasets/{dataset_id}https://api.ddog-gov.com/api/v2/llm-obs/v1/{project_id}/datasets/{dataset_id}https://api.datadoghq.com/api/v2/llm-obs/v1/{project_id}/datasets/{dataset_id}https://api.us3.datadoghq.com/api/v2/llm-obs/v1/{project_id}/datasets/{dataset_id}https://api.us5.datadoghq.com/api/v2/llm-obs/v1/{project_id}/datasets/{dataset_id}

Overview

Partially update an existing LLM Observability dataset within the specified project.

Arguments

Path Parameters

Name

Type

Description

project_id [required]

string

The ID of the LLM Observability project.

dataset_id [required]

string

The ID of the LLM Observability dataset.

Request

Body Data (required)

Update dataset payload.

Expand All

Field

Type

Description

data [required]

object

Data object for updating an LLM Observability dataset.

attributes [required]

object

Attributes for updating an LLM Observability dataset.

description

string

Updated description of the dataset.

metadata

object

Updated metadata associated with the dataset.

name

string

Updated name of the dataset.

type [required]

enum

Resource type of an LLM Observability dataset. Allowed enum values: datasets

{
  "data": {
    "attributes": {
      "description": "string",
      "metadata": {},
      "name": "string"
    },
    "type": "datasets"
  }
}

Response

OK

Response containing a single LLM Observability dataset.

Expand All

Field

Type

Description

data [required]

object

Data object for an LLM Observability dataset.

attributes [required]

object

Attributes of an LLM Observability dataset.

created_at [required]

date-time

Timestamp when the dataset was created.

current_version [required]

int64

Current version number of the dataset.

description [required]

string

Description of the dataset.

metadata [required]

object

Arbitrary metadata associated with the dataset.

name [required]

string

Name of the dataset.

updated_at [required]

date-time

Timestamp when the dataset was last updated.

id [required]

string

Unique identifier of the dataset.

type [required]

enum

Resource type of an LLM Observability dataset. Allowed enum values: datasets

{
  "data": {
    "attributes": {
      "created_at": "2024-01-15T10:30:00Z",
      "current_version": 1,
      "description": "",
      "metadata": {},
      "name": "My LLM Dataset",
      "updated_at": "2024-01-15T10:30:00Z"
    },
    "id": "9f64e5c7-dc5a-45c8-a17c-1b85f0bec97d",
    "type": "datasets"
  }
}

Bad Request

API error response.

Expand All

Field

Type

Description

errors [required]

[object]

A list of errors.

detail

string

A human-readable explanation specific to this occurrence of the error.

meta

object

Non-standard meta-information about the error

source

object

References to the source of the error.

header

string

A string indicating the name of a single request header which caused the error.

parameter

string

A string indicating which URI query parameter caused the error.

pointer

string

A JSON pointer to the value in the request document that caused the error.

status

string

Status code of the response.

title

string

Short human-readable summary of the error.

{
  "errors": [
    {
      "detail": "Missing required attribute in body",
      "meta": {},
      "source": {
        "header": "Authorization",
        "parameter": "limit",
        "pointer": "/data/attributes/title"
      },
      "status": "400",
      "title": "Bad Request"
    }
  ]
}

Unauthorized

API error response.

Expand All

Field

Type

Description

errors [required]

[object]

A list of errors.

detail

string

A human-readable explanation specific to this occurrence of the error.

meta

object

Non-standard meta-information about the error

source

object

References to the source of the error.

header

string

A string indicating the name of a single request header which caused the error.

parameter

string

A string indicating which URI query parameter caused the error.

pointer

string

A JSON pointer to the value in the request document that caused the error.

status

string

Status code of the response.

title

string

Short human-readable summary of the error.

{
  "errors": [
    {
      "detail": "Missing required attribute in body",
      "meta": {},
      "source": {
        "header": "Authorization",
        "parameter": "limit",
        "pointer": "/data/attributes/title"
      },
      "status": "400",
      "title": "Bad Request"
    }
  ]
}

Forbidden

API error response.

Expand All

Field

Type

Description

errors [required]

[object]

A list of errors.

detail

string

A human-readable explanation specific to this occurrence of the error.

meta

object

Non-standard meta-information about the error

source

object

References to the source of the error.

header

string

A string indicating the name of a single request header which caused the error.

parameter

string

A string indicating which URI query parameter caused the error.

pointer

string

A JSON pointer to the value in the request document that caused the error.

status

string

Status code of the response.

title

string

Short human-readable summary of the error.

{
  "errors": [
    {
      "detail": "Missing required attribute in body",
      "meta": {},
      "source": {
        "header": "Authorization",
        "parameter": "limit",
        "pointer": "/data/attributes/title"
      },
      "status": "400",
      "title": "Bad Request"
    }
  ]
}

Not Found

API error response.

Expand All

Field

Type

Description

errors [required]

[object]

A list of errors.

detail

string

A human-readable explanation specific to this occurrence of the error.

meta

object

Non-standard meta-information about the error

source

object

References to the source of the error.

header

string

A string indicating the name of a single request header which caused the error.

parameter

string

A string indicating which URI query parameter caused the error.

pointer

string

A JSON pointer to the value in the request document that caused the error.

status

string

Status code of the response.

title

string

Short human-readable summary of the error.

{
  "errors": [
    {
      "detail": "Missing required attribute in body",
      "meta": {},
      "source": {
        "header": "Authorization",
        "parameter": "limit",
        "pointer": "/data/attributes/title"
      },
      "status": "400",
      "title": "Bad Request"
    }
  ]
}

Too many requests

API error response.

Expand All

Field

Type

Description

errors [required]

[string]

A list of errors.

{
  "errors": [
    "Bad Request"
  ]
}

Code Example

                  # Path parameters
export project_id="a33671aa-24fd-4dcd-9b33-a8ec7dde7751"
export dataset_id="9f64e5c7-dc5a-45c8-a17c-1b85f0bec97d"
# Curl command
curl -X PATCH "https://api.ap1.datadoghq.com"https://api.ap2.datadoghq.com"https://api.datadoghq.eu"https://api.ddog-gov.com"https://api.datadoghq.com"https://api.us3.datadoghq.com"https://api.us5.datadoghq.com/api/v2/llm-obs/v1/${project_id}/datasets/${dataset_id}" \
-H "Accept: application/json" \
-H "Content-Type: application/json" \
-H "DD-API-KEY: ${DD_API_KEY}" \
-H "DD-APPLICATION-KEY: ${DD_APP_KEY}" \
-d @- << EOF
{
  "data": {
    "attributes": {},
    "type": "datasets"
  }
}
EOF

Update LLM Observability dataset records

Note: This endpoint is in preview and is subject to change. If you have any feedback, contact Datadog support.

PATCH https://api.ap1.datadoghq.com/api/v2/llm-obs/v1/{project_id}/datasets/{dataset_id}/recordshttps://api.ap2.datadoghq.com/api/v2/llm-obs/v1/{project_id}/datasets/{dataset_id}/recordshttps://api.datadoghq.eu/api/v2/llm-obs/v1/{project_id}/datasets/{dataset_id}/recordshttps://api.ddog-gov.com/api/v2/llm-obs/v1/{project_id}/datasets/{dataset_id}/recordshttps://api.datadoghq.com/api/v2/llm-obs/v1/{project_id}/datasets/{dataset_id}/recordshttps://api.us3.datadoghq.com/api/v2/llm-obs/v1/{project_id}/datasets/{dataset_id}/recordshttps://api.us5.datadoghq.com/api/v2/llm-obs/v1/{project_id}/datasets/{dataset_id}/records

Overview

Update one or more existing records in an LLM Observability dataset.

Arguments

Path Parameters

Name

Type

Description

project_id [required]

string

The ID of the LLM Observability project.

dataset_id [required]

string

The ID of the LLM Observability dataset.

Request

Body Data (required)

Update records payload.

Expand All

Field

Type

Description

data [required]

object

Data object for updating records in an LLM Observability dataset.

attributes [required]

object

Attributes for updating records in an LLM Observability dataset.

records [required]

[object]

List of records to update.

expected_output

object <oneOf>

Represents any valid JSON value.

Option 1

string

Option 2

double

Option 3

object

Option 4

[ <oneOf>]

Option 1

string

Option 2

double

Option 3

object

Option 4

boolean

Option 5

boolean

id [required]

string

Unique identifier of the record to update.

input

object <oneOf>

Represents any valid JSON value.

Option 1

string

Option 2

double

Option 3

object

Option 4

[ <oneOf>]

Option 1

string

Option 2

double

Option 3

object

Option 4

boolean

Option 5

boolean

metadata

object

Updated metadata associated with the record.

type [required]

enum

Resource type of LLM Observability dataset records. Allowed enum values: records

{
  "data": {
    "attributes": {
      "records": [
        {
          "expected_output": {
            "type": "undefined"
          },
          "id": "rec-7c3f5a1b-9e2d-4f8a-b1c6-3d7e9f0a2b4c",
          "input": {
            "type": "undefined"
          },
          "metadata": {}
        }
      ]
    },
    "type": "records"
  }
}

Response

OK

Response containing records after a create or update operation.

Expand All

Field

Type

Description

data [required]

[object]

List of affected dataset records.

records [required]

[object]

List of affected dataset records.

created_at [required]

date-time

Timestamp when the record was created.

dataset_id [required]

string

Identifier of the dataset this record belongs to.

expected_output [required]

object <oneOf>

Represents any valid JSON value.

Option 1

string

Option 2

double

Option 3

object

Option 4

[ <oneOf>]

Option 1

string

Option 2

double

Option 3

object

Option 4

boolean

Option 5

boolean

id [required]

string

Unique identifier of the record.

input [required]

object <oneOf>

Represents any valid JSON value.

Option 1

string

Option 2

double

Option 3

object

Option 4

[ <oneOf>]

Option 1

string

Option 2

double

Option 3

object

Option 4

boolean

Option 5

boolean

metadata [required]

object

Arbitrary metadata associated with the record.

updated_at [required]

date-time

Timestamp when the record was last updated.

{
  "data": [
    {
      "records": [
        {
          "created_at": "2024-01-15T10:30:00Z",
          "dataset_id": "9f64e5c7-dc5a-45c8-a17c-1b85f0bec97d",
          "expected_output": {
            "type": "undefined"
          },
          "id": "rec-7c3f5a1b-9e2d-4f8a-b1c6-3d7e9f0a2b4c",
          "input": {
            "type": "undefined"
          },
          "metadata": {},
          "updated_at": "2024-01-15T10:30:00Z"
        }
      ]
    }
  ]
}

Bad Request

API error response.

Expand All

Field

Type

Description

errors [required]

[object]

A list of errors.

detail

string

A human-readable explanation specific to this occurrence of the error.

meta

object

Non-standard meta-information about the error

source

object

References to the source of the error.

header

string

A string indicating the name of a single request header which caused the error.

parameter

string

A string indicating which URI query parameter caused the error.

pointer

string

A JSON pointer to the value in the request document that caused the error.

status

string

Status code of the response.

title

string

Short human-readable summary of the error.

{
  "errors": [
    {
      "detail": "Missing required attribute in body",
      "meta": {},
      "source": {
        "header": "Authorization",
        "parameter": "limit",
        "pointer": "/data/attributes/title"
      },
      "status": "400",
      "title": "Bad Request"
    }
  ]
}

Unauthorized

API error response.

Expand All

Field

Type

Description

errors [required]

[object]

A list of errors.

detail

string

A human-readable explanation specific to this occurrence of the error.

meta

object

Non-standard meta-information about the error

source

object

References to the source of the error.

header

string

A string indicating the name of a single request header which caused the error.

parameter

string

A string indicating which URI query parameter caused the error.

pointer

string

A JSON pointer to the value in the request document that caused the error.

status

string

Status code of the response.

title

string

Short human-readable summary of the error.

{
  "errors": [
    {
      "detail": "Missing required attribute in body",
      "meta": {},
      "source": {
        "header": "Authorization",
        "parameter": "limit",
        "pointer": "/data/attributes/title"
      },
      "status": "400",
      "title": "Bad Request"
    }
  ]
}

Forbidden

API error response.

Expand All

Field

Type

Description

errors [required]

[object]

A list of errors.

detail

string

A human-readable explanation specific to this occurrence of the error.

meta

object

Non-standard meta-information about the error

source

object

References to the source of the error.

header

string

A string indicating the name of a single request header which caused the error.

parameter

string

A string indicating which URI query parameter caused the error.

pointer

string

A JSON pointer to the value in the request document that caused the error.

status

string

Status code of the response.

title

string

Short human-readable summary of the error.

{
  "errors": [
    {
      "detail": "Missing required attribute in body",
      "meta": {},
      "source": {
        "header": "Authorization",
        "parameter": "limit",
        "pointer": "/data/attributes/title"
      },
      "status": "400",
      "title": "Bad Request"
    }
  ]
}

Not Found

API error response.

Expand All

Field

Type

Description

errors [required]

[object]

A list of errors.

detail

string

A human-readable explanation specific to this occurrence of the error.

meta

object

Non-standard meta-information about the error

source

object

References to the source of the error.

header

string

A string indicating the name of a single request header which caused the error.

parameter

string

A string indicating which URI query parameter caused the error.

pointer

string

A JSON pointer to the value in the request document that caused the error.

status

string

Status code of the response.

title

string

Short human-readable summary of the error.

{
  "errors": [
    {
      "detail": "Missing required attribute in body",
      "meta": {},
      "source": {
        "header": "Authorization",
        "parameter": "limit",
        "pointer": "/data/attributes/title"
      },
      "status": "400",
      "title": "Bad Request"
    }
  ]
}

Too many requests

API error response.

Expand All

Field

Type

Description

errors [required]

[string]

A list of errors.

{
  "errors": [
    "Bad Request"
  ]
}

Code Example

                  # Path parameters
export project_id="a33671aa-24fd-4dcd-9b33-a8ec7dde7751"
export dataset_id="9f64e5c7-dc5a-45c8-a17c-1b85f0bec97d"
# Curl command
curl -X PATCH "https://api.ap1.datadoghq.com"https://api.ap2.datadoghq.com"https://api.datadoghq.eu"https://api.ddog-gov.com"https://api.datadoghq.com"https://api.us3.datadoghq.com"https://api.us5.datadoghq.com/api/v2/llm-obs/v1/${project_id}/datasets/${dataset_id}/records" \
-H "Accept: application/json" \
-H "Content-Type: application/json" \
-H "DD-API-KEY: ${DD_API_KEY}" \
-H "DD-APPLICATION-KEY: ${DD_APP_KEY}" \
-d @- << EOF
{
  "data": {
    "attributes": {
      "records": [
        {
          "id": "rec-7c3f5a1b-9e2d-4f8a-b1c6-3d7e9f0a2b4c"
        }
      ]
    },
    "type": "records"
  }
}
EOF

Delete LLM Observability datasets

Note: This endpoint is in preview and is subject to change. If you have any feedback, contact Datadog support.

POST https://api.ap1.datadoghq.com/api/v2/llm-obs/v1/{project_id}/datasets/deletehttps://api.ap2.datadoghq.com/api/v2/llm-obs/v1/{project_id}/datasets/deletehttps://api.datadoghq.eu/api/v2/llm-obs/v1/{project_id}/datasets/deletehttps://api.ddog-gov.com/api/v2/llm-obs/v1/{project_id}/datasets/deletehttps://api.datadoghq.com/api/v2/llm-obs/v1/{project_id}/datasets/deletehttps://api.us3.datadoghq.com/api/v2/llm-obs/v1/{project_id}/datasets/deletehttps://api.us5.datadoghq.com/api/v2/llm-obs/v1/{project_id}/datasets/delete

Overview

Delete one or more LLM Observability datasets within the specified project.

Arguments

Path Parameters

Name

Type

Description

project_id [required]

string

The ID of the LLM Observability project.

Request

Body Data (required)

Delete datasets payload.

Expand All

Field

Type

Description

data [required]

object

Data object for deleting LLM Observability datasets.

attributes [required]

object

Attributes for deleting LLM Observability datasets.

dataset_ids [required]

[string]

List of dataset IDs to delete.

type [required]

enum

Resource type of an LLM Observability dataset. Allowed enum values: datasets

{
  "data": {
    "attributes": {
      "dataset_ids": [
        "9f64e5c7-dc5a-45c8-a17c-1b85f0bec97d"
      ]
    },
    "type": "datasets"
  }
}

Response

No Content

Bad Request

API error response.

Expand All

Field

Type

Description

errors [required]

[object]

A list of errors.

detail

string

A human-readable explanation specific to this occurrence of the error.

meta

object

Non-standard meta-information about the error

source

object

References to the source of the error.

header

string

A string indicating the name of a single request header which caused the error.

parameter

string

A string indicating which URI query parameter caused the error.

pointer

string

A JSON pointer to the value in the request document that caused the error.

status

string

Status code of the response.

title

string

Short human-readable summary of the error.

{
  "errors": [
    {
      "detail": "Missing required attribute in body",
      "meta": {},
      "source": {
        "header": "Authorization",
        "parameter": "limit",
        "pointer": "/data/attributes/title"
      },
      "status": "400",
      "title": "Bad Request"
    }
  ]
}

Unauthorized

API error response.

Expand All

Field

Type

Description

errors [required]

[object]

A list of errors.

detail

string

A human-readable explanation specific to this occurrence of the error.

meta

object

Non-standard meta-information about the error

source

object

References to the source of the error.

header

string

A string indicating the name of a single request header which caused the error.

parameter

string

A string indicating which URI query parameter caused the error.

pointer

string

A JSON pointer to the value in the request document that caused the error.

status

string

Status code of the response.

title

string

Short human-readable summary of the error.

{
  "errors": [
    {
      "detail": "Missing required attribute in body",
      "meta": {},
      "source": {
        "header": "Authorization",
        "parameter": "limit",
        "pointer": "/data/attributes/title"
      },
      "status": "400",
      "title": "Bad Request"
    }
  ]
}

Forbidden

API error response.

Expand All

Field

Type

Description

errors [required]

[object]

A list of errors.

detail

string

A human-readable explanation specific to this occurrence of the error.

meta

object

Non-standard meta-information about the error

source

object

References to the source of the error.

header

string

A string indicating the name of a single request header which caused the error.

parameter

string

A string indicating which URI query parameter caused the error.

pointer

string

A JSON pointer to the value in the request document that caused the error.

status

string

Status code of the response.

title

string

Short human-readable summary of the error.

{
  "errors": [
    {
      "detail": "Missing required attribute in body",
      "meta": {},
      "source": {
        "header": "Authorization",
        "parameter": "limit",
        "pointer": "/data/attributes/title"
      },
      "status": "400",
      "title": "Bad Request"
    }
  ]
}

Not Found

API error response.

Expand All

Field

Type

Description

errors [required]

[object]

A list of errors.

detail

string

A human-readable explanation specific to this occurrence of the error.

meta

object

Non-standard meta-information about the error

source

object

References to the source of the error.

header

string

A string indicating the name of a single request header which caused the error.

parameter

string

A string indicating which URI query parameter caused the error.

pointer

string

A JSON pointer to the value in the request document that caused the error.

status

string

Status code of the response.

title

string

Short human-readable summary of the error.

{
  "errors": [
    {
      "detail": "Missing required attribute in body",
      "meta": {},
      "source": {
        "header": "Authorization",
        "parameter": "limit",
        "pointer": "/data/attributes/title"
      },
      "status": "400",
      "title": "Bad Request"
    }
  ]
}

Too many requests

API error response.

Expand All

Field

Type

Description

errors [required]

[string]

A list of errors.

{
  "errors": [
    "Bad Request"
  ]
}

Code Example

                  # Path parameters
export project_id="a33671aa-24fd-4dcd-9b33-a8ec7dde7751"
# Curl command
curl -X POST "https://api.ap1.datadoghq.com"https://api.ap2.datadoghq.com"https://api.datadoghq.eu"https://api.ddog-gov.com"https://api.datadoghq.com"https://api.us3.datadoghq.com"https://api.us5.datadoghq.com/api/v2/llm-obs/v1/${project_id}/datasets/delete" \
-H "Content-Type: application/json" \
-H "DD-API-KEY: ${DD_API_KEY}" \
-H "DD-APPLICATION-KEY: ${DD_APP_KEY}" \
-d @- << EOF
{
  "data": {
    "attributes": {
      "dataset_ids": [
        "9f64e5c7-dc5a-45c8-a17c-1b85f0bec97d"
      ]
    },
    "type": "datasets"
  }
}
EOF

Delete LLM Observability dataset records

Note: This endpoint is in preview and is subject to change. If you have any feedback, contact Datadog support.

POST https://api.ap1.datadoghq.com/api/v2/llm-obs/v1/{project_id}/datasets/{dataset_id}/records/deletehttps://api.ap2.datadoghq.com/api/v2/llm-obs/v1/{project_id}/datasets/{dataset_id}/records/deletehttps://api.datadoghq.eu/api/v2/llm-obs/v1/{project_id}/datasets/{dataset_id}/records/deletehttps://api.ddog-gov.com/api/v2/llm-obs/v1/{project_id}/datasets/{dataset_id}/records/deletehttps://api.datadoghq.com/api/v2/llm-obs/v1/{project_id}/datasets/{dataset_id}/records/deletehttps://api.us3.datadoghq.com/api/v2/llm-obs/v1/{project_id}/datasets/{dataset_id}/records/deletehttps://api.us5.datadoghq.com/api/v2/llm-obs/v1/{project_id}/datasets/{dataset_id}/records/delete

Overview

Delete one or more records from an LLM Observability dataset.

Arguments

Path Parameters

Name

Type

Description

project_id [required]

string

The ID of the LLM Observability project.

dataset_id [required]

string

The ID of the LLM Observability dataset.

Request

Body Data (required)

Delete records payload.

Expand All

Field

Type

Description

data [required]

object

Data object for deleting records from an LLM Observability dataset.

attributes [required]

object

Attributes for deleting records from an LLM Observability dataset.

record_ids [required]

[string]

List of record IDs to delete.

type [required]

enum

Resource type of LLM Observability dataset records. Allowed enum values: records

{
  "data": {
    "attributes": {
      "record_ids": [
        "rec-7c3f5a1b-9e2d-4f8a-b1c6-3d7e9f0a2b4c"
      ]
    },
    "type": "records"
  }
}

Response

No Content

Bad Request

API error response.

Expand All

Field

Type

Description

errors [required]

[object]

A list of errors.

detail

string

A human-readable explanation specific to this occurrence of the error.

meta

object

Non-standard meta-information about the error

source

object

References to the source of the error.

header

string

A string indicating the name of a single request header which caused the error.

parameter

string

A string indicating which URI query parameter caused the error.

pointer

string

A JSON pointer to the value in the request document that caused the error.

status

string

Status code of the response.

title

string

Short human-readable summary of the error.

{
  "errors": [
    {
      "detail": "Missing required attribute in body",
      "meta": {},
      "source": {
        "header": "Authorization",
        "parameter": "limit",
        "pointer": "/data/attributes/title"
      },
      "status": "400",
      "title": "Bad Request"
    }
  ]
}

Unauthorized

API error response.

Expand All

Field

Type

Description

errors [required]

[object]

A list of errors.

detail

string

A human-readable explanation specific to this occurrence of the error.

meta

object

Non-standard meta-information about the error

source

object

References to the source of the error.

header

string

A string indicating the name of a single request header which caused the error.

parameter

string

A string indicating which URI query parameter caused the error.

pointer

string

A JSON pointer to the value in the request document that caused the error.

status

string

Status code of the response.

title

string

Short human-readable summary of the error.

{
  "errors": [
    {
      "detail": "Missing required attribute in body",
      "meta": {},
      "source": {
        "header": "Authorization",
        "parameter": "limit",
        "pointer": "/data/attributes/title"
      },
      "status": "400",
      "title": "Bad Request"
    }
  ]
}

Forbidden

API error response.

Expand All

Field

Type

Description

errors [required]

[object]

A list of errors.

detail

string

A human-readable explanation specific to this occurrence of the error.

meta

object

Non-standard meta-information about the error

source

object

References to the source of the error.

header

string

A string indicating the name of a single request header which caused the error.

parameter

string

A string indicating which URI query parameter caused the error.

pointer

string

A JSON pointer to the value in the request document that caused the error.

status

string

Status code of the response.

title

string

Short human-readable summary of the error.

{
  "errors": [
    {
      "detail": "Missing required attribute in body",
      "meta": {},
      "source": {
        "header": "Authorization",
        "parameter": "limit",
        "pointer": "/data/attributes/title"
      },
      "status": "400",
      "title": "Bad Request"
    }
  ]
}

Not Found

API error response.

Expand All

Field

Type

Description

errors [required]

[object]

A list of errors.

detail

string

A human-readable explanation specific to this occurrence of the error.

meta

object

Non-standard meta-information about the error

source

object

References to the source of the error.

header

string

A string indicating the name of a single request header which caused the error.

parameter

string

A string indicating which URI query parameter caused the error.

pointer

string

A JSON pointer to the value in the request document that caused the error.

status

string

Status code of the response.

title

string

Short human-readable summary of the error.

{
  "errors": [
    {
      "detail": "Missing required attribute in body",
      "meta": {},
      "source": {
        "header": "Authorization",
        "parameter": "limit",
        "pointer": "/data/attributes/title"
      },
      "status": "400",
      "title": "Bad Request"
    }
  ]
}

Too many requests

API error response.

Expand All

Field

Type

Description

errors [required]

[string]

A list of errors.

{
  "errors": [
    "Bad Request"
  ]
}

Code Example

                  # Path parameters
export project_id="a33671aa-24fd-4dcd-9b33-a8ec7dde7751"
export dataset_id="9f64e5c7-dc5a-45c8-a17c-1b85f0bec97d"
# Curl command
curl -X POST "https://api.ap1.datadoghq.com"https://api.ap2.datadoghq.com"https://api.datadoghq.eu"https://api.ddog-gov.com"https://api.datadoghq.com"https://api.us3.datadoghq.com"https://api.us5.datadoghq.com/api/v2/llm-obs/v1/${project_id}/datasets/${dataset_id}/records/delete" \
-H "Content-Type: application/json" \
-H "DD-API-KEY: ${DD_API_KEY}" \
-H "DD-APPLICATION-KEY: ${DD_APP_KEY}" \
-d @- << EOF
{
  "data": {
    "attributes": {
      "record_ids": [
        "rec-7c3f5a1b-9e2d-4f8a-b1c6-3d7e9f0a2b4c"
      ]
    },
    "type": "records"
  }
}
EOF

List LLM Observability experiments

Note: This endpoint is in preview and is subject to change. If you have any feedback, contact Datadog support.

GET https://api.ap1.datadoghq.com/api/v2/llm-obs/v1/experimentshttps://api.ap2.datadoghq.com/api/v2/llm-obs/v1/experimentshttps://api.datadoghq.eu/api/v2/llm-obs/v1/experimentshttps://api.ddog-gov.com/api/v2/llm-obs/v1/experimentshttps://api.datadoghq.com/api/v2/llm-obs/v1/experimentshttps://api.us3.datadoghq.com/api/v2/llm-obs/v1/experimentshttps://api.us5.datadoghq.com/api/v2/llm-obs/v1/experiments

Overview

List all LLM Observability experiments sorted by creation date, newest first.

Arguments

Query Strings

Name

Type

Description

filter[project_id]

string

Filter experiments by project ID. Required if filter[dataset_id] is not provided.

filter[dataset_id]

string

Filter experiments by dataset ID.

filter[id]

string

Filter experiments by experiment ID. Can be specified multiple times.

page[cursor]

string

Use the Pagination cursor to retrieve the next page of results.

page[limit]

integer

Maximum number of results to return per page.

Response

OK

Response containing a list of LLM Observability experiments.

Expand All

Field

Type

Description

data [required]

[object]

List of experiments.

attributes [required]

object

Attributes of an LLM Observability experiment.

config [required]

object

Configuration parameters for the experiment.

created_at [required]

date-time

Timestamp when the experiment was created.

dataset_id [required]

string

Identifier of the dataset used in this experiment.

description [required]

string

Description of the experiment.

metadata [required]

object

Arbitrary metadata associated with the experiment.

name [required]

string

Name of the experiment.

project_id [required]

string

Identifier of the project this experiment belongs to.

updated_at [required]

date-time

Timestamp when the experiment was last updated.

id [required]

string

Unique identifier of the experiment.

type [required]

enum

Resource type of an LLM Observability experiment. Allowed enum values: experiments

meta

object

Pagination cursor metadata.

after

string

Cursor for the next page of results.

{
  "data": [
    {
      "attributes": {
        "config": {},
        "created_at": "2024-01-15T10:30:00Z",
        "dataset_id": "9f64e5c7-dc5a-45c8-a17c-1b85f0bec97d",
        "description": "",
        "metadata": {},
        "name": "My Experiment v1",
        "project_id": "a33671aa-24fd-4dcd-9b33-a8ec7dde7751",
        "updated_at": "2024-01-15T10:30:00Z"
      },
      "id": "3fd6b5e0-8910-4b1c-a7d0-5b84de329012",
      "type": "experiments"
    }
  ],
  "meta": {
    "after": "string"
  }
}

Bad Request

API error response.

Expand All

Field

Type

Description

errors [required]

[object]

A list of errors.

detail

string

A human-readable explanation specific to this occurrence of the error.

meta

object

Non-standard meta-information about the error

source

object

References to the source of the error.

header

string

A string indicating the name of a single request header which caused the error.

parameter

string

A string indicating which URI query parameter caused the error.

pointer

string

A JSON pointer to the value in the request document that caused the error.

status

string

Status code of the response.

title

string

Short human-readable summary of the error.

{
  "errors": [
    {
      "detail": "Missing required attribute in body",
      "meta": {},
      "source": {
        "header": "Authorization",
        "parameter": "limit",
        "pointer": "/data/attributes/title"
      },
      "status": "400",
      "title": "Bad Request"
    }
  ]
}

Unauthorized

API error response.

Expand All

Field

Type

Description

errors [required]

[object]

A list of errors.

detail

string

A human-readable explanation specific to this occurrence of the error.

meta

object

Non-standard meta-information about the error

source

object

References to the source of the error.

header

string

A string indicating the name of a single request header which caused the error.

parameter

string

A string indicating which URI query parameter caused the error.

pointer

string

A JSON pointer to the value in the request document that caused the error.

status

string

Status code of the response.

title

string

Short human-readable summary of the error.

{
  "errors": [
    {
      "detail": "Missing required attribute in body",
      "meta": {},
      "source": {
        "header": "Authorization",
        "parameter": "limit",
        "pointer": "/data/attributes/title"
      },
      "status": "400",
      "title": "Bad Request"
    }
  ]
}

Forbidden

API error response.

Expand All

Field

Type

Description

errors [required]

[object]

A list of errors.

detail

string

A human-readable explanation specific to this occurrence of the error.

meta

object

Non-standard meta-information about the error

source

object

References to the source of the error.

header

string

A string indicating the name of a single request header which caused the error.

parameter

string

A string indicating which URI query parameter caused the error.

pointer

string

A JSON pointer to the value in the request document that caused the error.

status

string

Status code of the response.

title

string

Short human-readable summary of the error.

{
  "errors": [
    {
      "detail": "Missing required attribute in body",
      "meta": {},
      "source": {
        "header": "Authorization",
        "parameter": "limit",
        "pointer": "/data/attributes/title"
      },
      "status": "400",
      "title": "Bad Request"
    }
  ]
}

Too many requests

API error response.

Expand All

Field

Type

Description

errors [required]

[string]

A list of errors.

{
  "errors": [
    "Bad Request"
  ]
}

Code Example

                  # Curl command
curl -X GET "https://api.ap1.datadoghq.com"https://api.ap2.datadoghq.com"https://api.datadoghq.eu"https://api.ddog-gov.com"https://api.datadoghq.com"https://api.us3.datadoghq.com"https://api.us5.datadoghq.com/api/v2/llm-obs/v1/experiments" \
-H "Accept: application/json" \
-H "DD-API-KEY: ${DD_API_KEY}" \
-H "DD-APPLICATION-KEY: ${DD_APP_KEY}"

Create an LLM Observability experiment

Note: This endpoint is in preview and is subject to change. If you have any feedback, contact Datadog support.

POST https://api.ap1.datadoghq.com/api/v2/llm-obs/v1/experimentshttps://api.ap2.datadoghq.com/api/v2/llm-obs/v1/experimentshttps://api.datadoghq.eu/api/v2/llm-obs/v1/experimentshttps://api.ddog-gov.com/api/v2/llm-obs/v1/experimentshttps://api.datadoghq.com/api/v2/llm-obs/v1/experimentshttps://api.us3.datadoghq.com/api/v2/llm-obs/v1/experimentshttps://api.us5.datadoghq.com/api/v2/llm-obs/v1/experiments

Overview

Create a new LLM Observability experiment.

Request

Body Data (required)

Create experiment payload.

Expand All

Field

Type

Description

data [required]

object

Data object for creating an LLM Observability experiment.

attributes [required]

object

Attributes for creating an LLM Observability experiment.

config

object

Configuration parameters for the experiment.

dataset_id [required]

string

Identifier of the dataset used in this experiment.

dataset_version

int64

Version of the dataset to use. Defaults to the current version if not specified.

description

string

Description of the experiment.

ensure_unique

boolean

Whether to ensure the experiment name is unique. Defaults to true.

metadata

object

Arbitrary metadata associated with the experiment.

name [required]

string

Name of the experiment.

project_id [required]

string

Identifier of the project this experiment belongs to.

type [required]

enum

Resource type of an LLM Observability experiment. Allowed enum values: experiments

{
  "data": {
    "attributes": {
      "config": {},
      "dataset_id": "9f64e5c7-dc5a-45c8-a17c-1b85f0bec97d",
      "dataset_version": "integer",
      "description": "string",
      "ensure_unique": false,
      "metadata": {},
      "name": "My Experiment v1",
      "project_id": "a33671aa-24fd-4dcd-9b33-a8ec7dde7751"
    },
    "type": "experiments"
  }
}

Response

OK

Response containing a single LLM Observability experiment.

Expand All

Field

Type

Description

data [required]

object

Data object for an LLM Observability experiment.

attributes [required]

object

Attributes of an LLM Observability experiment.

config [required]

object

Configuration parameters for the experiment.

created_at [required]

date-time

Timestamp when the experiment was created.

dataset_id [required]

string

Identifier of the dataset used in this experiment.

description [required]

string

Description of the experiment.

metadata [required]

object

Arbitrary metadata associated with the experiment.

name [required]

string

Name of the experiment.

project_id [required]

string

Identifier of the project this experiment belongs to.

updated_at [required]

date-time

Timestamp when the experiment was last updated.

id [required]

string

Unique identifier of the experiment.

type [required]

enum

Resource type of an LLM Observability experiment. Allowed enum values: experiments

{
  "data": {
    "attributes": {
      "config": {},
      "created_at": "2024-01-15T10:30:00Z",
      "dataset_id": "9f64e5c7-dc5a-45c8-a17c-1b85f0bec97d",
      "description": "",
      "metadata": {},
      "name": "My Experiment v1",
      "project_id": "a33671aa-24fd-4dcd-9b33-a8ec7dde7751",
      "updated_at": "2024-01-15T10:30:00Z"
    },
    "id": "3fd6b5e0-8910-4b1c-a7d0-5b84de329012",
    "type": "experiments"
  }
}

Created

Response containing a single LLM Observability experiment.

Expand All

Field

Type

Description

data [required]

object

Data object for an LLM Observability experiment.

attributes [required]

object

Attributes of an LLM Observability experiment.

config [required]

object

Configuration parameters for the experiment.

created_at [required]

date-time

Timestamp when the experiment was created.

dataset_id [required]

string

Identifier of the dataset used in this experiment.

description [required]

string

Description of the experiment.

metadata [required]

object

Arbitrary metadata associated with the experiment.

name [required]

string

Name of the experiment.

project_id [required]

string

Identifier of the project this experiment belongs to.

updated_at [required]

date-time

Timestamp when the experiment was last updated.

id [required]

string

Unique identifier of the experiment.

type [required]

enum

Resource type of an LLM Observability experiment. Allowed enum values: experiments

{
  "data": {
    "attributes": {
      "config": {},
      "created_at": "2024-01-15T10:30:00Z",
      "dataset_id": "9f64e5c7-dc5a-45c8-a17c-1b85f0bec97d",
      "description": "",
      "metadata": {},
      "name": "My Experiment v1",
      "project_id": "a33671aa-24fd-4dcd-9b33-a8ec7dde7751",
      "updated_at": "2024-01-15T10:30:00Z"
    },
    "id": "3fd6b5e0-8910-4b1c-a7d0-5b84de329012",
    "type": "experiments"
  }
}

Bad Request

API error response.

Expand All

Field

Type

Description

errors [required]

[object]

A list of errors.

detail

string

A human-readable explanation specific to this occurrence of the error.

meta

object

Non-standard meta-information about the error

source

object

References to the source of the error.

header

string

A string indicating the name of a single request header which caused the error.

parameter

string

A string indicating which URI query parameter caused the error.

pointer

string

A JSON pointer to the value in the request document that caused the error.

status

string

Status code of the response.

title

string

Short human-readable summary of the error.

{
  "errors": [
    {
      "detail": "Missing required attribute in body",
      "meta": {},
      "source": {
        "header": "Authorization",
        "parameter": "limit",
        "pointer": "/data/attributes/title"
      },
      "status": "400",
      "title": "Bad Request"
    }
  ]
}

Unauthorized

API error response.

Expand All

Field

Type

Description

errors [required]

[object]

A list of errors.

detail

string

A human-readable explanation specific to this occurrence of the error.

meta

object

Non-standard meta-information about the error

source

object

References to the source of the error.

header

string

A string indicating the name of a single request header which caused the error.

parameter

string

A string indicating which URI query parameter caused the error.

pointer

string

A JSON pointer to the value in the request document that caused the error.

status

string

Status code of the response.

title

string

Short human-readable summary of the error.

{
  "errors": [
    {
      "detail": "Missing required attribute in body",
      "meta": {},
      "source": {
        "header": "Authorization",
        "parameter": "limit",
        "pointer": "/data/attributes/title"
      },
      "status": "400",
      "title": "Bad Request"
    }
  ]
}

Forbidden

API error response.

Expand All

Field

Type

Description

errors [required]

[object]

A list of errors.

detail

string

A human-readable explanation specific to this occurrence of the error.

meta

object

Non-standard meta-information about the error

source

object

References to the source of the error.

header

string

A string indicating the name of a single request header which caused the error.

parameter

string

A string indicating which URI query parameter caused the error.

pointer

string

A JSON pointer to the value in the request document that caused the error.

status

string

Status code of the response.

title

string

Short human-readable summary of the error.

{
  "errors": [
    {
      "detail": "Missing required attribute in body",
      "meta": {},
      "source": {
        "header": "Authorization",
        "parameter": "limit",
        "pointer": "/data/attributes/title"
      },
      "status": "400",
      "title": "Bad Request"
    }
  ]
}

Too many requests

API error response.

Expand All

Field

Type

Description

errors [required]

[string]

A list of errors.

{
  "errors": [
    "Bad Request"
  ]
}

Code Example

                  # Curl command
curl -X POST "https://api.ap1.datadoghq.com"https://api.ap2.datadoghq.com"https://api.datadoghq.eu"https://api.ddog-gov.com"https://api.datadoghq.com"https://api.us3.datadoghq.com"https://api.us5.datadoghq.com/api/v2/llm-obs/v1/experiments" \
-H "Accept: application/json" \
-H "Content-Type: application/json" \
-H "DD-API-KEY: ${DD_API_KEY}" \
-H "DD-APPLICATION-KEY: ${DD_APP_KEY}" \
-d @- << EOF
{
  "data": {
    "attributes": {
      "dataset_id": "9f64e5c7-dc5a-45c8-a17c-1b85f0bec97d",
      "name": "My Experiment v1",
      "project_id": "a33671aa-24fd-4dcd-9b33-a8ec7dde7751"
    },
    "type": "experiments"
  }
}
EOF

Update an LLM Observability experiment

Note: This endpoint is in preview and is subject to change. If you have any feedback, contact Datadog support.

PATCH https://api.ap1.datadoghq.com/api/v2/llm-obs/v1/experiments/{experiment_id}https://api.ap2.datadoghq.com/api/v2/llm-obs/v1/experiments/{experiment_id}https://api.datadoghq.eu/api/v2/llm-obs/v1/experiments/{experiment_id}https://api.ddog-gov.com/api/v2/llm-obs/v1/experiments/{experiment_id}https://api.datadoghq.com/api/v2/llm-obs/v1/experiments/{experiment_id}https://api.us3.datadoghq.com/api/v2/llm-obs/v1/experiments/{experiment_id}https://api.us5.datadoghq.com/api/v2/llm-obs/v1/experiments/{experiment_id}

Overview

Partially update an existing LLM Observability experiment.

Arguments

Path Parameters

Name

Type

Description

experiment_id [required]

string

The ID of the LLM Observability experiment.

Request

Body Data (required)

Update experiment payload.

Expand All

Field

Type

Description

data [required]

object

Data object for updating an LLM Observability experiment.

attributes [required]

object

Attributes for updating an LLM Observability experiment.

description

string

Updated description of the experiment.

name

string

Updated name of the experiment.

type [required]

enum

Resource type of an LLM Observability experiment. Allowed enum values: experiments

{
  "data": {
    "attributes": {
      "description": "string",
      "name": "string"
    },
    "type": "experiments"
  }
}

Response

OK

Response containing a single LLM Observability experiment.

Expand All

Field

Type

Description

data [required]

object

Data object for an LLM Observability experiment.

attributes [required]

object

Attributes of an LLM Observability experiment.

config [required]

object

Configuration parameters for the experiment.

created_at [required]

date-time

Timestamp when the experiment was created.

dataset_id [required]

string

Identifier of the dataset used in this experiment.

description [required]

string

Description of the experiment.

metadata [required]

object

Arbitrary metadata associated with the experiment.

name [required]

string

Name of the experiment.

project_id [required]

string

Identifier of the project this experiment belongs to.

updated_at [required]

date-time

Timestamp when the experiment was last updated.

id [required]

string

Unique identifier of the experiment.

type [required]

enum

Resource type of an LLM Observability experiment. Allowed enum values: experiments

{
  "data": {
    "attributes": {
      "config": {},
      "created_at": "2024-01-15T10:30:00Z",
      "dataset_id": "9f64e5c7-dc5a-45c8-a17c-1b85f0bec97d",
      "description": "",
      "metadata": {},
      "name": "My Experiment v1",
      "project_id": "a33671aa-24fd-4dcd-9b33-a8ec7dde7751",
      "updated_at": "2024-01-15T10:30:00Z"
    },
    "id": "3fd6b5e0-8910-4b1c-a7d0-5b84de329012",
    "type": "experiments"
  }
}

Bad Request

API error response.

Expand All

Field

Type

Description

errors [required]

[object]

A list of errors.

detail

string

A human-readable explanation specific to this occurrence of the error.

meta

object

Non-standard meta-information about the error

source

object

References to the source of the error.

header

string

A string indicating the name of a single request header which caused the error.

parameter

string

A string indicating which URI query parameter caused the error.

pointer

string

A JSON pointer to the value in the request document that caused the error.

status

string

Status code of the response.

title

string

Short human-readable summary of the error.

{
  "errors": [
    {
      "detail": "Missing required attribute in body",
      "meta": {},
      "source": {
        "header": "Authorization",
        "parameter": "limit",
        "pointer": "/data/attributes/title"
      },
      "status": "400",
      "title": "Bad Request"
    }
  ]
}

Unauthorized

API error response.

Expand All

Field

Type

Description

errors [required]

[object]

A list of errors.

detail

string

A human-readable explanation specific to this occurrence of the error.

meta

object

Non-standard meta-information about the error

source

object

References to the source of the error.

header

string

A string indicating the name of a single request header which caused the error.

parameter

string

A string indicating which URI query parameter caused the error.

pointer

string

A JSON pointer to the value in the request document that caused the error.

status

string

Status code of the response.

title

string

Short human-readable summary of the error.

{
  "errors": [
    {
      "detail": "Missing required attribute in body",
      "meta": {},
      "source": {
        "header": "Authorization",
        "parameter": "limit",
        "pointer": "/data/attributes/title"
      },
      "status": "400",
      "title": "Bad Request"
    }
  ]
}

Forbidden

API error response.

Expand All

Field

Type

Description

errors [required]

[object]

A list of errors.

detail

string

A human-readable explanation specific to this occurrence of the error.

meta

object

Non-standard meta-information about the error

source

object

References to the source of the error.

header

string

A string indicating the name of a single request header which caused the error.

parameter

string

A string indicating which URI query parameter caused the error.

pointer

string

A JSON pointer to the value in the request document that caused the error.

status

string

Status code of the response.

title

string

Short human-readable summary of the error.

{
  "errors": [
    {
      "detail": "Missing required attribute in body",
      "meta": {},
      "source": {
        "header": "Authorization",
        "parameter": "limit",
        "pointer": "/data/attributes/title"
      },
      "status": "400",
      "title": "Bad Request"
    }
  ]
}

Not Found

API error response.

Expand All

Field

Type

Description

errors [required]

[object]

A list of errors.

detail

string

A human-readable explanation specific to this occurrence of the error.

meta

object

Non-standard meta-information about the error

source

object

References to the source of the error.

header

string

A string indicating the name of a single request header which caused the error.

parameter

string

A string indicating which URI query parameter caused the error.

pointer

string

A JSON pointer to the value in the request document that caused the error.

status

string

Status code of the response.

title

string

Short human-readable summary of the error.

{
  "errors": [
    {
      "detail": "Missing required attribute in body",
      "meta": {},
      "source": {
        "header": "Authorization",
        "parameter": "limit",
        "pointer": "/data/attributes/title"
      },
      "status": "400",
      "title": "Bad Request"
    }
  ]
}

Too many requests

API error response.

Expand All

Field

Type

Description

errors [required]

[string]

A list of errors.

{
  "errors": [
    "Bad Request"
  ]
}

Code Example

                  # Path parameters
export experiment_id="3fd6b5e0-8910-4b1c-a7d0-5b84de329012"
# Curl command
curl -X PATCH "https://api.ap1.datadoghq.com"https://api.ap2.datadoghq.com"https://api.datadoghq.eu"https://api.ddog-gov.com"https://api.datadoghq.com"https://api.us3.datadoghq.com"https://api.us5.datadoghq.com/api/v2/llm-obs/v1/experiments/${experiment_id}" \
-H "Accept: application/json" \
-H "Content-Type: application/json" \
-H "DD-API-KEY: ${DD_API_KEY}" \
-H "DD-APPLICATION-KEY: ${DD_APP_KEY}" \
-d @- << EOF
{
  "data": {
    "attributes": {},
    "type": "experiments"
  }
}
EOF

Delete LLM Observability experiments

Note: This endpoint is in preview and is subject to change. If you have any feedback, contact Datadog support.

POST https://api.ap1.datadoghq.com/api/v2/llm-obs/v1/experiments/deletehttps://api.ap2.datadoghq.com/api/v2/llm-obs/v1/experiments/deletehttps://api.datadoghq.eu/api/v2/llm-obs/v1/experiments/deletehttps://api.ddog-gov.com/api/v2/llm-obs/v1/experiments/deletehttps://api.datadoghq.com/api/v2/llm-obs/v1/experiments/deletehttps://api.us3.datadoghq.com/api/v2/llm-obs/v1/experiments/deletehttps://api.us5.datadoghq.com/api/v2/llm-obs/v1/experiments/delete

Overview

Delete one or more LLM Observability experiments.

Request

Body Data (required)

Delete experiments payload.

Expand All

Field

Type

Description

data [required]

object

Data object for deleting LLM Observability experiments.

attributes [required]

object

Attributes for deleting LLM Observability experiments.

experiment_ids [required]

[string]

List of experiment IDs to delete.

type [required]

enum

Resource type of an LLM Observability experiment. Allowed enum values: experiments

{
  "data": {
    "attributes": {
      "experiment_ids": [
        "3fd6b5e0-8910-4b1c-a7d0-5b84de329012"
      ]
    },
    "type": "experiments"
  }
}

Response

No Content

Bad Request

API error response.

Expand All

Field

Type

Description

errors [required]

[object]

A list of errors.

detail

string

A human-readable explanation specific to this occurrence of the error.

meta

object

Non-standard meta-information about the error

source

object

References to the source of the error.

header

string

A string indicating the name of a single request header which caused the error.

parameter

string

A string indicating which URI query parameter caused the error.

pointer

string

A JSON pointer to the value in the request document that caused the error.

status

string

Status code of the response.

title

string

Short human-readable summary of the error.

{
  "errors": [
    {
      "detail": "Missing required attribute in body",
      "meta": {},
      "source": {
        "header": "Authorization",
        "parameter": "limit",
        "pointer": "/data/attributes/title"
      },
      "status": "400",
      "title": "Bad Request"
    }
  ]
}

Unauthorized

API error response.

Expand All

Field

Type

Description

errors [required]

[object]

A list of errors.

detail

string

A human-readable explanation specific to this occurrence of the error.

meta

object

Non-standard meta-information about the error

source

object

References to the source of the error.

header

string

A string indicating the name of a single request header which caused the error.

parameter

string

A string indicating which URI query parameter caused the error.

pointer

string

A JSON pointer to the value in the request document that caused the error.

status

string

Status code of the response.

title

string

Short human-readable summary of the error.

{
  "errors": [
    {
      "detail": "Missing required attribute in body",
      "meta": {},
      "source": {
        "header": "Authorization",
        "parameter": "limit",
        "pointer": "/data/attributes/title"
      },
      "status": "400",
      "title": "Bad Request"
    }
  ]
}

Forbidden

API error response.

Expand All

Field

Type

Description

errors [required]

[object]

A list of errors.

detail

string

A human-readable explanation specific to this occurrence of the error.

meta

object

Non-standard meta-information about the error

source

object

References to the source of the error.

header

string

A string indicating the name of a single request header which caused the error.

parameter

string

A string indicating which URI query parameter caused the error.

pointer

string

A JSON pointer to the value in the request document that caused the error.

status

string

Status code of the response.

title

string

Short human-readable summary of the error.

{
  "errors": [
    {
      "detail": "Missing required attribute in body",
      "meta": {},
      "source": {
        "header": "Authorization",
        "parameter": "limit",
        "pointer": "/data/attributes/title"
      },
      "status": "400",
      "title": "Bad Request"
    }
  ]
}

Too many requests

API error response.

Expand All

Field

Type

Description

errors [required]

[string]

A list of errors.

{
  "errors": [
    "Bad Request"
  ]
}

Code Example

                  # Curl command
curl -X POST "https://api.ap1.datadoghq.com"https://api.ap2.datadoghq.com"https://api.datadoghq.eu"https://api.ddog-gov.com"https://api.datadoghq.com"https://api.us3.datadoghq.com"https://api.us5.datadoghq.com/api/v2/llm-obs/v1/experiments/delete" \
-H "Content-Type: application/json" \
-H "DD-API-KEY: ${DD_API_KEY}" \
-H "DD-APPLICATION-KEY: ${DD_APP_KEY}" \
-d @- << EOF
{
  "data": {
    "attributes": {
      "experiment_ids": [
        "3fd6b5e0-8910-4b1c-a7d0-5b84de329012"
      ]
    },
    "type": "experiments"
  }
}
EOF

Push events for an LLM Observability experiment

Note: This endpoint is in preview and is subject to change. If you have any feedback, contact Datadog support.

POST https://api.ap1.datadoghq.com/api/v2/llm-obs/v1/experiments/{experiment_id}/eventshttps://api.ap2.datadoghq.com/api/v2/llm-obs/v1/experiments/{experiment_id}/eventshttps://api.datadoghq.eu/api/v2/llm-obs/v1/experiments/{experiment_id}/eventshttps://api.ddog-gov.com/api/v2/llm-obs/v1/experiments/{experiment_id}/eventshttps://api.datadoghq.com/api/v2/llm-obs/v1/experiments/{experiment_id}/eventshttps://api.us3.datadoghq.com/api/v2/llm-obs/v1/experiments/{experiment_id}/eventshttps://api.us5.datadoghq.com/api/v2/llm-obs/v1/experiments/{experiment_id}/events

Overview

Push spans and metrics for an LLM Observability experiment.

Arguments

Path Parameters

Name

Type

Description

experiment_id [required]

string

The ID of the LLM Observability experiment.

Request

Body Data (required)

Experiment events payload.

Expand All

Field

Type

Description

data [required]

object

Data object for pushing experiment events.

attributes [required]

object

Attributes for pushing experiment events including spans and metrics.

metrics

[object]

List of metrics to push for the experiment.

assessment

enum

Assessment result for an LLM Observability experiment metric. Allowed enum values: pass,fail

boolean_value

boolean

Boolean value. Used when metric_type is boolean.

categorical_value

string

Categorical value. Used when metric_type is categorical.

error

object

Error details for an experiment metric evaluation.

message

string

Error message associated with the metric evaluation.

json_value

object

JSON value. Used when metric_type is json.

label [required]

string

Label or name for the metric.

metadata

object

Arbitrary metadata associated with the metric.

metric_type [required]

enum

Type of metric recorded for an LLM Observability experiment. Allowed enum values: score,categorical,boolean,json

reasoning

string

Human-readable reasoning for the metric value.

score_value

double

Numeric score value. Used when metric_type is score.

span_id [required]

string

The ID of the span this metric measures.

tags

[string]

List of tags associated with the metric.

timestamp_ms [required]

int64

Timestamp when the metric was recorded, in milliseconds since Unix epoch.

spans

[object]

List of spans to push for the experiment.

dataset_id [required]

string

Dataset ID associated with this span.

duration [required]

int64

Duration of the span in nanoseconds.

meta

object

Metadata associated with an experiment span.

error

object

Error details for an experiment span.

message

string

Error message.

stack

string

Stack trace of the error.

type

string

The error type or exception class name.

expected_output

object

Expected output for the span, used for evaluation.

input

object <oneOf>

Represents any valid JSON value.

Option 1

string

Option 2

double

Option 3

object

Option 4

[ <oneOf>]

Option 1

string

Option 2

double

Option 3

object

Option 4

boolean

Option 5

boolean

output

object <oneOf>

Represents any valid JSON value.

Option 1

string

Option 2

double

Option 3

object

Option 4

[ <oneOf>]

Option 1

string

Option 2

double

Option 3

object

Option 4

boolean

Option 5

boolean

name [required]

string

Name of the span.

project_id [required]

string

Project ID associated with this span.

span_id [required]

string

Unique identifier of the span.

start_ns [required]

int64

Start time of the span in nanoseconds since Unix epoch.

status [required]

enum

Status of the span. Allowed enum values: ok,error

tags

[string]

List of tags associated with the span.

trace_id [required]

string

Trace ID for the span.

type [required]

enum

Resource type for LLM Observability experiment events. Allowed enum values: events

{
  "data": {
    "attributes": {
      "metrics": [
        {
          "assessment": "pass",
          "boolean_value": false,
          "categorical_value": "string",
          "error": {
            "message": "string"
          },
          "json_value": {},
          "label": "faithfulness",
          "metadata": {},
          "metric_type": "score",
          "reasoning": "string",
          "score_value": "number",
          "span_id": "span-7a1b2c3d",
          "tags": [],
          "timestamp_ms": 1705314600000
        }
      ],
      "spans": [
        {
          "dataset_id": "9f64e5c7-dc5a-45c8-a17c-1b85f0bec97d",
          "duration": 1500000000,
          "meta": {
            "error": {
              "message": "Model response timed out",
              "stack": "Traceback (most recent call last):\n  File \"main.py\", line 10, in <module>\n    response = model.generate(input)\n  File \"model.py\", line 45, in generate\n    raise TimeoutError(\"Model response timed out\")\nTimeoutError: Model response timed out",
              "type": "TimeoutError"
            },
            "expected_output": {},
            "input": {
              "type": "undefined"
            },
            "output": {
              "type": "undefined"
            }
          },
          "name": "llm_call",
          "project_id": "a33671aa-24fd-4dcd-9b33-a8ec7dde7751",
          "span_id": "span-7a1b2c3d",
          "start_ns": 1705314600000000000,
          "status": "ok",
          "tags": [],
          "trace_id": "abc123def456"
        }
      ]
    },
    "type": "events"
  }
}

Response

Accepted

Bad Request

API error response.

Expand All

Field

Type

Description

errors [required]

[object]

A list of errors.

detail

string

A human-readable explanation specific to this occurrence of the error.

meta

object

Non-standard meta-information about the error

source

object

References to the source of the error.

header

string

A string indicating the name of a single request header which caused the error.

parameter

string

A string indicating which URI query parameter caused the error.

pointer

string

A JSON pointer to the value in the request document that caused the error.

status

string

Status code of the response.

title

string

Short human-readable summary of the error.

{
  "errors": [
    {
      "detail": "Missing required attribute in body",
      "meta": {},
      "source": {
        "header": "Authorization",
        "parameter": "limit",
        "pointer": "/data/attributes/title"
      },
      "status": "400",
      "title": "Bad Request"
    }
  ]
}

Unauthorized

API error response.

Expand All

Field

Type

Description

errors [required]

[object]

A list of errors.

detail

string

A human-readable explanation specific to this occurrence of the error.

meta

object

Non-standard meta-information about the error

source

object

References to the source of the error.

header

string

A string indicating the name of a single request header which caused the error.

parameter

string

A string indicating which URI query parameter caused the error.

pointer

string

A JSON pointer to the value in the request document that caused the error.

status

string

Status code of the response.

title

string

Short human-readable summary of the error.

{
  "errors": [
    {
      "detail": "Missing required attribute in body",
      "meta": {},
      "source": {
        "header": "Authorization",
        "parameter": "limit",
        "pointer": "/data/attributes/title"
      },
      "status": "400",
      "title": "Bad Request"
    }
  ]
}

Forbidden

API error response.

Expand All

Field

Type

Description

errors [required]

[object]

A list of errors.

detail

string

A human-readable explanation specific to this occurrence of the error.

meta

object

Non-standard meta-information about the error

source

object

References to the source of the error.

header

string

A string indicating the name of a single request header which caused the error.

parameter

string

A string indicating which URI query parameter caused the error.

pointer

string

A JSON pointer to the value in the request document that caused the error.

status

string

Status code of the response.

title

string

Short human-readable summary of the error.

{
  "errors": [
    {
      "detail": "Missing required attribute in body",
      "meta": {},
      "source": {
        "header": "Authorization",
        "parameter": "limit",
        "pointer": "/data/attributes/title"
      },
      "status": "400",
      "title": "Bad Request"
    }
  ]
}

Not Found

API error response.

Expand All

Field

Type

Description

errors [required]

[object]

A list of errors.

detail

string

A human-readable explanation specific to this occurrence of the error.

meta

object

Non-standard meta-information about the error

source

object

References to the source of the error.

header

string

A string indicating the name of a single request header which caused the error.

parameter

string

A string indicating which URI query parameter caused the error.

pointer

string

A JSON pointer to the value in the request document that caused the error.

status

string

Status code of the response.

title

string

Short human-readable summary of the error.

{
  "errors": [
    {
      "detail": "Missing required attribute in body",
      "meta": {},
      "source": {
        "header": "Authorization",
        "parameter": "limit",
        "pointer": "/data/attributes/title"
      },
      "status": "400",
      "title": "Bad Request"
    }
  ]
}

Too many requests

API error response.

Expand All

Field

Type

Description

errors [required]

[string]

A list of errors.

{
  "errors": [
    "Bad Request"
  ]
}

Code Example

                  # Path parameters
export experiment_id="3fd6b5e0-8910-4b1c-a7d0-5b84de329012"
# Curl command
curl -X POST "https://api.ap1.datadoghq.com"https://api.ap2.datadoghq.com"https://api.datadoghq.eu"https://api.ddog-gov.com"https://api.datadoghq.com"https://api.us3.datadoghq.com"https://api.us5.datadoghq.com/api/v2/llm-obs/v1/experiments/${experiment_id}/events" \
-H "Accept: application/json" \
-H "Content-Type: application/json" \
-H "DD-API-KEY: ${DD_API_KEY}" \
-H "DD-APPLICATION-KEY: ${DD_APP_KEY}" \
-d @- << EOF
{
  "data": {
    "attributes": {
      "metrics": [
        {
          "label": "faithfulness",
          "metric_type": "score",
          "span_id": "span-7a1b2c3d",
          "timestamp_ms": 1705314600000
        }
      ],
      "spans": [
        {
          "dataset_id": "9f64e5c7-dc5a-45c8-a17c-1b85f0bec97d",
          "duration": 1500000000,
          "name": "llm_call",
          "project_id": "a33671aa-24fd-4dcd-9b33-a8ec7dde7751",
          "span_id": "span-7a1b2c3d",
          "start_ns": 1705314600000000000,
          "status": "ok",
          "trace_id": "abc123def456"
        }
      ]
    },
    "type": "events"
  }
}
EOF