Language

English Français 日本語 한국어 Español

Datadog Site

Site help
US1 US3 US5 EU AP1 AP2 US1-FED US2-FED

Search LLM Observability experimentation entities

Note: This endpoint is in preview and is subject to change. If you have any feedback, contact Datadog support.

POST https://api.ap1.datadoghq.com/api/v2/llm-obs/v1/experimentation/searchhttps://api.ap2.datadoghq.com/api/v2/llm-obs/v1/experimentation/searchhttps://api.datadoghq.eu/api/v2/llm-obs/v1/experimentation/searchhttps://api.ddog-gov.com/api/v2/llm-obs/v1/experimentation/searchhttps://api.us2.ddog-gov.com/api/v2/llm-obs/v1/experimentation/searchhttps://api.datadoghq.com/api/v2/llm-obs/v1/experimentation/searchhttps://api.us3.datadoghq.com/api/v2/llm-obs/v1/experimentation/searchhttps://api.us5.datadoghq.com/api/v2/llm-obs/v1/experimentation/search

Overview

Search across LLM Observability experimentation entities — projects, datasets, dataset records, experiments, and experiment runs — using cursor-based pagination.

The filter.scope field controls which entity types are returned. At least one valid scope must be provided.

Returns 200 OK when all results fit in a single page. Returns 206 Partial Content with a cursor in meta.after when additional pages are available.

Request

Body Data (required)

Experimentation search payload.

Expand All

Field

Type

Description

data [required]

object

Data object for an experimentation search request.

attributes [required]

object

Attributes for an experimentation search request.

content_preview

object

Options to control content preview truncation.

limit

int64

Maximum number of characters to include in content previews.

filter [required]

object

Filter criteria for an experimentation search request.

include_deleted

boolean

When true, include soft-deleted entities alongside active ones.

is_deleted

boolean

When true, return only soft-deleted entities.

query

string

Free-text search query.

scope [required]

[string]

Entity types to search. Valid values are projects, datasets, dataset_records, experiments, and experiment_runs.

version

int64

Filter dataset records by a specific dataset version.

include

object

Additional data to include in the response.

user_data

boolean

When true, enrich results with author user data (name and email).

page

object

Cursor-based pagination parameters.

cursor

string

Opaque cursor returned from a previous response to fetch the next page.

limit

int64

Maximum number of results per page.

type [required]

enum

Resource type for experimentation search and analytics operations. Allowed enum values: experimentation

{
  "data": {
    "attributes": {
      "content_preview": {
        "limit": 500
      },
      "filter": {
        "include_deleted": false,
        "is_deleted": false,
        "query": "my experiment",
        "scope": [
          "experiments"
        ],
        "version": "integer"
      },
      "include": {
        "user_data": false
      },
      "page": {
        "cursor": "string",
        "limit": 100
      }
    },
    "type": "experimentation"
  }
}

Response

OK — all results returned in a single page.

Response to a cursor-based experimentation search. Returns 200 OK when all results fit in one page; 206 Partial Content when a next-page cursor is available.

Expand All

Field

Type

Description

data [required]

object

JSON:API data object for an experimentation search response.

attributes [required]

object

The matching experimentation entities grouped by type.

dataset_records

[object]

Matching dataset records. Present when dataset_records is included in filter.scope.

created_at [required]

date-time

Timestamp when the record was created.

dataset_id [required]

string

Identifier of the dataset this record belongs to.

expected_output [required]

object <oneOf>

Represents any valid JSON value.

Option 1

string

A scalar string value.

Option 2

double

A scalar numeric value.

Option 3

object

An arbitrary object value with additional properties.

Option 4

[ <oneOf>]

An array of arbitrary values.

Option 1

string

A scalar string value.

Option 2

double

A scalar numeric value.

Option 3

object

An arbitrary object value with additional properties.

Option 4

boolean

A scalar boolean value.

Option 5

boolean

A scalar boolean value.

id [required]

string

Unique identifier of the record.

input [required]

object <oneOf>

Represents any valid JSON value.

Option 1

string

A scalar string value.

Option 2

double

A scalar numeric value.

Option 3

object

An arbitrary object value with additional properties.

Option 4

[ <oneOf>]

An array of arbitrary values.

Option 1

string

A scalar string value.

Option 2

double

A scalar numeric value.

Option 3

object

An arbitrary object value with additional properties.

Option 4

boolean

A scalar boolean value.

Option 5

boolean

A scalar boolean value.

metadata [required]

object

Arbitrary metadata associated with the record.

updated_at [required]

date-time

Timestamp when the record was last updated.

datasets

[object]

Matching datasets. Present when datasets is included in filter.scope.

attributes [required]

object

Attributes of an LLM Observability dataset.

created_at [required]

date-time

Timestamp when the dataset was created.

current_version [required]

int64

Current version number of the dataset.

description [required]

string

Description of the dataset.

metadata [required]

object

Arbitrary metadata associated with the dataset.

name [required]

string

Name of the dataset.

updated_at [required]

date-time

Timestamp when the dataset was last updated.

id [required]

string

Unique identifier of the dataset.

type [required]

enum

Resource type of an LLM Observability dataset. Allowed enum values: datasets

experiment_runs

[object]

Matching experiment runs. Present when experiment_runs is included in filter.scope.

aggregate_data

object

Aggregated metric data for this run.

created_at

date-time

Timestamp when the run was created.

experiment_id

string

Identifier of the experiment this run belongs to.

id

string

Unique identifier of the experiment run.

run_number

int32

Sequential number of this run within the experiment.

experiments

[object]

Matching experiments. Present when experiments is included in filter.scope.

aggregate_data

object

Pre-computed aggregate metrics for this experiment run, including eval score distributions, token costs, and error rates.

author

object

User data for the author of an experiment. Only present when include[user_data] is true.

email

string

Email address of the user.

handle

string

Username or handle associated with the user's Datadog account.

icon

string

URL of the user's icon.

id

string

Unique identifier of the user.

name

string

Display name of the user.

config [required]

object

Configuration parameters for the experiment.

created_at [required]

date-time

Timestamp when the experiment was created.

dataset_id [required]

string

Identifier of the dataset used in this experiment.

dataset_name

string

Name of the dataset used in this experiment. Only present when include[dataset_names] is true.

dataset_version

int64

Version of the dataset used in this experiment.

deleted_at

date-time

Timestamp when the experiment was soft-deleted, if applicable.

description [required]

string

Description of the experiment.

error

string

Error message describing why the experiment failed, if applicable.

experiment

string

Logical name of the experiment, shared across all runs of the same pipeline.

metadata [required]

object

Arbitrary metadata associated with the experiment.

name [required]

string

Name of the experiment.

parent_experiment_id

string

Identifier of the parent (baseline) experiment this experiment was run against, if any.

project_id [required]

string

Identifier of the project this experiment belongs to.

run_count

int32

Expected number of runs for this experiment.

status

enum

Execution status of an LLM Observability experiment. Allowed enum values: running,completed,failed,interrupted

updated_at [required]

date-time

Timestamp when the experiment was last updated.

projects

[object]

Matching projects. Present when projects is included in filter.scope.

attributes [required]

object

Attributes of an LLM Observability project.

created_at [required]

date-time

Timestamp when the project was created.

description [required]

string

Description of the project.

name [required]

string

Name of the project.

updated_at [required]

date-time

Timestamp when the project was last updated.

id [required]

string

Unique identifier of the project.

type [required]

enum

Resource type of an LLM Observability project. Allowed enum values: projects

id [required]

string

Server-generated identifier for this search result.

type [required]

enum

Resource type for experimentation search and analytics operations. Allowed enum values: experimentation

meta

object

Pagination cursor metadata.

after

string

Cursor for the next page of results.

{
  "data": {
    "attributes": {
      "dataset_records": [
        {
          "created_at": "2024-01-15T10:30:00Z",
          "dataset_id": "9f64e5c7-dc5a-45c8-a17c-1b85f0bec97d",
          "expected_output": {
            "description": "undefined",
            "type": "undefined"
          },
          "id": "rec-7c3f5a1b-9e2d-4f8a-b1c6-3d7e9f0a2b4c",
          "input": {
            "description": "undefined",
            "type": "undefined"
          },
          "metadata": {},
          "updated_at": "2024-01-15T10:30:00Z"
        }
      ],
      "datasets": [
        {
          "attributes": {
            "created_at": "2024-01-15T10:30:00Z",
            "current_version": 1,
            "description": "",
            "metadata": {},
            "name": "My LLM Dataset",
            "updated_at": "2024-01-15T10:30:00Z"
          },
          "id": "9f64e5c7-dc5a-45c8-a17c-1b85f0bec97d",
          "type": "datasets"
        }
      ],
      "experiment_runs": [
        {
          "aggregate_data": {},
          "created_at": "2024-01-15T10:30:00Z",
          "experiment_id": "3fd6b5e0-8910-4b1c-a7d0-5b84de329012",
          "id": "7a1b2c3d-4e5f-6789-abcd-ef0123456789",
          "run_number": 1
        }
      ],
      "experiments": [
        {
          "aggregate_data": {},
          "author": {
            "email": "jane.doe@example.com",
            "handle": "jane.doe@example.com",
            "icon": "https://example.com/icon.png",
            "id": "00000000-0000-0000-0000-000000000010",
            "name": "Jane Doe"
          },
          "config": {},
          "created_at": "2024-01-15T10:30:00Z",
          "dataset_id": "9f64e5c7-dc5a-45c8-a17c-1b85f0bec97d",
          "dataset_name": "string",
          "dataset_version": "integer",
          "deleted_at": "2019-09-19T10:00:00.000Z",
          "description": "",
          "error": "string",
          "experiment": "my-pipeline",
          "metadata": {},
          "name": "My Experiment v1",
          "parent_experiment_id": "3fd6b5e0-8910-4b1c-a7d0-5b84de329012",
          "project_id": "a33671aa-24fd-4dcd-9b33-a8ec7dde7751",
          "run_count": "integer",
          "status": "completed",
          "updated_at": "2024-01-15T10:30:00Z"
        }
      ],
      "projects": [
        {
          "attributes": {
            "created_at": "2024-01-15T10:30:00Z",
            "description": "",
            "name": "My LLM Project",
            "updated_at": "2024-01-15T10:30:00Z"
          },
          "id": "a33671aa-24fd-4dcd-9b33-a8ec7dde7751",
          "type": "projects"
        }
      ]
    },
    "id": "00000000-0000-0000-0000-000000000001",
    "type": "experimentation"
  },
  "meta": {
    "after": "string"
  }
}

Partial Content — more results are available. Use `meta.after` as the next `page.cursor`.

Response to a cursor-based experimentation search. Returns 200 OK when all results fit in one page; 206 Partial Content when a next-page cursor is available.

Expand All

Field

Type

Description

data [required]

object

JSON:API data object for an experimentation search response.

attributes [required]

object

The matching experimentation entities grouped by type.

dataset_records

[object]

Matching dataset records. Present when dataset_records is included in filter.scope.

created_at [required]

date-time

Timestamp when the record was created.

dataset_id [required]

string

Identifier of the dataset this record belongs to.

expected_output [required]

object <oneOf>

Represents any valid JSON value.

Option 1

string

A scalar string value.

Option 2

double

A scalar numeric value.

Option 3

object

An arbitrary object value with additional properties.

Option 4

[ <oneOf>]

An array of arbitrary values.

Option 1

string

A scalar string value.

Option 2

double

A scalar numeric value.

Option 3

object

An arbitrary object value with additional properties.

Option 4

boolean

A scalar boolean value.

Option 5

boolean

A scalar boolean value.

id [required]

string

Unique identifier of the record.

input [required]

object <oneOf>

Represents any valid JSON value.

Option 1

string

A scalar string value.

Option 2

double

A scalar numeric value.

Option 3

object

An arbitrary object value with additional properties.

Option 4

[ <oneOf>]

An array of arbitrary values.

Option 1

string

A scalar string value.

Option 2

double

A scalar numeric value.

Option 3

object

An arbitrary object value with additional properties.

Option 4

boolean

A scalar boolean value.

Option 5

boolean

A scalar boolean value.

metadata [required]

object

Arbitrary metadata associated with the record.

updated_at [required]

date-time

Timestamp when the record was last updated.

datasets

[object]

Matching datasets. Present when datasets is included in filter.scope.

attributes [required]

object

Attributes of an LLM Observability dataset.

created_at [required]

date-time

Timestamp when the dataset was created.

current_version [required]

int64

Current version number of the dataset.

description [required]

string

Description of the dataset.

metadata [required]

object

Arbitrary metadata associated with the dataset.

name [required]

string

Name of the dataset.

updated_at [required]

date-time

Timestamp when the dataset was last updated.

id [required]

string

Unique identifier of the dataset.

type [required]

enum

Resource type of an LLM Observability dataset. Allowed enum values: datasets

experiment_runs

[object]

Matching experiment runs. Present when experiment_runs is included in filter.scope.

aggregate_data

object

Aggregated metric data for this run.

created_at

date-time

Timestamp when the run was created.

experiment_id

string

Identifier of the experiment this run belongs to.

id

string

Unique identifier of the experiment run.

run_number

int32

Sequential number of this run within the experiment.

experiments

[object]

Matching experiments. Present when experiments is included in filter.scope.

aggregate_data

object

Pre-computed aggregate metrics for this experiment run, including eval score distributions, token costs, and error rates.

author

object

User data for the author of an experiment. Only present when include[user_data] is true.

email

string

Email address of the user.

handle

string

Username or handle associated with the user's Datadog account.

icon

string

URL of the user's icon.

id

string

Unique identifier of the user.

name

string

Display name of the user.

config [required]

object

Configuration parameters for the experiment.

created_at [required]

date-time

Timestamp when the experiment was created.

dataset_id [required]

string

Identifier of the dataset used in this experiment.

dataset_name

string

Name of the dataset used in this experiment. Only present when include[dataset_names] is true.

dataset_version

int64

Version of the dataset used in this experiment.

deleted_at

date-time

Timestamp when the experiment was soft-deleted, if applicable.

description [required]

string

Description of the experiment.

error

string

Error message describing why the experiment failed, if applicable.

experiment

string

Logical name of the experiment, shared across all runs of the same pipeline.

metadata [required]

object

Arbitrary metadata associated with the experiment.

name [required]

string

Name of the experiment.

parent_experiment_id

string

Identifier of the parent (baseline) experiment this experiment was run against, if any.

project_id [required]

string

Identifier of the project this experiment belongs to.

run_count

int32

Expected number of runs for this experiment.

status

enum

Execution status of an LLM Observability experiment. Allowed enum values: running,completed,failed,interrupted

updated_at [required]

date-time

Timestamp when the experiment was last updated.

projects

[object]

Matching projects. Present when projects is included in filter.scope.

attributes [required]

object

Attributes of an LLM Observability project.

created_at [required]

date-time

Timestamp when the project was created.

description [required]

string

Description of the project.

name [required]

string

Name of the project.

updated_at [required]

date-time

Timestamp when the project was last updated.

id [required]

string

Unique identifier of the project.

type [required]

enum

Resource type of an LLM Observability project. Allowed enum values: projects

id [required]

string

Server-generated identifier for this search result.

type [required]

enum

Resource type for experimentation search and analytics operations. Allowed enum values: experimentation

meta

object

Pagination cursor metadata.

after

string

Cursor for the next page of results.

{
  "data": {
    "attributes": {
      "dataset_records": [
        {
          "created_at": "2024-01-15T10:30:00Z",
          "dataset_id": "9f64e5c7-dc5a-45c8-a17c-1b85f0bec97d",
          "expected_output": {
            "description": "undefined",
            "type": "undefined"
          },
          "id": "rec-7c3f5a1b-9e2d-4f8a-b1c6-3d7e9f0a2b4c",
          "input": {
            "description": "undefined",
            "type": "undefined"
          },
          "metadata": {},
          "updated_at": "2024-01-15T10:30:00Z"
        }
      ],
      "datasets": [
        {
          "attributes": {
            "created_at": "2024-01-15T10:30:00Z",
            "current_version": 1,
            "description": "",
            "metadata": {},
            "name": "My LLM Dataset",
            "updated_at": "2024-01-15T10:30:00Z"
          },
          "id": "9f64e5c7-dc5a-45c8-a17c-1b85f0bec97d",
          "type": "datasets"
        }
      ],
      "experiment_runs": [
        {
          "aggregate_data": {},
          "created_at": "2024-01-15T10:30:00Z",
          "experiment_id": "3fd6b5e0-8910-4b1c-a7d0-5b84de329012",
          "id": "7a1b2c3d-4e5f-6789-abcd-ef0123456789",
          "run_number": 1
        }
      ],
      "experiments": [
        {
          "aggregate_data": {},
          "author": {
            "email": "jane.doe@example.com",
            "handle": "jane.doe@example.com",
            "icon": "https://example.com/icon.png",
            "id": "00000000-0000-0000-0000-000000000010",
            "name": "Jane Doe"
          },
          "config": {},
          "created_at": "2024-01-15T10:30:00Z",
          "dataset_id": "9f64e5c7-dc5a-45c8-a17c-1b85f0bec97d",
          "dataset_name": "string",
          "dataset_version": "integer",
          "deleted_at": "2019-09-19T10:00:00.000Z",
          "description": "",
          "error": "string",
          "experiment": "my-pipeline",
          "metadata": {},
          "name": "My Experiment v1",
          "parent_experiment_id": "3fd6b5e0-8910-4b1c-a7d0-5b84de329012",
          "project_id": "a33671aa-24fd-4dcd-9b33-a8ec7dde7751",
          "run_count": "integer",
          "status": "completed",
          "updated_at": "2024-01-15T10:30:00Z"
        }
      ],
      "projects": [
        {
          "attributes": {
            "created_at": "2024-01-15T10:30:00Z",
            "description": "",
            "name": "My LLM Project",
            "updated_at": "2024-01-15T10:30:00Z"
          },
          "id": "a33671aa-24fd-4dcd-9b33-a8ec7dde7751",
          "type": "projects"
        }
      ]
    },
    "id": "00000000-0000-0000-0000-000000000001",
    "type": "experimentation"
  },
  "meta": {
    "after": "string"
  }
}

Bad Request

API error response.

Expand All

Field

Type

Description

errors [required]

[object]

A list of errors.

detail

string

A human-readable explanation specific to this occurrence of the error.

meta

object

Non-standard meta-information about the error

source

object

References to the source of the error.

header

string

A string indicating the name of a single request header which caused the error.

parameter

string

A string indicating which URI query parameter caused the error.

pointer

string

A JSON pointer to the value in the request document that caused the error.

status

string

Status code of the response.

title

string

Short human-readable summary of the error.

{
  "errors": [
    {
      "detail": "Missing required attribute in body",
      "meta": {},
      "source": {
        "header": "Authorization",
        "parameter": "limit",
        "pointer": "/data/attributes/title"
      },
      "status": "400",
      "title": "Bad Request"
    }
  ]
}

Unauthorized

API error response.

Expand All

Field

Type

Description

errors [required]

[object]

A list of errors.

detail

string

A human-readable explanation specific to this occurrence of the error.

meta

object

Non-standard meta-information about the error

source

object

References to the source of the error.

header

string

A string indicating the name of a single request header which caused the error.

parameter

string

A string indicating which URI query parameter caused the error.

pointer

string

A JSON pointer to the value in the request document that caused the error.

status

string

Status code of the response.

title

string

Short human-readable summary of the error.

{
  "errors": [
    {
      "detail": "Missing required attribute in body",
      "meta": {},
      "source": {
        "header": "Authorization",
        "parameter": "limit",
        "pointer": "/data/attributes/title"
      },
      "status": "400",
      "title": "Bad Request"
    }
  ]
}

Forbidden

API error response.

Expand All

Field

Type

Description

errors [required]

[object]

A list of errors.

detail

string

A human-readable explanation specific to this occurrence of the error.

meta

object

Non-standard meta-information about the error

source

object

References to the source of the error.

header

string

A string indicating the name of a single request header which caused the error.

parameter

string

A string indicating which URI query parameter caused the error.

pointer

string

A JSON pointer to the value in the request document that caused the error.

status

string

Status code of the response.

title

string

Short human-readable summary of the error.

{
  "errors": [
    {
      "detail": "Missing required attribute in body",
      "meta": {},
      "source": {
        "header": "Authorization",
        "parameter": "limit",
        "pointer": "/data/attributes/title"
      },
      "status": "400",
      "title": "Bad Request"
    }
  ]
}

Too many requests

API error response.

Expand All

Field

Type

Description

errors [required]

[string]

A list of errors.

{
  "errors": [
    "Bad Request"
  ]
}

Internal Server Error

API error response.

Expand All

Field

Type

Description

errors [required]

[object]

A list of errors.

detail

string

A human-readable explanation specific to this occurrence of the error.

meta

object

Non-standard meta-information about the error

source

object

References to the source of the error.

header

string

A string indicating the name of a single request header which caused the error.

parameter

string

A string indicating which URI query parameter caused the error.

pointer

string

A JSON pointer to the value in the request document that caused the error.

status

string

Status code of the response.

title

string

Short human-readable summary of the error.

{
  "errors": [
    {
      "detail": "Missing required attribute in body",
      "meta": {},
      "source": {
        "header": "Authorization",
        "parameter": "limit",
        "pointer": "/data/attributes/title"
      },
      "status": "400",
      "title": "Bad Request"
    }
  ]
}

Code Example

                  ## default
# 

# Curl command
curl -X POST "https://api.ap1.datadoghq.com"https://api.ap2.datadoghq.com"https://api.datadoghq.eu"https://api.ddog-gov.com"https://api.us2.ddog-gov.com"https://api.datadoghq.com"https://api.us3.datadoghq.com"https://api.us5.datadoghq.com/api/v2/llm-obs/v1/experimentation/search" \
-H "Accept: application/json" \
-H "Content-Type: application/json" \
-H "DD-API-KEY: ${DD_API_KEY}" \
-H "DD-APPLICATION-KEY: ${DD_APP_KEY}" \
-d @- << EOF
{
  "data": {
    "attributes": {
      "filter": {
        "query": "@project_id:a33671aa-24fd-4dcd-9b33-a8ec7dde7751",
        "scope": [
          "experiments"
        ]
      },
      "page": {
        "limit": 50
      }
    },
    "type": "experimentation"
  }
}
EOF