Language

English Français 日本語 한국어 Español

Datadog Site

Site help
US1 US3 US5 EU AP1 AP2 US1-FED US2-FED

List LLM Observability experiment events (v2)

Note: This endpoint is in preview and is subject to change. If you have any feedback, contact Datadog support.

GET https://api.ap1.datadoghq.com/api/v2/llm-obs/v2/experiments/{experiment_id}/eventshttps://api.ap2.datadoghq.com/api/v2/llm-obs/v2/experiments/{experiment_id}/eventshttps://api.datadoghq.eu/api/v2/llm-obs/v2/experiments/{experiment_id}/eventshttps://api.ddog-gov.com/api/v2/llm-obs/v2/experiments/{experiment_id}/eventshttps://api.us2.ddog-gov.com/api/v2/llm-obs/v2/experiments/{experiment_id}/eventshttps://api.datadoghq.com/api/v2/llm-obs/v2/experiments/{experiment_id}/eventshttps://api.us3.datadoghq.com/api/v2/llm-obs/v2/experiments/{experiment_id}/eventshttps://api.us5.datadoghq.com/api/v2/llm-obs/v2/experiments/{experiment_id}/events

Overview

Retrieve spans and experiment-level summary metrics for a given experiment. Returns the full events payload without pagination. Deprecated: use ListLLMObsExperimentEventsV3 instead.

Arguments

Path Parameters

Name

Type

Description

experiment_id [required]

string

The ID of the LLM Observability experiment.

Response

OK

Response for listing experiment events (v2/v3). Returns spans and summary metrics in a single resource.

Expand All

Field

Type

Description

data [required]

object

JSON:API data object for an experiment events response.

attributes [required]

object

Attributes of an experiment events response.

spans [required]

[object]

Experiment spans, each enriched with their associated evaluation metrics.

dataset_record_id

string

ID of the dataset record this span evaluated.

duration

double

Duration of the span in nanoseconds.

eval_metrics

[object]

Evaluation metrics associated with this span.

assessment

enum

Assessment result for an LLM Observability experiment metric. Allowed enum values: pass,fail

boolean_value

boolean

Boolean value. Present when metric_type is boolean.

categorical_value

string

Categorical value. Present when metric_type is categorical.

eval_source_type

string

Source type of the evaluation.

id

string

Unique identifier of the evaluation metric event.

json_value

object

JSON value. Present when metric_type is json.

label

string

Label or name for the metric.

metadata

object

Arbitrary metadata associated with the metric.

metric_source

string

Source of the metric. Either custom (user-submitted) or summary (experiment-level aggregate).

metric_type

enum

Type of metric recorded for an LLM Observability experiment. Allowed enum values: score,categorical,boolean,json

reasoning

string

Human-readable reasoning for the metric value.

score_value

double

Numeric score. Present when metric_type is score.

span_id

string

Span ID this metric is associated with.

tags

[string]

Tags associated with the metric.

timestamp_ms

int64

Timestamp when the metric was recorded, in milliseconds since Unix epoch.

trace_id

string

Trace ID linking this metric to a span.

id

string

Unique identifier of the span.

meta

object

Metadata associated with an experiment span.

error

object

Error details for an experiment span.

message

string

Error message.

stack

string

Stack trace of the error.

type

string

The error type or exception class name.

expected_output

object

Expected output for the span, used for evaluation.

input

object <oneOf>

Represents any valid JSON value.

Option 1

string

A scalar string value.

Option 2

double

A scalar numeric value.

Option 3

object

An arbitrary object value with additional properties.

Option 4

[ <oneOf>]

An array of arbitrary values.

Option 1

string

A scalar string value.

Option 2

double

A scalar numeric value.

Option 3

object

An arbitrary object value with additional properties.

Option 4

boolean

A scalar boolean value.

Option 5

boolean

A scalar boolean value.

output

object <oneOf>

Represents any valid JSON value.

Option 1

string

A scalar string value.

Option 2

double

A scalar numeric value.

Option 3

object

An arbitrary object value with additional properties.

Option 4

[ <oneOf>]

An array of arbitrary values.

Option 1

string

A scalar string value.

Option 2

double

A scalar numeric value.

Option 3

object

An arbitrary object value with additional properties.

Option 4

boolean

A scalar boolean value.

Option 5

boolean

A scalar boolean value.

metrics

object

Numeric metrics attached to the span.

<any-key>

double

name

string

Name of the span.

parent_id

string

Parent span ID, if any.

span_id

string

Span ID.

start_ns

int64

Start time in nanoseconds since Unix epoch.

status

enum

Status of the span. Allowed enum values: ok,error

tags

[string]

Tags associated with the span.

trace_id

string

Trace ID.

summary_metrics [required]

[object]

Experiment-level summary evaluation metrics (not tied to individual spans).

assessment

enum

Assessment result for an LLM Observability experiment metric. Allowed enum values: pass,fail

boolean_value

boolean

Boolean value. Present when metric_type is boolean.

categorical_value

string

Categorical value. Present when metric_type is categorical.

eval_source_type

string

Source type of the evaluation.

id

string

Unique identifier of the evaluation metric event.

json_value

object

JSON value. Present when metric_type is json.

label

string

Label or name for the metric.

metadata

object

Arbitrary metadata associated with the metric.

metric_source

string

Source of the metric. Either custom (user-submitted) or summary (experiment-level aggregate).

metric_type

enum

Type of metric recorded for an LLM Observability experiment. Allowed enum values: score,categorical,boolean,json

reasoning

string

Human-readable reasoning for the metric value.

score_value

double

Numeric score. Present when metric_type is score.

span_id

string

Span ID this metric is associated with.

tags

[string]

Tags associated with the metric.

timestamp_ms

int64

Timestamp when the metric was recorded, in milliseconds since Unix epoch.

trace_id

string

Trace ID linking this metric to a span.

id [required]

string

Identifier for this events resource.

type [required]

enum

Resource type for an experiment events collection. Allowed enum values: experiment_events

meta

object

Pagination cursor metadata.

after

string

Cursor for the next page of results.

{
  "data": {
    "attributes": {
      "spans": [
        {
          "dataset_record_id": "string",
          "duration": 1500000000,
          "eval_metrics": [
            {
              "assessment": "pass",
              "boolean_value": false,
              "categorical_value": "string",
              "eval_source_type": "managed",
              "id": "00000000-0000-0000-0000-000000000001",
              "json_value": {},
              "label": "faithfulness",
              "metadata": {},
              "metric_source": "custom",
              "metric_type": "score",
              "reasoning": "string",
              "score_value": "number",
              "span_id": "span-7a1b2c3d",
              "tags": [],
              "timestamp_ms": 1705314600000,
              "trace_id": "abc123def456"
            }
          ],
          "id": "00000000-0000-0000-0000-000000000001",
          "meta": {
            "error": {
              "message": "Model response timed out",
              "stack": "Traceback (most recent call last):\n  File \"main.py\", line 10, in <module>\n    response = model.generate(input)\n  File \"model.py\", line 45, in generate\n    raise TimeoutError(\"Model response timed out\")\nTimeoutError: Model response timed out",
              "type": "TimeoutError"
            },
            "expected_output": {},
            "input": {
              "description": "undefined",
              "type": "undefined"
            },
            "output": {
              "description": "undefined",
              "type": "undefined"
            }
          },
          "metrics": {
            "<any-key>": "number"
          },
          "name": "llm_call",
          "parent_id": "string",
          "span_id": "span-7a1b2c3d",
          "start_ns": 1705314600000000000,
          "status": "ok",
          "tags": [],
          "trace_id": "abc123def456"
        }
      ],
      "summary_metrics": [
        {
          "assessment": "pass",
          "boolean_value": false,
          "categorical_value": "string",
          "eval_source_type": "managed",
          "id": "00000000-0000-0000-0000-000000000001",
          "json_value": {},
          "label": "faithfulness",
          "metadata": {},
          "metric_source": "custom",
          "metric_type": "score",
          "reasoning": "string",
          "score_value": "number",
          "span_id": "span-7a1b2c3d",
          "tags": [],
          "timestamp_ms": 1705314600000,
          "trace_id": "abc123def456"
        }
      ]
    },
    "id": "3fd6b5e0-8910-4b1c-a7d0-5b84de329012",
    "type": "experiment_events"
  },
  "meta": {
    "after": "string"
  }
}

Bad Request

API error response.

Expand All

Field

Type

Description

errors [required]

[object]

A list of errors.

detail

string

A human-readable explanation specific to this occurrence of the error.

meta

object

Non-standard meta-information about the error

source

object

References to the source of the error.

header

string

A string indicating the name of a single request header which caused the error.

parameter

string

A string indicating which URI query parameter caused the error.

pointer

string

A JSON pointer to the value in the request document that caused the error.

status

string

Status code of the response.

title

string

Short human-readable summary of the error.

{
  "errors": [
    {
      "detail": "Missing required attribute in body",
      "meta": {},
      "source": {
        "header": "Authorization",
        "parameter": "limit",
        "pointer": "/data/attributes/title"
      },
      "status": "400",
      "title": "Bad Request"
    }
  ]
}

Unauthorized

API error response.

Expand All

Field

Type

Description

errors [required]

[object]

A list of errors.

detail

string

A human-readable explanation specific to this occurrence of the error.

meta

object

Non-standard meta-information about the error

source

object

References to the source of the error.

header

string

A string indicating the name of a single request header which caused the error.

parameter

string

A string indicating which URI query parameter caused the error.

pointer

string

A JSON pointer to the value in the request document that caused the error.

status

string

Status code of the response.

title

string

Short human-readable summary of the error.

{
  "errors": [
    {
      "detail": "Missing required attribute in body",
      "meta": {},
      "source": {
        "header": "Authorization",
        "parameter": "limit",
        "pointer": "/data/attributes/title"
      },
      "status": "400",
      "title": "Bad Request"
    }
  ]
}

Forbidden

API error response.

Expand All

Field

Type

Description

errors [required]

[object]

A list of errors.

detail

string

A human-readable explanation specific to this occurrence of the error.

meta

object

Non-standard meta-information about the error

source

object

References to the source of the error.

header

string

A string indicating the name of a single request header which caused the error.

parameter

string

A string indicating which URI query parameter caused the error.

pointer

string

A JSON pointer to the value in the request document that caused the error.

status

string

Status code of the response.

title

string

Short human-readable summary of the error.

{
  "errors": [
    {
      "detail": "Missing required attribute in body",
      "meta": {},
      "source": {
        "header": "Authorization",
        "parameter": "limit",
        "pointer": "/data/attributes/title"
      },
      "status": "400",
      "title": "Bad Request"
    }
  ]
}

Not Found

API error response.

Expand All

Field

Type

Description

errors [required]

[object]

A list of errors.

detail

string

A human-readable explanation specific to this occurrence of the error.

meta

object

Non-standard meta-information about the error

source

object

References to the source of the error.

header

string

A string indicating the name of a single request header which caused the error.

parameter

string

A string indicating which URI query parameter caused the error.

pointer

string

A JSON pointer to the value in the request document that caused the error.

status

string

Status code of the response.

title

string

Short human-readable summary of the error.

{
  "errors": [
    {
      "detail": "Missing required attribute in body",
      "meta": {},
      "source": {
        "header": "Authorization",
        "parameter": "limit",
        "pointer": "/data/attributes/title"
      },
      "status": "400",
      "title": "Bad Request"
    }
  ]
}

Too many requests

API error response.

Expand All

Field

Type

Description

errors [required]

[string]

A list of errors.

{
  "errors": [
    "Bad Request"
  ]
}

Internal Server Error

API error response.

Expand All

Field

Type

Description

errors [required]

[object]

A list of errors.

detail

string

A human-readable explanation specific to this occurrence of the error.

meta

object

Non-standard meta-information about the error

source

object

References to the source of the error.

header

string

A string indicating the name of a single request header which caused the error.

parameter

string

A string indicating which URI query parameter caused the error.

pointer

string

A JSON pointer to the value in the request document that caused the error.

status

string

Status code of the response.

title

string

Short human-readable summary of the error.

{
  "errors": [
    {
      "detail": "Missing required attribute in body",
      "meta": {},
      "source": {
        "header": "Authorization",
        "parameter": "limit",
        "pointer": "/data/attributes/title"
      },
      "status": "400",
      "title": "Bad Request"
    }
  ]
}

Code Example

                  # Path parameters
export experiment_id="3fd6b5e0-8910-4b1c-a7d0-5b84de329012"
# Curl command
curl -X GET "https://api.ap1.datadoghq.com"https://api.ap2.datadoghq.com"https://api.datadoghq.eu"https://api.ddog-gov.com"https://api.us2.ddog-gov.com"https://api.datadoghq.com"https://api.us3.datadoghq.com"https://api.us5.datadoghq.com/api/v2/llm-obs/v2/experiments/${experiment_id}/events" \
-H "Accept: application/json" \
-H "DD-API-KEY: ${DD_API_KEY}" \
-H "DD-APPLICATION-KEY: ${DD_APP_KEY}"