Language

English Français 日本語 한국어 Español

Datadog Site

Site help
US1 US3 US5 EU AP1 AP2 US1-FED US2-FED

Search LLM Observability spans

Note: This endpoint is in preview and is subject to change. If you have any feedback, contact Datadog support.

POST https://api.ap1.datadoghq.com/api/v2/llm-obs/v1/spans/events/searchhttps://api.ap2.datadoghq.com/api/v2/llm-obs/v1/spans/events/searchhttps://api.datadoghq.eu/api/v2/llm-obs/v1/spans/events/searchhttps://api.ddog-gov.com/api/v2/llm-obs/v1/spans/events/searchhttps://api.us2.ddog-gov.com/api/v2/llm-obs/v1/spans/events/searchhttps://api.datadoghq.com/api/v2/llm-obs/v1/spans/events/searchhttps://api.us3.datadoghq.com/api/v2/llm-obs/v1/spans/events/searchhttps://api.us5.datadoghq.com/api/v2/llm-obs/v1/spans/events/search

Overview

Search LLM Observability spans using structured filters in the request body.

Request

Body Data (required)

Search spans payload.

Expand All

Field

Type

Description

data [required]

object

Data object for an LLM Observability spans search request.

attributes [required]

object

Attributes of an LLM Observability spans search request.

filter

object

Filter criteria for an LLM Observability span search.

from

string

Start of the time range. Accepts ISO 8601 or relative format (e.g., now-15m). Defaults to now-15m.

ml_app

string

Filter by ML application name.

query

string

Search query using LLM Observability query syntax. Supports attribute filters using the field:value syntax (e.g. session_id, trace_id, ml_app, meta.span.kind). When provided, structured field filters (span_id, trace_id, etc.) are ignored.

span_id

string

Filter by exact span ID.

span_kind

string

Filter by span kind (e.g., llm, agent, tool, task, workflow).

span_name

string

Filter by span name.

tags

object

Filter by tag key-value pairs.

<any-key>

string

to

string

End of the time range. Accepts ISO 8601 or relative format (e.g., now). Defaults to now.

trace_id

string

Filter by exact trace ID.

options

object

Additional options for a span search request.

include_attachments

boolean

Whether to include attachment data in the response. Defaults to true.

time_offset

int64

Offset in seconds applied to both from and to timestamps.

page

object

Pagination settings for a span search request.

cursor

string

Cursor from the previous response to retrieve the next page.

limit

int64

Maximum number of spans to return. Defaults to 10.

sort

string

Sort order for the results. Use - prefix for descending order.

type [required]

enum

Resource type for an LLM Observability spans search request. Allowed enum values: spans

{
  "data": {
    "attributes": {
      "filter": {
        "from": "now-900s",
        "ml_app": "my-llm-app",
        "query": "@session_id:abc123def456",
        "span_id": "abc123def456",
        "span_kind": "llm",
        "span_name": "llm_call",
        "tags": {
          "<any-key>": "string"
        },
        "to": "now",
        "trace_id": "trace-9a8b7c6d5e4f"
      },
      "options": {
        "include_attachments": true,
        "time_offset": 0
      },
      "page": {
        "cursor": "eyJzdGFydCI6MTAwfQ==",
        "limit": 10
      },
      "sort": "-start_ns"
    },
    "type": "spans"
  }
}

Response

OK

Response containing a list of LLM Observability spans.

Expand All

Field

Type

Description

data [required]

[object]

List of spans matching the query.

attributes [required]

object

Attributes of an LLM Observability span.

duration [required]

double

Duration of the span in nanoseconds.

evaluation

object

Evaluation metrics keyed by evaluator name.

<any-key>

object

An evaluation metric associated with an LLM Observability span.

assessment

string

Assessment result (e.g., pass or fail).

eval_metric_type

string

Type of the evaluation metric (e.g., score, categorical, boolean).

reasoning

string

Human-readable reasoning for the evaluation result.

status

string

Status of the evaluation execution.

tags

[string]

Tags associated with the evaluation metric.

value

Value of the evaluation result.

input

object

Input or output content of an LLM Observability span.

messages

[object]

List of messages in the input or output.

content

string

Text content of the message.

id

string

Unique identifier of the message.

role

string

Role of the message sender (e.g., user, assistant, system).

tool_calls

[object]

Tool calls made in this message.

arguments

object

Arguments passed to the tool.

name

string

Name of the tool called.

tool_id

string

Identifier of the tool call.

type

string

Type of the tool call.

tool_results

[object]

Tool results returned in this message.

name

string

Name of the tool that produced this result.

result

string

Result value returned by the tool.

tool_id

string

Identifier of the corresponding tool call.

type

string

Type of the tool result.

value

string

Plain-text value of the input or output.

intent

string

Detected intent of the span.

metadata

object

Arbitrary metadata associated with the span.

metrics

object

Numeric metrics associated with the span (e.g., token counts).

<any-key>

double

ml_app [required]

string

Name of the ML application this span belongs to.

model_name

string

Name of the model used in this span.

model_provider

string

Provider of the model used in this span.

name [required]

string

Name of the span.

output

object

Input or output content of an LLM Observability span.

messages

[object]

List of messages in the input or output.

content

string

Text content of the message.

id

string

Unique identifier of the message.

role

string

Role of the message sender (e.g., user, assistant, system).

tool_calls

[object]

Tool calls made in this message.

arguments

object

Arguments passed to the tool.

name

string

Name of the tool called.

tool_id

string

Identifier of the tool call.

type

string

Type of the tool call.

tool_results

[object]

Tool results returned in this message.

name

string

Name of the tool that produced this result.

result

string

Result value returned by the tool.

tool_id

string

Identifier of the corresponding tool call.

type

string

Type of the tool result.

value

string

Plain-text value of the input or output.

parent_id

string

Identifier of the parent span, if any.

span_id [required]

string

Unique identifier of the span.

span_kind [required]

string

Kind of span (e.g., llm, agent, tool, task, workflow).

start_ns [required]

int64

Start time of the span in nanoseconds since Unix epoch.

status [required]

string

Status of the span (e.g., ok, error).

tags

[string]

Tags associated with the span.

tool_definitions

[object]

Tool definitions available to the span.

description

string

Description of what the tool does.

name

string

Name of the tool.

schema

object

JSON schema describing the tool's input parameters.

version

string

Version of the tool definition.

trace_id [required]

string

Trace identifier this span belongs to.

id [required]

string

Unique identifier of the span.

type [required]

enum

Resource type for an LLM Observability span. Allowed enum values: span

links

object

Pagination links accompanying the spans response.

next

string

URL to retrieve the next page of results.

meta [required]

object

Metadata accompanying the spans response.

elapsed [required]

int64

Time elapsed for the query in milliseconds.

page [required]

object

Pagination cursor for the spans response.

after

string

Cursor to retrieve the next page of results. Absent when there are no more results.

request_id [required]

string

Unique identifier for the request.

status [required]

string

Status of the query execution.

{
  "data": [
    {
      "attributes": {
        "duration": 1500000000,
        "evaluation": {
          "<any-key>": {
            "assessment": "pass",
            "eval_metric_type": "score",
            "reasoning": "string",
            "status": "string",
            "tags": [],
            "value": "undefined"
          }
        },
        "input": {
          "messages": [
            {
              "content": "string",
              "id": "string",
              "role": "string",
              "tool_calls": [
                {
                  "arguments": {},
                  "name": "string",
                  "tool_id": "string",
                  "type": "string"
                }
              ],
              "tool_results": [
                {
                  "name": "string",
                  "result": "string",
                  "tool_id": "string",
                  "type": "string"
                }
              ]
            }
          ],
          "value": "string"
        },
        "intent": "string",
        "metadata": {},
        "metrics": {
          "<any-key>": "number"
        },
        "ml_app": "my-llm-app",
        "model_name": "gpt-4o",
        "model_provider": "openai",
        "name": "llm_call",
        "output": {
          "messages": [
            {
              "content": "string",
              "id": "string",
              "role": "string",
              "tool_calls": [
                {
                  "arguments": {},
                  "name": "string",
                  "tool_id": "string",
                  "type": "string"
                }
              ],
              "tool_results": [
                {
                  "name": "string",
                  "result": "string",
                  "tool_id": "string",
                  "type": "string"
                }
              ]
            }
          ],
          "value": "string"
        },
        "parent_id": "string",
        "span_id": "abc123def456",
        "span_kind": "llm",
        "start_ns": 1705314600000000000,
        "status": "ok",
        "tags": [],
        "tool_definitions": [
          {
            "description": "string",
            "name": "string",
            "schema": {},
            "version": "string"
          }
        ],
        "trace_id": "trace-9a8b7c6d5e4f"
      },
      "id": "abc123def456",
      "type": "span"
    }
  ],
  "links": {
    "next": "https://api.datadoghq.com/api/v2/llm-obs/v1/spans/events?page[cursor]=eyJzdGFydCI6MTAwfQ=="
  },
  "meta": {
    "elapsed": 132,
    "page": {
      "after": "eyJzdGFydCI6MTAwfQ=="
    },
    "request_id": "req-abc123",
    "status": "done"
  }
}

Bad Request

API error response.

Expand All

Field

Type

Description

errors [required]

[object]

A list of errors.

detail

string

A human-readable explanation specific to this occurrence of the error.

meta

object

Non-standard meta-information about the error

source

object

References to the source of the error.

header

string

A string indicating the name of a single request header which caused the error.

parameter

string

A string indicating which URI query parameter caused the error.

pointer

string

A JSON pointer to the value in the request document that caused the error.

status

string

Status code of the response.

title

string

Short human-readable summary of the error.

{
  "errors": [
    {
      "detail": "Missing required attribute in body",
      "meta": {},
      "source": {
        "header": "Authorization",
        "parameter": "limit",
        "pointer": "/data/attributes/title"
      },
      "status": "400",
      "title": "Bad Request"
    }
  ]
}

Unauthorized

API error response.

Expand All

Field

Type

Description

errors [required]

[object]

A list of errors.

detail

string

A human-readable explanation specific to this occurrence of the error.

meta

object

Non-standard meta-information about the error

source

object

References to the source of the error.

header

string

A string indicating the name of a single request header which caused the error.

parameter

string

A string indicating which URI query parameter caused the error.

pointer

string

A JSON pointer to the value in the request document that caused the error.

status

string

Status code of the response.

title

string

Short human-readable summary of the error.

{
  "errors": [
    {
      "detail": "Missing required attribute in body",
      "meta": {},
      "source": {
        "header": "Authorization",
        "parameter": "limit",
        "pointer": "/data/attributes/title"
      },
      "status": "400",
      "title": "Bad Request"
    }
  ]
}

Forbidden

API error response.

Expand All

Field

Type

Description

errors [required]

[object]

A list of errors.

detail

string

A human-readable explanation specific to this occurrence of the error.

meta

object

Non-standard meta-information about the error

source

object

References to the source of the error.

header

string

A string indicating the name of a single request header which caused the error.

parameter

string

A string indicating which URI query parameter caused the error.

pointer

string

A JSON pointer to the value in the request document that caused the error.

status

string

Status code of the response.

title

string

Short human-readable summary of the error.

{
  "errors": [
    {
      "detail": "Missing required attribute in body",
      "meta": {},
      "source": {
        "header": "Authorization",
        "parameter": "limit",
        "pointer": "/data/attributes/title"
      },
      "status": "400",
      "title": "Bad Request"
    }
  ]
}

Too many requests

API error response.

Expand All

Field

Type

Description

errors [required]

[string]

A list of errors.

{
  "errors": [
    "Bad Request"
  ]
}

Code Example

                  ## Search spans for an ML app
# 

# Curl command
curl -X POST "https://api.ap1.datadoghq.com"https://api.ap2.datadoghq.com"https://api.datadoghq.eu"https://api.ddog-gov.com"https://api.us2.ddog-gov.com"https://api.datadoghq.com"https://api.us3.datadoghq.com"https://api.us5.datadoghq.com/api/v2/llm-obs/v1/spans/events/search" \
-H "Accept: application/json" \
-H "Content-Type: application/json" \
-H "DD-API-KEY: ${DD_API_KEY}" \
-H "DD-APPLICATION-KEY: ${DD_APP_KEY}" \
-d @- << EOF
{
  "data": {
    "attributes": {
      "filter": {
        "from": "now-900s",
        "ml_app": "my-llm-app",
        "span_kind": "llm",
        "to": "now"
      },
      "options": {
        "include_attachments": true
      },
      "page": {
        "limit": 10
      }
    },
    "type": "spans"
  }
}
EOF
## Search all spans in a session
# 

# Curl command
curl -X POST "https://api.ap1.datadoghq.com"https://api.ap2.datadoghq.com"https://api.datadoghq.eu"https://api.ddog-gov.com"https://api.us2.ddog-gov.com"https://api.datadoghq.com"https://api.us3.datadoghq.com"https://api.us5.datadoghq.com/api/v2/llm-obs/v1/spans/events/search" \
-H "Accept: application/json" \
-H "Content-Type: application/json" \
-H "DD-API-KEY: ${DD_API_KEY}" \
-H "DD-APPLICATION-KEY: ${DD_APP_KEY}" \
-d @- << EOF
{
  "data": {
    "attributes": {
      "filter": {
        "from": "now-900s",
        "query": "@session_id:abc123def456",
        "to": "now"
      },
      "options": {
        "include_attachments": true
      },
      "page": {
        "limit": 50
      }
    },
    "type": "spans"
  }
}
EOF