Agent Observability 실험 스팬 목록 나열(v1)

Note: This endpoint is in preview and is subject to change. If you have any feedback, contact Datadog support.

GET https://api.ap1.datadoghq.com/api/v2/llm-obs/v1/experiments/{experiment_id}/eventshttps://api.ap2.datadoghq.com/api/v2/llm-obs/v1/experiments/{experiment_id}/eventshttps://api.datadoghq.eu/api/v2/llm-obs/v1/experiments/{experiment_id}/eventshttps://api.ddog-gov.com/api/v2/llm-obs/v1/experiments/{experiment_id}/eventshttps://api.us2.ddog-gov.com/api/v2/llm-obs/v1/experiments/{experiment_id}/eventshttps://api.uk1.datadoghq.com/api/v2/llm-obs/v1/experiments/{experiment_id}/eventshttps://api.datadoghq.com/api/v2/llm-obs/v1/experiments/{experiment_id}/eventshttps://api.us3.datadoghq.com/api/v2/llm-obs/v1/experiments/{experiment_id}/eventshttps://api.us5.datadoghq.com/api/v2/llm-obs/v1/experiments/{experiment_id}/events

개요

Retrieve spans with their evaluation metrics for a given experiment. Returns spans only, with no summary metrics and no pagination. Deprecated in favor of ListLLMObsExperimentEventsV3.

인수

경로 파라미터

이름

유형

설명

experiment_id [required]

string

The ID of the Agent Observability experiment.

응답

OK

Response for listing experiment spans (v1). Returns only spans with their evaluation metrics. No summary metrics or pagination are included. Deprecated in favor of ListLLMObsExperimentEventsV3.

Expand All

항목

유형

설명

data [required]

[object]

List of experiment spans with their evaluation metrics.

attributes [required]

object

An experiment span enriched with its associated evaluation metrics.

dataset_record_id

string

ID of the dataset record this span evaluated.

duration

double

Duration of the span in nanoseconds.

eval_metrics

[object]

Evaluation metrics associated with this span.

assessment

enum

Assessment result for an Agent Observability experiment metric. Allowed enum values: pass,fail

boolean_value

boolean

Boolean value. Present when metric_type is boolean.

categorical_value

string

Categorical value. Present when metric_type is categorical.

eval_source_type

string

Source type of the evaluation.

id

string

Unique identifier of the evaluation metric event.

json_value

object

JSON value. Present when metric_type is json.

label

string

Label or name for the metric.

metadata

object

Arbitrary metadata associated with the metric.

metric_source

string

Source of the metric. Either custom (user-submitted) or summary (experiment-level aggregate).

metric_type

enum

Type of metric recorded for an Agent Observability experiment. Allowed enum values: score,categorical,boolean,json

reasoning

string

Human-readable reasoning for the metric value.

score_value

double

Numeric score. Present when metric_type is score.

span_id

string

Span ID this metric is associated with.

tags

[string]

Tags associated with the metric.

timestamp_ms

int64

Timestamp when the metric was recorded, in milliseconds since Unix epoch.

trace_id

string

Trace ID linking this metric to a span.

id

string

Unique identifier of the span.

meta

object

Metadata associated with an experiment span.

error

object

Error details for an experiment span.

message

string

Error message.

stack

string

Stack trace of the error.

type

string

The error type or exception class name.

expected_output

object

Expected output for the span, used for evaluation.

input

object <oneOf>

Represents any valid JSON value.

Object 1

string

A scalar value represented as a string.

Object 2

double

A scalar numeric value.

Object 3

object

An arbitrary object value with additional properties.

Object 4

[ <oneOf>]

An array of arbitrary values.

Object 1

string

A scalar value represented as a string.

Object 2

double

A scalar numeric value.

Object 3

object

An arbitrary object value with additional properties.

Object 4

boolean

A scalar boolean value.

Object 5

boolean

A scalar boolean value.

output

object <oneOf>

Represents any valid JSON value.

Object 1

string

A scalar value represented as a string.

Object 2

double

A scalar numeric value.

Object 3

object

An arbitrary object value with additional properties.

Object 4

[ <oneOf>]

An array of arbitrary values.

Object 1

string

A scalar value represented as a string.

Object 2

double

A scalar numeric value.

Object 3

object

An arbitrary object value with additional properties.

Object 4

boolean

A scalar boolean value.

Object 5

boolean

A scalar boolean value.

metrics

object

Numeric metrics attached to the span.

<any-key>

double

name

string

Name of the span.

parent_id

string

Parent span ID, if any.

span_id

string

Span ID.

start_ns

int64

Start time in nanoseconds since Unix epoch.

status

enum

Status of the span. Allowed enum values: ok,error

tags

[string]

Tags associated with the span.

trace_id

string

Trace ID.

id [required]

string

Unique identifier of the span.

type [required]

enum

Resource type for a span item in an experiment spans response. Allowed enum values: experiments

{
  "data": [
    {
      "attributes": {
        "dataset_record_id": "string",
        "duration": 1500000000,
        "eval_metrics": [
          {
            "assessment": "pass",
            "boolean_value": false,
            "categorical_value": "string",
            "eval_source_type": "managed",
            "id": "00000000-0000-0000-0000-000000000001",
            "json_value": {},
            "label": "faithfulness",
            "metadata": {},
            "metric_source": "custom",
            "metric_type": "score",
            "reasoning": "string",
            "score_value": "number",
            "span_id": "span-7a1b2c3d",
            "tags": [],
            "timestamp_ms": 1705314600000,
            "trace_id": "abc123def456"
          }
        ],
        "id": "00000000-0000-0000-0000-000000000001",
        "meta": {
          "error": {
            "message": "Model response timed out",
            "stack": "Traceback (most recent call last):\n  File \"main.py\", line 10, in <module>\n    response = model.generate(input)\n  File \"model.py\", line 45, in generate\n    raise TimeoutError(\"Model response timed out\")\nTimeoutError: Model response timed out",
            "type": "TimeoutError"
          },
          "expected_output": {},
          "input": {
            "description": "undefined",
            "type": "undefined"
          },
          "output": {
            "description": "undefined",
            "type": "undefined"
          }
        },
        "metrics": {
          "<any-key>": "number"
        },
        "name": "llm_call",
        "parent_id": "string",
        "span_id": "span-7a1b2c3d",
        "start_ns": 1705314600000000000,
        "status": "ok",
        "tags": [],
        "trace_id": "abc123def456"
      },
      "id": "00000000-0000-0000-0000-000000000001",
      "type": "experiments"
    }
  ]
}

Bad Request

API error response.

Expand All

항목

유형

설명

errors [required]

[object]

A list of errors.

detail

string

A human-readable explanation specific to this occurrence of the error.

meta

object

Non-standard meta-information about the error

source

object

References to the source of the error.

header

string

A string indicating the name of a single request header which caused the error.

parameter

string

A string indicating which URI query parameter caused the error.

pointer

string

A JSON pointer to the value in the request document that caused the error.

status

string

Status code of the response.

title

string

Short human-readable summary of the error.

{
  "errors": [
    {
      "detail": "Missing required attribute in body",
      "meta": {},
      "source": {
        "header": "Authorization",
        "parameter": "limit",
        "pointer": "/data/attributes/title"
      },
      "status": "400",
      "title": "Bad Request"
    }
  ]
}

Unauthorized

API error response.

Expand All

항목

유형

설명

errors [required]

[object]

A list of errors.

detail

string

A human-readable explanation specific to this occurrence of the error.

meta

object

Non-standard meta-information about the error

source

object

References to the source of the error.

header

string

A string indicating the name of a single request header which caused the error.

parameter

string

A string indicating which URI query parameter caused the error.

pointer

string

A JSON pointer to the value in the request document that caused the error.

status

string

Status code of the response.

title

string

Short human-readable summary of the error.

{
  "errors": [
    {
      "detail": "Missing required attribute in body",
      "meta": {},
      "source": {
        "header": "Authorization",
        "parameter": "limit",
        "pointer": "/data/attributes/title"
      },
      "status": "400",
      "title": "Bad Request"
    }
  ]
}

Forbidden

API error response.

Expand All

항목

유형

설명

errors [required]

[object]

A list of errors.

detail

string

A human-readable explanation specific to this occurrence of the error.

meta

object

Non-standard meta-information about the error

source

object

References to the source of the error.

header

string

A string indicating the name of a single request header which caused the error.

parameter

string

A string indicating which URI query parameter caused the error.

pointer

string

A JSON pointer to the value in the request document that caused the error.

status

string

Status code of the response.

title

string

Short human-readable summary of the error.

{
  "errors": [
    {
      "detail": "Missing required attribute in body",
      "meta": {},
      "source": {
        "header": "Authorization",
        "parameter": "limit",
        "pointer": "/data/attributes/title"
      },
      "status": "400",
      "title": "Bad Request"
    }
  ]
}

Not Found

API error response.

Expand All

항목

유형

설명

errors [required]

[object]

A list of errors.

detail

string

A human-readable explanation specific to this occurrence of the error.

meta

object

Non-standard meta-information about the error

source

object

References to the source of the error.

header

string

A string indicating the name of a single request header which caused the error.

parameter

string

A string indicating which URI query parameter caused the error.

pointer

string

A JSON pointer to the value in the request document that caused the error.

status

string

Status code of the response.

title

string

Short human-readable summary of the error.

{
  "errors": [
    {
      "detail": "Missing required attribute in body",
      "meta": {},
      "source": {
        "header": "Authorization",
        "parameter": "limit",
        "pointer": "/data/attributes/title"
      },
      "status": "400",
      "title": "Bad Request"
    }
  ]
}

Too many requests

API error response.

Expand All

항목

유형

설명

errors [required]

[string]

A list of errors.

{
  "errors": [
    "Bad Request"
  ]
}

Internal Server Error

API error response.

Expand All

항목

유형

설명

errors [required]

[object]

A list of errors.

detail

string

A human-readable explanation specific to this occurrence of the error.

meta

object

Non-standard meta-information about the error

source

object

References to the source of the error.

header

string

A string indicating the name of a single request header which caused the error.

parameter

string

A string indicating which URI query parameter caused the error.

pointer

string

A JSON pointer to the value in the request document that caused the error.

status

string

Status code of the response.

title

string

Short human-readable summary of the error.

{
  "errors": [
    {
      "detail": "Missing required attribute in body",
      "meta": {},
      "source": {
        "header": "Authorization",
        "parameter": "limit",
        "pointer": "/data/attributes/title"
      },
      "status": "400",
      "title": "Bad Request"
    }
  ]
}

코드 예시

                  # Path parameters
export experiment_id="3fd6b5e0-8910-4b1c-a7d0-5b84de329012"
# Curl command
curl -X GET "https://api.ap1.datadoghq.com"https://api.ap2.datadoghq.com"https://api.datadoghq.eu"https://api.ddog-gov.com"https://api.us2.ddog-gov.com"https://api.uk1.datadoghq.com"https://api.datadoghq.com"https://api.us3.datadoghq.com"https://api.us5.datadoghq.com/api/v2/llm-obs/v1/experiments/${experiment_id}/events" \ -H "Accept: application/json" \ -H "DD-API-KEY: ${DD_API_KEY}" \ -H "DD-APPLICATION-KEY: ${DD_APP_KEY}"
"""
List Agent Observability experiment spans (v1) returns "OK" response
"""

from datadog_api_client import ApiClient, Configuration
from datadog_api_client.v2.api.agent_observability_api import AgentObservabilityApi

configuration = Configuration()
configuration.unstable_operations["list_llm_obs_experiment_events_v1"] = True
with ApiClient(configuration) as api_client:
    api_instance = AgentObservabilityApi(api_client)
    response = api_instance.list_llm_obs_experiment_events_v1(
        experiment_id="experiment_id",
    )

    print(response)

Instructions

First install the library and its dependencies and then save the example to example.py and run following commands:

    
DD_SITE="datadoghq.comus3.datadoghq.comus5.datadoghq.comdatadoghq.euap1.datadoghq.comap2.datadoghq.comuk1.datadoghq.comddog-gov.comus2.ddog-gov.com" DD_API_KEY="<DD_API_KEY>" DD_APP_KEY="<DD_APP_KEY>" python3 "example.py"
# List Agent Observability experiment spans (v1) returns "OK" response

require "datadog_api_client"
DatadogAPIClient.configure do |config|
  config.unstable_operations["v2.list_llm_obs_experiment_events_v1".to_sym] = true
end
api_instance = DatadogAPIClient::V2::AgentObservabilityAPI.new
p api_instance.list_llm_obs_experiment_events_v1("experiment_id")

Instructions

First install the library and its dependencies and then save the example to example.rb and run following commands:

    
DD_SITE="datadoghq.comus3.datadoghq.comus5.datadoghq.comdatadoghq.euap1.datadoghq.comap2.datadoghq.comuk1.datadoghq.comddog-gov.comus2.ddog-gov.com" DD_API_KEY="<DD_API_KEY>" DD_APP_KEY="<DD_APP_KEY>" rb "example.rb"
// List Agent Observability experiment spans (v1) returns "OK" response

package main

import (
	"context"
	"encoding/json"
	"fmt"
	"os"

	"github.com/DataDog/datadog-api-client-go/v2/api/datadog"
	"github.com/DataDog/datadog-api-client-go/v2/api/datadogV2"
)

func main() {
	ctx := datadog.NewDefaultContext(context.Background())
	configuration := datadog.NewConfiguration()
	configuration.SetUnstableOperationEnabled("v2.ListLLMObsExperimentEventsV1", true)
	apiClient := datadog.NewAPIClient(configuration)
	api := datadogV2.NewAgentObservabilityApi(apiClient)
	resp, r, err := api.ListLLMObsExperimentEventsV1(ctx, "experiment_id")

	if err != nil {
		fmt.Fprintf(os.Stderr, "Error when calling `AgentObservabilityApi.ListLLMObsExperimentEventsV1`: %v\n", err)
		fmt.Fprintf(os.Stderr, "Full HTTP response: %v\n", r)
	}

	responseContent, _ := json.MarshalIndent(resp, "", "  ")
	fmt.Fprintf(os.Stdout, "Response from `AgentObservabilityApi.ListLLMObsExperimentEventsV1`:\n%s\n", responseContent)
}

Instructions

First install the library and its dependencies and then save the example to main.go and run following commands:

    
DD_SITE="datadoghq.comus3.datadoghq.comus5.datadoghq.comdatadoghq.euap1.datadoghq.comap2.datadoghq.comuk1.datadoghq.comddog-gov.comus2.ddog-gov.com" DD_API_KEY="<DD_API_KEY>" DD_APP_KEY="<DD_APP_KEY>" go run "main.go"
// List Agent Observability experiment spans (v1) returns "OK" response

import com.datadog.api.client.ApiClient;
import com.datadog.api.client.ApiException;
import com.datadog.api.client.v2.api.AgentObservabilityApi;
import com.datadog.api.client.v2.model.LLMObsExperimentSpansResponse;

public class Example {
  public static void main(String[] args) {
    ApiClient defaultClient = ApiClient.getDefaultApiClient();
    defaultClient.setUnstableOperationEnabled("v2.listLLMObsExperimentEventsV1", true);
    AgentObservabilityApi apiInstance = new AgentObservabilityApi(defaultClient);

    try {
      LLMObsExperimentSpansResponse result =
          apiInstance.listLLMObsExperimentEventsV1("3fd6b5e0-8910-4b1c-a7d0-5b84de329012");
      System.out.println(result);
    } catch (ApiException e) {
      System.err.println(
          "Exception when calling AgentObservabilityApi#listLLMObsExperimentEventsV1");
      System.err.println("Status code: " + e.getCode());
      System.err.println("Reason: " + e.getResponseBody());
      System.err.println("Response headers: " + e.getResponseHeaders());
      e.printStackTrace();
    }
  }
}

Instructions

First install the library and its dependencies and then save the example to Example.java and run following commands:

    
DD_SITE="datadoghq.comus3.datadoghq.comus5.datadoghq.comdatadoghq.euap1.datadoghq.comap2.datadoghq.comuk1.datadoghq.comddog-gov.comus2.ddog-gov.com" DD_API_KEY="<DD_API_KEY>" DD_APP_KEY="<DD_APP_KEY>" java "Example.java"
// List Agent Observability experiment spans (v1) returns "OK" response
use datadog_api_client::datadog;
use datadog_api_client::datadogV2::api_agent_observability::AgentObservabilityAPI;

#[tokio::main]
async fn main() {
    let mut configuration = datadog::Configuration::new();
    configuration.set_unstable_operation_enabled("v2.ListLLMObsExperimentEventsV1", true);
    let api = AgentObservabilityAPI::with_config(configuration);
    let resp = api
        .list_llm_obs_experiment_events_v1("experiment_id".to_string())
        .await;
    if let Ok(value) = resp {
        println!("{:#?}", value);
    } else {
        println!("{:#?}", resp.unwrap_err());
    }
}

Instructions

First install the library and its dependencies and then save the example to src/main.rs and run following commands:

    
DD_SITE="datadoghq.comus3.datadoghq.comus5.datadoghq.comdatadoghq.euap1.datadoghq.comap2.datadoghq.comuk1.datadoghq.comddog-gov.comus2.ddog-gov.com" DD_API_KEY="<DD_API_KEY>" DD_APP_KEY="<DD_APP_KEY>" cargo run
/**
 * List Agent Observability experiment spans (v1) returns "OK" response
 */

import { client, v2 } from "@datadog/datadog-api-client";

const configuration = client.createConfiguration();
configuration.unstableOperations["v2.listLLMObsExperimentEventsV1"] = true;
const apiInstance = new v2.AgentObservabilityApi(configuration);

const params: v2.AgentObservabilityApiListLLMObsExperimentEventsV1Request = {
  experimentId: "experiment_id",
};

apiInstance
  .listLLMObsExperimentEventsV1(params)
  .then((data: v2.LLMObsExperimentSpansResponse) => {
    console.log(
      "API called successfully. Returned data: " + JSON.stringify(data)
    );
  })
  .catch((error: any) => console.error(error));

Instructions

First install the library and its dependencies and then save the example to example.ts and run following commands:

    
DD_SITE="datadoghq.comus3.datadoghq.comus5.datadoghq.comdatadoghq.euap1.datadoghq.comap2.datadoghq.comuk1.datadoghq.comddog-gov.comus2.ddog-gov.com" DD_API_KEY="<DD_API_KEY>" DD_APP_KEY="<DD_APP_KEY>" tsc "example.ts"