Get annotated queue interactions

Note: This endpoint is in preview and is subject to change. If you have any feedback, contact Datadog support.

GET https://api.ap1.datadoghq.com/api/v2/llm-obs/v1/annotation-queues/{queue_id}/annotated-interactionshttps://api.ap2.datadoghq.com/api/v2/llm-obs/v1/annotation-queues/{queue_id}/annotated-interactionshttps://api.datadoghq.eu/api/v2/llm-obs/v1/annotation-queues/{queue_id}/annotated-interactionshttps://api.ddog-gov.com/api/v2/llm-obs/v1/annotation-queues/{queue_id}/annotated-interactionshttps://api.us2.ddog-gov.com/api/v2/llm-obs/v1/annotation-queues/{queue_id}/annotated-interactionshttps://api.datadoghq.com/api/v2/llm-obs/v1/annotation-queues/{queue_id}/annotated-interactionshttps://api.us3.datadoghq.com/api/v2/llm-obs/v1/annotation-queues/{queue_id}/annotated-interactionshttps://api.us5.datadoghq.com/api/v2/llm-obs/v1/annotation-queues/{queue_id}/annotated-interactions

Overview

Retrieve all interactions (traces and sessions) and their annotations for a given annotation queue.

Arguments

Path Parameters

Name

Type

Description

queue_id [required]

string

The ID of the LLM Observability annotation queue.

Response

OK

Response containing the annotated interactions for an annotation queue.

Expand All

Field

Type

Description

data [required]

object

Data object for annotated interactions.

attributes [required]

object

Attributes containing the list of annotated interactions.

annotated_interactions [required]

[ <oneOf>]

List of interactions with their annotations.

Option 1

object

A trace, experiment trace, or session interaction with its associated annotations.

annotations [required]

[object]

List of annotations for this interaction.

created_at [required]

date-time

Timestamp when the annotation was created.

created_by [required]

string

Identifier of the user who created the annotation.

id [required]

string

Unique identifier of the annotation.

interaction_id [required]

string

Identifier of the interaction this annotation belongs to.

label_values [required]

object

Label values for this annotation.

modified_at [required]

date-time

Timestamp when the annotation was last modified.

modified_by [required]

string

Identifier of the user who last modified the annotation.

content_id [required]

string

Upstream entity identifier supplied by the caller.

created_at [required]

date-time

Timestamp when the interaction was added to the queue.

id [required]

string

Unique identifier of the interaction.

modified_at [required]

date-time

Timestamp when the interaction was last updated.

type [required]

enum

Type of an upstream-entity interaction. Allowed enum values: trace,experiment_trace,session

Option 2

object

A display_block interaction with its associated annotations.

annotations [required]

[object]

List of annotations for this interaction.

created_at [required]

date-time

Timestamp when the annotation was created.

created_by [required]

string

Identifier of the user who created the annotation.

id [required]

string

Unique identifier of the annotation.

interaction_id [required]

string

Identifier of the interaction this annotation belongs to.

label_values [required]

object

Label values for this annotation.

modified_at [required]

date-time

Timestamp when the annotation was last modified.

modified_by [required]

string

Identifier of the user who last modified the annotation.

content_id [required]

string

Server-generated deterministic identifier derived from the block list.

display_block [required]

[object]

List of content blocks that make up a display_block interaction. Must contain at least one block.

alt

string

Alternative text for an image block.

content

Block payload. A string for markdown, header, and text; an arbitrary JSON value (object, array, or scalar) for json. Omitted for image, widget, and llmobs_trace.

height

int64

Optional rendered height. Must be positive when set.

interactionType

enum

Upstream interaction type referenced by an llmobs_trace block. Restricted to trace or experiment_trace. Allowed enum values: trace,experiment_trace

label

string

Optional label rendered alongside the block.

level

enum

Visual size for a header block. Allowed enum values: sm,md,lg,xl

tileDef

Tile definition for a widget block. Required for widget. The schema is owned by the frontend renderer.

timeFrame

object

Unix-millis time range used by chart blocks.

end [required]

int64

End of the range, in Unix milliseconds.

start [required]

int64

Start of the range, in Unix milliseconds.

traceId

string

Trace identifier. Required for llmobs_trace blocks.

type [required]

enum

Discriminator for a single display_block content block. Adding a variant requires coordinated changes in the frontend renderer. Allowed enum values: markdown,header,text,json,image,widget,llmobs_trace

url

string

URL of the image. Required for image blocks.

id [required]

string

Unique identifier of the interaction.

type [required]

enum

Type discriminator for a display_block interaction. Allowed enum values: display_block

id [required]

string

The annotation queue ID.

type [required]

enum

Resource type for annotated interactions. Allowed enum values: annotated_interactions

{
  "data": {
    "attributes": {
      "annotated_interactions": [
        {
          "annotations": [
            {
              "created_at": "2024-01-15T10:30:00Z",
              "created_by": "00000000-0000-0000-0000-000000000002",
              "id": "annotation-789",
              "interaction_id": "interaction-456",
              "label_values": [
                {}
              ],
              "modified_at": "2024-01-15T10:30:00Z",
              "modified_by": "00000000-0000-0000-0000-000000000002"
            }
          ],
          "content_id": "trace-abc-123",
          "created_at": "2024-01-15T10:30:00Z",
          "id": "interaction-456",
          "modified_at": "2024-01-15T10:30:00Z",
          "type": "trace"
        }
      ]
    },
    "id": "00000000-0000-0000-0000-000000000001",
    "type": "annotated_interactions"
  }
}

Bad Request

API error response.

Expand All

Field

Type

Description

errors [required]

[object]

A list of errors.

detail

string

A human-readable explanation specific to this occurrence of the error.

meta

object

Non-standard meta-information about the error

source

object

References to the source of the error.

header

string

A string indicating the name of a single request header which caused the error.

parameter

string

A string indicating which URI query parameter caused the error.

pointer

string

A JSON pointer to the value in the request document that caused the error.

status

string

Status code of the response.

title

string

Short human-readable summary of the error.

{
  "errors": [
    {
      "detail": "Missing required attribute in body",
      "meta": {},
      "source": {
        "header": "Authorization",
        "parameter": "limit",
        "pointer": "/data/attributes/title"
      },
      "status": "400",
      "title": "Bad Request"
    }
  ]
}

Unauthorized

API error response.

Expand All

Field

Type

Description

errors [required]

[object]

A list of errors.

detail

string

A human-readable explanation specific to this occurrence of the error.

meta

object

Non-standard meta-information about the error

source

object

References to the source of the error.

header

string

A string indicating the name of a single request header which caused the error.

parameter

string

A string indicating which URI query parameter caused the error.

pointer

string

A JSON pointer to the value in the request document that caused the error.

status

string

Status code of the response.

title

string

Short human-readable summary of the error.

{
  "errors": [
    {
      "detail": "Missing required attribute in body",
      "meta": {},
      "source": {
        "header": "Authorization",
        "parameter": "limit",
        "pointer": "/data/attributes/title"
      },
      "status": "400",
      "title": "Bad Request"
    }
  ]
}

Forbidden

API error response.

Expand All

Field

Type

Description

errors [required]

[object]

A list of errors.

detail

string

A human-readable explanation specific to this occurrence of the error.

meta

object

Non-standard meta-information about the error

source

object

References to the source of the error.

header

string

A string indicating the name of a single request header which caused the error.

parameter

string

A string indicating which URI query parameter caused the error.

pointer

string

A JSON pointer to the value in the request document that caused the error.

status

string

Status code of the response.

title

string

Short human-readable summary of the error.

{
  "errors": [
    {
      "detail": "Missing required attribute in body",
      "meta": {},
      "source": {
        "header": "Authorization",
        "parameter": "limit",
        "pointer": "/data/attributes/title"
      },
      "status": "400",
      "title": "Bad Request"
    }
  ]
}

Not Found

API error response.

Expand All

Field

Type

Description

errors [required]

[object]

A list of errors.

detail

string

A human-readable explanation specific to this occurrence of the error.

meta

object

Non-standard meta-information about the error

source

object

References to the source of the error.

header

string

A string indicating the name of a single request header which caused the error.

parameter

string

A string indicating which URI query parameter caused the error.

pointer

string

A JSON pointer to the value in the request document that caused the error.

status

string

Status code of the response.

title

string

Short human-readable summary of the error.

{
  "errors": [
    {
      "detail": "Missing required attribute in body",
      "meta": {},
      "source": {
        "header": "Authorization",
        "parameter": "limit",
        "pointer": "/data/attributes/title"
      },
      "status": "400",
      "title": "Bad Request"
    }
  ]
}

Too many requests

API error response.

Expand All

Field

Type

Description

errors [required]

[string]

A list of errors.

{
  "errors": [
    "Bad Request"
  ]
}

Code Example

                  # Path parameters
export queue_id="00000000-0000-0000-0000-000000000001"
# Curl command
curl -X GET "https://api.ap1.datadoghq.com"https://api.ap2.datadoghq.com"https://api.datadoghq.eu"https://api.ddog-gov.com"https://api.us2.ddog-gov.com"https://api.datadoghq.com"https://api.us3.datadoghq.com"https://api.us5.datadoghq.com/api/v2/llm-obs/v1/annotation-queues/${queue_id}/annotated-interactions" \ -H "Accept: application/json" \ -H "DD-API-KEY: ${DD_API_KEY}" \ -H "DD-APPLICATION-KEY: ${DD_APP_KEY}"
"""
Get annotated queue interactions returns "OK" response
"""

from datadog_api_client import ApiClient, Configuration
from datadog_api_client.v2.api.llm_observability_api import LLMObservabilityApi

configuration = Configuration()
configuration.unstable_operations["get_llm_obs_annotated_interactions"] = True
with ApiClient(configuration) as api_client:
    api_instance = LLMObservabilityApi(api_client)
    response = api_instance.get_llm_obs_annotated_interactions(
        queue_id="queue_id",
    )

    print(response)

Instructions

First install the library and its dependencies and then save the example to example.py and run following commands:

    
DD_SITE="datadoghq.comus3.datadoghq.comus5.datadoghq.comdatadoghq.euap1.datadoghq.comap2.datadoghq.comddog-gov.comus2.ddog-gov.com" DD_API_KEY="<DD_API_KEY>" DD_APP_KEY="<DD_APP_KEY>" python3 "example.py"
# Get annotated queue interactions returns "OK" response

require "datadog_api_client"
DatadogAPIClient.configure do |config|
  config.unstable_operations["v2.get_llm_obs_annotated_interactions".to_sym] = true
end
api_instance = DatadogAPIClient::V2::LLMObservabilityAPI.new
p api_instance.get_llm_obs_annotated_interactions("queue_id")

Instructions

First install the library and its dependencies and then save the example to example.rb and run following commands:

    
DD_SITE="datadoghq.comus3.datadoghq.comus5.datadoghq.comdatadoghq.euap1.datadoghq.comap2.datadoghq.comddog-gov.comus2.ddog-gov.com" DD_API_KEY="<DD_API_KEY>" DD_APP_KEY="<DD_APP_KEY>" rb "example.rb"
// Get annotated queue interactions returns "OK" response

package main

import (
	"context"
	"encoding/json"
	"fmt"
	"os"

	"github.com/DataDog/datadog-api-client-go/v2/api/datadog"
	"github.com/DataDog/datadog-api-client-go/v2/api/datadogV2"
)

func main() {
	ctx := datadog.NewDefaultContext(context.Background())
	configuration := datadog.NewConfiguration()
	configuration.SetUnstableOperationEnabled("v2.GetLLMObsAnnotatedInteractions", true)
	apiClient := datadog.NewAPIClient(configuration)
	api := datadogV2.NewLLMObservabilityApi(apiClient)
	resp, r, err := api.GetLLMObsAnnotatedInteractions(ctx, "queue_id")

	if err != nil {
		fmt.Fprintf(os.Stderr, "Error when calling `LLMObservabilityApi.GetLLMObsAnnotatedInteractions`: %v\n", err)
		fmt.Fprintf(os.Stderr, "Full HTTP response: %v\n", r)
	}

	responseContent, _ := json.MarshalIndent(resp, "", "  ")
	fmt.Fprintf(os.Stdout, "Response from `LLMObservabilityApi.GetLLMObsAnnotatedInteractions`:\n%s\n", responseContent)
}

Instructions

First install the library and its dependencies and then save the example to main.go and run following commands:

    
DD_SITE="datadoghq.comus3.datadoghq.comus5.datadoghq.comdatadoghq.euap1.datadoghq.comap2.datadoghq.comddog-gov.comus2.ddog-gov.com" DD_API_KEY="<DD_API_KEY>" DD_APP_KEY="<DD_APP_KEY>" go run "main.go"
// Get annotated queue interactions returns "OK" response

import com.datadog.api.client.ApiClient;
import com.datadog.api.client.ApiException;
import com.datadog.api.client.v2.api.LlmObservabilityApi;
import com.datadog.api.client.v2.model.LLMObsAnnotatedInteractionsResponse;

public class Example {
  public static void main(String[] args) {
    ApiClient defaultClient = ApiClient.getDefaultApiClient();
    defaultClient.setUnstableOperationEnabled("v2.getLLMObsAnnotatedInteractions", true);
    LlmObservabilityApi apiInstance = new LlmObservabilityApi(defaultClient);

    try {
      LLMObsAnnotatedInteractionsResponse result =
          apiInstance.getLLMObsAnnotatedInteractions("00000000-0000-0000-0000-000000000001");
      System.out.println(result);
    } catch (ApiException e) {
      System.err.println(
          "Exception when calling LlmObservabilityApi#getLLMObsAnnotatedInteractions");
      System.err.println("Status code: " + e.getCode());
      System.err.println("Reason: " + e.getResponseBody());
      System.err.println("Response headers: " + e.getResponseHeaders());
      e.printStackTrace();
    }
  }
}

Instructions

First install the library and its dependencies and then save the example to Example.java and run following commands:

    
DD_SITE="datadoghq.comus3.datadoghq.comus5.datadoghq.comdatadoghq.euap1.datadoghq.comap2.datadoghq.comddog-gov.comus2.ddog-gov.com" DD_API_KEY="<DD_API_KEY>" DD_APP_KEY="<DD_APP_KEY>" java "Example.java"
// Get annotated queue interactions returns "OK" response
use datadog_api_client::datadog;
use datadog_api_client::datadogV2::api_llm_observability::LLMObservabilityAPI;

#[tokio::main]
async fn main() {
    let mut configuration = datadog::Configuration::new();
    configuration.set_unstable_operation_enabled("v2.GetLLMObsAnnotatedInteractions", true);
    let api = LLMObservabilityAPI::with_config(configuration);
    let resp = api
        .get_llm_obs_annotated_interactions("queue_id".to_string())
        .await;
    if let Ok(value) = resp {
        println!("{:#?}", value);
    } else {
        println!("{:#?}", resp.unwrap_err());
    }
}

Instructions

First install the library and its dependencies and then save the example to src/main.rs and run following commands:

    
DD_SITE="datadoghq.comus3.datadoghq.comus5.datadoghq.comdatadoghq.euap1.datadoghq.comap2.datadoghq.comddog-gov.comus2.ddog-gov.com" DD_API_KEY="<DD_API_KEY>" DD_APP_KEY="<DD_APP_KEY>" cargo run
/**
 * Get annotated queue interactions returns "OK" response
 */

import { client, v2 } from "@datadog/datadog-api-client";

const configuration = client.createConfiguration();
configuration.unstableOperations["v2.getLLMObsAnnotatedInteractions"] = true;
const apiInstance = new v2.LLMObservabilityApi(configuration);

const params: v2.LLMObservabilityApiGetLLMObsAnnotatedInteractionsRequest = {
  queueId: "queue_id",
};

apiInstance
  .getLLMObsAnnotatedInteractions(params)
  .then((data: v2.LLMObsAnnotatedInteractionsResponse) => {
    console.log(
      "API called successfully. Returned data: " + JSON.stringify(data)
    );
  })
  .catch((error: any) => console.error(error));

Instructions

First install the library and its dependencies and then save the example to example.ts and run following commands:

    
DD_SITE="datadoghq.comus3.datadoghq.comus5.datadoghq.comdatadoghq.euap1.datadoghq.comap2.datadoghq.comddog-gov.comus2.ddog-gov.com" DD_API_KEY="<DD_API_KEY>" DD_APP_KEY="<DD_APP_KEY>" tsc "example.ts"