--- title: Get annotated queue interactions description: Datadog, the leading service for cloud-scale monitoring. breadcrumbs: Docs > API Reference > LLM Observability --- # Get annotated queue interactions{% #get-annotated-queue-interactions %} {% tab title="v2" %} **Note**: This endpoint is in preview and is subject to change. If you have any feedback, contact [Datadog support](https://docs.datadoghq.com/help/). | Datadog site | API endpoint | | ----------------- | ------------------------------------------------------------------------------------------------------- | | ap1.datadoghq.com | GET https://api.ap1.datadoghq.com/api/v2/llm-obs/v1/annotation-queues/{queue_id}/annotated-interactions | | ap2.datadoghq.com | GET https://api.ap2.datadoghq.com/api/v2/llm-obs/v1/annotation-queues/{queue_id}/annotated-interactions | | app.datadoghq.eu | GET https://api.datadoghq.eu/api/v2/llm-obs/v1/annotation-queues/{queue_id}/annotated-interactions | | app.ddog-gov.com | GET https://api.ddog-gov.com/api/v2/llm-obs/v1/annotation-queues/{queue_id}/annotated-interactions | | us2.ddog-gov.com | GET https://api.us2.ddog-gov.com/api/v2/llm-obs/v1/annotation-queues/{queue_id}/annotated-interactions | | app.datadoghq.com | GET https://api.datadoghq.com/api/v2/llm-obs/v1/annotation-queues/{queue_id}/annotated-interactions | | us3.datadoghq.com | GET https://api.us3.datadoghq.com/api/v2/llm-obs/v1/annotation-queues/{queue_id}/annotated-interactions | | us5.datadoghq.com | GET https://api.us5.datadoghq.com/api/v2/llm-obs/v1/annotation-queues/{queue_id}/annotated-interactions | ### Overview Retrieve all interactions (traces and sessions) and their annotations for a given annotation queue. ### Arguments #### Path Parameters | Name | Type | Description | | -------------------------- | ------ | ------------------------------------------------- | | queue_id [*required*] | string | The ID of the LLM Observability annotation queue. | ### Response {% tab title="200" %} OK {% tab title="Model" %} Response containing the annotated interactions for an annotation queue. | Parent field | Field | Type | Description | | ---------------------- | ---------------------------------------- | --------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | | | data [*required*] | object | Data object for annotated interactions. | | data | attributes [*required*] | object | Attributes containing the list of annotated interactions. | | attributes | annotated_interactions [*required*] | [ ] | List of interactions with their annotations. | | annotated_interactions | Option 1 | object | A trace, experiment trace, or session interaction with its associated annotations. | | Option 1 | annotations [*required*] | [object] | List of annotations for this interaction. | | annotations | created_at [*required*] | date-time | Timestamp when the annotation was created. | | annotations | created_by [*required*] | string | Identifier of the user who created the annotation. | | annotations | id [*required*] | string | Unique identifier of the annotation. | | annotations | interaction_id [*required*] | string | Identifier of the interaction this annotation belongs to. | | annotations | label_values [*required*] | object | The label values for this annotation. | | annotations | modified_at [*required*] | date-time | Timestamp when the annotation was last modified. | | annotations | modified_by [*required*] | string | Identifier of the user who last modified the annotation. | | Option 1 | content_id [*required*] | string | Upstream entity identifier supplied by the caller. | | Option 1 | created_at [*required*] | date-time | Timestamp when the interaction was added to the queue. | | Option 1 | id [*required*] | string | Unique identifier of the interaction. | | Option 1 | modified_at [*required*] | date-time | Timestamp when the interaction was last updated. | | Option 1 | type [*required*] | enum | Type of an upstream-entity interaction. Allowed enum values: `trace,experiment_trace,session` | | annotated_interactions | Option 2 | object | A display_block interaction with its associated annotations. | | Option 2 | annotations [*required*] | [object] | List of annotations for this interaction. | | annotations | created_at [*required*] | date-time | Timestamp when the annotation was created. | | annotations | created_by [*required*] | string | Identifier of the user who created the annotation. | | annotations | id [*required*] | string | Unique identifier of the annotation. | | annotations | interaction_id [*required*] | string | Identifier of the interaction this annotation belongs to. | | annotations | label_values [*required*] | object | The label values for this annotation. | | annotations | modified_at [*required*] | date-time | Timestamp when the annotation was last modified. | | annotations | modified_by [*required*] | string | Identifier of the user who last modified the annotation. | | Option 2 | content_id [*required*] | string | Server-generated deterministic identifier derived from the block list. | | Option 2 | display_block [*required*] | [object] | List of content blocks that make up a `display_block` interaction. Must contain at least one block. | | display_block | alt | string | Alternative text for an `image` block. | | display_block | content | | Block payload. A string for `markdown`, `header`, and `text`; an arbitrary JSON value (object, array, or scalar) for `json`. Omitted for `image`, `widget`, and `llmobs_trace`. | | display_block | height | int64 | Optional rendered height. Must be positive when set. | | display_block | interactionType | enum | Upstream interaction type referenced by an `llmobs_trace` block. Restricted to `trace` or `experiment_trace`. Allowed enum values: `trace,experiment_trace` | | display_block | label | string | Optional label rendered alongside the block. | | display_block | level | enum | Visual size for a `header` block. Allowed enum values: `sm,md,lg,xl` | | display_block | tileDef | | Tile definition for a `widget` block. Required for `widget`. The schema is owned by the frontend renderer. | | display_block | timeFrame | object | Unix-millis time range used by chart blocks. | | timeFrame | end [*required*] | int64 | End of the range, in Unix milliseconds. | | timeFrame | start [*required*] | int64 | Start of the range, in Unix milliseconds. | | display_block | traceId | string | Trace identifier. Required for `llmobs_trace` blocks. | | display_block | type [*required*] | enum | Discriminator for a single `display_block` content block. Adding a variant requires coordinated changes in the frontend renderer. Allowed enum values: `markdown,header,text,json,image,widget,llmobs_trace` | | display_block | url | string | URL of the image. Required for `image` blocks. | | Option 2 | id [*required*] | string | Unique identifier of the interaction. | | Option 2 | type [*required*] | enum | Type discriminator for a `display_block` interaction. Allowed enum values: `display_block` | | data | id [*required*] | string | The queue ID. | | data | type [*required*] | enum | Resource type for annotated interactions. Allowed enum values: `annotated_interactions` | {% /tab %} {% tab title="Example" %} ```json { "data": { "attributes": { "annotated_interactions": [ { "annotations": [ { "created_at": "2024-01-15T10:30:00Z", "created_by": "00000000-0000-0000-0000-000000000002", "id": "annotation-789", "interaction_id": "interaction-456", "label_values": { "quality": "good" }, "modified_at": "2024-01-15T10:30:00Z", "modified_by": "00000000-0000-0000-0000-000000000002" } ], "content_id": "trace-abc-123", "created_at": "2024-01-15T10:30:00Z", "id": "interaction-456", "modified_at": "2024-01-15T10:30:00Z", "type": "trace" } ] }, "id": "00000000-0000-0000-0000-000000000001", "type": "annotated_interactions" } } ``` {% /tab %} {% /tab %} {% tab title="400" %} Bad Request {% tab title="Model" %} API error response. | Parent field | Field | Type | Description | | ------------ | ------------------------ | -------- | ------------------------------------------------------------------------------- | | | errors [*required*] | [object] | A list of errors. | | errors | detail | string | A human-readable explanation specific to this occurrence of the error. | | errors | meta | object | Non-standard meta-information about the error | | errors | source | object | References to the source of the error. | | source | header | string | A string indicating the name of a single request header which caused the error. | | source | parameter | string | A string indicating which URI query parameter caused the error. | | source | pointer | string | A JSON pointer to the value in the request document that caused the error. | | errors | status | string | Status code of the response. | | errors | title | string | Short human-readable summary of the error. | {% /tab %} {% tab title="Example" %} ```json { "errors": [ { "detail": "Missing required attribute in body", "meta": {}, "source": { "header": "Authorization", "parameter": "limit", "pointer": "/data/attributes/title" }, "status": "400", "title": "Bad Request" } ] } ``` {% /tab %} {% /tab %} {% tab title="401" %} Unauthorized {% tab title="Model" %} API error response. | Parent field | Field | Type | Description | | ------------ | ------------------------ | -------- | ------------------------------------------------------------------------------- | | | errors [*required*] | [object] | A list of errors. | | errors | detail | string | A human-readable explanation specific to this occurrence of the error. | | errors | meta | object | Non-standard meta-information about the error | | errors | source | object | References to the source of the error. | | source | header | string | A string indicating the name of a single request header which caused the error. | | source | parameter | string | A string indicating which URI query parameter caused the error. | | source | pointer | string | A JSON pointer to the value in the request document that caused the error. | | errors | status | string | Status code of the response. | | errors | title | string | Short human-readable summary of the error. | {% /tab %} {% tab title="Example" %} ```json { "errors": [ { "detail": "Missing required attribute in body", "meta": {}, "source": { "header": "Authorization", "parameter": "limit", "pointer": "/data/attributes/title" }, "status": "400", "title": "Bad Request" } ] } ``` {% /tab %} {% /tab %} {% tab title="403" %} Forbidden {% tab title="Model" %} API error response. | Parent field | Field | Type | Description | | ------------ | ------------------------ | -------- | ------------------------------------------------------------------------------- | | | errors [*required*] | [object] | A list of errors. | | errors | detail | string | A human-readable explanation specific to this occurrence of the error. | | errors | meta | object | Non-standard meta-information about the error | | errors | source | object | References to the source of the error. | | source | header | string | A string indicating the name of a single request header which caused the error. | | source | parameter | string | A string indicating which URI query parameter caused the error. | | source | pointer | string | A JSON pointer to the value in the request document that caused the error. | | errors | status | string | Status code of the response. | | errors | title | string | Short human-readable summary of the error. | {% /tab %} {% tab title="Example" %} ```json { "errors": [ { "detail": "Missing required attribute in body", "meta": {}, "source": { "header": "Authorization", "parameter": "limit", "pointer": "/data/attributes/title" }, "status": "400", "title": "Bad Request" } ] } ``` {% /tab %} {% /tab %} {% tab title="404" %} Not Found {% tab title="Model" %} API error response. | Parent field | Field | Type | Description | | ------------ | ------------------------ | -------- | ------------------------------------------------------------------------------- | | | errors [*required*] | [object] | A list of errors. | | errors | detail | string | A human-readable explanation specific to this occurrence of the error. | | errors | meta | object | Non-standard meta-information about the error | | errors | source | object | References to the source of the error. | | source | header | string | A string indicating the name of a single request header which caused the error. | | source | parameter | string | A string indicating which URI query parameter caused the error. | | source | pointer | string | A JSON pointer to the value in the request document that caused the error. | | errors | status | string | Status code of the response. | | errors | title | string | Short human-readable summary of the error. | {% /tab %} {% tab title="Example" %} ```json { "errors": [ { "detail": "Missing required attribute in body", "meta": {}, "source": { "header": "Authorization", "parameter": "limit", "pointer": "/data/attributes/title" }, "status": "400", "title": "Bad Request" } ] } ``` {% /tab %} {% /tab %} {% tab title="429" %} Too many requests {% tab title="Model" %} API error response. | Field | Type | Description | | ------------------------ | -------- | ----------------- | | errors [*required*] | [string] | A list of errors. | {% /tab %} {% tab title="Example" %} ```json { "errors": [ "Bad Request" ] } ``` {% /tab %} {% /tab %} ### Code Example ##### \# Path parameters export queue_id="00000000-0000-0000-0000-000000000001" \# Curl command curl -X GET "https://api.datadoghq.com/api/v2/llm-obs/v1/annotation-queues/${queue_id}/annotated-interactions" \ -H "Accept: application/json" \ -H "DD-API-KEY: ${DD_API_KEY}" \ -H "DD-APPLICATION-KEY: ${DD_APP_KEY}" ##### ```python """ Get annotated queue interactions returns "OK" response """ from datadog_api_client import ApiClient, Configuration from datadog_api_client.v2.api.llm_observability_api import LLMObservabilityApi configuration = Configuration() configuration.unstable_operations["get_llm_obs_annotated_interactions"] = True with ApiClient(configuration) as api_client: api_instance = LLMObservabilityApi(api_client) response = api_instance.get_llm_obs_annotated_interactions( queue_id="queue_id", ) print(response) ``` #### Instructions First [install the library and its dependencies](https://docs.datadoghq.com/api/latest.md?code-lang=python) and then save the example to `example.py` and run following commands: DD_SITE="datadoghq.com" DD_API_KEY="" DD_APP_KEY="" python3 "example.py" ##### ```ruby # Get annotated queue interactions returns "OK" response require "datadog_api_client" DatadogAPIClient.configure do |config| config.unstable_operations["v2.get_llm_obs_annotated_interactions".to_sym] = true end api_instance = DatadogAPIClient::V2::LLMObservabilityAPI.new p api_instance.get_llm_obs_annotated_interactions("queue_id") ``` #### Instructions First [install the library and its dependencies](https://docs.datadoghq.com/api/latest.md?code-lang=ruby) and then save the example to `example.rb` and run following commands: DD_SITE="datadoghq.com" DD_API_KEY="" DD_APP_KEY="" rb "example.rb" ##### ```go // Get annotated queue interactions returns "OK" response package main import ( "context" "encoding/json" "fmt" "os" "github.com/DataDog/datadog-api-client-go/v2/api/datadog" "github.com/DataDog/datadog-api-client-go/v2/api/datadogV2" ) func main() { ctx := datadog.NewDefaultContext(context.Background()) configuration := datadog.NewConfiguration() configuration.SetUnstableOperationEnabled("v2.GetLLMObsAnnotatedInteractions", true) apiClient := datadog.NewAPIClient(configuration) api := datadogV2.NewLLMObservabilityApi(apiClient) resp, r, err := api.GetLLMObsAnnotatedInteractions(ctx, "queue_id") if err != nil { fmt.Fprintf(os.Stderr, "Error when calling `LLMObservabilityApi.GetLLMObsAnnotatedInteractions`: %v\n", err) fmt.Fprintf(os.Stderr, "Full HTTP response: %v\n", r) } responseContent, _ := json.MarshalIndent(resp, "", " ") fmt.Fprintf(os.Stdout, "Response from `LLMObservabilityApi.GetLLMObsAnnotatedInteractions`:\n%s\n", responseContent) } ``` #### Instructions First [install the library and its dependencies](https://docs.datadoghq.com/api/latest.md?code-lang=go) and then save the example to `main.go` and run following commands: DD_SITE="datadoghq.com" DD_API_KEY="" DD_APP_KEY="" go run "main.go" ##### ```java // Get annotated queue interactions returns "OK" response import com.datadog.api.client.ApiClient; import com.datadog.api.client.ApiException; import com.datadog.api.client.v2.api.LlmObservabilityApi; import com.datadog.api.client.v2.model.LLMObsAnnotatedInteractionsResponse; public class Example { public static void main(String[] args) { ApiClient defaultClient = ApiClient.getDefaultApiClient(); defaultClient.setUnstableOperationEnabled("v2.getLLMObsAnnotatedInteractions", true); LlmObservabilityApi apiInstance = new LlmObservabilityApi(defaultClient); try { LLMObsAnnotatedInteractionsResponse result = apiInstance.getLLMObsAnnotatedInteractions("00000000-0000-0000-0000-000000000001"); System.out.println(result); } catch (ApiException e) { System.err.println( "Exception when calling LlmObservabilityApi#getLLMObsAnnotatedInteractions"); System.err.println("Status code: " + e.getCode()); System.err.println("Reason: " + e.getResponseBody()); System.err.println("Response headers: " + e.getResponseHeaders()); e.printStackTrace(); } } } ``` #### Instructions First [install the library and its dependencies](https://docs.datadoghq.com/api/latest.md?code-lang=java) and then save the example to `Example.java` and run following commands: DD_SITE="datadoghq.com" DD_API_KEY="" DD_APP_KEY="" java "Example.java" ##### ```rust // Get annotated queue interactions returns "OK" response use datadog_api_client::datadog; use datadog_api_client::datadogV2::api_llm_observability::LLMObservabilityAPI; #[tokio::main] async fn main() { let mut configuration = datadog::Configuration::new(); configuration.set_unstable_operation_enabled("v2.GetLLMObsAnnotatedInteractions", true); let api = LLMObservabilityAPI::with_config(configuration); let resp = api .get_llm_obs_annotated_interactions("queue_id".to_string()) .await; if let Ok(value) = resp { println!("{:#?}", value); } else { println!("{:#?}", resp.unwrap_err()); } } ``` #### Instructions First [install the library and its dependencies](https://docs.datadoghq.com/api/latest.md?code-lang=rust) and then save the example to `src/main.rs` and run following commands: DD_SITE="datadoghq.com" DD_API_KEY="" DD_APP_KEY="" cargo run ##### ```typescript /** * Get annotated queue interactions returns "OK" response */ import { client, v2 } from "@datadog/datadog-api-client"; const configuration = client.createConfiguration(); configuration.unstableOperations["v2.getLLMObsAnnotatedInteractions"] = true; const apiInstance = new v2.LLMObservabilityApi(configuration); const params: v2.LLMObservabilityApiGetLLMObsAnnotatedInteractionsRequest = { queueId: "queue_id", }; apiInstance .getLLMObsAnnotatedInteractions(params) .then((data: v2.LLMObsAnnotatedInteractionsResponse) => { console.log( "API called successfully. Returned data: " + JSON.stringify(data) ); }) .catch((error: any) => console.error(error)); ``` #### Instructions First [install the library and its dependencies](https://docs.datadoghq.com/api/latest.md?code-lang=typescript) and then save the example to `example.ts` and run following commands: DD_SITE="datadoghq.com" DD_API_KEY="" DD_APP_KEY="" tsc "example.ts" {% /tab %}