Add annotation queue interactions

v2 (latest)

Note: This endpoint is in preview and is subject to change. If you have any feedback, contact Datadog support.

POST https://api.ap1.datadoghq.com/api/v2/llm-obs/v1/annotation-queues/{queue_id}/interactionshttps://api.ap2.datadoghq.com/api/v2/llm-obs/v1/annotation-queues/{queue_id}/interactionshttps://api.datadoghq.eu/api/v2/llm-obs/v1/annotation-queues/{queue_id}/interactionshttps://api.ddog-gov.com/api/v2/llm-obs/v1/annotation-queues/{queue_id}/interactionshttps://api.us2.ddog-gov.com/api/v2/llm-obs/v1/annotation-queues/{queue_id}/interactionshttps://api.datadoghq.com/api/v2/llm-obs/v1/annotation-queues/{queue_id}/interactionshttps://api.us3.datadoghq.com/api/v2/llm-obs/v1/annotation-queues/{queue_id}/interactionshttps://api.us5.datadoghq.com/api/v2/llm-obs/v1/annotation-queues/{queue_id}/interactions

Overview

Add one or more interactions to an annotation queue. At least one interaction must be provided. Each interaction has a type:

trace, experiment_trace, session: content_id references the upstream entity; the server fetches the actual content.
display_block: omit content_id and provide the rendered content in display_block. The server generates content_id as a deterministic hash of the block list.

Items of different types can be mixed in a single request.

Arguments

Path Parameters

Name

Type

Description

queue_id [required]

string

The ID of the LLM Observability annotation queue.

Request

Body Data (required)

Add interactions payload.

Expand All

Field

Type

Description

data [required]

object

Data object for adding interactions to an annotation queue.

attributes [required]

object

Attributes for adding interactions to an annotation queue.

interactions [required]

[ <oneOf>]

List of interactions to add to the queue. Must contain at least one item.

{
  "data": {
    "attributes": {
      "interactions": [
        {
          "content_id": "trace-abc-123",
          "type": "trace"
        }
      ]
    },
    "type": "interactions"
  }
}

Response

Created

Response containing the result of adding interactions to an annotation queue.

Expand All

Field

Type

Description

data [required]

object

Data object for the interaction addition response.

attributes [required]

object

Attributes of the interaction addition response.

interactions [required]

[ <oneOf>]

List of interactions that were processed.

{
  "data": {
    "attributes": {
      "interactions": [
        {
          "already_existed": false,
          "content_id": "trace-abc-123",
          "created_at": "2024-01-15T10:30:00Z",
          "id": "00000000-0000-0000-0000-000000000000",
          "modified_at": "2024-01-15T10:30:00Z",
          "type": "trace"
        }
      ]
    },
    "id": "00000000-0000-0000-0000-000000000001",
    "type": "interactions"
  }
}

Bad Request

API error response.

Expand All

Field

Type

Description

errors [required]

[object]

A list of errors.

{
  "errors": [
    {
      "detail": "Missing required attribute in body",
      "meta": {},
      "source": {
        "header": "Authorization",
        "parameter": "limit",
        "pointer": "/data/attributes/title"
      },
      "status": "400",
      "title": "Bad Request"
    }
  ]
}

Unauthorized

API error response.

Expand All

Field

Type

Description

errors [required]

[object]

A list of errors.

{
  "errors": [
    {
      "detail": "Missing required attribute in body",
      "meta": {},
      "source": {
        "header": "Authorization",
        "parameter": "limit",
        "pointer": "/data/attributes/title"
      },
      "status": "400",
      "title": "Bad Request"
    }
  ]
}

Forbidden

API error response.

Expand All

Field

Type

Description

errors [required]

[object]

A list of errors.

{
  "errors": [
    {
      "detail": "Missing required attribute in body",
      "meta": {},
      "source": {
        "header": "Authorization",
        "parameter": "limit",
        "pointer": "/data/attributes/title"
      },
      "status": "400",
      "title": "Bad Request"
    }
  ]
}

Not Found

API error response.

Expand All

Field

Type

Description

errors [required]

[object]

A list of errors.

{
  "errors": [
    {
      "detail": "Missing required attribute in body",
      "meta": {},
      "source": {
        "header": "Authorization",
        "parameter": "limit",
        "pointer": "/data/attributes/title"
      },
      "status": "400",
      "title": "Bad Request"
    }
  ]
}

Too many requests

API error response.

Expand All

Field

Type

Description

errors [required]

[string]

A list of errors.

{
  "errors": [
    "Bad Request"
  ]
}

Code Example

                  ## default
# 

# Path parameters
export queue_id="00000000-0000-0000-0000-000000000001"
# Curl command
curl -X POST "https://api.ap1.datadoghq.com"https://api.ap2.datadoghq.com"https://api.datadoghq.eu"https://api.ddog-gov.com"https://api.us2.ddog-gov.com"https://api.datadoghq.com"https://api.us3.datadoghq.com"https://api.us5.datadoghq.com/api/v2/llm-obs/v1/annotation-queues/${queue_id}/interactions" \
-H "Accept: application/json" \
-H "Content-Type: application/json" \
-H "DD-API-KEY: ${DD_API_KEY}" \
-H "DD-APPLICATION-KEY: ${DD_APP_KEY}" \
-d @- << EOF
{
  "data": {
    "attributes": {
      "interactions": [
        {
          "content_id": "trace-abc-123",
          "type": "trace"
        }
      ]
    },
    "type": "interactions"
  }
}
EOF
## Add a display_block interaction
# 

# Path parameters
export queue_id="00000000-0000-0000-0000-000000000001"
# Curl command
curl -X POST "https://api.ap1.datadoghq.com"https://api.ap2.datadoghq.com"https://api.datadoghq.eu"https://api.ddog-gov.com"https://api.us2.ddog-gov.com"https://api.datadoghq.com"https://api.us3.datadoghq.com"https://api.us5.datadoghq.com/api/v2/llm-obs/v1/annotation-queues/${queue_id}/interactions" \
-H "Accept: application/json" \
-H "Content-Type: application/json" \
-H "DD-API-KEY: ${DD_API_KEY}" \
-H "DD-APPLICATION-KEY: ${DD_APP_KEY}" \
-d @- << EOF
{
  "data": {
    "attributes": {
      "interactions": [
        {
          "display_block": [
            {
              "content": "## Triage Instructions",
              "type": "markdown"
            },
            {
              "content": "Inputs",
              "level": "md",
              "type": "header"
            },
            {
              "content": {
                "experiment_id": "abc-123"
              },
              "label": "Experiments",
              "type": "json"
            }
          ],
          "type": "display_block"
        }
      ]
    },
    "type": "interactions"
  }
}
EOF

                

"""
Add annotation queue interactions returns "Created" response
"""

from datadog_api_client import ApiClient, Configuration
from datadog_api_client.v2.api.llm_observability_api import LLMObservabilityApi
from datadog_api_client.v2.model.llm_obs_annotation_queue_interaction_item import LLMObsAnnotationQueueInteractionItem
from datadog_api_client.v2.model.llm_obs_annotation_queue_interactions_data_attributes_request import (
    LLMObsAnnotationQueueInteractionsDataAttributesRequest,
)
from datadog_api_client.v2.model.llm_obs_annotation_queue_interactions_data_request import (
    LLMObsAnnotationQueueInteractionsDataRequest,
)
from datadog_api_client.v2.model.llm_obs_annotation_queue_interactions_request import (
    LLMObsAnnotationQueueInteractionsRequest,
)
from datadog_api_client.v2.model.llm_obs_annotation_queue_interactions_type import LLMObsAnnotationQueueInteractionsType
from datadog_api_client.v2.model.llm_obs_interaction_type import LLMObsInteractionType

body = LLMObsAnnotationQueueInteractionsRequest(
    data=LLMObsAnnotationQueueInteractionsDataRequest(
        attributes=LLMObsAnnotationQueueInteractionsDataAttributesRequest(
            interactions=[
                LLMObsAnnotationQueueInteractionItem(
                    content_id="trace-abc-123",
                    type=LLMObsInteractionType.TRACE,
                ),
            ],
        ),
        type=LLMObsAnnotationQueueInteractionsType.INTERACTIONS,
    ),
)

configuration = Configuration()
configuration.unstable_operations["create_llm_obs_annotation_queue_interactions"] = True
with ApiClient(configuration) as api_client:
    api_instance = LLMObservabilityApi(api_client)
    response = api_instance.create_llm_obs_annotation_queue_interactions(queue_id="queue_id", body=body)

    print(response)

Instructions

First install the library and its dependencies and then save the example to example.py and run following commands:

    DD_SITE="datadoghq.comus3.datadoghq.comus5.datadoghq.comdatadoghq.euap1.datadoghq.comap2.datadoghq.comddog-gov.comus2.ddog-gov.com" DD_API_KEY="<DD_API_KEY>" DD_APP_KEY="<DD_APP_KEY>" python3 "example.py"

# Add annotation queue interactions returns "Created" response

require "datadog_api_client"
DatadogAPIClient.configure do |config|
  config.unstable_operations["v2.create_llm_obs_annotation_queue_interactions".to_sym] = true
end
api_instance = DatadogAPIClient::V2::LLMObservabilityAPI.new

body = DatadogAPIClient::V2::LLMObsAnnotationQueueInteractionsRequest.new({
  data: DatadogAPIClient::V2::LLMObsAnnotationQueueInteractionsDataRequest.new({
    attributes: DatadogAPIClient::V2::LLMObsAnnotationQueueInteractionsDataAttributesRequest.new({
      interactions: [
        DatadogAPIClient::V2::LLMObsAnnotationQueueInteractionItem.new({
          content_id: "trace-abc-123",
          type: DatadogAPIClient::V2::LLMObsInteractionType::TRACE,
        }),
      ],
    }),
    type: DatadogAPIClient::V2::LLMObsAnnotationQueueInteractionsType::INTERACTIONS,
  }),
})
p api_instance.create_llm_obs_annotation_queue_interactions("queue_id", body)

Instructions

First install the library and its dependencies and then save the example to example.rb and run following commands:

    DD_SITE="datadoghq.comus3.datadoghq.comus5.datadoghq.comdatadoghq.euap1.datadoghq.comap2.datadoghq.comddog-gov.comus2.ddog-gov.com" DD_API_KEY="<DD_API_KEY>" DD_APP_KEY="<DD_APP_KEY>" rb "example.rb"

// Add annotation queue interactions returns "Created" response

package main

import (
	"context"
	"encoding/json"
	"fmt"
	"os"

	"github.com/DataDog/datadog-api-client-go/v2/api/datadog"
	"github.com/DataDog/datadog-api-client-go/v2/api/datadogV2"
)

func main() {
	body := datadogV2.LLMObsAnnotationQueueInteractionsRequest{
		Data: datadogV2.LLMObsAnnotationQueueInteractionsDataRequest{
			Attributes: datadogV2.LLMObsAnnotationQueueInteractionsDataAttributesRequest{
				Interactions: []datadogV2.LLMObsAnnotationQueueInteractionItem{
					{
						ContentId: "trace-abc-123",
						Type:      datadogV2.LLMOBSINTERACTIONTYPE_TRACE,
					},
				},
			},
			Type: datadogV2.LLMOBSANNOTATIONQUEUEINTERACTIONSTYPE_INTERACTIONS,
		},
	}
	ctx := datadog.NewDefaultContext(context.Background())
	configuration := datadog.NewConfiguration()
	configuration.SetUnstableOperationEnabled("v2.CreateLLMObsAnnotationQueueInteractions", true)
	apiClient := datadog.NewAPIClient(configuration)
	api := datadogV2.NewLLMObservabilityApi(apiClient)
	resp, r, err := api.CreateLLMObsAnnotationQueueInteractions(ctx, "queue_id", body)

	if err != nil {
		fmt.Fprintf(os.Stderr, "Error when calling `LLMObservabilityApi.CreateLLMObsAnnotationQueueInteractions`: %v\n", err)
		fmt.Fprintf(os.Stderr, "Full HTTP response: %v\n", r)
	}

	responseContent, _ := json.MarshalIndent(resp, "", "  ")
	fmt.Fprintf(os.Stdout, "Response from `LLMObservabilityApi.CreateLLMObsAnnotationQueueInteractions`:\n%s\n", responseContent)
}

Instructions

First install the library and its dependencies and then save the example to main.go and run following commands:

    DD_SITE="datadoghq.comus3.datadoghq.comus5.datadoghq.comdatadoghq.euap1.datadoghq.comap2.datadoghq.comddog-gov.comus2.ddog-gov.com" DD_API_KEY="<DD_API_KEY>" DD_APP_KEY="<DD_APP_KEY>" go run "main.go"

// Add annotation queue interactions returns "Created" response

import com.datadog.api.client.ApiClient;
import com.datadog.api.client.ApiException;
import com.datadog.api.client.v2.api.LlmObservabilityApi;
import com.datadog.api.client.v2.model.LLMObsAnnotationQueueInteractionItem;
import com.datadog.api.client.v2.model.LLMObsAnnotationQueueInteractionsDataAttributesRequest;
import com.datadog.api.client.v2.model.LLMObsAnnotationQueueInteractionsDataRequest;
import com.datadog.api.client.v2.model.LLMObsAnnotationQueueInteractionsRequest;
import com.datadog.api.client.v2.model.LLMObsAnnotationQueueInteractionsResponse;
import com.datadog.api.client.v2.model.LLMObsAnnotationQueueInteractionsType;
import com.datadog.api.client.v2.model.LLMObsInteractionType;
import java.util.Collections;

public class Example {
  public static void main(String[] args) {
    ApiClient defaultClient = ApiClient.getDefaultApiClient();
    defaultClient.setUnstableOperationEnabled("v2.createLLMObsAnnotationQueueInteractions", true);
    LlmObservabilityApi apiInstance = new LlmObservabilityApi(defaultClient);

    LLMObsAnnotationQueueInteractionsRequest body =
        new LLMObsAnnotationQueueInteractionsRequest()
            .data(
                new LLMObsAnnotationQueueInteractionsDataRequest()
                    .attributes(
                        new LLMObsAnnotationQueueInteractionsDataAttributesRequest()
                            .interactions(
                                Collections.singletonList(
                                    new LLMObsAnnotationQueueInteractionItem()
                                        .contentId("trace-abc-123")
                                        .type(LLMObsInteractionType.TRACE))))
                    .type(LLMObsAnnotationQueueInteractionsType.INTERACTIONS));

    try {
      LLMObsAnnotationQueueInteractionsResponse result =
          apiInstance.createLLMObsAnnotationQueueInteractions(
              "00000000-0000-0000-0000-000000000001", body);
      System.out.println(result);
    } catch (ApiException e) {
      System.err.println(
          "Exception when calling LlmObservabilityApi#createLLMObsAnnotationQueueInteractions");
      System.err.println("Status code: " + e.getCode());
      System.err.println("Reason: " + e.getResponseBody());
      System.err.println("Response headers: " + e.getResponseHeaders());
      e.printStackTrace();
    }
  }
}

Instructions

First install the library and its dependencies and then save the example to Example.java and run following commands:

    DD_SITE="datadoghq.comus3.datadoghq.comus5.datadoghq.comdatadoghq.euap1.datadoghq.comap2.datadoghq.comddog-gov.comus2.ddog-gov.com" DD_API_KEY="<DD_API_KEY>" DD_APP_KEY="<DD_APP_KEY>" java "Example.java"

// Add annotation queue interactions returns "Created" response
use datadog_api_client::datadog;
use datadog_api_client::datadogV2::api_llm_observability::LLMObservabilityAPI;
use datadog_api_client::datadogV2::model::LLMObsAnnotationQueueInteractionItem;
use datadog_api_client::datadogV2::model::LLMObsAnnotationQueueInteractionsDataAttributesRequest;
use datadog_api_client::datadogV2::model::LLMObsAnnotationQueueInteractionsDataRequest;
use datadog_api_client::datadogV2::model::LLMObsAnnotationQueueInteractionsRequest;
use datadog_api_client::datadogV2::model::LLMObsAnnotationQueueInteractionsType;
use datadog_api_client::datadogV2::model::LLMObsInteractionType;

#[tokio::main]
async fn main() {
    let body = LLMObsAnnotationQueueInteractionsRequest::new(
        LLMObsAnnotationQueueInteractionsDataRequest::new(
            LLMObsAnnotationQueueInteractionsDataAttributesRequest::new(vec![
                LLMObsAnnotationQueueInteractionItem::new(
                    "trace-abc-123".to_string(),
                    LLMObsInteractionType::TRACE,
                ),
            ]),
            LLMObsAnnotationQueueInteractionsType::INTERACTIONS,
        ),
    );
    let mut configuration = datadog::Configuration::new();
    configuration
        .set_unstable_operation_enabled("v2.CreateLLMObsAnnotationQueueInteractions", true);
    let api = LLMObservabilityAPI::with_config(configuration);
    let resp = api
        .create_llm_obs_annotation_queue_interactions("queue_id".to_string(), body)
        .await;
    if let Ok(value) = resp {
        println!("{:#?}", value);
    } else {
        println!("{:#?}", resp.unwrap_err());
    }
}

Instructions

First install the library and its dependencies and then save the example to src/main.rs and run following commands:

    DD_SITE="datadoghq.comus3.datadoghq.comus5.datadoghq.comdatadoghq.euap1.datadoghq.comap2.datadoghq.comddog-gov.comus2.ddog-gov.com" DD_API_KEY="<DD_API_KEY>" DD_APP_KEY="<DD_APP_KEY>" cargo run

/**
 * Add annotation queue interactions returns "Created" response
 */

import { client, v2 } from "@datadog/datadog-api-client";

const configuration = client.createConfiguration();
configuration.unstableOperations["v2.createLLMObsAnnotationQueueInteractions"] =
  true;
const apiInstance = new v2.LLMObservabilityApi(configuration);

const params: v2.LLMObservabilityApiCreateLLMObsAnnotationQueueInteractionsRequest =
  {
    body: {
      data: {
        attributes: {
          interactions: [
            {
              contentId: "trace-abc-123",
              type: "trace",
            },
          ],
        },
        type: "interactions",
      },
    },
    queueId: "queue_id",
  };

apiInstance
  .createLLMObsAnnotationQueueInteractions(params)
  .then((data: v2.LLMObsAnnotationQueueInteractionsResponse) => {
    console.log(
      "API called successfully. Returned data: " + JSON.stringify(data)
    );
  })
  .catch((error: any) => console.error(error));

Instructions

First install the library and its dependencies and then save the example to example.ts and run following commands:

    DD_SITE="datadoghq.comus3.datadoghq.comus5.datadoghq.comdatadoghq.euap1.datadoghq.comap2.datadoghq.comddog-gov.comus2.ddog-gov.com" DD_API_KEY="<DD_API_KEY>" DD_APP_KEY="<DD_APP_KEY>" tsc "example.ts"

Add annotation queue interactions

Overview

Arguments

Path Parameters

Request

Body Data (required)

Response

Code Example

Add annotation queue interactions

Add annotation queue interactions

Instructions

Add annotation queue interactions

Instructions

Add annotation queue interactions

Instructions

Add annotation queue interactions

Instructions

Add annotation queue interactions

Instructions

Add annotation queue interactions

Instructions

How can I help you today?