Note : This endpoint is in preview and is subject to change.
If you have any feedback, contact Datadog support .
GET https://api.ap1.datadoghq.com/api/v2/llm-obs/v1/annotation-queues/{queue_id}/label-schema https://api.ap2.datadoghq.com/api/v2/llm-obs/v1/annotation-queues/{queue_id}/label-schema https://api.datadoghq.eu/api/v2/llm-obs/v1/annotation-queues/{queue_id}/label-schema https://api.ddog-gov.com/api/v2/llm-obs/v1/annotation-queues/{queue_id}/label-schema https://api.us2.ddog-gov.com/api/v2/llm-obs/v1/annotation-queues/{queue_id}/label-schema https://api.datadoghq.com/api/v2/llm-obs/v1/annotation-queues/{queue_id}/label-schema https://api.us3.datadoghq.com/api/v2/llm-obs/v1/annotation-queues/{queue_id}/label-schema https://api.us5.datadoghq.com/api/v2/llm-obs/v1/annotation-queues/{queue_id}/label-schema
Overview Retrieve the label schema for a given annotation queue.
Arguments Path Parameters The ID of the LLM Observability annotation queue.
Response OK
Response containing the label schema of an annotation queue.
Expand All
Data object for an annotation queue label schema.
Attributes of an annotation queue label schema.
annotation_schema [required ]
Schema defining the labels for an annotation queue.
List of label schema definitions.
Description of the label.
Whether this label includes an assessment field.
Whether this label includes a reasoning field.
Unique identifier of the label schema. Assigned by the server if not provided.
Whether the boolean label represents an assessment. Requires has_assessment to be true.
Whether score values must be integers. Applicable to score-type labels.
Whether this label is required for an annotation.
Maximum value for score-type labels.
Minimum value for score-type labels.
Name of the label. Must match the pattern ^[a-zA-Z0-9_-]+$ and be unique within the queue.
Type of a label in an annotation queue label schema.
Allowed enum values: score,categorical,boolean,text
Allowed values for categorical-type labels. Must contain at least one non-empty, unique value.
Unique identifier of the annotation queue.
Resource type of an LLM Observability annotation queue.
Allowed enum values: queues
{
"data" : {
"attributes" : {
"annotation_schema" : {
"label_schemas" : [
{
"description" : "Rating of the response quality." ,
"has_assessment" : false ,
"has_reasoning" : false ,
"id" : "abc-123" ,
"is_assessment" : false ,
"is_integer" : false ,
"is_required" : true ,
"max" : 5 ,
"min" : 0 ,
"name" : "quality" ,
"type" : "score" ,
"values" : [
"good" ,
"bad" ,
"neutral"
]
}
]
}
},
"id" : "00000000-0000-0000-0000-000000000001" ,
"type" : "queues"
}
} Unauthorized
API error response.
Expand All
A human-readable explanation specific to this occurrence of the error.
Non-standard meta-information about the error
References to the source of the error.
A string indicating the name of a single request header which caused the error.
A string indicating which URI query parameter caused the error.
A JSON pointer to the value in the request document that caused the error.
Status code of the response.
Short human-readable summary of the error.
{
"errors" : [
{
"detail" : "Missing required attribute in body" ,
"meta" : {},
"source" : {
"header" : "Authorization" ,
"parameter" : "limit" ,
"pointer" : "/data/attributes/title"
},
"status" : "400" ,
"title" : "Bad Request"
}
]
} Forbidden
API error response.
Expand All
A human-readable explanation specific to this occurrence of the error.
Non-standard meta-information about the error
References to the source of the error.
A string indicating the name of a single request header which caused the error.
A string indicating which URI query parameter caused the error.
A JSON pointer to the value in the request document that caused the error.
Status code of the response.
Short human-readable summary of the error.
{
"errors" : [
{
"detail" : "Missing required attribute in body" ,
"meta" : {},
"source" : {
"header" : "Authorization" ,
"parameter" : "limit" ,
"pointer" : "/data/attributes/title"
},
"status" : "400" ,
"title" : "Bad Request"
}
]
} Not Found
API error response.
Expand All
A human-readable explanation specific to this occurrence of the error.
Non-standard meta-information about the error
References to the source of the error.
A string indicating the name of a single request header which caused the error.
A string indicating which URI query parameter caused the error.
A JSON pointer to the value in the request document that caused the error.
Status code of the response.
Short human-readable summary of the error.
{
"errors" : [
{
"detail" : "Missing required attribute in body" ,
"meta" : {},
"source" : {
"header" : "Authorization" ,
"parameter" : "limit" ,
"pointer" : "/data/attributes/title"
},
"status" : "400" ,
"title" : "Bad Request"
}
]
} Too many requests
{
"errors" : [
"Bad Request"
]
} Internal Server Error
API error response.
Expand All
A human-readable explanation specific to this occurrence of the error.
Non-standard meta-information about the error
References to the source of the error.
A string indicating the name of a single request header which caused the error.
A string indicating which URI query parameter caused the error.
A JSON pointer to the value in the request document that caused the error.
Status code of the response.
Short human-readable summary of the error.
{
"errors" : [
{
"detail" : "Missing required attribute in body" ,
"meta" : {},
"source" : {
"header" : "Authorization" ,
"parameter" : "limit" ,
"pointer" : "/data/attributes/title"
},
"status" : "400" ,
"title" : "Bad Request"
}
]
} Code Example Copy
# Path parameters export queue_id = "00000000-0000-0000-0000-000000000001" # Curl command curl -X GET "https://api.ap1.datadoghq.com "https://api.ap2.datadoghq.com "https://api.datadoghq.eu "https://api.ddog-gov.com "https://api.us2.ddog-gov.com "https://api.datadoghq.com "https://api.us3.datadoghq.com "https://api.us5.datadoghq.com /api/v2/llm-obs/v1/annotation-queues/${queue_id}/label-schema " \
-H "Accept: application/json" \
-H "DD-API-KEY: ${DD_API_KEY} " \
-H "DD-APPLICATION-KEY: ${DD_APP_KEY} "
"""
Get annotation queue label schema returns "OK" response
"""
from datadog_api_client import ApiClient , Configuration
from datadog_api_client.v2.api.llm_observability_api import LLMObservabilityApi
configuration = Configuration ()
configuration . unstable_operations [ "get_llm_obs_annotation_queue_label_schema" ] = True
with ApiClient ( configuration ) as api_client :
api_instance = LLMObservabilityApi ( api_client )
response = api_instance . get_llm_obs_annotation_queue_label_schema (
queue_id = "queue_id" ,
)
print ( response )
Instructions First install the library and its dependencies and then save the example to example.py and run following commands:
DD_SITE = "datadoghq.com us3.datadoghq.com us5.datadoghq.com datadoghq.eu ap1.datadoghq.com ap2.datadoghq.com ddog-gov.com us2.ddog-gov.com " DD_API_KEY = "<DD_API_KEY>" DD_APP_KEY = "<DD_APP_KEY>" python3 "example.py"
# Get annotation queue label schema returns "OK" response
require "datadog_api_client"
DatadogAPIClient . configure do | config |
config . unstable_operations [ "v2.get_llm_obs_annotation_queue_label_schema" . to_sym ] = true
end
api_instance = DatadogAPIClient :: V2 :: LLMObservabilityAPI . new
p api_instance . get_llm_obs_annotation_queue_label_schema ( "queue_id" )
Instructions First install the library and its dependencies and then save the example to example.rb and run following commands:
DD_SITE = "datadoghq.com us3.datadoghq.com us5.datadoghq.com datadoghq.eu ap1.datadoghq.com ap2.datadoghq.com ddog-gov.com us2.ddog-gov.com " DD_API_KEY = "<DD_API_KEY>" DD_APP_KEY = "<DD_APP_KEY>" rb "example.rb"
// Get annotation queue label schema returns "OK" response
package main
import (
"context"
"encoding/json"
"fmt"
"os"
"github.com/DataDog/datadog-api-client-go/v2/api/datadog"
"github.com/DataDog/datadog-api-client-go/v2/api/datadogV2"
)
func main () {
ctx := datadog . NewDefaultContext ( context . Background ())
configuration := datadog . NewConfiguration ()
configuration . SetUnstableOperationEnabled ( "v2.GetLLMObsAnnotationQueueLabelSchema" , true )
apiClient := datadog . NewAPIClient ( configuration )
api := datadogV2 . NewLLMObservabilityApi ( apiClient )
resp , r , err := api . GetLLMObsAnnotationQueueLabelSchema ( ctx , "queue_id" )
if err != nil {
fmt . Fprintf ( os . Stderr , "Error when calling `LLMObservabilityApi.GetLLMObsAnnotationQueueLabelSchema`: %v\n" , err )
fmt . Fprintf ( os . Stderr , "Full HTTP response: %v\n" , r )
}
responseContent , _ := json . MarshalIndent ( resp , "" , " " )
fmt . Fprintf ( os . Stdout , "Response from `LLMObservabilityApi.GetLLMObsAnnotationQueueLabelSchema`:\n%s\n" , responseContent )
}
Instructions First install the library and its dependencies and then save the example to main.go and run following commands:
DD_SITE = "datadoghq.com us3.datadoghq.com us5.datadoghq.com datadoghq.eu ap1.datadoghq.com ap2.datadoghq.com ddog-gov.com us2.ddog-gov.com " DD_API_KEY = "<DD_API_KEY>" DD_APP_KEY = "<DD_APP_KEY>" go run "main.go"
// Get annotation queue label schema returns "OK" response
import com.datadog.api.client.ApiClient ;
import com.datadog.api.client.ApiException ;
import com.datadog.api.client.v2.api.LlmObservabilityApi ;
import com.datadog.api.client.v2.model.LLMObsAnnotationQueueLabelSchemaResponse ;
public class Example {
public static void main ( String [] args ) {
ApiClient defaultClient = ApiClient . getDefaultApiClient ();
defaultClient . setUnstableOperationEnabled ( "v2.getLLMObsAnnotationQueueLabelSchema" , true );
LlmObservabilityApi apiInstance = new LlmObservabilityApi ( defaultClient );
try {
LLMObsAnnotationQueueLabelSchemaResponse result =
apiInstance . getLLMObsAnnotationQueueLabelSchema ( "00000000-0000-0000-0000-000000000001" );
System . out . println ( result );
} catch ( ApiException e ) {
System . err . println (
"Exception when calling LlmObservabilityApi#getLLMObsAnnotationQueueLabelSchema" );
System . err . println ( "Status code: " + e . getCode ());
System . err . println ( "Reason: " + e . getResponseBody ());
System . err . println ( "Response headers: " + e . getResponseHeaders ());
e . printStackTrace ();
}
}
}
Instructions First install the library and its dependencies and then save the example to Example.java and run following commands:
DD_SITE = "datadoghq.com us3.datadoghq.com us5.datadoghq.com datadoghq.eu ap1.datadoghq.com ap2.datadoghq.com ddog-gov.com us2.ddog-gov.com " DD_API_KEY = "<DD_API_KEY>" DD_APP_KEY = "<DD_APP_KEY>" java "Example.java"
// Get annotation queue label schema returns "OK" response
use datadog_api_client ::datadog ;
use datadog_api_client ::datadogV2 ::api_llm_observability ::LLMObservabilityAPI ;
#[tokio::main]
async fn main () {
let mut configuration = datadog ::Configuration ::new ();
configuration . set_unstable_operation_enabled ( "v2.GetLLMObsAnnotationQueueLabelSchema" , true );
let api = LLMObservabilityAPI ::with_config ( configuration );
let resp = api
. get_llm_obs_annotation_queue_label_schema ( "queue_id" . to_string ())
. await ;
if let Ok ( value ) = resp {
println! ( " {:#?} " , value );
} else {
println! ( " {:#?} " , resp . unwrap_err ());
}
}
Instructions First install the library and its dependencies and then save the example to src/main.rs and run following commands:
DD_SITE = "datadoghq.com us3.datadoghq.com us5.datadoghq.com datadoghq.eu ap1.datadoghq.com ap2.datadoghq.com ddog-gov.com us2.ddog-gov.com " DD_API_KEY = "<DD_API_KEY>" DD_APP_KEY = "<DD_APP_KEY>" cargo run
/**
* Get annotation queue label schema returns "OK" response
*/
import { client , v2 } from "@datadog/datadog-api-client" ;
const configuration = client . createConfiguration ();
configuration . unstableOperations [ "v2.getLLMObsAnnotationQueueLabelSchema" ] =
true ;
const apiInstance = new v2 . LLMObservabilityApi ( configuration );
const params : v2.LLMObservabilityApiGetLLMObsAnnotationQueueLabelSchemaRequest =
{
queueId : "queue_id" ,
};
apiInstance
. getLLMObsAnnotationQueueLabelSchema ( params )
. then (( data : v2.LLMObsAnnotationQueueLabelSchemaResponse ) => {
console . log (
"API called successfully. Returned data: " + JSON . stringify ( data )
);
})
. catch (( error : any ) => console . error ( error ));
Instructions First install the library and its dependencies and then save the example to example.ts and run following commands:
DD_SITE = "datadoghq.com us3.datadoghq.com us5.datadoghq.com datadoghq.eu ap1.datadoghq.com ap2.datadoghq.com ddog-gov.com us2.ddog-gov.com " DD_API_KEY = "<DD_API_KEY>" DD_APP_KEY = "<DD_APP_KEY>" tsc "example.ts"