Create or replace the label schema for a given annotation queue.
The label schema defines the labels annotators can apply to interactions in the queue.
Label names must be unique within the queue and match the pattern ^[a-zA-Z0-9_-]+$.
Each label must have a valid type: score, categorical, boolean, or text.
Data object for updating an annotation queue label schema.
attributes [required]
object
Attributes for updating an annotation queue label schema.
annotation_schema [required]
object
Schema defining the labels for an annotation queue.
label_schemas [required]
[object]
List of label schema definitions.
description
string
Description of the label.
has_assessment
boolean
Whether this label includes an assessment field.
has_reasoning
boolean
Whether this label includes a reasoning field.
id
string
Unique identifier of the label schema. Assigned by the server if not provided.
is_assessment
boolean
Whether the boolean label represents an assessment. Requires has_assessment to be true.
is_integer
boolean
Whether score values must be integers. Applicable to score-type labels.
is_required
boolean
Whether this label is required for an annotation.
max
double
Maximum value for score-type labels.
min
double
Minimum value for score-type labels.
name [required]
string
Name of the label. Must match the pattern ^[a-zA-Z0-9_-]+$ and be unique within the queue.
type [required]
enum
Type of a label in an annotation queue label schema.
Allowed enum values: score,categorical,boolean,text
values
[string]
Allowed values for categorical-type labels. Must contain at least one non-empty, unique value.
type [required]
enum
Resource type of an LLM Observability annotation queue.
Allowed enum values: queues
{"data":{"attributes":{"annotation_schema":{"label_schemas":[{"description":"Rating of the response quality.","has_assessment":false,"has_reasoning":false,"id":"abc-123","is_assessment":false,"is_integer":false,"is_required":true,"max":5,"min":0,"name":"quality","type":"score","values":["good","bad","neutral"]}]}},"type":"queues"}}
Response containing the label schema of an annotation queue.
Expand All
Field
Type
Description
data [required]
object
Data object for an annotation queue label schema.
attributes [required]
object
Attributes of an annotation queue label schema.
annotation_schema [required]
object
Schema defining the labels for an annotation queue.
label_schemas [required]
[object]
List of label schema definitions.
description
string
Description of the label.
has_assessment
boolean
Whether this label includes an assessment field.
has_reasoning
boolean
Whether this label includes a reasoning field.
id
string
Unique identifier of the label schema. Assigned by the server if not provided.
is_assessment
boolean
Whether the boolean label represents an assessment. Requires has_assessment to be true.
is_integer
boolean
Whether score values must be integers. Applicable to score-type labels.
is_required
boolean
Whether this label is required for an annotation.
max
double
Maximum value for score-type labels.
min
double
Minimum value for score-type labels.
name [required]
string
Name of the label. Must match the pattern ^[a-zA-Z0-9_-]+$ and be unique within the queue.
type [required]
enum
Type of a label in an annotation queue label schema.
Allowed enum values: score,categorical,boolean,text
values
[string]
Allowed values for categorical-type labels. Must contain at least one non-empty, unique value.
id [required]
string
Unique identifier of the annotation queue.
type [required]
enum
Resource type of an LLM Observability annotation queue.
Allowed enum values: queues
{"data":{"attributes":{"annotation_schema":{"label_schemas":[{"description":"Rating of the response quality.","has_assessment":false,"has_reasoning":false,"id":"abc-123","is_assessment":false,"is_integer":false,"is_required":true,"max":5,"min":0,"name":"quality","type":"score","values":["good","bad","neutral"]}]}},"id":"00000000-0000-0000-0000-000000000001","type":"queues"}}
DD_SITE="datadoghq.comus3.datadoghq.comus5.datadoghq.comdatadoghq.euap1.datadoghq.comap2.datadoghq.comddog-gov.comus2.ddog-gov.com"DD_API_KEY="<DD_API_KEY>"DD_APP_KEY="<DD_APP_KEY>"cargo run
/**
* Update annotation queue label schema returns "OK" response
*/import{client,v2}from"@datadog/datadog-api-client";constconfiguration=client.createConfiguration();configuration.unstableOperations["v2.updateLLMObsAnnotationQueueLabelSchema"]=true;constapiInstance=newv2.LLMObservabilityApi(configuration);constparams: v2.LLMObservabilityApiUpdateLLMObsAnnotationQueueLabelSchemaRequest={body:{data:{attributes:{annotationSchema:{labelSchemas:[{description:"Rating of the response quality.",hasAssessment: false,hasReasoning: false,id:"abc-123",isAssessment: false,isInteger: false,isRequired: true,max: 5.0,min: 0.0,name:"quality",type:"score",values:["good","bad","neutral"],},],},},type:"queues",},},queueId:"queue_id",};apiInstance.updateLLMObsAnnotationQueueLabelSchema(params).then((data: v2.LLMObsAnnotationQueueLabelSchemaResponse)=>{console.log("API called successfully. Returned data: "+JSON.stringify(data));}).catch((error: any)=>console.error(error));
