Create or update annotations on interactions in a queue. Each annotation is matched
by interaction_id and the requesting user’s identity.
Results and errors in the response are linked to request items by interaction_id.
Errors for individual items are returned in the errors field without blocking the rest of the batch.
List of annotations to create or update. Must contain at least one item.
interaction_id [required]
string
ID of the interaction to annotate.
label_values [required]
[object]
Label values for this annotation. Each entry references a label schema by ID
and provides the corresponding value validated against the schema type constraints.
assessment
enum
Assessment result for a label value.
Allowed enum values: pass,fail
label_schema_id [required]
string
ID of the label schema this value corresponds to.
reasoning
string
Free text reasoning for this label value.
value [required]
<oneOf>
The value for this label. Must comply with the label schema type constraints.
Option 1
double
A scalar numeric value.
Option 2
string
A scalar string value.
Option 3
[string]
For categorical-type labels allowing multiple selections.
Option 4
boolean
A scalar boolean value.
type [required]
enum
Resource type for LLM Observability annotations.
Allowed enum values: annotations
{"data":{"attributes":{"annotations":[{"interaction_id":"00000000-0000-0000-0000-000000000001","label_values":[{"assessment":"pass","label_schema_id":"abc-123","reasoning":"The response was accurate and well-structured.","value":0}]}]},"type":"annotations"}}
Response containing the created or updated annotations.
Expand All
Field
Type
Description
data [required]
object
Data object for the annotations response.
attributes [required]
object
Attributes of the annotations response.
annotations [required]
[object]
Successfully created or updated annotations.
created_at [required]
date-time
Timestamp when the annotation was created.
created_by [required]
string
Identifier of the user who created the annotation.
id [required]
string
Unique identifier of the annotation.
interaction_id [required]
string
Identifier of the interaction this annotation belongs to.
label_values [required]
[object]
Label values for this annotation. Each entry references a label schema by ID
and provides the corresponding value.
assessment
enum
Assessment result for a label value.
Allowed enum values: pass,fail
label_schema_id [required]
string
ID of the label schema this value corresponds to.
name_when_saved
string
Name of the label schema at the time the annotation was created.
reasoning
string
Free text reasoning for this label value.
type
enum
Type of a label in an annotation queue label schema.
Allowed enum values: score,categorical,boolean,text
value [required]
<oneOf>
The value for this label. Must comply with the label schema type constraints.
Option 1
double
A scalar numeric value.
Option 2
string
A scalar string value.
Option 3
[string]
For categorical-type labels allowing multiple selections.
Option 4
boolean
A scalar boolean value.
modified_at [required]
date-time
Timestamp when the annotation was last modified.
modified_by [required]
string
Identifier of the user who last modified the annotation.
errors
[object]
Partial errors for annotations that could not be processed.
annotation_id
string
ID of the annotation that failed, if applicable.
error [required]
string
Error message.
interaction_id [required]
string
ID of the interaction that failed.
id [required]
string
The annotation queue ID.
type [required]
enum
Resource type for LLM Observability annotations.
Allowed enum values: annotations
{"data":{"attributes":{"annotations":[{"created_at":"2024-01-15T10:30:00Z","created_by":"00000000-0000-0000-0000-000000000002","id":"annotation-789","interaction_id":"interaction-456","label_values":[{"assessment":"pass","label_schema_id":"abc-123","name_when_saved":"quality","reasoning":"The response was accurate and well-structured.","type":"score","value":0}],"modified_at":"2024-01-15T10:30:00Z","modified_by":"00000000-0000-0000-0000-000000000002"}],"errors":[{"annotation_id":"00000000-0000-0000-0000-000000000000","error":"interaction not found","interaction_id":"00000000-0000-0000-0000-000000000001"}]},"id":"00000000-0000-0000-0000-000000000001","type":"annotations"}}
"""
Create or update annotations returns "OK — annotations created or updated. Per-item errors are listed in `errors`."
response
"""fromdatadog_api_clientimportApiClient,Configurationfromdatadog_api_client.v2.api.llm_observability_apiimportLLMObservabilityApifromdatadog_api_client.v2.model.llm_obs_annotation_label_valueimportLLMObsAnnotationLabelValuefromdatadog_api_client.v2.model.llm_obs_annotations_data_attributes_requestimport(LLMObsAnnotationsDataAttributesRequest,)fromdatadog_api_client.v2.model.llm_obs_annotations_data_requestimportLLMObsAnnotationsDataRequestfromdatadog_api_client.v2.model.llm_obs_annotations_requestimportLLMObsAnnotationsRequestfromdatadog_api_client.v2.model.llm_obs_annotations_typeimportLLMObsAnnotationsTypefromdatadog_api_client.v2.model.llm_obs_upsert_annotation_itemimportLLMObsUpsertAnnotationItembody=LLMObsAnnotationsRequest(data=LLMObsAnnotationsDataRequest(attributes=LLMObsAnnotationsDataAttributesRequest(annotations=[LLMObsUpsertAnnotationItem(interaction_id="00000000-0000-0000-0000-000000000001",label_values=[LLMObsAnnotationLabelValue(label_schema_id="abc-123",value="good",),LLMObsAnnotationLabelValue(label_schema_id="ef56gh78",value="positive",),],),],),type=LLMObsAnnotationsType.ANNOTATIONS,),)configuration=Configuration()configuration.unstable_operations["upsert_llm_obs_annotations"]=TruewithApiClient(configuration)asapi_client:api_instance=LLMObservabilityApi(api_client)response=api_instance.upsert_llm_obs_annotations(queue_id="queue_id",body=body)print(response)
# Create or update annotations returns "OK — annotations created or updated. Per-item errors are listed in `errors`."# responserequire"datadog_api_client"DatadogAPIClient.configuredo|config|config.unstable_operations["v2.upsert_llm_obs_annotations".to_sym]=trueendapi_instance=DatadogAPIClient::V2::LLMObservabilityAPI.newbody=DatadogAPIClient::V2::LLMObsAnnotationsRequest.new({data:DatadogAPIClient::V2::LLMObsAnnotationsDataRequest.new({attributes:DatadogAPIClient::V2::LLMObsAnnotationsDataAttributesRequest.new({annotations:[DatadogAPIClient::V2::LLMObsUpsertAnnotationItem.new({interaction_id:"00000000-0000-0000-0000-000000000001",label_values:[DatadogAPIClient::V2::LLMObsAnnotationLabelValue.new({label_schema_id:"abc-123",value:"good",}),DatadogAPIClient::V2::LLMObsAnnotationLabelValue.new({label_schema_id:"ef56gh78",value:"positive",}),],}),],}),type:DatadogAPIClient::V2::LLMObsAnnotationsType::ANNOTATIONS,}),})papi_instance.upsert_llm_obs_annotations("queue_id",body)
// Create or update annotations returns "OK — annotations created or updated. Per-item errors are listed in `errors`."// responsepackagemainimport("context""encoding/json""fmt""os""github.com/DataDog/datadog-api-client-go/v2/api/datadog""github.com/DataDog/datadog-api-client-go/v2/api/datadogV2")funcmain(){body:=datadogV2.LLMObsAnnotationsRequest{Data:datadogV2.LLMObsAnnotationsDataRequest{Attributes:datadogV2.LLMObsAnnotationsDataAttributesRequest{Annotations:[]datadogV2.LLMObsUpsertAnnotationItem{{InteractionId:"00000000-0000-0000-0000-000000000001",LabelValues:[]datadogV2.LLMObsAnnotationLabelValue{{LabelSchemaId:"abc-123",Value:datadogV2.LLMObsAnnotationLabelValueValue{AnyValueString:datadog.PtrString("good")},},{LabelSchemaId:"ef56gh78",Value:datadogV2.LLMObsAnnotationLabelValueValue{AnyValueString:datadog.PtrString("positive")},},},},},},Type:datadogV2.LLMOBSANNOTATIONSTYPE_ANNOTATIONS,},}ctx:=datadog.NewDefaultContext(context.Background())configuration:=datadog.NewConfiguration()configuration.SetUnstableOperationEnabled("v2.UpsertLLMObsAnnotations",true)apiClient:=datadog.NewAPIClient(configuration)api:=datadogV2.NewLLMObservabilityApi(apiClient)resp,r,err:=api.UpsertLLMObsAnnotations(ctx,"queue_id",body)iferr!=nil{fmt.Fprintf(os.Stderr,"Error when calling `LLMObservabilityApi.UpsertLLMObsAnnotations`: %v\n",err)fmt.Fprintf(os.Stderr,"Full HTTP response: %v\n",r)}responseContent,_:=json.MarshalIndent(resp,""," ")fmt.Fprintf(os.Stdout,"Response from `LLMObservabilityApi.UpsertLLMObsAnnotations`:\n%s\n",responseContent)}
// Create or update annotations returns "OK — annotations created or updated.
// Per-item errors are listed in `errors`." response
usedatadog_api_client::datadog;usedatadog_api_client::datadogV2::api_llm_observability::LLMObservabilityAPI;usedatadog_api_client::datadogV2::model::LLMObsAnnotationLabelValue;usedatadog_api_client::datadogV2::model::LLMObsAnnotationLabelValueValue;usedatadog_api_client::datadogV2::model::LLMObsAnnotationsDataAttributesRequest;usedatadog_api_client::datadogV2::model::LLMObsAnnotationsDataRequest;usedatadog_api_client::datadogV2::model::LLMObsAnnotationsRequest;usedatadog_api_client::datadogV2::model::LLMObsAnnotationsType;usedatadog_api_client::datadogV2::model::LLMObsUpsertAnnotationItem;#[tokio::main]asyncfnmain(){letbody=LLMObsAnnotationsRequest::new(LLMObsAnnotationsDataRequest::new(LLMObsAnnotationsDataAttributesRequest::new(vec![LLMObsUpsertAnnotationItem::new("00000000-0000-0000-0000-000000000001".to_string(),vec![LLMObsAnnotationLabelValue::new("abc-123".to_string(),LLMObsAnnotationLabelValueValue::AnyValueString("good".to_string()),),LLMObsAnnotationLabelValue::new("ef56gh78".to_string(),LLMObsAnnotationLabelValueValue::AnyValueString("positive".to_string()),),],)]),LLMObsAnnotationsType::ANNOTATIONS,));letmutconfiguration=datadog::Configuration::new();configuration.set_unstable_operation_enabled("v2.UpsertLLMObsAnnotations",true);letapi=LLMObservabilityAPI::with_config(configuration);letresp=api.upsert_llm_obs_annotations("queue_id".to_string(),body).await;ifletOk(value)=resp{println!("{:#?}",value);}else{println!("{:#?}",resp.unwrap_err());}}
DD_SITE="datadoghq.comus3.datadoghq.comus5.datadoghq.comdatadoghq.euap1.datadoghq.comap2.datadoghq.comuk1.datadoghq.comddog-gov.comus2.ddog-gov.com"DD_API_KEY="<DD_API_KEY>"DD_APP_KEY="<DD_APP_KEY>"cargo run
/**
* Create or update annotations returns "OK — annotations created or updated. Per-item errors are listed in `errors`."
* response
*/import{client,v2}from"@datadog/datadog-api-client";constconfiguration=client.createConfiguration();configuration.unstableOperations["v2.upsertLLMObsAnnotations"]=true;constapiInstance=newv2.LLMObservabilityApi(configuration);constparams: v2.LLMObservabilityApiUpsertLLMObsAnnotationsRequest={body:{data:{attributes:{annotations:[{interactionId:"00000000-0000-0000-0000-000000000001",labelValues:[{labelSchemaId:"abc-123",value:"good",},{labelSchemaId:"ef56gh78",value:"positive",},],},],},type:"annotations",},},queueId:"queue_id",};apiInstance.upsertLLMObsAnnotations(params).then((data: v2.LLMObsAnnotationsResponse)=>{console.log("API called successfully. Returned data: "+JSON.stringify(data));}).catch((error: any)=>console.error(error));