--- title: Create or update annotations description: Datadog, the leading service for cloud-scale monitoring. breadcrumbs: Docs > API Reference > LLM Observability --- # Create or update annotations{% #create-or-update-annotations %} Copy pageCopied {% tab title="v2" %} **Note**: This endpoint is in preview and is subject to change. If you have any feedback, contact [Datadog support](https://docs.datadoghq.com/help/). | Datadog site | API endpoint | | ----------------- | --------------------------------------------------------------------------------------------- | | ap1.datadoghq.com | POST https://api.ap1.datadoghq.com/api/v2/llm-obs/v1/annotation-queues/{queue_id}/annotations | | ap2.datadoghq.com | POST https://api.ap2.datadoghq.com/api/v2/llm-obs/v1/annotation-queues/{queue_id}/annotations | | app.datadoghq.eu | POST https://api.datadoghq.eu/api/v2/llm-obs/v1/annotation-queues/{queue_id}/annotations | | app.ddog-gov.com | POST https://api.ddog-gov.com/api/v2/llm-obs/v1/annotation-queues/{queue_id}/annotations | | us2.ddog-gov.com | POST https://api.us2.ddog-gov.com/api/v2/llm-obs/v1/annotation-queues/{queue_id}/annotations | | app.datadoghq.com | POST https://api.datadoghq.com/api/v2/llm-obs/v1/annotation-queues/{queue_id}/annotations | | us3.datadoghq.com | POST https://api.us3.datadoghq.com/api/v2/llm-obs/v1/annotation-queues/{queue_id}/annotations | | us5.datadoghq.com | POST https://api.us5.datadoghq.com/api/v2/llm-obs/v1/annotation-queues/{queue_id}/annotations | ### Overview Create or update annotations on interactions in a queue. Each annotation is matched by `interaction_id` and the requesting user's identity. Results and errors in the response are linked to request items by `interaction_id`. Errors for individual items are returned in the `errors` field without blocking the rest of the batch. ### Arguments #### Path Parameters | Name | Type | Description | | -------------------------- | ------ | ------------------------------------------------- | | queue_id [*required*] | string | The ID of the LLM Observability annotation queue. | ### Request #### Body Data (required) Payload for creating or updating annotations. {% tab title="Model" %} | Parent field | Field | Type | Description | | ------------ | --------------------------------- | ------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------- | | | data [*required*] | object | Data object for creating or updating annotations. | | data | attributes [*required*] | object | Attributes for creating or updating annotations. | | attributes | annotations [*required*] | [object] | List of annotations to create or update. Must contain at least one item. | | annotations | interaction_id [*required*] | string | ID of the interaction to annotate. | | annotations | label_values [*required*] | [object] | Label values for this annotation. Each entry references a label schema by ID and provides the corresponding value validated against the schema type constraints. | | label_values | assessment | enum | Assessment result for a label value. Allowed enum values: `pass,fail` | | label_values | label_schema_id [*required*] | string | ID of the label schema this value corresponds to. | | label_values | reasoning | string | Free text reasoning for this label value. | | label_values | value [*required*] | | The value for this label. Must comply with the label schema type constraints. | | value | Option 1 | double | A scalar numeric value. | | value | Option 2 | string | A scalar string value. | | value | Option 3 | [string] | For categorical-type labels allowing multiple selections. | | value | Option 4 | boolean | A scalar boolean value. | | data | type [*required*] | enum | Resource type for LLM Observability annotations. Allowed enum values: `annotations` | {% /tab %} {% tab title="Example" %} ```json { "data": { "attributes": { "annotations": [ { "interaction_id": "00000000-0000-0000-0000-000000000001", "label_values": [ { "assessment": "pass", "label_schema_id": "abc-123", "reasoning": "The response was accurate and well-structured.", "value": 0 } ] } ] }, "type": "annotations" } } ``` {% /tab %} ### Response {% tab title="200" %} OK — annotations created or updated. Per-item errors are listed in `errors`. {% tab title="Model" %} Response containing the created or updated annotations. | Parent field | Field | Type | Description | | ------------ | --------------------------------- | ------------- | ------------------------------------------------------------------------------------------------------------------ | | | data [*required*] | object | Data object for the annotations response. | | data | attributes [*required*] | object | Attributes of the annotations response. | | attributes | annotations [*required*] | [object] | Successfully created or updated annotations. | | annotations | created_at [*required*] | date-time | Timestamp when the annotation was created. | | annotations | created_by [*required*] | string | Identifier of the user who created the annotation. | | annotations | id [*required*] | string | Unique identifier of the annotation. | | annotations | interaction_id [*required*] | string | Identifier of the interaction this annotation belongs to. | | annotations | label_values [*required*] | [object] | Label values for this annotation. Each entry references a label schema by ID and provides the corresponding value. | | label_values | assessment | enum | Assessment result for a label value. Allowed enum values: `pass,fail` | | label_values | label_schema_id [*required*] | string | ID of the label schema this value corresponds to. | | label_values | name_when_saved | string | Name of the label schema at the time the annotation was created. | | label_values | reasoning | string | Free text reasoning for this label value. | | label_values | type | enum | Type of a label in an annotation queue label schema. Allowed enum values: `score,categorical,boolean,text` | | label_values | value [*required*] | | The value for this label. Must comply with the label schema type constraints. | | value | Option 1 | double | A scalar numeric value. | | value | Option 2 | string | A scalar string value. | | value | Option 3 | [string] | For categorical-type labels allowing multiple selections. | | value | Option 4 | boolean | A scalar boolean value. | | annotations | modified_at [*required*] | date-time | Timestamp when the annotation was last modified. | | annotations | modified_by [*required*] | string | Identifier of the user who last modified the annotation. | | attributes | errors | [object] | Partial errors for annotations that could not be processed. | | errors | annotation_id | string | ID of the annotation that failed, if applicable. | | errors | error [*required*] | string | Error message. | | errors | interaction_id [*required*] | string | ID of the interaction that failed. | | data | id [*required*] | string | The annotation queue ID. | | data | type [*required*] | enum | Resource type for LLM Observability annotations. Allowed enum values: `annotations` | {% /tab %} {% tab title="Example" %} ```json { "data": { "attributes": { "annotations": [ { "created_at": "2024-01-15T10:30:00Z", "created_by": "00000000-0000-0000-0000-000000000002", "id": "annotation-789", "interaction_id": "interaction-456", "label_values": [ { "assessment": "pass", "label_schema_id": "abc-123", "name_when_saved": "quality", "reasoning": "The response was accurate and well-structured.", "type": "score", "value": 0 } ], "modified_at": "2024-01-15T10:30:00Z", "modified_by": "00000000-0000-0000-0000-000000000002" } ], "errors": [ { "annotation_id": "00000000-0000-0000-0000-000000000000", "error": "interaction not found", "interaction_id": "00000000-0000-0000-0000-000000000001" } ] }, "id": "00000000-0000-0000-0000-000000000001", "type": "annotations" } } ``` {% /tab %} {% /tab %} {% tab title="400" %} Bad Request {% tab title="Model" %} API error response. | Parent field | Field | Type | Description | | ------------ | ------------------------ | -------- | ------------------------------------------------------------------------------- | | | errors [*required*] | [object] | A list of errors. | | errors | detail | string | A human-readable explanation specific to this occurrence of the error. | | errors | meta | object | Non-standard meta-information about the error | | errors | source | object | References to the source of the error. | | source | header | string | A string indicating the name of a single request header which caused the error. | | source | parameter | string | A string indicating which URI query parameter caused the error. | | source | pointer | string | A JSON pointer to the value in the request document that caused the error. | | errors | status | string | Status code of the response. | | errors | title | string | Short human-readable summary of the error. | {% /tab %} {% tab title="Example" %} ```json { "errors": [ { "detail": "Missing required attribute in body", "meta": {}, "source": { "header": "Authorization", "parameter": "limit", "pointer": "/data/attributes/title" }, "status": "400", "title": "Bad Request" } ] } ``` {% /tab %} {% /tab %} {% tab title="401" %} Unauthorized {% tab title="Model" %} API error response. | Parent field | Field | Type | Description | | ------------ | ------------------------ | -------- | ------------------------------------------------------------------------------- | | | errors [*required*] | [object] | A list of errors. | | errors | detail | string | A human-readable explanation specific to this occurrence of the error. | | errors | meta | object | Non-standard meta-information about the error | | errors | source | object | References to the source of the error. | | source | header | string | A string indicating the name of a single request header which caused the error. | | source | parameter | string | A string indicating which URI query parameter caused the error. | | source | pointer | string | A JSON pointer to the value in the request document that caused the error. | | errors | status | string | Status code of the response. | | errors | title | string | Short human-readable summary of the error. | {% /tab %} {% tab title="Example" %} ```json { "errors": [ { "detail": "Missing required attribute in body", "meta": {}, "source": { "header": "Authorization", "parameter": "limit", "pointer": "/data/attributes/title" }, "status": "400", "title": "Bad Request" } ] } ``` {% /tab %} {% /tab %} {% tab title="403" %} Forbidden {% tab title="Model" %} API error response. | Parent field | Field | Type | Description | | ------------ | ------------------------ | -------- | ------------------------------------------------------------------------------- | | | errors [*required*] | [object] | A list of errors. | | errors | detail | string | A human-readable explanation specific to this occurrence of the error. | | errors | meta | object | Non-standard meta-information about the error | | errors | source | object | References to the source of the error. | | source | header | string | A string indicating the name of a single request header which caused the error. | | source | parameter | string | A string indicating which URI query parameter caused the error. | | source | pointer | string | A JSON pointer to the value in the request document that caused the error. | | errors | status | string | Status code of the response. | | errors | title | string | Short human-readable summary of the error. | {% /tab %} {% tab title="Example" %} ```json { "errors": [ { "detail": "Missing required attribute in body", "meta": {}, "source": { "header": "Authorization", "parameter": "limit", "pointer": "/data/attributes/title" }, "status": "400", "title": "Bad Request" } ] } ``` {% /tab %} {% /tab %} {% tab title="404" %} Not Found — the queue does not exist. {% tab title="Model" %} API error response. | Parent field | Field | Type | Description | | ------------ | ------------------------ | -------- | ------------------------------------------------------------------------------- | | | errors [*required*] | [object] | A list of errors. | | errors | detail | string | A human-readable explanation specific to this occurrence of the error. | | errors | meta | object | Non-standard meta-information about the error | | errors | source | object | References to the source of the error. | | source | header | string | A string indicating the name of a single request header which caused the error. | | source | parameter | string | A string indicating which URI query parameter caused the error. | | source | pointer | string | A JSON pointer to the value in the request document that caused the error. | | errors | status | string | Status code of the response. | | errors | title | string | Short human-readable summary of the error. | {% /tab %} {% tab title="Example" %} ```json { "errors": [ { "detail": "Missing required attribute in body", "meta": {}, "source": { "header": "Authorization", "parameter": "limit", "pointer": "/data/attributes/title" }, "status": "400", "title": "Bad Request" } ] } ``` {% /tab %} {% /tab %} {% tab title="429" %} Too many requests {% tab title="Model" %} API error response. | Field | Type | Description | | ------------------------ | -------- | ----------------- | | errors [*required*] | [string] | A list of errors. | {% /tab %} {% tab title="Example" %} ```json { "errors": [ "Bad Request" ] } ``` {% /tab %} {% /tab %} ### Code Example ##### \## default # \# Path parameters export queue_id="00000000-0000-0000-0000-000000000001" \# Curl command curl -X POST "https://api.datadoghq.com/api/v2/llm-obs/v1/annotation-queues/${queue_id}/annotations" \ -H "Accept: application/json" \ -H "Content-Type: application/json" \ -H "DD-API-KEY: ${DD_API_KEY}" \ -H "DD-APPLICATION-KEY: ${DD_APP_KEY}" \ -d @- << EOF { "data": { "attributes": { "annotations": [ { "interaction_id": "00000000-0000-0000-0000-000000000001", "label_values": [ { "label_schema_id": "abc-123", "value": "good" }, { "label_schema_id": "ef56gh78", "value": "positive" } ] } ] }, "type": "annotations" } } EOF {% /tab %}