--- title: Aggregate LLM Observability experimentation description: Datadog, the leading service for cloud-scale monitoring. breadcrumbs: Docs > API Reference > LLM Observability --- # Aggregate LLM Observability experimentation{% #aggregate-llm-observability-experimentation %} Copy pageCopied {% tab title="v2" %} **Note**: This endpoint is in preview and is subject to change. If you have any feedback, contact [Datadog support](https://docs.datadoghq.com/help/). | Datadog site | API endpoint | | ----------------- | ------------------------------------------------------------------------------ | | ap1.datadoghq.com | POST https://api.ap1.datadoghq.com/api/v2/llm-obs/v1/experimentation/analytics | | ap2.datadoghq.com | POST https://api.ap2.datadoghq.com/api/v2/llm-obs/v1/experimentation/analytics | | app.datadoghq.eu | POST https://api.datadoghq.eu/api/v2/llm-obs/v1/experimentation/analytics | | app.ddog-gov.com | POST https://api.ddog-gov.com/api/v2/llm-obs/v1/experimentation/analytics | | us2.ddog-gov.com | POST https://api.us2.ddog-gov.com/api/v2/llm-obs/v1/experimentation/analytics | | app.datadoghq.com | POST https://api.datadoghq.com/api/v2/llm-obs/v1/experimentation/analytics | | us3.datadoghq.com | POST https://api.us3.datadoghq.com/api/v2/llm-obs/v1/experimentation/analytics | | us5.datadoghq.com | POST https://api.us5.datadoghq.com/api/v2/llm-obs/v1/experimentation/analytics | ### Overview Execute an analytics aggregation over LLM Observability experimentation data. Use this endpoint to compute metrics (for example average eval scores) grouped by fields such as `span_id` or `experiment_id`. At least one `compute` definition and one `index` must be provided. ### Request #### Body Data (required) Analytics payload. {% tab title="Model" %} | Parent field | Field | Type | Description | | ------------ | ---------------------------- | -------- | --------------------------------------------------------------------------------------------------------- | | | data [*required*] | object | Data object for an analytics request. | | data | attributes [*required*] | object | Attributes for an analytics request. | | attributes | aggregate [*required*] | object | Analytics aggregation parameters. | | aggregate | compute [*required*] | [object] | List of metric computations to perform. | | compute | metric [*required*] | string | Name of the metric to compute. | | compute | name | string | Optional alias for this computation in the response. | | aggregate | dataset_version | int64 | Filter to a specific dataset version. | | aggregate | group_by | [object] | Fields to group results by. | | group_by | field [*required*] | string | Field name to group by. | | aggregate | indexes [*required*] | [string] | Data indexes to query. At least one is required. | | aggregate | limit | int32 | Maximum number of results to return. | | aggregate | search [*required*] | object | Search query for filtering analytics data. | | search | query [*required*] | string | Filter expression. | | aggregate | time | object | Unix-millisecond time range for filtering analytics data. | | time | from [*required*] | int64 | Start of the time range in milliseconds since Unix epoch. | | time | to [*required*] | int64 | End of the time range in milliseconds since Unix epoch. | | data | type [*required*] | enum | Resource type for experimentation search and analytics operations. Allowed enum values: `experimentation` | {% /tab %} {% tab title="Example" %} ```json { "data": { "attributes": { "aggregate": { "compute": [ { "metric": "score_value", "name": "avg_faithfulness" } ], "dataset_version": "integer", "group_by": [ { "field": "span_id" } ], "indexes": [ "experiment-evals" ], "limit": 1000, "search": { "query": "@experiment_id:3fd6b5e0-8910-4b1c-a7d0-5b84de329012" }, "time": { "from": 1705312200000, "to": 1705315800000 } } }, "type": "experimentation" } } ``` {% /tab %} ### Response {% tab title="200" %} OK {% tab title="Model" %} Response to an analytics query. | Parent field | Field | Type | Description | | ------------ | ---------------------------- | -------- | --------------------------------------------------------------------------------------------------------- | | | data [*required*] | object | JSON:API data object for an analytics response. | | data | attributes [*required*] | object | Attributes of an analytics response. | | attributes | hit_count [*required*] | int64 | Total number of events matched by the query before grouping. | | attributes | result [*required*] | object | Analytics query result containing all buckets. | | result | values [*required*] | [object] | List of result buckets. | | values | by | object | The group-by field values for this bucket. | | values | metrics [*required*] | object | Computed metric values for this bucket. | | data | id [*required*] | string | Server-generated identifier for this analytics result. | | data | type [*required*] | enum | Resource type for experimentation search and analytics operations. Allowed enum values: `experimentation` | {% /tab %} {% tab title="Example" %} ```json { "data": { "attributes": { "hit_count": 1500, "result": { "values": [ { "by": { "span_id": "span-7a1b2c3d" }, "metrics": { "score_value": 0.85 } } ] } }, "id": "00000000-0000-0000-0000-000000000001", "type": "experimentation" } } ``` {% /tab %} {% /tab %} {% tab title="400" %} Bad Request {% tab title="Model" %} API error response. | Parent field | Field | Type | Description | | ------------ | ------------------------ | -------- | ------------------------------------------------------------------------------- | | | errors [*required*] | [object] | A list of errors. | | errors | detail | string | A human-readable explanation specific to this occurrence of the error. | | errors | meta | object | Non-standard meta-information about the error | | errors | source | object | References to the source of the error. | | source | header | string | A string indicating the name of a single request header which caused the error. | | source | parameter | string | A string indicating which URI query parameter caused the error. | | source | pointer | string | A JSON pointer to the value in the request document that caused the error. | | errors | status | string | Status code of the response. | | errors | title | string | Short human-readable summary of the error. | {% /tab %} {% tab title="Example" %} ```json { "errors": [ { "detail": "Missing required attribute in body", "meta": {}, "source": { "header": "Authorization", "parameter": "limit", "pointer": "/data/attributes/title" }, "status": "400", "title": "Bad Request" } ] } ``` {% /tab %} {% /tab %} {% tab title="401" %} Unauthorized {% tab title="Model" %} API error response. | Parent field | Field | Type | Description | | ------------ | ------------------------ | -------- | ------------------------------------------------------------------------------- | | | errors [*required*] | [object] | A list of errors. | | errors | detail | string | A human-readable explanation specific to this occurrence of the error. | | errors | meta | object | Non-standard meta-information about the error | | errors | source | object | References to the source of the error. | | source | header | string | A string indicating the name of a single request header which caused the error. | | source | parameter | string | A string indicating which URI query parameter caused the error. | | source | pointer | string | A JSON pointer to the value in the request document that caused the error. | | errors | status | string | Status code of the response. | | errors | title | string | Short human-readable summary of the error. | {% /tab %} {% tab title="Example" %} ```json { "errors": [ { "detail": "Missing required attribute in body", "meta": {}, "source": { "header": "Authorization", "parameter": "limit", "pointer": "/data/attributes/title" }, "status": "400", "title": "Bad Request" } ] } ``` {% /tab %} {% /tab %} {% tab title="403" %} Forbidden {% tab title="Model" %} API error response. | Parent field | Field | Type | Description | | ------------ | ------------------------ | -------- | ------------------------------------------------------------------------------- | | | errors [*required*] | [object] | A list of errors. | | errors | detail | string | A human-readable explanation specific to this occurrence of the error. | | errors | meta | object | Non-standard meta-information about the error | | errors | source | object | References to the source of the error. | | source | header | string | A string indicating the name of a single request header which caused the error. | | source | parameter | string | A string indicating which URI query parameter caused the error. | | source | pointer | string | A JSON pointer to the value in the request document that caused the error. | | errors | status | string | Status code of the response. | | errors | title | string | Short human-readable summary of the error. | {% /tab %} {% tab title="Example" %} ```json { "errors": [ { "detail": "Missing required attribute in body", "meta": {}, "source": { "header": "Authorization", "parameter": "limit", "pointer": "/data/attributes/title" }, "status": "400", "title": "Bad Request" } ] } ``` {% /tab %} {% /tab %} {% tab title="429" %} Too many requests {% tab title="Model" %} API error response. | Field | Type | Description | | ------------------------ | -------- | ----------------- | | errors [*required*] | [string] | A list of errors. | {% /tab %} {% tab title="Example" %} ```json { "errors": [ "Bad Request" ] } ``` {% /tab %} {% /tab %} {% tab title="500" %} Internal Server Error {% tab title="Model" %} API error response. | Parent field | Field | Type | Description | | ------------ | ------------------------ | -------- | ------------------------------------------------------------------------------- | | | errors [*required*] | [object] | A list of errors. | | errors | detail | string | A human-readable explanation specific to this occurrence of the error. | | errors | meta | object | Non-standard meta-information about the error | | errors | source | object | References to the source of the error. | | source | header | string | A string indicating the name of a single request header which caused the error. | | source | parameter | string | A string indicating which URI query parameter caused the error. | | source | pointer | string | A JSON pointer to the value in the request document that caused the error. | | errors | status | string | Status code of the response. | | errors | title | string | Short human-readable summary of the error. | {% /tab %} {% tab title="Example" %} ```json { "errors": [ { "detail": "Missing required attribute in body", "meta": {}, "source": { "header": "Authorization", "parameter": "limit", "pointer": "/data/attributes/title" }, "status": "400", "title": "Bad Request" } ] } ``` {% /tab %} {% /tab %} ### Code Example ##### \## default # \# Curl command curl -X POST "https://api.datadoghq.com/api/v2/llm-obs/v1/experimentation/analytics" \ -H "Accept: application/json" \ -H "Content-Type: application/json" \ -H "DD-API-KEY: ${DD_API_KEY}" \ -H "DD-APPLICATION-KEY: ${DD_APP_KEY}" \ -d @- << EOF { "data": { "attributes": { "aggregate": { "compute": [ { "metric": "score_value", "name": "avg_faithfulness" } ], "group_by": [ { "field": "span_id" } ], "indexes": [ "experiment-evals" ], "search": { "query": "@experiment_id:3fd6b5e0-8910-4b1c-a7d0-5b84de329012 @label:faithfulness" } } }, "type": "experimentation" } } EOF {% /tab %}