--- title: List patterns topics with clustered points description: Datadog, the leading service for cloud-scale monitoring. breadcrumbs: Docs > API Reference > LLM Observability --- # List patterns topics with clustered points{% #list-patterns-topics-with-clustered-points %} Copy pageCopied {% tab title="v2" %} **Note**: This endpoint is in preview and is subject to change. If you have any feedback, contact [Datadog support](https://docs.datadoghq.com/help/). | Datadog site | API endpoint | | ----------------- | ---------------------------------------------------------------------------------------------- | | ap1.datadoghq.com | GET https://api.ap1.datadoghq.com/api/v2/llm-obs/v1/topic-discovery-topics/with-cluster-points | | ap2.datadoghq.com | GET https://api.ap2.datadoghq.com/api/v2/llm-obs/v1/topic-discovery-topics/with-cluster-points | | app.datadoghq.eu | GET https://api.datadoghq.eu/api/v2/llm-obs/v1/topic-discovery-topics/with-cluster-points | | app.ddog-gov.com | GET https://api.ddog-gov.com/api/v2/llm-obs/v1/topic-discovery-topics/with-cluster-points | | us2.ddog-gov.com | GET https://api.us2.ddog-gov.com/api/v2/llm-obs/v1/topic-discovery-topics/with-cluster-points | | app.datadoghq.com | GET https://api.datadoghq.com/api/v2/llm-obs/v1/topic-discovery-topics/with-cluster-points | | us3.datadoghq.com | GET https://api.us3.datadoghq.com/api/v2/llm-obs/v1/topic-discovery-topics/with-cluster-points | | us5.datadoghq.com | GET https://api.us5.datadoghq.com/api/v2/llm-obs/v1/topic-discovery-topics/with-cluster-points | ### Overview List the topics discovered by a patterns run, with the clustered points attached inline to each leaf topic. When no run is specified, the most recent completed run is used. ### Arguments #### Query Strings | Name | Type | Description | | --------------------------- | ------- | --------------------------------------------------------------------------------------------------------------------------------- | | config_id [*required*] | string | The ID of the patterns configuration. | | run_id | string | The ID of a specific patterns run. Defaults to the most recent completed run. | | include_metrics | boolean | When true, enrich each clustered point with span metrics such as status, duration, token counts, estimated cost, and evaluations. | ### Response {% tab title="200" %} OK {% tab title="Model" %} Response containing the topics, and the clustered points of their leaf topics, discovered by an LLM Observability patterns run. | Parent field | Field | Type | Description | | --------------- | --------------------------------- | --------- | ---------------------------------------------------------------------------------------------------------------------------------------------------- | | | data [*required*] | object | Data object of an LLM Observability patterns topics-with-clustered-points response. | | data | attributes [*required*] | object | Attributes of an LLM Observability patterns topics-with-clustered-points response. | | attributes | completed_at | date-time | Timestamp when the run completed. Null if the run has not completed. | | attributes | config_id [*required*] | string | Identifier of the configuration that produced the run. | | attributes | config_snapshot | object | Snapshot of the configuration used for a patterns run. | | config_snapshot | account_id | string | Integration account ID used for a bring-your-own-model run. | | config_snapshot | evp_query | string | Query that selected the spans for the run. | | config_snapshot | hierarchy_depth | int32 | Depth of the topic hierarchy generated. | | config_snapshot | integration_provider | string | Integration provider used for a bring-your-own-model run. | | config_snapshot | model_name | string | Model name used for a bring-your-own-model run. | | config_snapshot | num_records | int32 | Maximum number of records processed for the run. | | config_snapshot | sampling_ratio | double | Fraction of matching spans sampled for the run. | | attributes | created_at [*required*] | date-time | Timestamp when the run was created. | | attributes | previous_run_id [*required*] | string | Identifier of the run that completed immediately before this one. Empty if none. | | attributes | run_id [*required*] | string | Identifier of the run that produced the topics. | | attributes | topics [*required*] | [object] | List of discovered topics with their clustered points. | | topics | cluster_points | [object] | List of clustered points attached to a topic. | | cluster_points | duration | double | Duration of the source span in nanoseconds. Included only when metrics are requested. | | cluster_points | estimated_total_cost | double | Estimated total cost of the source span. Included only when metrics are requested. | | cluster_points | evaluation | object | Evaluation results for the source span keyed by evaluation name. Included only when metrics are requested. | | cluster_points | input_tokens | double | Number of input tokens of the source span. Included only when metrics are requested. | | cluster_points | output_tokens | double | Number of output tokens of the source span. Included only when metrics are requested. | | cluster_points | span_id [*required*] | string | Identifier of the source span. | | cluster_points | status | string | Status of the source span. Included only when metrics are requested. | | cluster_points | total_tokens | double | Total number of tokens of the source span. Included only when metrics are requested. | | topics | created_at [*required*] | date-time | Timestamp when the topic was created. | | topics | description [*required*] | string | Description of the topic. | | topics | first_seen_at [*required*] | date-time | Timestamp when the topic was first seen. | | topics | hierarchy_level [*required*] | int64 | Level of the topic in the hierarchy. Level 0 is a leaf topic. | | topics | id [*required*] | string | Unique identifier of the topic. | | topics | is_validated [*required*] | boolean | Whether the topic has been validated. | | topics | name [*required*] | string | Name of the topic. | | topics | parent_topic_id [*required*] | string | Identifier of the parent topic. Empty for top-level topics. | | topics | point_count [*required*] | int64 | Number of data points assigned to the topic. | | topics | run_id [*required*] | string | Identifier of the run that produced the topic. | | data | id [*required*] | string | Identifier of the run the topics belong to. | | data | type [*required*] | enum | Resource type of an LLM Observability patterns topics-with-clustered-points response. Allowed enum values: `get_topics_with_cluster_points_response` | {% /tab %} {% tab title="Example" %} ```json { "data": { "attributes": { "completed_at": "2024-01-15T10:45:00Z", "config_id": "a7c8d9e0-1234-5678-9abc-def012345678", "config_snapshot": { "account_id": "1000000001", "evp_query": "@ml_app:support-bot", "hierarchy_depth": 2, "integration_provider": "openai", "model_name": "gpt-4o", "num_records": 1000, "sampling_ratio": 0.1 }, "created_at": "2024-01-15T10:30:00Z", "previous_run_id": "", "run_id": "3fd6b5e0-8910-4b1c-a7d0-5b84de329012", "topics": [ { "cluster_points": [ { "duration": 1500000, "estimated_total_cost": 0.0021, "evaluation": {}, "input_tokens": 128, "output_tokens": 64, "span_id": "1234567890123456789", "status": "ok", "total_tokens": 192 } ], "created_at": "2024-01-15T10:30:00Z", "description": "Questions about invoices, charges, and refunds.", "first_seen_at": "2024-01-15T10:30:00Z", "hierarchy_level": 0, "id": "5c1fae90-2b6d-4e3a-9f12-7a0c4d8e6b21", "is_validated": true, "name": "Billing questions", "parent_topic_id": "", "point_count": 125, "run_id": "3fd6b5e0-8910-4b1c-a7d0-5b84de329012" } ] }, "id": "3fd6b5e0-8910-4b1c-a7d0-5b84de329012", "type": "get_topics_with_cluster_points_response" } } ``` {% /tab %} {% /tab %} {% tab title="400" %} Bad Request {% tab title="Model" %} API error response. | Parent field | Field | Type | Description | | ------------ | ------------------------ | -------- | ------------------------------------------------------------------------------- | | | errors [*required*] | [object] | A list of errors. | | errors | detail | string | A human-readable explanation specific to this occurrence of the error. | | errors | meta | object | Non-standard meta-information about the error | | errors | source | object | References to the source of the error. | | source | header | string | A string indicating the name of a single request header which caused the error. | | source | parameter | string | A string indicating which URI query parameter caused the error. | | source | pointer | string | A JSON pointer to the value in the request document that caused the error. | | errors | status | string | Status code of the response. | | errors | title | string | Short human-readable summary of the error. | {% /tab %} {% tab title="Example" %} ```json { "errors": [ { "detail": "Missing required attribute in body", "meta": {}, "source": { "header": "Authorization", "parameter": "limit", "pointer": "/data/attributes/title" }, "status": "400", "title": "Bad Request" } ] } ``` {% /tab %} {% /tab %} {% tab title="401" %} Unauthorized {% tab title="Model" %} API error response. | Parent field | Field | Type | Description | | ------------ | ------------------------ | -------- | ------------------------------------------------------------------------------- | | | errors [*required*] | [object] | A list of errors. | | errors | detail | string | A human-readable explanation specific to this occurrence of the error. | | errors | meta | object | Non-standard meta-information about the error | | errors | source | object | References to the source of the error. | | source | header | string | A string indicating the name of a single request header which caused the error. | | source | parameter | string | A string indicating which URI query parameter caused the error. | | source | pointer | string | A JSON pointer to the value in the request document that caused the error. | | errors | status | string | Status code of the response. | | errors | title | string | Short human-readable summary of the error. | {% /tab %} {% tab title="Example" %} ```json { "errors": [ { "detail": "Missing required attribute in body", "meta": {}, "source": { "header": "Authorization", "parameter": "limit", "pointer": "/data/attributes/title" }, "status": "400", "title": "Bad Request" } ] } ``` {% /tab %} {% /tab %} {% tab title="403" %} Forbidden {% tab title="Model" %} API error response. | Parent field | Field | Type | Description | | ------------ | ------------------------ | -------- | ------------------------------------------------------------------------------- | | | errors [*required*] | [object] | A list of errors. | | errors | detail | string | A human-readable explanation specific to this occurrence of the error. | | errors | meta | object | Non-standard meta-information about the error | | errors | source | object | References to the source of the error. | | source | header | string | A string indicating the name of a single request header which caused the error. | | source | parameter | string | A string indicating which URI query parameter caused the error. | | source | pointer | string | A JSON pointer to the value in the request document that caused the error. | | errors | status | string | Status code of the response. | | errors | title | string | Short human-readable summary of the error. | {% /tab %} {% tab title="Example" %} ```json { "errors": [ { "detail": "Missing required attribute in body", "meta": {}, "source": { "header": "Authorization", "parameter": "limit", "pointer": "/data/attributes/title" }, "status": "400", "title": "Bad Request" } ] } ``` {% /tab %} {% /tab %} {% tab title="404" %} Not Found {% tab title="Model" %} API error response. | Parent field | Field | Type | Description | | ------------ | ------------------------ | -------- | ------------------------------------------------------------------------------- | | | errors [*required*] | [object] | A list of errors. | | errors | detail | string | A human-readable explanation specific to this occurrence of the error. | | errors | meta | object | Non-standard meta-information about the error | | errors | source | object | References to the source of the error. | | source | header | string | A string indicating the name of a single request header which caused the error. | | source | parameter | string | A string indicating which URI query parameter caused the error. | | source | pointer | string | A JSON pointer to the value in the request document that caused the error. | | errors | status | string | Status code of the response. | | errors | title | string | Short human-readable summary of the error. | {% /tab %} {% tab title="Example" %} ```json { "errors": [ { "detail": "Missing required attribute in body", "meta": {}, "source": { "header": "Authorization", "parameter": "limit", "pointer": "/data/attributes/title" }, "status": "400", "title": "Bad Request" } ] } ``` {% /tab %} {% /tab %} {% tab title="429" %} Too many requests {% tab title="Model" %} API error response. | Field | Type | Description | | ------------------------ | -------- | ----------------- | | errors [*required*] | [string] | A list of errors. | {% /tab %} {% tab title="Example" %} ```json { "errors": [ "Bad Request" ] } ``` {% /tab %} {% /tab %} {% tab title="500" %} Internal Server Error {% tab title="Model" %} API error response. | Parent field | Field | Type | Description | | ------------ | ------------------------ | -------- | ------------------------------------------------------------------------------- | | | errors [*required*] | [object] | A list of errors. | | errors | detail | string | A human-readable explanation specific to this occurrence of the error. | | errors | meta | object | Non-standard meta-information about the error | | errors | source | object | References to the source of the error. | | source | header | string | A string indicating the name of a single request header which caused the error. | | source | parameter | string | A string indicating which URI query parameter caused the error. | | source | pointer | string | A JSON pointer to the value in the request document that caused the error. | | errors | status | string | Status code of the response. | | errors | title | string | Short human-readable summary of the error. | {% /tab %} {% tab title="Example" %} ```json { "errors": [ { "detail": "Missing required attribute in body", "meta": {}, "source": { "header": "Authorization", "parameter": "limit", "pointer": "/data/attributes/title" }, "status": "400", "title": "Bad Request" } ] } ``` {% /tab %} {% /tab %} ### Code Example ##### \# Required query arguments export config_id="a7c8d9e0-1234-5678-9abc-def012345678" \# Curl command curl -X GET "https://api.datadoghq.com/api/v2/llm-obs/v1/topic-discovery-topics/with-cluster-points?config_id=${config_id}" \ -H "Accept: application/json" \ -H "DD-API-KEY: ${DD_API_KEY}" \ -H "DD-APPLICATION-KEY: ${DD_APP_KEY}" {% /tab %}