--- title: List experiment holdouts description: "List the holdouts an experiment is attached to, in evaluation order: MANAGED holdouts first, starting with the GLOBAL holdout, followed by EXTERNAL…" breadcrumbs: Docs > API > List experiment holdouts --- > For the complete documentation index, see [llms.txt](https://docs.datadoghq.com/llms.txt). # List experiment holdouts {% callout %} This product or feature is not supported for the following sites: app.ddog-gov.com, us2.ddog-gov.com {% /callout %} ## v2 (latest) | Datadog site | API endpoint | | ----------------- | ----------------------------------------------------------------------------- | | app.datadoghq.com | GET https://api.datadoghq.com/api/v2/experiments/{experiment_id}/holdouts | | us3.datadoghq.com | GET https://api.us3.datadoghq.com/api/v2/experiments/{experiment_id}/holdouts | | us5.datadoghq.com | GET https://api.us5.datadoghq.com/api/v2/experiments/{experiment_id}/holdouts | | app.datadoghq.eu | GET https://api.datadoghq.eu/api/v2/experiments/{experiment_id}/holdouts | | ap1.datadoghq.com | GET https://api.ap1.datadoghq.com/api/v2/experiments/{experiment_id}/holdouts | | ap2.datadoghq.com | GET https://api.ap2.datadoghq.com/api/v2/experiments/{experiment_id}/holdouts | | uk1.datadoghq.com | GET https://api.uk1.datadoghq.com/api/v2/experiments/{experiment_id}/holdouts | | app.ddog-gov.com | GET https://api.ddog-gov.com/api/v2/experiments/{experiment_id}/holdouts | | us2.ddog-gov.com | GET https://api.us2.ddog-gov.com/api/v2/experiments/{experiment_id}/holdouts | ### Overview List the holdouts an experiment is attached to, in evaluation order: MANAGED holdouts first, starting with the GLOBAL holdout, followed by EXTERNAL holdouts. For a draft experiment, the response includes the GLOBAL holdout that the experiment would join if it started. The response excludes archived and deleted holdouts, and MANAGED holdouts past their analysis window. EXTERNAL holdouts are listed until they are archived. MANAGED holdouts are listed only for experiments that use a Datadog feature flag, because Datadog serves holdouts only through its flags. The order matches the experiment's `holdout_ids` attribute. This endpoint requires the `product_analytics_experiments_read` permission. OAuth apps require the `product_analytics_experiments_read` authorization [scope](https://docs.datadoghq.com/api/latest/scopes.md#experiments) to access this endpoint. ### Arguments #### Path Parameters | Field | Type | Description | | -------------------------- | ------ | --------------------------- | | experiment_id [*required*] | string | The UUID of the experiment. | ### Response {% tabs %} {% tab label="200" %} OK {% tabs %} {% tab label="Model" %} Response containing the holdouts an experiment is attached to. | Parent field | Field | Type | Description | | ------------ | ------------------ | -------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | | data [*required*] | [object] | Holdouts the experiment is attached to, in evaluation order. | | data | attributes | object | Attributes of a holdout that the experiment is attached to. | | attributes | evaluation_order | int64 | Zero-based position of this MANAGED holdout in the experiment's evaluation order. The GLOBAL holdout is evaluated first. Null for EXTERNAL holdouts. | | attributes | holdout_type | enum | How Datadog serves a MANAGED holdout. GLOBAL holdouts apply to every eligible experiment of a subject type. SELECTIVE holdouts apply only to the experiments attached to them. Null for EXTERNAL holdouts. Allowed enum values: `GLOBAL`, `SELECTIVE`, `UNKNOWN`. | | attributes | key | string | Key of the holdout. | | attributes | kind | enum | Whether Datadog serves the holdout. Datadog serves MANAGED holdouts. EXTERNAL holdouts record what your own feature flagging system holds back, and Datadog does not serve them. Allowed enum values: `MANAGED`, `EXTERNAL`, `UNKNOWN`. | | attributes | name | string | Display name of the holdout. | | attributes | state | enum | Lifecycle phase of the holdout. PENDING holdouts have not started including experiments. INCLUDING holdouts are adding experiments. WAITING_FOR_ANALYSIS holdouts have stopped including experiments but have not started analysis. ANALYSING holdouts are in their analysis window. COMPLETE applies only to EXTERNAL holdouts that have ended their analysis window but are not archived. Allowed enum values: `PENDING`, `INCLUDING`, `WAITING_FOR_ANALYSIS`, `ANALYSING`, `COMPLETE`, `UNKNOWN`. | | attributes | traffic_allocation | double | Share of subjects the holdout holds back, from 0 to 1. | | data | id [*required*] | uuid | Identifier of the holdout. | | data | type [*required*] | enum | Experiment holdout resource type. Allowed enum values: `experiment-holdouts`. Default: `experiment-holdouts`. | {% /tab %} {% tab label="Example" %} ```json { "data": [ { "attributes": { "evaluation_order": 0, "holdout_type": "GLOBAL", "key": "q3-global-holdout", "kind": "MANAGED", "name": "Q3 global holdout", "state": "INCLUDING", "traffic_allocation": 0.05 }, "id": "e1e2e3e4-0000-0000-0000-000000000000", "type": "experiment-holdouts" } ] } ``` {% /tab %} {% /tabs %} {% /tab %} {% tab label="400" %} Malformed experiment ID (not a valid UUID). {% tabs %} {% tab label="Model" %} API error response. | Parent field | Field | Type | Description | | ------------ | ------------------- | -------- | ------------------------------------------------------------------------------- | | | errors [*required*] | [object] | A list of errors. | | errors | detail | string | A human-readable explanation specific to this occurrence of the error. | | errors | meta | object | Non-standard meta-information about the error | | errors | source | object | References to the source of the error. | | source | header | string | A string indicating the name of a single request header which caused the error. | | source | parameter | string | A string indicating which URI query parameter caused the error. | | source | pointer | string | A JSON pointer to the value in the request document that caused the error. | | errors | status | string | Status code of the response. | | errors | title | string | Short human-readable summary of the error. | {% /tab %} {% tab label="Example" %} ```json { "errors": [ { "detail": "Missing required attribute in body", "meta": {}, "source": { "header": "Authorization", "parameter": "limit", "pointer": "/data/attributes/title" }, "status": "400", "title": "Bad Request" } ] } ``` {% /tab %} {% /tabs %} {% /tab %} {% tab label="401" %} Missing or invalid authentication. {% tabs %} {% tab label="Model" %} API error response. | Parent field | Field | Type | Description | | ------------ | ------------------- | -------- | ------------------------------------------------------------------------------- | | | errors [*required*] | [object] | A list of errors. | | errors | detail | string | A human-readable explanation specific to this occurrence of the error. | | errors | meta | object | Non-standard meta-information about the error | | errors | source | object | References to the source of the error. | | source | header | string | A string indicating the name of a single request header which caused the error. | | source | parameter | string | A string indicating which URI query parameter caused the error. | | source | pointer | string | A JSON pointer to the value in the request document that caused the error. | | errors | status | string | Status code of the response. | | errors | title | string | Short human-readable summary of the error. | {% /tab %} {% tab label="Example" %} ```json { "errors": [ { "detail": "Missing required attribute in body", "meta": {}, "source": { "header": "Authorization", "parameter": "limit", "pointer": "/data/attributes/title" }, "status": "400", "title": "Bad Request" } ] } ``` {% /tab %} {% /tabs %} {% /tab %} {% tab label="403" %} The caller lacks the `product_analytics_experiments_read` permission or lacks permission to view this experiment. {% tabs %} {% tab label="Model" %} API error response. | Parent field | Field | Type | Description | | ------------ | ------------------- | -------- | ------------------------------------------------------------------------------- | | | errors [*required*] | [object] | A list of errors. | | errors | detail | string | A human-readable explanation specific to this occurrence of the error. | | errors | meta | object | Non-standard meta-information about the error | | errors | source | object | References to the source of the error. | | source | header | string | A string indicating the name of a single request header which caused the error. | | source | parameter | string | A string indicating which URI query parameter caused the error. | | source | pointer | string | A JSON pointer to the value in the request document that caused the error. | | errors | status | string | Status code of the response. | | errors | title | string | Short human-readable summary of the error. | {% /tab %} {% tab label="Example" %} ```json { "errors": [ { "detail": "Missing required attribute in body", "meta": {}, "source": { "header": "Authorization", "parameter": "limit", "pointer": "/data/attributes/title" }, "status": "400", "title": "Bad Request" } ] } ``` {% /tab %} {% /tabs %} {% /tab %} {% tab label="404" %} No experiment with this ID exists for the organization. {% tabs %} {% tab label="Model" %} API error response. | Parent field | Field | Type | Description | | ------------ | ------------------- | -------- | ------------------------------------------------------------------------------- | | | errors [*required*] | [object] | A list of errors. | | errors | detail | string | A human-readable explanation specific to this occurrence of the error. | | errors | meta | object | Non-standard meta-information about the error | | errors | source | object | References to the source of the error. | | source | header | string | A string indicating the name of a single request header which caused the error. | | source | parameter | string | A string indicating which URI query parameter caused the error. | | source | pointer | string | A JSON pointer to the value in the request document that caused the error. | | errors | status | string | Status code of the response. | | errors | title | string | Short human-readable summary of the error. | {% /tab %} {% tab label="Example" %} ```json { "errors": [ { "detail": "Missing required attribute in body", "meta": {}, "source": { "header": "Authorization", "parameter": "limit", "pointer": "/data/attributes/title" }, "status": "400", "title": "Bad Request" } ] } ``` {% /tab %} {% /tabs %} {% /tab %} {% tab label="429" %} Too many requests {% tabs %} {% tab label="Model" %} API error response. | Field | Type | Description | | ------------------- | -------- | ----------------- | | errors [*required*] | [string] | A list of errors. | {% /tab %} {% tab label="Example" %} ```json { "errors": [ "Bad Request" ] } ``` {% /tab %} {% /tabs %} {% /tab %} {% /tabs %} ### Code Example {% tabs %} {% tab label="Curl" %} ```bash # Path parameters export experiment_id="550e8400-e29b-41d4-a716-446655440000" # Set your Datadog site and credentials export DD_SITE="datadoghq.com" # Or us3.datadoghq.com, us5.datadoghq.com, datadoghq.eu, ap1.datadoghq.com, ap2.datadoghq.com, uk1.datadoghq.com, ddog-gov.com, us2.ddog-gov.com # Use a Personal Access Token or Service Access Token export DD_BEARER_TOKEN="" curl -X GET "https://api.${DD_SITE}/api/v2/experiments/${experiment_id}/holdouts" \ -H "Accept: application/json" \ -H "Authorization: Bearer ${DD_BEARER_TOKEN}" ``` {% /tab %} {% /tabs %}