---
title: Experiments
description: >-
  Plan, run, and analyze randomized experiments across your stack with Datadog
  Experiments.
breadcrumbs: Docs > Experiments
---

> For the complete documentation index, see [llms.txt](https://docs.datadoghq.com/llms.txt).

# Experiments

{% callout %}
# Important note for users on the following Datadog sites: app.ddog-gov.com, us2.ddog-gov.com

{% alert level="danger" %}
This product is not supported for your selected [Datadog site](https://docs.datadoghq.com/getting_started/site.md). ({% placeholder "user-datadog-site-name" /%}).
{% /alert %}

{% /callout %}

## Overview{% #overview %}

Datadog Experiments is a composable platform for end-to-end experimentation. An experiment in Datadog consists of two components:

1. A **randomized assignment** of [subjects](https://docs.datadoghq.com/experiments/concepts/subject_types.md) (typically users) to two or more variations—either from a [Datadog Feature Flag](https://docs.datadoghq.com/feature_flags.md), or your randomization system of choice
1. A set of **metrics** to compare between variants—computed either within Datadog or with warehouse native analytics.

To get started, select a link from the table below. Otherwise, read on to learn more about Datadog Experiments.

| Quick Links                                                                                                                                                |
| ---------------------------------------------------------------------------------------------------------------------------------------------------------- |
| [Connect a data warehouse](https://docs.datadoghq.com/experiments/guide/connecting_a_data_warehouse.md)                                                    | Set up Snowflake, BigQuery, Redshift, or Databricks for warehouse-native experiment analysis |
| [Create a warehouse-native metric](https://docs.datadoghq.com/experiments/defining_metrics.md?tab=warehouse)                                               | Define Metric SQL Models and experiment metrics from warehouse data                          |
| [Create a metric from Product Analytics or Real User Monitoring data](https://docs.datadoghq.com/experiments/defining_metrics.md?tab=productanalyticsorum) | Build experiment metrics from client-side RUM and Product Analytics events                   |
| [Launch an experiment using Datadog Feature Flags](https://docs.datadoghq.com/experiments/plan_and_launch_experiments.md)                                  | Plan your hypothesis, configure randomization with Feature Flags, and start your experiment  |
| [Analyze an experiment that's already been randomized](https://docs.datadoghq.com/experiments/concepts/exposure_sql.md)                                    | Define exposure data in your warehouse when randomization runs outside Datadog Feature Flags |
| [Understand experiment diagnostics](https://docs.datadoghq.com/experiments/diagnostics.md)                                                                 | Interpret automated checks for exposures, metrics, randomization, and analysis health        |

## Randomization{% #randomization %}

Every experiment needs a way to assign subjects to a control or treatment variant. Datadog supports two approaches.

### Datadog Feature Flags{% #datadog-feature-flags %}

[Datadog Feature Flags](https://docs.datadoghq.com/feature_flags.md) is the default way to randomize experiments. Create a flag, implement it with the [Feature Flags SDK](https://docs.datadoghq.com/getting_started/feature_flags.md#feature-flags-sdks), and pass a stable subject identifier as the `targetingKey` so the same user always receives the same variant. Datadog uses deterministic hashing to keep assignments consistent across sessions and devices.

When you [plan and launch an experiment](https://docs.datadoghq.com/experiments/plan_and_launch_experiments.md), link it to a feature flag to define traffic splits, targeting rules, and rollout behavior. You can also create an experiment directly from a flag's detail page. To randomize by a unit other than user—for example, an organization—see [Subject Types](https://docs.datadoghq.com/experiments/concepts/subject_types.md).

### Bring your own randomization{% #bring-your-own-randomization %}

If you randomize subjects outside Datadog—for example, with an in-house system—use [Exposure SQL Models](https://docs.datadoghq.com/experiments/concepts/exposure_sql.md) to tell Datadog who was exposed to each experiment and when. Exposure SQL Models query exposure records from your [connected warehouse](https://docs.datadoghq.com/experiments/guide/connecting_a_data_warehouse.md) and map them to Datadog fields such as subject key, timestamp, experiment ID, and variant ID.

Datadog deduplicates exposure data automatically: if a user appears in multiple variants for the same experiment, that user is excluded from the analysis. When exposures come from your warehouse instead of Feature Flags, metrics built on Datadog SDK events are not supported—you need [warehouse-native metrics](https://docs.datadoghq.com/experiments/defining_metrics.md?tab=warehouse).

## Metrics{% #metrics %}

Experiment metrics define what you measure to decide whether a change succeeded. Create at least one primary metric before launching an experiment, and add secondary metrics as guardrails for unintended effects on performance, engagement, or revenue.

### Warehouse native mode{% #warehouse-native-mode %}

In warehouse native mode, Datadog runs experiment analysis directly in Snowflake, BigQuery, Redshift, or Databricks. After you [connect your warehouse](https://docs.datadoghq.com/experiments/guide/connecting_a_data_warehouse.md), create a **Metric SQL Model** that maps warehouse tables to Datadog, then define metrics from that model. Map each model to one or more [subject types](https://docs.datadoghq.com/experiments/concepts/subject_types.md) and specify a timestamp column so Datadog can join metric events to experiment exposures.

Warehouse mode is required when you use [Exposure SQL Models](https://docs.datadoghq.com/experiments/concepts/exposure_sql.md) for randomization. It also suits teams whose source of truth for business metrics already lives in the warehouse.

### Product Analytics and RUM{% #product-analytics-and-rum %}

For client-side experiments, build metrics from events collected by the [Real User Monitoring (RUM)](https://docs.datadoghq.com/real_user_monitoring.md) and [Product Analytics](https://docs.datadoghq.com/product_analytics.md#getting-started) SDKs. Define metrics from actions, views, sessions, and other event types, then choose an aggregation method such as count of events, count of unique users, or sum of a property.

This path works when randomization runs through [Datadog Feature Flags](https://docs.datadoghq.com/feature_flags.md) and you want to measure user behavior, funnel conversion, or application performance without querying a warehouse. Product Analytics and RUM metrics are available in near real time as experiments launch.

## Statistics{% #statistics %}

Datadog applies statistical analysis to compare variants and estimate lift. When you set up an experiment, choose an [analysis method](https://docs.datadoghq.com/experiments/statistics/analysis_methods.md)—sequential frequentist, fixed-sample frequentist, or Bayesian—and optionally run a [sample size calculation](https://docs.datadoghq.com/experiments/plan_and_launch_experiments.md#run-a-sample-size-calculation-optional) to estimate how long the experiment needs to run. After results are in, use [Global Lift](https://docs.datadoghq.com/experiments/statistics/global_lift.md) to understand how a targeted experiment lift translates to impact on your company-wide metric total, and [Cumulative Impact](https://docs.datadoghq.com/experiments/concepts/cumulative_impact.md) to aggregate noise-adjusted effects across many experiments on the same metric.

{% image
   source="https://docs.dd-static.net/images/product_analytics/experiment/overview_metrics_view-1.55e7b161fbea40fa077982ddd509c40f.png?auto=format&fit=max&w=850 1x, https://docs.dd-static.net/images/product_analytics/experiment/overview_metrics_view-1.55e7b161fbea40fa077982ddd509c40f.png?auto=format&fit=max&w=850&dpr=2 2x"
   alt="The Experiments metrics view showing business, funnel, and performance metrics with control and variant values and relative lift for each metric. A tooltip is open on the Revenue metric showing Non-CUPED values for Revenue per User, Total Revenue, and User Assignment Count across the control and variant groups." /%}

## Further reading{% #further-reading %}

- [Measure the business impact of every product change with Datadog Experiments](https://www.datadoghq.com/blog/experiments)
- [Feature Flags](https://docs.datadoghq.com/feature_flags.md)
- [Product Analytics](https://docs.datadoghq.com/product_analytics.md)