To set this up for Databricks, connect a Databricks service account to Datadog and configure your experiment settings. This guide covers:

• Granting permissions to the service principal
• Connecting Databricks to Datadog
• Configuring experiment settings in Datadog

Prerequisites

Datadog Experiments connects to Databricks through the Datadog Databricks integration. If you already have a Databricks integration configured for the workspace you plan to use, skip to Step 1. Otherwise, expand the section below to create a service principal.

Step 1: Grant permissions to the service principal

In your Databricks Workspace, open the SQL Editor to run the following commands and grant the service principal permissions for warehouse-native experiment analysis.

Grant read access to source tables

Grant the service principal read access to the tables containing your experiment metrics. Run both GRANT USE commands, then run the GRANT SELECT option that matches your access needs. Replace <catalog>, <schema>, <table>, and <principal> with the appropriate values.

GRANT USE CATALOG ON CATALOG <catalog> TO `<principal>`;
GRANT USE SCHEMA ON SCHEMA <catalog>.<schema> TO `<principal>`;

-- Option 1: Give read access to a single table
GRANT SELECT ON TABLE <catalog>.<schema>.<table> TO `<principal>`;

-- Option 2: Give read access to all tables in the schema
GRANT SELECT ON ALL TABLES IN SCHEMA <catalog>.<schema> TO `<principal>`;

Create an output schema

Run the following commands to create a schema where Datadog Experiments can write intermediate results and temporary tables. Replace datadog_experiments_output with your output schema name, and <catalog> and <principal> with the appropriate values.

CREATE SCHEMA IF NOT EXISTS <catalog>.datadog_experiments_output;
GRANT USE SCHEMA ON SCHEMA <catalog>.datadog_experiments_output TO `<principal>`;
GRANT CREATE TABLE ON SCHEMA <catalog>.datadog_experiments_output TO `<principal>`;

Configure a volume for temporary data staging

Datadog Experiments uses a volume to temporarily save exposure data before copying it into a Databricks table. Run the following commands to create and grant access to this volume. Replace datadog_experiments_output with your output schema name, and <catalog> and <principal> with the appropriate values.

CREATE VOLUME IF NOT EXISTS <catalog>.datadog_experiments_output.datadog_experiments_volume;
GRANT READ VOLUME ON VOLUME <catalog>.datadog_experiments_output.datadog_experiments_volume TO `<principal>`;
GRANT WRITE VOLUME ON VOLUME <catalog>.datadog_experiments_output.datadog_experiments_volume TO `<principal>`;

Grant SQL warehouse access

Grant the service principal access to the SQL warehouse that Datadog Experiments uses to run queries.

1. Navigate to SQL Warehouses in your Databricks Workspace.
2. Select the warehouse for Datadog Experiments.
3. At the top right corner, click Permissions.
4. Grant the service principal the Can use permission.
5. Close the Manage permissions modal.

Step 2: Connect Databricks to Datadog

To connect your Databricks Workspace to Datadog for warehouse-native experiment analysis:

1. Navigate to Datadog's integrations page and search for Databricks.
2. Click the Databricks tile to open its modal.
3. Select the Configure tab and click Add Databricks Workspace. If this is your first Databricks account, the setup form appears automatically.
4. Under the Connect a new Databricks Workspace section, enter:
   • Workspace Name.
   • Workspace URL.
   • Client ID.
   • Client Secret.
   • System Tables SQL Warehouse ID.
5. Toggle off Jobs Monitoring and all other products.
6. Toggle off the Metrics - Model Serving resource.
7. Click Save Databricks Workspace.

To set this up for Amazon Redshift, connect a Redshift cluster to Datadog using the AWS integration and configure your experiment settings. This guide covers:

• Preparing the Redshift cluster
• Creating AWS resources and granting IAM permissions
• Configuring experiment settings in Datadog

Prerequisites

Datadog Experiments connects to Redshift through Datadog's Amazon Web Services (AWS) integration. If you already have the AWS integration configured for the account containing your Redshift cluster, skip to Step 1.

Step 1: Prepare the Redshift cluster

Create a Datadog service user and a dedicated schema for Datadog to store experiment results and intermediate tables.

Create a Datadog service user in your Redshift database

Run the following command to create a service user with a strong password that Datadog can use to execute queries. Replace datadog_experiments_user with your user value and Your_Strong_Password with your password.

CREATE USER datadog_experiments_user PASSWORD 'Your_Strong_Password';

Create a Redshift output schema

Run the following commands to create a schema where Datadog can store experiment results and intermediate tables. Replace datadog_experiments_output with your schema name and datadog_experiments_user with your service user value.

CREATE SCHEMA IF NOT EXISTS datadog_experiments_output;
GRANT ALL ON SCHEMA datadog_experiments_output TO datadog_experiments_user;

Grant the service user read access to your metric data

Grant the service user read access to the tables or schemas that contain your source data. These are the tables you plan to use for experiment metrics, and are typically in a different schema than the output schema created above. Run the GRANT USAGE command, then run the GRANT SELECT option that matches your access needs. Replace datadog_experiments_user, <schema>, and <table> with the appropriate values.

GRANT USAGE ON SCHEMA <schema> TO datadog_experiments_user;

-- Option 1: Give read access to a single table
GRANT SELECT ON TABLE <schema>.<table> TO datadog_experiments_user;

-- Option 2: Give read access to all tables in the schema
GRANT SELECT ON ALL TABLES IN SCHEMA <schema> TO datadog_experiments_user;

Step 2: Create AWS resources and grant IAM permissions

Create an S3 bucket

Create an S3 bucket for importing exposure events into your warehouse. The bucket name must start with datadog-experimentation- (for example, datadog-experimentation-[aws_account_id]). You can use the bucket's default settings.

Grant additional IAM permissions

In addition to the permissions listed in the AWS integration documentation, Datadog Experiments requires additional IAM permissions to run warehouse-native experiment analysis.

Use the following table to gather the values for your environment, then add the policy statement below to the IAM role that your Datadog AWS integration uses.

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "RedshiftGetClusterCredentials",
      "Effect": "Allow",
      "Action": [
        "redshift:GetClusterCredentials"
      ],
      "Resource": [
        "[Redshift cluster ARN]",
        "[Redshift user ARN]",
        "[Redshift database ARN]"
      ]
    },
    {
      "Sid": "QueryRedshift",
      "Effect": "Allow",
      "Action": [
        "redshift-data:ExecuteStatement",
        "redshift-data:GetStatementResult",
        "redshift-data:DescribeStatement",
        "redshift-data:ListStatements",
        "redshift-data:CancelStatement"
      ],
      "Resource": "*"
    },
    {
      "Sid": "ListExperimentationBucket",
      "Effect": "Allow",
      "Action": [
        "s3:ListBucket"
      ],
      "Resource": "[S3 bucket ARN]"
    },
    {
      "Sid": "ReadWriteExperimentationBucket",
      "Effect": "Allow",
      "Action": [
        "s3:GetObject",
        "s3:PutObject",
        "s3:DeleteObject"
      ],
      "Resource": "[S3 bucket ARN]/*"
    }
  ]
}

To set this up for Snowflake, connect a Snowflake service account to Datadog and configure your experiment settings. This guide covers:

• Preparing a Snowflake service account
• Connecting it to Datadog
• Configuring experiment settings

Step 1: Prepare the Snowflake service account

The examples in this guide use datadog_experiments_user and datadog_experiments_role as the service account's user and role. Replace these with your own values.

Create a dedicated service user and role in Snowflake

1. Use the Snowflake documentation to create a public-private key pair for enhanced authentication. Datadog only supports unencrypted private keys.
2. Run the following commands in Snowflake to create the user and role in the service account. Replace <public_key> with the public key you generated in the previous step.

USE ROLE ACCOUNTADMIN;
CREATE ROLE IF NOT EXISTS datadog_experiments_role;
CREATE USER IF NOT EXISTS datadog_experiments_user
    RSA_PUBLIC_KEY = '<public_key>';
GRANT ROLE datadog_experiments_role TO USER datadog_experiments_user;
ALTER USER datadog_experiments_user SET DEFAULT_ROLE = datadog_experiments_role;

Grant privileges to the role

1. Identify the tables in Snowflake from which you intend to create metrics.
2. Run the following commands to grant read privileges to the new role, replacing <database>, <schema>, and <table> with their appropriate values. Run both GRANT USAGE commands, then run the GRANT SELECT option or options that match your access needs.

GRANT USAGE ON DATABASE <database> TO ROLE datadog_experiments_role;
GRANT USAGE ON SCHEMA <database>.<schema> TO ROLE datadog_experiments_role;

-- Option 1: Give read access to a single table
GRANT SELECT ON TABLE <database>.<schema>.<table> TO ROLE datadog_experiments_role;

-- Option 2: Give read access to all existing tables in the schema
GRANT SELECT ON ALL TABLES IN SCHEMA <database>.<schema> TO ROLE datadog_experiments_role;

-- Option 3: Give read access to all future tables in the schema
GRANT SELECT ON FUTURE TABLES IN SCHEMA <database>.<schema> TO ROLE datadog_experiments_role;

Grant the role access to the output schema

Datadog writes experiment exposure logs and intermediate metric results to tables in a dedicated output schema. Run the following commands to create the schema and grant the role full access. Replace <database> with the appropriate value.

CREATE SCHEMA IF NOT EXISTS <database>.datadog_experiments_output;
GRANT ALL ON SCHEMA <database>.datadog_experiments_output TO ROLE datadog_experiments_role;
GRANT ALL PRIVILEGES ON FUTURE TABLES IN SCHEMA <database>.datadog_experiments_output TO ROLE datadog_experiments_role;

Create a dedicated warehouse for Datadog Experiments (optional)

Creating a dedicated warehouse for Datadog Experiments is optional. Run the following commands to create one. Replace <wh_size> with the appropriate value.

CREATE WAREHOUSE IF NOT EXISTS datadog_experiments_wh
    WAREHOUSE_SIZE = <wh_size>
    AUTO_SUSPEND = 300
    INITIALLY_SUSPENDED = true;
GRANT ALL PRIVILEGES ON WAREHOUSE datadog_experiments_wh TO ROLE datadog_experiments_role;

Step 2: Connect Snowflake to Datadog

To connect your Snowflake account to Datadog for warehouse-native experiment analysis:

1. Navigate to Datadog's integrations page and search for Snowflake.
2. Click the Snowflake tile to open its modal.
3. Select the Configure tab and click Add Snowflake Account.
4. Add your Account URL. To find your account URL, see the Snowflake guide.
5. Toggle off all resources (these are not needed for experiment analysis).
6. Enter the Snowflake User Name you created in Step 1 (for example, datadog_experiments_user).
7. Scroll to the Configure a key pair authentication section and upload your unencrypted private key.
8. Click Save.

After you set up your Databricks integration and workspace, configure the experiment settings in Datadog:

1. Open Datadog Product Analytics.
2. In the left navigation, hover over Settings and click Experiments.
3. Select the Warehouse Connections tab.
4. Click Connect a data warehouse. If you already have a warehouse connected, click Edit instead.
5. Select the Databricks tile.
6. Using the Account dropdown, select the Databricks Workspace you configured in Step 2.
7. Enter the Catalog, Schema, and Volume name you configured in Step 1. If your catalog and schema do not appear in the dropdown, enter them manually to add them to the list.
8. Click Save.

After you save your warehouse connection, create experiment metrics using your Databricks data.

After you set up your AWS integration and Redshift cluster, configure the experiment settings in Datadog:

1. Open Datadog Product Analytics.
2. In the left navigation, hover over Settings and click Experiments.
3. Select the Warehouse Connections tab.
4. Click Connect a data warehouse. If you already have a warehouse connected, click Edit instead.
5. Select the Redshift tile.
6. Select your AWS account from the dropdown.
7. Under Cluster Connection, enter:
   • AWS region: The region your Redshift cluster is in (for example, us-east-1).
   • Cluster identifier: The name of your Redshift cluster.
   • Cluster endpoint: The full endpoint URL for your cluster.
   • Port: The port your cluster is listening on (default: 5439).
8. Under Database and Storage, enter:
   • Database: The name of the database containing your source tables.
   • Database user: The service user you created in Step 1 (for example, datadog_experiments_user).
   • Schema: The schema you created in Step 1 for Datadog Experiments to write to (for example, datadog_experiments_output).
   • Temp S3 bucket: The S3 bucket you created in Step 2 (for example, datadog-experimentation-[aws_account_id]).
9. Click Save.

After you save your warehouse connection, create experiment metrics using your Redshift data.

After you set up your Snowflake integration, configure the experiment settings in Datadog Product Analytics:

1. In the left navigation, hover over Settings, then click Experiments.
2. Select the Warehouse Connections tab.
3. Click Connect a data warehouse. If you already have a warehouse connected, click Edit instead.
4. Select the Snowflake tile.
5. Enter the Account, Role, Warehouse, Database, and Schema you configured in Step 1. If your database and schema do not appear in the dropdown, enter them manually to add them to the list.
6. Click Save.

After you save your warehouse connection, create experiment metrics using your Snowflake data.