Connect BigQuery for Warehouse-Native Experiment Analysis

This product is not supported for your selected Datadog site. ().
このページは日本語には対応しておりません。随時翻訳に取り組んでいます。
翻訳に関してご質問やご意見ございましたら、お気軽にご連絡ください。

Overview

Warehouse-native experiment analysis lets you run statistical computations directly in your data warehouse.

To set this up for BigQuery, connect a BigQuery service account to Datadog and configure your experiment settings. This guide covers:

Prerequisites

Datadog connects to BigQuery through a Google Cloud service account. If you already have a service account connected to Datadog, skip to Step 1. Otherwise, expand the section below to create one.

  1. Open your Google Cloud console.
  2. Navigate to IAM & Admin > Service Accounts.
  3. Click Create service account.
  4. Enter the following:
    1. Service account name.
    2. Service account ID.
    3. Service account description.
  5. Click Create and continue.
    1. Note: The Permissions and Principals with access settings are optional here. These are configured in Step 2.
  6. Click Done.

After you create the service account, continue to Step 1 to set up the Google Cloud resources.

If you plan to use other Google Cloud observability functionality in Datadog, see Datadog's Google Cloud Platform integration documentation to determine which resources to enable.

Step 1: Prepare the Google Cloud resources

Datadog Experiments uses a BigQuery dataset for caching experiment results and a Cloud Storage bucket for staging experiment records.

Create a BigQuery dataset

  1. Open your Google Cloud console.
  2. In the Search bar, search for BigQuery.
  3. In the Explorer panel, expand your project (for example, datadog-sandbox).
  4. Select Datasets, then click Create dataset.
    The BigQuery Datasets page in the Google Cloud console showing the datadog-sandbox project expanded in the left Explorer menu with Datasets selected, a list of datasets with columns for Dataset ID, Type, Location, Create time, and Label, and the Create dataset button highlighted in the top right.
  5. Enter a Dataset ID (for example, datadog_experiments_output).
  6. (Optional) Select a Data location from the dropdown, add Tags, and set Advanced options.
  7. Click Create dataset.

Create a Cloud Storage bucket

Create a Cloud Storage bucket that Datadog Experiments can use to stage experiment exposure records. See Google’s Create a bucket documentation.

Step 2: Grant permissions to the Datadog service account

The Datadog Experiments service account requires specific permissions to run warehouse-native experiment analysis.

Assign IAM roles at the project level

To assign IAM roles so Datadog Experiments can read and write data, and run jobs in your data warehouse:

  1. Open your Google Cloud console and navigate to IAM & Admin > IAM.
  2. Select the Allow tab and click Grant access.
  3. In the New principals field, enter the service account email.
  4. Using the Select a role dropdown, add the following roles:
    1. BigQuery Job User: Allows the service account to run BigQuery jobs.
    2. BigQuery Data Owner: Grants the service account full access to the Datadog Experiments output dataset.
    3. Storage Object User: Allows the service account to read and write objects in the storage bucket that Datadog Experiments uses.
    4. BigQuery Data Viewer: Allows the service account to read tables used in warehouse-native metrics.
  5. Click Save.
The Google Cloud IAM page showing the Grant access panel for a project, with the Grant access button highlighted on the left, a New principals field highlighted in the Add principals section, and a Select a role dropdown highlighted in the Assign roles section.

Grant read access to specific source tables

Repeat the following steps for each dataset you plan to use for experiment metrics:

  1. In the Google Cloud console Search bar, search for BigQuery.
  2. In the Explorer panel, expand your project (for example, datadog-sandbox).
  3. Click Datasets, then select the dataset containing your source tables.
  4. Click the Share dropdown and select Manage permissions.
    The BigQuery dataset page with the Share dropdown expanded and Manage permissions highlighted, showing additional options including Copy link, Authorize Views, Authorize Routines, Authorize Datasets, Manage Subscriptions, and Publish as Listing.
  5. Click Add principal.
  6. In the New principals field, enter the service account email.
  7. Using the Select a role dropdown, select the BigQuery Data Viewer role.
  8. Click Save.

Step 3: Configure experiment settings

Datadog supports one warehouse connection per organization. Connecting BigQuery replaces any existing warehouse connection (for example, Snowflake).

After you set up your Google Cloud resources and IAM roles, configure the experiment settings in Datadog:

  1. Open Datadog Product Analytics.
  2. In the left navigation, hover over Settings and click Experiments.
  3. Select the Warehouse Connections tab.
  4. Click Connect a data warehouse. If you already have a warehouse connected, click Edit instead.
  5. Select the BigQuery tile.
  6. Under Select BigQuery Account, enter:
    • GCP service account: The service account you are using for Datadog Experiments.
    • Project: Your Google Cloud project.
  7. Under Dataset and GCS Bucket, enter:
    • Dataset: The dataset you created in Step 1 (for example, datadog_experiments_output).
    • GCS Bucket: The Cloud Storage bucket you created in Step 1.
  8. Click Save.
The Edit Data Warehouse modal with BigQuery selected, showing two sections: Select BigQuery Account with fields for GCP service account and Project, and Dataset and GCS Bucket with fields for Dataset and GCS Bucket.

After you save your warehouse connection, create experiment metrics using your BigQuery data.

Further reading

お役に立つドキュメント、リンクや記事:

Defining metrics in Datadog ExperimentsDOCUMENTATION more more How to bridge speed and quality in experiments through unified dataBLOG more more