--- title: BigQuery description: >- Connect BigQuery to Datadog Data Observability to monitor data quality, track usage, and detect issues. breadcrumbs: >- Docs > Data Observability Overview > Quality Monitoring > Warehouse Integrations > BigQuery --- # BigQuery {% callout %} # Important note for users on the following Datadog sites: app.ddog-gov.com {% alert level="danger" %} This product is not supported for your selected [Datadog site](https://docs.datadoghq.com/getting_started/site). (). {% /alert %} {% /callout %} ## Overview{% #overview %} The BigQuery integration connects Datadog to your Google Cloud project to sync metadata, query history, and table-level metrics. Use it to monitor data freshness, detect anomalies, and trace lineage across your data stack. ## Prerequisites{% #prerequisites %} If your Google Cloud project restricts network access by IP, add the Datadog webhook IPs to your allowlist. For the list of IPs, see the `webhooks` section of IP ranges list. ## Configure the BigQuery integration in Datadog{% #configure-the-bigquery-integration-in-datadog %} To configure the BigQuery integration in Datadog: 1. Navigate to [**Datadog Data Observability** > **Settings**](https://app.datadoghq.com/datasets/settings/integrations). 1. Click the **Configure** button for the BigQuery option. You can now choose to use a service account that you've already connected to Datadog or set up a new one specifically for Data Observability. ### Using an existing service account{% #using-an-existing-service-account %} Choose this option if you've previously connected a service account to Datadog. 1. Select the **Use connected Service Account** option. 1. Select the service account from the dropdown list. 1. Click **Next**. 1. Turn on the **Enable Data Observability** toggle and click **Add Account**. Data Observability functionality requires additional roles and API access beyond the basic BigQuery integration. #### Additional roles{% #additional-roles %} Apply the following roles to the service account in your GCP IAM console. | Role | Role ID | Description | | --------------------------------------------------------------------------------------------------------- | ------------------------------- | ------------------------------------ | | [BigQuery Data Viewer](https://cloud.google.com/bigquery/docs/access-control#bigquery.dataViewer) | `roles/bigquery.dataViewer` | Provides visibility into datasets | | [BigQuery Resource Viewer](https://cloud.google.com/bigquery/docs/access-control#bigquery.resourceViewer) | `roles/bigquery.resourceViewer` | Provides visibility into jobs | | [Job User](https://cloud.google.com/bigquery/docs/access-control#bigquery.jobUser) | `roles/bigquery.jobUser` | Required to run data quality queries | #### Additional APIs{% #additional-apis %} Make sure the following APIs have been enabled in the project associated with the service account. | API | API ID | Description | | --------------------------------------------------------------------- | ------------------------- | --------------------------- | | [BigQuery API](https://cloud.google.com/bigquery/docs/reference/rest) | `bigquery.googleapis.com` | Required to access BigQuery | ### Using a new service account{% #using-a-new-service-account %} 1. Select the **Add new Service Account** option. 1. Choose one of the setup methods. **Quick Start** is recommended for most users since it requires no manual permissions setup. After you've completed the **Quick Start** flow, click **Next** to proceed to the next page and enable the **Enable Data Observability** toggle before clicking **Add Account**. #### Manual setup{% #manual-setup %} In your Google Cloud console's IAM page, create a service account with the following roles: | Role | Role ID | Description | | --------------------------------------------------------------------------------------------------------- | ------------------------------- | ------------------------------------------------------------------------------------------- | | [Monitoring Viewer](https://cloud.google.com/iam/docs/roles-permissions/monitoring#monitoring.viewer) | `roles/monitoring.viewer` | Provides read-only access to the monitoring data available in your Google Cloud environment | | [Cloud Asset Viewer](https://cloud.google.com/iam/docs/roles-permissions/cloudasset#cloudasset.viewer) | `roles/cloudasset.viewer` | Provides read-only access to cloud assets metadata | | [Browser](https://cloud.google.com/iam/docs/roles-permissions/browser#browser) | `roles/browser` | Provides read-only access to browse the hierarchy of a project | | [BigQuery Data Viewer](https://cloud.google.com/bigquery/docs/access-control#bigquery.dataViewer) | `roles/bigquery.dataViewer` | Provides visibility into datasets | | [BigQuery Resource Viewer](https://cloud.google.com/bigquery/docs/access-control#bigquery.resourceViewer) | `roles/bigquery.resourceViewer` | Provides visibility into jobs | | [Job User](https://cloud.google.com/bigquery/docs/access-control#bigquery.jobUser) | `roles/bigquery.jobUser` | Required to run data quality queries | | [Compute Viewer](https://cloud.google.com/compute/docs/access/iam#compute.viewer) | `roles/compute.viewer` | Provides read-only access to get and list Compute Engine resources | In Datadog, after you've selected the **Manual** setup method: 1. Copy the generated **Datadog Principal** to your clipboard. If this is your first time configuration GCP you'll need to first click the **Generate Principal** button. 1. In the Google Cloud console, under **Service Accounts**, find the service account you recently created and click on it. 1. Navigate to the **Principals with access** tab and click the **Grant access** button. 1. Paste the **Datadog Principal** into the **New principals** text box. 1. Under the **Assign roles** section, select the **Service Account Token Creator** role. This will allow the Datadog Principal to impersonate the service account you just created. 1. Click the **Save** button. Finally, you need to enable the following APIs for **every project** you want to monitor, including the project where the service account has been created. | API | API ID | Description | | -------------------------------------------------------------------------------------- | ------------------------------------- | -------------------------------------------------------------------------------------------- | | [Cloud Monitoring API](https://cloud.google.com/monitoring/api/enable-api) | `monitoring.googleapis.com` | Allows Datadog to query your Google Cloud metric data | | [Cloud Asset API](https://cloud.google.com/asset-inventory/docs/reference/rest) | `cloudasset.googleapis.com` | Allows Datadog to request Google Cloud resources and link relevant labels to metrics as tags | | [Compute Engine API](https://cloud.google.com/compute/docs/reference/rest/v1) | `compute.googleapis.com` | Allows Datadog to discover compute instance data | | [Cloud Resource Manager API](https://cloud.google.com/resource-manager/reference/rest) | `cloudresourcemanager.googleapis.com` | Allows Datadog to append metrics with the correct resources and tags | | [BigQuery API](https://cloud.google.com/bigquery/docs/reference/rest) | `bigquery.googleapis.com` | Required to access BigQuery | Once the service account has been created and the necessary roles and APIs have been applied, you can return to Datadog. 1. Click **Verify and save account**. 1. Click **Next**. 1. Turn on the **Enable Data Observability** toggle and click **Add Account**. ## Next steps{% #next-steps %} After you configure the integration, Datadog begins syncing your information schema and query history in the background. Initial syncs can take several hours depending on the size of your BigQuery deployment. After the initial sync completes, create a [Data Observability monitor](https://docs.datadoghq.com/monitors/types/data_observability/) to start alerting on freshness, row count, column-level metrics, and custom SQL metrics. ## Further reading{% #further-reading %} - [Learn about Data Observability](https://docs.datadoghq.com/data_observability/) - [Data Observability Monitors](https://docs.datadoghq.com/monitors/types/data_observability/)