---
title: dbt
description: >-
  Connect dbt Cloud or dbt Core to Datadog for job run metadata and model
  lineage.
breadcrumbs: 'Docs > Data Observability Overview > Data Observability: Jobs Monitoring > dbt'
---

# dbt

{% callout %}
# Important note for users on the following Datadog sites: app.ddog-gov.com

{% alert level="danger" %}
This product is not supported for your selected [Datadog site](https://docs.datadoghq.com/getting_started/site). ().
{% /alert %}

{% /callout %}

## Overview{% #overview %}

Datadog can access your dbt Cloud or dbt Core metadata to extract information about job runs, including run durations, models generated by dbt, and lineage relationships between models. Datadog matches tables in your warehouse with dbt models to determine the causality and consequences of table failure.

{% tab title="dbt Cloud" %}
Follow the steps below to connect dbt Cloud to Datadog.

## Generate an API token in dbt Cloud{% #generate-an-api-token-in-dbt-cloud %}

Create a service token in dbt Cloud so Datadog can access your account's metadata.

1. In dbt Cloud, go to **User Profile > API Tokens > Service Tokens**.
1. Click on **+ Create Service Token**.
1. Provide a name for the token.
1. Set the token permissions:
   - For dbt Cloud Enterprise plan, ensure that the token has **Developer** permissions.
   - For dbt Cloud Team plan, ensure that the token has **Account Admin** permissions.
1. Click **Save** and copy the generated API token.

## Connect your dbt Cloud account to Datadog{% #connect-your-dbt-cloud-account-to-datadog %}

Use the API token to configure the integration in Datadog.

1. Navigate to Datadog's [dbt Cloud integration tile](https://app.datadoghq.com/integrations/dbt-cloud).
1. If you have already created a dbt Cloud integration account, make sure you have updated it with the API token with permissions described above.
1. If not, create a new account. Fill in the **Account Name**, **Account Id**, **Account Url**, and **API Token** sections.
1. Click **Save** to save your settings.

## What's next{% #whats-next %}

After your next dbt job run, you should start seeing job run and lineage data in [Datadog Data Observability](https://app.datadoghq.com/datasets/catalog?integration=dbt), as shown below.

{% image
   source="https://datadog-docs.imgix.net/images/data_observability/data-obs-dbt-cloud-final.0a9482f0c463f82642a440c622b35e28.png?auto=format"
   alt="Data Observability overview showing dbt job runs as a stacked bar chart over time and a table of connected dbt Cloud accounts with their status." /%}

{% /tab %}

{% tab title="dbt Core" %}
Follow the steps below to connect dbt Core to Datadog.

**Note**: If you run dbt Core with an external orchestrator (such as Airflow) and want to correlate orchestrator tasks with dbt runs, follow the [Airflow integration instructions](https://docs.datadoghq.com/data_jobs/airflow/?tab=kubernetes) first.

## Retrieve your Datadog API key{% #retrieve-your-datadog-api-key %}

1. [Follow these instructions](https://docs.datadoghq.com/account_management/api-app-keys/#add-an-api-key-or-client-token) to create or retrieve a Datadog API key.

## Install openlineage-dbt{% #install-openlineage-dbt %}

1. Install the `openlineage-dbt` package. Reference [Using dbt with Amazon MWAA](https://docs.aws.amazon.com/mwaa/latest/userguide/samples-dbt.html) for setting up this package in your virtual environment.

   ```shell
   pip3 install openlineage-dbt>=1.39.0
   ```

## Set the environment variables{% #set-the-environment-variables %}

1. Set the following environment variables. Replace `datadoghq.com` with the relevant [Datadog site](https://docs.datadoghq.com/getting_started/site/#access-the-datadog-site) for your organization. For more information on predefined Datadog sites, see the [OpenLineage documentation](https://openlineage.io/docs/client/python/#predefined-datadog-sites).

   ```shell
   export DD_SITE=datadoghq.com
   export DD_API_KEY=<YOUR_DATADOG_API_KEY>
   export OPENLINEAGE__TRANSPORT__TYPE=datadog
   
   # OPENLINEAGE_NAMESPACE determines the Datadog tag value for the environment (similar to how the service tag identifies the application).
   # Typical values are dev, staging, or prod, but you can over ride it with any custom value.
   export OPENLINEAGE_NAMESPACE=<YOUR_ENV>
   
   # Optional, for debugging purposes
   export OPENLINEAGE_CLIENT_LOGGING=DEBUG
   ```

## Update the dbt invocation{% #update-the-dbt-invocation %}

1. Change your dbt invocations to use the OpenLineage wrapper (`dbt-ol`) instead of calling `dbt` directly. This applies to any dbt command you want to track in Datadog, such as `run`, `build`, and `test`. For the full list of available commands, see the [dbt documentation](https://docs.getdbt.com/docs/running-a-dbt-project/run-your-dbt-projects).

1. Add the `--consume-structured-logs` flag to view dbt jobs while the command is still running.

   ```shell
   # Run models
   dbt-ol run --consume-structured-logs --openlineage-dbt-job-name <YOUR_DBT_JOB_NAME>
   
   # Run tests (required to see test failures in Datadog)
   dbt-ol test --consume-structured-logs --openlineage-dbt-job-name <YOUR_DBT_JOB_NAME>
   
   # Run build (runs models, tests, seeds, and snapshots)
   dbt-ol build --consume-structured-logs --openlineage-dbt-job-name <YOUR_DBT_JOB_NAME>
   ```

## What's next{% #whats-next %}

After your next dbt job run, you should start seeing job run and lineage data in [Datadog Data Observability](https://app.datadoghq.com/datasets/catalog?integration=dbt), as shown below.

{% image
   source="https://datadog-docs.imgix.net/images/data_observability/data-obs-dbt-cloud-final.0a9482f0c463f82642a440c622b35e28.png?auto=format"
   alt="Data Observability overview showing dbt job runs and model lineage." /%}

{% /tab %}

## Further reading{% #further-reading %}

- [Learn about Data Observability](https://docs.datadoghq.com/data_observability/)
- [Understanding dbt: basics and best practices](https://www.datadoghq.com/blog/understanding-dbt/)