---
title: Quality Monitoring
description: >-
  Detect data freshness delays, unusual patterns, and column-level metric
  changes before they impact downstream systems.
breadcrumbs: Docs > Data Observability Overview > Quality Monitoring
---

> For the complete documentation index, see [llms.txt](https://docs.datadoghq.com/llms.txt).

# Quality Monitoring

{% callout %}
# Important note for users on the following Datadog sites: app.ddog-gov.com, us2.ddog-gov.com

{% alert level="danger" %}
This product is not supported for your selected [Datadog site](https://docs.datadoghq.com/getting_started/site.md). ({% placeholder "user-datadog-site-name" /%}).
{% /alert %}

{% /callout %}

## Overview{% #overview %}

Quality Monitoring detects issues such as data freshness delays, unusual data patterns, and changes in column-level metrics before they affect dashboards, machine learning models, or other downstream systems. It alerts you to potential problems and provides context to trace them back to upstream jobs or sources.

## Key capabilities{% #key-capabilities %}

With Quality Monitoring, you can:

- Detect delayed updates and unexpected row count behavior in your tables
- Surface changes in column-level metrics such as null counts or uniqueness
- Set up monitors using static thresholds or historical baselines
- Trace quality issues using lineage views that show upstream jobs and downstream impact
- Explore end-to-end dependencies in [Lineage](https://docs.datadoghq.com/data_observability/lineage.md) to identify root causes and downstream impact

## Supported data sources{% #supported-data-sources %}

Connect to these data warehouses:

- [Snowflake](https://docs.datadoghq.com/data_observability/quality_monitoring/data_warehouses/snowflake.md)
- [Databricks](https://docs.datadoghq.com/data_observability/quality_monitoring/data_warehouses/databricks.md)
- [BigQuery](https://docs.datadoghq.com/data_observability/quality_monitoring/data_warehouses/bigquery.md)
- [Redshift](https://docs.datadoghq.com/data_observability/quality_monitoring/data_warehouses/redshift.md)
- [ClickHouse Cloud](https://docs.datadoghq.com/data_observability/quality_monitoring/data_warehouses/clickhouse.md)
 
Connect to these data lake sources:

- [AWS Glue](https://docs.datadoghq.com/data_observability/quality_monitoring/data_lakes/aws_glue.md)
- [Amazon Athena](https://docs.datadoghq.com/data_observability/quality_monitoring/data_lakes/amazon_athena.md)
 
Connect to these transactional databases:

- [PostgreSQL](https://docs.datadoghq.com/data_observability/quality_monitoring/transactional_databases/postgres.md)
 
Integrate with these transformation and orchestration tools:

- [Databricks](https://docs.datadoghq.com/data_observability/jobs_monitoring/databricks.md)
- [Airflow](https://docs.datadoghq.com/data_observability/jobs_monitoring/airflow.md)
- [dbt Core](https://docs.datadoghq.com/data_observability/jobs_monitoring/dbt.md?tab=dbtcore)
- [dbt Cloud](https://docs.datadoghq.com/data_observability/jobs_monitoring/dbt.md?tab=dbtcloud)
- [Spark on Kubernetes](https://docs.datadoghq.com/data_observability/jobs_monitoring/kubernetes.md)
- [Spark on Amazon EMR](https://docs.datadoghq.com/data_observability/jobs_monitoring/emr.md)
- [Spark on Google Dataproc](https://docs.datadoghq.com/data_observability/jobs_monitoring/dataproc.md)
- [AWS Glue Jobs](https://docs.datadoghq.com/data_observability/jobs_monitoring/glue.md)
- [Custom Jobs using OpenLineage](https://docs.datadoghq.com/data_observability/jobs_monitoring/openlineage.md)
 
Trace lineage from these ELT tools:

- [Airbyte](https://docs.datadoghq.com/data_observability/quality_monitoring/elt/airbyte.md)
- [Fivetran](https://docs.datadoghq.com/data_observability/quality_monitoring/elt/fivetran.md)
- [Hightouch](https://docs.datadoghq.com/data_observability/quality_monitoring/elt/hightouch.md)
 
Track downstream impact in these BI tools:

- [Looker](https://docs.datadoghq.com/data_observability/quality_monitoring/business_intelligence/looker.md)
- [Tableau](https://docs.datadoghq.com/data_observability/quality_monitoring/business_intelligence/tableau.md)
- [Sigma](https://docs.datadoghq.com/data_observability/quality_monitoring/business_intelligence/sigma.md)
- [Metabase](https://docs.datadoghq.com/data_observability/quality_monitoring/business_intelligence/metabase.md)
- [Power BI](https://docs.datadoghq.com/data_observability/quality_monitoring/business_intelligence/powerbi.md)
 
## Data sent to Datadog{% #data-sent-to-datadog %}

Quality Monitoring runs aggregate queries in your data warehouse, such as `COUNT(*)`, `MAX(<timestamp_column>)`, null counts, and distinct counts. Only the resulting metric values are sent to Datadog, not row-level data from your tables.

[Custom SQL monitors](https://docs.datadoghq.com/monitors/types/data_observability.md?tab=customsql#select-your-metric-type) are the exception, because you control what the query returns:

- The query must return a value aliased as `dd_value`, and Datadog stores that value. Queries that return full rows, such as `SELECT *`, fail validation and are not stored.
- If the query includes a `GROUP BY` clause, Datadog also stores the value of each grouped column. Each group is evaluated as a separate series.

For example, the following query stores each distinct `region` value alongside its count:

```sql
SELECT region, COUNT(*) AS dd_value
FROM <database>.<schema>.orders
WHERE status = 'FAILED'
GROUP BY region
```

{% alert level="warning" %}
If you group by a column that contains sensitive data, such as `email` or `user_id`, those values are sent to Datadog. Group only by non-sensitive columns. If your warehouse supports column masking, you can also mask sensitive columns so that Datadog reads masked values.
{% /alert %}

Column masking support varies by warehouse:

| Warehouse  | Column masking                                                                                                                                            |
| ---------- | --------------------------------------------------------------------------------------------------------------------------------------------------------- |
| Databricks | [Unity Catalog column masks](https://docs.databricks.com/aws/en/data-governance/unity-catalog/filters-and-masks)                                          |
| Snowflake  | [Masking policies](https://docs.snowflake.com/en/user-guide/security-column-ddm-intro) (requires Enterprise Edition or higher)                            |
| BigQuery   | [Data masking](https://docs.cloud.google.com/bigquery/docs/column-data-masking-intro). Grant the Datadog service account the BigQuery Masked Reader role. |
| Redshift   | [Dynamic data masking](https://docs.aws.amazon.com/redshift/latest/dg/t_ddm.html)                                                                         |

## Further reading{% #further-reading %}

Additional helpful documentation, links, and articles:

- [Data Observability Overview](https://docs.datadoghq.com/data_observability.md)
- [Jobs Monitoring](https://docs.datadoghq.com/data_observability/jobs_monitoring.md)
- [Lineage](https://docs.datadoghq.com/data_observability/lineage.md)
- [CI/CD](https://docs.datadoghq.com/data_observability/cicd.md)
- [Data pipeline monitoring 101: Tracking health and performance across the data stack](https://www.datadoghq.com/blog/data-pipeline-monitoring/)
