Overview

Data Observability monitors use anomaly detection that learns from seasonality, trends, and user feedback to catch delayed data, incomplete loads, and unexpected value changes before they affect downstream dashboards, AI applications, or business decisions. Combined with end-to-end data and code lineage, these monitors help teams detect issues early, assess downstream impact, and route to the right owner.

Data Observability monitors support the following metric types:

Table-level metric types:

Column-level metric types:

Datadog collects metrics such as row count and freshness from warehouse system metadata (for example, INFORMATION_SCHEMA) when available. This avoids running a query against your warehouse and reduces compute costs. Not all warehouses expose system metadata. For metrics that cannot be collected from system metadata, the monitor runs a query directly against your warehouse to compute the value.

Data Observability monitors require Quality Monitoring to be set up with at least one supported data warehouse (for example, Snowflake, Databricks, or BigQuery).

To compare a metric across two data assets instead of tracking a single one, see Source to target monitors.

Monitor creation

To create a Data Observability monitor in Datadog, navigate to Data Observability > Monitors > New Monitor or Monitors > New Monitor > Data Observability. To view all existing Data Observability monitors, see the Data Observability Monitors page.

Choose data to monitor

First, select whether to monitor the Table or Column level:

Then, use the Edit tab to search for tables, views, or columns by typing key:value filters into the search field.

Filter by name or location:

Filter by tag:

Filter on any tag applied to your data assets by using the tag key as the filter key. For example, if your assets are tagged with owner, platform, or environment, search on those tags directly:

Tag filters support the same * wildcards and quoting as name filters, for example, owner:data-* or platform:"Snowflake Prod".

Filter by computed attribute:

In addition to your own tags, Datadog computes attributes for your data assets that you can filter on. The available computed attribute is:

lineage_score is bucketed into the discrete tiers listed above rather than taking a continuous value, so filter on one of those exact values. Match a single tier, or combine tiers with OR. For example, lineage_score:1.00 returns your most depended-on assets, and lineage_score:(0.90 OR 1.00) returns the top two tiers.

Combine any of these filters with AND or OR, use parentheses to group conditions, and prefix with - to exclude.

Examples:

Filter by lineage relationship:

To scope your selection to assets that are connected to another asset in your lineage graph, click Add Relation Filter. Choose Upstream of or Downstream of, then select a specific asset or use the same key:value filters to match a set of assets. For example, monitor every table that is upstream of a critical dashboard, or every column downstream of a specific source table.

Filter by hierarchy relationship:

To scope your selection to assets that are a parent or child of another asset in your lineage graph, click Add Relation Filter. Choose Parent of or Child of, then select a specific asset or use the same key:value filters to match a set of assets. For example, monitor every table that has a revenue column, or every table that is within a critical schema.

A single monitor can track up to 5,000 tables, views, or columns. This limit cannot be increased. If your query matches more, split them across multiple monitors.

Select your metric type

Choose a metric type based on the data quality signal you want to track. Each monitor tracks one metric type.

The Freshness metric type detects when data has not been updated within an expected time window. Use it to catch stale data before it affects downstream reports or models.

• Table freshness tracks the time elapsed since the table was last updated. Table freshness is not available for views or for data warehouses that do not provide updated timestamps for tables in system metadata. Use column-level freshness instead.
• Column freshness tracks the most recent date seen in a datetime column.

The Row Count metric type tracks row count changes in your tables. Use it to detect unexpected drops or spikes in data that could indicate pipeline failures or upstream issues.

Column metric types track column-level metrics to detect data drift or quality degradation. Select from the following:

Metric	Description
Uniqueness	The percentage of values in a column that are unique.
Nullness	The percentage of values in a column that are null.
Cardinality	The number of distinct values in a column.
Percent Zero	The percentage of values in a column that are equal to zero.
Percent Negative	The percentage of values in a column that are negative.
Min	The minimum of all values in a column.
Max	The maximum of all values in a column.
Mean	The average of all values in a column.
Standard Deviation	The measure of variation within values in a column.
Sum	The sum of all values in a column.

Some column metrics are only available for specific column types. Numeric metrics (Percent Zero, Percent Negative, Min, Max, Mean, Standard Deviation, Sum) require numeric columns.

The Custom SQL metric type tracks a custom metric value returned by a SQL query that you define. Use it when built-in metric types do not cover your use case, such as monitoring business-specific data quality rules.

1. Select a model type that describes the value returned by your query:
   • Default: The query returns a scalar value. Use this in most cases.
   • Freshness: The query returns the difference (in seconds) between the current time and the last time an event occurred.
   • Percentage: The query returns a percentage value between 0 and 100.
2. Write a SQL query that returns a single value aliased as dd_value, for example: SELECT COUNT(*) as dd_value FROM ANALYTICS_DB.PROD.ORDERS WHERE STATUS = 'FAILED'
3. Click Validate to verify your query syntax.

If your SQL query includes a GROUP BY clause, list the grouped columns as a comma-separated list in the Group by field (for example, column_a, column_b). Each group is evaluated independently.

Note: Each Custom SQL monitor counts as an individual monitored table for billing purposes.

Configure monitor

Detection method

Select a detection method:

• Anomaly: Alert when the metric deviates from an expected pattern. Threshold values are not required. The anomaly model requires 3 to 7 days to train (including a weekend), depending on how frequently the underlying data updates. During the training period, the monitor does not trigger alerts and will be visualized in blue. After training completes, the monitor will be shown in green when in a normal state and red when in an outlier state.
• Threshold: Alert when the metric crosses a fixed value. Set the comparison operator (above, above or equal to, below, below or equal to, equal to, or not equal to) and define a Critical threshold (required) and optionally a Warning threshold. For more details, see Configure Monitors.

WHERE clause

Add a WHERE clause to filter the data evaluated by the monitor. This is useful for monitoring specific segments of data or only recent records. For example:

• created_at >= DATEADD(day, -7, CURRENT_TIMESTAMP()) — only monitor rows from the past week.
• region = 'US' — only monitor data for a specific region.

Group by

You can add a Group by clause to split a single monitor into multiple groups, each evaluated independently. For example, grouping a row count monitor by a REGION column produces a separate alert for each geography.

The default limit is 500 groups per monitor. To increase this limit, contact Support.

Monitor schedule

Set how often the monitor evaluates your data:

• Hourly: The monitor runs every hour.
• Daily: The monitor runs once per day.
• Manual: The monitor runs only when triggered programmatically. Trigger these monitors using the Data Observability API on a schedule so enough historical data can accumulate for modeling to be useful. Currently, the UI does not support default metrics like row counts and freshness, so this workflow only applies to custom or column-level metrics.

Set alert conditions

Choose an aggregation type:

• Simple Alert: Send a single notification when any monitored table or column meets the condition.
• Multi Alert: Send a notification for each group meeting the condition. Customize which dimensions to group by (for example, table, schema, database) to control alert granularity. For example, grouping by schema only sends one alert per schema, bundling all affected tables together to reduce noise.

Example notification

{{#is_alert}}
Data quality issue detected on {{database.name}}.{{schema.name}}.{{table.name}}:
current value {{value}} has breached the threshold of {{threshold}}.
{{/is_alert}}

{{#is_recovery}}
Data quality issue on {{database.name}}.{{schema.name}}.{{table.name}} has recovered.
Current value {{value}} is within the threshold of {{threshold}}.
{{/is_recovery}}

{{#is_alert}}
Anomaly detected on {{database.name}}.{{schema.name}}.{{table.name}}:
observed value {{observed}} is outside the expected range of {{lower_bound}} to {{upper_bound}}
(predicted: {{predicted}}).
{{/is_alert}}

{{#is_recovery}}
{{database.name}}.{{schema.name}}.{{table.name}} has recovered.
Observed value {{observed}} is within the expected range.
{{/is_recovery}}

Source to target monitors

A source to target monitor compares the same metric on two data assets and alerts when the two values diverge. Other Data Observability monitors track whether a single asset is fresh or complete. A source to target monitor tracks whether the copy that landed in a destination matches what left the source.

When a pipeline moves data between systems, partial failures rarely look like failures. If 100,000 rows leave a source table and 99,850 rows arrive in the destination, a row count monitor on the destination alone sees a plausible value. Comparing the two assets surfaces the gap.

Use a source to target monitor to:

• Validate replication from Postgres into Databricks.
• Reconcile two databases inside the same Snowflake account, for example a quality database against production.
• Verify a migration from Redshift to BigQuery before cutover, by running both systems side by side and confirming that they match.
• Confirm that a transformation does not drop rows between its input and its output.

Source to target monitors are available in all regions except GovCloud.

Create a source to target monitor

1. Navigate to Monitors > New Monitor and select Source to Target.
2. Under Choose source, select the warehouse that holds the source data, then select the data to compare.
3. Under Choose target, do the same for the destination. The source and the target can be in different data warehouses or in the same one.
4. Under Select your metric type, choose the metric to compare. Source to target monitors support the same metric types as other Data Observability monitors, including row count, freshness, nullness, uniqueness, cardinality, and Custom SQL.
5. Set Format to control how the comparison is expressed:
   • Difference: the target value minus the source value. A negative value means the target has less than the source.
   • % Difference: the same difference as a percentage of the source value.
6. Configure the detection method, schedule, and notifications as described in Configure monitor.

The Preview Monitor Evaluation panel shows the identified source and target, along with a preview of the selected metric.

The monitored asset is the target, so the monitor appears on the target’s status page.

Compare a custom metric

When the metric type is Custom SQL, supply one query for the source and one query for the target. A WHERE clause is not accepted for this metric type. Include any filtering in each query.

Evaluation

The difference between the source and the target is recorded as its own metric, so a source to target monitor is evaluated by the same detection methods as any other Data Observability monitor, including anomaly detection. Both sides are measured on a synchronized schedule, so the two values are captured at the same time rather than following each warehouse’s default collection cadence.

Example monitors

Annotate bounds

For monitors using the Anomaly detection method, you can annotate bound ranges to provide feedback and improve the model over time. Unlike infrastructure metrics, data quality metrics are often business-specific, so use annotations to teach the model what behavior is normal for your data.

On a monitor’s status page, click Annotate Bounds, select a time range on the chart, and choose one of the following annotations:

Further Reading

Additional helpful documentation, links, and articles:

Data Observability OverviewDOCUMENTATION Quality MonitoringDOCUMENTATION Configure your monitor notificationsDOCUMENTATION Schedule a downtime to mute a monitorDOCUMENTATION Consult your monitor statusDOCUMENTATION