---
title: Monitor and query for unparsed logs
description: Datadog, the leading service for cloud-scale monitoring.
breadcrumbs: Docs > Log Management > Logs Guides > Monitor and query for unparsed logs
---

# Monitor and query for unparsed logs

## Overview{% #overview %}

Parsed logs are central to be able to use Datadog Log Management to its full capacity, for queries, monitors, aggregations or automatic enrichments such as sensitive data scanner. When you are scaling your volume of logs it can be challenging to identify and fix logs patterns that are not parsed by your pipelines.

To identify and control the volume of unparsed logs in your organization:

1. Detect unparsed logs
1. Query for unparsed logs
1. Create a metric to track for unparsed logs
1. Monitor the volume of unparsed logs

## Detect unparsed logs{% #detect-unparsed-logs %}

To determine if a specific log has been parsed by your pipelines, open the log and check the Event Attributes panel. If the log is unparsed, instead of showing attributes extracted from your log, the panel shows a message saying that no attributes were extracted:

{% image
   source="https://docs.dd-static.net/images/logs/guide/unparsed-logs/unparsed-log.56fdd1b7463cceae2228a5f5686121be.jpg?auto=format&fit=max&w=850 1x, https://docs.dd-static.net/images/logs/guide/unparsed-logs/unparsed-log.56fdd1b7463cceae2228a5f5686121be.jpg?auto=format&fit=max&w=850&dpr=2 2x"
   alt="Unparsed log details" /%}

You can start parsing and unparsed log by creating [custom pipelines](https://docs.datadoghq.com/logs/log_configuration/pipelines.md) or using a [log integration](https://docs.datadoghq.com/integrations.md#cat-log-collection) as the source of the log to take advantage of the automatic pipeline setup.

## Query for unparsed logs{% #query-for-unparsed-logs %}

If you have many logs, making one-by-one checking unviable, you can instead query for unparsed logs by using the filter `datadog.pipelines:false` in the [Log Explorer](https://docs.datadoghq.com/logs/explorer.md):

{% image
   source="https://docs.dd-static.net/images/logs/guide/unparsed-logs/datadog-pipeline-false-log-explorer.c667f27109cca0d402562dfcd770029e.jpg?auto=format&fit=max&w=850 1x, https://docs.dd-static.net/images/logs/guide/unparsed-logs/datadog-pipeline-false-log-explorer.c667f27109cca0d402562dfcd770029e.jpg?auto=format&fit=max&w=850&dpr=2 2x"
   alt="Query unparsed logs" /%}

This filter returns all indexed logs without custom attributes after the pipeline processing. [Pattern aggregation](https://docs.datadoghq.com/logs/explorer.md#patterns) shows an aggregated view of the common patterns in the unparsed logs, which can kickstart your creation of custom pipelines.

## Create a metric to track for unparsed logs{% #create-a-metric-to-track-for-unparsed-logs %}

Querying for unparsed logs lets you select the unparsed *indexed* logs. It's also a good practice to ensure that even the logs that you do not index are parsed, so that the content of your [archives](https://docs.datadoghq.com/logs/archives.md?tab=awss3) are structured.

To create a metric for unparsed logs, create a [custom metric](https://docs.datadoghq.com/logs/logs_to_metrics.md) using the `datadog.pipelines:false` query:

{% image
   source="https://docs.dd-static.net/images/logs/guide/unparsed-logs/logs-unparsed-metric.5b65ba78d405253db9ff16f796dc5f04.jpg?auto=format&fit=max&w=850 1x, https://docs.dd-static.net/images/logs/guide/unparsed-logs/logs-unparsed-metric.5b65ba78d405253db9ff16f796dc5f04.jpg?auto=format&fit=max&w=850&dpr=2 2x"
   alt="Generate logs.unparsed metric" /%}

As for any log-based metric, you can add dimensions in the `group by` field. The example above shows grouping by `service` and `team`. You should group by the dimensions that you are using to define the ownership of a log.

## Monitor the volume of unparsed logs{% #monitor-the-volume-of-unparsed-logs %}

To ensure that the logs parsing in your organization is kept under control, apply a quota for the unparsed logs volume. This approach is close to what is proposed with [daily quotas](https://docs.datadoghq.com/logs/indexes.md#set-daily-quota) for indexes.

To monitor the volume of unparsed logs:

1. Create a [metric monitor](https://docs.datadoghq.com/monitors/types/metric.md?tab=threshold#overview).
1. Use the previously created `logs.unparsed` metric.
1. Define the quota per `team`.
1. Ensure that the [alert conditions](https://docs.datadoghq.com/monitors/types/metric.md?tab=threshold#set-alert-conditions) fit when you want to be alerted.

{% image
   source="https://docs.dd-static.net/images/logs/guide/unparsed-logs/monitor-unparsed-logs-team.5fc628fff36026fad8d7f0ce06eac3b3.jpg?auto=format&fit=max&w=850 1x, https://docs.dd-static.net/images/logs/guide/unparsed-logs/monitor-unparsed-logs-team.5fc628fff36026fad8d7f0ce06eac3b3.jpg?auto=format&fit=max&w=850&dpr=2 2x"
   alt="Query unparsed logs" /%}

## Further Reading{% #further-reading %}

- [See how to explore your logs](https://docs.datadoghq.com/logs/explorer.md)
- [Log Parsing - Best Practices](https://docs.datadoghq.com/logs/faq/log-parsing-best-practice.md)