---
title: Logs Troubleshooting
description: Datadog, the leading service for cloud-scale monitoring.
breadcrumbs: Docs > Log Management > Logs Troubleshooting
---

# Logs Troubleshooting

If you experience unexpected behavior with Datadog Logs, there are a few common issues you can investigate and this guide may help resolve issues quickly. If you continue to have trouble, reach out to [Datadog support](https://docs.datadoghq.com/help/) for further assistance.

## Missing logs - data access restrictions{% #missing-logs---data-access-restrictions %}

You cannot see any logs in the [Log Explorer](https://app.datadoghq.com/logs) or [Live Tail](https://app.datadoghq.com/logs/livetail). This may be happening because your role is part of a restriction query.

If you are unable to access your Restriction Queries in Datadog, contact your Datadog Administrator to verify if your role is affected.

See [Check Restrictions Queries](https://docs.datadoghq.com/logs/guide/logs-rbac-permissions/?tab=ui#check-restriction-queries) for more information on configuring Logs RBAC data access controls.

**Legacy Permissions** can also restrict access to Logs, particularly in the [Log Explorer](https://app.datadoghq.com/logs). Depending on configuration, access may be limited to specific indexes or to a single index at a time. For more information on how Legacy Permissions are applied at the role and organization level, see [Legacy Permissions](https://docs.datadoghq.com/logs/guide/logs-rbac-permissions/?tab=ui#legacy-permissions).

## Missing logs - logs daily quota reached{% #missing-logs---logs-daily-quota-reached %}

You have not made any changes to your log configuration, but the [Log Explorer](https://app.datadoghq.com/logs) shows that logs are missing for today. This may be happening because you have reached your daily quota.

{% image
   source="https://datadog-docs.imgix.net/images/logs/troubleshooting/daily_quota_reached.a38f787b7d1e1850402949ecfb3c650a.png?auto=format"
   alt="A bar graph showing missing logs and a message saying daily quota reached" /%}

See [Set daily quota](https://docs.datadoghq.com/logs/log_configuration/indexes/#set-daily-quota) for more information on setting up, updating or removing the quota.

To verify if a daily quota has been reached historically, you can search in the Event Explorer with the tag `datadog_index:{index_name}`.

{% image
   source="https://datadog-docs.imgix.net/images/logs/troubleshooting/daily_quota_event.213f9740a196e63a48d09f672f6e4ac4.png?auto=format"
   alt="An event explaining the time at which a daily quota was reached" /%}

## Missing logs - timestamp outside of the ingestion window{% #missing-logs---timestamp-outside-of-the-ingestion-window %}

Logs with a timestamp further than 18 hours in the past are dropped at intake. Fix the issue at the source by checking which `service` and `source` are impacted with the `datadog.estimated_usage.logs.drop_count` metric.

## Missing logs - timestamp not aligned with timezone{% #missing-logs---timestamp-not-aligned-with-timezone %}

By default, Datadog parses all epoch timestamps in Logs using UTC. If incoming logs use a different timezone, timestamps may appear shifted by the corresponding offset from UTC. For example, if logs are sent from New York (EST -5), they may appear 5 hours in the past, or if logs are sent from Australia (AEST +10), they may appear 10 hours ahead of the expected time frame.

To adjust the timezone of the logs during processing, see the footnotes in Datadog's [Parsing](https://docs.datadoghq.com/logs/log_configuration/parsing/?tab=matchers#parsing-dates) guide on using the `timezone` parameter with the date matcher. Epoch timestamps can be adjusted using the timezone parameter in a Grok Parser processor to adjust localizations.

Epoch timestamps can be adjusted using the `timezone` parameter in a Grok Parser processor. Follow these steps to convert a localized timestamp to UTC using the example in Datadog's [Grok Parser](https://docs.datadoghq.com/logs/log_configuration/processors/?tab=ui#grok-parser) guide.

1. Navigate to the [Pipelines](https://app.datadoghq.com/logs/pipelines) page.
1. In **Pipelines**, select the correct pipeline matching to your logs.
1. Open the Grok Parser processor that is parsing your logs.
1. Given that a local host is logging in UTC+1, adjust the date matcher to account for this difference. The resulting rule will have a comma and a new string defining the timezone to UTC+1.
1. Verify that the [Log Date Remapper](https://docs.datadoghq.com/logs/log_configuration/processors/?tab=ui#log-date-remapper) is using the parsed attribute as the official timestamp for the matching logs.

Go to the [Log Explorer](https://app.datadoghq.com/logs) and verify that the logs appear in line with their original timestamp.

## Unable to parse timestamp key from JSON logs{% #unable-to-parse-timestamp-key-from-json-logs %}

Datadog requires timestamp attributes to use one of the supported date formats:

- ISO8601
- UNIX (the milliseconds EPOCH format)
- RFC3164

Timestamps that do not exactly match these formats may be dropped, even if they are similar (for example, epoch timestamps in nanoseconds).

If you are unable to convert the timestamp of JSON logs to a [recognized date format](https://docs.datadoghq.com/logs/log_configuration/pipelines/?tab=date#date-attribute) before they are ingested into Datadog, follow these steps to convert and map the timestamps using Datadog's [arithmetic processor](https://docs.datadoghq.com/logs/log_configuration/indexes/#set-daily-quota) and [log date remapper](https://docs.datadoghq.com/logs/log_configuration/pipelines/?tab=date#date-attribute):

1. Navigate to the [Pipelines](https://app.datadoghq.com/logs/pipelines) page.

1. In **Pipelines**, hover over **Preprocessing for JSON logs**, and click the pencil icon.

1. Remove `timestamp` from the reserved attribute mapping list. The attribute is not being parsed as the official timestamp of the log during preprocessing.

{% image
   source="https://datadog-docs.imgix.net/images/logs/troubleshooting/preprocessing_json_timestamp.2059d8913400e3bbb5140bbdf7694af1.png?auto=format"
   alt="The preprocessing for JSON logs configuration box with the date attributes, which includes timestamp by default" /%}

Set up the [arithmetic processor](https://docs.datadoghq.com/logs/log_configuration/processors/?tab=ui#arithmetic-processor) so that the formula multiplies your timestamp by 1000 to convert it to milliseconds. The formula's result is a new attribute.

Set up the [log date remapper](https://docs.datadoghq.com/logs/log_configuration/processors/?tab=ui#log-date-remapper) to use the new attribute as the official timestamp.

Go to [Log Explorer](https://app.datadoghq.com/logs) to see new JSON logs with their mapped timestamp.

## Truncated logs{% #truncated-logs %}

Logs above 1MB are truncated. Fix the issue at the source by checking which `service` and `source` are impacted with the `datadog.estimated_usage.logs.truncated_count` and `datadog.estimated_usage.logs.truncated_bytes` metrics.

## Truncated log messages{% #truncated-log-messages %}

There is an additional truncation in fields that applies only to indexed logs: the value is truncated to 75 KiB for the message field and 25 KiB for non-message fields. Datadog stores the full text, and it remains visible in regular list queries in the Log Explorer. However, the truncated version is displayed when performing a grouped query, such as when grouping logs by that truncated field or performing similar operations that display that specific field.

## Logs present in Live Tail, but missing from Logs Explorer{% #logs-present-in-live-tail-but-missing-from-logs-explorer %}

[Logging Without Limits™](https://docs.datadoghq.com/logs/guide/getting-started-lwl/) decouples log ingestion from indexing, allowing you to retain the logs that matter most. When exclusion filters applied to indexes are too broad, they may exclude more logs than intended.

Review both exclusion filters and index filters carefully. Parsed and unparsed JSON logs can match index filters in unexpected ways, particularly when free-text search is used to exclude short strings. This can result in entire logs being dropped from indexing, even when they contain other valuable data. For details on the differences between full-text and free-text search, see [Search Syntax](https://docs.datadoghq.com/logs/explorer/search_syntax/#full-text-search).

## Estimated Usage Metrics{% #estimated-usage-metrics %}

If Logs do not appear to be indexed, or at a smaller or higher rate than expected, check Estimated Usage Metric volumes in your [Log Management dashboard](https://app.datadoghq.com/dashboard/lists/preset/3?q=Log%20Management%20estimated%20usage&p=1).

Depending on the metric, tags such as `datadog_index`, `datadog_is_excluded`, `service,` and `status` are available for filtering. Use these tags to filter metrics such as `datadog.estimated_usage.logs.ingested_events` by exclusion status and by the index that is indexing or excluding the logs.

If the `datadog_index` tag is set to `N/A` for a metric datapoint, the corresponding logs do not match any index in your organization. Review index order and filter queries to identify potential exclusions.

**Note**: Estimated Usage Metrics do not respect [Daily Quotas](https://docs.datadoghq.com/logs/log_configuration/indexes/#set-daily-quota).

## Create a support ticket{% #create-a-support-ticket %}

If the above troubleshooting steps do not resolve your issues with missing logs in Datadog, create a [support ticket](https://help.datadoghq.com/hc/en-us/requests/new). If possible, include the following information:

| Information               | Description                                                                                                                                                                                                                                                                                                                                                                                                                                 |
| ------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| **Raw log sample**        | Collect the log directly from the source generating it, based on your architecture or logger configuration. Attach the log to the support ticket as a **text file** or **raw JSON**.                                                                                                                                                                                                                                                        |
| **Indexes configuration** | If the log appears in Live Tail but not in the Log Explorer, include the response from the [Get All Indexes](https://docs.datadoghq.com/api/latest/logs-indexes/#get-an-index) API call. If your organization has many indexes, review Estimated Usage Metrics to identify the relevant index, then include the response from the [Get an Index](https://docs.datadoghq.com/api/latest/logs-indexes/#get-an-index) API call for that index. |
| **Agent flare**           | If logs are sent using the Agent and do not appear anywhere in the Datadog UI, submit an [Agent Flare](https://docs.datadoghq.com/agent/troubleshooting/send_a_flare/?tab=agent) with the support ticket.                                                                                                                                                                                                                                   |
| **Other sources**         | If logs are sent using a source other than the Agent, include details on the originating source of the logs (for example, Lambda Forwarder or Kinesis Firehose).                                                                                                                                                                                                                                                                            |