Trace Retention

このページは日本語には対応しておりません。随時翻訳に取り組んでいます。翻訳に関してご質問やご意見ございましたら、お気軽にご連絡ください。

With APM, both the ingestion of traces as well as the retention of those traces for 15 days are fully customizable.

To track or monitor your volume of ingested and indexed data, see the Usage Metrics documentation.

Retention filters

After spans have been ingested by Datadog, some are kept for 15 days according to the retention filters that have been set on your account.

By default, two retention filters are enabled for your account:

The Error Default retention filter indexes error spans with status:error. The retention rate and the query can be configured. For instance, to capture production errors, set the query to status:error, env:production. Disable the retention filter if you do not want to capture the errors by default.
The Intelligent Retention Filter retains diverse error traces and traces from different latency distributions.

You can also create any number of additional tag-based retention filters for your services.

Note: Admin rights are required to create, modify, or disable retention filters.

In Datadog, on the Retention Filters tab, you can see a list of all retention filters:

Filter Name: The name of each filter used to index spans.
Filter Query: The tag-based query for each filter.
Retention Rate: A percentage from 0 to 100% of how many matching spans are indexed.
Spans Indexed: The number of spans indexed by the filter over the selected time period.
Last Updated: The date and user who last modified the retention filter.
Enabled toggle: Allows filters to be turned on and off.

The order of the retention filter list changes indexing behavior. If a span matches a retention filter early in the list, the span is either kept or dropped. Any matching retention filter lower on the list does not catch the already-processed span.

The Spans Indexed column for each retention filter is powered by the datadog.estimated_usage.apm.indexed_spans metric, which you can use to track your indexed span usage. For more information, read Usage Metrics, or see the dashboard available in your account.

Note: Retention filters do not affect what traces are collected by the Agent and sent to Datadog ("ingested"). The only way to change how much tracing data is ingested is through ingestion controls.

Datadog intelligent retention filter

The intelligent retention filter is always active for your services, and it keeps a proportion of traces to help you monitor the health of your applications. All service entry spans are indexed for the traces kept by the intelligent retention filter.

For 30 days, intelligent retention retains:

A representative selection of errors, ensuring error diversity (for example, response code 400s, 500s).
High latency in the p75, p90, and p95 quartiles.
All resources with any traffic that have associated traces in the past for any time window selection.
True maximum duration trace for each time window.

Note: Because intelligent retention chooses spans intentionally and not randomly, spans that are retained only by the intelligent filter are not included in the Trace explorer timeseries view. Aggregated views (timeseries, toplist, table) are available only for spans retained by custom retention filters.

Spans indexed by the intelligent retention filter are not counted towards the usage of indexed spans, and hence do not impact your bill.

If there are specific tags, or attributes for which you want to retain more spans than what Intelligent Retention retains, then create your own retention filter.

Create your own retention filter

Decide which spans are indexed and retained for 15 days by creating, modifying, and disabling additional filters based on tags. Set a percentage of spans matching each filter to be retained. Any span that is retained has its corresponding trace saved as well, and when it is viewed in the Trace Explorer, the complete trace is available.

Note: In order for you to search by tag in the Trace Explorer, the span that directly contains the searched-upon tag must have been indexed by a retention filter.

Define the retention query by adding any span tag. Choose to retain all spans with the defined tags, only service entry spans (selected by default), or only trace root spans.
Set a percentage of spans matching these tags to be indexed.
Name the filter.
Save the new filter.

When you create a new filter or edit the retention rate of an existing filter, Datadog displays an estimate of the percentage change in global indexing volume.

Filters are retained in a serial order. If you have an upstream filter that retains spans with the resource:POST /hello_world tag, those spans do not show up in the Edit window of a downstream filter that searches for spans with the same tag because they have been retained by the upstream filter.

For example, you can create filters to keep all traces for: