New announcements for Serverless, Network, RUM, and more from Dash! New announcements from Dash!

Generate Metrics from Ingested Logs

This feature is in private beta. Contact Contact Datadog support to enquire about enabling the feature for your account.

Overview

Datadog’s Logging without Limits* lets you dynamically decide what to include or exclude from your indexes. Log-based metrics are a cost-efficient way to summarize log data from the entire ingest stream. This means that even if you use exclusion filters to limit indexes to operationally critical logs, you can still visualize trends and anomalies over all of your log data at full granularity for 15 months.

With log-based metrics, generate custom metrics from the count of logs that match a filter query and apply log attributes and tag keys as metric tags following the <KEY>:<VALUE> format.

Generate a log-based metric

To generate a new log-based metric, go to the Configuration page of your Datadog account and select the Generate Metrics tab, then the New Metric+ button.

Add a new log-based metric

  1. Input a query to filter the log stream: The query syntax is the same as for the Log Explorer Search. Only logs ingested with a timestamp within the past 20 minutes are considered for aggregation.
  2. Add dimensions to group by: Add attributes or tag key to apply to the generated log-based metric to transform them into Tags following the <KEY>:<VALUE> format. Log-based metrics are considered as custom metrics. Avoid grouping by unbounded or extremely high cardinality attributes like timestamps, user IDs, request IDs, or session IDs to avert impacting your billing.
  3. Name your metric: Log-based metric names must follow the naming metric convention.

Note: Data points for Log-based metrics are generated at one minute intervals.

Update a log-based metric

After a metric is created, only these fields can be updated:

  • Stream filter query
  • Aggregation groups

To change the metric type or name, a new metric must be created.

Feedback

Datadog would love to receive your feedback on this feature. You can submit your feedback directly from this form.

Further Reading


*Logging without Limits is a trademark of Datadog, Inc.