Generate Metrics from Ingested Logs
Rapport de recherche Datadog : Bilan sur l'adoption de l'informatique sans serveur Rapport : Bilan sur l'adoption de l'informatique sans serveur

Generate Metrics from Ingested Logs

Cette page n'est pas encore disponible en français, sa traduction est en cours.
Si vous avez des questions ou des retours sur notre projet de traduction actuel, n'hésitez pas à nous contacter.

Overview

Datadog’s Logging without Limits* lets you dynamically decide what to include or exclude from your indexes. Log-based metrics are a cost-efficient way to summarize log data from the entire ingest stream. This means that even if you use exclusion filters to limit indexes to operationally critical logs, you can still visualize trends and anomalies over all of your log data at full granularity for 15 months.

With log-based metrics, you can record a count of logs that match a query or summarize a numeric value contained in a log, such as request duration.

Generate a log-based metric

To generate a new log-based metric, go to the Configuration page of your Datadog account and select the Generate Metrics tab, then the New Metric+ button.

Add a new log-based metric

  1. Input a query to filter the log stream: The query syntax is the same as for the Log Explorer Search. Only logs ingested with a timestamp within the past 20 minutes are considered for aggregation.
  2. Select the field you would like to track: Select * to generate a count of all logs matching your query or enter a log attribute (e.g., @network.bytes_written) to aggregate a numeric value and create its corresponding count, min, max, sum, and avg aggregated metrics.
  3. Add dimensions to group by: By default, metrics generated from logs will not have any tags unless explicitly added. Any attribute or tag dimension that exists in your logs can be used to create metric Tags. Log-based metrics are considered custom metrics. Avoid grouping by unbounded or extremely high cardinality attributes like timestamps, user IDs, request IDs, or session IDs to avert impacting your billing.
  4. Name your metric: Log-based metric names must follow the naming metric convention.

Note: Data points for Log-based metrics are generated at ten second intervals.

Update a log-based metric

After a metric is created, only these fields can be updated:

  • Stream filter query: to change the set of matching logs to be aggregated into metrics.
  • Aggregation groups: to update the tags or manage the cardinality of the generated metrics.

To change the metric type or name, a new metric must be created.

Usage metrics are estimates of your current Datadog usage in near real-time. They enable you to:

  • Graph your estimated usage.
  • Create monitors around your estimated usage.
  • Get instant alerts of spikes or drops in your usage.
  • Assess the potential impact of code changes on your usage in near real-time.

Log Management usage metrics come with three tags that can be used for more granular monitoring:

TagDescription
 datadog_index Indicates the routing query that matches a log to an intended index. 
 datadog_is_excluded Indicates whether or not a log matches an exclusion query. 
 service The service attribute of the log event. 

Further Reading


*Logging without Limits is a trademark of Datadog, Inc.