Generate custom metrics from ingested spans to track trends, power dashboards, and trigger monitors—even for spans and traces that are not retained for full trace analysis.

Custom metrics are created from spans ingested by Datadog APM, regardless of whether a retention filter indexes those spans. Extract numeric values from spans (such as counts, durations, or custom tags) or traces (end-to-end trace duration) and store them as long-lived custom metrics with 15-month retention.

Notes:

• Datadog automatically generates Trace Metrics that capture request counts, error rates, and latency distributions for 100% of your application traffic.
• Available spans for custom metric generation depend on your APM ingestion control settings. Dropped spans from sampling or filtering cannot generate metrics.

Use custom metrics from spans for:

• Fine-grained visibility into span-level latency, error rates, or tag-level performance
• Powering anomaly or forecast monitors with low-latency, high-resolution metrics.
• Extracting key signals for trending or alerting without retaining the full span.

When to use custom metrics from traces

Use custom metrics from traces for:

• Metrics derived from complete trace context, such as total trace duration or operations per trace.
• Alerting on conditions requiring full trace knowledge (for example, N+1 query detection or fan-out patterns).
• Extracting key signals for trending or alerting without retaining the full trace.

Create a metric from spans or traces

1. Navigate to APM > Generate Metrics.
2. Click New Metric.
3. Name your metric following the metric naming convention. Metric names starting with trace.* are not allowed.
4. Select the metric type: Spans or Traces. Both use the same query syntax as APM Search and Analytics.
5. Define the metric query to filter and include only the spans or traces you want to measure.
6. Choose the value to aggregate:
   • Select * to count all matching spans or traces.
   • Enter a numeric attribute (for example, @cassandra_row_count) to aggregate and track the count, min, max, sum, or percentiles.
7. Set grouping dimensions. By default, metrics have no tags unless you add them. Use any span attribute or tag to create metric tags.
8. Preview the result to view the real-time impact of your query through the data visualization and matching spans or traces in the live preview.
9. Click Create Metric.

Update existing metrics

After a metric is created, only two fields can be updated:

Note: To change the metric type or name, create a new metric and delete the old one.

Data availability

Metrics generated from traces are emitted after a trace completes. For long-running traces, the delay increases accordingly (for example, a 45-minute trace’s metric cannot be emitted until trace completion).

When using custom metrics from traces in monitors, account for this latency to avoid false positives.

Further Reading

Additional helpful documentation, links, and articles:

Customize trace ingestion and retain important tracesDOCUMENTATION Use Analytics queries and monitors based on retained tracesDOCUMENTATION Use advanced trace queries to create metrics from specific tracesDOCUMENTATION Monitor 100% of your application traffic with trace metricsDOCUMENTATION Create and manage span-based metrics with TerraformEXTERNAL SITE