LLM Observability Metrics

This product is not supported for your selected Datadog site. ().
このページは日本語には対応しておりません。随時翻訳に取り組んでいます。
翻訳に関してご質問やご意見ございましたら、お気軽にご連絡ください。

After you instrument your application with LLM Observability, you can access LLM Observability metrics for use in dashboards and monitors. These metrics capture span counts, error counts, token usage, and latency measures for your LLM applications. These metrics are calculated based on 100% of the application’s traffic.

Other tags set on spans are not available as tags on LLM Observability metrics.

Span metrics

Metric NameDescriptionMetric TypeTags
ml_obs.spanTotal number of spans with a span kindCountenv, error, ml_app, model_name, model_provider, service, span_kind, version
ml_obs.span.durationTotal duration of spans in secondsDistributionenv, error, ml_app, model_name, model_provider, service, span_kind, version
ml_obs.span.errorNumber of errors that occurred in the spanCountenv, error, ml_app, model_name, model_provider, service, span_kind, version

LLM token metrics

Metric NameDescriptionMetric TypeTags
ml_obs.span.llm.input.tokensNumber of tokens in the input sent to the LLMDistributionenv, error, ml_app, model_name, model_provider, service, version
ml_obs.span.llm.output.tokensNumber of tokens in the outputDistributionenv, error, ml_app, model_name, model_provider, service, version
ml_obs.span.llm.prompt.tokensNumber of tokens used in the promptDistributionenv, error, ml_app, model_name, model_provider, service, version
ml_obs.span.llm.completion.tokensTokens generated as a completion during the spanDistributionenv, error, ml_app, model_name, model_provider, service, version
ml_obs.span.llm.total.tokensTotal tokens consumed during the span (input + output + prompt)Distributionenv, error, ml_app, model_name, model_provider, service, version
ml_obs.span.llm.input.charactersNumber of characters in the input sent to the LLMDistributionenv, error, ml_app, model_name, model_provider, service, version
ml_obs.span.llm.output.charactersNumber of characters in the outputDistributionenv, error, ml_app, model_name, model_provider, service, version

Embedding metrics

Metric NameDescriptionMetric TypeTags
ml_obs.span.embedding.input.tokensNumber of input tokens used for generating an embeddingDistributionenv, error, ml_app, model_name, model_provider, service, version

LLM cost metrics

Metric NameDescriptionMetric TypeTags
ml_obs.span.llm.input.costEstimated input cost in an LLM spanDistributionenv, error, ml_app, model_name, model_provider, service, version, source
ml_obs.span.embedding.input.costEstimated input cost in an embedding spanDistributionenv, error, ml_app, model_name, model_provider, service, version, source
ml_obs.span.llm.output.costEstimated output cost in an LLM spanDistributionenv, error, ml_app, model_name, model_provider, service, version, source
ml_obs.span.llm.total.costEstimated total cost in an LLM spanDistributionenv, error, ml_app, model_name, model_provider, service, version, source
ml_obs.span.llm.input.cache_write.costEstimated cache write input cost in an LLM spanDistributionenv, error, ml_app, model_name, model_provider, service, version, source
ml_obs.span.llm.input.cache_read.costEstimated cache read input cost in an LLM spanDistributionenv, error, ml_app, model_name, model_provider, service, version, source
ml_obs.span.llm.input.non_cached.costEstimated non cached input cost in an LLM spanDistributionenv, error, ml_app, model_name, model_provider, service, version, source

Trace metrics

Metric NameDescriptionMetric TypeTags
ml_obs.traceNumber of tracesCountenv, error, ml_app, service, span_kind, version
ml_obs.trace.durationTotal duration of all traces across all spansDistributionenv, error, ml_app, service, span_kind, version
ml_obs.trace.errorNumber of errors that occurred during the traceCountenv, error, ml_app, service, span_kind, version

Estimated usage metrics

Metric NameDescriptionMetric TypeTags
ml_obs.estimated_usage.llm.input.tokensEstimated number of input tokens usedDistributionevaluation_name, ml_app, model_name, model_provider, model_server

Deprecated metrics

The following metrics are deprecated, and are maintained only for backward compatibility. Datadog strongly recommends using non-deprecated token metrics for all token usage measurement use cases.
Metric NameDescriptionMetric TypeTags
ml_obs.estimated_usage.llm.output.tokensEstimated number of output tokens generatedDistributionevaluation_name, ml_app, model_name, model_provider, model_server
ml_obs.estimated_usage.llm.total.tokensTotal estimated tokens (input + output) usedDistributionevaluation_name, ml_app, model_name, model_provider, model_server

Next steps

Make use of your LLM Observability metrics:


Further Reading

お役に立つドキュメント、リンクや記事:

Learn more about LLM ObservabilityDOCUMENTATION more more Create and manage monitors to notify your teams when it matters.DOCUMENTATION more more