LLM Observability Metrics
This product is not supported for your selected
Datadog site. (
).
After you instrument your application with LLM Observability, you can access LLM Observability metrics for use in dashboards and monitors. These metrics capture span counts, error counts, token usage, and latency measures for your LLM applications. These metrics are calculated based on 100% of the application’s traffic.
Other tags set on spans are not available as tags on LLM Observability metrics.
Span metrics
| Metric Name | Description | Metric Type | Tags |
|---|
ml_obs.span | Total number of spans with a span kind | Count | env, error, ml_app, model_name, model_provider, service, span_kind, version |
ml_obs.span.duration | Total duration of spans in seconds | Distribution | env, error, ml_app, model_name, model_provider, service, span_kind, version |
ml_obs.span.error | Number of errors that occurred in the span | Count | env, error, ml_app, model_name, model_provider, service, span_kind, version |
LLM token metrics
| Metric Name | Description | Metric Type | Tags |
|---|
ml_obs.span.llm.input.tokens | Number of tokens in the input sent to the LLM | Distribution | env, error, ml_app, model_name, model_provider, service, version |
ml_obs.span.llm.output.tokens | Number of tokens in the output | Distribution | env, error, ml_app, model_name, model_provider, service, version |
ml_obs.span.llm.prompt.tokens | Number of tokens used in the prompt | Distribution | env, error, ml_app, model_name, model_provider, service, version |
ml_obs.span.llm.completion.tokens | Tokens generated as a completion during the span | Distribution | env, error, ml_app, model_name, model_provider, service, version |
ml_obs.span.llm.total.tokens | Total tokens consumed during the span (input + output + prompt) | Distribution | env, error, ml_app, model_name, model_provider, service, version |
ml_obs.span.llm.input.characters | Number of characters in the input sent to the LLM | Distribution | env, error, ml_app, model_name, model_provider, service, version |
ml_obs.span.llm.output.characters | Number of characters in the output | Distribution | env, error, ml_app, model_name, model_provider, service, version |
Embedding metrics
| Metric Name | Description | Metric Type | Tags |
|---|
ml_obs.span.embedding.input.tokens | Number of input tokens used for generating an embedding | Distribution | env, error, ml_app, model_name, model_provider, service, version |
Trace metrics
| Metric Name | Description | Metric Type | Tags |
|---|
ml_obs.trace | Number of traces | Count | env, error, ml_app, service, span_kind, version |
ml_obs.trace.duration | Total duration of all traces across all spans | Distribution | env, error, ml_app, service, span_kind, version |
ml_obs.trace.error | Number of errors that occurred during the trace | Count | env, error, ml_app, service, span_kind, version |
Estimated usage metrics
| Metric Name | Description | Metric Type | Tags |
|---|
ml_obs.estimated_usage.llm.input.tokens | Estimated number of input tokens used | Distribution | evaluation_name, ml_app, model_name, model_provider, model_server |
Deprecated metrics
The following metrics are deprecated, and are maintained only for backward compatibility. Datadog strongly recommends using non-deprecated token metrics for all token usage measurement use cases.
| Metric Name | Description | Metric Type | Tags |
|---|
ml_obs.estimated_usage.llm.output.tokens | Estimated number of output tokens generated | Distribution | evaluation_name, ml_app, model_name, model_provider, model_server |
ml_obs.estimated_usage.llm.total.tokens | Total estimated tokens (input + output) used | Distribution | evaluation_name, ml_app, model_name, model_provider, model_server |
Next steps
Make use of your LLM Observability metrics:
Further Reading
Additional helpful documentation, links, and articles:
