Evaluation compatibility

Este producto no es compatible con el sitio Datadog seleccionado. ().
Esta página aún no está disponible en español. Estamos trabajando en su traducción.
Si tienes alguna pregunta o comentario sobre nuestro actual proyecto de traducción, no dudes en ponerte en contacto con nosotros.

Evaluation compatibility

The supported third party LLM providers are OpenAI, Azure OpenAI, Anthropic, and Bedrock Anthropic.

Managed evaluations

Managed evaluations are supported for the following configurations.

EvaluationDD-trace versionLLM ProviderApplicable span
Tool Selectionv3.12+OpenAI, Azure OpenAILLM only
Tool Argument Correctnessv3.12+OpenAI, Azure OpenAILLM only
Goal CompletenessFully supportedOpenAI, Azure OpenAILLM only
Hallucinationv2.18+OpenAILLM only
Failure to AnswerFully supportedAll third party LLM providersAll span kinds
SentimentFully supportedAll third party LLM providersAll span kinds
ToxicityFully supportedAll third party LLM providersAll span kinds
Prompt InjectionFully supportedAll third party LLM providersAll span kinds
Topic RelevancyFully supportedAll third party LLM providersAll span kinds
Language MismatchFully supportedSelf hostedAll span kinds

Custom LLM-as-a-judge evaluations

Custom LLM-as-a-judge evaluations are supported for the following configurations.

EvaluationDD-trace versionLLM ProviderApplicable span
BooleanFully supportedAll third party LLM providersAll span kinds
ScoreFully supportedOpenAI, Azure OpenAIAll span kinds
CategoricalFully supportedOpenAI, Azure OpenAIAll span kinds