Evaluation compatibility

Ce produit n'est pas pris en charge par le site Datadog que vous avez sélectionné. ().
Cette page n'est pas encore disponible en français, sa traduction est en cours.
Si vous avez des questions ou des retours sur notre projet de traduction actuel, n'hésitez pas à nous contacter.

Evaluation compatibility

The supported third party LLM providers are OpenAI, Azure OpenAI, Anthropic, Amazon Bedrock, Vertex AI, and AI Gateway.

Managed evaluations

Managed evaluations are supported for the following configurations.

EvaluationDD-trace versionLLM ProviderApplicable span
Language MismatchFully supportedSelf hostedAll span kinds

Custom LLM-as-a-judge evaluations

Custom LLM-as-a-judge evaluations are supported for the following configurations.

EvaluationDD-trace versionLLM ProviderApplicable span
BooleanFully supportedAll third party LLM providersAll span kinds
ScoreFully supportedAll third party LLM providersAll span kinds
CategoricalFully supportedAll third party LLM providersAll span kinds
JSONFully supportedAll third party LLM providersAll span kinds

Template LLM-as-a-judge evaluations

Existing templates for custom LLM-as-a-judge evaluations are supported for the following configurations.

EvaluationDD-trace versionLLM ProviderApplicable span
Failure to AnswerFully supportedAll third party LLM providersAll span kinds
HallucinationFully supportedAll third party LLM providersLLM only
SentimentFully supportedAll third party LLM providersAll span kinds
ToxicityFully supportedAll third party LLM providersAll span kinds
Prompt InjectionFully supportedAll third party LLM providersAll span kinds
Topic RelevancyFully supportedAll third party LLM providersAll span kinds
Tool SelectionFully supportedAll third party LLM providersLLM only
Tool Argument CorrectnessFully supportedAll third party LLM providersLLM only
Goal CompletenessFully supportedAll third party LLM providersLLM only