Evaluation compatibility

이 제품은 선택한 Datadog 사이트에서 지원되지 않습니다. ().
이 페이지는 아직 한국어로 제공되지 않습니다. 번역 작업 중입니다.
현재 번역 프로젝트에 대한 질문이나 피드백이 있으신 경우 언제든지 연락주시기 바랍니다.

Evaluation compatibility

The supported third party LLM providers are OpenAI, Azure OpenAI, Anthropic, and Bedrock Anthropic.

Managed evaluations

Managed evaluations are supported for the following configurations.

EvaluationDD-trace versionLLM ProviderApplicable span
Tool Selectionv3.12+OpenAI, Azure OpenAILLM only
Tool Argument Correctnessv3.12+OpenAI, Azure OpenAILLM only
Goal CompletenessFully supportedOpenAI, Azure OpenAILLM only
Hallucinationv2.18+OpenAILLM only
Language MismatchFully supportedSelf hostedAll span kinds

Custom LLM-as-a-judge evaluations

Custom LLM-as-a-judge evaluations are supported for the following configurations.

EvaluationDD-trace versionLLM ProviderApplicable span
BooleanFully supportedAll third party LLM providersAll span kinds
ScoreFully supportedOpenAI, Azure OpenAI, Anthropic, Vertex AIAll span kinds
CategoricalFully supportedOpenAI, Azure OpenAI, Anthropic, Vertex AIAll span kinds

Template LLM-as-a-judge evaluations

Existing templates for custom LLM-as-a-judge evaluations are supported for the following configurations.

EvaluationDD-trace versionLLM ProviderApplicable span
Failure to AnswerFully supportedAll third party LLM providersAll span kinds
SentimentFully supportedAll third party LLM providersAll span kinds
ToxicityFully supportedAll third party LLM providersAll span kinds
Prompt InjectionFully supportedAll third party LLM providersAll span kinds
Topic RelevancyFully supportedAll third party LLM providersAll span kinds