Configuration

이 페이지는 아직 한국어로 제공되지 않으며 번역 작업 중입니다. 번역에 관한 질문이나 의견이 있으시면 언제든지 저희에게 연락해 주십시오.

Overview

You can configure your LLM applications on the Configuration page with settings that can optimize your application’s performance and security.

Evaluations
Enables Datadog to assess your LLM application on dimensions like Quality, Security, and Safety. By enabling evaluations, you can assess the effectiveness of your application’s responses and maintain high standards for both performance and user safety. For more information about evaluations, see Terms and Concepts.
Topics
Helps identify irrelevant input for the topic relevancy out-of-the-box evaluation, ensuring your LLM application stays focused on its intended purpose.

Connect your OpenAI account

Connect your OpenAI account to LLM Observability with your OpenAI API key.

  1. In Datadog, navigate to LLM Observability > Settings > Integrations.
  2. Select Connect on the OpenAI tile.
  3. Follow the instructions on the tile.
    • Provide your OpenAI API key. Ensure that this key has write permission for model capabilities.
  4. Enable Use this API key to evaluate your LLM applications.
The OpenAI configuration tile in LLM Observability. Lists instructions for configuring OpenAI and providing your OpenAI API key.

Select and enable evaluations

  1. Navigate to LLM Observability > Settings > Evaluations.
  2. Click on the evaluation you want to enable.
  3. Select OpenAI as your LLM provider.
  4. Select the OpenAI account you want to run the evaluation on.
  5. Assign the LLM application you want to run the evaluation on.

After you click Save, LLM Observability invokes a GPT-4o mini model using the OpenAI API key you provided.

For more information about evaluations, see Terms and Concepts.

Provide topics for topic relevancy

Providing topics allows you to use the topic relevancy evaluation.

  1. Go to LLM Observability > Applications.
  2. Select the application you want to add topics for.
  3. At the bottom of the left sidebar, select Configuration.
  4. Add topics in the pop-up modal.

Topics can contain multiple words and should be as specific and descriptive as possible. For example, for an LLM application that was designed for incident management, add “observability”, “software engineering”, or “incident resolution”. If your application handles customer inquiries for an e-commerce store, you can use “Customer questions about purchasing furniture on an e-commerce store”.

Further Reading

추가 유용한 문서, 링크 및 기사:

Learn about EvaluationsDOCUMENTATION more more Learn how to submit EvaluationsDOCUMENTATION more more Learn how to set up LLM ObservabilityDOCUMENTATION more more