Overview

You can configure your LLM applications on the Configuration page with settings that can optimize your application’s performance and security.

Evaluations

Enables Datadog to assess your LLM application on dimensions like Quality, Security, and Safety. By enabling evaluations, you can assess the effectiveness of your application’s responses and maintain high standards for both performance and user safety. For more information about evaluations, see Terms and Concepts.

Topics

Helps identify irrelevant input for the topic relevancy out-of-the-box evaluation, ensuring your LLM application stays focused on its intended purpose.

Connect your OpenAI account

Connect your OpenAI account to LLM Observability with your OpenAI API key.

1. In Datadog, navigate to LLM Observability > Settings > Integrations.
2. Select Connect on the OpenAI tile.
3. Follow the instructions on the tile.
   • Provide your OpenAI API key. Ensure that this key has write permission for model capabilities.
4. Enable Invoke Model from this API key.

Select and enable evaluations

1. Navigate to LLM Observability > Settings > Evaluations.
2. Click on the evaluation you want to enable.
3. Select OpenAI as your LLM provider.
4. Select the OpenAI account you want to run the evaluation on.
5. Assign the LLM application you want to run the evaluation on.

After you click Save, LLM Observability invokes a GPT-4o mini model using the OpenAI API key you provided.

For more information about evaluations, see Terms and Concepts.

Provide topics for topic relevancy

Providing topics allows you to use the topic relevancy evaluation.

1. Go to LLM Observability > Applications.
2. Select the application you want to add topics for.
3. At the bottom of the left sidebar, select Configuration.
4. Add topics in the pop-up modal.

Topics can contain multiple words and should be as specific and descriptive as possible. For example, for an LLM application that was designed for incident management, add “observability”, “software engineering”, or “incident resolution”. If your application handles customer inquiries for an e-commerce store, you can use “Customer questions about purchasing furniture on an e-commerce store”.

Further Reading

Más enlaces, artículos y documentación útiles:

Learn about EvaluationsDOCUMENTATION Learn how to submit EvaluationsDOCUMENTATION Learn how to set up LLM ObservabilityDOCUMENTATION