Overview
You can configure your LLM applications on the Configuration page with settings that can optimize your application’s performance and security.
- Evaluations
- Enables Datadog to assess your LLM application on dimensions like Quality, Security, and Safety. By enabling evaluations, you can assess the effectiveness of your application’s responses and maintain high standards for both performance and user safety. For more information about evaluations, see Terms and Concepts.
- Topics
- Helps identify irrelevant input for the
topic relevancy
out-of-the-box evaluation, ensuring your LLM application stays focused on its intended purpose.
Connect your OpenAI account
Connect your OpenAI account to LLM Observability with your OpenAI API key.
- In Datadog, navigate to LLM Observability > Settings > Integrations.
- Select Connect on the OpenAI tile.
- Follow the instructions on the tile.
- Provide your OpenAI API key. Ensure that this key has write permission for model capabilities.
- Enable Invoke Model from this API key.
Select and enable evaluations
- Navigate to LLM Observability > Settings > Evaluations.
- Click on the evaluation you want to enable.
- Select OpenAI as your LLM provider.
- Select the OpenAI account you want to run the evaluation on.
- Assign the LLM application you want to run the evaluation on.
After you click Save, LLM Observability invokes a GPT-4o mini
model using the OpenAI API key you provided.
For more information about evaluations, see Terms and Concepts.
Provide topics for topic relevancy
Providing topics allows you to use the topic relevancy evaluation.
- Go to LLM Observability > Applications.
- Select the application you want to add topics for.
- At the bottom of the left sidebar, select Configuration.
- Add topics in the pop-up modal.
Topics can contain multiple words and should be as specific and descriptive as possible. For example, for an LLM application that was designed for incident management, add “observability”, “software engineering”, or “incident resolution”. If your application handles customer inquiries for an e-commerce store, you can use “Customer questions about purchasing furniture on an e-commerce store”.
Further Reading
Additional helpful documentation, links, and articles: