This product is not supported for your selected Datadog site. ().
Agent Observability, Experiment view. Heading: 'Comparing 6 offline experiments across 9 fields'. Line graph visualization charting the accuracy, correctness, duration, estimated cost, and other metrics of various offline experiments.

Agent Observability Offline Experiments supports the entire lifecycle of building LLM applications and agents. It helps you understand how changes to prompts, models, providers, or system architecture affect performance. With this feature, you can:

  • Create and version datasets
  • Run and manage offline experiments
  • Compare results to evaluate impact

Set up offline experiments

Follow the Set Up and Use Agent Observability Offline Experiments guide to install the SDK, configure credentials, create datasets, define tasks and evaluators, run offline experiments, and review results.

Cookbooks

For in-depth examples of what you can do with Offline Experiments, see Datadog’s provided Jupyter notebooks.

Further reading