---
title: Set Up and Use Agent Observability Offline Experiments
description: >-
  How to set up Agent Observability Offline Experiments and start running
  offline experiments.
breadcrumbs: >-
  Docs > Agent Observability > Agent Observability Guides > Set Up and Use Agent
  Observability Offline Experiments
---

> For the complete documentation index, see [llms.txt](https://docs.datadoghq.com/llms.txt).

# Set Up and Use Agent Observability Offline Experiments

{% callout %}
# Important note for users on the following Datadog sites: app.ddog-gov.com, us2.ddog-gov.com

{% alert level="danger" %}
This product is not supported for your selected [Datadog site](https://docs.datadoghq.com/getting_started/site.md). ({% placeholder "user-datadog-site-name" /%}).
{% /alert %}

{% /callout %}

This guide describes how to set up and use Agent Observability offline experiments with the Python or Node.js SDK. See [runnable Node.js experiment examples](https://github.com/DataDog/llm-observability/tree/main/experiments/nodejs).

## Prerequisites{% #prerequisites %}

To run offline experiments, you need a Datadog [API key](https://docs.datadoghq.com/account_management/api-app-keys.md#add-an-api-key-or-client-token) and a [Datadog application key](https://docs.datadoghq.com/account_management/api-app-keys.md#add-application-keys).

## Set up Agent Observability{% #set-up-agent-observability %}

If you have not already set up Agent Observability, select the Python or Node.js tab and follow the instructions for your application's language.

Replace `site` in the following examples with your Datadog site: <YOUR_DATADOG_SITE>.

1. Install the Agent Observability SDK:

   {% tab title="Python" %}

   ```shell
   pip install ddtrace>=4.3.0
   ```

   {% /tab %}

   {% tab title="Node.js" %}

   ```shell
   npm install "dd-trace@>=6.13.0"
   ```

   {% /tab %}

1. Enable Agent Observability:

   {% tab title="Python" %}
Pass `api_key` and `app_key` to `LLMObs.enable()`, or set the `DD_API_KEY` and `DD_APP_KEY` environment variables:

   ```python
   from ddtrace.llmobs import LLMObs
   
   LLMObs.enable(
       api_key="<YOUR_API_KEY>",  # defaults to DD_API_KEY environment variable
       app_key="<YOUR_APP_KEY>",  # defaults to DD_APP_KEY environment variable
       site="datadoghq.com",      # defaults to DD_SITE environment variable
       project_name="<YOUR_PROJECT>"  # defaults to DD_LLMOBS_PROJECT_NAME environment variable, or "default-project" if the environment variable is not set
   )
   ```

      {% /tab %}

   {% tab title="Node.js" %}
Set `DD_API_KEY` and `DD_APP_KEY` in your environment. The Offline Experiments client uses these variables for authentication. Then choose one of the following initialization methods.

**In-code initialization**

Initialize `dd-trace` in your application entrypoint:

   ```javascript
   const tracer = require('dd-trace').init({
     service: '<YOUR_SERVICE>',
     site: 'datadoghq.com',
     llmobs: {
       mlApp: '<YOUR_ML_APP_NAME>',
       projectName: '<YOUR_PROJECT>',
       agentlessEnabled: true,
     },
   })
   
   const { experiments } = tracer.llmobs
   ```

Configure the project and application names independently:

   - `projectName` identifies the project that contains your datasets and offline experiments. If omitted, it defaults to `default-project`.
   - `mlApp` identifies the LLM application used for Agent Observability traces. Neither `mlApp` nor `service` determines the Offline Experiments project name.

**Command-line initialization**

Alternatively, initialize `dd-trace` from the command line instead of calling `require('dd-trace').init(...)` in your application. Set the configuration and authentication environment variables when starting your application:

   ```shell
   DD_SITE=<YOUR_DATADOG_SITE> DD_API_KEY=<YOUR_API_KEY> DD_APP_KEY=<YOUR_APP_KEY> \
   DD_LLMOBS_ENABLED=1 DD_LLMOBS_ML_APP=<YOUR_ML_APP_NAME> DD_LLMOBS_PROJECT_NAME=<YOUR_PROJECT> \
   NODE_OPTIONS="--import dd-trace/initialize.mjs" node <YOUR_APP_ENTRYPOINT>
   ```

For more information, see the [Node.js tracer command-line setup](https://docs.datadoghq.com/llm_observability/instrument/sdk.md?tab=nodejs#command-line-setup).
   {% /tab %}

### APM trace correlation{% #apm-trace-correlation %}

To correlate experiment spans with [APM traces](https://docs.datadoghq.com/llm_observability/instrument/agent_observability_and_apm.md), run Agent Observability through a Datadog Agent. The Agent forwards trace data to APM.

{% tab title="Python" %}
APM trace correlation requires `agentless_enabled=False`, which is the default:

```python
LLMObs.enable(
    api_key="<YOUR_API_KEY>",
    app_key="<YOUR_APP_KEY>",
    site="datadoghq.com",
    agentless_enabled=False,  # default — required for APM trace correlation
    project_name="<YOUR_PROJECT>",
)
```

To run without a Datadog Agent, set `agentless_enabled` to `True`. Agentless runs do not generate corresponding APM spans for experiment spans.
{% /tab %}

{% tab title="Node.js" %}
APM trace correlation requires `agentlessEnabled: false`, which is the default. If you used the in-code initialization example in the setup section, change `agentlessEnabled` from `true` to `false`:

```javascript
const tracer = require('dd-trace').init({
  service: '<YOUR_SERVICE>',
  site: 'datadoghq.com',
  llmobs: {
    mlApp: '<YOUR_ML_APP_NAME>',
    projectName: '<YOUR_PROJECT>',
    agentlessEnabled: false, // default — required for APM trace correlation
  },
})
```

To run without a Datadog Agent, set `agentlessEnabled` to `true`. Agentless runs do not generate corresponding APM spans for experiment spans.
{% /tab %}

## Create a project{% #create-a-project %}

Projects organize your datasets and offline experiments. You can create a project in the Datadog UI or through the API. To create a project with an SDK, specify a project name that does not already exist.

{% tab title="Python" %}

```python
from ddtrace.llmobs import LLMObs

LLMObs.enable(
    ...,
    project_name="<YOUR_PROJECT>"  # defaults to DD_LLMOBS_PROJECT_NAME environment variable, or "default-project" if the environment variable is not set
)
```

{% /tab %}

{% tab title="Node.js" %}

```javascript
const tracer = require('dd-trace').init({
  llmobs: {
    mlApp: '<YOUR_ML_APP_NAME>',
    projectName: 'experiments-project',
  },
})

const { experiments } = tracer.llmobs
```

Set the default project with `llmobs.projectName` or the `DD_LLMOBS_PROJECT_NAME` environment variable. The Offline Experiments client creates the project when it first accesses it.

To use a different project for a specific operation, pass `projectName` to that operation. For example, pass it to `pullDataset()` to retrieve a dataset from another project. When creating a dataset and its experiment, use the same project name for both:

```javascript
async function main () {
  const dataset = experiments.createDataset('capital-cities', {
    projectName: 'other-project',
    records: [
      { inputData: { country: 'France' }, expectedOutput: 'Paris' },
      { inputData: { country: 'Japan' }, expectedOutput: 'Tokyo' },
    ],
  })

  const experiment = experiments.experiment({
    name: 'capital-cities-test',
    projectName: 'other-project',
    dataset,
    task: (inputData) => ({ France: 'Paris', Japan: 'Tokyo' })[inputData.country],
    evaluators: {
      exact_match: (_inputData, outputData, expectedOutput) => outputData === expectedOutput,
    },
  })

  const result = await experiment.run()
  console.log(`View experiment: ${result.url}`)
}

main().catch((error) => {
  console.error(error)
  process.exitCode = 1
})
```

{% /tab %}

## Create a dataset{% #create-a-dataset %}

A dataset is a collection of inputs, expected outputs, and metadata that represent scenarios you want to test your agent on. Each dataset belongs to a project.

- **Input** (required): Represents all the information that the agent can access in a task.
- **Expected output** (optional): Represents the ideal answer that the agent should output. Expected output is also called *ground truth*. You can use *expected output* to store the actual output of the app, as well as any intermediary results you want to assess.
- **Metadata** (optional): Contains information to categorize the record for further analysis, such as topics, tags, descriptions, and notes.

Create a dataset using the example for your SDK:

{% tab title="Python" %}
To create a dataset from a CSV file, use `LLMObs.create_dataset_from_csv()`:

```python
# Create dataset from CSV
dataset = LLMObs.create_dataset_from_csv(
    csv_path="questions.csv",
    dataset_name="capitals-of-the-world",
    project_name="capitals-project",              # Optional: defaults to the project name from LLMObs.enable
    description="Geography quiz dataset",         # Optional: Dataset description
    input_data_columns=["question", "category"],  # Columns to use as input
    expected_output_columns=["answer"],             # Optional: Columns to use as expected output
    metadata_columns=["difficulty"],                # Optional: Additional columns as metadata
    csv_delimiter=","                               # Optional: Defaults to comma
)

# Example "questions.csv":
# question,category,answer,difficulty
# What is the capital of Japan?,geography,Tokyo,medium
# What is the capital of Brazil?,geography,Brasília,medium
```

{% /tab %}

{% tab title="Node.js" %}
The Node.js SDK does not include a CSV-specific helper. Parse the CSV with the library of your choice, then pass the records to `createDataset()`:

```javascript
const dataset = experiments.createDataset('capitals-of-the-world', {
  description: 'Geography quiz dataset',
  records: [
    {
      inputData: { question: 'What is the capital of Japan?', category: 'geography' },
      expectedOutput: 'Tokyo',
      metadata: { difficulty: 'medium' },
    },
    {
      inputData: { question: 'What is the capital of Brazil?', category: 'geography' },
      expectedOutput: 'Brasília',
      metadata: { difficulty: 'medium' },
    },
  ],
})

// The dataset is pushed automatically when the experiment runs.
// To push it earlier, use: await dataset.push()
```

{% /tab %}

See [Datasets](https://docs.datadoghq.com/llm_observability/improve/datasets.md) for information about creating, retrieving, and managing datasets and how Datadog retains dataset versions.

## Create an experiment{% #create-an-experiment %}

An experiment runs your agent across scenarios in a dataset and uses evaluators to measure its performance. Compare offline experiments to assess how different application configurations perform.

An experiment has the following components:

- **Task**: Defines the core workflow you want to evaluate. It can range from a single LLM call to a more complex flow involving multiple LLM calls and RAG steps. The task is executed sequentially across all records in the dataset.
- **Evaluator**: A function, executed on each record, that measures how well the model or agent performs. Evaluators allow you to compare the output to either the expected output or the original input.
- **Summary evaluators**: Optional functions executed against all the data of the experiment (input, output, expected output, and evaluator results). Summary evaluators allow you to compute more advanced metrics such as precision, recall, and accuracy across your dataset.

To create an experiment:

### 1. Load a dataset

{% tab title="Python" %}

```python
from ddtrace.llmobs import LLMObs
from typing import Dict, Any, Optional, List

dataset = LLMObs.pull_dataset("capitals-of-the-world")
```

{% /tab %}

{% tab title="Node.js" %}

```javascript
const tracer = require('dd-trace').init()
const { experiments } = tracer.llmobs

const dataset = await experiments.pullDataset('capitals-of-the-world')
```

{% /tab %}

### 2. Define a task function that processes a single dataset record

{% tab title="Python" %}

```python
def task(input_data: Dict[str, Any], config: Optional[Dict[str, Any]] = None) -> str:
    question = input_data["question"]
    # Your LLM or processing logic here
    return "Beijing" if "China" in question else "Unknown"
```

{% /tab %}

{% tab title="Node.js" %}

```javascript
async function task (inputData, config, metadata) {
  const question = inputData.question
  // Your LLM or processing logic here
  return question.includes('China') ? 'Beijing' : 'Unknown'
}
```

{% /tab %}

A task can take any non-null type as `input_data` or `inputData` (string, number, Boolean, object, or array). Evaluators can use output of any type. This example returns a string. To include intermediate results for evaluation, return a dictionary or object.

Optionally, your task function can accept metadata from the dataset record:

{% tab title="Python" %}

```python
def task(input_data: Dict[str, Any], config: Optional[Dict[str, Any]] = None, metadata: Optional[Dict[str, Any]] = None) -> str:
    difficulty = metadata.get("difficulty", "unknown") if metadata else "unknown"
    question = input_data["question"]
    return "Beijing" if "China" in question else "Unknown"
```

{% /tab %}

{% tab title="Node.js" %}

```javascript
function task (inputData, config, metadata) {
  const difficulty = metadata?.difficulty ?? 'unknown'
  const question = inputData.question
  // Use difficulty in the task when it is part of your evaluation logic.
  return question.includes('China') ? 'Beijing' : 'Unknown'
}
```

{% /tab %}

Trace parts of your experiment task, for example, workflows or tool calls, using the same tracing APIs you use in production. Agent Observability also automatically traces and annotates calls to supported LLM frameworks and libraries.

{% tab title="Python" %}
For custom tracing, use the [Python tracing decorators](https://docs.datadoghq.com/llm_observability/instrument/sdk.md?tab=python#manual-instrumentation). For automatic instrumentation, see the [supported Python frameworks](https://docs.datadoghq.com/llm_observability/instrument/auto_instrumentation.md?tab=python).
{% /tab %}

{% tab title="Node.js" %}
For custom tracing, use `llmobs.trace()`. For automatic instrumentation, see the [supported Node.js frameworks](https://docs.datadoghq.com/llm_observability/instrument/auto_instrumentation.md?tab=nodejs).
{% /tab %}

#### Using OpenTelemetry spans inside offline experiments{% #using-opentelemetry-spans-inside-offline-experiments %}

If your application uses [OpenTelemetry instrumentation](https://docs.datadoghq.com/llm_observability/instrument/otel_instrumentation.md), you can create OTel spans inside your experiment task. With `DD_TRACE_OTEL_ENABLED=1`, the Datadog tracer acts as the OpenTelemetry TracerProvider, so OTel spans appear as children of the experiment span automatically.

{% tab title="Python" %}

```python
import json
from opentelemetry import trace

tracer = trace.get_tracer(__name__)

def task(input_data: Dict[str, Any], config: Optional[Dict[str, Any]] = None) -> str:
    question = input_data["question"]

    # OTel gen_ai span — automatically becomes a child of the experiment span
    with tracer.start_as_current_span("my-llm-call") as span:
        span.set_attribute("gen_ai.operation.name", "chat")
        span.set_attribute("gen_ai.system", "openai")
        span.set_attribute("gen_ai.request.model", "gpt-4o")
        span.set_attribute("gen_ai.usage.input_tokens", 25)
        span.set_attribute("gen_ai.usage.output_tokens", 8)
        span.set_attribute(
            "gen_ai.input.messages",
            json.dumps([{"role": "user", "parts": [{"type": "text", "content": question}]}]),
        )

        result = call_my_llm(question)

        span.set_attribute(
            "gen_ai.output.messages",
            json.dumps([{"role": "assistant", "parts": [{"type": "text", "content": result}]}]),
        )

    return result
```

{% /tab %}

{% tab title="Node.js" %}
Install the OpenTelemetry API if it is not already a dependency:

```shell
npm install @opentelemetry/api
```

```javascript
const { trace } = require('@opentelemetry/api')

const otelTracer = trace.getTracer('my-experiment')

async function task (inputData, config, metadata) {
  const question = inputData.question

  return otelTracer.startActiveSpan('my-llm-call', async (span) => {
    span.setAttribute('gen_ai.operation.name', 'chat')
    span.setAttribute('gen_ai.system', 'openai')
    span.setAttribute('gen_ai.request.model', 'gpt-4o')
    span.setAttribute('gen_ai.usage.input_tokens', 25)
    span.setAttribute('gen_ai.usage.output_tokens', 8)
    span.setAttribute('gen_ai.input.messages', JSON.stringify([
      { role: 'user', parts: [{ type: 'text', content: question }] },
    ]))

    const result = await callMyLlm(question)

    span.setAttribute('gen_ai.output.messages', JSON.stringify([
      { role: 'assistant', parts: [{ type: 'text', content: result }] },
    ]))
    span.end()
    return result
  })
}
```

{% /tab %}

To include OpenTelemetry spans in your experiment traces, set `DD_TRACE_OTEL_ENABLED` to `1` when running your experiment:

{% tab title="Python" %}

```shell
DD_TRACE_OTEL_ENABLED=1 python my_experiment.py
```

{% /tab %}

{% tab title="Node.js" %}

```shell
DD_TRACE_OTEL_ENABLED=1 node my_experiment.js
```

{% /tab %}

### 3. Define evaluators

Evaluators measure how well your model or agent performs on each record. Both SDKs support local evaluators defined as functions or classes. The Node.js SDK also supports managed remote evaluators, which run evaluations configured in Datadog.

For detailed information on building evaluators, including the full data model reference and best practices, see the [Evaluation Developer Guide](https://docs.datadoghq.com/llm_observability/investigate/evaluations/evaluation_developer_guide.md).

Datadog supports the following evaluator return types:

- **Boolean**: Returns true or false
- **Score**: Returns a numeric value (float)
- **Categorical**: Returns a labeled category (string)
- **JSON**: Returns structured data (dict or object)

{% tab title="Python" %}
#### Function-based evaluators{% #function-based-evaluators %}

```python
def exact_match(input_data: Dict[str, Any], output_data: str, expected_output: str) -> bool:
    return output_data == expected_output

def overlap(input_data: Dict[str, Any], output_data: str, expected_output: str) -> float:
    expected_output_set = set(expected_output)
    output_set = set(output_data)

    intersection = len(output_set.intersection(expected_output_set))
    union = len(output_set.union(expected_output_set))

    return intersection / union

def fake_llm_as_a_judge(input_data: Dict[str, Any], output_data: str, expected_output: str) -> EvaluatorResult:
    fake_llm_call = "excellent"
    return EvaluatorResult(
        value=fake_llm_call,
        reasoning="the model explains itself",
        assessment="pass", # or fail
        tags={"task": "judge_llm_call"},
    )

# Return multiple metrics from one evaluator call
from ddtrace.llmobs import EvaluatorResult, MultiEvaluatorResult

def multi_metric_evaluator(input_data, output_data, expected_output):
    correct = output_data == expected_output
    return MultiEvaluatorResult(
        {
            "correct": EvaluatorResult(value=correct, assessment="pass" if correct else "fail"),
            "length": len(str(output_data)),
        }
    )
    # Emitted as: multi_metric_evaluator-correct, multi_metric_evaluator-length
```

#### Class-based evaluators{% #class-based-evaluators %}

```python
from ddtrace.llmobs import BaseEvaluator, EvaluatorContext, EvaluatorResult

class SemanticSimilarityEvaluator(BaseEvaluator):
    def __init__(self, threshold: float = 0.8):
        super().__init__(name="semantic_similarity")
        self.threshold = threshold

    def evaluate(self, context: EvaluatorContext) -> EvaluatorResult:
        score = compute_similarity(context.output_data, context.expected_output)
        return EvaluatorResult(
            value=score,
            reasoning=f"Similarity score: {score:.2f}",
            assessment="pass" if score >= self.threshold else "fail",
        )
```

{% /tab %}

{% tab title="Node.js" %}
#### Function-based evaluators{% #function-based-evaluators %}

Return a Boolean, number, string, or JSON-serializable object. Use an object map to assign evaluator names explicitly.

```javascript
function exact_match (inputData, outputData, expectedOutput) {
  return outputData === expectedOutput
}

function overlap (inputData, outputData, expectedOutput) {
  const expected = new Set(String(expectedOutput))
  const output = new Set(String(outputData))
  const intersection = [...output].filter(value => expected.has(value)).length
  const union = new Set([...expected, ...output]).size
  return intersection / union
}

function fake_llm_as_a_judge (inputData, outputData, expectedOutput) {
  // Return a categorical metric. Use a separate evaluator for each metric.
  return 'excellent'
}
```

#### Class-based evaluators{% #class-based-evaluators %}

For reusable evaluators, extend `experiments.BaseEvaluator`. The `evaluate` method receives an `EvaluatorContext` and can return a value or a Promise. Return `experiments.EvaluatorResult` to attach reasoning, assessment, metadata, or tags to the evaluation metric.

```javascript
const tracer = require('dd-trace').init()
const { experiments } = tracer.llmobs

class ExactMatchEvaluator extends experiments.BaseEvaluator {
  constructor () {
    super('exact_match')
  }

  evaluate (context) {
    const passed = context.outputData === context.expectedOutput
    return new experiments.EvaluatorResult(passed, {
      assessment: passed ? 'pass' : 'fail',
      reasoning: passed ? 'The output matches the expected value.' : 'The output does not match the expected value.',
      metadata: { evaluator_version: 'v1' },
      tags: { evaluator_version: 'v1' },
    })
  }
}

const experiment = experiments.experiment({
  // ...name, dataset, and task...
  evaluators: [new ExactMatchEvaluator()],
})
```

#### Managed remote evaluators{% #managed-remote-evaluators %}

Use `experiments.RemoteEvaluator` to run a custom [LLM-as-a-judge evaluation](https://docs.datadoghq.com/llm_observability/investigate/evaluations/llm_as_a_judge_evaluations.md) configured in Datadog. Set `evalName` to the configured evaluation name. The default input mapping includes the task input and output, expected output, record metadata, and experiment span and trace IDs. You can provide `transformFn` to customize the request sent to Datadog.

```javascript
const remoteEvaluator = new experiments.RemoteEvaluator({
  evalName: '<YOUR_EVALUATOR_NAME>',
})

const experiment = experiments.experiment({
  // ...name, dataset, and task...
  evaluators: [remoteEvaluator],
})
```

You can mix managed remote evaluators with local function-based or class-based evaluators in the same experiment.
{% /tab %}

### 4. (Optional) Define summary evaluators

Summary evaluators receive aggregated results after all record-level evaluators finish. Use them to calculate dataset-level statistics, such as averages or pass rates. Both SDKs support function-based and class-based summary evaluators.

{% tab title="Python" %}
#### Function-based summary evaluators{% #function-based-summary-evaluators %}

Summary evaluator functions can take a list of any non-null type as `inputs` (string, number, Boolean, object, or array); `outputs` and `expected_outputs` can be lists of any type. `evaluators_results` is a dictionary of lists of results from evaluators, keyed by the name of the evaluator function.

```python
def num_exact_matches(inputs, outputs, expected_outputs, evaluators_results):
    return evaluators_results["exact_match"].count(True)
```

#### Class-based summary evaluators{% #class-based-summary-evaluators %}

For details on implementing `BaseSummaryEvaluator`, see the [Evaluation Developer Guide](https://docs.datadoghq.com/llm_observability/investigate/evaluations/evaluation_developer_guide.md).

```python
from ddtrace.llmobs import BaseSummaryEvaluator, SummaryEvaluatorContext

class AverageScoreEvaluator(BaseSummaryEvaluator):
    def __init__(self, target_evaluator: str):
        super().__init__(name="average_score")
        self.target_evaluator = target_evaluator

    def evaluate(self, context: SummaryEvaluatorContext):
        scores = context.evaluation_results.get(self.target_evaluator, [])
        if not scores:
            return None
        return sum(scores) / len(scores)
```

{% /tab %}

{% tab title="Node.js" %}
#### Function-based summary evaluators{% #function-based-summary-evaluators %}

Summary evaluator functions receive the inputs, outputs, expected outputs, and an object containing the results from each evaluator. The optional fifth argument contains record metadata.

```javascript
function num_exact_matches (inputs, outputs, expectedOutputs, evaluatorResults, metadata) {
  return (evaluatorResults.exact_match || []).filter(Boolean).length
}
```

#### Class-based summary evaluators{% #class-based-summary-evaluators %}

For reusable summary evaluators, extend `experiments.BaseSummaryEvaluator`. The `SummaryEvaluatorContext` includes the inputs, outputs, expected outputs, evaluator results, and record metadata.

```javascript
class PassRateEvaluator extends experiments.BaseSummaryEvaluator {
  constructor () {
    super('pass_rate')
  }

  evaluate (context) {
    const values = context.evaluationResults.exact_match || []
    if (values.length === 0) return null
    return values.filter(Boolean).length / values.length
  }
}
```

Add class-based summary evaluators to `summaryEvaluators` alongside function-based evaluators.
{% /tab %}

### 5. Create and run the experiment

Create an experiment with your dataset, task, and evaluators, then run it and inspect the results:

{% tab title="Python" %}

```python
experiment = LLMObs.experiment(
    name="capital-cities-test",
    task=task,
    dataset=dataset,
    evaluators=[exact_match, overlap, fake_llm_as_a_judge],
    summary_evaluators=[num_exact_matches], # optional
    description="Testing capital cities knowledge",
    config={
        "model_name": "gpt-4",
        "version": "1.0"
    },
)

# Run the experiment
results = experiment.run()  # Run on all dataset records

# Process results
for result in results.get("rows", []):
    print(f"Record {result['idx']}")
    print(f"Input: {result['input']}")
    print(f"Output: {result['output']}")
    print(f"Score: {result['evaluations']['evaluator']['value']}")
    if result['error']['message']:
        print(f"Error: {result['error']['message']}")
```

{% /tab %}

{% tab title="Node.js" %}

```javascript
const experiment = experiments.experiment({
  name: 'capital-cities-test',
  task,
  dataset,
  evaluators: {
    exact_match,
    overlap,
    fake_llm_as_a_judge,
  },
  summaryEvaluators: {
    num_exact_matches,
  },
  description: 'Testing capital cities knowledge',
  config: {
    model: 'gpt-4',
    version: '1.0',
  },
})

// Run the experiment. The dataset is pushed automatically if needed.
const results = await experiment.run()

// Process results
for (const row of results.rows) {
  console.log(`Record ${row.index}`)
  console.log(`Input: ${JSON.stringify(row.input)}`)
  console.log(`Output: ${JSON.stringify(row.output)}`)
  console.log(`Score: ${row.evaluations.exact_match}`)
  if (row.isError) {
    console.log(`Error: ${row.errorMessage}`)
  }
}
```

{% /tab %}

Configure experiment execution using the options supported by your SDK:

{% tab title="Python" %}

```python
# Increase execution speed
results = experiment.run(jobs=4)

# Test your pipeline on a subset of the data
results = experiment.run(sample_size=10)

# Stop the experiment if an error occurs
results = experiment.run(raise_errors=True)
```

{% /tab %}

{% tab title="Node.js" %}
The Node.js SDK runs dataset records sequentially. To manage concurrency, implement it in your task function. To stop the experiment if an error occurs, set `throwOnErrors` to `true`:

```javascript
const results = await experiment.run({
  throwOnErrors: true,
})
```

{% /tab %}

### 6. Review your experiment results in Datadog

{% tab title="Python" %}

```python
print(f"View experiment: {experiment.url}")
```

{% /tab %}

{% tab title="Node.js" %}

```javascript
console.log(`View experiment: ${results.url}`)
// The same URL is available from the experiment after it has run.
console.log(experiment.url())
```

{% /tab %}

**Note:** Offline Experiments traces are retained for 90 days.

## Further reading{% #further-reading %}

Additional helpful documentation, links, and articles:

- [Debug and evaluate your AI app from your coding agent with Datadog Agent Observability](https://www.datadoghq.com/blog/debug-and-evaluate-your-ai-app-from-your-coding-agent/)