---
title: End-User Feedback
description: >-
  Submit end-user feedback to Agent Observability and connect it to spans,
  traces, sessions, or external entities.
breadcrumbs: Docs > Agent Observability > Evaluations > End-User Feedback
---

# End-User Feedback

{% callout %}
# Important note for users on the following Datadog sites: app.ddog-gov.com, us2.ddog-gov.com

{% alert level="danger" %}
This product is not supported for your selected [Datadog site](https://docs.datadoghq.com/getting_started/site.md). ({% placeholder "user-datadog-site-name" /%}).
{% /alert %}

{% /callout %}

## Overview{% #overview %}

End-user feedback captures input from users of your LLM application in Agent Observability. Examples include thumbs-up or thumbs-down ratings, whether a user accepted an agent's change, and free-text comments about a response.

Feedback is different from an evaluation. Use feedback for signals submitted by an end user. Use [external evaluations](https://docs.datadoghq.com/llm_observability/evaluations/external_evaluations.md) for results produced by your own evaluator logic, where who submitted the evaluation is not relevant. Use [Annotation Queues](https://docs.datadoghq.com/llm_observability/evaluations/annotation_queues.md) for structured review workflows run by your team.

Submitted feedback appears when viewing Agent Observability sessions, traces, or spans.

## Submit feedback{% #submit-feedback %}

Submit feedback with the [Evaluations API](https://docs.datadoghq.com/llm_observability/instrumentation/api.md#evaluations-api) by setting `event_kind` to `feedback`.

Feedback events require:

- `event_kind: "feedback"`
- `submitter.id`, which identifies the user or agent that submitted the feedback
- Exactly one target field: `span_id`, `trace_id`, `session_id`, or `feedback_join_key`
- A value field that matches `metric_type`

Feedback events must not include `join_on`. If `eval_scope` is omitted, Datadog infers it from the target field. If `eval_scope` is provided, it must match the selected target.

### Target feedback{% #target-feedback %}

| Target          | Field               | Use when                                                                                                            |
| --------------- | ------------------- | ------------------------------------------------------------------------------------------------------------------- |
| Span            | `span_id`           | The feedback applies to one span.                                                                                   |
| Trace           | `trace_id`          | The feedback applies to an entire trace.                                                                            |
| Session         | `session_id`        | The feedback applies to an entire session.                                                                          |
| External entity | `feedback_join_key` | The feedback applies to a customer-defined entity, such as an incident ID, report ID, task ID, or release check ID. |

### Use a feedback join key{% #use-a-feedback-join-key %}

Use `feedback_join_key` when feedback is not tied to a single span, trace, or session. First, enrich your spans with the `feedback_join_key` tag related to the external entity using the SDK's [Enriching spans](https://docs.datadoghq.com/llm_observability/instrumentation/sdk.md?tab=python#enriching-spans) workflow or the [Spans API](https://docs.datadoghq.com/llm_observability/instrumentation/api.md?tab=model#spans-api). Then, submit feedback with the same `feedback_join_key`.

## Examples{% #examples %}

### Submit thumbs-down feedback for a span{% #submit-thumbs-down-feedback-for-a-span %}

```json
{
  "data": {
    "type": "evaluation_metric",
    "attributes": {
      "metrics": [
        {
          "event_kind": "feedback",
          "span_id": "20245611112024561111",
          "ml_app": "weather-bot",
          "timestamp_ms": 1765990800016,
          "metric_type": "categorical",
          "label": "thumbs",
          "categorical_value": "down",
          "assessment": "fail",
          "submitter": {
            "id": "user-123",
            "type": "user"
          }
        }
      ]
    }
  }
}
```

### Submit free-text feedback with a feedback join key{% #submit-free-text-feedback-with-a-feedback-join-key %}

```json
{
  "data": {
    "type": "evaluation_metric",
    "attributes": {
      "metrics": [
        {
          "event_kind": "feedback",
          "feedback_join_key": "incident-123",
          "ml_app": "incident-agent",
          "timestamp_ms": 1765990800016,
          "metric_type": "text",
          "label": "user_comment",
          "text_value": "The investigation missed the customer impact.",
          "assessment": "fail",
          "submitter": {
            "id": "user-123",
            "type": "user"
          }
        }
      ]
    }
  }
}
```

## Analyze feedback{% #analyze-feedback %}

To create a dashboard widget for feedback, create the widget as you would for an evaluation and add the filter `@event_kind:feedback`.

{% alert level="info" %}
Support for filtering spans, traces, or sessions by feedback is not available. For example, you cannot yet filter traces to only traces with thumbs-down feedback. Use dashboards scoped to `@event_kind:feedback` instead.
{% /alert %}

## Further Reading{% #further-reading %}

- [Learn about the Evaluations API](https://docs.datadoghq.com/llm_observability/instrumentation/api.md#evaluations-api)
- [Learn about submitting external evaluations](https://docs.datadoghq.com/llm_observability/evaluations/external_evaluations.md)
- [Learn about Annotation Queues](https://docs.datadoghq.com/llm_observability/evaluations/annotation_queues.md)