Overview

End-user feedback captures input from users of your LLM application in Agent Observability. Examples include thumbs-up or thumbs-down ratings, whether a user accepted an agent’s change, and free-text comments about a response.

Feedback is different from an evaluation. Use feedback for signals submitted by an end user. Use external evaluations for results produced by your own evaluator logic, where who submitted the evaluation is not relevant. Use Annotation Queues for structured review workflows run by your team.

Submitted feedback appears when viewing Agent Observability sessions, traces, or spans.

Submit feedback

Submit feedback with the Evaluations API by setting event_kind to feedback.

Feedback events require:

• event_kind: "feedback"
• submitter.id, which identifies the user or agent that submitted the feedback
• Exactly one target field: span_id, trace_id, session_id, or feedback_join_key
• A value field that matches metric_type

Feedback events must not include join_on. If eval_scope is omitted, Datadog infers it from the target field. If eval_scope is provided, it must match the selected target.

Target feedback

Use a feedback join key

Use feedback_join_key when feedback is not tied to a single span, trace, or session. First, enrich your spans with the feedback_join_key tag related to the external entity using the SDK’s Enriching spans workflow or the Spans API. Then, submit feedback with the same feedback_join_key.

Examples

Submit thumbs-down feedback for a span

{
  "data": {
    "type": "evaluation_metric",
    "attributes": {
      "metrics": [
        {
          "event_kind": "feedback",
          "span_id": "20245611112024561111",
          "ml_app": "weather-bot",
          "timestamp_ms": 1765990800016,
          "metric_type": "categorical",
          "label": "thumbs",
          "categorical_value": "down",
          "assessment": "fail",
          "submitter": {
            "id": "user-123",
            "type": "user"
          }
        }
      ]
    }
  }
}

Submit free-text feedback with a feedback join key

{
  "data": {
    "type": "evaluation_metric",
    "attributes": {
      "metrics": [
        {
          "event_kind": "feedback",
          "feedback_join_key": "incident-123",
          "ml_app": "incident-agent",
          "timestamp_ms": 1765990800016,
          "metric_type": "text",
          "label": "user_comment",
          "text_value": "The investigation missed the customer impact.",
          "assessment": "fail",
          "submitter": {
            "id": "user-123",
            "type": "user"
          }
        }
      ]
    }
  }
}

Analyze feedback

To create a dashboard widget for feedback, create the widget as you would for an evaluation and add the filter @event_kind:feedback. To search and filter spans and traces by feedback in the Trace Explorer, see Feedback queries.

Further Reading

Additional helpful documentation, links, and articles:

Learn about the Evaluations APIDOCUMENTATION Learn about submitting external evaluationsDOCUMENTATION Learn about Annotation QueuesDOCUMENTATION