Overview

Attach images and audio to LLM spans to troubleshoot voice agents and agents that process images. Inspect the images a model received and play its input and output audio directly in the trace view, alongside other span data. You don’t need to reconstruct the interaction from logs or separate storage. For voice conversations, view each turn’s transcript alongside its audio.

Media belongs to individual messages within an LLM span. Automatic instrumentation captures media from supported libraries. For other libraries, use manual instrumentation to attach media. For OpenTelemetry spans, see OpenTelemetry instrumentation.

Requirements

The following Datadog SDK versions support attaching media to messages. For automatic capture, see the integration-specific requirements.

No minimum Datadog SDK version applies to the Agent Observability HTTP API or OpenTelemetry instrumentation. OpenTelemetry instrumentation must emit the supported GenAI message parts.

Automatic instrumentation

Datadog’s LLM integrations populate media parts for the following libraries, with no code change on your part.

Integrations don’t fetch images referenced by a remote URL or a file ID. They capture only images that the application sends inline as base64 data. When an integration cannot capture media, it records a text marker such as [image] or [audio] in the message.

OpenTelemetry instrumentation

Datadog extracts media from OpenTelemetry spans that contain the following GenAI message parts. No additional instrumentation is needed if your application already emits these parts:

• A blob part with a mime_type and inline bytes becomes an image or audio part. When the part omits modality, Datadog infers it from the MIME type.
• A uri part carrying a base64 image data URI, such as data:image/png;base64,..., becomes an image part.

Only media carried inline as base64 bytes is rendered. A remote URL is recorded as a text reference and is never fetched. For audio playback, send audio in blob parts. The conventions specify blob for inline base64 data, so prefer blob for both audio and images.

For the full mapping, see Media in messages.

Manual instrumentation

Attach media to an LLM span by adding image_parts or audio_parts to a message in input_data or output_data.

Each media part takes the following fields:

Datadog stores media separately during ingestion and loads it on demand in the trace view. Inline media still counts toward the span event size limit. Before attaching media, check the supported formats and size limits.

The examples focus on annotation and omit SDK setup and model calls. For SDK setup, see the Agent Observability SDK reference.

import base64
from ddtrace.llmobs import LLMObs
from ddtrace.llmobs.decorators import llm

@llm(model_name="gpt-4o", model_provider="openai")
def describe_image(image_bytes):
    resp = ...  # multimodal (vision) LLM call
    LLMObs.annotate(
        input_data=[
            {
                "role": "user",
                "content": "What is in this image?",
                "image_parts": [
                    {"mime_type": "image/png", "content": base64.b64encode(image_bytes).decode("utf-8")}
                ],
            }
        ],
        output_data=[{"role": "assistant", "content": "The image shows a golden retriever puppy."}],
    )
    return resp

import base64
from ddtrace.llmobs import LLMObs
from ddtrace.llmobs.decorators import llm

@llm(model_name="gpt-realtime", model_provider="openai")
def voice_turn(user_audio_bytes):
    resp = ...  # multimodal (audio) LLM call
    LLMObs.annotate(
        input_data=[
            {
                "role": "user",
                "content": "Hey, how are you?",  # transcript of the input audio
                "audio_parts": [
                    {"mime_type": "audio/wav", "content": base64.b64encode(user_audio_bytes).decode("utf-8")}
                ],
            }
        ],
        output_data=[
            {
                "role": "assistant",
                "content": "Hey! I'm doing great, thanks for asking. How about you?",
                "audio_parts": [
                    {"mime_type": "audio/wav", "content": base64.b64encode(resp.audio_bytes).decode("utf-8")}
                ],
            }
        ],
    )
    return resp

function describeImage (imageBytes) {
  const resp = ... // multimodal (vision) LLM call
  llmobs.annotate({
    inputData: [
      {
        role: 'user',
        content: 'What is in this image?',
        imageParts: [{ mimeType: 'image/png', content: imageBytes.toString('base64') }]
      }
    ],
    outputData: [{ role: 'assistant', content: 'The image shows a golden retriever puppy.' }]
  })
  return resp
}
describeImage = llmobs.wrap({ kind: 'llm', modelName: 'gpt-4o', modelProvider: 'openai' }, describeImage)

function voiceTurn (userAudioBytes) {
  const resp = ... // multimodal (audio) LLM call
  llmobs.annotate({
    inputData: [
      {
        role: 'user',
        content: 'Hey, how are you?', // transcript of the input audio
        audioParts: [{ mimeType: 'audio/wav', content: userAudioBytes.toString('base64') }]
      }
    ],
    outputData: [
      {
        role: 'assistant',
        content: "Hey! I'm doing great, thanks for asking. How about you?",
        audioParts: [{ mimeType: 'audio/wav', content: resp.audioBuffer.toString('base64') }]
      }
    ]
  })
  return resp
}
voiceTurn = llmobs.wrap({ kind: 'llm', modelName: 'gpt-audio', modelProvider: 'openai' }, voiceTurn)

{
  "role": "user",
  "content": "What is in this image?",
  "image_parts": [
    {
      "mime_type": "image/png",
      "content": "iVBORw0KGgoAAAANSUhEUgAA..."
    }
  ]
}

View images and play audio

Open a trace in Trace Explorer and select an LLM span. Media attached to the span’s messages renders inside the message it belongs to.

Images appear as thumbnails. Select a thumbnail to open the image at full size. Up to 12 images render per message.

Audio appears as a player, with the message text shown alongside it. For a voice conversation, that text is the turn transcript. Select the player to load and play the audio.

Supported formats

Images render when the MIME type is one of image/png, image/jpeg, image/webp, or image/gif, and the declared MIME type matches the encoded bytes. SVG images do not render. The trace view makes no outbound requests, so an image referenced by a remote URL is shown as text rather than fetched.

Audio plays through the browser’s native audio player. Supported containers include audio/wav, audio/mpeg, audio/ogg, and audio/webm. Raw formats such as pcm16, g711_ulaw, and g711_alaw do not play. Convert raw audio to WAV before attaching it. The Python OpenAI Realtime API integration does this conversion for you.

Limits and behavior

• Span event size. A span event is capped at 5 MB. When a span exceeds the cap, its input and output are replaced with a placeholder and dropped_io is added to the span’s collection_errors attribute. The span itself is kept.
• Per-part size. Automatic instrumentation in Python caps a single inline media part at 4 MiB. Above the cap, the integration records a text marker such as [audio] or [image omitted: too large] and preserves the surrounding message text and model response.
• Sensitive Data Scanner. Media content is not scanned by Sensitive Data Scanner. Message text, tool arguments, and tool results are scanned as usual.

Best practices

• Use the typed media fields. Attach media with image_parts and audio_parts rather than embedding base64 strings in a message’s content. Only typed fields render in the trace view.
• Attach the transcript with the audio. Set the message content to the turn transcript so that the span is readable, searchable, and usable by evaluators even before the audio is played.

Further reading

Additional helpful documentation, links, and articles:

Agent Observability SDK ReferenceDOCUMENTATION Automatic Instrumentation for Agent ObservabilityDOCUMENTATION Agent Observability HTTP API ReferenceDOCUMENTATION