This product is not supported for your selected Datadog site. ().
Overview
Attach images and audio to LLM spans to troubleshoot voice agents and agents that process images. Inspect the images a model received and play its input and output audio directly in the trace view, alongside other span data. You don’t need to reconstruct the interaction from logs or separate storage. For voice conversations, view each turn’s transcript alongside its audio.
Input input_audio content and output audio, including the model’s audio transcript
>= 4.12.0
OpenAI Realtime API
Per-turn input and output audio, transcripts, token usage, and tool calls
>= 4.13.0
Amazon Nova 2 Sonic (aws-sdk-bedrock-runtime)
Per-turn input and output audio, transcripts, token usage, and tool calls
>= 4.17.0
ElevenLabs Agents (WebSocket)
Conversation input and output audio, transcripts, and client tool calls
>= 4.17.0
LiteLLM
Input and output audio, through the OpenAI chat completions format
>= 4.12.0
OpenAI chat completions and Responses API
Inline base64-encoded images
>= 4.15.0
OpenAI Agents
Inline base64-encoded images
>= 4.15.0
Anthropic
Inline base64-encoded images
>= 4.15.0
Nova 2 Sonic and ElevenLabs require ddtrace 4.17 or later. Nova 2 Sonic also requires Python 3.12+ and aws-sdk-bedrock-runtime 0.11.0+; ElevenLabs requires elevenlabs 2.70.0+.
Audio in streamed chat completions (stream=True) is not captured.
Integration
Media captured
Datadog SDK version
OpenAI chat completions
Input input_audio content and output audio, including the model’s audio transcript
v6: >= 6.7.0; v5: >= 5.118.0
OpenAI Agents
Input and output audio, through the OpenAI chat completions format
v6: >= 6.7.0; v5: >= 5.118.0
OpenAI Realtime API
Per-turn input and output audio, transcripts, token usage, and tool calls
v6: >= 6.17.0; v5: >= 5.128.0
OpenAI Agents Realtime SDK
Per-turn input and output audio, transcripts, token usage, and tool calls for WebSocket and SIP sideband connections
v6: >= 6.25.0; v5: >= 5.136.0
Amazon Nova 2 Sonic (@aws-sdk/client-bedrock-runtime)
Per-turn input and output audio, transcripts, token usage, and tool calls
v6: >= 6.22.0; v5: >= 5.133.0
ElevenLabs Agents (WebSocket)
Conversation input and output audio, transcripts, and client tool calls
v6: >= 6.25.0; v5: >= 5.136.0
OpenAI Agents Realtime SDK instrumentation requires @openai/agents-realtime 0.20.0 or later. To disable tracer-issued audio retrievals for SIP sidebands, set DD_OPENAI_REALTIME_SIDEBAND_AUDIO_RETRIEVAL_ENABLED=false.
Audio in streamed chat completions is not captured.
Voice conversation playback and turn transcripts captured with automatic instrumentation.
Integrations don’t fetch images referenced by a remote URL or a file ID. They capture only images that the application sends inline as base64 data. When an integration cannot capture media, it records a text marker such as [image] or [audio] in the message.
OpenTelemetry instrumentation
Datadog extracts media from OpenTelemetry spans that contain the following GenAI message parts. No additional instrumentation is needed if your application already emits these parts:
A blob part with a mime_type and inline bytes becomes an image or audio part. When the part omits modality, Datadog infers it from the MIME type.
A uri part carrying a base64 image data URI, such as data:image/png;base64,..., becomes an image part.
Only media carried inline as base64 bytes is rendered. A remote URL is recorded as a text reference and is never fetched. For audio playback, send audio in blob parts. The conventions specify blob for inline base64 data, so prefer blob for both audio and images.
Attach media to an LLM span by adding image_parts or audio_parts to a message in input_data or output_data.
Each media part takes the following fields:
Field
Type
Required
Description
mime_type
string
Yes
The media type of the attachment, such as image/png or audio/wav.
content
string
Yes
The base64-encoded media.
Datadog stores media separately during ingestion and loads it on demand in the trace view. Inline media still counts toward the span event size limit. Before attaching media, check the supported formats and size limits.
importbase64fromddtrace.llmobsimportLLMObsfromddtrace.llmobs.decoratorsimportllm@llm(model_name="gpt-4o",model_provider="openai")defdescribe_image(image_bytes):resp=...# multimodal (vision) LLM callLLMObs.annotate(input_data=[{"role":"user","content":"What is in this image?","image_parts":[{"mime_type":"image/png","content":base64.b64encode(image_bytes).decode("utf-8")}],}],output_data=[{"role":"assistant","content":"The image shows a golden retriever puppy."}],)returnresp
Attach audio and a transcript
importbase64fromddtrace.llmobsimportLLMObsfromddtrace.llmobs.decoratorsimportllm@llm(model_name="gpt-realtime",model_provider="openai")defvoice_turn(user_audio_bytes):resp=...# multimodal (audio) LLM callLLMObs.annotate(input_data=[{"role":"user","content":"Hey, how are you?",# transcript of the input audio"audio_parts":[{"mime_type":"audio/wav","content":base64.b64encode(user_audio_bytes).decode("utf-8")}],}],output_data=[{"role":"assistant","content":"Hey! I'm doing great, thanks for asking. How about you?","audio_parts":[{"mime_type":"audio/wav","content":base64.b64encode(resp.audio_bytes).decode("utf-8")}],}],)returnresp
If a media part is missing mime_type or content, annotation fails.
Node.js uses the camelCase field names imageParts and audioParts, with mimeType on each part.
Attach an image
functiondescribeImage(imageBytes){constresp=...// multimodal (vision) LLM call
llmobs.annotate({inputData:[{role:'user',content:'What is in this image?',imageParts:[{mimeType:'image/png',content:imageBytes.toString('base64')}]}],outputData:[{role:'assistant',content:'The image shows a golden retriever puppy.'}]})returnresp}describeImage=llmobs.wrap({kind:'llm',modelName:'gpt-4o',modelProvider:'openai'},describeImage)
Attach audio and a transcript
functionvoiceTurn(userAudioBytes){constresp=...// multimodal (audio) LLM call
llmobs.annotate({inputData:[{role:'user',content:'Hey, how are you?',// transcript of the input audio
audioParts:[{mimeType:'audio/wav',content:userAudioBytes.toString('base64')}]}],outputData:[{role:'assistant',content:"Hey! I'm doing great, thanks for asking. How about you?",audioParts:[{mimeType:'audio/wav',content:resp.audioBuffer.toString('base64')}]}]})returnresp}voiceTurn=llmobs.wrap({kind:'llm',modelName:'gpt-audio',modelProvider:'openai'},voiceTurn)
A media part that is missing mimeType or content is dropped. The rest of the message is still recorded.
Submit audio_parts and image_parts on the message objects you send to the Spans API endpoint. The field names match the Python SDK.
{"role":"user","content":"What is in this image?","image_parts":[{"mime_type":"image/png","content":"iVBORw0KGgoAAAANSUhEUgAA..."}]}
Open a trace in Trace Explorer and select an LLM span. Media attached to the span’s messages renders inside the message it belongs to.
Images appear as thumbnails. Select a thumbnail to open the image at full size. Up to 12 images render per message.
Audio appears as a player, with the message text shown alongside it. For a voice conversation, that text is the turn transcript. Select the player to load and play the audio.
Supported formats
Images render when the MIME type is one of image/png, image/jpeg, image/webp, or image/gif, and the declared MIME type matches the encoded bytes. SVG images do not render. The trace view makes no outbound requests, so an image referenced by a remote URL is shown as text rather than fetched.
Audio plays through the browser’s native audio player. Supported containers include audio/wav, audio/mpeg, audio/ogg, and audio/webm. Raw formats such as pcm16, g711_ulaw, and g711_alaw do not play. Convert raw audio to WAV before attaching it. The automatic OpenAI Realtime API, Nova 2 Sonic, and ElevenLabs integrations convert supported raw audio to WAV.
Limits and behavior
Span event size. A span event is capped at 5 MB. When a span exceeds the cap, its input and output are replaced with a placeholder and dropped_io is added to the span’s collection_errors attribute. The span itself is kept.
Per-part size. Automatic instrumentation in Python caps a single inline media part at 4 MiB. Above the cap, the integration records a text marker such as [audio] or [image omitted: too large] and preserves the surrounding message text and model response.
Sensitive Data Scanner. Media content is not scanned by Sensitive Data Scanner. Message text, tool arguments, and tool results are scanned as usual.
Best practices
Use the typed media fields. Attach media with image_parts and audio_parts rather than embedding base64 strings in a message’s content. Only typed fields render in the trace view.
Attach the transcript with the audio. Set the message content to the turn transcript so that the span is readable, searchable, and usable by evaluators even before the audio is played.
Further reading
Additional helpful documentation, links, and articles: