Example LLM Observability trace for Vercel AI SDK usage

Overview

Use the Vercel AI SDK integration to monitor, troubleshoot, and evaluate your applications that use the Vercel AI SDK.

The Vercel AI SDK auto-instrumentation allows you to:

• Visualize LLM calls with their associated token usage and costs
• See tool calls made by your application, with their respective inputs and outputs
• See embedded calls made by your application and their associated token usage and costs
• See the flow of handoffs between LLMs and the tools they call, allowing you to identify LLM retries and infinite tool calling loops

Setup

1. Install the dd-trace package:

npm install dd-trace

2. Run your application using the following Node options to initialize the LLM Observability auto-instrumentation:

DD_API_KEY=<YOUR_DD_API_KEY> DD_SITE=<YOUR_DD_SITE> DD_LLMOBS_ENABLED=true DD_LLMOBS_AGENTLESS_ENABLED=true DD_LLMOBS_ML_APP=<YOUR_ML_APP_NAME> node --import dd-trace/initialize.mjs <YOUR-APP>.js

3. (Optional) If using the Datadog Agent, ensure it is running with the following command:

docker run -d \
    --cgroupns host \
    --pid host \
    -v /var/run/docker.sock:/var/run/docker.sock:ro \
    -v /proc/:/host/proc/:ro \
    -v /sys/fs/cgroup/:/host/sys/fs/cgroup:ro \
    -e DD_API_KEY=<DATADOG_API_KEY> \
    -p 127.0.0.1:8126:8126/tcp \
    -p 127.0.0.1:8125:8125/udp \
    -e DD_DOGSTATSD_NON_LOCAL_TRAFFIC=true \
    -e DD_APM_ENABLED=true \
    gcr.io/datadoghq/agent:latest

4. Before running your application with the Agent, unset the DD_LLMOBS_AGENTLESS_ENABLED, DD_SITE, and DD_API_KEY environment variables.

Uninstallation

Unset the DD_ environment variables, and remove the --import dd-trace/initialize.mjs from the command line for your deployment using the Vercel AI SDK.

Support

Need help? Contact Datadog Support.

Further Reading

Additional helpful documentation, links, and articles:

Auto InstrumentationDOCUMENTATION