--- title: Amazon Bedrock description: Amazon Bedrock makes AI foundation models available through an API. breadcrumbs: Docs > Integrations > Amazon Bedrock --- # Amazon Bedrock ## Overview{% #overview %} Monitor, troubleshoot, and evaluate your LLM-powered applications, such as chatbots or data extraction tools, using Amazon Bedrock. If you are building LLM applications, use LLM Observability to investigate the root cause of issues, monitor operational performance, and evaluate the quality, privacy, and safety of your LLM applications. See the [LLM Observability tracing view video](https://imgix.datadoghq.com/video/products/llm-observability/expedite-troubleshooting.mp4?fm=webm&fit=max) for an example of how you can investigate a trace. Amazon Bedrock is a fully managed service that makes [foundation models](https://aws.amazon.com/what-is/foundation-models/) (FMs) from Amazon and leading AI startups available through an API, so you can choose from various FMs to find the model that's best suited for your use case. Enable this integration to see all your Bedrock metrics in Datadog. ## Setup{% #setup %} ### LLM Observability: Get end-to-end visibility into your LLM application using Amazon Bedrock{% #llm-observability-get-end-to-end-visibility-into-your-llm-application-using-amazon-bedrock %} You can enable LLM Observability in different environments. Follow the appropriate setup based on your scenario: #### Installation for Python{% #installation-for-python %} ##### If you do not have the Datadog Agent:{% #if-you-do-not-have-the-datadog-agent %} 1. Install the `ddtrace` package: ```shell pip install ddtrace ``` Start your application with the following command, enabling Agentless mode: ```shell DD_SITE= DD_API_KEY= DD_LLMOBS_ENABLED=1 DD_LLMOBS_AGENTLESS_ENABLED=1 DD_LLMOBS_ML_APP= ddtrace-run python .py ``` ##### If you already have the Datadog Agent installed:{% #if-you-already-have-the-datadog-agent-installed %} 1. Make sure the Agent is running and that APM and StatsD are enabled. For example, use the following command with Docker: ```shell docker run -d \ --cgroupns host \ --pid host \ -v /var/run/docker.sock:/var/run/docker.sock:ro \ -v /proc/:/host/proc/:ro \ -v /sys/fs/cgroup/:/host/sys/fs/cgroup:ro \ -e DD_API_KEY= \ -p 127.0.0.1:8126:8126/tcp \ -p 127.0.0.1:8125:8125/udp \ -e DD_DOGSTATSD_NON_LOCAL_TRAFFIC=true \ -e DD_APM_ENABLED=true \ gcr.io/datadoghq/agent:latest ``` If you haven't already, install the `ddtrace` package: ```shell pip install ddtrace ``` Start your application using the `ddtrace-run` command to automatically enable tracing: ```shell DD_SITE= DD_API_KEY= DD_LLMOBS_ENABLED=1 DD_LLMOBS_ML_APP= ddtrace-run python .py ``` **Note**: If the Agent is running on a custom host or port, set `DD_AGENT_HOST` and `DD_TRACE_AGENT_PORT` accordingly. ##### If you are running LLM Observability in a serverless environment (AWS Lambda):{% #if-you-are-running-llm-observability-in-a-serverless-environment-aws-lambda %} 1. Install the **Datadog-Python** and **Datadog-Extension** Lambda layers as part of your AWS Lambda setup. 1. Enable LLM Observability by setting the following environment variables: ```shell DD_SITE= DD_API_KEY= DD_LLMOBS_ENABLED=1 DD_LLMOBS_ML_APP= ``` **Note**: In serverless environments, Datadog automatically flushes spans when the Lambda function finishes running. ##### Automatic Amazon Bedrock tracing{% #automatic-amazon-bedrock-tracing %} The Amazon Bedrock integration is automatically enabled when LLM Observability is configured. This captures latency, errors, input and output messages, as well as token usage for Amazon Bedrock calls. The following methods are traced for both synchronous and streamed Amazon Bedrock operations: - `InvokeModel()` - `InvokeModelWithResponseStream()` No additional setup is required for these methods. ##### Validation{% #validation %} Validate that LLM Observability is properly capturing spans by checking your application logs for successful span creation. You can also run the following command to check the status of the `ddtrace` integration: ```shell ddtrace-run --info ``` Look for the following message to confirm the setup: ```shell Agent error: None ``` ##### Debugging{% #debugging %} If you encounter issues during setup, enable debug logging by passing the `--debug` flag: ```shell ddtrace-run --debug ``` This displays any errors related to data transmission or instrumentation, including issues with Amazon Bedrock traces. #### Installation for Node.js{% #installation-for-nodejs %} ##### If you do not have the Datadog Agent:{% #if-you-do-not-have-the-datadog-agent-1 %} 1. Install the `dd-trace` package: ```shell npm install dd-trace ``` 1. Start your application with the following command, enabling agentless mode: ```shell DD_SITE= DD_API_KEY= DD_LLMOBS_ENABLED=1 DD_LLMOBS_AGENTLESS_ENABLED=1 DD_LLMOBS_ML_APP= node -r 'dd-trace/init' .js ``` ##### If you already have the Datadog Agent installed:{% #if-you-already-have-the-datadog-agent-installed-1 %} 1. Make sure the Agent is running and that APM is enabled. For example, use the following command with Docker: ```shell docker run -d \ --cgroupns host \ --pid host \ -v /var/run/docker.sock:/var/run/docker.sock:ro \ -v /proc/:/host/proc/:ro \ -v /sys/fs/cgroup/:/host/sys/fs/cgroup:ro \ -e DD_API_KEY= \ -p 127.0.0.1:8126:8126/tcp \ -p 127.0.0.1:8125:8125/udp \ -e DD_DOGSTATSD_NON_LOCAL_TRAFFIC=true \ -e DD_APM_ENABLED=true \ gcr.io/datadoghq/agent:latest ``` 1. Install the Datadog APM Node.js library. ```shell npm install dd-trace ``` 1. Start your application using the `-r dd-trace/init` or `NODE_OPTIONS='--require dd-trace/init'` command to automatically enable tracing: ```shell DD_SITE= DD_API_KEY= DD_LLMOBS_ENABLED=1 DD_LLMOBS_ML_APP= node -r 'dd-trace/init' .js ``` **Note**: If the Agent is running on a custom host or port, set `DD_AGENT_HOST` and `DD_TRACE_AGENT_PORT` accordingly. ##### If you are running LLM Observability in a serverless environment (AWS Lambda):{% #if-you-are-running-llm-observability-in-a-serverless-environment-aws-lambda-1 %} 1. Enable LLM Observability by setting the following environment variables: ```shell DD_SITE= DD_API_KEY= DD_LLMOBS_ENABLED=1 DD_LLMOBS_ML_APP= ``` 1. Before the lambda finishes, call `llmobs.flush()`: ```js const llmobs = require('dd-trace').llmobs; // or, if dd-trace was not initialized via NODE_OPTIONS const llmobs = require('dd-trace').init({ llmobs: { mlApp: , } }).llmobs; // with DD_API_KEY and DD_SITE being set at the environment level async function handler (event, context) { ... llmobs.flush() return ... } ``` ### APM: Get Usage Metrics for Python Applications{% #apm-get-usage-metrics-for-python-applications %} If you haven't already, set up the [Amazon Web Services integration](https://docs.datadoghq.com/integrations/amazon_web_services.md) first. ### Metric collection{% #metric-collection %} 1. In the [AWS integration page](https://app.datadoghq.com/integrations/amazon-web-services), ensure that `Bedrock` is enabled under the `Metric Collection` tab. 1. Install the [Datadog - Amazon Bedrock integration](https://app.datadoghq.com/integrations/amazon-bedrock). ## Data Collected{% #data-collected %} ### Metrics{% #metrics %} | | | | | **aws.bedrock.content\_filtered\_count**(count) | The total number of times the text output content was filtered.*Shown as time* | | **aws.bedrock.input\_token\_count**(gauge) | The average number of input tokens used in prompts invoked for a model.*Shown as token* | | **aws.bedrock.input\_token\_count.maximum**(gauge) | The maximum number of input tokens used in prompts invoked for a model.*Shown as token* | | **aws.bedrock.input\_token\_count.minimum**(gauge) | The minimum number of input tokens used in prompts invoked for a model.*Shown as token* | | **aws.bedrock.input\_token\_count.sum**(count) | The total number of input tokens used in prompts invoked for a model.*Shown as token* | | **aws.bedrock.invocation\_client\_errors**(count) | The number of client invocation errors.*Shown as error* | | **aws.bedrock.invocation\_latency**(gauge) | Average latency of the invocations in milliseconds.*Shown as millisecond* | | **aws.bedrock.invocation\_latency.maximum**(gauge) | The maximum invocation latency over a one-minute period.*Shown as millisecond* | | **aws.bedrock.invocation\_latency.minimum**(gauge) | The minimum invocation latency over a one-minute period.*Shown as millisecond* | | **aws.bedrock.invocation\_latency.p90**(gauge) | The 90th percentile of invocation latency over a one-minute period.*Shown as millisecond* | | **aws.bedrock.invocation\_latency.p95**(gauge) | The 95th percentile of invocation latency over a 1 minute period.*Shown as millisecond* | | **aws.bedrock.invocation\_latency.p99**(gauge) | The 99th percentile of invocation latency over a one-minute period.*Shown as millisecond* | | **aws.bedrock.invocation\_server\_errors**(count) | The number of server invocation errors.*Shown as error* | | **aws.bedrock.invocation\_throttles**(count) | The number of invocation throttles.*Shown as throttle* | | **aws.bedrock.invocations**(count) | The number of invocations sent to a model endpoint.*Shown as invocation* | | **aws.bedrock.output\_image\_count**(gauge) | The average number of output images returned by model invocations over a 1 minute period.*Shown as item* | | **aws.bedrock.output\_token\_count**(gauge) | The average number of output tokens returned by model invocations over a 1 minute period.*Shown as token* | | **aws.bedrock.output\_token\_count.maximum**(gauge) | The maximum number of output tokens returned by model invocations over a one-minute period.*Shown as token* | | **aws.bedrock.output\_token\_count.minimum**(gauge) | The minimum number of output tokens returned by model invocations over a one-minute period.*Shown as token* | | **aws.bedrock.output\_token\_count.sum**(count) | The total number of output tokens returned by all model invocations.*Shown as token* | | **aws.bedrockagentcore.active\_streaming\_connections**(gauge) | The current number of active WebSocket connections per agent.*Shown as connection* | | **aws.bedrockagentcore.allow\_decisions**(count) | The number of policy decisions that resulted in ALLOW. | | **aws.bedrockagentcore.api\_key\_fetch\_failures**(count) | The number of failed API key fetch operations by exception type.*Shown as error* | | **aws.bedrockagentcore.api\_key\_fetch\_success**(count) | The number of successful API key fetch operations. | | **aws.bedrockagentcore.connection\_duration**(gauge) | The amount of time that connection stays open.*Shown as millisecond* | | **aws.bedrockagentcore.connection\_duration.maximum**(gauge) | The maximum amount of time that connection stays open.*Shown as millisecond* | | **aws.bedrockagentcore.connection\_duration.minimum**(gauge) | The minimum amount of time that connection stays open.*Shown as millisecond* | | **aws.bedrockagentcore.connection\_duration.p90**(gauge) | The 90th percentile amount of time that connection stays open.*Shown as millisecond* | | **aws.bedrockagentcore.connection\_duration.p99**(gauge) | The 99th percentile amount of time that connection stays open.*Shown as millisecond* | | **aws.bedrockagentcore.cpu\_used\_vcpu\_hours**(count) | The total amount of virtual CPU consumed in vCPU-hours.*Shown as hour* | | **aws.bedrockagentcore.creation\_count**(count) | The number of created memory events and memory records. | | **aws.bedrockagentcore.deny\_decisions**(count) | The number of policy decisions that resulted in DENY. | | **aws.bedrockagentcore.determining\_policies**(count) | The number of determining policies for a request. | | **aws.bedrockagentcore.duration**(gauge) | The time elapsed between receiving the request and sending the final response token.*Shown as millisecond* | | **aws.bedrockagentcore.duration.maximum**(gauge) | The maximum time elapsed between receiving the request and sending the final response token.*Shown as millisecond* | | **aws.bedrockagentcore.duration.minimum**(gauge) | The minimum time elapsed between receiving the request and sending the final response token.*Shown as millisecond* | | **aws.bedrockagentcore.duration.p90**(gauge) | The 90th percentile time elapsed between receiving the request and sending the final response token.*Shown as millisecond* | | **aws.bedrockagentcore.duration.p99**(gauge) | The 99th percentile time elapsed between receiving the request and sending the final response token.*Shown as millisecond* | | **aws.bedrockagentcore.errors**(count) | The total number of system and user errors.*Shown as error* | | **aws.bedrockagentcore.failed\_extraction**(count) | The number of failed extraction jobs.*Shown as error* | | **aws.bedrockagentcore.inbound\_authorization\_failure**(count) | The total number of inbound authorization failures.*Shown as error* | | **aws.bedrockagentcore.inbound\_authorization\_success**(count) | The total number of successful inbound authorization operations. | | **aws.bedrockagentcore.inbound\_streaming\_bytes\_processed**(count) | The total number of bytes processed in WebSocket frames received from clients to agent containers.*Shown as byte* | | **aws.bedrockagentcore.invocations**(count) | The total number of requests made to the Data Plane API.*Shown as invocation* | | **aws.bedrockagentcore.latency**(gauge) | The time elapsed between receiving the request and sending the first response token.*Shown as millisecond* | | **aws.bedrockagentcore.latency.maximum**(gauge) | Maximum time elapsed between receiving the request and sending the first response token.*Shown as millisecond* | | **aws.bedrockagentcore.latency.minimum**(gauge) | The minimum time elapsed between receiving the request and sending the first response token.*Shown as millisecond* | | **aws.bedrockagentcore.latency.p90**(gauge) | The 90th percentile time elapsed between receiving the request and sending the first response token.*Shown as millisecond* | | **aws.bedrockagentcore.latency.p99**(gauge) | The 99th percentile time elapsed between receiving the request and sending the first response token.*Shown as millisecond* | | **aws.bedrockagentcore.memory\_used\_gb\_hours**(count) | The total amount of memory consumed in GB-hours.*Shown as hour* | | **aws.bedrockagentcore.no\_determining\_policies**(count) | The number of requests denied due to no determining policies. | | **aws.bedrockagentcore.outbound\_streaming\_bytes\_processed**(count) | The total number of bytes processed in WebSocket frames sent from agent containers to clients.*Shown as byte* | | **aws.bedrockagentcore.resource\_access\_token\_fetch\_failures**(count) | The number of failed OAuth2 token fetch operations by exception type.*Shown as error* | | **aws.bedrockagentcore.resource\_access\_token\_fetch\_success**(count) | The number of successful OAuth2 token fetch operations from credential providers. | | **aws.bedrockagentcore.sessions**(gauge) | The number of agent sessions.*Shown as connection* | | **aws.bedrockagentcore.sessions.maximum**(gauge) | The maximum number of agent sessions.*Shown as connection* | | **aws.bedrockagentcore.sessions.minimum**(gauge) | The minimum number of agent sessions.*Shown as connection* | | **aws.bedrockagentcore.stream\_publishing\_failure**(count) | The number of memory record events that failed to publish to Kinesis Data Stream.*Shown as error* | | **aws.bedrockagentcore.system\_errors**(count) | The number of server-side errors encountered during request processing.*Shown as error* | | **aws.bedrockagentcore.take\_over\_count**(count) | The total number of browser user takeover events. | | **aws.bedrockagentcore.take\_over\_release\_count**(count) | The total number of browser user control release events. | | **aws.bedrockagentcore.target\_execution\_time**(gauge) | The time taken to execute the target over Lambda, OpenAPI, or other integrations.*Shown as millisecond* | | **aws.bedrockagentcore.target\_execution\_time.maximum**(gauge) | The maximum time taken to execute the target over Lambda, OpenAPI, or other integrations.*Shown as millisecond* | | **aws.bedrockagentcore.target\_execution\_time.minimum**(gauge) | The minimum time taken to execute the target over Lambda, OpenAPI, or other integrations.*Shown as millisecond* | | **aws.bedrockagentcore.target\_execution\_time.p90**(gauge) | The 90th percentile time taken to execute the target over Lambda, OpenAPI, or other integrations.*Shown as millisecond* | | **aws.bedrockagentcore.target\_execution\_time.p99**(gauge) | The 99th percentile time taken to execute the target over Lambda, OpenAPI, or other integrations.*Shown as millisecond* | | **aws.bedrockagentcore.target\_type\_apigateway**(count) | The total number of requests served by API Gateway targets. | | **aws.bedrockagentcore.target\_type\_lambda**(count) | The total number of requests served by Lambda targets. | | **aws.bedrockagentcore.target\_type\_mcp**(count) | The total number of requests served by MCP targets. | | **aws.bedrockagentcore.target\_type\_openapi**(count) | The total number of requests served by OpenAPI targets. | | **aws.bedrockagentcore.target\_type\_smithy**(count) | The total number of requests served by Smithy targets. | | **aws.bedrockagentcore.throttles**(count) | The number of requests throttled by the service due to exceeding allowed rate limits.*Shown as throttle* | | **aws.bedrockagentcore.token\_count**(count) | The number of tokens used.*Shown as token* | | **aws.bedrockagentcore.total\_mismatched\_policies**(count) | The number of failed policies for a request due to missing attribute or type mismatch. | | **aws.bedrockagentcore.user\_errors**(count) | The number of client-side errors resulting from invalid requests.*Shown as error* | | **aws.bedrockagentcore.workload\_access\_token\_fetch\_failures**(count) | The number of failed workload access token fetch operations by exception type.*Shown as error* | | **aws.bedrockagentcore.workload\_access\_token\_fetch\_success**(count) | The number of successful workload access token fetch operations. | ### Events{% #events %} The Amazon Bedrock integration does not include any events. ### Service Checks{% #service-checks %} The Amazon Bedrock integration does not include any service checks. ## Troubleshooting{% #troubleshooting %} Need help? Contact [Datadog support](https://docs.datadoghq.com/help/). ## Further Reading{% #further-reading %} - [Monitor Amazon Bedrock with Datadog](https://www.datadoghq.com/blog/monitor-amazon-bedrock-with-datadog/)