--- title: Set Up AI Guard description: >- Set up AI Guard to evaluate and block unsafe prompts, responses, and tool calls in your AI agents. breadcrumbs: Docs > Datadog Security > AI Guard > Set Up AI Guard --- > For the complete documentation index, see [llms.txt](https://docs.datadoghq.com/llms.txt). # Set Up AI Guard {% callout %} # Important note for users on the following Datadog sites: app.ddog-gov.com, us2.ddog-gov.com {% alert level="danger" %} AI Guard isn't available in the {% placeholder "user-datadog-site-name" /%} site. {% /alert %} {% /callout %} Set up AI Guard to start evaluating agent prompts, responses, and tool calls for threats. After setup, the default policy monitors every evaluation, and you can [start blocking unsafe requests](https://docs.datadoghq.com/security/ai_guard/policies.md#blocking-policy) when you're ready. AI Guard connects to AI agents through several integration methods: automatic integrations, manual integrations, or the SDK. The setup steps detailed here are the same for each of these methods. When you're instrumenting your application, you choose the method that fits your agent's language and libraries. AI Guard protects each agent as a separate service. Complete these steps for every service you want to protect. ## 1. Check prerequisites{% #prerequisites %} Confirm you have the Datadog permissions, the Datadog Agent, and the Datadog SDK version that AI Guard needs. ### Permissions{% #permissions %} Some setup steps require specific Datadog permissions. An admin might need to create a role with these permissions and assign it to you: | Permission | ID | Type | Description | | --------------------------------- | ------------------------------------------------- | ----- | ----------------------------------------------------------------------------------------------------------------- | | AI Guard Evaluate | `ai_guard_evaluate` | Write | Required to call the AI Guard evaluation API and to create an application key with the `ai_guard_evaluate` scope. | | AI Guard View | `ai_guard_view` | Read | Required to view the AI Guard UI, including signals, spans, and read-only settings. | | AI Guard Write | `ai_guard_write` | Write | Required to change AI Guard configuration, such as policies, custom rules, and sensitive data scanning. | | APM Retention Filters Write | `apm_retention_filter_write` | Write | Required to create the retention filter for AI Guard spans. | | Security Notification Rules Read | `security_monitoring_notification_profiles_read` | Read | Required to view notification rules for AI Guard security signals. | | Security Notification Rules Write | `security_monitoring_notification_profiles_write` | Write | Required to create and edit notification rules for AI Guard security signals. | | Security Rules Read | `security_monitoring_rules_read` | Read | Required to view AI Guard detection rules. | | Security Rules Write | `security_monitoring_rules_write` | Write | Required to create and edit AI Guard detection rules. | | Security Signals Read | `security_monitoring_signals_read` | Read | Required to view AI Guard security signals. | | Security Signals Write | `security_monitoring_signals_write` | Write | Required to triage AI Guard security signals. | | User Access Manage | `user_access_manage` | Write | Required only to restrict access to AI Guard spans with Data Access Control. | ### Datadog Agent and Datadog SDK{% #datadog-agent-and-datadog-sdk %} Every integration method runs on the Datadog SDK and sends AI Guard data through the Datadog Agent. The Agent must be running and reachable from your application. AI Guard requires these minimum Datadog SDK versions: | Language | Datadog SDK | Minimum version | | -------- | --------------- | ----------------- | | Python | `dd-trace-py` | 4.13.0 | | Node.js | `dd-trace-js` | 5.122.0 or 6.11.0 | | Java | `dd-trace-java` | 1.54.0 | | Ruby | `dd-trace-rb` | 2.44.0 | Some features require a later version, such as streamed response evaluation and sensitive data redaction. Each page lists its own requirements. To take advantage of all AI Guard features, use the latest version of the Datadog SDK for your language. {% alert level="info" %} By default, running the Datadog SDK and the Agent turns on other Datadog products, which are billed separately from AI Guard: - The Datadog SDK sends full APM traces. - To turn off APM tracing but keep AI Guard, set `DD_APM_TRACING_ENABLED=false`. AI Guard spans still reach Datadog, and indexed AI Guard spans count toward your APM indexed span usage. See Create a retention filter for AI Guard spans. - Don't set `DD_TRACE_ENABLED=false`, because that setting turns off AI Guard spans and integrations too. - The Agent reports Infrastructure Monitoring data. To turn off Infrastructure Monitoring, set `DD_INFRASTRUCTURE_MODE=none`. This setting requires Agent 7.77.0 or later. {% /alert %} ## 2. Create API and application keys{% #create-keys %} AI Guard authenticates every evaluation with a Datadog API key and application key, so your application needs both before it can send evaluations. 1. Create an API key and an application key. See [API and Application Keys](https://docs.datadoghq.com/account_management/api-app-keys.md). 1. When you add [scopes](https://docs.datadoghq.com/account_management/api-app-keys.md#scopes) to the application key, add the `ai_guard_evaluate` scope. The user who creates the application key must have the AI Guard Evaluate permission. 1. Store both keys where your application reads its environment variables. Each integration page lists the variables to set. ## 3. Instrument your application{% #instrumentation %} Instrument each AI agent with the integration method that matches its language and libraries, so AI Guard can evaluate model traffic. AI Guard identifies each protected agent by its `DD_SERVICE` and `DD_ENV` values, and applies policies by service and environment. Use the same service name that the agent reports to APM. The AI Guard Discover tab lists the AI agent services in your organization and shows which ones AI Guard protects, matched by service name. ### Find the agent language and libraries{% #find-the-agent-language-and-libraries %} If you don't know which language or AI libraries an agent uses, check its APM service page. 1. In APM, open the agent's service page. 1. Find the language icon next to the service name. 1. Open the operation dropdown. Operations such as `openai.request`, `anthropic.request`, `langchain.request`, or `litellm.request` show which AI libraries the agent calls. ### Choose an integration method{% #choose-an-integration-method %} The appropriate integration method depends on how an agent reaches its model and which language and libraries the agent uses. Choose the first integration method that applies to the agent: 1. If the agent's model traffic goes through LiteLLM Proxy, use the [LiteLLM Proxy integration](https://docs.datadoghq.com/security/ai_guard/setup/manual_integrations.md#litellm-proxy). 1. If the agent uses one of these AI frameworks or client libraries, use the matching [automatic integration](https://docs.datadoghq.com/security/ai_guard/setup/automatic_integrations.md) or [manual integration](https://docs.datadoghq.com/security/ai_guard/setup/manual_integrations.md): | Framework or library | Language | Method | | -------------------- | --------------- | --------------------- | | LangChain | Python | Automatic integration | | OpenAI SDK | Python, Node.js | Automatic integration | | Anthropic SDK | Python, Node.js | Automatic integration | | AI SDK | Node.js | Automatic integration | | RubyLLM | Ruby | Automatic integration | | Amazon Strands | Python | Manual integration | 1. If no integration covers the agent's framework, but the agent is written in Python, Node.js, Java, or Ruby, use the [SDK](https://docs.datadoghq.com/security/ai_guard/setup/sdk.md). 1. If none of these apply, use the [HTTP API](https://docs.datadoghq.com/security/ai_guard/setup/http_api.md). HTTP API evaluations don't create spans, so they don't appear on any AI Guard page in Datadog and don't generate signals. Skip the remaining setup steps. ## 4. Create a retention filter for AI Guard spans{% #retention-filter %} Create a retention filter so Datadog keeps every AI Guard span. By default, APM samples spans. Without a retention filter, even though AI Guard still evaluates your agent's requests, Datadog can drop the corresponding spans, preventing them from appearing in AI Guard. You must have the APM Retention Filters Write permission to complete this task. 1. Create a [custom retention filter](https://docs.datadoghq.com/tracing/trace_pipeline/trace_retention.md#create-your-own-retention-filter) with these settings: - Retention query: `resource_name:ai_guard` in All Spans. - Span rate: 100%. - Trace rate: 100%. 1. Drag the AI Guard filter to the top of the Custom Retention Filters list. New filters are added to the bottom of the list, and Datadog applies only the first filter that matches a span, so a filter higher in the list with a lower rate can drop AI Guard spans. Keeping AI Guard spans can add to your APM bill. APM bills separately for spans that Datadog receives, called *ingestion*, and spans that a retention filter keeps for search, called *indexing*. AI Guard spans aren't billed for ingestion. The AI Guard spans this filter keeps are indexed spans, so they count toward your APM indexed span usage. Organizations that don't use APM, or that already index a high volume of traces, are the most likely to see indexed AI Guard spans on their bill. ## 5. (Optional) Limit access to AI Guard spans{% #limit-access %} Optionally restrict who can view AI Guard spans, because they can contain sensitive prompts, responses, and tool call data from your agents. You must have the User Access Manage permission to complete this task. 1. In Organization Settings > Data Access Control, create a restricted dataset scoped to APM data. 1. Apply the filter `resource_name:ai_guard`. 1. Grant access to the dataset to specific roles or teams. ## Verify the setup{% #verify %} Confirm that AI Guard is evaluating agent traffic. 1. Send several requests through the agent. 1. In AI Guard, verify the setup: - On the Discover tab, find the agent service. After its evaluations arrive, the service appears under Protected by AI Guard. - On the Investigate tab, confirm that the agent's evaluations appear. ## Troubleshooting: AI Guard spans don't appear{% #troubleshooting-ai-guard-spans-dont-appear %} If AI Guard spans for the agent don't appear, check these causes in order: 1. **AI Guard isn't enabled.** Set `DD_AI_GUARD_ENABLED=true`. The variable defaults to `false`, and when it's unset: - Automatic integrations don't evaluate anything. - SDK calls in Node.js, Java, and Ruby return `ALLOW` without calling AI Guard. - Manual integrations and Python SDK calls still evaluate, but AI Guard spans don't include the evaluated messages. 1. **The application key is missing the scope.** If evaluation calls fail with a 401 or 403 error, confirm the application key has the `ai_guard_evaluate` scope. 1. **The Agent isn't reachable.** Confirm the Datadog Agent is running and that your application can connect to it. 1. **The retention filter is missing or out of order.** Confirm a retention filter matches `resource_name:ai_guard` at 100% span and trace rates, and that it sits above any other filter that matches AI Guard spans. See the Create a retention filter for AI Guard spans section. 1. **The service or environment isn't set.** Confirm `DD_SERVICE` and `DD_ENV` are set, so AI Guard can match the agent to its policy and show it in Discover. To confirm the Datadog SDK reaches AI Guard, check the `datadog.ai_guard.evaluations` metric for your service. If the metric is above zero but spans don't appear, the cause is span retention, not AI Guard.