Get Started with AI Guard
AI Guard isn't available in the site.
AI Guard helps secure your AI apps and agents in real time against prompt injection, jailbreaking, tool misuse, and sensitive data exfiltration attacks. AI Guard can also detect sensitive data such as PII and secrets in LLM conversations. This page describes how to set it up so you can keep your data secure against these AI-based threats.
For an overview on AI Guard, see AI Guard.
Setup
To set up AI Guard, you need to create API keys, install an SDK, configure retention filters, and set AI Guard policies including blocking, evaluation sensitivity, and sensitive data scanning.
For full setup instructions, see Set Up AI Guard.
View AI Guard data in Datadog
After completing the setup steps and using an SDK to instrument your code, you can view your data in Datadog on the AI Guard page.
You can't see data in Datadog for evaluations performed directly using the REST API.
Security signals
AI Guard generates security signals when it detects threats such as prompt injection, jailbreaking, or tool misuse. You can create custom detection rules, set thresholds for notifications, and investigate signals alongside other application security threats.
For more information, see AI Guard Security Signals.
Set up Datadog Monitors for alerting
To create monitors for alerting at certain thresholds, you can use Datadog Monitors. You can monitor AI Guard evaluations with either APM traces or with metrics. For both types of monitor, you should set your alert conditions, name for the alert, and define notifications; Datadog recommends using Slack.
APM monitor
Follow the instructions to create a new APM monitor, with its scope set to Trace Analytics.
- To monitor evaluation traffic, use the query
@ai_guard.action: (DENY OR ABORT). - To monitor blocked traffic, use the query
@ai_guard.blocked:true.
Metric monitor
Follow the instructions to create a new metric monitor.
- To monitor evaluation traffic, use the metric
datadog.ai_guard.evaluations with the tags action:deny OR action:abort. - To monitor blocked traffic, use the metric
datadog.ai_guard.evaluations with the tag blocking_enabled:true.
Evaluate conversations in AI Guard Playground
The AI Guard Playground lets you test AI Guard evaluations directly from the Datadog UI, without writing any code. Submit a conversation, including user input, assistant output, and tool calls, and see the evaluation result (action and reason) in real time.
Use the Playground to:
- Experiment with different prompt patterns and see how AI Guard responds.
- Verify that AI Guard correctly detects prompt injection, jailbreaking, or unsafe tool calls.
- Tweak the evaluation sensitivity threshold and see how it affects detection results. You can then adjust the threshold in AI Guard’s evaluation sensitivity settings.
- Test sensitive data scanning on your conversations.
- Share evaluation results with your team during development.
Further reading
Additional helpful documentation, links, and articles:
