--- title: Set Up AI Impact description: >- Configure AI detection sources and AI Impact settings so Datadog can classify pull requests as AI-assisted. breadcrumbs: Docs > Delivery Performance > AI Impact > Set Up AI Impact --- > For the complete documentation index, see [llms.txt](https://docs.datadoghq.com/llms.txt). # Set Up AI Impact {% callout %} # Important note for users on the following Datadog sites: app.ddog-gov.com, us2.ddog-gov.com {% alert level="danger" %} This product is not supported for your selected [Datadog site](https://docs.datadoghq.com/getting_started/site.md). ({% placeholder "user-datadog-site-name" /%}). {% /alert %} {% /callout %} {% callout %} # Important note for users on the following Datadog sites: app.datadoghq.com, us3.datadoghq.com, us5.datadoghq.com, app.datadoghq.eu, ap1.datadoghq.com, ap2.datadoghq.com, uk1.datadoghq.com {% callout %} ##### Join the Preview! AI Impact is available to all Datadog customers in Preview. {% /callout %} {% /callout %} [AI Impact](https://app.datadoghq.com/ci/dora/ai-impact) classifies each pull request as AI-assisted or non-AI based on the sources it detects for your AI coding tools. A source is a signal that an AI coding tool contributed to a commit or PR. Configure your AI coding tools and sources at [Software Delivery > Delivery Performance > Settings > AI Impact](https://app.datadoghq.com/ci/settings/dora/integrations). Datadog points you to the recommended source for the most accurate classification for each coding assistant, and preset mapping rules help detect activity from common review agents and autonomous agents. ## Prerequisites{% #prerequisites %} AI Impact requires setting up [DORA Metrics](https://docs.datadoghq.com/delivery_performance/dora_metrics/setup.md) with deployment, commit, and pull request data. No AI-specific configuration is required to see initial data, such as AI co-authored commits and some activity attributed to AI reviews and autonomous agents. For more complete measurement, configure your AI coding tools and sources as described below. ## How AI detection works{% #how-ai-detection-works %} Datadog classifies AI activity from the following types of AI tools: | AI tools | How detection works | | ----------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | Coding assistants | Detected by co-author patterns, configured integrations, pull request labels, or other data that indicate contributions from an AI tool. AI attribution can be either direct or inferred, depending on the tool's source. | | Review agents | Detected using mapping rules for pull request comments and GitHub reactions. A review agent is attributed to a pull request when a rule matches a review author to a named agent. | | Autonomous agents | Detected using mapping rules for author logins, author emails, or commit trailers. An autonomous agent is attributed to a pull request when it creates the first commit in the PR. | ### AI attribution modes for coding assistants{% #ai-attribution-modes-for-coding-assistants %} AI Impact classifies each pull request as AI-assisted or non-AI, and every metric is built on that classification. A PR is AI-assisted when at least one of its commits is AI-assisted. Each source emits signals that provide **direct attribution**, attribution **inferred from user activity**, or both. | Direct attribution | Inferred from user activity | | ---------------------------- | ------------------------------------------------------------------------------------------------------------------------ | | A commit is AI-assisted when | The tool reports AI contribution to that specific commit, for example, Cursor integration reporting AI-generated lines | The commit author created lines of code with the AI tool on the day the commit was created | | Evidence is tied to | The commit | The author and the calendar day | | What the metrics tell you | How AI-assisted code performs compared to code written without AI, and how users of one tool compare to users of another | How developers working with AI perform against those working without it, and how users of one tool compare to users of another | Direct attribution is more precise, because the signal is attached to the change itself. Inferred attribution covers tools that report usage without per-commit detail, and classifies every commit an active author made that day as AI-assisted. A user is active only on days the tool reports lines of code created by that user. By default, the [AI Impact dashboard](https://app.datadoghq.com/ci/dora/ai-impact) displays metrics using the most precise attribution mode that all integrated tools have in common, so that metrics are comparable across tools on an equal basis. For example, if you're using only Cursor, metrics use direct attribution because that's the most precise method available. If you're using both Cursor and Claude Code, metrics are inferred from user activity because direct attribution is not available for Claude Code. To select which attribution mode your metrics use in the [AI Impact dashboard](https://app.datadoghq.com/ci/dora/ai-impact), set the Default AI attribution UI filter in AI Impact settings. The setting affects only how data is displayed in the AI Impact UI, not what Datadog ingests. Both modes continue to collect data regardless of which mode you select. ## Set up coding assistants{% #set-up-coding-assistants %} [AI Impact Settings](https://app.datadoghq.com/ci/settings/dora/integrations) show which coding assistants and sources Datadog detects. Some coding assistants are detected automatically, when they add themselves as a co-author to the commits they generate. To more completely capture an AI coding assistant's activity, configure a source for it. ### Supported AI coding assistants and sources{% #supported-ai-coding-assistants-and-sources %} Each supported coding assistant has one recommended source per context, listed in the following table. The recommended source is the most accurate one that most users can configure without manual work per pull request, and it ingests data automatically after setup. | Coding assistant | Recommended source | Attribution mode | | ------------------------------------------------ | --------------------------------------------------------------------------------------------------------------------------- | ---------------- | | Claude Code (API or Platform) | [Anthropic Usage and Costs integration](https://docs.datadoghq.com/integrations/anthropic-usage-and-costs.md) | Inferred | | Claude Code (Enterprise plan) | [Claude Enterprise User Analytics integration](https://docs.datadoghq.com/integrations/claude-enterprise-user-analytics.md) | Inferred | | Claude Code (Teams plan) | OpenTelemetry | Inferred | | Claude Code (Amazon Bedrock or Google Vertex AI) | OpenTelemetry | Inferred | | Codex | [OpenAI Codex integration](https://docs.datadoghq.com/integrations/openai-codex.md) | Inferred | | Cursor | [Cursor integration](https://docs.datadoghq.com/integrations/cursor.md?tab=cursorintegrationindatadog#overview) | Direct | | GitHub Copilot | [GitHub Copilot integration](https://docs.datadoghq.com/integrations/github-copilot.md) | Inferred | To track other coding assistants or use a different source for a supported coding assistant, the following sources are available for all coding assistants: - PR labels: (Direct attribution) PRs with a designated label are marked as AI-assisted. - Public API: (Inferred from user activity) Send your own event data through the [Datadog Public API](https://docs.datadoghq.com/api/latest/dora-metrics/send-ai-tool-user-activity.md), and Datadog infers AI attribution from it. ### Configure a source for a coding assistant{% #configure-a-source-for-a-coding-assistant %} 1. In [AI Impact Settings](https://app.datadoghq.com/ci/settings/dora/integrations), open the Coding assistants tab. The tab lists the AI tools automatically detected through co-author detection, as well as any other configured assistants. 1. If the coding assistant you want to track is already listed, select it to configure a source. Otherwise, click Add a coding assistant and select the coding assistant. 1. For Claude Code, select the plan you use or the gateway you route through. The recommended source differs for each. 1. Follow the UI guidance to set up the source. Expand the following sections for details on each source type: {% collapsible-section #datadog-integrations %} #### Datadog integrations In many cases, the recommended source is a Datadog integration for the coding tool, which reports AI usage directly to Datadog. Of these, only the Cursor integration reports AI contribution directly per commit. The other integrations report user activity, which Datadog uses to infer AI assistance for the commits an active author created that day. When configuring a coding assistant in AI Impact Settings, click the relevant button in the Integration section to open the integration tile in Datadog and complete the setup. {% /collapsible-section %} {% collapsible-section #opentelemetry %} #### OpenTelemetry Use OpenTelemetry to report AI usage data from certain coding assistant deployments, such as Claude Code's Teams plan. Additional configuration is required for Claude Code running through an external gateway such as Amazon Bedrock or Google Vertex AI. 1. Configure Claude Code to send OpenTelemetry data to Datadog, following either [Option 2: OpenTelemetry (OTLP)](https://docs.datadoghq.com/ai_agents_console/setup.md#option-2-opentelemetry-otlp) or [Option 3: Forward data through the Datadog Agent](https://docs.datadoghq.com/ai_agents_console/setup.md#option-3-forward-data-through-the-datadog-agent). 1. Amazon Bedrock or Google Vertex AI: Set a per-user resource attribute in your Claude Code settings file, for example `~/.claude/settings.json`: ```json "OTEL_RESOURCE_ATTRIBUTES": "user.email=user@company.com" ``` AI Impact needs `user.email` to tie activity back to an individual developer and infer AI assistance for that developer's commits. 1. Restart Claude Code. Because `user.email` differs for every developer, it cannot ship in a single shared configuration file. Distribute the settings file with a Mobile Device Management (MDM) system or a provisioning script that writes the value at login. {% /collapsible-section %} {% collapsible-section #pr-label-detection %} #### PR label detection Pull request labels give direct attribution for any coding assistant, including assistants with no Datadog integration. Datadog marks every commit in a labeled pull request as AI-assisted. 1. When configuring a coding assistant in AI Impact Settings, click Set Up in the PR Labels section. 1. Enter one or more labels that your team uses to identify AI-assisted pull requests, and click Save. Each rule applies to the whole pull request, so if any commits were written without AI assistance, they are classified as being AI-assisted. Label detection depends on consistent labeling by your team, which can be difficult to sustain across a large organization. Use it for a coding assistant with no Datadog integration, or for a single team that can maintain the practice. {% /collapsible-section %} {% collapsible-section #datadog-public-api %} #### Datadog public API Use the [Datadog public API](https://docs.datadoghq.com/api/latest/dora-metrics/send-ai-tool-user-activity.md) to send your own AI usage data to Datadog when your organization routes AI coding tools through an internal gateway. The API is also an option for tracking a coding assistant that has no Datadog integration. Datadog infers AI assistance from the user activity you report. {% /collapsible-section %} {% collapsible-section #co-author-signatures %} #### Co-author signatures Some coding assistants add themselves as a co-author to the commits they generate. Datadog detects these co-author signatures automatically—no additional setup is required. Because the signature is attached to the commit, co-author detection provides direct attribution. Co-author detection can undercount AI activity, as not all coding assistants add themselves as co-authors, and a rebase or squash can strip the co-author signature. To measure a coding assistant more fully, configure another source for it. {% /collapsible-section %} ## Set up review agents{% #set-up-review-agents %} Review agents review pull requests rather than writing the code in them. Datadog detects review agents from pull request comments and GitHub reactions, and attributes a review agent to a pull request when a mapping rule matches a review author to a named agent. To track a review agent with a mapping rule: 1. Open the Review agents tab in AI Impact settings. 1. Click Add Review Agent. 1. Enter an Agent name to identify the review agent in the UI. 1. Define one or more rules that match the agent's author login associated with review comments or reactions on PRs. ## Set up autonomous agents{% #set-up-autonomous-agents %} Autonomous agents open pull requests on their own rather than assisting a developer in an editor. Datadog detects most autonomous agents automatically from known agent patterns. You can also define mapping rules to identify agents that Datadog does not automatically detect. To track an agent with a mapping rule: 1. Open the Autonomous agents tab in AI Impact settings. 1. Click Add Autonomous Agent. 1. Enter an Agent name to identify the autonomous agent in the UI. 1. Define one or more rules that identify the agent's pull requests. ### Identify automation bots{% #identify-automation-bots %} Automation bots, such as dependency updaters and release bots, are not AI agents, and their activity is not relevant to AI Impact. Identifying a bot prevents its activity from affecting your AI Impact metrics. To detect automation bot-authored commits and PRs, enable automatic bot detection (for GitHub) and define custom mapping rules in [Delivery Performance general settings](https://app.datadoghq.com/ci/settings/dora). ## Further Reading{% #further-reading %} Additional helpful documentation, links, and articles: - [Learn about AI Impact](https://docs.datadoghq.com/delivery_performance/ai_impact.md) - [Set up DORA Metrics](https://docs.datadoghq.com/delivery_performance/dora_metrics/setup.md) - [Set up Agent Console for coding agents](https://docs.datadoghq.com/ai_agents_console/setup.md)