--- title: AI Impact description: >- Measure the impact of AI coding assistants on your software delivery performance. breadcrumbs: Docs > Delivery Performance > AI Impact --- > For the complete documentation index, see [llms.txt](https://docs.datadoghq.com/llms.txt). # AI Impact {% callout %} # Important note for users on the following Datadog sites: app.ddog-gov.com, us2.ddog-gov.com {% alert level="danger" %} This product is not supported for your selected [Datadog site](https://docs.datadoghq.com/getting_started/site.md). ({% placeholder "user-datadog-site-name" /%}). {% /alert %} {% /callout %} {% callout %} # Important note for users on the following Datadog sites: app.datadoghq.com, us3.datadoghq.com, us5.datadoghq.com, app.datadoghq.eu, ap1.datadoghq.com, ap2.datadoghq.com, uk1.datadoghq.com {% callout %} ##### Join the Preview! AI Impact is available to all Datadog customers in Preview. {% /callout %} {% /callout %} ## Overview{% #overview %} AI Impact measures how AI coding assistants affect your software delivery performance. Measuring their impact requires configuring an integration with a supported provider. ## Setup{% #setup %} ### Prerequisites{% #prerequisites %} - [DORA Metrics](https://docs.datadoghq.com/delivery_performance/dora_metrics/setup.md) set up with deployment, commit, and pull request data. - An integration configured with a supported AI coding tool provider. ### Supported tools{% #supported-tools %} | Tool | Direct attribution | Inferred from user activity | | ----------------------------------------------------------------------------------------------------------------------------- | ------------------ | --------------------------- | | [Cursor](https://docs.datadoghq.com/integrations/cursor.md?tab=cursorintegrationindatadog#overview) | yes | yes | | [Claude Code Platform (API)](https://docs.datadoghq.com/integrations/anthropic-usage-and-costs.md) | yes | | [Claude Code Enterprise](https://app.datadoghq.com/integrations?search=claude&integrationId=claude-enterprise-user-analytics) | yes | | [GitHub Copilot](https://docs.datadoghq.com/integrations/github-copilot.md) | yes | | [Codex](https://docs.datadoghq.com/integrations/openai-codex.md) | yes | ### AI attribution{% #ai-attribution %} AI Impact classifies each pull request as AI-assisted or non-AI, and every metric is built on that classification. A PR is AI-assisted when at least one of its commits is AI-assisted. Two attribution modes are available, depending on the signal your tools provide. | Direct attribution | Inferred from user activity | | ---------------------------- | ------------------------------------------------------------------------------------------------------------------------ | | A commit is AI-assisted when | The tool reports AI contribution to that specific commit, for example, Cursor integration reporting AI-generated lines | The commit author created lines of code with the AI tool on the day the commit was created | | Evidence is tied to | The commit | The author and the calendar day | | What the metrics tell you | How AI-assisted code performs compared to code written without AI, and how users of one tool compare to users of another | How developers working with AI perform against those working without it, and how users of one tool compare to users of another | Direct attribution is the more precise of the two, because the signal is attached to the change itself. The inferred from user activity mode covers tools that report usage without per-commit detail, and classifies every commit an active author made that day as AI-assisted. A user is active only on days the tool reports lines of code created by that user. By default, Datadog selects the most precise attribution mode that all integrated tools have in common, so that metrics are comparable across tools on an equal basis. For example, if you're using only Cursor, metrics use direct attribution because that's the most precise method available. If you're using both Cursor and Claude Code, metrics are inferred from user activity because direct attribution is not available for Claude Code. ## Impact metrics{% #impact-metrics %} | Metric | Definition | | ------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | AI-assisted PRs | PRs containing at least one AI-assisted commit, divided by total PRs. | | PR Throughput | Number of PRs deployed per user per day for AI-assisted authors compared to non-assisted authors. | | PR Cycle Time | Median time from a PR's first commit to merge for AI-assisted PRs compared to non-assisted PRs. | | Change Failure Rate | Failure rate weighted by the proportion of AI-assisted commits in each deployment, compared to the weighted rate for non-assisted commits. For example, if a failed deployment has 3 out of 10 commits assisted by AI, only 30% of that failure is attributed to AI. | | Recovery Time | Median recovery time of failed deployments containing AI-assisted commits compared to deployments without. | {% alert level="info" %} Change Failure Rate only includes deployments linked to code changes. Configuration-only or infrastructure deployments are excluded to help the comparison reflect the impact of AI on code-related failures. This differs from standard DORA Change Failure Rate, which counts all deployment types. {% /alert %} {% alert level="info" %} For GitHub only, PR-level metrics exclude PRs whose commits are entirely bot-authored. This keeps automated activity out of the non-AI baseline. {% /alert %} ## Further Reading{% #further-reading %} Additional helpful documentation, links, and articles: - [Learn about DORA Metrics](https://docs.datadoghq.com/delivery_performance/dora_metrics.md) - [Set up DORA Metrics](https://docs.datadoghq.com/delivery_performance/dora_metrics/setup.md) - [Learn how DORA metrics are calculated](https://docs.datadoghq.com/delivery_performance/dora_metrics/calculation.md) - [5 pitfalls to avoid when measuring developer experience in the AI era](https://www.datadoghq.com/blog/devex-measurement-pitfalls-ai-era/) - [Measure the real impact of AI coding tools on software delivery with Datadog AI Impact](https://www.datadoghq.com/blog/ai-impact/) - [How to measure developer experience in the AI era](https://www.datadoghq.com/blog/how-to-measure-developer-experience-in-the-ai-era/)