CI Visibility is not available in the selected site () at this time.
Quality Gates are in private beta. To request access, complete the form.Request Access
Quality Gates allows you to gate your workflows based on signals in Datadog. You can create two types of rules: Static Analysis and Tests.
For example, you can block code from being merged if it introduces new flaky tests or code security violations; issues that wouldn’t normally fail your CI/CD pipelines and end up deployed to production.
Using Quality Gates, you have control over what is merged into the default branch, and ultimately on what is deployed.
This allows you to ensure that the code running in production is meeting high quality standards, reducing incidents, and minimizing unwanted behaviors.
Set up Quality Gates
There are two main steps required to set up and use Quality Gates:
- Create one or more rules on the Datadog site.
- Invoke Quality Gates by using the
datadog-ci gate evaluate command in your CI pipeline.
Create a rule
To create Quality Gates rules for your organization, your user account must have the
- In Datadog, navigate to CI > Quality Gate Rules and click + New Rule.
- Define the rule scope. The rule scope defines when the rule should be evaluated. For example, you can create a rule that it is evaluated only on specific repositories and branches. To define the scope for a rule, switch to
select when to evaluate and add included or excluded values for the scope. See rule scopes for more information.
- Select the rule type. You can choose between
Static Analysis and
- Define the rule condition. The rule condition states in which scenario the rule fails, failing the related pipeline (if the rule is blocking). You can select one of the existing rule conditions for the rule type you have selected.
The following example shows how to create a static analysis rule that fails when there are one or more static analysis violations with “error” status and “security” category being introduced in a specific commit:
- Select whether the rule should block the pipeline or not when it fails.
Non-blocking rules are helpful when rolling out a new rule and wanting to verify its behavior before making it blocking.
- Select the time window over which the query is evaluated.
- Select a rule name that describes the rule that you are creating.
- Click Save Rule.
Invoking quality gates
To use quality gates, your
datadog-ci version should be
2.19.0 or later.
You can invoke the Quality Gates evaluation by calling the
datadog-ci gate evaluate command.
- Retrieves all the rules that have rule scopes and custom scopes matching the current pipeline context (repository, branch, or custom scope passed in the command).
- Evaluates all the matching rules.
- If one or more blocking rules fail, the command fails as well, blocking the pipeline.
For the command to work properly, it's important that the events (tests, static analysis violations)
are sent to Datadog before the
datadog-ci gate evaluate command is executed.
Otherwise, the rules might have an incorrect behavior due to the absence of the events.
The command requires the
DD_APP_KEY environment variables to point to your Datadog API Key
and Datadog Application Key. Also, you need to set the
true. Optionally, you can specify the
DD_SITE environment variable to point to a specific datadog site (default value is
DD_BETA_COMMANDS_ENABLED=true DD_SITE= DD_API_KEY=<API_KEY> DD_APP_KEY=<APP_KEY> datadog-ci gate evaluate
The behavior of the command can be modified using the following flags:
--fail-on-empty: when this flag is specified, the command fails if no matching rules were found in Datadog
based on the current pipeline context. By default, the command succeeds.
--fail-if-unavailable: when this flag is specified, the command fails if one or more rules could not be evaluated because of an internal issue.
By default, the command succeeds.
--timeout: the command stops its execution after the specified timeout in seconds. The default timeout is 10 minutes.
The command typically completes within a few minutes, but it could take longer due to processing delays in Datadog.
--no-wait: by default, the command waits a certain amount of time for the events (tests, static analysis violations) to arrive to Datadog.
This step is important as it makes sure that the events are queryable in Datadog before the rules are executed,
avoiding incorrect evaluations. If, in your pipeline, the job containing the
datadog-ci gate evaluate command is
called several minutes after the related events are sent to Datadog, you could skip this waiting time by specifying the
If used incorrectly, this flag may result in inaccurate rule evaluations.
Custom scopes can be added by using the following option one or more times:
datadog-ci gate evaluate --scope team:backend --scope team:frontend
The command logs can be examined to understand what was the overall gate evaluation status, along with information
about all the rules that were evaluated.
When creating a rule, you can define a scope, which states when the rule should be evaluated. If the rule scope is
always evaluate, the rule is evaluated on all repositories and branches.
datadog-ci gate evaluate command is invoked, the rules having a scope matching the command context are evaluated.
For each scope (for example,
branch), you can either select included or excluded values.
When included values are selected, the rule is evaluated if one or more included values are part of the command context.
When excluded values are selected, the rule is not evaluated if any of the excluded values are part of the command context.
For example, to create a rule that is evaluated in all branches except
main of the
example-repository repository, you can define the following scope:
If a rule does not contain a scope, it is evaluated for all values for that scope.
For example, if a rule does not contain the
repository scope, it is evaluated for all repositories.
In addition to branch and repository, you can define custom scopes to filter rules that are evaluated for a specific CI pipeline.
To add a custom scope when creating a rule:
- Click + Add Filter and select Custom Scope.
- Define the scope name, for example,
- Define the scope’s included or excluded values.
repository scopes, custom scopes need to be provided in the
datadog-ci gate evaluate command using the
For example, you can create a rule that is evaluated for the
example-repository repository, but only when the team is
The rule would be evaluated when this command is invoked in a CI pipeline of the
datadog-ci gate evaluate --scope team:backend
The rule would not be evaluated when the following commands are invoked instead:
datadog-ci gate evaluate, as it does not define any team.
datadog-ci gate evaluate --scope team:api --scope team:frontend, as it defines teams different from
To edit a Quality Gates rule, click the Edit icon to the right of the Creator avatar on the Quality Gates Rules page.
To delete a Quality Gates rule, click the Delete icon next to the Edit button on the Quality Gates Rules page. Alternatively, you can edit the rule and click Delete Rule.
Enable GitHub check creation
You can automatically create a GitHub check for each rule evaluated. When this feature is enabled, the evaluation results appear directly in GitHub.
The check contains additional information about the rule evaluation, such as the failure reason and the matching events in Datadog.
To enable GitHub Checks:
- Go to the GitHub integration tile. If you do not have this integration set up, or you don’t have a GitHub app within the integration, follow the GitHub integration documentation to set one up.
Checks: Write access to the GitHub application.
After the permission is granted, you should be able to see the checks in GitHub.
Note: a current limitation is that you cannot re-run a check to re-run the related Quality Gates rule.
Only users with the
quality_gate_rules_write permission can create and edit Quality Gate rules. Users with the
quality_gate_rules_read permission can view Quality Gate rules.
For more information, see the RBAC Permissions documentation.
Track changes in rules
You can view information about who created, modified, and deleted Quality Gates rules in Audit Trail.
Additional helpful documentation, links, and articles: