Datadog provides default rules so you can get started quickly with Scorecards, but you can also create custom rules.

Create custom rules

To add and evaluate custom rules using the Scorecards API:

1. Specify the name of the rule, the scorecard it belongs to, a rule description, and an owner to pass to /scorecard/rules.
2. Send an outcome of pass, fail, or skip for each {rule, service} tuple that you are evaluating to /scorecard/outcomes/batch.
3. View an overview of outcomes and remarks in the Scorecards dashboard.

After initial setup, rules can also be enabled or disabled through the API.

To evaluate and add custom rules in the Scorecards UI:

1. Click Create Rule on the Scorecards page.
2. Specify the name of the rule, the scorecard it belongs to, a rule description, and the owning team.
3. Navigate to the rule you created and select Edit Outcome next to the service that you want to evaluate.
4. Select the relevant outcome of pass, fail, or skip and add an optional remark describing the reason for the outcome.
5. View an overview of outcomes and remarks in the Scorecards dashboard.

Evaluate custom rules using Workflow Automation

Workflow Automation allows you to automate the evaluation of your custom rules in Datadog using the Update scorecard rule outcome action. To set up a custom rule evaluation, create a Workflow from scratch or use one of the Scorecards blueprints.

To set up a custom rule evaluation using Workflow Automation:

1. Create a custom rule in Scorecards.
2. Create a Workflow.
3. Set a schedule for your Workflow to run on.
4. Click plus (+) icon to add a step.
5. Use the List service definitions action to fetch all defined services from Service Catalog.
6. Insert a For loop to iterate over each service one-by-one.
7. Select the action needed to fetch your evaluation data.
8. Transform the returned data using a custom JavaScript function to generate pass/fail outcomes for each service.
9. Use the Update scorecard rule outcome action to send results to Scorecards.
10. Run the Workflow and see your evaluations populate in Scorecards for your custom rule.

Further reading

Additional helpful documentation, links, and articles:

Service CatalogDOCUMENTATION Scorecards APIDOCUMENTATION Prioritize and promote service observability best practices with ScorecardsBLOG Formalize best practices with custom ScorecardsBLOG Track DORA Metrics with DatadogDOCUMENTATION