- 重要な情報
- アプリ内
- インフラストラクチャー
- アプリケーションパフォーマンス
- 継続的インテグレーション
- ログ管理
- セキュリティ
- UX モニタリング
- 管理
Use Incident Settings to customize aspects of the Incident Management experience for your entire organization. The individual settings are categorized and divided into different sub-sections. The main categories are: General, Notifications, and Remediation.
The General subsection of Incident Settings is used to define your organization’s severity levels and status levels, and to declare incident helper text.
Use severity level settings to:
SEV-0
or SEV-1
(defaults to SEV-1
).Note: If you attempt to delete a severity that is referenced in a notification rule, you are prompted to confirm your decision. Choosing to proceed disables the impacted notification rules as they are no longer valid. Deleting a severity or changing the starting severity does not automatically update any Incident Management Analytics queries.
Use status level settings to:
Completed
status.Note: Deleting the Completed
status does not automatically update any incidents that are already in the Completed
status, nor does it automatically update any Incident Management Analytics queries that explicitly reference it. Any notification rule that references the Completed
status is disabled, as that rule is no longer valid.
For the Declare Incident Helper Text settings, you can customize the helper text that appears alongside the severity and status level descriptions in the Incident Creation Modal. The helper text has Markdown support, which allows indented lists, text formatting, and hyperlinks to other instruction resources for incident responders.
Property fields are key pieces of metadata you can tag your incidents with. This makes it easier to search for specific subsets of incidents on the Homepage and make more robust queries in Incident Management Analytics. There are five default property fields:
Root Cause
Services
Teams
Detection Method
Summary
If you have Datadog APM configured, the Services
property field automatically leverages your APM Service names. To edit the values of Services
or Teams
, upload a CSV of the values you wish to associate with each field. Your CSV file must start with your field’s name in the top row, with the desired values listed immediately below it.
You can add more property fields to your settings by selecting one of your existing key:value
pair metric tags. When you do this, the key of your property field is the start case of your metric tag’s key (each word is capitalized and separated by spaces), and the values for the property field are equal to the values reported by the metric tag.
Property fields are organized into three tables that correspond to where the fields will appear in the Overview section of the Incident Details page:
What Happened
Why It Happened
Attributes
You can move any property field into a different table or reorder them in the same table by dragging and dropping the field using the drag handle icon. Preview what your property fields look like by clicking the Preview button on the top right.
In addition to the five default fields and the fields based on metric tags, you can also create custom property fields and mark them as required at the creation of an incident. There are four kinds of custom fields you can create:
Single-Select, Multi-Select, and Number custom fields are searchable facets in the Incident Homepage and Incident Management Analytics for easy filtering of incidents. Number fields are measures in Incident Management Analytics that can be graphed and visualized in Dashboards and Notebooks.
The responder types settings provide you with the ability to create custom roles to assign to your incident responders and to specify if those roles are meant to be held by one person or multiple people per incident. These roles are unrelated to the Role Based Access Control (RBAC) system. Responder types allow your responders to understand what their responsibilities are in an incident based on the definitions of your own incident response process. By default there are two roles:
Incident Commander
- The individual responsible for leading the response teamResponder
- An individual that actively contributes to investigating an incident and resolving its underlying issueNote: The Incident Commander
responder type appears in Incident Settings so that you may customize its description. Incident Commander
cannot be deleted as a responder type, nor can its name or status as a One person role
be changed. The Responder
role is a generic fallback role if a responder is not otherwise assigned a different role, and does not appear in Incident Settings.
To create a custom responder type:
One person role
or a Multi person role
. A One person role
can be held by a single person per incident, while a Multi person role
can be held by an unlimited number of people per incident.The integrations settings provide you with additional configurations for setting up the Incident Management features of the Datadog Slack App. There are two settings to configure:
You can configure either of these settings to use any Slack workspace you have configured in your organization’s Slack integration tile.
By default, dedicated incident channels use incident-{public_id}
as their name template.
The incident
prefix can be changed to any string composed of lowercase letters, numbers, and dashes. Datadog recommends you keep your prefix short as Slack enforces an 80 character limit in channel names. Aside from {public_id}
, you can also add {date_created}
and {title}
as variables in the channel name template.
Notes:
{public_id}
, there is a chance two incidents will have duplicate channel names. In this case, the Datadog Slack App automatically appends a random lowercase letter or number to the end of your channel name to prevent the channel creation process from failing.{title}
, the Datadog Slack App automatically renames the channel if an incident’s title changes.The incident updates channel sends a message whenever an incident is declared or changes status, severity, or incident commander.
Message templates are dynamic, reusable messages that can be used in manual incident notifications, or automated notification rules. Message templates leverage template variables, such as {{incident.severity}}
, to dynamically inject the corresponding value from the incident that the notification is being sent for. Message templates have Markdown support so that incident notifications can include text formatting, tables, indented lists, and hyperlinks. To better organize a large number of message templates, each template requires a category during the creation process.
To create a message template:
Note: Template variables are supported in both the message’s title and body.
Notification rules allow you to configure scenarios when specific stakeholders should be automatically notified of an incident. You can use notification rules to ensure key stakeholders are always made aware of high priority incidents, to notify external systems whenever a new incident is declared or updated, or to notify specific responders when a particular service or team experiences an incident.
Example: Set a notification rule to automatically notify team stakeholders whenever a SEV-1 or SEV-2 for service:web-store
AND application:purchasing
incident is declared and when that incident moves through different states of progression.
To configure a new notification rule:
key:value
pairs you want notifications to be sent for. By default, these filters are empty, and a notification rule triggers for any incident.You can perform the following operations to manage your notification rules.
Postmortem templates are dynamic, reusable templates used to create a Datadog Notebook that is automatically populated with incident information after an incident has been resolved. Postmortem templates leverage template variables, such as {{incident.severity}}
, to dynamically inject the corresponding value from the incident that the postmortem is being created for. Postmortem templates have Markdown support so that the resulting notebook includes text formatting, tables, indented lists, and hyperlinks.
To create a postmortem template: