Webhooks

Webhooks

Crawler Crawler

Overview

Webhooks enable you to:

  • Connect to your services.
  • Alert your services when a metric alert is triggered.

Setup

Go to the Webhooks integration tile and enter the URL and name of the webhook you want to use.

Usage

To use your webhook, add @webhook-<WEBHOOK_NAME> in the text of the metric alert you want to trigger the webhook. It triggers a POST request to the URL you set with the following content in JSON format. The timeout for any individual request is 15 seconds. Datadog only issues a retry if there is an internal error (badly formed notification message), or if Datadog receives a 5XX response from the webhook endpoint. Missed connections are retried 5 times.

Note: Custom headers must be in JSON format.

To add your own custom fields to the request, you can also specify your own payload in the Payload field. If you want your payload to be URL-encoded, check the Encode as form checkbox and specify your payload in JSON format. Use the following variables:

$AGGREG_KEY
ID to aggregate events belonging together.
Example: 9bd4ac313a4d1e8fae2482df7b77628
$ALERT_CYCLE_KEY
ID to link events from the time an alert triggers until it resolves.
$ALERT_ID
ID of alert.
Example: 1234
$ALERT_METRIC
Name of the metric if it’s an alert.
Example: system.load.1
$ALERT_PRIORITY
Priority of the alerting monitor.
Example: P1, P2
$ALERT_QUERY
Query of the monitor that triggered the webhook.
$ALERT_SCOPE
Comma-separated list of tags triggering the alert.
Example: availability-zone:us-east-1a, role:computing-node
$ALERT_STATUS
Summary of the alert status.
Example: system.load.1 over host:my-host was > 0 at least once during the last 1m
$ALERT_TITLE
Title of the alert.
$ALERT_TRANSITION
Type of alert notification.
Example: Recovered, Triggered/Re-Triggered, No Data/Re-No Data, Warn/Re-Warn, Renotify
$ALERT_TYPE
Type of the alert.
$DATE
Date (epoch) where the event happened.
Example: 1406662672000
$EMAIL
Email of the user posting the event that triggered the webhook.
$EVENT_MSG
Text of the event.
Example: @webhook-url Sending to the webhook
$EVENT_TITLE
Title of the event.
Example: [Triggered] [Memory Alert]
$EVENT_TYPE
Type of the event.
Example: metric_alert_monitor, event_alert, or service_check.
$HOSTNAME
The hostname of the server associated with the event, if there is one.
$ID
ID of the event.
Example: 1234567
$INCIDENT_ATTACHMENTS
List of JSON objects with the incident’s attachments, such as postmortem and documents.
Example: [{"attachment_type": "postmortem", "attachment": {"url": "https://app.datadoghq.com/notebook/123","title": "Postmortem IR-1"}}]
$INCIDENT_COMMANDER
JSON object with the incident commander’s handle, uuid, name, email, and icon.
$INCIDENT_CUSTOMER_IMPACT
JSON object with an incident’s customer impact status, duration, and scope.
Example: {"customer_impacted": true, "customer_impact_duration": 300 ,"customer_impact_scope": "scope here"}
$INCIDENT_FIELDS
JSON object mapping each of an incident’s fields to its values.
Example: {"state": "active", "datacenter": ["eu1", "us1"]}
$INCIDENT_PUBLIC_ID
Public ID of the associated incident.
Example: 123
$INCIDENT_TITLE
Title of the incident.
$INCIDENT_URL
URL of the incident.
Example: https://app.datadoghq.com/incidents/1
$INCIDENT_MSG
The message of the incident notification.
$LAST_UPDATED
Date when the event was last updated.
$LINK
URL of the event.
Example: https://app.datadoghq.com/event/jump_to?event_id=123456
$LOGS_SAMPLE
Logs sample from log monitor alerts.
$METRIC_NAMESPACE
Namespace of the metric if it’s an alert.
$ORG_ID
ID of your organization.
Example: 11023
$ORG_NAME
Name of your organization.
Example: Datadog
$PRIORITY
Priority of the event.
Example: normal or low
$SECURITY_RULE_NAME
The name of the security rule.
$SECURITY_SIGNAL_ID
The unique identifier of the signal.
Example: AAAAA-AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
$SECURITY_SIGNAL_SEVERITY
The severity of the security signal.
Example: medium
$SECURITY_SIGNAL_TITLE
The title of the security signal.
$SECURITY_SIGNAL_MSG
The message of the security signal.
$SECURITY_SIGNAL_ATTRIBUTES
The security signal attributes.
Example: {"network":{"client":{"ip":"1.2.3.4"}}}
$SECURITY_RULE_ID
The security rule ID.
Example: aaa-aaa-aaa
$SECURITY_RULE_QUERY
The query or queries associated with the security rule.
Example: ["@evt.name:authentication"]
$SECURITY_RULE_GROUP_BY_FIELDS
The security group by key value pairs.
Example: {"@usr.name":"john.doe@your_domain.com"}
$SECURITY_RULE_TYPE
The security rule type.
Example: log_detection
$SNAPSHOT
URL of the image if the event contains a snapshot.
Example: https://p.datadoghq.com/path-to-snapshot
$SYNTHETICS_TEST_NAME
Name of the Synthetics test.
$SYNTHETICS_FIRST_FAILING_STEP_NAME
Name of the first failing step of the Synthetics test.
$TAGS
Comma-separated list of the event tags.
Example: monitor, name:myService, role:computing-node
$TEXT_ONLY_MSG
Text of the event without Markdown formatting.
$USER
User posting the event that triggered the webhook.
Example: rudy
$USERNAME
Username of the user posting the event that triggered the webhook.

Authentication

If you want to post your webhooks to a service requiring authentication, you can use basic HTTP authentication by modifing your URL from https://my.service.example.com to https://<USERNAME>:<PASSWORD>@my.service.example.com.

Multiple webhooks

In a monitor alert, if 2 or more webhook endpoints are notified, then a webhook queue is created on a per service level. For instance, if you reach out to PagerDuty and Slack, a retry on the Slack webhook does not affect the PagerDuty one.

However, in the PagerDuty scope, certain events always go before others—specifically, an “Acknowledge” payload always goes before “Resolution”. If an “Acknowledge” ping fails, the “Resolution” ping is queued due to the retry logic.

Examples

Sending SMS through Twilio

Use as URL: https://<ACCOUNT_ID>:<AUTH_TOKENT>@api.twilio.com/2010-04-01/Accounts/<ACCOUNT_ID>/Messages.json

and as a payload:

{
    "To": "+1347XXXXXXX",
    "From": "+1347XXXXXX",
    "Body": "$EVENT_TITLE \n Related Graph: $SNAPSHOT"
}

Replace To with your phone number and From with the one Twilio attributed to you. Check the Encode as form checkbox.

Creating an issue in Jira

Use as URL: https://<JIRA_USER_NAME>:<JIRA_PASSWORD>@<YOUR_DOMAIN>.atlassian.net/rest/api/2/issue

and as a payload:

{
    "fields": {
        "project": {
            "key": "YOUR-PROJECT-KEY"
        },
        "issuetype": {
            "name": "Task"
        },
        "description": "There's an issue. See the graph: $SNAPSHOT and event: $LINK",
        "summary": "$EVENT_TITLE"
    }
}

Do not check the “Encode as form” checkbox.