Network Performance Monitoring is now generally available! Network Monitoring is now available!
home Home docs Docs api API

Webhooks

Crawler Crawler

Overview

Webhooks enable you to:

  • Connect to your services.
  • Alert your services when a metric alert is triggered.

Setup

Configuration

Go to the Webhooks integration tile and enter the URL and name of the webhook you want to use.

Usage

To use your webhook, add @webhook-*name_of_the_webhook* in the text of the metric alert you want to trigger the webhook. It triggers a POST request to the URL you set with the following content in JSON format. The timeout for any individual request is 15 seconds.

You can also specify your own payload in order to add your own custom fields to the request. Check the Use Custom Payload checkbox and specify your own custom payload, using the following variables. If you want your payload to be URL-encoded, check the Encode as form payload and specify your payload in JSON format.

VariableMeaning
$AGGREG_KEYID to aggregate events belonging together (e.g.: 9bd4ac313a4d1e8fae2482df7b77628).
$ALERT_CYCLE_KEYID to link events from the time an alert triggers until it resolves.
$ALERT_IDID of alert (e.g.: 1234).
$ALERT_METRICName of the metric if it’s an alert (e.g.: system.load.1).
$ALERT_QUERYQuery of the monitor that triggered the webhook.
$ALERT_SCOPEComma-separated list of tags triggering the alert (e.g.: availability-zone:us-east-1a, role:computing-node).
$ALERT_STATUSSummary of the alert status (e.g.: system.load.1 over host:my-host was > 0 at least once during the last 1m).
$ALERT_TITLETitle of the alert.
$ALERT_TRANSITIONType of alert notification (values: Recovered, Triggered/Re-Triggered, No Data/Re-No Data, Warn/Re-Warn, Null, Renotify).
$ALERT_TYPEType of the alert.
$DATEDate (epoch) where the event happened (e.g.: 1406662672000).
$EMAILEmail of the user posting the event that triggered the webhook.
$EVENT_MSGText of the event (e.g.: @webhook-url Sending to the webhook).
$EVENT_TITLETitle of the event (e.g.: [Triggered] [Memory Alert]).
$EVENT_TYPEType of the event *(values: metric_alert_monitor, event_alert, or `service_check.
$HOSTNAMEThe hostname of the server associated with the event, if there is one.
$IDID of the event (e.g.: 1234567).
$LAST_UPDATEDDate when the event was last updated.
$LINKURL of the event (e.g.: https://app.datadoghq.com/event/jump_to?event_id=123456).
$LOGS_SAMPLELogs sample from log monitor alerts.
$METRIC_NAMESPACENamespace of the metric if it’s an alert.
$ORG_IDID of your organization (e.g.: 11023).
$ORG_NAMEName of your organization (e.g.: Datadog).
$PRIORITYPriority of the event (values: normal or low).
$SNAPSHOTURL of the image if the event contains a snapshot (e.g.: https://url.to.snpashot.com/).
$TAGSComma-separated list of the event tags (e.g.: monitor, name:myService, role:computing-node).
$TEXT_ONLY_MSGText of the event without Markdown formatting.
$USERUser posting the event that triggered the webhook (e.g.: rudy).
$USERNAMEUsername of the user posting the event that triggered the webhook.

If you want to post your webhooks to a service requiring authentication, you can use basic HTTP authentication by modifing your URL from https://my.service.com to https://username:password@my.service.com.

Multiple webhooks

In a monitor alert, if 2 or more webhook endpoints are notified, then a webhook queue is created on a per service level. For instance, if you reach out to PagerDuty and Slack, a retry on the Slack webhook does not affect the PagerDuty one.

However, in the PagerDuty scope, certain events always go before others—specifically, an “Acknowledge” payload always goes before “Resolution”. If an “Acknowledge” ping fails, the “Resolution” ping is queued due to the retry logic.

How does the queue behave if a webhook fails?

A service X webhook failure does not affect the webhook of service Y.

What causes a webhook to fail (5XX response or internal error)?

Datadog only issues a retry if there is an internal error (badly formed notification message), or if Datadog receives a 5XX response from the webhook endpoint. Missed connections are retried 5 times.

Examples

Sending SMS through Twilio

Use as URL: https://{Your-Account-id}:{Your-Auth-Token}@api.twilio.com/2010-04-01/Accounts/{Your-Account-id}/Messages.json

and as a payload:

    {
        "To":"+1347XXXXXXX",
        "From":"+1347XXXXXX",
        "Body":"$EVENT_TITLE \n Related Graph: $SNAPSHOT"
    }

Replace To with your phone number and From with the one Twilio attributed to you. Check the Encode as form checkbox.

Creating an issue in Jira

Use as URL: https://{Your-Jira-Username}:{Your-Jira-Password}@{Your-Domain}.atlassian.net/rest/api/2/issue

and as a payload:

{
    "fields": {
        "project": {
            "key": "YOUR-PROJECT-KEY"
            },
        "issuetype": {
            "name": "Task"
        },
        "description": "There's an issue. See the graph: $SNAPSHOT and event: $LINK",
        "summary": "$EVENT_TITLE"
    }
}

Do not check the “Encode as form” checkbox.

Table of Contents