Logging is here!

Microsoft Azure

Crawler Crawler

Overview

Connect to Microsoft Azure in order to:

  • Get metrics from Azure VMs with or without installing the Agent
  • Tag your Azure VMs with Azure-specific information (e.g. location)
  • Get metrics for other services: Application Gateway, App Service (Web & Mobile), Batch Service, Event Hub, IOT Hub, Logic App, Redis Cache, Server Farm (App Service Plan), SQL Database, SQL Elastic Pool, and Virtual Machine Scale Set

Related integrations include:

App Service easy-to-use service for deploying and scaling web, mobile, API and business logic applications
Batch Service managed task scheduler and processor
Event Hub large scale data stream managed service
IOT Hub connect, monitor, and manage billions of IOT assets
Logic App quickly build powerful integration solutions
Redis Cache managed data cache
Storage blob, file, queue, and table storage
SQL Database highly scalable relational database in the cloud
SQL Database Elastic Pool manage the performance of multiple databases
Virtual Machine virtual machine management service
Virtual Machine Scale Set deploy, manage, and autoscale a set of identical VMs

Setup

Installation

You can integrate your Microsoft Azure account with Datadog using the Azure CLI tool or the Azure portal.

Integrating through the Azure CLI

To integrate Datadog with Azure using the Azure CLI, make sure you have Azure CLI installed.

First, login to the Azure account you want to integrate with Datadog

For Azure CLI 2.0

az login

Run the account show command and copy & paste the Tenant ID value into the form on the Azure setup tile under “Tenant Name/ID”

az account show
  • Create an application as a service principal using the format below.
  • Grant the Service Principal the “reader” role for the subscription(s) you would like to monitor.
  • The appID generated from this command must be pasted into the “Client ID” text box in the Azure installation form in Datadog.
  • Add --name {some-name} to use a hand-picked name. Otherwise Azure will generate a unique one. The Name will not be used in any way in the setup process.
  • Add --password {some-password} to use a hand-picked password. Otherwise Azure will generate a unique one. This password must be copied and pasted into the “Client Secret” text box in the Azure installation form in Datadog.
az ad sp create-for-rbac --role reader --scopes /subscriptions/{subscription_id}

For Azure CLI 1.0

azure login

Run the account show command and copy & paste the Tenant ID value into the form on the Azure setup tile under “Tenant Name/ID”

azure account show
  • Create an application as a service principal using the format below. The name is NOT used in any way and is simply required as part of the setup process.
  • The password you choose must be copied and pasted into the form on the Azure setup tile under “Client Secret”.
  • You must also pass the “Client ID” of the application into Datadog. The “Client ID” is the unique ID generated from this command, shown under Service Principal Name. It is not the Object ID.
azure ad sp create -n {name} -p {password}
  • Grant the Service Principal the “Reader” role for the subscription you are interested in monitoring.
  • Use the Object Id returned from the previous command to fill in {object-Id}. {subscription-Id} is the azure subscription you would like to monitor, and is listed as ID in azure account show or through the portal
azure role assignment create --objectId {object-Id} -o Reader -c /subscriptions/{subscription-Id}/

For Azure CLI < 1.0

azure login

Run the account show command and copy & paste the Tenant ID value into the form on the Azure setup tile under “Tenant Name/ID”

azure account show
  • Create an Active Directory application using the format below.
  • The name, home-page, and identifiter-uris will be NOT used in any way and are simply required as part of the setup process.
  • The password you choose must be copied and pasted into the form on the Azure setup tile under “Client Secret”
azure ad app create --name "DatadogAuth" --home-page "http://app.datadoghq.com" --identifier-uris "http://app.datadoghq.com" --password "SomePassword"
  • Create a Service Principal using the AppId returned from the last command.
  • Copy and paste this AppId into the form on the Azure setup tile under “Client ID”

azure cli <0.10.2:

azure ad sp create {app-id}

azure cli >= 0.10.2:

azure ad sp create -a {app-id}
  • Grant the Service Principal the “Reader” role for the subscription you are interested in monitoring.
  • Use the Object Id returned from the previous command to fill in {object-Id} {subscription-Id} is the azure subscription you would like to monitor, and is listed as ID in azure account show or through the portal
azure role assignment create --objectId {object-Id} --roleName Reader --subscription {subscription-Id}

Integrating through the Azure Portals

  1. Get your tenant name and pass it to Datadog.
  2. Create a web application in your Active Directory and pass the correct credentials to Datadog.
  3. Give this application read-access to any subscriptions you would like to monitor.
Getting your Tenant Name
  1. Navigate to portal.azure.com
  2. In the leftmost blade, select “Azure Active Directory”
  3. Under properties, copy the Directory ID Value

    Azure tenant name
  4. Paste the ID under “Tenant Name/ID” in the form on the Azure setup tile

    tenant name form
Creating the Web Application
  1. Navigate to the “App Registrations” tab within your Azure Active Directory.
  2. Press “Add”
  3. Enter a name and Sign-on URL for this app.
    • These will NOT be used in any way and are simply required as part of the setup process.
    • Leave Application “Type as Web app/ API”
  4. Press “Create”

    azure create ad
  5. Once it is created, select the App from the list of App Registrations

  6. Copy the “Application ID” and paste the value into “Client ID” in the form on the Azure setup tile

    azure client id
    client id form
  7. For the same app, go to “All settings”

  8. Go to “Keys”

  9. Enter a new Client Secret key and press Save

    • Make sure to note when the key will expire!
  10. When the Secret Key is shown, copy and paste it in “Client Secret” in the form on the Azure setup tile

    azure client secret
    client secret
  11. Click “Install Integration” to complete the application creation process

Giving Read Permissions to the Application

  1. Navigate to “Subscriptions” on the left hand menu

    subscriptions icon
  2. Click on the subscription you would like to monitor

  3. Click on “Access control (IAM)” in the lefthand menu

    access control button
  4. Click “Add”

    add user button
  5. Select “Reader” as a role

    reader icon
  6. Search/select for the name of the Application you just created (i.e. Datadog Auth)

  7. Click Select

  8. Click OK

  9. Repeat this process for any other subscriptions you would like to monitor

  10. Diagnostics must be enabled for ARM deployed VMs to collect metrics. See the instructions below

In a few minutes, metrics from applications under that subscription will begin to appear!

azure vm screenboard
Naviate to the Azure VM Default Dashboard to see this dashboard populate with your infrastructure’s data

Learn more about how to monitor Azure VM performance metrics with our series of posts. We detail the key performance metrics, how to collect them, and how to use Datadog to monitor Azure VMs.

Configuration

Optionally, you can limit the Azure VMs that are pulled into Datadog by entering tags in the “Optionally filter to VMs with tag” textbox. This comma separated list of tags (in the form ‘key:value’) defines a filter that we will use when collecting metrics from Azure VMs. Wildcards, such as ‘?’ (for single characters) and ‘*’ (for multiple characters) can also be used. Only VMs that match one of the defined tags will be imported into Datadog. The rest will be ignored.

VMs matching a given tag can also be excluded by adding ‘!’ before the tag. For example:

datadog:monitored,env:production,!env:staging,instance-type:c1.*
filter_form

Deploy Agents

  1. Navigate to your VM in the Azure Portal > Settings > Extenstions > Add > Select Datadog Agent. Use an API key found here
  2. Manually deploy Agents by following the instructions here
  3. Install based on operating system or CICD tool using these instructions

Data Collected

Metrics

azure.analysisservices_servers.command_pool_job_queue_length
(count)
Number of jobs in the command thread pool queue
shown as job
azure.analysisservices_servers.current_user_sessions
(count)
Number of current user sessions
shown as session
azure.analysisservices_servers.memory_metric
(gauge)
Memory. Range 0-25 GB for S1, 0-50 GB for S2 and 0-100 GB for S4
shown as byte
azure.analysisservices_servers.processing_pool_job_queue_length
(count)
Number of jobs in the processing pool queue
shown as job
azure.analysisservices_servers.qpu_metric
(gauge)
Query Processing Units. Range 0-100 for S1, 0-200 for S2 and 0-400 for S4
shown as unit
azure.analysisservices_servers.query_pool_busy_threads
(count)
Number of busy threads in the query pool
shown as thread
azure.analysisservices_servers.successfull_connections_per_sec
(rate)
Rate of succesful connection completions
shown as connection
azure.analysisservices_servers.total_connection_failures
(count)
Total failed connections
shown as connection
azure.analysisservices_servers.total_connection_requests
(count)
Total connection requests
shown as request
azure.apimanagement_service.failed_requests
(count)
Number of failed gateway requests
shown as request
azure.apimanagement_service.other_requests
(count)
Number of other gateway requests
shown as request
azure.apimanagement_service.successful_requests
(count)
Number of successful gateway requests
shown as request
azure.apimanagement_service.total_requests
(count)
Total number of gateway requests
shown as request
azure.apimanagement_service.unauthorized_requests
(count)
Number of unauthorized gateway requests
shown as request
azure.cognitiveservices_accounts.blocked_calls
(count)
Number of calls that exceeded quota or rate limit
shown as request
azure.cognitiveservices_accounts.client_errors
(count)
Number of calls with client side errors (4xx)
shown as error
azure.cognitiveservices_accounts.data_in
(gauge)
Incoming data size
shown as byte
azure.cognitiveservices_accounts.data_out
(gauge)
Outgoing data size
shown as byte
azure.cognitiveservices_accounts.latency
(gauge)
Latency of calls
shown as millisecond
azure.cognitiveservices_accounts.server_errors
(count)
Number of calls with internal server errors (5xx)
shown as error
azure.cognitiveservices_accounts.successful_calls
(count)
Number of succesful calls
shown as request
azure.cognitiveservices_accounts.total_calls
(count)
Total number of calls
shown as request
azure.cognitiveservices_accounts.total_errors
(count)
Total number of calls with errors
shown as error
azure.network_applicationgateways.throughput
(rate)
Application gateway network throughput
shown as byte
azure.network_loadbalancers.byte_count
(count)
Total number of bytes processed per front-end
shown as byte
azure.network_loadbalancers.dip_availability
(count)
Average count of availability of DIP endpoints based on probe results
shown as unit
azure.network_loadbalancers.packet_count
(count)
Total number of packets processed per front-end
shown as packet
azure.network_loadbalancers.snat_connection_count
(count)
Total number of new SNAT connections that is outbound connections that are masqueraded to the Public IP address front-end
shown as connection
azure.network_loadbalancers.syn_count
(count)
Total number of SYN packets received
shown as packet
azure.network_loadbalancers.vip_availability
(count)
Average count of availability of VIP endpoints based on probe results
shown as unit
azure.search_searchservices.search_latency
(gauge)
Search latency for the search service
shown as second
azure.search_searchservices.search_queries_per_second
(rate)
Rate of search queries for the search service
shown as query
azure.search_searchservices.throttled_search_queries_percentage
(gauge)
Percentage of throttled search queries for the search service
shown as percent
azure.servicebus_namespaces.cpu_per_namespace
(gauge)
Service bus premium namespace CPU usage metric
shown as percent
azure.servicebus_namespaces.mem_usage_per_namespace
(gauge)
Service bus premium namespace memory usage metric
shown as percent
azure.streamanalytics_streamingjobs.aml_callout_failed_requests
(count)
AML callout function failed requests
shown as request
azure.streamanalytics_streamingjobs.aml_callout_input_events
(count)
AML callout function input events
shown as event
azure.streamanalytics_streamingjobs.aml_callout_requests
(count)
AML callout function requests
shown as request
azure.streamanalytics_streamingjobs.conversion_errors
(count)
Number of data conversion errors
shown as error
azure.streamanalytics_streamingjobs.dropped_or_adjusted_events
(count)
Number of out of order events
shown as event
azure.streamanalytics_streamingjobs.errors
(count)
Number of runtime errors
shown as error
azure.streamanalytics_streamingjobs.input_event_bytes
(gauge)
Input event size in bytes
shown as byte
azure.streamanalytics_streamingjobs.input_events
(count)
Number of input events
shown as event
azure.streamanalytics_streamingjobs.late_input_events
(count)
Number of late input events
shown as event
azure.streamanalytics_streamingjobs.output_events
(count)
Number of output events
shown as event
azure.streamanalytics_streamingjobs.resource_utilization
(gauge)
Streaming units utilization percentage
shown as percent

View the specific metrics we collect for each Azure service integration:

Events

The Datadog-Azure integrations send all your azure events in your Event stream.

Troubleshooting

Here are some common issues you might be seeing.

I don’t know my tenant name

  1. Navigate to portal.azure.com
  2. In the leftmost blade, select “Azure Active Directory”
  3. Under properties, it is the Directory ID

    azure tenant name

Your tenant name is also available from the URL after navigating to the classic portal. It is the text in between (not including) the @ and # symbol

azure tenant url

Unable to login

If you have experienced an error logging in while trying to install the integration, please reach out to support@datadoghq.com. When possible, please attach a screenshot.

No Metrics Are Showing Up

Please ensure you completed step three of the installation process, which entails giving read permissions to the Azure application (created in step two) for the subscriptions you want to monitor.

For ARM deployed virtual machines, you must also turn on Diagnostics and select the VM metrics you would like to collect. See Enable Diagnostics below for instructions.

Missing Metrics?

For ARM virtual machines, ensure you have enabled diagnostics and selected the metrics you would like to collect using the instructions below.

For other missing metrics, please reach out to support@datadoghq.com.

Enable diagnostics

Turning on Diagnostics allows ARM deployed VMs to collect logging information which includes metrics for CPU, Network, etc. To do this, first go to Azure Portal then follow the instructions below.

After locating your VM:

  1. Click on Diagnostics settings under the Monitoring section
  2. Shift the status to On
  3. Select the metrics you would like to collect (we recommend “Basic metrics”, “Network and web metrics”, and “.Net metrics”. Un-checking logs collection could save you some storage space. Linux Virtual Machines only collect “Basic” and “Boot” diagnostics)
  4. Click Save to save your changes

    azure enable diagnostics