Setting Up Database Monitoring for Azure Database for MySQL
Database Monitoring is not supported for this site.
Database Monitoring provides deep visibility into your MySQL databases by exposing query metrics, query samples, explain plans, connection data, system metrics, and telemetry for the InnoDB storage engine.
The Agent collects telemetry directly from the database by logging in as a read-only user. Complete the following steps to enable Database Monitoring with your MySQL database:
- Configure database parameters
- Grant the Agent access to the database
- Install the Agent
- Install the Azure MySQL integration
Before you begin
- Supported MySQL versions
- 5.7, or 8.0+
- Supported Azure MySQL deployment types
- MySQL on Azure VMs, Single Server, Flexible Server
- Supported Agent versions
- 7.36.1+
- Performance impact
- The default Agent configuration for Database Monitoring is conservative, but you can adjust settings such as the collection interval and query sampling rate to better suit your needs. For most workloads, the Agent represents less than one percent of query execution time on the database and less than one percent of CPU.
Database Monitoring runs as an integration on top of the base Agent (see benchmarks). - Proxies, load balancers, and connection poolers
- The Agent must connect directly to the host being monitored, preferably through the instance endpoint. The Agent should not connect to the database through a proxy, load balancer, or connection pooler. While this can be an anti-pattern for client applications, each Agent must have knowledge of the underlying hostname and should stick to a single host for its lifetime, even in cases of failover. If the Datadog Agent connects to different hosts while it is running, the values of metrics will be incorrect.
- Data security considerations
- See Sensitive information for information about what data the Agent collects from your databases and how to ensure it is secure.
Configure the following in the server parameters and then restart the server for the settings to take effect:
The agent also requires performance_schema.events_statements_*
consumers to be set to ON
to collect currently running queries. By default, Azure MySQL Database enables performance schema consumers so no additional configuration is required.
Grant the Agent access
The Datadog Agent requires read-only access to the database in order to collect statistics and queries.
Create the datadog
user and grant basic permissions:
CREATE USER datadog@'%' IDENTIFIED by '<UNIQUEPASSWORD>';
ALTER USER datadog@'%' WITH MAX_USER_CONNECTIONS 5;
GRANT REPLICATION CLIENT ON *.* TO datadog@'%';
GRANT PROCESS ON *.* TO datadog@'%';
GRANT SELECT ON performance_schema.* TO datadog@'%';
Create the datadog
user and grant basic permissions:
CREATE USER datadog@'%' IDENTIFIED BY '<UNIQUEPASSWORD>';
GRANT REPLICATION CLIENT ON *.* TO datadog@'%' WITH MAX_USER_CONNECTIONS 5;
GRANT PROCESS ON *.* TO datadog@'%';
GRANT SELECT ON performance_schema.* TO datadog@'%';
Create the following schema:
CREATE SCHEMA IF NOT EXISTS datadog;
GRANT EXECUTE ON datadog.* to datadog@'%';
GRANT CREATE TEMPORARY TABLES ON datadog.* TO datadog@'%';
Create the explain_statement
procedure to enable the Agent to collect explain plans:
DELIMITER $$
CREATE PROCEDURE datadog.explain_statement(IN query TEXT)
SQL SECURITY DEFINER
BEGIN
SET @explain := CONCAT('EXPLAIN FORMAT=json ', query);
PREPARE stmt FROM @explain;
EXECUTE stmt;
DEALLOCATE PREPARE stmt;
END $$
DELIMITER ;
Additionally, create this procedure in every schema from which you want to collect explain plans. Replace <YOUR_SCHEMA>
with your database schema:
DELIMITER $$
CREATE PROCEDURE <YOUR_SCHEMA>.explain_statement(IN query TEXT)
SQL SECURITY DEFINER
BEGIN
SET @explain := CONCAT('EXPLAIN FORMAT=json ', query);
PREPARE stmt FROM @explain;
EXECUTE stmt;
DEALLOCATE PREPARE stmt;
END $$
DELIMITER ;
GRANT EXECUTE ON PROCEDURE <YOUR_SCHEMA>.explain_statement TO datadog@'%';
Install the Agent
To monitor Azure hosts, install the Datadog Agent in your infrastructure and configure it to connect to each instance endpoint remotely. The Agent does not need to run on the database, it only needs to connect to it. For additional Agent installation methods not mentioned here, see the Agent installation instructions.
To configure this check for an Agent running on a host, for example when you provision a small virtual machine for the Agent to collect from the database:
Edit the mysql.d/conf.yaml
file, in the conf.d/
folder at the root of your Agent’s configuration directory to start collecting your MySQL metrics. See the sample mysql.d/conf.yaml for all available configuration options, including those for custom metrics.
Add this configuration block to your mysql.d/conf.yaml
to collect MySQL metrics:
init_config:
instances:
- dbm: true
host: '<AZURE_INSTANCE_ENDPOINT>'
port: 3306
username: datadog
password: '<YOUR_CHOSEN_PASSWORD>' # from the CREATE USER step earlier
# After adding your project and instance, configure the Datadog Azure integration to pull additional cloud data such as CPU and Memory.
azure:
deployment_type: '<DEPLOYMENT_TYPE>'
fully_qualified_domain_name: '<AZURE_INSTANCE_ENDPOINT>'
See the MySQL integration spec for additional information on setting deployment_type
and name
fields.
Note: Wrap your password in single quotes in case a special character is present.
Restart the Agent to start sending MySQL metrics to Datadog.
To configure the Database Monitoring Agent running in a Docker container, you can set the Autodiscovery Integration Templates as Docker labels on your agent container.
Note: The Agent must have read permission on the Docker socket for Autodiscovery of labels to work.
Command line
Execute the following command to run the agent from your command line. Replace the values to match your account and environment:
export DD_API_KEY=xxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
export DD_AGENT_VERSION=7.36.1
docker run -e "DD_API_KEY=${DD_API_KEY}" \
-v /var/run/docker.sock:/var/run/docker.sock:ro \
-l com.datadoghq.ad.check_names='["mysql"]' \
-l com.datadoghq.ad.init_configs='[{}]' \
-l com.datadoghq.ad.instances='[{
"dbm": true,
"host": "<AZURE_INSTANCE_ENDPOINT>",
"port": 3306,
"username": "datadog",
"password": "<UNIQUEPASSWORD>",
"azure": {
"deployment_type": "<DEPLOYMENT_TYPE>",
"fully_qualified_domain_name": "<AZURE_INSTANCE_ENDPOINT>"
}
}]' \
gcr.io/datadoghq/agent:${DD_AGENT_VERSION}
Dockerfile
Labels can also be specified in a Dockerfile
, so you can build and deploy a custom agent without changing any infrastructure configuration:
FROM datadog/agent:7.36.1
LABEL "com.datadoghq.ad.check_names"='["mysql"]'
LABEL "com.datadoghq.ad.init_configs"='[{}]'
LABEL "com.datadoghq.ad.instances"='[{"dbm": true, "host": "<AZURE_INSTANCE_ENDPOINT>", "port": 3306,"username": "datadog","password": "<UNIQUEPASSWORD>", "azure": {"deployment_type": "<DEPLOYMENT_TYPE>", "fully_qualified_domain_name": "<AZURE_INSTANCE_ENDPOINT>"}}]'
See the MySQL integration spec for additional information on setting deployment_type
and name
fields.
To avoid exposing the datadog
user’s password in plain text, use the Agent’s secret management package and declare the password using the ENC[]
syntax, or see the Autodiscovery template variables documentation on how to pass in the password as an environment variable.
If you have a Kubernetes cluster, use the Datadog Cluster Agent for Database Monitoring.
Follow the instructions to enable the cluster checks if not already enabled in your Kubernetes cluster. You can declare the MySQL configuration with static files mounted in the Cluster Agent container, or by using service annotations:
Command line with Helm
Execute the following Helm command to install the Datadog Cluster Agent on your Kubernetes cluster. Replace the values to match your account and environment:
helm repo add datadog https://helm.datadoghq.com
helm repo update
helm install <RELEASE_NAME> \
--set 'datadog.apiKey=<DATADOG_API_KEY>' \
--set 'clusterAgent.enabled=true' \
--set 'clusterAgent.confd.mysql\.yaml=cluster_check: true
init_config:
instances:
- dbm: true
host: <INSTANCE_ADDRESS>
port: 3306
username: datadog
password: "<UNIQUEPASSWORD>"
azure:
deployment_type: "<DEPLOYMENT_TYPE>"
fully_qualified_domain_name: "<AZURE_INSTANCE_ENDPOINT>"' \
datadog/datadog
To configure a cluster check with a mounted configuration file, mount the configuration file in the Cluster Agent container on the path /conf.d/mysql.yaml
:
cluster_check: true # Make sure to include this flag
init_config:
instances:
- dbm: true
host: '<AZURE_INSTANCE_ENDPOINT>'
port: 3306
username: datadog
password: '<UNIQUEPASSWORD>'
# After adding your project and instance, configure the Datadog Azure integration to pull additional cloud data such as CPU, Memory, etc.
azure:
deployment_type: '<DEPLOYMENT_TYPE>'
fully_qualified_domain_name: '<AZURE_INSTANCE_ENDPOINT>'
Rather than mounting a file, you can declare the instance configuration as a Kubernetes Service. To configure this check for an Agent running on Kubernetes, create a Service in the same namespace as the Datadog Cluster Agent:
apiVersion: v1
kind: Service
metadata:
name: mysql
labels:
tags.datadoghq.com/env: '<ENV>'
tags.datadoghq.com/service: '<SERVICE>'
annotations:
ad.datadoghq.com/service.check_names: '["mysql"]'
ad.datadoghq.com/service.init_configs: '[{}]'
ad.datadoghq.com/service.instances: |
[
{
"dbm": true,
"host": "<AZURE_INSTANCE_ENDPOINT>",
"port": 3306,
"username": "datadog",
"password": "<UNIQUEPASSWORD>",
"azure": {
"deployment_type": "<DEPLOYMENT_TYPE>",
"fully_qualified_domain_name": "<AZURE_INSTANCE_ENDPOINT>"
}
}
]
spec:
ports:
- port: 3306
protocol: TCP
targetPort: 3306
name: mysql
See the MySQL integration spec for additional information on setting deployment_type
and name
fields.
The Cluster Agent automatically registers this configuration and begins running the MySQL check.
To avoid exposing the datadog
user’s password in plain text, use the Agent’s secret management package and declare the password using the ENC[]
syntax.
Validate
Run the Agent’s status subcommand and look for mysql
under the Checks section. Or visit the Databases page to get started.
Example Agent Configurations
One agent connecting to multiple hosts
It is common to configure a single Agent host to connect to multiple remote database instances (see Agent installation architectures for DBM). To connect to multiple hosts, create an entry for each host in the MySQL integration config.
In these cases, Datadog recommends limiting the number of instances per Agent to a maximum of 10 database instances to guarantee reliable performance.
init_config:
instances:
- dbm: true
host: example-service-primary.example-host.com
port: 3306
username: datadog
password: '<PASSWORD>'
tags:
- 'env:prod'
- 'team:team-discovery'
- 'service:example-service'
- dbm: true
host: example-service-replica-1.example-host.com
port: 3306
username: datadog
password: '<PASSWORD>'
options:
replication: true
tags:
- 'env:prod'
- 'team:team-discovery'
- 'service:example-service'
- dbm: true
host: example-service-replica-2.example-host.com
port: 3306
username: datadog
password: '<PASSWORD>'
options:
replication: true
tags:
- 'env:prod'
- 'team:team-discovery'
- 'service:example-service'
[...]
Storing passwords securely
While it is possible to declare passwords directly in the Agent configuration files, it is a more secure practice to encrypt and store database credentials elsewhere using secret management software such as Vault. The Agent is able to read these credentials using the ENC[]
syntax. Review the secrets management documentation for the required setup to store these credentials. The following example shows how to declare and use those credentials:
init_config:
instances:
- dbm: true
host: localhost
port: 3306
username: datadog
password: 'ENC[datadog_user_database_password]'
Running custom queries
To collect custom metrics, use the custom_queries
option. See the sample mysql.d/conf.yaml for more details.
init_config:
instances:
- dbm: true
host: localhost
port: 3306
username: datadog
password: '<PASSWORD>'
custom_queries:
- query: SELECT age, salary, hours_worked, name FROM hr.employees;
columns:
- name: custom.employee_age
type: gauge
- name: custom.employee_salary
type: gauge
- name: custom.employee_hours
type: count
- name: name
type: tag
tags:
- 'table:employees'
Working with hosts through a proxy
If the Agent must connect through a proxy such as the Cloud SQL Auth proxy, all telemetry is tagged with the hostname of the proxy rather than the database instance. Use the reported_hostname
option to set a custom override of the hostname detected by the Agent.
init_config:
instances:
- dbm: true
host: localhost
port: 5000
username: datadog
password: '<PASSWORD>'
reported_hostname: example-service-primary
- dbm: true
host: localhost
port: 5001
username: datadog
password: '<PASSWORD>'
reported_hostname: example-service-replica-1
Install the Azure MySQL Integration
To collect more comprehensive database metrics from Azure, install the MySQL integration (optional).
Troubleshooting
If you have installed and configured the integrations and Agent as described, and it is not working as expected, see Troubleshooting.
Further reading
Additional helpful documentation, links, and articles: