Amazon RDS

The standard integration requires selecting RDS on the left side of the AWS integration tile. This allows you to receive metrics about your instance as often as your CloudWatch integration allows. All RDS Engine types are supported.

The preset dashboard for this integration includes the following metric information: connection, replication lag, read operations and latency, computer, RAM, write operations and latency, and disk metrics.

The enhanced integration requires additional configuration and is available for MySQL, Aurora, MariaDB, SQL Server, Oracle, and PostgreSQL engines. Additional metrics are available but an AWS Lambda is required to submit the metrics to Datadog. The higher granularity and additional required services may result in additional AWS charges.

The preset dashboard for this integration includes the following metric information: loads, uptime, CPU Utilization, tasks, memory, SWAP, network receive, network transmit, CPU used per process, memory used per process, disk ops, filesystem used (pct), tasks running, and system CPU utilization.

The native database integration is optional and available for MySQL, Aurora, MariaDB, SQL Server, and PostgreSQL engine types. To get the metrics from RDS and the ones from the native integration to match up, use the dbinstanceidentifier tag on the native integration based on the identifier you assign to the RDS instance. The RDS instances are automatically assigned the tag.

There are 3 preset dashboards available for this configuration: MySQL, Aurora, and PostgreSQL. Each dashboard includes the following metric information: query volume, disk I/O, connections, replication, and AWS resource metrics.

Note: These dashboard display metrics both from AWS CloudWatch and from the individual database engine itself. Enable one of the integrations, MySQL, Aurora, or PostgreSQL, for all integration metrics.

For the standard RDS integration, set up the Amazon Web Services integration first.

Enable enhanced monitoring for your RDS instance during instance creation or afterwards by choosing Modify under Instance Actions. It is recommended to choose 15 for Monitoring Granularity.

The following instructions use KMS and the Lambda Management Console to create an encrypted version of your Datadog API key which can only be used with the RDS Enhanced Monitoring Lambda function. If you already have an encrypted API key from another Lambda such as the Log Forwarder, see the the Lambda function’s README for other options.

Create your KMS key

1. Open the KMS home at https://console.aws.amazon.com/kms/home.
2. Go into Customer managed keys.
3. Choose Create Key.
4. Enter an Alias for the key, such as lambda-datadog-key. Note: An alias cannot begin with aws. Aliases that begin with aws are reserved by Amazon Web Services to represent AWS-managed CMKs in your account.
5. Add the appropriate administrators to determine who can administer this key.
6. No roles need to be added.
7. Save your KMS key.

Create your Lambda function

1. From the Lambda Management Console, create a new Lambda Function. Your Lambda function must be in the same region as the KMS key you created.
2. Choose Serverless Application Repository, search for and select Datadog-RDS-Enhanced
3. Give the application a unique name.
4. Paste the Id of the key created in the previous section in the KMSKeyId parameter and deploy.
5. Once the application is deployed, open up the newly created Lambda Function (click on the function under “Resource”).
6. Click on the Configuration tab and go to the Environment variables section. For the environment variable kmsEncryptedKeys, add in your Datadog API key in the full JSON format in the value field like follows: {"api_key":"<YOUR_API_KEY>"}.
7. Open the Encryption configuration section and select Enable helpers for encryption in transit.
8. In the KMS key to encrypt at rest section, select Use a customer master key and enter the same KMS key created earlier.
9. Press the Encrypt button next to the JSON blob you just entered and in the popup, choose the same KMS key created earlier as well.
10. Press Save.
11. Create a new trigger using the RDSOSMetrics CloudWatch log group as the source.
12. Give the filter a name and optional filter pattern and press Save.

When clicking on test button for your lambda function you might get this error:

{
    "stackTrace": [
        [
            "/var/task/lambda_function.py",
            109,
            "lambda_handler",
            "event = json.loads(gzip.GzipFile(fileobj=StringIO(event['awslogs']['data'].decode('base64'))).read())"
        ]
    ],
    "errorType": "KeyError",
    "errorMessage": "'awslogs'"
}

This can be ignored. The Test button doesn’t work with this setup.

1. Navigate to the AWS Console and open the RDS section to find the instance you want to monitor.

2. Make note of the endpoint URL, for example: mysqlrds.blah.us-east1.rds.amazonaws.com:3306, which is used to configure the Agent. Also make a note of the DB Instance identifier, for example: mysqlrds, which is used to create graphs and dashboards.

1. In the AWS integration tile, ensure that RDS is checked under metric collection.

2. Add the following permissions to your Datadog IAM policy in order to collect Amazon RDS metrics. For more information, see the RDS policies on the AWS website.

   AWS Permission	Description
   rds:DescribeDBInstances	Describe RDS instances to add tags.
   rds:ListTagsForResource	Add custom tags on RDS instances.
   rds:DescribeEvents	Add events related to RDS databases.

3. Install the Datadog - AWS RDS integration.

1. In the AWS integration tile, ensure that RDS is checked under metric collection.

2. Add the following permissions to your Datadog IAM policy in order to collect Amazon RDS metrics. For more information, see the RDS policies on the AWS website.

   AWS Permission	Description
   rds:DescribeDBInstances	Describe RDS instances to add tags.
   rds:ListTagsForResource	Add custom tags on RDS instances.
   rds:DescribeEvents	Add events related to RDS databases.

3. Install the Datadog - AWS RDS integration.

Configure an Agent and connect to your RDS instance by editing the appropriate yaml file in your conf.d directory and then restart your Agent:

For RDS Aurora, edit the YAML file of the database flavor you are using.

If you are using MySQL or MariaDB, then edit mysql.yaml:

init_config:

instances:
    # The endpoint URL from the AWS console
    - server: 'mysqlrds.blah.us-east1-rds.amazonaws.com'
      user: '<USERNAME>'
      pass: '<PASSWORD>'
      port: 3306
      tags:
          - 'dbinstanceidentifier:<INSTANCE_NAME>'

If you are using PostgreSQL, then edit postgres.yaml:

init_config:

instances:
    - host: 'mysqlrds.blah.us-east1-rds.amazonaws.com'
      port: 5432
      username: '<USERNAME>'
      password: '<PASSWORD>'
      dbname: '<DB_NAME>'
      tags:
          - 'dbinstanceidentifier:<DB_INSTANCE_NAME>'

If you are using Microsoft SQL Server, then edit sqlserver.yaml:

init_config:

instances:
    - host: 'mysqlrds.blah.us-east1-rds.amazonaws.com,1433'
      username: '<USERNAME>'
      password: '<PASSWORD>'
      tags:
          - 'dbinstanceidentifier:<DB_INSTANCE_NAME>'

Validation

Run the Agent’s status subcommand and look for something like this under the Checks section:

Checks
======

[...]

  mysql
  -----
      - instance #0 [OK]
      - Collected 8 metrics & 0 events