Amazon Redshift

Overview

Amazon Redshift is a fast, fully managed, petabyte-scale data warehouse service that makes it simple and cost-effective to efficiently analyze all your data.

Enable this integration to see all your Redshift metrics in Datadog.

Setup

Installation

If you haven’t already, set up the Amazon Web Services integration first.

Metric collection

  1. In the AWS integration page, ensure that Redshift is enabled under the Metric Collection tab.

  2. Add these permissions to your Datadog IAM policy in order to collect Amazon Redshift metrics:

    • redshift:DescribeClusters: List all Redshift Clusters in your account.
    • redshift:DescribeLoggingStatus: Get S3 bucket where Redshift logs are stored.
    • tag:GetResources: Get custom tags on your Redshift clusters.

    For more information, see the Redshift policies on the AWS website.

  3. Install the Datadog - Amazon Redshift integration.

Log collection

Enable logging

Enable the logging on your Redshift Cluster first to collect your logs. Redshift logs can be written to an Amazon S3 bucket and consumed by a Lambda function. For more information, see Configuring auditing using the console.

Send logs to Datadog

  1. If you haven’t already, set up the Datadog Forwarder Lambda function in your AWS account.

  2. Once the Lambda function is installed, there are two ways to collect your Redshift logs:

    • Automatically: Redshift logs are managed automatically if you grant Datadog access with a set of permissions. See Automatically Set Up Triggers for more information on configuring automatic log collection on the Datadog Forwarder Lambda function.
    • Manually: In the AWS console, add a trigger on the S3 bucket that contains your Redshift logs. See the manual installation steps.

Manual installation steps

  1. If you haven’t already, set up the Datadog Forwarder Lambda function in your AWS account.
  2. Once set up, go to the Datadog Forwarder Lambda function. In the Function Overview section, click Add Trigger.
  3. Select the S3 trigger for the Trigger Configuration.
  4. Select the S3 bucket that contains your Redshift logs.
  5. Leave the event type as All object create events.
  6. Click Add to add the trigger to your Lambda.

Go to the Log Explorer to start exploring your logs.

For more information on collecting AWS Services logs, see Send AWS Services Logs with the Datadog Lambda Function.

Data Collected

Metrics

aws.redshift.commit_queue_length
(count)		The number of transactions ahead of a transaction in the commit queue.
Shown as transaction
aws.redshift.concurrency_scaling_active_clusters
(count)		The number of concurrency scaling clusters that are actively processing queries at any given time.
aws.redshift.concurrency_scaling_seconds
(gauge)		The number of seconds used by concurrency scaling clusters that have active query processing activity.
Shown as second
aws.redshift.cpuutilization
(gauge)		The percentage of CPU utilization. For clusters, this metric represents an aggregation of all nodes (leader and compute) CPU utilization values.
Shown as percent
aws.redshift.database_connections
(gauge)		The number of database connections to a cluster.
Shown as connection
aws.redshift.health_status
(gauge)		Indicates the health of the cluster. 1 indicates healthy, and 0 indicates unhealthy.
aws.redshift.maintenance_mode
(gauge)		Indicates whether the cluster is in maintenance mode. 1 indicates on, and 0 indicates off.
aws.redshift.max_configured_concurrency_scaling_clusters
(count)		The maximum number of concurrency scaling clusters configured from the parameter group.
aws.redshift.network_receive_throughput
(rate)		The rate at which the node or cluster receives data.
Shown as byte
aws.redshift.network_transmit_throughput
(rate)		The rate at which the node or cluster writes data.
Shown as byte
aws.redshift.num_exceeded_schema_quotas
(count)		The number of schemas with exceeded quotas.
aws.redshift.percentage_disk_space_used
(gauge)		The percent of disk space used.
Shown as percent
aws.redshift.percentage_quota_used
(gauge)		The percentage of disk or storage space used relative to the configured schema quota.
Shown as percent
aws.redshift.queries_completed_per_second
(count)		The average number of queries completed per second. Reported in five-minute intervals.
Shown as query
aws.redshift.query_duration
(gauge)		The average amount of time to complete a query. Reported in five-minute intervals.
Shown as microsecond
aws.redshift.query_runtime_breakdown
(gauge)		AWS Redshift query runtime breakdown
aws.redshift.read_iops
(rate)		The average number of disk read operations per second.
Shown as operation
aws.redshift.read_latency
(gauge)		The average amount of time taken for disk read I/O operations.
Shown as second
aws.redshift.read_throughput
(rate)		The average number of bytes read from disk per second.
Shown as byte
aws.redshift.schema_quota
(gauge)		The configured quota for a schema.
Shown as byte
aws.redshift.storage_used
(gauge)		The disk or storage space used by a schema.
Shown as byte
aws.redshift.total_table_count
(count)		The number of user tables open at a particular point in time. This total does not include Spectrum tables.
Shown as table
aws.redshift.wlmqueries_completed_per_second
(count)		The average number of queries completed per second for a workload management (WLM) queue. Reported in five-minute intervals.
Shown as query
aws.redshift.wlmquery_duration
(gauge)		The average length of time to complete a query for a workload management (WLM) queue. Reported in five-minute intervals.
Shown as microsecond
aws.redshift.wlmqueue_length
(count)		The number of queries waiting to enter a workload management (WLM) queue.
Shown as query
aws.redshift.wlmqueue_wait_time
(gauge)		The total time queries spent waiting in the workload management (WLM) queue.
Shown as millisecond
aws.redshift.wlmrunning_queries
(count)		The number of queries running from both the main cluster and Concurrency Scaling cluster per WLM queue.
Shown as query
aws.redshift.write_iops
(rate)		The average number of write operations per second.
Shown as operation
aws.redshift.write_latency
(gauge)		The average amount of time taken for disk write I/O operations.
Shown as second
aws.redshift.write_throughput
(rate)		The average number of bytes written to disk per second.
Shown as byte

Each of the metrics retrieved from AWS are assigned the same tags that appear in the AWS console, including but not limited to host name, security-groups, and more.

Events

The Amazon Redshift integration does not include any events.

Service Checks

The Amazon Redshift integration does not include any service checks.

Troubleshooting

Need help? Contact Datadog support.