Amazon Redshift
Security Monitoring is now available Security Monitoring is now available

Amazon Redshift

Crawler Crawler

Overview

Amazon Redshift is a fast, fully managed, petabyte-scale data warehouse service that makes it simple and cost-effective to efficiently analyze all your data.

Enable this integration to see all your Redshift metrics in Datadog.

Setup

Installation

If you haven’t already, set up the Amazon Web Services integration first.

Metric collection

  1. In the AWS integration tile, ensure that Redshift is checked under metric collection.

  2. Add these permissions to your Datadog IAM policy in order to collect Amazon Redshift metrics:

    • redshift:DescribeClusters: List all Redshift Clusters in your account.
    • redshift:DescribeLoggingStatus: Get S3 bucket where Redshift logs are stored.
    • tag:GetResources: Get custom tags on your Redshift cluters.

    For more information on Redshift policies, review the documentation on the AWS website.

  3. Install the Datadog - AWS Redshift integration.

Log collection

Enable AWS Redshift logging

Enable the logging on your Redshift Cluster first to collect your logs. Redshift logs can be written to an AWS S3 bucket and consumed by a Lambda function. For more information, refer to the AWS documentation

Send logs to Datadog

  1. If you haven’t already, set up the Datadog log collection AWS Lambda function.

  2. Once the lambda function is installed, there are two ways to collect your Redshift logs:

Manual installation steps

  1. If you haven’t already, set up the Datadog log collection AWS Lambda function.
  2. Once the lambda function is installed, manually add a trigger on the S3 bucket that contains your Redshift logs in the AWS console, in your Lambda, click on S3 in the trigger list:
    S3 trigger configuration
    Configure your trigger by choosing the S3 bucket that contains your Redshift logs and change the event type to Object Created (All) then click on the add button.
    S3 Lambda trigger configuration

Once done, go in your Datadog Log section to start exploring your logs!

Data Collected

Metrics

aws.redshift.commit_queue_length
(count)		The number of transactions ahead of a transaction in the commit queue.
Shown as transaction
aws.redshift.concurrency_scaling_active_clusters
(count)		The number of concurrency scaling clusters that are actively processing queries at any given time.
aws.redshift.concurrency_scaling_seconds
(gauge)		The number of seconds used by concurrency scaling clusters that have active query processing activity.
Shown as second
aws.redshift.cpuutilization
(gauge)		The percentage of CPU utilization. For clusters, this metric represents an aggregation of all nodes (leader and compute) CPU utilization values.
Shown as percent
aws.redshift.database_connections
(gauge)		The number of database connections to a cluster.
Shown as connection
aws.redshift.health_status
(gauge)		Indicates the health of the cluster. 1 indicates healthy, and 0 indicates unhealthy.
aws.redshift.maintenance_mode
(gauge)		Indicates whether the cluster is in maintenance mode. 1 indicates on, and 0 indicates off.
aws.redshift.max_configured_concurrency_scaling_clusters
(count)		The maximum number of concurrency scaling clusters configured from the parameter group.
aws.redshift.network_receive_throughput
(rate)		The rate at which the node or cluster receives data.
Shown as byte
aws.redshift.network_transmit_throughput
(rate)		The rate at which the node or cluster writes data.
Shown as byte
aws.redshift.percentage_disk_space_used
(gauge)		The percent of disk space used.
Shown as percent
aws.redshift.queries_completed_per_second
(count)		The average number of queries completed per second. Reported in five-minute intervals.
Shown as query
aws.redshift.query_duration
(gauge)		The average amount of time to complete a query. Reported in five-minute intervals.
Shown as microsecond
aws.redshift.query_runtime_breakdown
(gauge)		AWS Redshift query runtime breakdown
aws.redshift.read_iops
(rate)		The average number of disk read operations per second.
Shown as operation
aws.redshift.read_latency
(gauge)		The average amount of time taken for disk read I/O operations.
Shown as second
aws.redshift.read_throughput
(rate)		The average number of bytes read from disk per second.
Shown as byte
aws.redshift.total_table_count
(count)		The number of user tables open at a particular point in time. This total does not include Spectrum tables.
Shown as table
aws.redshift.wlmqueries_completed_per_second
(count)		The average number of queries completed per second for a workload management (WLM) queue. Reported in five-minute intervals.
Shown as query
aws.redshift.wlmquery_duration
(gauge)		The average length of time to complete a query for a workload management (WLM) queue. Reported in five-minute intervals.
Shown as microsecond
aws.redshift.wlmqueue_length
(count)		The number of queries waiting to enter a workload management (WLM) queue.
Shown as query
aws.redshift.wlmqueue_wait_time
(gauge)		The total time queries spent waiting in the workload management (WLM) queue.
Shown as millisecond
aws.redshift.wlmrunning_queries
(count)		The number of queries running from both the main cluster and Concurrency Scaling cluster per WLM queue.
Shown as query
aws.redshift.write_iops
(rate)		The average number of write operations per second.
Shown as operation
aws.redshift.write_latency
(gauge)		The average amount of time taken for disk write I/O operations.
Shown as second
aws.redshift.write_throughput
(rate)		The average number of bytes written to disk per second.
Shown as byte

Each of the metrics retrieved from AWS will be assigned the same tags that appear in the AWS console, including but not limited to host name, security-groups, and more.

Events

The AWS Redshift integration does not include any events.

Service Checks

The AWS Redshift integration does not include any service checks.

Troubleshooting

Need help? Contact Datadog support.

On this Page