Amazon Redshift

Versión de integración1.0.0

To find out if this integration is available in your organization, see your Datadog Integrations page or ask your organization administrator.

To initiate an exception request to enable this integration for your organization, email support@ddog-gov.com.

Esta página aún no está disponible en español. Estamos trabajando en su traducción.
Si tienes alguna pregunta o comentario sobre nuestro actual proyecto de traducción, no dudes en ponerte en contacto con nosotros.

Overview

Amazon Redshift is a fast, fully managed, petabyte-scale data warehouse service that makes it simple and cost-effective to efficiently analyze all your data.

Enable this integration to see all your Redshift metrics in Datadog.

Setup

Installation

If you haven’t already, set up the Amazon Web Services integration first.

Metric collection

  1. In the AWS integration page, ensure that Redshift is enabled under the Metric Collection tab.

  2. Add these permissions to your Datadog IAM policy in order to collect Amazon Redshift metrics:

    • redshift:DescribeClusters: List all Redshift Clusters in your account.
    • redshift:DescribeLoggingStatus: Get S3 bucket where Redshift logs are stored.
    • tag:GetResources: Get custom tags on your Redshift clusters.

    For more information, see the Redshift policies on the AWS website.

  3. Install the Datadog - Amazon Redshift integration.

Log collection

Enable logging

Enable the logging on your Redshift Cluster first to collect your logs. Redshift logs can be written to an Amazon S3 bucket and consumed by a Lambda function. For more information, see Configuring auditing using the console.

Send logs to Datadog

  1. If you haven’t already, set up the Datadog Forwarder Lambda function in your AWS account.

  2. Once the Lambda function is installed, there are two ways to collect your Redshift logs:

    • Automatically: Redshift logs are managed automatically if you grant Datadog access with a set of permissions. See Automatically Set Up Triggers for more information on configuring automatic log collection on the Datadog Forwarder Lambda function.
    • Manually: In the AWS console, add a trigger on the S3 bucket that contains your Redshift logs. See the manual installation steps.

Manual installation steps

  1. If you haven’t already, set up the Datadog Forwarder Lambda function in your AWS account.
  2. Once set up, go to the Datadog Forwarder Lambda function. In the Function Overview section, click Add Trigger.
  3. Select the S3 trigger for the Trigger Configuration.
  4. Select the S3 bucket that contains your Redshift logs.
  5. Leave the event type as All object create events.
  6. Click Add to add the trigger to your Lambda.

Go to the Log Explorer to start exploring your logs.

For more information on collecting AWS Services logs, see Send AWS Services Logs with the Datadog Lambda Function.

Data Collected

Metrics

aws.redshift.auto_copy_error_by_cluster
(count)		The number of auto-copy job errors for the cluster.
Shown as error
aws.redshift.auto_copy_error_by_job
(count)		The number of errors for an auto-copy job.
Shown as error
aws.redshift.auto_copy_sqs_polling_throttled
(count)		The number of times auto-copy polling of the Amazon SQS event notification queue was throttled.
Shown as throttle
aws.redshift.commit_queue_length
(count)		The number of transactions ahead of a transaction in the commit queue.
Shown as transaction
aws.redshift.concurrency_scaling_active_clusters
(count)		The number of concurrency scaling clusters that are actively processing queries at any given time.
aws.redshift.concurrency_scaling_seconds
(gauge)		The number of seconds used by concurrency scaling clusters that have active query processing activity.
Shown as second
aws.redshift.cpuutilization
(gauge)		The percentage of CPU utilization. For clusters, this metric represents an aggregation of all nodes (leader and compute) CPU utilization values.
Shown as percent
aws.redshift.database_connections
(gauge)		The number of database connections to a cluster.
Shown as connection
aws.redshift.extra_compute_for_automatic_optimization_seconds
(count)		The number of seconds of extra compute resources used to run automatic optimization tasks.
Shown as second
aws.redshift.health_status
(gauge)		Indicates the health of the cluster. 1 indicates healthy, and 0 indicates unhealthy.
aws.redshift.integration_auto_remediation_triggered
(count)		The number of tables that auto-remediation moved to resynchronization during the reporting interval.
Shown as table
aws.redshift.integration_data_transferred
(count)		The amount of data transferred by the zero-ETL integration, in logical bytes.
Shown as byte
aws.redshift.integration_deleted_rows
(count)		The number of rows deleted by the zero-ETL integration.
Shown as row
aws.redshift.integration_duplicate_rows_detected
(count)		The number of tables with duplicate rows identified during the reporting interval.
Shown as table
aws.redshift.integration_inserted_rows
(count)		The number of rows inserted by the zero-ETL integration.
Shown as row
aws.redshift.integration_lag
(gauge)		The lag from the time data is committed to the source to the time the data is available for queries in Amazon Redshift.
Shown as second
aws.redshift.integration_latest_applied_change
(gauge)		The time, in Unix epoch seconds, when the zero-ETL integration last completed ingestion on Amazon Redshift.
Shown as second
aws.redshift.integration_latest_detected_change
(gauge)		The time, in Unix epoch seconds, when the zero-ETL integration last staged a source change in the replication queue.
Shown as second
aws.redshift.integration_latest_source_commit
(gauge)		The time, in Unix epoch seconds, of the latest source commit observed by the zero-ETL integration.
Shown as second
aws.redshift.integration_num_tables_failed_replication
(gauge)		The number of tables that failed replication.
Shown as table
aws.redshift.integration_num_tables_replicated
(gauge)		The number of tables that have been replicated from the source database to Amazon Redshift.
Shown as table
aws.redshift.integration_state
(gauge)		The state of the zero-ETL integration.
aws.redshift.integration_updated_rows
(count)		The number of rows updated by the zero-ETL integration.
Shown as row
aws.redshift.maintenance_mode
(gauge)		Indicates whether the cluster is in maintenance mode. 1 indicates on, and 0 indicates off.
aws.redshift.max_configured_concurrency_scaling_clusters
(count)		The maximum number of concurrency scaling clusters configured from the parameter group.
aws.redshift.network_receive_throughput
(rate)		The rate at which the node or cluster receives data.
Shown as byte
aws.redshift.network_transmit_throughput
(rate)		The rate at which the node or cluster writes data.
Shown as byte
aws.redshift.num_exceeded_schema_quotas
(count)		The number of schemas with exceeded quotas.
aws.redshift.percentage_disk_space_used
(gauge)		The percent of disk space used.
Shown as percent
aws.redshift.percentage_quota_used
(gauge)		The percentage of disk or storage space used relative to the configured schema quota.
Shown as percent
aws.redshift.queries_completed_per_second
(count)		The average number of queries completed per second. Reported in five-minute intervals.
Shown as query
aws.redshift.query_duration
(gauge)		The average amount of time to complete a query. Reported in five-minute intervals.
Shown as microsecond
aws.redshift.query_runtime_breakdown
(gauge)		AWS Redshift query runtime breakdown
aws.redshift.read_iops
(rate)		The average number of disk read operations per second.
Shown as operation
aws.redshift.read_latency
(gauge)		The average amount of time taken for disk read I/O operations.
Shown as second
aws.redshift.read_throughput
(rate)		The average number of bytes read from disk per second.
Shown as byte
aws.redshift.redshift_managed_storage_billed_used_space
(gauge)		The amount of Amazon Redshift managed storage space that is billed.
Shown as megabyte
aws.redshift.redshift_managed_storage_ds_used_space
(gauge)		The amount of Amazon Redshift managed storage space used by data.
Shown as megabyte
aws.redshift.redshift_managed_storage_total_capacity
(gauge)		The total Amazon Redshift managed storage capacity.
Shown as megabyte
aws.redshift.schema_quota
(gauge)		The configured quota for a schema.
Shown as byte
aws.redshift.storage_used
(gauge)		The disk or storage space used by a schema.
Shown as byte
aws.redshift.total_table_count
(count)		The number of user tables open at a particular point in time. This total does not include Spectrum tables.
Shown as table
aws.redshift.usage_limit_available
(gauge)		The amount available under a usage limit. Measured in minutes for compute features and in terabytes for data scanning features.
aws.redshift.usage_limit_consumed
(gauge)		The amount consumed under a usage limit. Measured in minutes for compute features and in terabytes for data scanning features.
aws.redshift.user_queries_failed
(count)		The number of user queries that failed.
Shown as query
aws.redshift.user_queries_succeeded
(count)		The number of user queries that succeeded.
Shown as query
aws.redshift.user_query_duration
(gauge)		The average amount of time to complete a user query.
Shown as microsecond
aws.redshift.wlmqueries_completed_per_second
(count)		The average number of queries completed per second for a workload management (WLM) queue. Reported in five-minute intervals.
Shown as query
aws.redshift.wlmquery_duration
(gauge)		The average length of time to complete a query for a workload management (WLM) queue. Reported in five-minute intervals.
Shown as microsecond
aws.redshift.wlmqueue_length
(count)		The number of queries waiting to enter a workload management (WLM) queue.
Shown as query
aws.redshift.wlmqueue_wait_time
(gauge)		The total time queries spent waiting in the workload management (WLM) queue.
Shown as millisecond
aws.redshift.wlmrunning_queries
(count)		The number of queries running from both the main cluster and Concurrency Scaling cluster per WLM queue.
Shown as query
aws.redshift.write_iops
(rate)		The average number of write operations per second.
Shown as operation
aws.redshift.write_latency
(gauge)		The average amount of time taken for disk write I/O operations.
Shown as second
aws.redshift.write_throughput
(rate)		The average number of bytes written to disk per second.
Shown as byte

Each of the metrics retrieved from AWS are assigned the same tags that appear in the AWS console, including but not limited to host name, security-groups, and more.

Events

The Amazon Redshift integration does not include any events.

Service Checks

The Amazon Redshift integration does not include any service checks.

Troubleshooting

Need help? Contact Datadog support.