Amazon MSK Connect

Integration version1.0.0

To find out if this integration is available in your organization, see your Datadog Integrations page or ask your organization administrator.

To initiate an exception request to enable this integration for your organization, email support@ddog-gov.com.

Overview

Amazon MSK Connect is a feature of Amazon MSK that makes it easy to stream data to and from your Apache Kafka clusters.

Enable this integration to see all your Amazon MSK Connect metrics in Datadog.

Setup

Installation

If you haven’t already, set up the Amazon Web Services integration first.

Metric collection

  1. In the AWS integration tile, ensure that KafkaConnect is enabled under the Metric Collection tab.
  2. Install the Datadog - Amazon MSK Connect integration.

Data Collected

Metrics

aws.kafkaconnect.bytes_in_per_sec
(gauge)
The metadata bytes transferred to the Kafka Connect framework per second for communication between workers.
Shown as byte
aws.kafkaconnect.bytes_out_per_sec
(gauge)
The metadata bytes transferred from the Kafka Connect framework per second for communication between workers.
Shown as byte
aws.kafkaconnect.cpu_utilization
(gauge)
The percentage of CPU consumption by system and user.
Shown as percent
aws.kafkaconnect.cpu_utilization.maximum
(gauge)
The maximum percentage of CPU consumption by system and user.
Shown as percent
aws.kafkaconnect.errored_task_count
(gauge)
The number of tasks that have errored out.
Shown as error
aws.kafkaconnect.memory_utilization
(gauge)
The percentage of total memory utilized on a worker instance.
Shown as percent
aws.kafkaconnect.memory_utilization.maximum
(gauge)
The maximum percentage of total memory utilized on a worker instance.
Shown as percent
aws.kafkaconnect.rebalance_completed_total
(count)
The total number of rebalances completed by the connector.
aws.kafkaconnect.rebalance_time_avg
(gauge)
The average time spent by the connector on rebalancing.
Shown as millisecond
aws.kafkaconnect.rebalance_time_max
(gauge)
The maximum time spent by the connector on rebalancing.
Shown as millisecond
aws.kafkaconnect.rebalance_time_since_last
(gauge)
The time since the connector completed the most recent rebalance.
Shown as millisecond
aws.kafkaconnect.running_task_count
(gauge)
The number of tasks currently running in the connector.
Shown as task
aws.kafkaconnect.running_task_count.maximum
(gauge)
The maximum number of tasks currently running in the connector.
Shown as task
aws.kafkaconnect.running_task_count.minimum
(gauge)
The minimum number of tasks currently running in the connector.
Shown as task
aws.kafkaconnect.sink_consumer_byte_rate
(gauge)
The average bytes consumed per second by the Kafka Connect Sink consumer before transformations.
Shown as byte
aws.kafkaconnect.sink_record_read_rate
(gauge)
The average number of records read per second from the Apache Kafka or Amazon MSK cluster.
Shown as record
aws.kafkaconnect.sink_record_send_rate
(gauge)
The average number of records per second output from transformations and sent to the destination.
Shown as record
aws.kafkaconnect.source_producer_byte_rate
(gauge)
The average bytes produced per second by the Kafka Connect Source producer after transformations.
Shown as byte
aws.kafkaconnect.source_record_poll_rate
(gauge)
The average number of records produced or polled per second.
Shown as record
aws.kafkaconnect.source_record_write_rate
(gauge)
The average number of records per second output from transformations and written to the cluster.
Shown as record
aws.kafkaconnect.task_startup_attempts_total
(count)
The total number of task startup attempts by the connector.
Shown as task
aws.kafkaconnect.task_startup_success_percentage
(gauge)
The average percentage of successful task starts for the connector.
Shown as percent
aws.kafkaconnect.task_startup_success_percentage.maximum
(gauge)
The maximum average percentage of successful task starts for the connector.
Shown as percent
aws.kafkaconnect.worker_count
(gauge)
The number of workers running in the connector.
Shown as worker

Events

The Amazon MSK Connect integration does not include any events.

Service Checks

The Amazon MSK Connect integration does not include any service checks.

Troubleshooting

Need help? Contact Datadog support.