For AI agents: A markdown version of this page is available at https://docs.datadoghq.com/integrations/alibaba-cloud-kafka.md. A documentation index is available at /llms.txt.

Alibaba Cloud Kafka

Integration version1.0.0

To find out if this integration is available in your organization, see your Datadog Integrations page or ask your organization administrator.

To initiate an exception request to enable this integration for your organization, email support@ddog-gov.com.

Overview

Alibaba Cloud Message Queue for Apache Kafka is a fully managed, high-throughput distributed messaging service compatible with the Apache Kafka ecosystem. It is used for real-time data pipelines, log collection, and stream processing.

Use this integration to monitor instance throughput, disk usage, consumer group lag, and connection metrics.

Setup

Installation

After you set up the Alibaba Cloud integration, ensure that the Kafka service is included in your Alibaba Cloud monitoring configuration.

Data Collected

Metrics

alibabacloud.kafka.cluster_message_input_v3
(rate)
Input byte for V3 clusters.
Shown as byte
alibabacloud.kafka.disk_instance_ratio_v3
(gauge)
Disk Usage by Instance.
Shown as percent
alibabacloud.kafka.group_message_num_output_onetopic_onepartition_v3
(rate)
Number of messages per topic per partition.
Shown as message
alibabacloud.kafka.group_message_num_output_onetopic_v3
(rate)
Number of messages per topic.
Shown as message
alibabacloud.kafka.group_message_num_output_v3
(rate)
Number of messages.
Shown as message
alibabacloud.kafka.instance_batch_size_tp50v2
(gauge)
Instance Batch Size TP50.
Shown as byte
alibabacloud.kafka.instance_batch_size_tp999v2
(gauge)
Instance Batch Size TP999.
Shown as byte
alibabacloud.kafka.instance_cpu_usage_v2
(gauge)
Instance CPU Utilization.
Shown as percent
alibabacloud.kafka.instance_disk_log_size_v3
(gauge)
Instance Disk log size.
Shown as byte
alibabacloud.kafka.instance_internet_rx_rate_v3
(rate)
Instance inbound internet traffic.
Shown as byte
alibabacloud.kafka.instance_internet_tx_rate_v3
(rate)
Instance outbound internet traffic.
Shown as byte
alibabacloud.kafka.instance_max_connection_ratio_v3
(gauge)
Usage of Maximum Connections per Node (Internet + Internal Network).
Shown as percent
alibabacloud.kafka.instance_max_connection_v3
(gauge)
Instance max connections.
Shown as instance
alibabacloud.kafka.instance_max_internet_connection_ratio_v3
(gauge)
Usage of Maximum Connections per Node (Internet).
Shown as percent
alibabacloud.kafka.instance_max_internet_connection_v3
(gauge)
Instance max internet connections.
Shown as instance
alibabacloud.kafka.instance_max_node_input_ratio_v3
(gauge)
Instance Max Node Input Ratio.
Shown as percent
alibabacloud.kafka.instance_max_node_output_ratio_v3
(gauge)
Instance Max Node Output Ratio.
Shown as percent
alibabacloud.kafka.instance_message_input_ratio_v3
(gauge)
Instance message input ratio.
Shown as percent
alibabacloud.kafka.instance_message_input_v3
(rate)
Instance message input.
Shown as byte
alibabacloud.kafka.instance_message_num_input_v3
(rate)
Instance message number input.
Shown as message
alibabacloud.kafka.instance_message_num_output_v3
(rate)
Instance message number output.
Shown as message
alibabacloud.kafka.instance_message_output_ratio_v3
(gauge)
Instance message output ratio.
Shown as percent
alibabacloud.kafka.instance_message_output_v3
(rate)
Instance message output.
Shown as byte
alibabacloud.kafka.instance_reqs_input_v3
(rate)
Instance request input rate.
Shown as instance
alibabacloud.kafka.instance_reqs_output_v3
(rate)
Instance request output rate.
Shown as instance
alibabacloud.kafka.instance_throttle_time_p99_input_v3
(gauge)
Instance throttle p99 input.
Shown as millisecond
alibabacloud.kafka.instance_throttle_time_p99_output_v3
(gauge)
Instance throttle p99 output.
Shown as millisecond
alibabacloud.kafka.instance_total_connection_v3
(gauge)
Instance total connections.
Shown as connection
alibabacloud.kafka.instance_total_internet_connection_V3
(gauge)
Instance total internet connections.
Shown as connection
alibabacloud.kafka.message_accumulation_onetopic_v3
(gauge)
Message accumulation per topic.
Shown as message
alibabacloud.kafka.message_accumulation_v3
(gauge)
Message accumulation.
Shown as message
alibabacloud.kafka.partition_instance_ratio_v3
(gauge)
Partition Usage by Instance
Shown as percent
alibabacloud.kafka.topic_abnormal_ha_partition_v3
(gauge)
Topic abnormal ha partition number.
Shown as instance
alibabacloud.kafka.topic_message_input_v3
(rate)
Topic message input.
Shown as byte
alibabacloud.kafka.topic_message_num_input_v3
(rate)
Topic message input number.
Shown as message
alibabacloud.kafka.topic_message_num_output_v3
(rate)
Topic message output number.
Shown as message
alibabacloud.kafka.topic_message_output_input_ratio_v3
(gauge)
Topic message input/output ratio.
Shown as percent
alibabacloud.kafka.topic_message_output_v3
(rate)
Topic message output.
Shown as byte
alibabacloud.kafka.topic_normal_ha_partition_num_v3
(gauge)
Topic normal ha partition number.
Shown as instance
alibabacloud.kafka.topic_reqs_input_v3
(rate)
Topic requests input.
Shown as instance
alibabacloud.kafka.topic_reqs_output_v3
(rate)
Topic requests output.
Shown as instance
alibabacloud.kafka.instance_fetch_throttle_queue_size_v2
(gauge)
Throttled Consumption Requests of Instance.
Shown as instance
alibabacloud.kafka.instance_ha_event_v2
(gauge)
Instance HA Event.
Shown as instance
alibabacloud.kafka.instance_internet_rx_rate_by_node
(rate)
Instance inbound traffic per node.
Shown as byte
alibabacloud.kafka.instance_internet_rx_utilization_by_node
(gauge)
Instance inbound traffic utilization per node.
Shown as percent
alibabacloud.kafka.instance_internet_tx_rate_by_node
(rate)
Instance outbound traffic per node.
Shown as byte
alibabacloud.kafka.instance_internet_tx_utilization_by_node
(gauge)
Instance outbound traffic utilization per node.
Shown as percent
alibabacloud.kafka.instance_max_connection
(gauge)
Instance max connections.
Shown as connection
alibabacloud.kafka.instance_max_internet_connection
(gauge)
Instance max internet connections.
Shown as connection
alibabacloud.kafka.instance_max_read_iops_v2
(gauge)
Maximum Read IOPS per Node.
Shown as operation
alibabacloud.kafka.instance_max_write_iops_v2
(gauge)
Maximum Write IOPS per Node.
Shown as operation
alibabacloud.kafka.instance_message_input_ratio_v2
(gauge)
Instance message input ratio.
Shown as percent
alibabacloud.kafka.instance_message_output_ratio_v2
(gauge)
Instance message output ratio.
Shown as percent
alibabacloud.kafka.instance_produce_throttle_queue_size_v2
(gauge)
Throttled Sending Requests of Instance.
Shown as instance
alibabacloud.kafka.instance_rebalance_time_v2
(gauge)
Maximum Rebalance Time of Instance.
Shown as millisecond
alibabacloud.kafka.instance_total_connection
(gauge)
Instance total connections.
Shown as connection
alibabacloud.kafka.instance_total_internet_connection
(gauge)
Instance total internet connections.
Shown as connection
alibabacloud.kafka.partition_instance_ratio_v2
(gauge)
Partition instance ratio.
Shown as percent
alibabacloud.kafka.cluster_message_input
(rate)
(Deprecated) Cluster message input rate.
Shown as byte
alibabacloud.kafka.group_message_num_output
(rate)
Message number output.
Shown as message
alibabacloud.kafka.group_message_num_output_onetopic
(rate)
Message number output per topic.
Shown as message
alibabacloud.kafka.group_message_num_output_onetopic_onepartition
(rate)
(Deprecated) Message number output per topic per partition.
Shown as message
alibabacloud.kafka.group_topic_accumulation_consume_cost_time
(gauge)
Estimated Time to Clear Topic Backlog for Consumer Group.
Shown as second
alibabacloud.kafka.instance_disk_capacity.maximum
(gauge)
Instance disk capacity.
Shown as percent
alibabacloud.kafka.instance_disk_log_size.maximum
(gauge)
Instance disk log size.
Shown as byte
alibabacloud.kafka.instance_internet_rx.rate
(rate)
Instance internet inbound traffic.
Shown as bit
alibabacloud.kafka.instance_internet_rxrate
(rate)
(Deprecated) Instance internet inbound traffic rate.
Shown as bit
alibabacloud.kafka.instance_internet_tx.rate
(rate)
Instance internet outbound traffic.
Shown as bit
alibabacloud.kafka.instance_internet_txrate
(rate)
(Deprecated) Instance internet outbound traffic rate.
Shown as bit
alibabacloud.kafka.instance_message_input
(rate)
Instance message input.
Shown as byte
alibabacloud.kafka.instance_message_num_input
(rate)
Instance message number input.
Shown as message
alibabacloud.kafka.instance_message_num_output
(rate)
Instance message number output.
Shown as message
alibabacloud.kafka.instance_message_output
(rate)
Instance message output.
Shown as byte
alibabacloud.kafka.instance_reqs_input
(rate)
Instance requests input.
Shown as instance
alibabacloud.kafka.instance_reqs_output
(rate)
Instance requests output.
Shown as instance
alibabacloud.kafka.message_accumulation
(gauge)
Message accumulation.
Shown as message
alibabacloud.kafka.message_accumulation_onetopic
(gauge)
Message accumulation per topic.
Shown as message
alibabacloud.kafka.topic_message_input
(rate)
Topic message input.
Shown as byte
alibabacloud.kafka.topic_message_num_input
(rate)
Topic message number input.
Shown as message
alibabacloud.kafka.topic_message_num_output
(rate)
Topic message number output.
Shown as message
alibabacloud.kafka.topic_message_output
(rate)
Topic message output.
Shown as byte
alibabacloud.kafka.topic_message_output_input_ratio
(gauge)
Topic message output/input ratio.
Shown as percent
alibabacloud.kafka.topic_reqs_input
(rate)
Topic requests input.
Shown as instance
alibabacloud.kafka.topic_reqs_output
(rate)
Topic requests output.
Shown as instance

Events

Alibaba Cloud Kafka does not include any events.

Service Checks

Alibaba Cloud Kafka does not include any service checks.

Troubleshooting

Need help? Contact Datadog support.