---
title: Kafka Monitoring
description: >-
  Monitor Kafka cluster health, connect services to topics, and inspect schemas
  and messages with Data Streams Monitoring's Kafka Monitoring.
breadcrumbs: Docs > Data Streams Monitoring > Kafka Monitoring
---

# Kafka Monitoring

{% callout %}
# Important note for users on the following Datadog sites: app.ddog-gov.com

{% alert level="danger" %}
This product is not supported for your selected [Datadog site](https://docs.datadoghq.com/getting_started/site.md). ().
{% /alert %}

{% /callout %}

With Data Streams Monitoring's Kafka Monitoring, a Datadog Agent check connects to your Kafka cluster and starts collecting health and performance metrics. Kafka Monitoring allows you to:

- **Monitor Kafka health**: See cluster, broker, topic, and partition health with throughput, lag, and replication metrics
- **Pinpoint root cause**: Correlate configuration and schema changes with lag, throughput, and errors, and trace issues to the exact topic, schema version, or configuration change
- **Connect services to topics**: See which producers and consumers interact with each topic, with linked owners, repos, on-call rotations, traces, and error logs
- **Inspect topic schemas and messages**: View schemas, compare versions, and access messages to debug poison payloads or explore the topic

To get started, see [Kafka Monitoring Setup](https://docs.datadoghq.com/data_streams/kafka/setup.md).

## Workflows{% #workflows %}

### Monitor cluster health and performance{% #monitor-cluster-health-and-performance %}

The Clusters, Topics, and Brokers tabs display health status across your entire Kafka infrastructure. For each topic, you can see partition count, under-replicated and offline partitions, message throughput, and consumer lag.

{% image
   source="https://docs.dd-static.net/images/data_streams/kafka_clusters_overview-2.eaa471b4f409f4be35d9bc3b61692201.png?auto=format&fit=max&w=850 1x, https://docs.dd-static.net/images/data_streams/kafka_clusters_overview-2.eaa471b4f409f4be35d9bc3b61692201.png?auto=format&fit=max&w=850&dpr=2 2x"
   alt="The Kafka Monitoring clusters view showing cluster list with broker counts, topic names, replication status, and messages-in rate" /%}

Click into any topic to see a detailed summary, including incoming message rate, maximum lag across all partitions, and whether current lag is approaching the retention limit.

{% image
   source="https://docs.dd-static.net/images/data_streams/kafka_topic_summary-2.23fb40c09ce05e82cb8fadb60941fdbf.png?auto=format&fit=max&w=850 1x, https://docs.dd-static.net/images/data_streams/kafka_topic_summary-2.23fb40c09ce05e82cb8fadb60941fdbf.png?auto=format&fit=max&w=850&dpr=2 2x"
   alt="Topic detail summary page showing incoming message rate of 0.8 msg/sec, current lag of 1.15 seconds, and lag-vs-retention status" /%}

From any metric, you can create Datadog monitors, SLOs, and dashboards.

### Correlate configuration and schema changes with health metrics{% #correlate-configuration-and-schema-changes-with-health-metrics %}

Change events are overlaid directly on throughput and lag graphs, so you can see whether a configuration or schema change coincided with a degradation.

{% image
   source="https://docs.dd-static.net/images/data_streams/kafka_topics_lag_change-2.7610fae263830f9d1503c93943ab8b9f.png?auto=format&fit=max&w=850 1x, https://docs.dd-static.net/images/data_streams/kafka_topics_lag_change-2.7610fae263830f9d1503c93943ab8b9f.png?auto=format&fit=max&w=850&dpr=2 2x"
   alt="Topics view with a topic_config change annotation at 17:02:42 overlaid on the lag-by-topic graph, showing a spike correlated with the change event" /%}

To identify exactly what changed, click on detected changes on the overlay and select View config change.

{% image
   source="https://docs.dd-static.net/images/data_streams/lag-by-topic-overlay.ebe66a8ed5de8bcd2ade6819fe80466b.png?auto=format&fit=max&w=850 1x, https://docs.dd-static.net/images/data_streams/lag-by-topic-overlay.ebe66a8ed5de8bcd2ade6819fe80466b.png?auto=format&fit=max&w=850&dpr=2 2x"
   alt="Topic configuration diff view comparing version 625 and 626, with max.message.bytes changed from 1000012 to 1024 highlighted" /%}

### Connect producer and consumer services to topics{% #connect-producer-and-consumer-services-to-topics %}

The Producers and Consumers sections of each topic show which services are reading from and writing to that topic. Hovering over a service shows ownership information from the Service Catalog: team, code repository, on-call engineer, and Slack channel.

{% image
   source="https://docs.dd-static.net/images/data_streams/kafka_topic_service_ownership.a40df6698200c0a73aa7a76f8b4f6dc1.png?auto=format&fit=max&w=850 1x, https://docs.dd-static.net/images/data_streams/kafka_topic_service_ownership.a40df6698200c0a73aa7a76f8b4f6dc1.png?auto=format&fit=max&w=850&dpr=2 2x"
   alt="Topic producers and consumers view with a service panel open showing ownership team (Frameworks), code repo, on-call engineer, Slack channel, and health status" /%}

Use this information to contact the right team when a consumer is lagging or a producer is misbehaving.

### Inspect topic schemas and messages{% #inspect-topic-schemas-and-messages %}

The Schema section shows the current schema for a topic's key or value, with version history. Use the version selector to compare schemas across versions.

The Messages section lets you retrieve messages by partition and offset to inspect payloads directly. This is useful for debugging poison payloads or verifying message structure after a schema change. See [Enable message inspection](https://docs.datadoghq.com/data_streams/kafka/setup.md#enable-message-inspection) for the additional prerequisites and permissions required to retrieve messages.

{% image
   source="https://docs.dd-static.net/images/data_streams/kafka_schema_messages.d188facca9bc9e9930cbf45053f2bf6b.png?auto=format&fit=max&w=850 1x, https://docs.dd-static.net/images/data_streams/kafka_schema_messages.d188facca9bc9e9930cbf45053f2bf6b.png?auto=format&fit=max&w=850&dpr=2 2x"
   alt="Topic schema and messages view showing a Protobuf schema definition and a table of recent messages with date, partition, offset, and message value" /%}