With Data Streams Monitoring’s Kafka Monitoring, a Datadog Agent check connects to your Kafka cluster and starts collecting health and performance metrics. Kafka Monitoring allows you to:

• Monitor Kafka health: See cluster, broker, topic, and partition health with throughput, lag, and replication metrics
• Pinpoint root cause: Correlate configuration and schema changes with lag, throughput, and errors, and trace issues to the exact topic, schema version, or configuration change
• Connect services to topics: See which producers and consumers interact with each topic, with linked owners, repos, on-call rotations, traces, and error logs
• Inspect topic schemas and messages: View schemas, compare versions, and access messages to debug poison payloads or explore the topic

Setup

Go to the Kafka Monitoring setup page and click Get Started. Then choose your environment and follow the instructions. To request assistance, choose Request a pairing session.

The setup page provides environment-specific configuration instructions. You can copy the instructions directly to an AI agent with Copy for AI.

Workflows

Monitor cluster health and performance

The Clusters, Topics, and Brokers tabs display health status across your entire Kafka infrastructure. For each topic, you can see partition count, under-replicated and offline partitions, message throughput, and consumer lag.

Click into any topic to see a detailed summary, including incoming message rate, maximum lag across all partitions, and whether current lag is approaching the retention limit.

From any metric, you can create Datadog monitors, SLOs, and dashboards.

Correlate configuration and schema changes with health metrics

Change events are overlaid directly on throughput and lag graphs, so you can see whether a configuration or schema change coincided with a degradation.

To identify exactly what changed, click on detected changes on the overlay and select View config change.

Connect producer and consumer services to topics

The Producers and Consumers sections of each topic show which services are reading from and writing to that topic. Hovering over a service shows ownership information from the Service Catalog: team, code repository, on-call engineer, and Slack channel.

Use this information to contact the right team when a consumer is lagging or a producer is misbehaving.

Inspect topic schemas and messages

The Schema section shows the current schema for a topic’s key or value, with version history. Use the version selector to compare schemas across versions.

The Messages section lets you retrieve messages by partition and offset to inspect payloads directly. This is useful for debugging poison payloads or verifying message structure after a schema change.

Note: Message viewing is not enabled by default and requires additional setup. Access is controlled by per-user permissions, so you can roll out Kafka Monitoring without exposing message content. Configure message viewing separately after completing the initial setup.