Amazon Managed Streaming for Apache Kafka
Security Monitoring is now available Security Monitoring is now available

Amazon Managed Streaming for Apache Kafka

Agent Check Agent Check

Overview

Amazon Managed Streaming for Apache Kafka (MSK) is a fully managed service that makes it easy to build and run applications that use Apache Kafka to process streaming data.

You can collect metrics from this integration in two ways - with the Datadog Agent or with a Crawler that collects metrics from Cloudwatch.

Agent Check

The Agent check monitors Amazon Managed Streaming for Apache Kafka (Amazon MSK) through the Datadog Agent.

Setup

Follow the instructions below to install and configure this check for an Agent running on a host. For containerized environments, see the Autodiscovery Integration Templates for guidance on applying these instructions.

Installation

  1. Create a client machine if one does not already exist
  2. Ensure the client machine has been granted the permission policy arn:aws:iam::aws:policy/AmazonMSKReadOnlyAccess or equivalent credentials are available
  3. Install the Datadog Agent

Configuration

  1. Edit the amazon_msk.d/conf.yaml file, in the conf.d/ folder at the root of your Agent’s configuration directory to start collecting your Amazon MSK performance data. See the sample amazon_msk.d/conf.yaml for all available configuration options.

  2. Restart the Agent.

Validation

Run the Agent’s status subcommand and look for amazon_msk under the Checks section.

Data Collected

Metrics

aws.msk.go.gc.duration.seconds.count
(gauge)
GC invocation durations count.
aws.msk.go.gc.duration.seconds.quantile
(gauge)
GC invocation duration quantile.
Shown as second
aws.msk.go.gc.duration.seconds.sum
(gauge)
GC invocation duration total.
Shown as second
aws.msk.go.goroutines
(gauge)
Number of goroutines that currently exist.
Shown as thread
aws.msk.go.info
(gauge)
Information about the Go environment.
aws.msk.go.memstats.alloc.bytes
(gauge)
Number of bytes allocated and still in use.
Shown as byte
aws.msk.go.memstats.alloc.bytes.total
(count)
Total number of bytes allocated, even if freed.
Shown as byte
aws.msk.go.memstats.buck.hash.sys.bytes
(gauge)
Number of bytes used by the profiling bucket hash table.
Shown as byte
aws.msk.go.memstats.frees.total
(count)
Total number of frees.
aws.msk.go.memstats.gc.cpu.fraction
(gauge)
The fraction of this program's available CPU time used by the GC since the program started.
aws.msk.go.memstats.gc.sys.bytes
(gauge)
Number of bytes used for garbage collection system metadata.
Shown as byte
aws.msk.go.memstats.heap.alloc.bytes
(gauge)
Number of heap bytes allocated and still in use.
Shown as byte
aws.msk.go.memstats.heap.idle.bytes
(gauge)
Number of heap bytes waiting to be used.
Shown as byte
aws.msk.go.memstats.heap.inuse.bytes
(gauge)
Number of heap bytes that are in use.
Shown as byte
aws.msk.go.memstats.heap.objects
(gauge)
Number of allocated objects.
aws.msk.go.memstats.heap.released.bytes
(gauge)
Number of heap bytes released to OS.
Shown as byte
aws.msk.go.memstats.heap.sys.bytes
(gauge)
Number of heap bytes obtained from system.
Shown as byte
aws.msk.go.memstats.last.gc.time.seconds
(gauge)
Number of seconds since 1970 of last garbage collection.
Shown as second
aws.msk.go.memstats.lookups.total
(count)
Total number of pointer lookups.
aws.msk.go.memstats.mallocs.total
(count)
Total number of mallocs.
aws.msk.go.memstats.mcache.inuse.bytes
(gauge)
Number of bytes in use by mcache structures.
Shown as byte
aws.msk.go.memstats.mcache.sys.bytes
(gauge)
Number of bytes used for mcache structures obtained from system.
Shown as byte
aws.msk.go.memstats.mspan.inuse.bytes
(gauge)
Number of bytes in use by mspan structures.
Shown as byte
aws.msk.go.memstats.mspan.sys.bytes
(gauge)
Number of bytes used for mspan structures obtained from system.
Shown as byte
aws.msk.go.memstats.next.gc.bytes
(gauge)
Number of heap bytes when next garbage collection will take place.
Shown as byte
aws.msk.go.memstats.other.sys.bytes
(gauge)
Number of bytes used for other system allocations.
Shown as byte
aws.msk.go.memstats.stack.inuse.bytes
(gauge)
Number of bytes in use by the stack allocator.
Shown as byte
aws.msk.go.memstats.stack.sys.bytes
(gauge)
Number of bytes obtained from system for stack allocator.
Shown as byte
aws.msk.go.memstats.sys.bytes
(gauge)
Number of bytes obtained from system.
Shown as byte
aws.msk.go.threads
(gauge)
Number of OS threads created.
Shown as thread
aws.msk.jmx.config.reload.failure.total
(count)
Number of times configuration have failed to be reloaded.
aws.msk.jmx.config.reload.success.total
(count)
Number of times configuration have successfully been reloaded.
aws.msk.jmx.exporter.build.info
(gauge)
A metric with a constant '1' value labeled with the version of the JMX exporter.
aws.msk.jmx.scrape.duration.seconds
(gauge)
Time this JMX scrape took, in seconds.
Shown as second
aws.msk.jmx.scrape.error
(gauge)
Non-zero if this scrape failed.
aws.msk.kafka.cluster.Partition.Value
(gauge)
Attribute exposed for management (kafka.cluster<>Value)
aws.msk.kafka.controller.ControllerChannelManager.50thPercentile
(gauge)
Attribute exposed for management (kafka.controller<>50thPercentile)
aws.msk.kafka.controller.ControllerChannelManager.75thPercentile
(gauge)
Attribute exposed for management (kafka.controller<>75thPercentile)
aws.msk.kafka.controller.ControllerChannelManager.95thPercentile
(gauge)
Attribute exposed for management (kafka.controller<>95thPercentile)
aws.msk.kafka.controller.ControllerChannelManager.98thPercentile
(gauge)
Attribute exposed for management (kafka.controller<>98thPercentile)
aws.msk.kafka.controller.ControllerChannelManager.999thPercentile
(gauge)
Attribute exposed for management (kafka.controller<>999thPercentile)
aws.msk.kafka.controller.ControllerChannelManager.99thPercentile
(gauge)
Attribute exposed for management (kafka.controller<>99thPercentile)
aws.msk.kafka.controller.ControllerChannelManager.Count
(gauge)
Attribute exposed for management (kafka.controller<>Count)
aws.msk.kafka.controller.ControllerChannelManager.FifteenMinuteRate
(gauge)
Attribute exposed for management (kafka.controller<>FifteenMinuteRate)
aws.msk.kafka.controller.ControllerChannelManager.FiveMinuteRate
(gauge)
Attribute exposed for management (kafka.controller<>FiveMinuteRate)
aws.msk.kafka.controller.ControllerChannelManager.Max
(gauge)
Attribute exposed for management (kafka.controller<>Max)
aws.msk.kafka.controller.ControllerChannelManager.Mean
(gauge)
Attribute exposed for management (kafka.controller<>Mean)
aws.msk.kafka.controller.ControllerChannelManager.MeanRate
(gauge)
Attribute exposed for management (kafka.controller<>MeanRate)
aws.msk.kafka.controller.ControllerChannelManager.Min
(gauge)
Attribute exposed for management (kafka.controller<>Min)
aws.msk.kafka.controller.ControllerChannelManager.OneMinuteRate
(gauge)
Attribute exposed for management (kafka.controller<>OneMinuteRate)
aws.msk.kafka.controller.ControllerChannelManager.StdDev
(gauge)
Attribute exposed for management (kafka.controller<>StdDev)
aws.msk.kafka.controller.ControllerChannelManager.Value
(gauge)
Attribute exposed for management (kafka.controller<>Value)
aws.msk.kafka.controller.ControllerEventManager.50thPercentile
(gauge)
Attribute exposed for management (kafka.controller<>50thPercentile)
aws.msk.kafka.controller.ControllerEventManager.75thPercentile
(gauge)
Attribute exposed for management (kafka.controller<>75thPercentile)
aws.msk.kafka.controller.ControllerEventManager.95thPercentile
(gauge)
Attribute exposed for management (kafka.controller<>95thPercentile)
aws.msk.kafka.controller.ControllerEventManager.98thPercentile
(gauge)
Attribute exposed for management (kafka.controller<>98thPercentile)
aws.msk.kafka.controller.ControllerEventManager.999thPercentile
(gauge)
Attribute exposed for management (kafka.controller<>999thPercentile)
aws.msk.kafka.controller.ControllerEventManager.99thPercentile
(gauge)
Attribute exposed for management (kafka.controller<>99thPercentile)
aws.msk.kafka.controller.ControllerEventManager.Count
(gauge)
Attribute exposed for management (kafka.controller<>Count)
aws.msk.kafka.controller.ControllerEventManager.Max
(gauge)
Attribute exposed for management (kafka.controller<>Max)
aws.msk.kafka.controller.ControllerEventManager.Mean
(gauge)
Attribute exposed for management (kafka.controller<>Mean)
aws.msk.kafka.controller.ControllerEventManager.Min
(gauge)
Attribute exposed for management (kafka.controller<>Min)
aws.msk.kafka.controller.ControllerEventManager.StdDev
(gauge)
Attribute exposed for management (kafka.controller<>StdDev)
aws.msk.kafka.controller.ControllerEventManager.Value
(gauge)
Attribute exposed for management (kafka.controller<>Value)
aws.msk.kafka.controller.ControllerStats.50thPercentile
(gauge)
Attribute exposed for management (kafka.controller<>50thPercentile)
aws.msk.kafka.controller.ControllerStats.75thPercentile
(gauge)
Attribute exposed for management (kafka.controller<>75thPercentile)
aws.msk.kafka.controller.ControllerStats.95thPercentile
(gauge)
Attribute exposed for management (kafka.controller<>95thPercentile)
aws.msk.kafka.controller.ControllerStats.98thPercentile
(gauge)
Attribute exposed for management (kafka.controller<>98thPercentile)
aws.msk.kafka.controller.ControllerStats.999thPercentile
(gauge)
Attribute exposed for management (kafka.controller<>999thPercentile)
aws.msk.kafka.controller.ControllerStats.99thPercentile
(gauge)
Attribute exposed for management (kafka.controller<>99thPercentile)
aws.msk.kafka.controller.ControllerStats.Count
(gauge)
Attribute exposed for management (kafka.controller<>Count)
aws.msk.kafka.controller.ControllerStats.FifteenMinuteRate
(gauge)
Attribute exposed for management (kafka.controller<>FifteenMinuteRate)
aws.msk.kafka.controller.ControllerStats.FiveMinuteRate
(gauge)
Attribute exposed for management (kafka.controller<>FiveMinuteRate)
aws.msk.kafka.controller.ControllerStats.Max
(gauge)
Attribute exposed for management (kafka.controller<>Max)
aws.msk.kafka.controller.ControllerStats.Mean
(gauge)
Attribute exposed for management (kafka.controller<>Mean)
aws.msk.kafka.controller.ControllerStats.MeanRate
(gauge)
Attribute exposed for management (kafka.controller<>MeanRate)
aws.msk.kafka.controller.ControllerStats.Min
(gauge)
Attribute exposed for management (kafka.controller<>Min)
aws.msk.kafka.controller.ControllerStats.OneMinuteRate
(gauge)
Attribute exposed for management (kafka.controller<>OneMinuteRate)
aws.msk.kafka.controller.ControllerStats.StdDev
(gauge)
Attribute exposed for management (kafka.controller<>StdDev)
aws.msk.kafka.controller.KafkaController.Value
(gauge)
Attribute exposed for management (kafka.controller<>Value)
aws.msk.kafka.coordinator.group.GroupMetadataManager.Value
(gauge)
Attribute exposed for management (kafka.coordinator.group<>Value)
aws.msk.kafka.coordinator.transaction.TransactionMarkerChannelManager.Value
(gauge)
Attribute exposed for management (kafka.coordinator.transaction<>Value)
aws.msk.kafka.log.Log.Value
(gauge)
Attribute exposed for management (kafka.log<>Value)
aws.msk.kafka.log.LogCleaner.Value
(gauge)
Attribute exposed for management (kafka.log<>Value)
aws.msk.kafka.log.LogCleanerManager.Value
(gauge)
Attribute exposed for management (kafka.log<>Value)
aws.msk.kafka.log.LogManager.Value
(gauge)
Attribute exposed for management (kafka.log<>Value)
aws.msk.kafka.network.Acceptor.Count
(gauge)
Attribute exposed for management (kafka.network<>Count)
aws.msk.kafka.network.Acceptor.FifteenMinuteRate
(gauge)
Attribute exposed for management (kafka.network<>FifteenMinuteRate)
aws.msk.kafka.network.Acceptor.FiveMinuteRate
(gauge)
Attribute exposed for management (kafka.network<>FiveMinuteRate)
aws.msk.kafka.network.Acceptor.MeanRate
(gauge)
Attribute exposed for management (kafka.network<>MeanRate)
aws.msk.kafka.network.Acceptor.OneMinuteRate
(gauge)
Attribute exposed for management (kafka.network<>OneMinuteRate)
aws.msk.kafka.network.Processor.Value
(gauge)
Attribute exposed for management (kafka.network<>Value)
aws.msk.kafka.network.RequestChannel.Value
(gauge)
Attribute exposed for management (kafka.network<>Value)
aws.msk.kafka.network.RequestMetrics.50thPercentile
(gauge)
Attribute exposed for management (kafka.network<>50thPercentile)
aws.msk.kafka.network.RequestMetrics.75thPercentile
(gauge)
Attribute exposed for management (kafka.network<>75thPercentile)
aws.msk.kafka.network.RequestMetrics.95thPercentile
(gauge)
Attribute exposed for management (kafka.network<>95thPercentile)
aws.msk.kafka.network.RequestMetrics.98thPercentile
(gauge)
Attribute exposed for management (kafka.network<>98thPercentile)
aws.msk.kafka.network.RequestMetrics.999thPercentile
(gauge)
Attribute exposed for management (kafka.network<>999thPercentile)
aws.msk.kafka.network.RequestMetrics.99thPercentile
(gauge)
Attribute exposed for management (kafka.network<>99thPercentile)
aws.msk.kafka.network.RequestMetrics.Count
(gauge)
Attribute exposed for management (kafka.network<>Count)
aws.msk.kafka.network.RequestMetrics.FifteenMinuteRate
(gauge)
Attribute exposed for management (kafka.network<>FifteenMinuteRate)
aws.msk.kafka.network.RequestMetrics.FiveMinuteRate
(gauge)
Attribute exposed for management (kafka.network<>FiveMinuteRate)
aws.msk.kafka.network.RequestMetrics.Max
(gauge)
Attribute exposed for management (kafka.network<>Max)
aws.msk.kafka.network.RequestMetrics.Mean
(gauge)
Attribute exposed for management (kafka.network<>Mean)
aws.msk.kafka.network.RequestMetrics.MeanRate
(gauge)
Attribute exposed for management (kafka.network<>MeanRate)
aws.msk.kafka.network.RequestMetrics.Min
(gauge)
Attribute exposed for management (kafka.network<>Min)
aws.msk.kafka.network.RequestMetrics.OneMinuteRate
(gauge)
Attribute exposed for management (kafka.network<>OneMinuteRate)
aws.msk.kafka.network.RequestMetrics.StdDev
(gauge)
Attribute exposed for management (kafka.network<>StdDev)
aws.msk.kafka.network.SocketServer.Value
(gauge)
Attribute exposed for management (kafka.network<>Value)
aws.msk.kafka.security.SimpleAclAuthorizer.Count
(gauge)
Attribute exposed for management (kafka.security<>Count)
aws.msk.kafka.security.SimpleAclAuthorizer.FifteenMinuteRate
(gauge)
Attribute exposed for management (kafka.security<>FifteenMinuteRate)
aws.msk.kafka.security.SimpleAclAuthorizer.FiveMinuteRate
(gauge)
Attribute exposed for management (kafka.security<>FiveMinuteRate)
aws.msk.kafka.security.SimpleAclAuthorizer.MeanRate
(gauge)
Attribute exposed for management (kafka.security<>MeanRate)
aws.msk.kafka.security.SimpleAclAuthorizer.OneMinuteRate
(gauge)
Attribute exposed for management (kafka.security<>OneMinuteRate)
aws.msk.kafka.server.BrokerTopicMetrics.Count
(gauge)
Attribute exposed for management (kafka.server<>Count)
aws.msk.kafka.server.BrokerTopicMetrics.FifteenMinuteRate
(gauge)
Attribute exposed for management (kafka.server<>FifteenMinuteRate)
aws.msk.kafka.server.BrokerTopicMetrics.FiveMinuteRate
(gauge)
Attribute exposed for management (kafka.server<>FiveMinuteRate)
aws.msk.kafka.server.BrokerTopicMetrics.MeanRate
(gauge)
Attribute exposed for management (kafka.server<>MeanRate)
aws.msk.kafka.server.BrokerTopicMetrics.OneMinuteRate
(gauge)
Attribute exposed for management (kafka.server<>OneMinuteRate)
aws.msk.kafka.server.DelayedFetchMetrics.Count
(gauge)
Attribute exposed for management (kafka.server<>Count)
aws.msk.kafka.server.DelayedFetchMetrics.FifteenMinuteRate
(gauge)
Attribute exposed for management (kafka.server<>FifteenMinuteRate)
aws.msk.kafka.server.DelayedFetchMetrics.FiveMinuteRate
(gauge)
Attribute exposed for management (kafka.server<>FiveMinuteRate)
aws.msk.kafka.server.DelayedFetchMetrics.MeanRate
(gauge)
Attribute exposed for management (kafka.server<>MeanRate)
aws.msk.kafka.server.DelayedFetchMetrics.OneMinuteRate
(gauge)
Attribute exposed for management (kafka.server<>OneMinuteRate)
aws.msk.kafka.server.DelayedOperationPurgatory.Value
(gauge)
Attribute exposed for management (kafka.server<>Value)
aws.msk.kafka.server.Fetch.queue.size
(gauge)
Tracks the size of the delay queue (kafka.server<>queue-size)
aws.msk.kafka.server.FetchSessionCache.Count
(gauge)
Attribute exposed for management (kafka.server<>Count)
aws.msk.kafka.server.FetchSessionCache.FifteenMinuteRate
(gauge)
Attribute exposed for management (kafka.server<>FifteenMinuteRate)
aws.msk.kafka.server.FetchSessionCache.FiveMinuteRate
(gauge)
Attribute exposed for management (kafka.server<>FiveMinuteRate)
aws.msk.kafka.server.FetchSessionCache.MeanRate
(gauge)
Attribute exposed for management (kafka.server<>MeanRate)
aws.msk.kafka.server.FetchSessionCache.OneMinuteRate
(gauge)
Attribute exposed for management (kafka.server<>OneMinuteRate)
aws.msk.kafka.server.FetchSessionCache.Value
(gauge)
Attribute exposed for management (kafka.server<>Value)
aws.msk.kafka.server.FetcherLagMetrics.Value
(gauge)
Attribute exposed for management (kafka.server<>Value)
aws.msk.kafka.server.FetcherStats.Count
(gauge)
Attribute exposed for management (kafka.server<>Count)
aws.msk.kafka.server.FetcherStats.FifteenMinuteRate
(gauge)
Attribute exposed for management (kafka.server<>FifteenMinuteRate)
aws.msk.kafka.server.FetcherStats.FiveMinuteRate
(gauge)
Attribute exposed for management (kafka.server<>FiveMinuteRate)
aws.msk.kafka.server.FetcherStats.MeanRate
(gauge)
Attribute exposed for management (kafka.server<>MeanRate)
aws.msk.kafka.server.FetcherStats.OneMinuteRate
(gauge)
Attribute exposed for management (kafka.server<>OneMinuteRate)
aws.msk.kafka.server.KafkaRequestHandlerPool.Count
(gauge)
Attribute exposed for management (kafka.server<>Count)
aws.msk.kafka.server.KafkaRequestHandlerPool.FifteenMinuteRate
(gauge)
Attribute exposed for management (kafka.server<>FifteenMinuteRate)
aws.msk.kafka.server.KafkaRequestHandlerPool.FiveMinuteRate
(gauge)
Attribute exposed for management (kafka.server<>FiveMinuteRate)
aws.msk.kafka.server.KafkaRequestHandlerPool.MeanRate
(gauge)
Attribute exposed for management (kafka.server<>MeanRate)
aws.msk.kafka.server.KafkaRequestHandlerPool.OneMinuteRate
(gauge)
Attribute exposed for management (kafka.server<>OneMinuteRate)
aws.msk.kafka.server.KafkaServer.Value
(gauge)
Attribute exposed for management (kafka.server<>Value)
aws.msk.kafka.server.LeaderReplication.byte.rate
(gauge)
Tracking byte-rate for LeaderReplication (kafka.server<>byte-rate)
aws.msk.kafka.server.Produce.queue.size
(gauge)
Tracks the size of the delay queue (kafka.server<>queue-size)
aws.msk.kafka.server.ReplicaAlterLogDirsManager.Value
(gauge)
Attribute exposed for management (kafka.server<>Value)
aws.msk.kafka.server.ReplicaFetcherManager.Value
(gauge)
Attribute exposed for management (kafka.server<>Value)
aws.msk.kafka.server.ReplicaManager.Count
(gauge)
Attribute exposed for management (kafka.server<>Count)
aws.msk.kafka.server.ReplicaManager.FifteenMinuteRate
(gauge)
Attribute exposed for management (kafka.server<>FifteenMinuteRate)
aws.msk.kafka.server.ReplicaManager.FiveMinuteRate
(gauge)
Attribute exposed for management (kafka.server<>FiveMinuteRate)
aws.msk.kafka.server.ReplicaManager.MeanRate
(gauge)
Attribute exposed for management (kafka.server<>MeanRate)
aws.msk.kafka.server.ReplicaManager.OneMinuteRate
(gauge)
Attribute exposed for management (kafka.server<>OneMinuteRate)
aws.msk.kafka.server.ReplicaManager.Value
(gauge)
Attribute exposed for management (kafka.server<>Value)
aws.msk.kafka.server.Request.queue.size
(gauge)
Tracks the size of the delay queue (kafka.server<>queue-size)
aws.msk.kafka.server.SessionExpireListener.Count
(gauge)
Attribute exposed for management (kafka.server<>Count)
aws.msk.kafka.server.SessionExpireListener.FifteenMinuteRate
(gauge)
Attribute exposed for management (kafka.server<>FifteenMinuteRate)
aws.msk.kafka.server.SessionExpireListener.FiveMinuteRate
(gauge)
Attribute exposed for management (kafka.server<>FiveMinuteRate)
aws.msk.kafka.server.SessionExpireListener.MeanRate
(gauge)
Attribute exposed for management (kafka.server<>MeanRate)
aws.msk.kafka.server.SessionExpireListener.OneMinuteRate
(gauge)
Attribute exposed for management (kafka.server<>OneMinuteRate)
aws.msk.kafka.server.ZooKeeperClientMetrics.50thPercentile
(gauge)
Attribute exposed for management (kafka.server<>50thPercentile)
aws.msk.kafka.server.ZooKeeperClientMetrics.75thPercentile
(gauge)
Attribute exposed for management (kafka.server<>75thPercentile)
aws.msk.kafka.server.ZooKeeperClientMetrics.95thPercentile
(gauge)
Attribute exposed for management (kafka.server<>95thPercentile)
aws.msk.kafka.server.ZooKeeperClientMetrics.98thPercentile
(gauge)
Attribute exposed for management (kafka.server<>98thPercentile)
aws.msk.kafka.server.ZooKeeperClientMetrics.999thPercentile
(gauge)
Attribute exposed for management (kafka.server<>999thPercentile)
aws.msk.kafka.server.ZooKeeperClientMetrics.99thPercentile
(gauge)
Attribute exposed for management (kafka.server<>99thPercentile)
aws.msk.kafka.server.ZooKeeperClientMetrics.Count
(gauge)
Attribute exposed for management (kafka.server<>Count)
aws.msk.kafka.server.ZooKeeperClientMetrics.Max
(gauge)
Attribute exposed for management (kafka.server<>Max)
aws.msk.kafka.server.ZooKeeperClientMetrics.Mean
(gauge)
Attribute exposed for management (kafka.server<>Mean)
aws.msk.kafka.server.ZooKeeperClientMetrics.Min
(gauge)
Attribute exposed for management (kafka.server<>Min)
aws.msk.kafka.server.ZooKeeperClientMetrics.StdDev
(gauge)
Attribute exposed for management (kafka.server<>StdDev)
aws.msk.kafka.server.controller.channel.metrics.connection.close.rate
(gauge)
The number of connections closed per second (kafka.server<>connection-close-rate)
Shown as connection
aws.msk.kafka.server.controller.channel.metrics.connection.close.total
(gauge)
The total number of connections closed (kafka.server<>connection-close-total)
Shown as connection
aws.msk.kafka.server.controller.channel.metrics.connection.count
(gauge)
The current number of active connections. (kafka.server<>connection-count)
Shown as connection
aws.msk.kafka.server.controller.channel.metrics.connection.creation.rate
(gauge)
The number of new connections established per second (kafka.server<>connection-creation-rate)
aws.msk.kafka.server.controller.channel.metrics.connection.creation.total
(gauge)
The total number of new connections established (kafka.server<>connection-creation-total)
aws.msk.kafka.server.controller.channel.metrics.failed.authentication.rate
(gauge)
The number of connections with failed authentication per second (kafka.server<>failed-authentication-rate)
aws.msk.kafka.server.controller.channel.metrics.failed.authentication.total
(gauge)
The total number of connections with failed authentication (kafka.server<>failed-authentication-total)
aws.msk.kafka.server.controller.channel.metrics.failed.reauthentication.rate
(gauge)
The number of failed re-authentication of connections per second (kafka.server<>failed-reauthentication-rate)
aws.msk.kafka.server.controller.channel.metrics.failed.reauthentication.total
(gauge)
The total number of failed re-authentication of connections (kafka.server<>failed-reauthentication-total)
aws.msk.kafka.server.controller.channel.metrics.incoming.byte.rate
(gauge)
The number of bytes read off all sockets per second (kafka.server<>incoming-byte-rate)
Shown as byte
aws.msk.kafka.server.controller.channel.metrics.incoming.byte.total
(gauge)
The total number of bytes read off all sockets (kafka.server<>incoming-byte-total)
Shown as byte
aws.msk.kafka.server.controller.channel.metrics.io.ratio
(gauge)
The fraction of time the I/O thread spent doing I/O (kafka.server<>io-ratio)
aws.msk.kafka.server.controller.channel.metrics.io.time.ns.avg
(gauge)
The average length of time for I/O per select call in nanoseconds. (kafka.server<>io-time-ns-avg)
aws.msk.kafka.server.controller.channel.metrics.io.wait.ratio
(gauge)
The fraction of time the I/O thread spent waiting (kafka.server<>io-wait-ratio)
aws.msk.kafka.server.controller.channel.metrics.io.wait.time.ns.avg
(gauge)
The average length of time the I/O thread spent waiting for a socket ready for reads or writes in nanoseconds. (kafka.server<>io-wait-time-ns-avg)
aws.msk.kafka.server.controller.channel.metrics.io.waittime.total
(gauge)
The total time the I/O thread spent waiting (kafka.server<>io-waittime-total)
aws.msk.kafka.server.controller.channel.metrics.iotime.total
(gauge)
The total time the I/O thread spent doing I/O (kafka.server<>iotime-total)
aws.msk.kafka.server.controller.channel.metrics.network.io.rate
(gauge)
The number of network operations (reads or writes) on all connections per second (kafka.server<>network-io-rate)
aws.msk.kafka.server.controller.channel.metrics.network.io.total
(gauge)
The total number of network operations (reads or writes) on all connections (kafka.server<>network-io-total)
aws.msk.kafka.server.controller.channel.metrics.outgoing.byte.rate
(gauge)
The number of outgoing bytes sent to all servers per second (kafka.server<>outgoing-byte-rate)
Shown as byte
aws.msk.kafka.server.controller.channel.metrics.outgoing.byte.total
(gauge)
The total number of outgoing bytes sent to all servers (kafka.server<>outgoing-byte-total)
Shown as byte
aws.msk.kafka.server.controller.channel.metrics.reauthentication.latency.avg
(gauge)
The average latency observed due to re-authentication (kafka.server<>reauthentication-latency-avg)
aws.msk.kafka.server.controller.channel.metrics.reauthentication.latency.max
(gauge)
The max latency observed due to re-authentication (kafka.server<>reauthentication-latency-max)
aws.msk.kafka.server.controller.channel.metrics.request.rate
(gauge)
The number of requests sent per second (kafka.server<>request-rate)
aws.msk.kafka.server.controller.channel.metrics.request.size.avg
(gauge)
The average size of requests sent. (kafka.server<>request-size-avg)
aws.msk.kafka.server.controller.channel.metrics.request.size.max
(gauge)
The maximum size of any request sent. (kafka.server<>request-size-max)
aws.msk.kafka.server.controller.channel.metrics.request.total
(gauge)
The total number of requests sent (kafka.server<>request-total)
aws.msk.kafka.server.controller.channel.metrics.response.rate
(gauge)
The number of responses received per second (kafka.server<>response-rate)
aws.msk.kafka.server.controller.channel.metrics.response.total
(gauge)
The total number of responses received (kafka.server<>response-total)
aws.msk.kafka.server.controller.channel.metrics.select.rate
(gauge)
The number of times the I/O layer checked for new I/O to perform per second (kafka.server<>select-rate)
aws.msk.kafka.server.controller.channel.metrics.select.total
(gauge)
The total number of times the I/O layer checked for new I/O to perform (kafka.server<>select-total)
aws.msk.kafka.server.controller.channel.metrics.successful.authentication.no.reauth.total
(gauge)
The total number of connections with successful authentication where the client does not support re-authentication (kafka.server<>successful-authentication-no-reauth-total)
aws.msk.kafka.server.controller.channel.metrics.successful.authentication.rate
(gauge)
The number of connections with successful authentication per second (kafka.server<>successful-authentication-rate)
aws.msk.kafka.server.controller.channel.metrics.successful.authentication.total
(gauge)
The total number of connections with successful authentication (kafka.server<>successful-authentication-total)
aws.msk.kafka.server.controller.channel.metrics.successful.reauthentication.rate
(gauge)
The number of successful re-authentication of connections per second (kafka.server<>successful-reauthentication-rate)
aws.msk.kafka.server.controller.channel.metrics.successful.reauthentication.total
(gauge)
The total number of successful re-authentication of connections (kafka.server<>successful-reauthentication-total)
aws.msk.kafka.server.kafka.metrics.count.count
(gauge)
total number of registered metrics (kafka.server<>count)
aws.msk.kafka.server.replica.fetcher.metrics.connection.close.rate
(gauge)
The number of connections closed per second (kafka.server<>connection-close-rate)
aws.msk.kafka.server.replica.fetcher.metrics.connection.close.total
(gauge)
The total number of connections closed (kafka.server<>connection-close-total)
aws.msk.kafka.server.replica.fetcher.metrics.connection.count
(gauge)
The current number of active connections. (kafka.server<>connection-count)
aws.msk.kafka.server.replica.fetcher.metrics.connection.creation.rate
(gauge)
The number of new connections established per second (kafka.server<>connection-creation-rate)
aws.msk.kafka.server.replica.fetcher.metrics.connection.creation.total
(gauge)
The total number of new connections established (kafka.server<>connection-creation-total)
aws.msk.kafka.server.replica.fetcher.metrics.failed.authentication.rate
(gauge)
The number of connections with failed authentication per second (kafka.server<>failed-authentication-rate)
aws.msk.kafka.server.replica.fetcher.metrics.failed.authentication.total
(gauge)
The total number of connections with failed authentication (kafka.server<>failed-authentication-total)
aws.msk.kafka.server.replica.fetcher.metrics.incoming.byte.rate
(gauge)
The number of bytes read off all sockets per second (kafka.server<>incoming-byte-rate)
Shown as byte
aws.msk.kafka.server.replica.fetcher.metrics.incoming.byte.total
(gauge)
The total number of bytes read off all sockets (kafka.server<>incoming-byte-total)
Shown as byte
aws.msk.kafka.server.replica.fetcher.metrics.io.ratio
(gauge)
The fraction of time the I/O thread spent doing I/O (kafka.server<>io-ratio)
aws.msk.kafka.server.replica.fetcher.metrics.io.time.ns.avg
(gauge)
The average length of time for I/O per select call in nanoseconds. (kafka.server<>io-time-ns-avg)
aws.msk.kafka.server.replica.fetcher.metrics.io.wait.ratio
(gauge)
The fraction of time the I/O thread spent waiting (kafka.server<>io-wait-ratio)
aws.msk.kafka.server.replica.fetcher.metrics.io.wait.time.ns.avg
(gauge)
The average length of time the I/O thread spent waiting for a socket ready for reads or writes in nanoseconds. (kafka.server<>io-wait-time-ns-avg)
aws.msk.kafka.server.replica.fetcher.metrics.io.waittime.total
(gauge)
The total time the I/O thread spent waiting (kafka.server<>io-waittime-total)
aws.msk.kafka.server.replica.fetcher.metrics.iotime.total
(gauge)
The total time the I/O thread spent doing I/O (kafka.server<>iotime-total)
aws.msk.kafka.server.replica.fetcher.metrics.network.io.rate
(gauge)
The number of network operations (reads or writes) on all connections per second (kafka.server<>network-io-rate)
aws.msk.kafka.server.replica.fetcher.metrics.network.io.total
(gauge)
The total number of network operations (reads or writes) on all connections (kafka.server<>network-io-total)
aws.msk.kafka.server.replica.fetcher.metrics.outgoing.byte.rate
(gauge)
The number of outgoing bytes sent to all servers per second (kafka.server<>outgoing-byte-rate)
Shown as byte
aws.msk.kafka.server.replica.fetcher.metrics.outgoing.byte.total
(gauge)
The total number of outgoing bytes sent to all servers (kafka.server<>outgoing-byte-total)
Shown as byte
aws.msk.kafka.server.replica.fetcher.metrics.request.rate
(gauge)
The number of requests sent per second (kafka.server<>request-rate)
aws.msk.kafka.server.replica.fetcher.metrics.request.size.avg
(gauge)
The average size of requests sent. (kafka.server<>request-size-avg)
aws.msk.kafka.server.replica.fetcher.metrics.request.size.max
(gauge)
The maximum size of any request sent. (kafka.server<>request-size-max)
aws.msk.kafka.server.replica.fetcher.metrics.request.total
(gauge)
The total number of requests sent (kafka.server<>request-total)
aws.msk.kafka.server.replica.fetcher.metrics.response.rate
(gauge)
The number of responses received per second (kafka.server<>response-rate)
aws.msk.kafka.server.replica.fetcher.metrics.response.total
(gauge)
The total number of responses received (kafka.server<>response-total)
aws.msk.kafka.server.replica.fetcher.metrics.select.rate
(gauge)
The number of times the I/O layer checked for new I/O to perform per second (kafka.server<>select-rate)
aws.msk.kafka.server.replica.fetcher.metrics.select.total
(gauge)
The total number of times the I/O layer checked for new I/O to perform (kafka.server<>select-total)
aws.msk.kafka.server.replica.fetcher.metrics.successful.authentication.rate
(gauge)
The number of connections with successful authentication per second (kafka.server<>successful-authentication-rate)
aws.msk.kafka.server.replica.fetcher.metrics.successful.authentication.total
(gauge)
The total number of connections with successful authentication (kafka.server<>successful-authentication-total)
aws.msk.kafka.server.socket.server.metrics.MemoryPoolAvgDepletedPercent
(gauge)
(kafka.server<>MemoryPoolAvgDepletedPercent)
aws.msk.kafka.server.socket.server.metrics.MemoryPoolDepletedTimeTotal
(gauge)
(kafka.server<>MemoryPoolDepletedTimeTotal)
aws.msk.kafka.server.socket.server.metrics.connection.close.rate
(gauge)
The number of connections closed per second (kafka.server<>connection-close-rate)
aws.msk.kafka.server.socket.server.metrics.connection.close.total
(gauge)
The total number of connections closed (kafka.server<>connection-close-total)
aws.msk.kafka.server.socket.server.metrics.connection.count
(gauge)
The current number of active connections. (kafka.server<>connection-count)
aws.msk.kafka.server.socket.server.metrics.connection.creation.rate
(gauge)
The number of new connections established per second (kafka.server<>connection-creation-rate)
aws.msk.kafka.server.socket.server.metrics.connection.creation.total
(gauge)
The total number of new connections established (kafka.server<>connection-creation-total)
aws.msk.kafka.server.socket.server.metrics.expired.connections.killed.count
(gauge)
(kafka.server<>expired-connections-killed-count)
aws.msk.kafka.server.socket.server.metrics.failed.authentication.rate
(gauge)
The number of connections with failed authentication per second (kafka.server<>failed-authentication-rate)
aws.msk.kafka.server.socket.server.metrics.failed.authentication.total
(gauge)
The total number of connections with failed authentication (kafka.server<>failed-authentication-total)
aws.msk.kafka.server.socket.server.metrics.failed.reauthentication.rate
(gauge)
The number of failed re-authentication of connections per second (kafka.server<>failed-reauthentication-rate)
aws.msk.kafka.server.socket.server.metrics.failed.reauthentication.total
(gauge)
The total number of failed re-authentication of connections (kafka.server<>failed-reauthentication-total)
aws.msk.kafka.server.socket.server.metrics.incoming.byte.rate
(gauge)
The number of bytes read off all sockets per second (kafka.server<>incoming-byte-rate)
aws.msk.kafka.server.socket.server.metrics.incoming.byte.total
(gauge)
The total number of bytes read off all sockets (kafka.server<>incoming-byte-total)
aws.msk.kafka.server.socket.server.metrics.io.ratio
(gauge)
The fraction of time the I/O thread spent doing I/O (kafka.server<>io-ratio)
aws.msk.kafka.server.socket.server.metrics.io.time.ns.avg
(gauge)
The average length of time for I/O per select call in nanoseconds. (kafka.server<>io-time-ns-avg)
aws.msk.kafka.server.socket.server.metrics.io.wait.ratio
(gauge)
The fraction of time the I/O thread spent waiting (kafka.server<>io-wait-ratio)
aws.msk.kafka.server.socket.server.metrics.io.wait.time.ns.avg
(gauge)
The average length of time the I/O thread spent waiting for a socket ready for reads or writes in nanoseconds. (kafka.server<>io-wait-time-ns-avg)
aws.msk.kafka.server.socket.server.metrics.io.waittime.total
(gauge)
The total time the I/O thread spent waiting (kafka.server<>io-waittime-total)
aws.msk.kafka.server.socket.server.metrics.iotime.total
(gauge)
The total time the I/O thread spent doing I/O (kafka.server<>iotime-total)
aws.msk.kafka.server.socket.server.metrics.network.io.rate
(gauge)
The number of network operations (reads or writes) on all connections per second (kafka.server<>network-io-rate)
aws.msk.kafka.server.socket.server.metrics.network.io.total
(gauge)
The total number of network operations (reads or writes) on all connections (kafka.server<>network-io-total)
aws.msk.kafka.server.socket.server.metrics.outgoing.byte.rate
(gauge)
The number of outgoing bytes sent to all servers per second (kafka.server<>outgoing-byte-rate)
aws.msk.kafka.server.socket.server.metrics.outgoing.byte.total
(gauge)
The total number of outgoing bytes sent to all servers (kafka.server<>outgoing-byte-total)
aws.msk.kafka.server.socket.server.metrics.reauthentication.latency.avg
(gauge)
The average latency observed due to re-authentication (kafka.server<>reauthentication-latency-avg)
aws.msk.kafka.server.socket.server.metrics.reauthentication.latency.max
(gauge)
The max latency observed due to re-authentication (kafka.server<>reauthentication-latency-max)
aws.msk.kafka.server.socket.server.metrics.request.rate
(gauge)
The number of requests sent per second (kafka.server<>request-rate)
aws.msk.kafka.server.socket.server.metrics.request.size.avg
(gauge)
The average size of requests sent. (kafka.server<>request-size-avg)
aws.msk.kafka.server.socket.server.metrics.request.size.max
(gauge)
The maximum size of any request sent. (kafka.server<>request-size-max)
aws.msk.kafka.server.socket.server.metrics.request.total
(gauge)
The total number of requests sent (kafka.server<>request-total)
aws.msk.kafka.server.socket.server.metrics.response.rate
(gauge)
The number of responses received per second (kafka.server<>response-rate)
aws.msk.kafka.server.socket.server.metrics.response.total
(gauge)
The total number of responses received (kafka.server<>response-total)
aws.msk.kafka.server.socket.server.metrics.select.rate
(gauge)
The number of times the I/O layer checked for new I/O to perform per second (kafka.server<>select-rate)
aws.msk.kafka.server.socket.server.metrics.select.total
(gauge)
The total number of times the I/O layer checked for new I/O to perform (kafka.server<>select-total)
aws.msk.kafka.server.socket.server.metrics.successful.authentication.no.reauth.total
(gauge)
The total number of connections with successful authentication where the client does not support re-authentication (kafka.server<>successful-authentication-no-reauth-total)
aws.msk.kafka.server.socket.server.metrics.successful.authentication.rate
(gauge)
The number of connections with successful authentication per second (kafka.server<>successful-authentication-rate)
aws.msk.kafka.server.socket.server.metrics.successful.authentication.total
(gauge)
The total number of connections with successful authentication (kafka.server<>successful-authentication-total)
aws.msk.kafka.server.socket.server.metrics.successful.reauthentication.rate
(gauge)
The number of successful re-authentication of connections per second (kafka.server<>successful-reauthentication-rate)
aws.msk.kafka.server.socket.server.metrics.successful.reauthentication.total
(gauge)
The total number of successful re-authentication of connections (kafka.server<>successful-reauthentication-total)
aws.msk.kafka.server.txn.marker.channel.metrics.connection.close.rate
(gauge)
The number of connections closed per second (kafka.server<>connection-close-rate)
aws.msk.kafka.server.txn.marker.channel.metrics.connection.close.total
(gauge)
The total number of connections closed (kafka.server<>connection-close-total)
aws.msk.kafka.server.txn.marker.channel.metrics.connection.count
(gauge)
The current number of active connections. (kafka.server<>connection-count)
aws.msk.kafka.server.txn.marker.channel.metrics.connection.creation.rate
(gauge)
The number of new connections established per second (kafka.server<>connection-creation-rate)
aws.msk.kafka.server.txn.marker.channel.metrics.connection.creation.total
(gauge)
The total number of new connections established (kafka.server<>connection-creation-total)
aws.msk.kafka.server.txn.marker.channel.metrics.failed.authentication.rate
(gauge)
The number of connections with failed authentication per second (kafka.server<>failed-authentication-rate)
aws.msk.kafka.server.txn.marker.channel.metrics.failed.authentication.total
(gauge)
The total number of connections with failed authentication (kafka.server<>failed-authentication-total)
aws.msk.kafka.server.txn.marker.channel.metrics.failed.reauthentication.rate
(gauge)
The number of failed re-authentication of connections per second (kafka.server<>failed-reauthentication-rate)
aws.msk.kafka.server.txn.marker.channel.metrics.failed.reauthentication.total
(gauge)
The total number of failed re-authentication of connections (kafka.server<>failed-reauthentication-total)
aws.msk.kafka.server.txn.marker.channel.metrics.incoming.byte.rate
(gauge)
The number of bytes read off all sockets per second (kafka.server<>incoming-byte-rate)
aws.msk.kafka.server.txn.marker.channel.metrics.incoming.byte.total
(gauge)
The total number of bytes read off all sockets (kafka.server<>incoming-byte-total)
aws.msk.kafka.server.txn.marker.channel.metrics.io.ratio
(gauge)
The fraction of time the I/O thread spent doing I/O (kafka.server<>io-ratio)
aws.msk.kafka.server.txn.marker.channel.metrics.io.time.ns.avg
(gauge)
The average length of time for I/O per select call in nanoseconds. (kafka.server<>io-time-ns-avg)
aws.msk.kafka.server.txn.marker.channel.metrics.io.wait.ratio
(gauge)
The fraction of time the I/O thread spent waiting (kafka.server<>io-wait-ratio)
aws.msk.kafka.server.txn.marker.channel.metrics.io.wait.time.ns.avg
(gauge)
The average length of time the I/O thread spent waiting for a socket ready for reads or writes in nanoseconds. (kafka.server<>io-wait-time-ns-avg)
aws.msk.kafka.server.txn.marker.channel.metrics.io.waittime.total
(gauge)
The total time the I/O thread spent waiting (kafka.server<>io-waittime-total)
aws.msk.kafka.server.txn.marker.channel.metrics.iotime.total
(gauge)
The total time the I/O thread spent doing I/O (kafka.server<>iotime-total)
aws.msk.kafka.server.txn.marker.channel.metrics.network.io.rate
(gauge)
The number of network operations (reads or writes) on all connections per second (kafka.server<>network-io-rate)
aws.msk.kafka.server.txn.marker.channel.metrics.network.io.total
(gauge)
The total number of network operations (reads or writes) on all connections (kafka.server<>network-io-total)
aws.msk.kafka.server.txn.marker.channel.metrics.outgoing.byte.rate
(gauge)
The number of outgoing bytes sent to all servers per second (kafka.server<>outgoing-byte-rate)
aws.msk.kafka.server.txn.marker.channel.metrics.outgoing.byte.total
(gauge)
The total number of outgoing bytes sent to all servers (kafka.server<>outgoing-byte-total)
aws.msk.kafka.server.txn.marker.channel.metrics.reauthentication.latency.avg
(gauge)
The average latency observed due to re-authentication (kafka.server<>reauthentication-latency-avg)
aws.msk.kafka.server.txn.marker.channel.metrics.reauthentication.latency.max
(gauge)
The max latency observed due to re-authentication (kafka.server<>reauthentication-latency-max)
aws.msk.kafka.server.txn.marker.channel.metrics.request.rate
(gauge)
The number of requests sent per second (kafka.server<>request-rate)
aws.msk.kafka.server.txn.marker.channel.metrics.request.size.avg
(gauge)
The average size of requests sent. (kafka.server<>request-size-avg)
aws.msk.kafka.server.txn.marker.channel.metrics.request.size.max
(gauge)
The maximum size of any request sent. (kafka.server<>request-size-max)
aws.msk.kafka.server.txn.marker.channel.metrics.request.total
(gauge)
The total number of requests sent (kafka.server<>request-total)
aws.msk.kafka.server.txn.marker.channel.metrics.response.rate
(gauge)
The number of responses received per second (kafka.server<>response-rate)
aws.msk.kafka.server.txn.marker.channel.metrics.response.total
(gauge)
The total number of responses received (kafka.server<>response-total)
aws.msk.kafka.server.txn.marker.channel.metrics.select.rate
(gauge)
The number of times the I/O layer checked for new I/O to perform per second (kafka.server<>select-rate)
aws.msk.kafka.server.txn.marker.channel.metrics.select.total
(gauge)
The total number of times the I/O layer checked for new I/O to perform (kafka.server<>select-total)
aws.msk.kafka.server.txn.marker.channel.metrics.successful.authentication.no.reauth.total
(gauge)
The total number of connections with successful authentication where the client does not support re-authentication (kafka.server<>successful-authentication-no-reauth-total)
aws.msk.kafka.server.txn.marker.channel.metrics.successful.authentication.rate
(gauge)
The number of connections with successful authentication per second (kafka.server<>successful-authentication-rate)
aws.msk.kafka.server.txn.marker.channel.metrics.successful.authentication.total
(gauge)
The total number of connections with successful authentication (kafka.server<>successful-authentication-total)
aws.msk.kafka.server.txn.marker.channel.metrics.successful.reauthentication.rate
(gauge)
The number of successful re-authentication of connections per second (kafka.server<>successful-reauthentication-rate)
aws.msk.kafka.server.txn.marker.channel.metrics.successful.reauthentication.total
(gauge)
The total number of successful re-authentication of connections (kafka.server<>successful-reauthentication-total)
aws.msk.kafka.utils.Throttler.Count
(gauge)
Attribute exposed for management (kafka.utils<>Count)
aws.msk.kafka.utils.Throttler.FifteenMinuteRate
(gauge)
Attribute exposed for management (kafka.utils<>FifteenMinuteRate)
aws.msk.kafka.utils.Throttler.FiveMinuteRate
(gauge)
Attribute exposed for management (kafka.utils<>FiveMinuteRate)
aws.msk.kafka.utils.Throttler.MeanRate
(gauge)
Attribute exposed for management (kafka.utils<>MeanRate)
aws.msk.kafka.utils.Throttler.OneMinuteRate
(gauge)
Attribute exposed for management (kafka.utils<>OneMinuteRate)
aws.msk.node.cpu.seconds.total
(count)
Seconds the cpus spent in each mode.
aws.msk.node.exporter.build.info
(gauge)
A metric with a constant '1' value labeled by version, revision, branch, and goversion from which node_exporter was built.
aws.msk.node.filesystem.avail.bytes
(gauge)
Filesystem space available to non-root users in bytes.
Shown as byte
aws.msk.node.filesystem.device.error
(gauge)
Whether an error occurred while getting statistics for the given device.
aws.msk.node.filesystem.files
(gauge)
Filesystem total file nodes.
aws.msk.node.filesystem.files.free
(gauge)
Filesystem total free file nodes.
aws.msk.node.filesystem.free.bytes
(gauge)
Filesystem free space in bytes.
Shown as byte
aws.msk.node.filesystem.readonly
(gauge)
Filesystem read-only status.
aws.msk.node.filesystem.size.bytes
(gauge)
Filesystem size in bytes.
Shown as byte
aws.msk.node.scrape.collector.duration.seconds
(gauge)
node_exporter: Duration of a collector scrape.
Shown as second
aws.msk.node.scrape.collector.success
(gauge)
node_exporter: Whether a collector succeeded.
aws.msk.process.cpu.seconds.total
(count)
Total user and system CPU time spent in seconds.
aws.msk.process.max.fds
(gauge)
Maximum number of open file descriptors.
aws.msk.process.open.fds
(gauge)
Number of open file descriptors.
aws.msk.process.resident.memory.bytes
(gauge)
Resident memory size in bytes.
Shown as byte
aws.msk.process.start.time.seconds
(gauge)
Start time of the process since unix epoch in seconds.
Shown as second
aws.msk.process.virtual.memory.bytes
(gauge)
Virtual memory size in bytes.
Shown as byte
aws.msk.process.virtual.memory.max.bytes
(gauge)
Maximum amount of virtual memory available in bytes.
Shown as byte
aws.msk.promhttp.metric.handler.requests.in.flight
(gauge)
Current number of scrapes being served.
aws.msk.promhttp.metric.handler.requests.total
(count)
Total number of scrapes by HTTP status code.

Service Checks

aws.msk.can_connect:
Returns CRITICAL if the Agent is unable to discover nodes of the MSK cluster. Otherwise, returns OK.

aws.msk.prometheus.health:
Returns CRITICAL if the check cannot access a metrics endpoint. Otherwise, returns OK.

Events

The Amazon MSK check does not include any events.

Crawler

Enable this integration to see MSK metrics from Cloudwatch in Datadog.

Setup

Installation

If you haven’t already, set up the Amazon Web Services integration first.

Metric collection

  1. In the AWS integration tile, ensure that MSK is checked under metric collection.

  2. Install the Datadog - Amazon MSK integration.

Log collection

Enable logging

Configure Amazon MSK to send logs either to a S3 bucket or to Cloudwatch.

Note: If you log to a S3 bucket, make sure that amazon_msk is set as Target prefix.

Send logs to Datadog

  1. If you haven’t already, set up the Datadog log collection AWS Lambda function.
  2. Once the lambda function is installed, manually add a trigger on the S3 bucket or Cloudwatch log group that contains your Amazon MSK logs in the AWS console:

Data Collected

aws.kafka.zookeeper_request_latency_ms_mean
(gauge)
Mean latency in milliseconds for ZooKeeper requests from broker.
aws.kafka.active_controller_count
(gauge)
Only one controller per cluster should be active at any given time.
aws.kafka.global_partition_count
(gauge)
Total number of partitions across all brokers in the cluster.
aws.kafka.global_topic_count
(gauge)
Total number of partitions across all brokers in the cluster.
aws.kafka.offline_partitions_count
(gauge)
Total number of partitions that are offline in the cluster.
aws.kafka.swap_used
(gauge)
The size in bytes of swap memory that is in use for the broker.
Shown as byte
aws.kafka.swap_free
(gauge)
The size in bytes of swap memory that is available for the broker.
Shown as byte
aws.kafka.memory_used
(gauge)
The size in bytes of memory that is in use for the broker.
Shown as byte
aws.kafka.memory_buffered
(gauge)
The size in bytes of buffered memory for the broker.
Shown as byte
aws.kafka.memory_free
(gauge)
The size in bytes of memory that is free and available for the broker.
Shown as byte
aws.kafka.memory_cached
(gauge)
The size in bytes of cached memory for the broker.
Shown as byte
aws.kafka.cpu_user
(gauge)
The percentage of CPU in user space.
Shown as percent
aws.kafka.cpu_system
(gauge)
The percentage of CPU in kernel space.
Shown as percent
aws.kafka.cpu_idle
(gauge)
The percentage of CPU idle time.
Shown as percent
aws.kafka.root_disk_used
(gauge)
The percentage of the root disk used by the broker.
Shown as percent
aws.kafka.kafka_app_logs_disk_used
(gauge)
The percentage of disk space used for application logs.
Shown as percent
aws.kafka.kafka_data_logs_disk_used
(gauge)
The percentage of disk space used for data logs.
Shown as percent
aws.kafka.network_rx_errors
(count)
The number of network receive errors for the broker.
aws.kafka.network_tx_errors
(count)
The number of network transmit errors for the broker.
aws.kafka.network_rx_dropped
(count)
The number of dropped receive packages.
aws.kafka.network_tx_dropped
(count)
The number of dropped transmit packages.
aws.kafka.network_rx_packets
(count)
The number of packets recieved by the broker.
aws.kafka.network_tx_packets
(count)
The number of packets transmitted by the broker.
aws.kafka.messages_in_per_sec
(gauge)
The number of incoming messages per second for the broker.
aws.kafka.network_processor_avg_idle_percent
(gauge)
The average percentage of the time the network processors are idle.
aws.kafka.request_handler_avg_idle_percent
(gauge)
The average percentage of the time the request handler threads are idle.
aws.kafka.leader_count
(gauge)
The number of leader replicas.
aws.kafka.partition_count
(gauge)
The number of partitions for the broker.
aws.kafka.produce_local_time_ms_mean
(gauge)
The mean time in milliseconds for the follower to send a response.
Shown as millisecond
aws.kafka.produce_message_conversions_time_ms_mean
(gauge)
The mean time in milliseconds spent on message format conversions.
Shown as millisecond
aws.kafka.produce_request_queue_time_ms_mean
(gauge)
The mean time in milliseconds that request messages spend in the queue.
Shown as millisecond
aws.kafka.produce_response_queue_time_ms_mean
(gauge)
The mean time in milliseconds that response messages spend in the queue.
Shown as millisecond
aws.kafka.produce_response_send_time_ms_mean
(gauge)
The mean time in milliseconds spent on sending response messages.
Shown as millisecond
aws.kafka.produce_total_time_ms_mean
(gauge)
The mean produce time in milliseconds.
Shown as millisecond
aws.kafka.request_bytes_mean
(gauge)
The mean number of request bytes for the broker.
aws.kafka.under_minlsr_partition_count
(gauge)
The number of under minlsr partitions for the broker
aws.kafka.under_replicated_partitions
(gauge)
The number of under-replicated partitions for the broker.
aws.kafka.bytes_in_per_sec
(gauge)
The number of bytes per second received from clients.
Shown as byte
aws.kafka.bytes_out_per_sec
(gauge)
The number of bytes per second sent to clients.
Shown as byte
aws.kafka.messages_in_per_sec
(gauge)
The number of messages received from clients per second.
aws.kafka.fetch_message_conversions_per_sec
(gauge)
The number of fetch message conversions per second for the broker.
aws.kafka.produce_message_conversions_per_sec
(gauge)
The number of produce message conversions per second for the broker.
aws.kafka.fetch_consumer_total_time_ms_mean
(gauge)
The mean total time in milliseconds that consumers spend on fetching data from the broker.
Shown as millisecond
aws.kafka.fetch_follower_total_time_ms_mean
(gauge)
The mean total time in milliseconds that followers spend on fetching data from the broker.
Shown as millisecond
aws.kafka.fetch_consumer_request_queue_time_ms_mean
(gauge)
The mean time in milliseconds that the consumer request waits in the request queue.
Shown as millisecond
aws.kafka.fetch_follower_request_queue_time_ms_mean
(gauge)
The mean time in milliseconds that the follower request waits in the request queue.
Shown as millisecond
aws.kafka.fetch_consumer_local_time_ms_mean
(gauge)
The mean time in milliseconds that the consumer request is processed at the leader.
Shown as millisecond
aws.kafka.fetch_follower_local_time_ms_mean
(gauge)
The mean time in milliseconds that the follower request is processed at the leader.
Shown as millisecond
aws.kafka.fetch_consumer_response_queue_time_ms_mean
(gauge)
The mean time in milliseconds that the consumer request waits in the response queue.
Shown as millisecond
aws.kafka.fetch_follower_response_queue_time_ms_mean
(gauge)
The mean time in milliseconds that the follower request waits in the response queue.
Shown as millisecond
aws.kafka.consumer_response_send_time_ms_mean
(gauge)
The mean time in milliseconds for the consumer to send a response.
Shown as millisecond
aws.kafka.fetch_follower_response_send_time_ms_mean
(gauge)
The mean time in milliseconds for the follower to send a response.
Shown as millisecond
aws.kafka.produce_throttle_time
(gauge)
The average produce throttle time in milliseconds.
Shown as millisecond
aws.kafka.produce_throttle_byte_rate
(gauge)
The number of throttled bytes per second.
aws.kafka.produce_throttle_queue_size
(gauge)
The number of messages in the throttle queue.
aws.kafka.fetch_throttle_time
(gauge)
The average fetch throttle time in milliseconds.
Shown as millisecond
aws.kafka.fetch_throttle_byte_rate
(gauge)
The number of throttled bytes per second.
aws.kafka.fetch_throttle_queue_size
(gauge)
The number of messages in the throttle queue.
aws.kafka.request_throttle_time
(gauge)
The average request throttle time in milliseconds.
Shown as millisecond
aws.kafka.request_time
(gauge)
The average time spent in broker network and I/O threads to process requests that are exempt throttled.
aws.kafka.request_throttle_queue_size
(gauge)
The number of messages in the throttle queue.
aws.kafka.request_exempt_from_throttle_time
(gauge)
The average time spent in broker network and I/O threads to process requests that are exempt from throttling.

The Amazon MSK crawler does not include any events or service checks.

Troubleshooting

Need help? Contact Datadog support.