Amazon Managed Streaming for Apache Kafka

docs>integrations>Amazon Managed Streaming for Apache Kafka

概要

Amazon Managed Streaming for Apache Kafka (MSK) は、Apache Kafka を使用してストリーミングデータを処理するアプリケーションを、簡単に構築して実行できるフルマネージド型のサービスです。

このインテグレーションからメトリクスを収集する方法は、Datadog Agent を使用する方法と、クローラーを使用して CloudWatch からメトリクスを収集する方法の 2 通りあります。

Agent チェック

Agent チェックは、Datadog Agent を通じて、Amazon Managed Streaming for Apache Kafka (Amazon MSK) を監視します。

セットアップ

ホストで実行されている Agent 用にこのチェックをインストールおよび構成する場合は、以下の手順に従ってください。コンテナ環境の場合は、オートディスカバリーのインテグレーションテンプレートのガイドを参照してこの手順を行ってください。

インストール

まだ作成していない場合は、クライアントマシンを作成します
クライアントマシンにアクセス許可ポリシー（arn:aws:iam::aws:policy/AmazonMSKReadOnlyAccess）が付与されているか、これに相当する資格情報が使用できることを確認します
MSK 側で Prometheus によるオープンモニタリングを有効にし、JmxExporter および NodeExporter を有効にします。
Datadog Agentをインストール

コンフィギュレーション

Agent のコンフィギュレーションディレクトリのルートにある conf.d/ フォルダーの amazon_msk.d/conf.yaml ファイルを編集し、Amazon MSK のパフォーマンスデータの収集を開始します。使用可能なすべてのコンフィギュレーションオプションの詳細については、サンプル amazon_msk.d/conf.yamlを参照してください。
Agent を再起動します。

検証

Agent の status サブコマンドを実行し、Checks セクションで amazon_msk を探します。

収集データ

メトリクス

aws.msk.go.gc.duration.seconds.count (gauge)	GC invocation durations count.
aws.msk.go.gc.duration.seconds.quantile (gauge)	GC invocation duration quantile. Shown as second
aws.msk.go.gc.duration.seconds.sum (gauge)	GC invocation duration total. Shown as second
aws.msk.go.goroutines (gauge)	Number of goroutines that currently exist. Shown as thread
aws.msk.go.info (gauge)	Information about the Go environment.
aws.msk.go.memstats.alloc.bytes (gauge)	Number of bytes allocated and still in use. Shown as byte
aws.msk.go.memstats.alloc.bytes.total (count)	Total number of bytes allocated, even if freed. Shown as byte
aws.msk.go.memstats.buck.hash.sys.bytes (gauge)	Number of bytes used by the profiling bucket hash table. Shown as byte
aws.msk.go.memstats.frees.total (count)	Total number of frees.
aws.msk.go.memstats.gc.cpu.fraction (gauge)	The fraction of this program's available CPU time used by the GC since the program started.
aws.msk.go.memstats.gc.sys.bytes (gauge)	Number of bytes used for garbage collection system metadata. Shown as byte
aws.msk.go.memstats.heap.alloc.bytes (gauge)	Number of heap bytes allocated and still in use. Shown as byte
aws.msk.go.memstats.heap.idle.bytes (gauge)	Number of heap bytes waiting to be used. Shown as byte
aws.msk.go.memstats.heap.inuse.bytes (gauge)	Number of heap bytes that are in use. Shown as byte
aws.msk.go.memstats.heap.objects (gauge)	Number of allocated objects.
aws.msk.go.memstats.heap.released.bytes (gauge)	Number of heap bytes released to OS. Shown as byte
aws.msk.go.memstats.heap.sys.bytes (gauge)	Number of heap bytes obtained from system. Shown as byte
aws.msk.go.memstats.last.gc.time.seconds (gauge)	Number of seconds since 1970 of last garbage collection. Shown as second
aws.msk.go.memstats.lookups.total (count)	Total number of pointer lookups.
aws.msk.go.memstats.mallocs.total (count)	Total number of mallocs.
aws.msk.go.memstats.mcache.inuse.bytes (gauge)	Number of bytes in use by mcache structures. Shown as byte
aws.msk.go.memstats.mcache.sys.bytes (gauge)	Number of bytes used for mcache structures obtained from system. Shown as byte
aws.msk.go.memstats.mspan.inuse.bytes (gauge)	Number of bytes in use by mspan structures. Shown as byte
aws.msk.go.memstats.mspan.sys.bytes (gauge)	Number of bytes used for mspan structures obtained from system. Shown as byte
aws.msk.go.memstats.next.gc.bytes (gauge)	Number of heap bytes when next garbage collection will take place. Shown as byte
aws.msk.go.memstats.other.sys.bytes (gauge)	Number of bytes used for other system allocations. Shown as byte
aws.msk.go.memstats.stack.inuse.bytes (gauge)	Number of bytes in use by the stack allocator. Shown as byte
aws.msk.go.memstats.stack.sys.bytes (gauge)	Number of bytes obtained from system for stack allocator. Shown as byte
aws.msk.go.memstats.sys.bytes (gauge)	Number of bytes obtained from system. Shown as byte
aws.msk.go.threads (gauge)	Number of OS threads created. Shown as thread
aws.msk.jmx.config.reload.failure.total (count)	Number of times configuration have failed to be reloaded.
aws.msk.jmx.config.reload.success.total (count)	Number of times configuration have successfully been reloaded.
aws.msk.jmx.exporter.build.info (gauge)	A metric with a constant '1' value labeled with the version of the JMX exporter.
aws.msk.jmx.scrape.duration.seconds (gauge)	Time this JMX scrape took, in seconds. Shown as second
aws.msk.jmx.scrape.error (gauge)	Non-zero if this scrape failed.
aws.msk.kafka.cluster.Partition.Value (gauge)	Attribute exposed for management (kafka.cluster<>Value)
aws.msk.kafka.consumer.group.ConsumerLagMetrics.Value (gauge)	Attribute exposed for management (kafka.consumer.group<>Value)
aws.msk.kafka.controller.ControllerChannelManager.50thPercentile (gauge)	Attribute exposed for management (kafka.controller<>50thPercentile)
aws.msk.kafka.controller.ControllerChannelManager.75thPercentile (gauge)	Attribute exposed for management (kafka.controller<>75thPercentile)
aws.msk.kafka.controller.ControllerChannelManager.95thPercentile (gauge)	Attribute exposed for management (kafka.controller<>95thPercentile)
aws.msk.kafka.controller.ControllerChannelManager.98thPercentile (gauge)	Attribute exposed for management (kafka.controller<>98thPercentile)
aws.msk.kafka.controller.ControllerChannelManager.999thPercentile (gauge)	Attribute exposed for management (kafka.controller<>999thPercentile)
aws.msk.kafka.controller.ControllerChannelManager.99thPercentile (gauge)	Attribute exposed for management (kafka.controller<>99thPercentile)
aws.msk.kafka.controller.ControllerChannelManager.Count (gauge)	Attribute exposed for management (kafka.controller<>Count)
aws.msk.kafka.controller.ControllerChannelManager.FifteenMinuteRate (gauge)	Attribute exposed for management (kafka.controller<>FifteenMinuteRate)
aws.msk.kafka.controller.ControllerChannelManager.FiveMinuteRate (gauge)	Attribute exposed for management (kafka.controller<>FiveMinuteRate)
aws.msk.kafka.controller.ControllerChannelManager.Max (gauge)	Attribute exposed for management (kafka.controller<>Max)
aws.msk.kafka.controller.ControllerChannelManager.Mean (gauge)	Attribute exposed for management (kafka.controller<>Mean)
aws.msk.kafka.controller.ControllerChannelManager.MeanRate (gauge)	Attribute exposed for management (kafka.controller<>MeanRate)
aws.msk.kafka.controller.ControllerChannelManager.Min (gauge)	Attribute exposed for management (kafka.controller<>Min)
aws.msk.kafka.controller.ControllerChannelManager.OneMinuteRate (gauge)	Attribute exposed for management (kafka.controller<>OneMinuteRate)
aws.msk.kafka.controller.ControllerChannelManager.StdDev (gauge)	Attribute exposed for management (kafka.controller<>StdDev)
aws.msk.kafka.controller.ControllerChannelManager.Value (gauge)	Attribute exposed for management (kafka.controller<>Value)
aws.msk.kafka.controller.ControllerEventManager.50thPercentile (gauge)	Attribute exposed for management (kafka.controller<>50thPercentile)
aws.msk.kafka.controller.ControllerEventManager.75thPercentile (gauge)	Attribute exposed for management (kafka.controller<>75thPercentile)
aws.msk.kafka.controller.ControllerEventManager.95thPercentile (gauge)	Attribute exposed for management (kafka.controller<>95thPercentile)
aws.msk.kafka.controller.ControllerEventManager.98thPercentile (gauge)	Attribute exposed for management (kafka.controller<>98thPercentile)
aws.msk.kafka.controller.ControllerEventManager.999thPercentile (gauge)	Attribute exposed for management (kafka.controller<>999thPercentile)
aws.msk.kafka.controller.ControllerEventManager.99thPercentile (gauge)	Attribute exposed for management (kafka.controller<>99thPercentile)
aws.msk.kafka.controller.ControllerEventManager.Count (gauge)	Attribute exposed for management (kafka.controller<>Count)
aws.msk.kafka.controller.ControllerEventManager.Max (gauge)	Attribute exposed for management (kafka.controller<>Max)
aws.msk.kafka.controller.ControllerEventManager.Mean (gauge)	Attribute exposed for management (kafka.controller<>Mean)
aws.msk.kafka.controller.ControllerEventManager.Min (gauge)	Attribute exposed for management (kafka.controller<>Min)
aws.msk.kafka.controller.ControllerEventManager.StdDev (gauge)	Attribute exposed for management (kafka.controller<>StdDev)
aws.msk.kafka.controller.ControllerEventManager.Value (gauge)	Attribute exposed for management (kafka.controller<>Value)
aws.msk.kafka.controller.ControllerStats.50thPercentile (gauge)	Attribute exposed for management (kafka.controller<>50thPercentile)
aws.msk.kafka.controller.ControllerStats.75thPercentile (gauge)	Attribute exposed for management (kafka.controller<>75thPercentile)
aws.msk.kafka.controller.ControllerStats.95thPercentile (gauge)	Attribute exposed for management (kafka.controller<>95thPercentile)
aws.msk.kafka.controller.ControllerStats.98thPercentile (gauge)	Attribute exposed for management (kafka.controller<>98thPercentile)
aws.msk.kafka.controller.ControllerStats.999thPercentile (gauge)	Attribute exposed for management (kafka.controller<>999thPercentile)
aws.msk.kafka.controller.ControllerStats.99thPercentile (gauge)	Attribute exposed for management (kafka.controller<>99thPercentile)
aws.msk.kafka.controller.ControllerStats.Count (gauge)	Attribute exposed for management (kafka.controller<>Count)
aws.msk.kafka.controller.ControllerStats.FifteenMinuteRate (gauge)	Attribute exposed for management (kafka.controller<>FifteenMinuteRate)
aws.msk.kafka.controller.ControllerStats.FiveMinuteRate (gauge)	Attribute exposed for management (kafka.controller<>FiveMinuteRate)
aws.msk.kafka.controller.ControllerStats.Max (gauge)	Attribute exposed for management (kafka.controller<>Max)
aws.msk.kafka.controller.ControllerStats.Mean (gauge)	Attribute exposed for management (kafka.controller<>Mean)
aws.msk.kafka.controller.ControllerStats.MeanRate (gauge)	Attribute exposed for management (kafka.controller<>MeanRate)
aws.msk.kafka.controller.ControllerStats.Min (gauge)	Attribute exposed for management (kafka.controller<>Min)
aws.msk.kafka.controller.ControllerStats.OneMinuteRate (gauge)	Attribute exposed for management (kafka.controller<>OneMinuteRate)
aws.msk.kafka.controller.ControllerStats.StdDev (gauge)	Attribute exposed for management (kafka.controller<>StdDev)
aws.msk.kafka.controller.KafkaController.Value (gauge)	Attribute exposed for management (kafka.controller<>Value)
aws.msk.kafka.coordinator.group.GroupMetadataManager.Value (gauge)	Attribute exposed for management (kafka.coordinator.group<>Value)
aws.msk.kafka.coordinator.transaction.TransactionMarkerChannelManager.Value (gauge)	Attribute exposed for management (kafka.coordinator.transaction<>Value)
aws.msk.kafka.log.Log.Value (gauge)	Attribute exposed for management (kafka.log<>Value)
aws.msk.kafka.log.LogCleaner.Value (gauge)	Attribute exposed for management (kafka.log<>Value)
aws.msk.kafka.log.LogCleanerManager.Value (gauge)	Attribute exposed for management (kafka.log<>Value)
aws.msk.kafka.log.LogManager.Value (gauge)	Attribute exposed for management (kafka.log<>Value)
aws.msk.kafka.network.Acceptor.Count (gauge)	Attribute exposed for management (kafka.network<>Count)
aws.msk.kafka.network.Acceptor.FifteenMinuteRate (gauge)	Attribute exposed for management (kafka.network<>FifteenMinuteRate)
aws.msk.kafka.network.Acceptor.FiveMinuteRate (gauge)	Attribute exposed for management (kafka.network<>FiveMinuteRate)
aws.msk.kafka.network.Acceptor.MeanRate (gauge)	Attribute exposed for management (kafka.network<>MeanRate)
aws.msk.kafka.network.Acceptor.OneMinuteRate (gauge)	Attribute exposed for management (kafka.network<>OneMinuteRate)
aws.msk.kafka.network.Processor.Value (gauge)	Attribute exposed for management (kafka.network<>Value)
aws.msk.kafka.network.RequestChannel.Value (gauge)	Attribute exposed for management (kafka.network<>Value)
aws.msk.kafka.network.RequestMetrics.50thPercentile (gauge)	Attribute exposed for management (kafka.network<>50thPercentile)
aws.msk.kafka.network.RequestMetrics.75thPercentile (gauge)	Attribute exposed for management (kafka.network<>75thPercentile)
aws.msk.kafka.network.RequestMetrics.95thPercentile (gauge)	Attribute exposed for management (kafka.network<>95thPercentile)
aws.msk.kafka.network.RequestMetrics.98thPercentile (gauge)	Attribute exposed for management (kafka.network<>98thPercentile)
aws.msk.kafka.network.RequestMetrics.999thPercentile (gauge)	Attribute exposed for management (kafka.network<>999thPercentile)
aws.msk.kafka.network.RequestMetrics.99thPercentile (gauge)	Attribute exposed for management (kafka.network<>99thPercentile)
aws.msk.kafka.network.RequestMetrics.Count (gauge)	Attribute exposed for management (kafka.network<>Count)
aws.msk.kafka.network.RequestMetrics.FifteenMinuteRate (gauge)	Attribute exposed for management (kafka.network<>FifteenMinuteRate)
aws.msk.kafka.network.RequestMetrics.FiveMinuteRate (gauge)	Attribute exposed for management (kafka.network<>FiveMinuteRate)
aws.msk.kafka.network.RequestMetrics.Max (gauge)	Attribute exposed for management (kafka.network<>Max)
aws.msk.kafka.network.RequestMetrics.Mean (gauge)	Attribute exposed for management (kafka.network<>Mean)
aws.msk.kafka.network.RequestMetrics.MeanRate (gauge)	Attribute exposed for management (kafka.network<>MeanRate)
aws.msk.kafka.network.RequestMetrics.Min (gauge)	Attribute exposed for management (kafka.network<>Min)
aws.msk.kafka.network.RequestMetrics.OneMinuteRate (gauge)	Attribute exposed for management (kafka.network<>OneMinuteRate)
aws.msk.kafka.network.RequestMetrics.StdDev (gauge)	Attribute exposed for management (kafka.network<>StdDev)
aws.msk.kafka.network.SocketServer.Value (gauge)	Attribute exposed for management (kafka.network<>Value)
aws.msk.kafka.security.SimpleAclAuthorizer.Count (gauge)	Attribute exposed for management (kafka.security<>Count)
aws.msk.kafka.security.SimpleAclAuthorizer.FifteenMinuteRate (gauge)	Attribute exposed for management (kafka.security<>FifteenMinuteRate)
aws.msk.kafka.security.SimpleAclAuthorizer.FiveMinuteRate (gauge)	Attribute exposed for management (kafka.security<>FiveMinuteRate)
aws.msk.kafka.security.SimpleAclAuthorizer.MeanRate (gauge)	Attribute exposed for management (kafka.security<>MeanRate)
aws.msk.kafka.security.SimpleAclAuthorizer.OneMinuteRate (gauge)	Attribute exposed for management (kafka.security<>OneMinuteRate)
aws.msk.kafka.server.BrokerTopicMetrics.Count (gauge)	Attribute exposed for management (kafka.server<>Count)
aws.msk.kafka.server.BrokerTopicMetrics.FifteenMinuteRate (gauge)	Attribute exposed for management (kafka.server<>FifteenMinuteRate)
aws.msk.kafka.server.BrokerTopicMetrics.FiveMinuteRate (gauge)	Attribute exposed for management (kafka.server<>FiveMinuteRate)
aws.msk.kafka.server.BrokerTopicMetrics.MeanRate (gauge)	Attribute exposed for management (kafka.server<>MeanRate)
aws.msk.kafka.server.BrokerTopicMetrics.OneMinuteRate (gauge)	Attribute exposed for management (kafka.server<>OneMinuteRate)
aws.msk.kafka.server.DelayedFetchMetrics.Count (gauge)	Attribute exposed for management (kafka.server<>Count)
aws.msk.kafka.server.DelayedFetchMetrics.FifteenMinuteRate (gauge)	Attribute exposed for management (kafka.server<>FifteenMinuteRate)
aws.msk.kafka.server.DelayedFetchMetrics.FiveMinuteRate (gauge)	Attribute exposed for management (kafka.server<>FiveMinuteRate)
aws.msk.kafka.server.DelayedFetchMetrics.MeanRate (gauge)	Attribute exposed for management (kafka.server<>MeanRate)
aws.msk.kafka.server.DelayedFetchMetrics.OneMinuteRate (gauge)	Attribute exposed for management (kafka.server<>OneMinuteRate)
aws.msk.kafka.server.DelayedOperationPurgatory.Value (gauge)	Attribute exposed for management (kafka.server<>Value)
aws.msk.kafka.server.Fetch.queue.size (gauge)	Tracks the size of the delay queue (kafka.server<>queue-size)
aws.msk.kafka.server.FetchSessionCache.Count (gauge)	Attribute exposed for management (kafka.server<>Count)
aws.msk.kafka.server.FetchSessionCache.FifteenMinuteRate (gauge)	Attribute exposed for management (kafka.server<>FifteenMinuteRate)
aws.msk.kafka.server.FetchSessionCache.FiveMinuteRate (gauge)	Attribute exposed for management (kafka.server<>FiveMinuteRate)
aws.msk.kafka.server.FetchSessionCache.MeanRate (gauge)	Attribute exposed for management (kafka.server<>MeanRate)
aws.msk.kafka.server.FetchSessionCache.OneMinuteRate (gauge)	Attribute exposed for management (kafka.server<>OneMinuteRate)
aws.msk.kafka.server.FetchSessionCache.Value (gauge)	Attribute exposed for management (kafka.server<>Value)
aws.msk.kafka.server.FetcherLagMetrics.Value (gauge)	Attribute exposed for management (kafka.server<>Value)
aws.msk.kafka.server.FetcherStats.Count (gauge)	Attribute exposed for management (kafka.server<>Count)
aws.msk.kafka.server.FetcherStats.FifteenMinuteRate (gauge)	Attribute exposed for management (kafka.server<>FifteenMinuteRate)
aws.msk.kafka.server.FetcherStats.FiveMinuteRate (gauge)	Attribute exposed for management (kafka.server<>FiveMinuteRate)
aws.msk.kafka.server.FetcherStats.MeanRate (gauge)	Attribute exposed for management (kafka.server<>MeanRate)
aws.msk.kafka.server.FetcherStats.OneMinuteRate (gauge)	Attribute exposed for management (kafka.server<>OneMinuteRate)
aws.msk.kafka.server.KafkaRequestHandlerPool.Count (gauge)	Attribute exposed for management (kafka.server<>Count)
aws.msk.kafka.server.KafkaRequestHandlerPool.FifteenMinuteRate (gauge)	Attribute exposed for management (kafka.server<>FifteenMinuteRate)
aws.msk.kafka.server.KafkaRequestHandlerPool.FiveMinuteRate (gauge)	Attribute exposed for management (kafka.server<>FiveMinuteRate)
aws.msk.kafka.server.KafkaRequestHandlerPool.MeanRate (gauge)	Attribute exposed for management (kafka.server<>MeanRate)
aws.msk.kafka.server.KafkaRequestHandlerPool.OneMinuteRate (gauge)	Attribute exposed for management (kafka.server<>OneMinuteRate)
aws.msk.kafka.server.KafkaServer.Value (gauge)	Attribute exposed for management (kafka.server<>Value)
aws.msk.kafka.server.LeaderReplication.byte.rate (gauge)	Tracking byte-rate for LeaderReplication (kafka.server<>byte-rate)
aws.msk.kafka.server.Produce.queue.size (gauge)	Tracks the size of the delay queue (kafka.server<>queue-size)
aws.msk.kafka.server.ReplicaAlterLogDirsManager.Value (gauge)	Attribute exposed for management (kafka.server<>Value)
aws.msk.kafka.server.ReplicaFetcherManager.Value (gauge)	Attribute exposed for management (kafka.server<>Value)
aws.msk.kafka.server.ReplicaManager.Count (gauge)	Attribute exposed for management (kafka.server<>Count)
aws.msk.kafka.server.ReplicaManager.FifteenMinuteRate (gauge)	Attribute exposed for management (kafka.server<>FifteenMinuteRate)
aws.msk.kafka.server.ReplicaManager.FiveMinuteRate (gauge)	Attribute exposed for management (kafka.server<>FiveMinuteRate)
aws.msk.kafka.server.ReplicaManager.MeanRate (gauge)	Attribute exposed for management (kafka.server<>MeanRate)
aws.msk.kafka.server.ReplicaManager.OneMinuteRate (gauge)	Attribute exposed for management (kafka.server<>OneMinuteRate)
aws.msk.kafka.server.ReplicaManager.Value (gauge)	Attribute exposed for management (kafka.server<>Value)
aws.msk.kafka.server.Request.queue.size (gauge)	Tracks the size of the delay queue (kafka.server<>queue-size)
aws.msk.kafka.server.SessionExpireListener.Count (gauge)	Attribute exposed for management (kafka.server<>Count)
aws.msk.kafka.server.SessionExpireListener.FifteenMinuteRate (gauge)	Attribute exposed for management (kafka.server<>FifteenMinuteRate)
aws.msk.kafka.server.SessionExpireListener.FiveMinuteRate (gauge)	Attribute exposed for management (kafka.server<>FiveMinuteRate)
aws.msk.kafka.server.SessionExpireListener.MeanRate (gauge)	Attribute exposed for management (kafka.server<>MeanRate)
aws.msk.kafka.server.SessionExpireListener.OneMinuteRate (gauge)	Attribute exposed for management (kafka.server<>OneMinuteRate)
aws.msk.kafka.server.ZooKeeperClientMetrics.50thPercentile (gauge)	Attribute exposed for management (kafka.server<>50thPercentile)
aws.msk.kafka.server.ZooKeeperClientMetrics.75thPercentile (gauge)	Attribute exposed for management (kafka.server<>75thPercentile)
aws.msk.kafka.server.ZooKeeperClientMetrics.95thPercentile (gauge)	Attribute exposed for management (kafka.server<>95thPercentile)
aws.msk.kafka.server.ZooKeeperClientMetrics.98thPercentile (gauge)	Attribute exposed for management (kafka.server<>98thPercentile)
aws.msk.kafka.server.ZooKeeperClientMetrics.999thPercentile (gauge)	Attribute exposed for management (kafka.server<>999thPercentile)
aws.msk.kafka.server.ZooKeeperClientMetrics.99thPercentile (gauge)	Attribute exposed for management (kafka.server<>99thPercentile)
aws.msk.kafka.server.ZooKeeperClientMetrics.Count (gauge)	Attribute exposed for management (kafka.server<>Count)
aws.msk.kafka.server.ZooKeeperClientMetrics.Max (gauge)	Attribute exposed for management (kafka.server<>Max)
aws.msk.kafka.server.ZooKeeperClientMetrics.Mean (gauge)	Attribute exposed for management (kafka.server<>Mean)
aws.msk.kafka.server.ZooKeeperClientMetrics.Min (gauge)	Attribute exposed for management (kafka.server<>Min)
aws.msk.kafka.server.ZooKeeperClientMetrics.StdDev (gauge)	Attribute exposed for management (kafka.server<>StdDev)
aws.msk.kafka.server.controller.channel.metrics.connection.close.rate (gauge)	The number of connections closed per second (kafka.server<>connection-close-rate) Shown as connection
aws.msk.kafka.server.controller.channel.metrics.connection.close.total (gauge)	The total number of connections closed (kafka.server<>connection-close-total) Shown as connection
aws.msk.kafka.server.controller.channel.metrics.connection.count (gauge)	The current number of active connections. (kafka.server<>connection-count) Shown as connection
aws.msk.kafka.server.controller.channel.metrics.connection.creation.rate (gauge)	The number of new connections established per second (kafka.server<>connection-creation-rate)
aws.msk.kafka.server.controller.channel.metrics.connection.creation.total (gauge)	The total number of new connections established (kafka.server<>connection-creation-total)
aws.msk.kafka.server.controller.channel.metrics.failed.authentication.rate (gauge)	The number of connections with failed authentication per second (kafka.server<>failed-authentication-rate)
aws.msk.kafka.server.controller.channel.metrics.failed.authentication.total (gauge)	The total number of connections with failed authentication (kafka.server<>failed-authentication-total)
aws.msk.kafka.server.controller.channel.metrics.failed.reauthentication.rate (gauge)	The number of failed re-authentication of connections per second (kafka.server<>failed-reauthentication-rate)
aws.msk.kafka.server.controller.channel.metrics.failed.reauthentication.total (gauge)	The total number of failed re-authentication of connections (kafka.server<>failed-reauthentication-total)
aws.msk.kafka.server.controller.channel.metrics.incoming.byte.rate (gauge)	The number of bytes read off all sockets per second (kafka.server<>incoming-byte-rate) Shown as byte
aws.msk.kafka.server.controller.channel.metrics.incoming.byte.total (gauge)	The total number of bytes read off all sockets (kafka.server<>incoming-byte-total) Shown as byte
aws.msk.kafka.server.controller.channel.metrics.io.ratio (gauge)	The fraction of time the I/O thread spent doing I/O (kafka.server<>io-ratio)
aws.msk.kafka.server.controller.channel.metrics.io.time.ns.avg (gauge)	The average length of time for I/O per select call in nanoseconds. (kafka.server<>io-time-ns-avg)
aws.msk.kafka.server.controller.channel.metrics.io.wait.ratio (gauge)	The fraction of time the I/O thread spent waiting (kafka.server<>io-wait-ratio)
aws.msk.kafka.server.controller.channel.metrics.io.wait.time.ns.avg (gauge)	The average length of time the I/O thread spent waiting for a socket ready for reads or writes in nanoseconds. (kafka.server<>io-wait-time-ns-avg)
aws.msk.kafka.server.controller.channel.metrics.io.waittime.total (gauge)	The total time the I/O thread spent waiting (kafka.server<>io-waittime-total)
aws.msk.kafka.server.controller.channel.metrics.iotime.total (gauge)	The total time the I/O thread spent doing I/O (kafka.server<>iotime-total)
aws.msk.kafka.server.controller.channel.metrics.network.io.rate (gauge)	The number of network operations (reads or writes) on all connections per second (kafka.server<>network-io-rate)
aws.msk.kafka.server.controller.channel.metrics.network.io.total (gauge)	The total number of network operations (reads or writes) on all connections (kafka.server<>network-io-total)
aws.msk.kafka.server.controller.channel.metrics.outgoing.byte.rate (gauge)	The number of outgoing bytes sent to all servers per second (kafka.server<>outgoing-byte-rate) Shown as byte
aws.msk.kafka.server.controller.channel.metrics.outgoing.byte.total (gauge)	The total number of outgoing bytes sent to all servers (kafka.server<>outgoing-byte-total) Shown as byte
aws.msk.kafka.server.controller.channel.metrics.reauthentication.latency.avg (gauge)	The average latency observed due to re-authentication (kafka.server<>reauthentication-latency-avg)
aws.msk.kafka.server.controller.channel.metrics.reauthentication.latency.max (gauge)	The max latency observed due to re-authentication (kafka.server<>reauthentication-latency-max)
aws.msk.kafka.server.controller.channel.metrics.request.rate (gauge)	The number of requests sent per second (kafka.server<>request-rate)
aws.msk.kafka.server.controller.channel.metrics.request.size.avg (gauge)	The average size of requests sent. (kafka.server<>request-size-avg)
aws.msk.kafka.server.controller.channel.metrics.request.size.max (gauge)	The maximum size of any request sent. (kafka.server<>request-size-max)
aws.msk.kafka.server.controller.channel.metrics.request.total (gauge)	The total number of requests sent (kafka.server<>request-total)
aws.msk.kafka.server.controller.channel.metrics.response.rate (gauge)	The number of responses received per second (kafka.server<>response-rate)
aws.msk.kafka.server.controller.channel.metrics.response.total (gauge)	The total number of responses received (kafka.server<>response-total)
aws.msk.kafka.server.controller.channel.metrics.select.rate (gauge)	The number of times the I/O layer checked for new I/O to perform per second (kafka.server<>select-rate)
aws.msk.kafka.server.controller.channel.metrics.select.total (gauge)	The total number of times the I/O layer checked for new I/O to perform (kafka.server<>select-total)
aws.msk.kafka.server.controller.channel.metrics.successful.authentication.no.reauth.total (gauge)	The total number of connections with successful authentication where the client does not support re-authentication (kafka.server<>successful-authentication-no-reauth-total)
aws.msk.kafka.server.controller.channel.metrics.successful.authentication.rate (gauge)	The number of connections with successful authentication per second (kafka.server<>successful-authentication-rate)
aws.msk.kafka.server.controller.channel.metrics.successful.authentication.total (gauge)	The total number of connections with successful authentication (kafka.server<>successful-authentication-total)
aws.msk.kafka.server.controller.channel.metrics.successful.reauthentication.rate (gauge)	The number of successful re-authentication of connections per second (kafka.server<>successful-reauthentication-rate)
aws.msk.kafka.server.controller.channel.metrics.successful.reauthentication.total (gauge)	The total number of successful re-authentication of connections (kafka.server<>successful-reauthentication-total)
aws.msk.kafka.server.kafka.metrics.count.count (gauge)	total number of registered metrics (kafka.server<>count)
aws.msk.kafka.server.replica.fetcher.metrics.connection.close.rate (gauge)	The number of connections closed per second (kafka.server<>connection-close-rate)
aws.msk.kafka.server.replica.fetcher.metrics.connection.close.total (gauge)	The total number of connections closed (kafka.server<>connection-close-total)
aws.msk.kafka.server.replica.fetcher.metrics.connection.count (gauge)	The current number of active connections. (kafka.server<>connection-count)
aws.msk.kafka.server.replica.fetcher.metrics.connection.creation.rate (gauge)	The number of new connections established per second (kafka.server<>connection-creation-rate)
aws.msk.kafka.server.replica.fetcher.metrics.connection.creation.total (gauge)	The total number of new connections established (kafka.server<>connection-creation-total)
aws.msk.kafka.server.replica.fetcher.metrics.failed.authentication.rate (gauge)	The number of connections with failed authentication per second (kafka.server<>failed-authentication-rate)
aws.msk.kafka.server.replica.fetcher.metrics.failed.authentication.total (gauge)	The total number of connections with failed authentication (kafka.server<>failed-authentication-total)
aws.msk.kafka.server.replica.fetcher.metrics.incoming.byte.rate (gauge)	The number of bytes read off all sockets per second (kafka.server<>incoming-byte-rate) Shown as byte
aws.msk.kafka.server.replica.fetcher.metrics.incoming.byte.total (gauge)	The total number of bytes read off all sockets (kafka.server<>incoming-byte-total) Shown as byte
aws.msk.kafka.server.replica.fetcher.metrics.io.ratio (gauge)	The fraction of time the I/O thread spent doing I/O (kafka.server<>io-ratio)
aws.msk.kafka.server.replica.fetcher.metrics.io.time.ns.avg (gauge)	The average length of time for I/O per select call in nanoseconds. (kafka.server<>io-time-ns-avg)
aws.msk.kafka.server.replica.fetcher.metrics.io.wait.ratio (gauge)	The fraction of time the I/O thread spent waiting (kafka.server<>io-wait-ratio)
aws.msk.kafka.server.replica.fetcher.metrics.io.wait.time.ns.avg (gauge)	The average length of time the I/O thread spent waiting for a socket ready for reads or writes in nanoseconds. (kafka.server<>io-wait-time-ns-avg)
aws.msk.kafka.server.replica.fetcher.metrics.io.waittime.total (gauge)	The total time the I/O thread spent waiting (kafka.server<>io-waittime-total)
aws.msk.kafka.server.replica.fetcher.metrics.iotime.total (gauge)	The total time the I/O thread spent doing I/O (kafka.server<>iotime-total)
aws.msk.kafka.server.replica.fetcher.metrics.network.io.rate (gauge)	The number of network operations (reads or writes) on all connections per second (kafka.server<>network-io-rate)
aws.msk.kafka.server.replica.fetcher.metrics.network.io.total (gauge)	The total number of network operations (reads or writes) on all connections (kafka.server<>network-io-total)
aws.msk.kafka.server.replica.fetcher.metrics.outgoing.byte.rate (gauge)	The number of outgoing bytes sent to all servers per second (kafka.server<>outgoing-byte-rate) Shown as byte
aws.msk.kafka.server.replica.fetcher.metrics.outgoing.byte.total (gauge)	The total number of outgoing bytes sent to all servers (kafka.server<>outgoing-byte-total) Shown as byte
aws.msk.kafka.server.replica.fetcher.metrics.request.rate (gauge)	The number of requests sent per second (kafka.server<>request-rate)
aws.msk.kafka.server.replica.fetcher.metrics.request.size.avg (gauge)	The average size of requests sent. (kafka.server<>request-size-avg)
aws.msk.kafka.server.replica.fetcher.metrics.request.size.max (gauge)	The maximum size of any request sent. (kafka.server<>request-size-max)
aws.msk.kafka.server.replica.fetcher.metrics.request.total (gauge)	The total number of requests sent (kafka.server<>request-total)
aws.msk.kafka.server.replica.fetcher.metrics.response.rate (gauge)	The number of responses received per second (kafka.server<>response-rate)
aws.msk.kafka.server.replica.fetcher.metrics.response.total (gauge)	The total number of responses received (kafka.server<>response-total)
aws.msk.kafka.server.replica.fetcher.metrics.select.rate (gauge)	The number of times the I/O layer checked for new I/O to perform per second (kafka.server<>select-rate)
aws.msk.kafka.server.replica.fetcher.metrics.select.total (gauge)	The total number of times the I/O layer checked for new I/O to perform (kafka.server<>select-total)
aws.msk.kafka.server.replica.fetcher.metrics.successful.authentication.rate (gauge)	The number of connections with successful authentication per second (kafka.server<>successful-authentication-rate)
aws.msk.kafka.server.replica.fetcher.metrics.successful.authentication.total (gauge)	The total number of connections with successful authentication (kafka.server<>successful-authentication-total)
aws.msk.kafka.server.socket.server.metrics.MemoryPoolAvgDepletedPercent (gauge)	(kafka.server<>MemoryPoolAvgDepletedPercent)
aws.msk.kafka.server.socket.server.metrics.MemoryPoolDepletedTimeTotal (gauge)	(kafka.server<>MemoryPoolDepletedTimeTotal)
aws.msk.kafka.server.socket.server.metrics.connection.close.rate (gauge)	The number of connections closed per second (kafka.server<>connection-close-rate)
aws.msk.kafka.server.socket.server.metrics.connection.close.total (gauge)	The total number of connections closed (kafka.server<>connection-close-total)
aws.msk.kafka.server.socket.server.metrics.connection.count (gauge)	The current number of active connections. (kafka.server<>connection-count)
aws.msk.kafka.server.socket.server.metrics.connection.creation.rate (gauge)	The number of new connections established per second (kafka.server<>connection-creation-rate)
aws.msk.kafka.server.socket.server.metrics.connection.creation.total (gauge)	The total number of new connections established (kafka.server<>connection-creation-total)
aws.msk.kafka.server.socket.server.metrics.expired.connections.killed.count (gauge)	(kafka.server<>expired-connections-killed-count)
aws.msk.kafka.server.socket.server.metrics.failed.authentication.rate (gauge)	The number of connections with failed authentication per second (kafka.server<>failed-authentication-rate)
aws.msk.kafka.server.socket.server.metrics.failed.authentication.total (gauge)	The total number of connections with failed authentication (kafka.server<>failed-authentication-total)
aws.msk.kafka.server.socket.server.metrics.failed.reauthentication.rate (gauge)	The number of failed re-authentication of connections per second (kafka.server<>failed-reauthentication-rate)
aws.msk.kafka.server.socket.server.metrics.failed.reauthentication.total (gauge)	The total number of failed re-authentication of connections (kafka.server<>failed-reauthentication-total)
aws.msk.kafka.server.socket.server.metrics.incoming.byte.rate (gauge)	The number of bytes read off all sockets per second (kafka.server<>incoming-byte-rate)
aws.msk.kafka.server.socket.server.metrics.incoming.byte.total (gauge)	The total number of bytes read off all sockets (kafka.server<>incoming-byte-total)
aws.msk.kafka.server.socket.server.metrics.io.ratio (gauge)	The fraction of time the I/O thread spent doing I/O (kafka.server<>io-ratio)
aws.msk.kafka.server.socket.server.metrics.io.time.ns.avg (gauge)	The average length of time for I/O per select call in nanoseconds. (kafka.server<>io-time-ns-avg)
aws.msk.kafka.server.socket.server.metrics.io.wait.ratio (gauge)	The fraction of time the I/O thread spent waiting (kafka.server<>io-wait-ratio)
aws.msk.kafka.server.socket.server.metrics.io.wait.time.ns.avg (gauge)	The average length of time the I/O thread spent waiting for a socket ready for reads or writes in nanoseconds. (kafka.server<>io-wait-time-ns-avg)
aws.msk.kafka.server.socket.server.metrics.io.waittime.total (gauge)	The total time the I/O thread spent waiting (kafka.server<>io-waittime-total)
aws.msk.kafka.server.socket.server.metrics.iotime.total (gauge)	The total time the I/O thread spent doing I/O (kafka.server<>iotime-total)
aws.msk.kafka.server.socket.server.metrics.network.io.rate (gauge)	The number of network operations (reads or writes) on all connections per second (kafka.server<>network-io-rate)
aws.msk.kafka.server.socket.server.metrics.network.io.total (gauge)	The total number of network operations (reads or writes) on all connections (kafka.server<>network-io-total)
aws.msk.kafka.server.socket.server.metrics.outgoing.byte.rate (gauge)	The number of outgoing bytes sent to all servers per second (kafka.server<>outgoing-byte-rate)
aws.msk.kafka.server.socket.server.metrics.outgoing.byte.total (gauge)	The total number of outgoing bytes sent to all servers (kafka.server<>outgoing-byte-total)
aws.msk.kafka.server.socket.server.metrics.reauthentication.latency.avg (gauge)	The average latency observed due to re-authentication (kafka.server<>reauthentication-latency-avg)
aws.msk.kafka.server.socket.server.metrics.reauthentication.latency.max (gauge)	The max latency observed due to re-authentication (kafka.server<>reauthentication-latency-max)
aws.msk.kafka.server.socket.server.metrics.request.rate (gauge)	The number of requests sent per second (kafka.server<>request-rate)
aws.msk.kafka.server.socket.server.metrics.request.size.avg (gauge)	The average size of requests sent. (kafka.server<>request-size-avg)
aws.msk.kafka.server.socket.server.metrics.request.size.max (gauge)	The maximum size of any request sent. (kafka.server<>request-size-max)
aws.msk.kafka.server.socket.server.metrics.request.total (gauge)	The total number of requests sent (kafka.server<>request-total)
aws.msk.kafka.server.socket.server.metrics.response.rate (gauge)	The number of responses received per second (kafka.server<>response-rate)
aws.msk.kafka.server.socket.server.metrics.response.total (gauge)	The total number of responses received (kafka.server<>response-total)
aws.msk.kafka.server.socket.server.metrics.select.rate (gauge)	The number of times the I/O layer checked for new I/O to perform per second (kafka.server<>select-rate)
aws.msk.kafka.server.socket.server.metrics.select.total (gauge)	The total number of times the I/O layer checked for new I/O to perform (kafka.server<>select-total)
aws.msk.kafka.server.socket.server.metrics.successful.authentication.no.reauth.total (gauge)	The total number of connections with successful authentication where the client does not support re-authentication (kafka.server<>successful-authentication-no-reauth-total)
aws.msk.kafka.server.socket.server.metrics.successful.authentication.rate (gauge)	The number of connections with successful authentication per second (kafka.server<>successful-authentication-rate)
aws.msk.kafka.server.socket.server.metrics.successful.authentication.total (gauge)	The total number of connections with successful authentication (kafka.server<>successful-authentication-total)
aws.msk.kafka.server.socket.server.metrics.successful.reauthentication.rate (gauge)	The number of successful re-authentication of connections per second (kafka.server<>successful-reauthentication-rate)
aws.msk.kafka.server.socket.server.metrics.successful.reauthentication.total (gauge)	The total number of successful re-authentication of connections (kafka.server<>successful-reauthentication-total)
aws.msk.kafka.server.txn.marker.channel.metrics.connection.close.rate (gauge)	The number of connections closed per second (kafka.server<>connection-close-rate)
aws.msk.kafka.server.txn.marker.channel.metrics.connection.close.total (gauge)	The total number of connections closed (kafka.server<>connection-close-total)
aws.msk.kafka.server.txn.marker.channel.metrics.connection.count (gauge)	The current number of active connections. (kafka.server<>connection-count)
aws.msk.kafka.server.txn.marker.channel.metrics.connection.creation.rate (gauge)	The number of new connections established per second (kafka.server<>connection-creation-rate)
aws.msk.kafka.server.txn.marker.channel.metrics.connection.creation.total (gauge)	The total number of new connections established (kafka.server<>connection-creation-total)
aws.msk.kafka.server.txn.marker.channel.metrics.failed.authentication.rate (gauge)	The number of connections with failed authentication per second (kafka.server<>failed-authentication-rate)
aws.msk.kafka.server.txn.marker.channel.metrics.failed.authentication.total (gauge)	The total number of connections with failed authentication (kafka.server<>failed-authentication-total)
aws.msk.kafka.server.txn.marker.channel.metrics.failed.reauthentication.rate (gauge)	The number of failed re-authentication of connections per second (kafka.server<>failed-reauthentication-rate)
aws.msk.kafka.server.txn.marker.channel.metrics.failed.reauthentication.total (gauge)	The total number of failed re-authentication of connections (kafka.server<>failed-reauthentication-total)
aws.msk.kafka.server.txn.marker.channel.metrics.incoming.byte.rate (gauge)	The number of bytes read off all sockets per second (kafka.server<>incoming-byte-rate)
aws.msk.kafka.server.txn.marker.channel.metrics.incoming.byte.total (gauge)	The total number of bytes read off all sockets (kafka.server<>incoming-byte-total)
aws.msk.kafka.server.txn.marker.channel.metrics.io.ratio (gauge)	The fraction of time the I/O thread spent doing I/O (kafka.server<>io-ratio)
aws.msk.kafka.server.txn.marker.channel.metrics.io.time.ns.avg (gauge)	The average length of time for I/O per select call in nanoseconds. (kafka.server<>io-time-ns-avg)
aws.msk.kafka.server.txn.marker.channel.metrics.io.wait.ratio (gauge)	The fraction of time the I/O thread spent waiting (kafka.server<>io-wait-ratio)
aws.msk.kafka.server.txn.marker.channel.metrics.io.wait.time.ns.avg (gauge)	The average length of time the I/O thread spent waiting for a socket ready for reads or writes in nanoseconds. (kafka.server<>io-wait-time-ns-avg)
aws.msk.kafka.server.txn.marker.channel.metrics.io.waittime.total (gauge)	The total time the I/O thread spent waiting (kafka.server<>io-waittime-total)
aws.msk.kafka.server.txn.marker.channel.metrics.iotime.total (gauge)	The total time the I/O thread spent doing I/O (kafka.server<>iotime-total)
aws.msk.kafka.server.txn.marker.channel.metrics.network.io.rate (gauge)	The number of network operations (reads or writes) on all connections per second (kafka.server<>network-io-rate)
aws.msk.kafka.server.txn.marker.channel.metrics.network.io.total (gauge)	The total number of network operations (reads or writes) on all connections (kafka.server<>network-io-total)
aws.msk.kafka.server.txn.marker.channel.metrics.outgoing.byte.rate (gauge)	The number of outgoing bytes sent to all servers per second (kafka.server<>outgoing-byte-rate)
aws.msk.kafka.server.txn.marker.channel.metrics.outgoing.byte.total (gauge)	The total number of outgoing bytes sent to all servers (kafka.server<>outgoing-byte-total)
aws.msk.kafka.server.txn.marker.channel.metrics.reauthentication.latency.avg (gauge)	The average latency observed due to re-authentication (kafka.server<>reauthentication-latency-avg)
aws.msk.kafka.server.txn.marker.channel.metrics.reauthentication.latency.max (gauge)	The max latency observed due to re-authentication (kafka.server<>reauthentication-latency-max)
aws.msk.kafka.server.txn.marker.channel.metrics.request.rate (gauge)	The number of requests sent per second (kafka.server<>request-rate)
aws.msk.kafka.server.txn.marker.channel.metrics.request.size.avg (gauge)	The average size of requests sent. (kafka.server<>request-size-avg)
aws.msk.kafka.server.txn.marker.channel.metrics.request.size.max (gauge)	The maximum size of any request sent. (kafka.server<>request-size-max)
aws.msk.kafka.server.txn.marker.channel.metrics.request.total (gauge)	The total number of requests sent (kafka.server<>request-total)
aws.msk.kafka.server.txn.marker.channel.metrics.response.rate (gauge)	The number of responses received per second (kafka.server<>response-rate)
aws.msk.kafka.server.txn.marker.channel.metrics.response.total (gauge)	The total number of responses received (kafka.server<>response-total)
aws.msk.kafka.server.txn.marker.channel.metrics.select.rate (gauge)	The number of times the I/O layer checked for new I/O to perform per second (kafka.server<>select-rate)
aws.msk.kafka.server.txn.marker.channel.metrics.select.total (gauge)	The total number of times the I/O layer checked for new I/O to perform (kafka.server<>select-total)
aws.msk.kafka.server.txn.marker.channel.metrics.successful.authentication.no.reauth.total (gauge)	The total number of connections with successful authentication where the client does not support re-authentication (kafka.server<>successful-authentication-no-reauth-total)
aws.msk.kafka.server.txn.marker.channel.metrics.successful.authentication.rate (gauge)	The number of connections with successful authentication per second (kafka.server<>successful-authentication-rate)
aws.msk.kafka.server.txn.marker.channel.metrics.successful.authentication.total (gauge)	The total number of connections with successful authentication (kafka.server<>successful-authentication-total)
aws.msk.kafka.server.txn.marker.channel.metrics.successful.reauthentication.rate (gauge)	The number of successful re-authentication of connections per second (kafka.server<>successful-reauthentication-rate)
aws.msk.kafka.server.txn.marker.channel.metrics.successful.reauthentication.total (gauge)	The total number of successful re-authentication of connections (kafka.server<>successful-reauthentication-total)
aws.msk.kafka.utils.Throttler.Count (gauge)	Attribute exposed for management (kafka.utils<>Count)
aws.msk.kafka.utils.Throttler.FifteenMinuteRate (gauge)	Attribute exposed for management (kafka.utils<>FifteenMinuteRate)
aws.msk.kafka.utils.Throttler.FiveMinuteRate (gauge)	Attribute exposed for management (kafka.utils<>FiveMinuteRate)
aws.msk.kafka.utils.Throttler.MeanRate (gauge)	Attribute exposed for management (kafka.utils<>MeanRate)
aws.msk.kafka.utils.Throttler.OneMinuteRate (gauge)	Attribute exposed for management (kafka.utils<>OneMinuteRate)
aws.msk.node.cpu.seconds.total (count)	Seconds the cpus spent in each mode.
aws.msk.node.exporter.build.info (gauge)	A metric with a constant '1' value labeled by version, revision, branch, and goversion from which node_exporter was built.
aws.msk.node.filesystem.avail.bytes (gauge)	Filesystem space available to non-root users in bytes. Shown as byte
aws.msk.node.filesystem.device.error (gauge)	Whether an error occurred while getting statistics for the given device.
aws.msk.node.filesystem.files (gauge)	Filesystem total file nodes.
aws.msk.node.filesystem.files.free (gauge)	Filesystem total free file nodes.
aws.msk.node.filesystem.free.bytes (gauge)	Filesystem free space in bytes. Shown as byte
aws.msk.node.filesystem.readonly (gauge)	Filesystem read-only status.
aws.msk.node.filesystem.size.bytes (gauge)	Filesystem size in bytes. Shown as byte
aws.msk.node.scrape.collector.duration.seconds (gauge)	node_exporter: Duration of a collector scrape. Shown as second
aws.msk.node.scrape.collector.success (gauge)	node_exporter: Whether a collector succeeded.
aws.msk.process.cpu.seconds.total (count)	Total user and system CPU time spent in seconds.
aws.msk.process.max.fds (gauge)	Maximum number of open file descriptors.
aws.msk.process.open.fds (gauge)	Number of open file descriptors.
aws.msk.process.resident.memory.bytes (gauge)	Resident memory size in bytes. Shown as byte
aws.msk.process.start.time.seconds (gauge)	Start time of the process since unix epoch in seconds. Shown as second
aws.msk.process.virtual.memory.bytes (gauge)	Virtual memory size in bytes. Shown as byte
aws.msk.process.virtual.memory.max.bytes (gauge)	Maximum amount of virtual memory available in bytes. Shown as byte
aws.msk.promhttp.metric.handler.requests.in.flight (gauge)	Current number of scrapes being served.
aws.msk.promhttp.metric.handler.requests.total (count)	Total number of scrapes by HTTP status code.
aws.msk.kafka.network.request.ErrorsPerSec (gauge)	Shown as error
aws.msk.kafka.network.request.LocalTimeMs (gauge)	Shown as millisecond
aws.msk.kafka.network.request.MessageConversionsTimeMs (gauge)	Shown as millisecond
aws.msk.kafka.network.request.RemoteTimeMs (gauge)	Shown as millisecond
aws.msk.kafka.network.request.RequestBytes (gauge)	Shown as byte
aws.msk.kafka.network.request.RequestQueueTimeMs (gauge)	Shown as millisecond
aws.msk.kafka.network.request.RequestsPerSec (gauge)	Shown as request
aws.msk.kafka.network.request.ResponseQueueTimeMs (gauge)	Shown as millisecond
aws.msk.kafka.network.request.ResponseSendTimeMs (gauge)	Shown as millisecond
aws.msk.kafka.network.request.TemporaryMemoryBytes (gauge)	Shown as byte
aws.msk.kafka.network.request.ThrottleTimeMs (gauge)	Shown as millisecond
aws.msk.kafka.network.request.TotalTimeMs (gauge)	Shown as millisecond
aws.msk.kafka.server.broker_topics.BytesInPerSec (gauge)	Shown as byte
aws.msk.kafka.server.broker_topics.BytesOutPerSec (gauge)	Shown as byte
aws.msk.kafka.server.broker_topics.BytesRejectedPerSec (gauge)	Shown as byte
aws.msk.kafka.server.broker_topics.FailedFetchRequestsPerSec (gauge)	Shown as request
aws.msk.kafka.server.broker_topics.FailedProduceRequestsPerSec (gauge)	Shown as request
aws.msk.kafka.server.broker_topics.FetchMessageConversionsPerSec (gauge)
aws.msk.kafka.server.broker_topics.MessagesInPerSec (gauge)	Shown as message
aws.msk.kafka.server.broker_topics.ProduceMessageConversionsPerSec (gauge)
aws.msk.kafka.server.broker_topics.ReplicationBytesInPerSec (gauge)	Shown as byte
aws.msk.kafka.server.broker_topics.ReplicationBytesOutPerSec (gauge)	Shown as byte
aws.msk.kafka.server.broker_topics.TotalFetchRequestsPerSec (gauge)	Shown as byte
aws.msk.kafka.server.broker_topics.TotalProduceRequestsPerSec (gauge)	Shown as byte
aws.msk.kafka.server.replica_manager.LeaderCount (gauge)
aws.msk.kafka.server.replica_manager.OfflineReplicaCount (gauge)
aws.msk.kafka.server.replica_manager.PartitionCount (gauge)
aws.msk.kafka.server.replica_manager.UnderMinIsrPartitionCount (gauge)
aws.msk.kafka.server.replica_manager.UnderReplicatedPartitions (gauge)

サービスのチェック

aws.msk.can_connect:
Agent が MSK クラスターのノードを発見できない場合は CRITICAL を返します。それ以外の場合は OK を返します。

aws.msk.prometheus.health:
チェックがメトリクスのエンドポイントにアクセスできない場合は CRITICAL を返します。それ以外の場合は OK を返します。

イベント

Amazon MSK チェックには、イベントは含まれません。

Crawler

このインテグレーションを有効にすると、CloudWatch からの MSK メトリクスを Datadog に表示できます。

セットアップ

インストール

Amazon Web Services インテグレーションをまだセットアップしていない場合は、最初にセットアップします。

メトリクスの収集

AWS インテグレーションページで、Metric Collection タブの下にある Kafka が有効になっていることを確認します。
Datadog - Amazon MSK インテグレーションをインストールします。

ログの収集

ログの有効化

Amazon MSK から S3 バケットまたは CloudWatch のいずれかにログを送信するよう構成します。

注:

S3 バケットにログを送る場合は、Target prefix が amazon_msk に設定されているかを確認してください。
CloudWatch のロググループにログを送る場合は、その名前に msk という部分文字列が含まれていることを確認してください。

ログを Datadog に送信する方法

Datadog ログコレクション AWS Lambda 関数をまだ実行していない場合は、セットアップします。
Lambda 関数がインストールされたら、AWS コンソールから、Amazon MSK ログを含む S3 バケットまたは CloudWatch のロググループに手動でトリガーを追加します。
- S3 バケットに手動トリガーを追加
- CloudWatch ロググループに手動トリガーを追加

収集データ

aws.kafka.zookeeper_request_latency_ms_mean （gauge）	ブローカーからの ZooKeeper リクエストの平均レイテンシー（ミリ秒）。
aws.kafka.active_controller_count （gauge）	クラスターごとに、常に 1 つのコントローラーだけをアクティブにする必要があります。
aws.kafka.global_partition_count （gauge）	クラスター内のすべてのブローカーのパーティションの合計数。
aws.kafka.global_topic_count （gauge）	クラスター内のすべてのブローカーのトピックの合計数。
aws.kafka.offline_partitions_count （gauge）	クラスター内でオフラインになっているパーティションの合計数。
aws.kafka.swap_used （gauge）	ブローカーに使用されているスワップメモリのサイズ。バイト数で表示
aws.kafka.swap_free （gauge）	ブローカーで使用可能なスワップメモリのサイズ。バイト数で表示
aws.kafka.memory_used （gauge）	ブローカーに使用されているメモリのサイズ。バイト数で表示
aws.kafka.memory_buffered （gauge）	ブローカーに使用されているバッファメモリのサイズ。バイト数で表示
aws.kafka.memory_free （gauge）	ブローカーが使用可能な空きメモリのサイズ。バイト数で表示
aws.kafka.memory_cached （gauge）	ブローカーのキャッシュメモリのサイズ。バイト数で表示
aws.kafka.cpu_user （gauge）	ユーザースペースの CPU の割合。パーセントで表示
aws.kafka.cpu_system （gauge）	カーネルスペースの CPU の割合。パーセントで表示
aws.kafka.cpu_idle （gauge）	CPU アイドル時間の割合。パーセントで表示
aws.kafka.root_disk_used （gauge）	ブローカーに使用されているルートディスクの割合。パーセントで表示
aws.kafka.kafka_app_logs_disk_used （gauge）	アプリケーションログに使用されるディスク容量の割合。パーセントで表示
aws.kafka.kafka_data_logs_disk_used （gauge）	データログに使用されているディスク容量の割合。パーセントで表示
aws.kafka.network_rx_errors （count）	ブローカーのネットワーク受信エラーの数。
aws.kafka.network_tx_errors （count）	ブローカーのネットワーク送信エラーの数。
aws.kafka.network_rx_dropped （count）	ドロップされた受信パッケージの数。
aws.kafka.network_tx_dropped （count）	ドロップされた送信パッケージの数。
aws.kafka.network_rx_packets （count）	ブローカーによって受信されたパケットの数。
aws.kafka.network_tx_packets （count）	ブローカーによって送信されたパケットの数。
aws.kafka.messages_in_per_sec （gauge）	ブローカーの 1 秒あたりの受信メッセージ数。
aws.kafka.network_processor_avg_idle_percent （gauge）	ネットワークプロセッサがアイドル状態の平均時間の割合。
aws.kafka.request_handler_avg_idle_percent （gauge）	リクエストハンドラーのスレッドがアイドル状態の平均時間の割合。
aws.kafka.leader_count （gauge）	リーダーレプリカの数。
aws.kafka.partition_count （gauge）	ブローカーのパーティションの数。
aws.kafka.produce_local_time_ms_mean （gauge）	フォロワーが応答を送信するまでの平均時間。ミリ秒で表示
aws.kafka.produce_message_conversions_time_ms_mean （gauge）	メッセージ形式の変換に費やされた平均時間。ミリ秒で表示
aws.kafka.produce_request_queue_time_ms_mean （gauge）	リクエストメッセージがキューに費やした平均時間。ミリ秒で表示
aws.kafka.produce_response_queue_time_ms_mean （gauge）	応答メッセージがキューに費やした平均時間。ミリ秒で表示
aws.kafka.produce_response_send_time_ms_mean （gauge）	応答メッセージの送信に費やされた平均時間。ミリ秒で表示
aws.kafka.produce_total_time_ms_mean （gauge）	ミリ秒単位の平均生成時間。ミリ秒で表示
aws.kafka.request_bytes_mean （gauge）	ブローカーのリクエストバイトの平均数。
aws.kafka.under_minlsr_partition_count （gauge）	ブローカーの minIsr 未満パーティションの数。
aws.kafka.under_replicated_partitions （gauge）	ブローカーのレプリケートされていないパーティションの数。
aws.kafka.bytes_in_per_sec （gauge）	クライアントから受信した 1 秒あたりのバイト数。バイト数で表示
aws.kafka.bytes_out_per_sec （gauge）	クライアントに送信された 1 秒あたりのバイト数。バイト数で表示
aws.kafka.messages_in_per_sec （gauge）	クライアントから受信した 1 秒あたりのメッセージ数。
aws.kafka.fetch_message_conversions_per_sec （gauge）	ブローカーの 1 秒あたりのフェッチメッセージ変換回数。
aws.kafka.produce_message_conversions_per_sec （gauge）	ブローカーの 1 秒あたりの生成メッセージ変換回数。
aws.kafka.fetch_consumer_total_time_ms_mean （gauge）	コンシューマーがブローカーからデータを取得するのに費やす平均合計時間。ミリ秒で表示
aws.kafka.fetch_follower_total_time_ms_mean （gauge）	フォロワーがブローカーからデータを取得するのに費やす平均合計時間。ミリ秒で表示
aws.kafka.fetch_consumer_request_queue_time_ms_mean （gauge）	コンシューマーのリクエストがリクエストキューで待機する平均時間。ミリ秒で表示
aws.kafka.fetch_follower_request_queue_time_ms_mean （gauge）	フォロワーのリクエストがリクエストキューで待機する平均時間。ミリ秒で表示
aws.kafka.fetch_consumer_local_time_ms_mean （gauge）	コンシューマーのリクエストがリーダーで処理される平均時間。ミリ秒で表示
aws.kafka.fetch_follower_local_time_ms_mean （gauge）	フォロワーのリクエストがリーダーで処理される平均時間。ミリ秒で表示
aws.kafka.fetch_consumer_response_queue_time_ms_mean （gauge）	コンシューマーのリクエストが応答キューで待機する平均時間。ミリ秒で表示
aws.kafka.fetch_follower_response_queue_time_ms_mean （gauge）	フォロワーのリクエストが応答キューで待機する平均時間。ミリ秒で表示
aws.kafka.consumer_response_send_time_ms_mean （gauge）	コンシューマーが応答を送信するための平均時間。ミリ秒で表示
aws.kafka.fetch_follower_response_send_time_ms_mean （gauge）	フォロワーが応答を送信するための平均時間。ミリ秒で表示
aws.kafka.produce_throttle_time （gauge）	平均スロットル生成時間。ミリ秒で表示
aws.kafka.produce_throttle_byte_rate （gauge）	1 秒あたりのスロットルバイト数。
aws.kafka.produce_throttle_queue_size （gauge）	スロットルキュー内のメッセージ数。
aws.kafka.fetch_throttle_time （gauge）	スロットル取得の平均時間ミリ秒で表示
aws.kafka.fetch_throttle_byte_rate （gauge）	1 秒あたりのスロットルバイト数。
aws.kafka.fetch_throttle_queue_size （gauge）	スロットルキュー内のメッセージ数。
aws.kafka.request_throttle_time （gauge）	リクエストスロットルの平均時間。ミリ秒で表示
aws.kafka.request_time （gauge）	ブローカーネットワークおよび I/O スレッドでリクエストを処理するのに費やされた平均時間。
aws.kafka.request_throttle_queue_size （gauge）	スロットルキュー内のメッセージ数。
aws.kafka.request_exempt_from_throttle_time （gauge）	ブローカーネットワークおよび I/O スレッドで、スロットリングから除外されたリクエストを処理するのに費やされた平均時間。

Amazon MSK クローラーには、イベントやサービスのチェック機能は含まれません。

トラブルシューティング

ご不明な点は、Datadog のサポートチームまでお問合せください。