Amazon Kinesis は、膨大な分散型データストリームをリアルタイムに処理するクラウドベースのフルマネージド型サービスです。
このインテグレーションを有効にすると、すべての Kinesis メトリクスが Datadog に表示され、カスタム Kinesis タグが収集されます。
Amazon Web Services インテグレーションをまだセットアップしていない場合は、最初にセットアップします。これ以外に必要なインストール手順はありません。
AWS インテグレーションページで、Metric Collection タブの下にある Kinesis が有効になっていることを確認します。
Amazon Kinesis のメトリクスを収集するために、次のアクセス許可を Datadog IAM ポリシーに追加します。
kinesis:ListStreams: 使用できるストリームを一覧表示します。kinesis:DescribeStream: Kinesis ストリームのタグと新しいメトリクスを追加します。kinesis:ListTagsForStream: カスタムタグを追加します。詳細については、AWS ウェブサイト上の Kinesis ポリシーを参照してください。
Datadog - Amazon Kinesis インテグレーションをインストールします。
Datadog は、Amazon Kinesis Delivery ストリームのデフォルトの宛先の 1 つです。 AWS は Amazon Kinesis Data Firehose を完全に管理しているため、ストリーミングログ用の追加のインフラストラクチャーや転送構成を維持する必要はありません。
AWS Firehose コンソールで Kinesis Firehose Delivery Stream を設定するか、CloudFormation テンプレートを使用して宛先を自動的に設定できます。
ただし、S3 バケットにログを送る場合は、AWS Lambda 関数を使用します。amazon_kinesis が Target prefix として設定されていることを確認します。
Datadog Forwarder Lambda 関数をまだセットアップしていない場合は、セットアップします。
Lambda 関数がインストールされたら、AWS コンソールから、Amazon Kinesis ログを含む S3 バケットまたは CloudWatch のロググループに手動でトリガーを追加します。
| aws.kinesis.get_records_bytes (gauge) | Average number of bytes per GetRecords operation Shown as byte |
| aws.kinesis.get_records_bytes.maximum (gauge) | Maximum number of bytes per GetRecords operation Shown as byte |
| aws.kinesis.get_records_bytes.minimum (gauge) | Minimum number of bytes per GetRecords operation Shown as byte |
| aws.kinesis.get_records_bytes.sum (count) | Total number of bytes returned over all GetRecords operations Shown as byte |
| aws.kinesis.get_records_iterator_age (gauge) | Average difference between the current time and when the last record of a GetRecords call was written to the stream. Shown as second |
| aws.kinesis.get_records_iterator_age.maximum (gauge) | Maximum difference between the current time and when the last record of a GetRecords call was written to the stream. Shown as second |
| aws.kinesis.get_records_iterator_age_milliseconds (gauge) | Difference between the current time and when the last record of a GetRecords call was written to the stream. Shown as millisecond |
| aws.kinesis.get_records_iterator_age_milliseconds.maximum (gauge) | Maximum difference between the current time and when the last record of a GetRecords call was written to the stream. Shown as millisecond |
| aws.kinesis.get_records_latency (gauge) | Average time taken per GetRecords operation. Shown as millisecond |
| aws.kinesis.get_records_latency.maximum (gauge) | Maximum time taken per GetRecords operation. Shown as millisecond |
| aws.kinesis.get_records_records (gauge) | Average number of records per GetRecords operation Shown as record |
| aws.kinesis.get_records_records.maximum (gauge) | Maximum number of records per GetRecords operation Shown as record |
| aws.kinesis.get_records_records.minimum (gauge) | Minimum number of records per GetRecords operation Shown as record |
| aws.kinesis.get_records_records.sum (count) | Total number of records returned over all GetRecords operations Shown as record |
| aws.kinesis.get_records_success (count) | Sum of successful GetRecords operations per stream. Shown as event |
| aws.kinesis.get_records_success.average (count) | Average number of successful GetRecords operations per stream. Shown as event |
| aws.kinesis.incoming_bytes (gauge) | Average number of bytes successfully put to the Amazon Kinesis stream per operation. Shown as byte |
| aws.kinesis.incoming_bytes.sum (count) | Total number of bytes successfully put to the Amazon Kinesis stream. Shown as byte |
| aws.kinesis.incoming_records (count) | Total number of records sucessfully put to the Amazon Kinesis stream. Shown as record |
| aws.kinesis.incoming_records.average (gauge) | Average number of records sucessfully put to the Amazon Kinesis stream per operation. Shown as record |
| aws.kinesis.iterator_age_milliseconds (gauge) | The age of the last record in all GetRecords calls made against a shard, measured over the specified time period. Shown as millisecond |
| aws.kinesis.outgoing_bytes (gauge) | Average number of bytes retrieved in a GetRecords operation Shown as byte |
| aws.kinesis.outgoing_bytes.sum (count) | Total number of bytes retrieved from the Amazon Kinesis stream Shown as byte |
| aws.kinesis.outgoing_records (count) | The number of records retrieved from the shard, measured over the specified time period. Shown as record |
| aws.kinesis.outgoing_records.average (gauge) | The average number of records retrieved from the shard, measured over the specified time period. Shown as record |
| aws.kinesis.put_record_bytes (gauge) | Average number of bytes per PutRecord operation Shown as byte |
| aws.kinesis.put_record_bytes.maximum (gauge) | Maximum number of bytes per PutRecord operation Shown as byte |
| aws.kinesis.put_record_bytes.minimum (gauge) | Minimum number of bytes per PutRecord operation Shown as byte |
| aws.kinesis.put_record_bytes.sum (count) | Total number of bytes for all PutRecord operation Shown as byte |
| aws.kinesis.put_record_latency (gauge) | Average time taken per PutRecord operation. Shown as millisecond |
| aws.kinesis.put_record_latency.maximum (gauge) | Maximum time taken per PutRecord operation. Shown as millisecond |
| aws.kinesis.put_record_success (count) | Sum of successful PutRecord operations per Amazon Kinesis stream. Shown as event |
| aws.kinesis.put_record_success.average (count) | Average number of successful PutRecord operations per Amazon Kinesis stream. Shown as event |
| aws.kinesis.put_records_bytes (gauge) | Average number of bytes per PutRecords operation. Shown as byte |
| aws.kinesis.put_records_bytes.maximum (gauge) | Maximum number of bytes per PutRecords operation. Shown as byte |
| aws.kinesis.put_records_bytes.minimum (gauge) | Minimum number of bytes per PutRecords operation. Shown as byte |
| aws.kinesis.put_records_bytes.sum (count) | Total number of bytes for all PutRecords operation. Shown as byte |
| aws.kinesis.put_records_failed_records (gauge) | The number of records rejected due to internal failures in a PutRecords operation per Kinesis data stream, summed over the specified time period. Shown as record |
| aws.kinesis.put_records_failed_records.average (gauge) | The number of records rejected due to internal failures in a PutRecords operation per Kinesis data stream, averaged over the specified time period. Shown as record |
| aws.kinesis.put_records_failed_records.maximum (gauge) | The number of records rejected due to internal failures in a PutRecords operation per Kinesis data stream, maximum over the specified time period. Shown as record |
| aws.kinesis.put_records_failed_records.minimum (gauge) | The number of records rejected due to internal failures in a PutRecords operation per Kinesis data stream, minimum over the specified time period. Shown as record |
| aws.kinesis.put_records_latency (gauge) | Average time taken per PutRecords operation. Shown as millisecond |
| aws.kinesis.put_records_latency.maximum (gauge) | Maximum time taken for all PutRecords operation. Shown as millisecond |
| aws.kinesis.put_records_records (gauge) | Average number of records per PutRecords operation Shown as record |
| aws.kinesis.put_records_records.maximum (gauge) | Maximum number of records for PutRecords operations Shown as record |
| aws.kinesis.put_records_records.minimum (gauge) | Minimum number of records for PutRecords operations Shown as record |
| aws.kinesis.put_records_records.sum (count) | Total number of records for PutRecords operations Shown as record |
| aws.kinesis.put_records_success (count) | Sum of successful PutRecords operations per Amazon Kinesis stream. Shown as event |
| aws.kinesis.put_records_success.average (count) | Average number of successful PutRecords operations per Amazon Kinesis stream. Shown as event |
| aws.kinesis.put_records_successful_records (gauge) | The number of successful records in a PutRecords operation per Kinesis data stream, summed over the specified time period. Shown as record |
| aws.kinesis.put_records_successful_records.average (gauge) | The number of successful records in a PutRecords operation per Kinesis data stream, averaged over the specified time period. Shown as record |
| aws.kinesis.put_records_successful_records.maximum (gauge) | The number of successful records in a PutRecords operation per Kinesis data stream, maximum over the specified time period. Shown as record |
| aws.kinesis.put_records_successful_records.minimum (gauge) | The number of successful records in a PutRecords operation per Kinesis data stream, minimum over the specified time period. Shown as record |
| aws.kinesis.put_records_throttled_records (gauge) | The number of records rejected due to throttling in a PutRecords operation per Kinesis data stream, summed over the specified time period. Shown as record |
| aws.kinesis.put_records_throttled_records.average (gauge) | The number of records rejected due to throttling in a PutRecords operation per Kinesis data stream, averaged over the specified time period. Shown as record |
| aws.kinesis.put_records_throttled_records.maximum (gauge) | The number of records rejected due to throttling in a PutRecords operation per Kinesis data stream, maximum over the specified time period. Shown as record |
| aws.kinesis.put_records_throttled_records.minimum (gauge) | The number of records rejected due to throttling in a PutRecords operation per Kinesis data stream, minimum over the specified time period. Shown as record |
| aws.kinesis.put_records_total_records (gauge) | The total number of records sent in a PutRecords operation per Kinesis data stream, summed over the specified time period. Shown as record |
| aws.kinesis.put_records_total_records.average (gauge) | The total number of records sent in a PutRecords operation per Kinesis data stream, averaged over the specified time period. Shown as record |
| aws.kinesis.put_records_total_records.maximum (gauge) | The total number of records sent in a PutRecords operation per Kinesis data stream, maximum over the specified time period. Shown as record |
| aws.kinesis.put_records_total_records.minimum (gauge) | The total number of records sent in a PutRecords operation per Kinesis data stream, minimum over the specified time period. Shown as record |
| aws.kinesis.read_provisioned_throughput_exceeded (count) | Number of GetRecords calls throttled for the stream Shown as record |
| aws.kinesis.read_provisioned_throughput_exceeded.average (count) | Average of GetRecords calls throttled for the stream Shown as record |
| aws.kinesis.read_provisioned_throughput_exceeded.maximum (count) | Maximum number of GetRecords calls throttled for the stream Shown as record |
| aws.kinesis.read_provisioned_throughput_exceeded.minimum (count) | Minimum number of GetRecords calls throttled for the stream Shown as record |
| aws.kinesis.subscribe_to_shard_event_bytes (gauge) | The number of bytes received from the shard, averaged over the specified time period. Shown as byte |
| aws.kinesis.subscribe_to_shard_event_bytes.maximum (gauge) | The number of bytes received from the shard, maximum over the specified time period. Shown as byte |
| aws.kinesis.subscribe_to_shard_event_bytes.minimum (gauge) | The number of bytes received from the shard, minimum over the specified time period. Shown as byte |
| aws.kinesis.subscribe_to_shard_event_bytes.sum (gauge) | The number of bytes received from the shard, summed over the specified time period. Shown as byte |
| aws.kinesis.subscribe_to_shard_event_millis_behind_latest (gauge) | The difference between the current time and when the last record of the SubscribeToShard event was written to the stream. Shown as millisecond |
| aws.kinesis.subscribe_to_shard_event_millis_behind_latest.maximum (gauge) | The difference between the current time and when the last record of the SubscribeToShard event was written to the stream. Shown as millisecond |
| aws.kinesis.subscribe_to_shard_event_records (count) | The number of records received from the shard, averaged over the specified time period. Shown as record |
| aws.kinesis.subscribe_to_shard_event_records.maximum (count) | The number of records received from the shard, maximum over the specified time period. Shown as record |
| aws.kinesis.subscribe_to_shard_event_records.minimum (count) | The number of records received from the shard, minimum over the specified time period. Shown as record |
| aws.kinesis.subscribe_to_shard_event_records.sum (count) | The number of records received from the shard, summed over the specified time period. Shown as record |
| aws.kinesis.subscribe_to_shard_event_success (count) | This metric is emitted every time an event is published successfully. It is only emitted when there's an active subscription. Shown as event |
| aws.kinesis.subscribe_to_shard_event_success.average (count) | This metric is emitted every time an event is published successfully. It is only emitted when there's an active subscription. Shown as event |
| aws.kinesis.subscribe_to_shard_rate_exceeded (count) | This metric is emitted when a new subscription attempt fails because there already is an active subscription by the same consumer or if you exceed the number of calls per second allowed for this operation. Shown as record |
| aws.kinesis.subscribe_to_shard_rate_exceeded.average (count) | This metric is emitted when a new subscription attempt fails because there already is an active subscription by the same consumer or if you exceed the number of calls per second allowed for this operation. Shown as record |
| aws.kinesis.subscribe_to_shard_rate_exceeded.maximum (count) | This metric is emitted when a new subscription attempt fails because there already is an active subscription by the same consumer or if you exceed the number of calls per second allowed for this operation. Shown as record |
| aws.kinesis.subscribe_to_shard_rate_exceeded.minimum (count) | This metric is emitted when a new subscription attempt fails because there already is an active subscription by the same consumer or if you exceed the number of calls per second allowed for this operation. Shown as record |
| aws.kinesis.subscribe_to_shard_success (gauge) | This metric records whether the SubscribeToShard subscription was successfully established. The subscription only lives for at most 5 minutes. Therefore, this metric gets emitted at least once every 5 minutes. Shown as event |
| aws.kinesis.subscribe_to_shard_success.average (gauge) | This metric records whether the SubscribeToShard subscription was successfully established. The subscription only lives for at most 5 minutes. Therefore, this metric gets emitted at least once every 5 minutes. Shown as event |
| aws.kinesis.write_provisioned_throughput_exceeded (count) | Number of records rejected due to throttling for the stream Shown as record |
| aws.kinesis.write_provisioned_throughput_exceeded.average (count) | Average of records rejected due to throttling for the stream Shown as record |
| aws.kinesis.write_provisioned_throughput_exceeded.maximum (count) | Maximum number of records rejected due to throttling for the stream Shown as record |
| aws.kinesis.write_provisioned_throughput_exceeded.minimum (count) | Minimum number of records rejected due to throttling for the stream Shown as record |
AWS から取得される各メトリクスには、ホスト名やセキュリティ グループなど、AWS コンソールに表示されるのと同じタグが割り当てられます。
Amazon Kinesis インテグレーションには、イベントは含まれません。
Amazon Kinesis インテグレーションには、サービスのチェック機能は含まれません。
ご不明な点は、Datadog のサポートチームまでお問合せください。