Amazon Kinesis
セキュリティモニタリングが使用可能です セキュリティモニタリングが使用可能です

Amazon Kinesis

Crawler Crawler

概要

Amazon Kinesis は、膨大な分散型データストリームをリアルタイムに処理するクラウドベースのフルマネージド型サービスです。

このインテグレーションを有効にすると、すべての Kinesis メトリクスが Datadog に表示され、カスタム Kinesis タグが収集されます。

セットアップ

インストール

Amazon Web Services インテグレーションをまだセットアップしていない場合は、最初にセットアップします。これ以外に必要なインストール手順はありません。

メトリクスの収集

  1. AWS インテグレーションタイルのメトリクス収集で、Kinesis をオンにします。
  2. Amazon Kinesis のメトリクスを収集するために、次のアクセス許可を Datadog IAM ポリシーに追加します。

    • kinesis:ListStreams: 使用できるストリームを一覧表示します。
    • kinesis:DescribeStream: Kinesis ストリームのタグと新しいメトリクスを追加します。
    • kinesis:ListTagsForStream: カスタムタグを追加します。

    Kinesis ポリシーの詳細については、AWS Web サイトのガイドを参照してください。

  3. Datadog - AWS Kinesis インテグレーションをインストールします。

ログの収集

ログの有効化

Amazon Kinesis から S3 バケットまたは CloudWatch のいずれかにログを送信するよう構成します。

: S3 バケットにログを送る場合は、Target prefixamazon_kinesis に設定されているかを確認してください。

Datadog へのログの送信

  1. Datadog ログ コレクション AWS Lambda 関数をまだ実行していない場合は、セットアップします。
  2. lambda 関数がインストールされたら、AWS コンソールから、Amazon Kinesis ログを含む S3 バケットまたは CloudWatch のロググループに手動でトリガーを追加します。

収集データ

メトリクス

aws.kinesis.get_records_bytes
(gauge)
Average number of bytes per GetRecords operation
Shown as byte
aws.kinesis.get_records_bytes.maximum
(gauge)
Maximum number of bytes per GetRecords operation
Shown as byte
aws.kinesis.get_records_bytes.minimum
(gauge)
Minimum number of bytes per GetRecords operation
Shown as byte
aws.kinesis.get_records_bytes.sum
(count)
Total number of bytes returned over all GetRecords operations
Shown as byte
aws.kinesis.get_records_iterator_age
(gauge)
Average difference between the current time and when the last record of a GetRecords call was written to the stream.
Shown as second
aws.kinesis.get_records_iterator_age_milliseconds
(gauge)
Difference between the current time and when the last record of a GetRecords call was written to the stream.
Shown as millisecond
aws.kinesis.get_records_iterator_age_milliseconds.maximum
(gauge)
Maximum difference between the current time and when the last record of a GetRecords call was written to the stream.
Shown as millisecond
aws.kinesis.get_records_iterator_age.maximum
(gauge)
Maximum difference between the current time and when the last record of a GetRecords call was written to the stream.
Shown as second
aws.kinesis.get_records_latency
(gauge)
Average time taken per GetRecords operation.
Shown as millisecond
aws.kinesis.get_records_latency.maximum
(gauge)
Maximum time taken per GetRecords operation.
Shown as millisecond
aws.kinesis.get_records_records
(gauge)
Average number of records per GetRecords operation
Shown as record
aws.kinesis.get_records_records.maximum
(gauge)
Maximum number of records per GetRecords operation
Shown as record
aws.kinesis.get_records_records.minimum
(gauge)
Minimum number of records per GetRecords operation
Shown as record
aws.kinesis.get_records_records.sum
(count)
Total number of records returned over all GetRecords operations
Shown as record
aws.kinesis.get_records_success
(count)
Sum of successful GetRecords operations per stream.
Shown as event
aws.kinesis.get_records_success.average
(count)
Average number of successful GetRecords operations per stream.
Shown as event
aws.kinesis.incoming_bytes
(gauge)
Average number of bytes successfully put to the Amazon Kinesis stream per operation.
Shown as byte
aws.kinesis.incoming_bytes.sum
(count)
Total number of bytes successfully put to the Amazon Kinesis stream.
Shown as byte
aws.kinesis.incoming_records
(count)
Total number of records sucessfully put to the Amazon Kinesis stream.
Shown as record
aws.kinesis.incoming_records.average
(gauge)
Average number of records sucessfully put to the Amazon Kinesis stream per operation.
Shown as record
aws.kinesis.iterator_age_milliseconds
(gauge)
The age of the last record in all GetRecords calls made against a shard, measured over the specified time period.
Shown as millisecond
aws.kinesis.outgoing_bytes
(gauge)
Average number of bytes retrieved in a GetRecords operation
Shown as byte
aws.kinesis.outgoing_bytes.sum
(count)
Total number of bytes retrieved from the Amazon Kinesis stream
Shown as byte
aws.kinesis.outgoing_records
(count)
The number of records retrieved from the shard, measured over the specified time period.
Shown as record
aws.kinesis.put_record_bytes
(gauge)
Average number of bytes per PutRecord operation
Shown as byte
aws.kinesis.put_record_bytes.maximum
(gauge)
Maximum number of bytes per PutRecord operation
Shown as byte
aws.kinesis.put_record_bytes.minimum
(gauge)
Minimum number of bytes per PutRecord operation
Shown as byte
aws.kinesis.put_record_bytes.sum
(count)
Total number of bytes for all PutRecord operation
Shown as byte
aws.kinesis.put_record_latency
(gauge)
Average time taken per PutRecord operation.
Shown as millisecond
aws.kinesis.put_record_latency.maximum
(gauge)
Maximum time taken per PutRecord operation.
Shown as millisecond
aws.kinesis.put_record_success
(count)
Sum of successful PutRecord operations per Amazon Kinesis stream.
Shown as event
aws.kinesis.put_record_success.average
(count)
Average number of successful PutRecord operations per Amazon Kinesis stream.
Shown as event
aws.kinesis.put_records_bytes
(gauge)
Average number of bytes per PutRecords operation.
Shown as byte
aws.kinesis.put_records_bytes.maximum
(gauge)
Maximum number of bytes per PutRecords operation.
Shown as byte
aws.kinesis.put_records_bytes.minimum
(gauge)
Minimum number of bytes per PutRecords operation.
Shown as byte
aws.kinesis.put_records_bytes.sum
(count)
Total number of bytes for all PutRecords operation.
Shown as byte
aws.kinesis.put_records_latency
(gauge)
Average time taken per PutRecords operation.
Shown as millisecond
aws.kinesis.put_records_latency.maximum
(gauge)
Maximum time taken for all PutRecords operation.
Shown as millisecond
aws.kinesis.put_records_records
(gauge)
Average number of records per PutRecords operation
Shown as record
aws.kinesis.put_records_records.maximum
(gauge)
Maximum number of records for PutRecords operations
Shown as record
aws.kinesis.put_records_records.minimum
(gauge)
Minimum number of records for PutRecords operations
Shown as record
aws.kinesis.put_records_records.sum
(count)
Total number of records for PutRecords operations
Shown as record
aws.kinesis.put_records_success
(count)
Sum of successful PutRecords operations per Amazon Kinesis stream.
Shown as event
aws.kinesis.put_records_success.average
(count)
Average number of successful PutRecords operations per Amazon Kinesis stream.
Shown as event
aws.kinesis.read_provisioned_throughput_exceeded
(count)
Number of GetRecords calls throttled for the stream
Shown as record
aws.kinesis.read_provisioned_throughput_exceeded.average
(count)
Average of GetRecords calls throttled for the stream
Shown as record
aws.kinesis.read_provisioned_throughput_exceeded.maximum
(count)
Maximum number of GetRecords calls throttled for the stream
Shown as record
aws.kinesis.read_provisioned_throughput_exceeded.minimum
(count)
Minimum number of GetRecords calls throttled for the stream
Shown as record
aws.kinesis.subscribe_to_shard_event.bytes
(gauge)
The number of bytes received from the shard, measured over the specified time period.
Shown as byte
aws.kinesis.subscribe_to_shard_event.millis_behind_latest
(gauge)
The difference between the current time and when the last record of the SubscribeToShard event was written to the stream.
Shown as millisecond
aws.kinesis.subscribe_to_shard_event.records
(count)
The number of records received from the shard, measured over the specified time period.
Shown as record
aws.kinesis.subscribe_to_shard_event.success
(count)
This metric is emitted every time an event is published successfully.
aws.kinesis.subscribe_to_shard.rate_exceeded
(gauge)
This metric is emitted when a new subscription attempt fails because there already is an active subscription by the same consumer or if you exceed the number of calls per second allowed for this operation.
aws.kinesis.subscribe_to_shard.success
(gauge)
This metric records whether the SubscribeToShard subscription was successfully established.
aws.kinesis.write_provisioned_throughput_exceeded
(count)
Number of records rejected due to throttling for the stream
Shown as record
aws.kinesis.write_provisioned_throughput_exceeded.average
(count)
Average of records rejected due to throttling for the stream
Shown as record
aws.kinesis.write_provisioned_throughput_exceeded.maximum
(count)
Maximum number of records rejected due to throttling for the stream
Shown as record
aws.kinesis.write_provisioned_throughput_exceeded.minimum
(count)
Minimum number of records rejected due to throttling for the stream
Shown as record

AWS から取得される各メトリクスには、ホスト名やセキュリティグループなど、AWS コンソールに表示されるタグと同じタグが割り当てられます。

イベント

AWS Kinesis インテグレーションには、イベントは含まれません。

サービスのチェック

AWS AWS Kinesis インテグレーションには、サービスのチェック機能は含まれません。

トラブルシューティング

ご不明な点は、Datadog のサポートチームまでお問合せください。