Amazon Firehose
セキュリティモニタリングが使用可能です セキュリティモニタリングが使用可能です

Amazon Firehose

Crawler Crawler

概要

Amazon Firehose は、AWS にストリーミングデータをロードする最も簡単な方法です。

このインテグレーションを有効にすると、Datadog にすべての Firehose メトリクスを表示できます。

セットアップ

インストール

Amazon Web Services インテグレーションをまだセットアップしていない場合は、最初にセットアップします。

メトリクスの収集

  1. AWS インテグレーションタイルのメトリクス収集で、Firehose をオンにします。
  2. Datadog - AWS Firehose インテグレーションをインストールします。

ログの収集

ログの有効化

Amazon Firehose から S3 バケットまたは CloudWatch のいずれかにログを送信するよう構成します。

: S3 バケットにログを送る場合は、Target prefixamazon_firehose に設定されているかを確認してください。

Datadog へのログの送信

  1. Datadog ログコレクション AWS Lambda 関数 をまだ設定していない場合は、設定を行ってください。
  2. lambda 関数がインストールされたら、AWS コンソールから、Amazon Firehose ログを含む S3 バケットまたは CloudWatch のロググループに手動でトリガーを追加します。

収集データ

メトリクス

aws.firehose.delivery_to_elasticsearch_bytes
(count)
The total number of bytes indexed to Amazon ElasticSearch.
Shown as byte
aws.firehose.delivery_to_elasticsearch_records
(count)
The total number of records indexed to Amazon ElasticSearch.
aws.firehose.delivery_to_elasticsearch_success
(gauge)
Fraction of records successfully indexed to Amazon ElasticSearch.
Shown as fraction
aws.firehose.delivery_to_elasticsearch_success.sum
(count)
Total number of records successfully indexed to Amazon ElasticSearch.
aws.firehose.delivery_to_redshift_bytes
(gauge)
The average number of bytes copied to Amazon Redshift per delivery.
Shown as byte
aws.firehose.delivery_to_redshift_bytes.sum
(count)
The total number of bytes copied to Amazon Redshift.
Shown as byte
aws.firehose.delivery_to_redshift_records
(count)
The total number of records copied to Amazon Redshift.
aws.firehose.delivery_to_redshift_records.average
(gauge)
The average number of records copied to Amazon Redshift per delivery.
aws.firehose.delivery_to_redshift_success
(gauge)
Fraction of successful Redshift COPY commands
Shown as fraction
aws.firehose.delivery_to_redshift_success.sum
(count)
Total number of successful Redshift COPY commands
aws.firehose.delivery_to_s_3bytes
(gauge)
The average number of bytes delivered to Amazon S3 per delivery.
Shown as byte
aws.firehose.delivery_to_s_3bytes.sum
(count)
The total number of bytes delivered to Amazon S3.
Shown as byte
aws.firehose.delivery_to_s_3data_freshness
(gauge)
Age (from getting into Firehose to now) of the oldest record in Firehose. Any record older than this age has been delivered to the S3 bucket.
Shown as second
aws.firehose.delivery_to_s_3records
(count)
The total number of records delivered to Amazon S3.
aws.firehose.delivery_to_s_3records.average
(gauge)
The average number of records delivered to Amazon S3 per delivery.
aws.firehose.delivery_to_s_3success
(gauge)
Fraction of successful deliveries to S3
Shown as fraction
aws.firehose.delivery_to_s_3success.sum
(count)
Sum of successful deliveries to S3
aws.firehose.delivery_to_splunk_bytes
(gauge)
The average number of bytes delivered to Splunk per delivery.
Shown as byte
aws.firehose.delivery_to_splunk_bytes.sum
(count)
The total number of bytes delivered to Splunk.
Shown as byte
aws.firehose.delivery_to_splunk_data_ack_latency
(gauge)
The average approximate duration it takes to receive an acknowledgement from Splunk after Kinesis Data Firehose sends it data.
Shown as second
aws.firehose.delivery_to_splunk_data_ack_latency.maximum
(gauge)
The maximum approximate duration it takes to receive an acknowledgement from Splunk after Kinesis Data Firehose sends it data.
Shown as second
aws.firehose.delivery_to_splunk_data_freshness
(gauge)
Average age (from getting into Firehose to now) of the oldest record in Firehose. Any record older than this age has been delivered to Splunk.
Shown as second
aws.firehose.delivery_to_splunk_data_freshness.maximum
(gauge)
Maximum age (from getting into Firehose to now) of the oldest record in Firehose. Any record older than this age has been delivered to Splunk.
Shown as second
aws.firehose.delivery_to_splunk_records
(count)
The total number of records delivered to Splunk.
aws.firehose.delivery_to_splunk_records.average
(gauge)
The average number of records delivered to Splunk per delivery.
aws.firehose.delivery_to_splunk_success
(gauge)
Fraction of successfully indexed records
Shown as fraction
aws.firehose.delivery_to_splunk_success.maximum
(gauge)
Maximum value for fraction of successfully indexed records
Shown as fraction
aws.firehose.delivery_to_splunk_success.minimum
(gauge)
Minimum value for fraction of successfully indexed records
Shown as fraction
aws.firehose.describe_delivery_stream_latency
(gauge)
The average time taken per DescribeDeliveryStream operation.
Shown as millisecond
aws.firehose.describe_delivery_stream_latency.maximum
(gauge)
The maximum time taken per DescribeDeliveryStream operation.
Shown as millisecond
aws.firehose.describe_delivery_stream_requests
(count)
Total number of DescribeDeliveryStream requests.
aws.firehose.incoming_bytes
(gauge)
The number of bytes ingested into the Firehose delivery stream.
Shown as byte
aws.firehose.incoming_records
(count)
The number of records ingested into the Firehose delivery stream.
aws.firehose.list_delivery_stream_latency
(gauge)
The average time taken per ListDeliveryStream operation.
Shown as millisecond
aws.firehose.list_delivery_stream_latency.maximum
(gauge)
The maximum time taken per ListDeliveryStream operation.
Shown as millisecond
aws.firehose.list_delivery_stream_requests
(count)
Total number of ListDeliveryStream requests.
aws.firehose.list_delivery_streams_latency
(gauge)
The average time taken per ListDeliveryStreams operation.
Shown as millisecond
aws.firehose.list_delivery_streams_latency.maximum
(gauge)
The maximum time taken per ListDeliveryStreams operation.
Shown as millisecond
aws.firehose.list_delivery_streams_requests
(count)
Total number of ListDeliveryStreams requests.
aws.firehose.put_record_bytes
(count)
The total number of bytes put to the Firehose delivery stream using PutRecord.
Shown as byte
aws.firehose.put_record_bytes.average
(gauge)
The average number of bytes per PutRecord request
Shown as byte
aws.firehose.put_record_latency
(gauge)
The average time taken per PutRecord operation.
Shown as millisecond
aws.firehose.put_record_latency.maximum
(gauge)
The maximum time taken per PutRecord operation.
Shown as millisecond
aws.firehose.put_record_requests
(count)
Total number of PutRecord requests.
aws.firehose.put_record_batch_bytes
(count)
The total number of bytes put to the Firehose delivery stream using PutRecordBatch.
Shown as byte
aws.firehose.put_record_batch_bytes.average
(gauge)
The average number of bytes per PutRecordBatch request.
Shown as byte
aws.firehose.put_record_batch_latency
(gauge)
The average time taken per PutRecordBatch operation.
Shown as millisecond
aws.firehose.put_record_batch_latency.maximum
(gauge)
The maximum time taken per PutRecordBatch operation.
Shown as millisecond
aws.firehose.put_record_batch_records
(count)
Total number of records from PutRecordBatch operations.
aws.firehose.put_record_batch_records.average
(gauge)
Average number of records per PutRecordBatch operation.
aws.firehose.put_record_batch_requests
(count)
Total number of PutRecordBatch requests.
aws.firehose.throttled_describe_stream
(count)
Total number of times the DescribeStream operation is throttled when the data source is a Kinesis data stream.
Shown as millisecond
aws.firehose.throttled_get_records
(count)
The total number of times the GetRecords operation is throttled when the data source is a Kinesis data stream.
Shown as millisecond
aws.firehose.throttled_get_shard_iterator
(count)
The total number of times the GetShardIterator operation is throttled when the data source is a Kinesis data stream.
Shown as millisecond
aws.firehose.update_delivery_stream_latency
(gauge)
The average time taken per UpdateDeliveryStream operation.
Shown as millisecond
aws.firehose.update_delivery_stream_latency.maximum
(gauge)
The maximum time taken per UpdateDeliveryStream operation.
Shown as millisecond
aws.firehose.update_delivery_stream_requests
(count)
Total number of UpdateDeliveryStream requests.
aws.firehose.update_firehose_latency
(gauge)
The average time taken per UpdateFirehose operation.
Shown as millisecond
aws.firehose.update_firehose_latency.maximum
(gauge)
The maximum time taken per UpdateFirehose operation.
Shown as millisecond
aws.firehose.update_firehose_requests
(count)
Total number of UpdateFirehose requests.

AWS から取得される各メトリクスには、ホスト名やセキュリティグループなど、AWS コンソールに表示されるタグと同じタグが割り当てられます。

イベント

AWS Kinesis Firehose インテグレーションには、イベントは含まれません。

サービスのチェック

AWS AWS Kinesis Firehose インテグレーションには、サービスのチェック機能は含まれません。

トラブルシューティング

ご不明な点は、Datadog のサポートチームまでお問合せください。