Datadog-AWS Firehose Integration

Overview

Amazon Kinesis Firehose is the easiest way to load streaming data into AWS.

Enable this integration to see in Datadog all your Firehose metrics.

Setup

Installation

If you haven’t already, set up the Amazon Web Services integration first. There are no other installation steps that need to be performed.

Configuration

In the Amazon Web Services integration tile, ensure that Firehose is checked under metric collection.

Data Collected

Metrics

aws.firehose.delivery_to_elasticsearch_bytes
(count)
The total number of bytes indexed to Amazon ElasticSearch.
shown as byte
aws.firehose.delivery_to_elasticsearch_records
(count)
The total number of records indexed to Amazon ElasticSearch.
shown as
aws.firehose.delivery_to_elasticsearch_success
(gauge)
Fraction of records successfully indexed to Amazon ElasticSearch.
shown as fraction
aws.firehose.delivery_to_elasticsearch_success.sum
(count)
Total number of records successfully indexed to Amazon ElasticSearch.
shown as
aws.firehose.delivery_to_redshift_bytes
(gauge)
The average number of bytes copied to Amazon Redshift per delivery.
shown as byte
aws.firehose.delivery_to_redshift_bytes.sum
(count)
The total number of bytes copied to Amazon Redshift.
shown as byte
aws.firehose.delivery_to_redshift_records
(count)
The total number of records copied to Amazon Redshift.
shown as
aws.firehose.delivery_to_redshift_records.average
(gauge)
The average number of records copied to Amazon Redshift per delivery.
shown as
aws.firehose.delivery_to_redshift_success
(gauge)
Fraction of successful Redshift COPY commands
shown as fraction
aws.firehose.delivery_to_redshift_success.sum
(count)
Total number of successful Redshift COPY commands
shown as
aws.firehose.delivery_to_s_3bytes
(gauge)
The average number of bytes delivered to Amazon S3 per delivery.
shown as byte
aws.firehose.delivery_to_s_3bytes.sum
(count)
The total number of bytes delivered to Amazon S3.
shown as byte
aws.firehose.delivery_to_s_3data_freshness
(gauge)
Age (from getting into Firehose to now) of the oldest record in Firehose. Any record older than this age has been delivered to the S3 bucket.
shown as second
aws.firehose.delivery_to_s_3records
(count)
The total number of records delivered to Amazon S3.
shown as
aws.firehose.delivery_to_s_3records.average
(gauge)
The average number of records delivered to Amazon S3 per delivery.
shown as
aws.firehose.delivery_to_s_3success
(gauge)
Fraction of successful deliveries to S3
shown as fraction
aws.firehose.delivery_to_s_3success.sum
(count)
Sum of successful deliveries to S3
shown as
aws.firehose.describe_delivery_stream_latency
(gauge)
The average time taken per DescribeDeliveryStream operation.
shown as millisecond
aws.firehose.describe_delivery_stream_latency.maximum
(gauge)
The maximum time taken per DescribeDeliveryStream operation.
shown as millisecond
aws.firehose.describe_delivery_stream_requests
(count)
Total number of DescribeDeliveryStream requests.
shown as
aws.firehose.incoming_bytes
(gauge)
The number of bytes ingested into the Firehose delivery stream.
shown as byte
aws.firehose.incoming_records
(count)
The number of records ingested into the Firehose delivery stream.
shown as
aws.firehose.list_delivery_stream_latency
(gauge)
The average time taken per ListDeliveryStream operation.
shown as millisecond
aws.firehose.list_delivery_stream_latency.maximum
(gauge)
The maximum time taken per ListDeliveryStream operation.
shown as millisecond
aws.firehose.list_delivery_stream_requests
(count)
Total number of ListDeliveryStream requests.
shown as
aws.firehose.list_delivery_streams_latency
(gauge)
The average time taken per ListDeliveryStreams operation.
shown as millisecond
aws.firehose.list_delivery_streams_latency.maximum
(gauge)
The maximum time taken per ListDeliveryStreams operation.
shown as millisecond
aws.firehose.list_delivery_streams_requests
(count)
Total number of ListDeliveryStreams requests.
shown as
aws.firehose.put_record_bytes
(count)
The total number of bytes put to the Firehose delivery stream using PutRecord.
shown as byte
aws.firehose.put_record_bytes.average
(gauge)
The average number of bytes per PutRecord request
shown as byte
aws.firehose.put_record_latency
(gauge)
The average time taken per PutRecord operation.
shown as millisecond
aws.firehose.put_record_latency.maximum
(gauge)
The maximum time taken per PutRecord operation.
shown as millisecond
aws.firehose.put_record_requests
(count)
Total number of PutRecord requests.
shown as
aws.firehose.put_record_batch_bytes
(count)
The total number of bytes put to the Firehose delivery stream using PutRecordBatch.
shown as byte
aws.firehose.put_record_batch_bytes.average
(gauge)
The average number of bytes per PutRecordBatch request.
shown as byte
aws.firehose.put_record_batch_latency
(gauge)
The average time taken per PutRecordBatch operation.
shown as millisecond
aws.firehose.put_record_batch_latency.maximum
(gauge)
The maximum time taken per PutRecordBatch operation.
shown as millisecond
aws.firehose.put_record_batch_records
(count)
Total number of records from PutRecordBatch operations.
shown as
aws.firehose.put_record_batch_records.average
(gauge)
Average number of records per PutRecordBatch operation.
shown as
aws.firehose.put_record_batch_requests
(count)
Total number of PutRecordBatch requests.
shown as
aws.firehose.update_delivery_stream_latency
(gauge)
The average time taken per UpdateDeliveryStream operation.
shown as millisecond
aws.firehose.update_delivery_stream_latency.maximum
(gauge)
The maximum time taken per UpdateDeliveryStream operation.
shown as millisecond
aws.firehose.update_delivery_stream_requests
(count)
Total number of UpdateDeliveryStream requests.
shown as
aws.firehose.update_firehose_latency
(gauge)
The average time taken per UpdateFirehose operation.
shown as millisecond
aws.firehose.update_firehose_latency.maximum
(gauge)
The maximum time taken per UpdateFirehose operation.
shown as millisecond
aws.firehose.update_firehose_requests
(count)
Total number of UpdateFirehose requests.
shown as

Each of the metrics retrieved from AWS will be assigned the same tags that appear in the AWS console, including but not limited to host name, security-groups, and more.