Amazon Kinesis is a fully managed, cloud-based service for real-time processing of large, distributed data streams.
Enable this integration to see in Datadog all your Kinesis metrics, and collect custom Kinesis tags.
If you haven’t already, set up the Amazon Web Services integration first. There are no other installation steps that need to be performed.
In the AWS integration tile, ensure that Kinesis is checked under metric collection.
Add those permissions to your Datadog IAM policy in order to collect Amazon Kinesis metrics:
kinesis:ListStreams: List available streams.kinesis:DescribeStream: Add tags and new metrics for kinesis streams.kinesis:ListTagsForStream: Add custom tags.For more information on Kinesis policies, review the documentation on the AWS website.
Install the Datadog - AWS Kinesis integration.
Configure Amazon Kinesis to send logs either to a S3 bucket or to Cloudwatch.
Note: If you log to a S3 bucket, make sure that amazon_kinesis is set as Target prefix.
Once the lambda function is installed, manually add a trigger on the S3 bucket or Cloudwatch log group that contains your Amazon Kinesis logs in the AWS console:
| aws.kinesis.get_records_bytes (gauge) | Average number of bytes per GetRecords operation Shown as byte |
| aws.kinesis.get_records_bytes.minimum (gauge) | Minimum number of bytes per GetRecords operation Shown as byte |
| aws.kinesis.get_records_bytes.maximum (gauge) | Maximum number of bytes per GetRecords operation Shown as byte |
| aws.kinesis.get_records_bytes.sum (count) | Total number of bytes returned over all GetRecords operations Shown as byte |
| aws.kinesis.get_records_records (gauge) | Average number of records per GetRecords operation Shown as record |
| aws.kinesis.get_records_records.minimum (gauge) | Minimum number of records per GetRecords operation Shown as record |
| aws.kinesis.get_records_records.maximum (gauge) | Maximum number of records per GetRecords operation Shown as record |
| aws.kinesis.get_records_records.sum (count) | Total number of records returned over all GetRecords operations Shown as record |
| aws.kinesis.get_records_iterator_age (gauge) | Average Difference between the current time and when the last record of a GetRecords call was written to the stream. Shown as second |
| aws.kinesis.get_records_iterator_age.maximum (gauge) | Maximum Difference between the current time and when the last record of a GetRecords call was written to the stream. Shown as second |
| aws.kinesis.get_records_iterator_age_milliseconds (gauge) | Difference between the current time and when the last record of a GetRecords call was written to the stream. Shown as millisecond |
| aws.kinesis.get_records_iterator_age_milliseconds.maximum (gauge) | Difference between the current time and when the last record of a GetRecords call was written to the stream. Shown as millisecond |
| aws.kinesis.get_records_latency (gauge) | Average time taken per GetRecords operation. Shown as millisecond |
| aws.kinesis.get_records_latency.maximum (gauge) | Maximum time taken per GetRecords operation. Shown as millisecond |
| aws.kinesis.get_records_success.average (count) | Average number of successful GetRecords operations per stream. Shown as event |
| aws.kinesis.get_records_success (count) | Sum of successful GetRecords operations per stream. Shown as event |
| aws.kinesis.incoming_bytes (gauge) | Average number of bytes successfully put to the Amazon Kinesis stream per operation. Shown as byte |
| aws.kinesis.incoming_bytes.sum (count) | Total number of bytes successfully put to the Amazon Kinesis stream. Shown as byte |
| aws.kinesis.incoming_records (count) | Total number of records sucessfully put to the Amazon Kinesis stream. Shown as record |
| aws.kinesis.incoming_records.average (gauge) | Average number of records sucessfully put to the Amazon Kinesis stream per operation. Shown as record |
| aws.kinesis.outgoing_bytes (gauge) | Average number of bytes retrieved in a GetRecords operation Shown as byte |
| aws.kinesis.outgoing_bytes.sum (count) | Total number of bytes retrieved from the Amazon Kinesis stream Shown as byte |
| aws.kinesis.put_record_bytes (gauge) | Average number of bytes per PutRecord operation Shown as byte |
| aws.kinesis.put_record_bytes.minimum (gauge) | Minimum number of bytes per PutRecord operation Shown as byte |
| aws.kinesis.put_record_bytes.maximum (gauge) | Maximum number of bytes per PutRecord operation Shown as byte |
| aws.kinesis.put_record_bytes.sum (count) | Total number of bytes for all PutRecord operation Shown as byte |
| aws.kinesis.put_record_latency (gauge) | Average time taken per PutRecord operation. Shown as millisecond |
| aws.kinesis.put_record_latency.maximum (gauge) | Maximum time taken per PutRecord operation. Shown as millisecond |
| aws.kinesis.put_record_success (count) | Sum of successful PutRecord operations per Amazon Kinesis stream. Shown as event |
| aws.kinesis.put_record_success.average (count) | Average number of successful PutRecord operations per Amazon Kinesis stream. Shown as event |
| aws.kinesis.put_records_bytes.sum (count) | Total number of bytes for all PutRecords operation. Shown as byte |
| aws.kinesis.put_records_bytes.maximum (gauge) | Maximum number of bytes per PutRecords operation. Shown as byte |
| aws.kinesis.put_records_bytes.minimum (gauge) | Minimum number of bytes per PutRecords operation. Shown as byte |
| aws.kinesis.put_records_bytes (gauge) | Average number of bytes per PutRecords operation. Shown as byte |
| aws.kinesis.put_records_latency (gauge) | Average time taken per PutRecords operation. Shown as millisecond |
| aws.kinesis.put_records_latency.maximum (gauge) | Maximum time taken for all PutRecords operation. Shown as millisecond |
| aws.kinesis.put_records_success.average (count) | Average number of successful PutRecords operations per Amazon Kinesis stream. Shown as event |
| aws.kinesis.put_records_success (count) | Sum of successful PutRecords operations per Amazon Kinesis stream. Shown as event |
| aws.kinesis.put_records_records (gauge) | Average number of records per PutRecords operation Shown as record |
| aws.kinesis.put_records_records.maximum (gauge) | Maximum number of records for PutRecords operations Shown as record |
| aws.kinesis.put_records_records.minimum (gauge) | Minimum number of records for PutRecords operations Shown as record |
| aws.kinesis.put_records_records.sum (count) | Total number of records for PutRecords operations Shown as record |
| aws.kinesis.write_provisioned_throughput_exceeded (count) | Number of records rejected due to throttling for the stream Shown as record |
| aws.kinesis.write_provisioned_throughput_exceeded.average (count) | Average of records rejected due to throttling for the stream Shown as record |
| aws.kinesis.write_provisioned_throughput_exceeded.minimum (count) | Minimum number of records rejected due to throttling for the stream Shown as record |
| aws.kinesis.write_provisioned_throughput_exceeded.maximum (count) | Maximum number of records rejected due to throttling for the stream Shown as record |
| aws.kinesis.read_provisioned_throughput_exceeded (count) | Number of GetRecords calls throttled for the stream Shown as record |
| aws.kinesis.read_provisioned_throughput_exceeded.average (count) | Average of GetRecords calls throttled for the stream Shown as record |
| aws.kinesis.read_provisioned_throughput_exceeded.minimum (count) | Minimum number of GetRecords calls throttled for the stream Shown as record |
| aws.kinesis.read_provisioned_throughput_exceeded.maximum (count) | Maximum number of GetRecords calls throttled for the stream Shown as record |
Each of the metrics retrieved from AWS will be assigned the same tags that appear in the AWS console, including but not limited to host name, security-groups, and more.
The AWS Kinesis integration does not include any events.
The AWS Kinesis integration does not include any service checks.
Need help? Contact Datadog support.