Google Bigtable

Overview

Bigtable is Google’s NoSQL Big Data database service. It’s the same database that powers many core Google services, including Search, Analytics, Maps, and Gmail.

Use the Datadog Google Cloud Platform integration to collect metrics from Google Bigtable.

Setup

Installation

If you haven’t already, set up the Google Cloud Platform integration first. There are no other installation steps.

Log collection

Google Bigtable logs are collected with Google Cloud Logging and sent to a Dataflow job through a Cloud Pub/Sub topic. If you haven’t already, set up logging with the Datadog Dataflow template.

Once this is done, export your Google Bigtable logs from Google Cloud Logging to the pub/sub:

  1. Go to the Google Cloud Logging page and filter the Google Bigtable logs.
  2. Click Create Export and name the sink.
  3. Choose “Cloud Pub/Sub” as the destination and select the pub/sub that was created for that purpose. Note: The pub/sub can be located in a different project.
  4. Click Create and wait for the confirmation message to show up.

Data Collected

Metrics

gcp.bigtable.backup.bytes_used
(gauge)		Backup storage used.
Shown as byte
gcp.bigtable.autoscaling.max_node_count
(gauge)		Maximum number of nodes in an autoscaled cluster.
Shown as node
gcp.bigtable.autoscaling.min_node_count
(gauge)		Minimum number of nodes in an autoscaled cluster.
Shown as node
gcp.bigtable.autoscaling.recommended_node_count_for_cpu
(gauge)		Recommended number of nodes in an autoscaled cluster based on CPU usage.
Shown as node
gcp.bigtable.autoscaling.recommended_node_count_for_storage
(gauge)		Recommended number of nodes in an autoscaled cluster based on storage usage.
Shown as node
gcp.bigtable.cluster.cpu_load
(gauge)		CPU load of a cluster.
gcp.bigtable.cluster.cpu_load_by_app_profile_by_method_by_table
(gauge)		CPU load of a cluster split by app profile, method, and table.
gcp.bigtable.cluster.cpu_load_hottest_node
(gauge)		CPU load of the busiest node in a cluster.
gcp.bigtable.cluster.disk_load
(gauge)		Utilization of HDD disks in a cluster.
gcp.bigtable.cluster.node_count
(gauge)		Number of nodes in a cluster.
Shown as node
gcp.bigtable.cluster.storage_utilization
(gauge)		Storage used as a fraction of total storage capacity.
gcp.bigtable.disk.bytes_used
(gauge)		Amount of compressed data for tables stored in a cluster.
Shown as byte
gcp.bigtable.disk.storage_capacity
(gauge)		Capacity of compressed data for tables that can be stored in a cluster.
Shown as byte
gcp.bigtable.replication.latencies.avg
(gauge)		Distribution of replication request latencies for a table.
Shown as millisecond
gcp.bigtable.replication.latencies.samplecount
(gauge)		Sample count for replication request latencies.
Shown as sample
gcp.bigtable.replication.latencies.sumsqdev
(gauge)		Sum of squared deviation for replication request latencies.
Shown as second
gcp.bigtable.replication.latency.avg
(gauge)		Distribution of replication request latencies for a table.
Shown as millisecond
gcp.bigtable.replication.latency.samplecount
(gauge)		Sample count for replication request latencies.
Shown as sample
gcp.bigtable.replication.latency.sumsqdev
(gauge)		Sum of squared deviation for replication request latencies.
Shown as second
gcp.bigtable.replication.max_delay
(gauge)		Upper bound for replication delay between clusters of a table.
Shown as second
gcp.bigtable.server.error_count
(count)		Number of server requests for a table that failed with an error.
Shown as error
gcp.bigtable.server.latencies.avg
(gauge)		Distribution of replication request latencies for a table.
Shown as millisecond
gcp.bigtable.server.latencies.samplecount
(gauge)		Sample count for replication request latencies.
Shown as sample
gcp.bigtable.server.latencies.sumsqdev
(gauge)		Sum of squared deviation for replication request latencies.
Shown as second
gcp.bigtable.server.modified_rows_count
(count)		Number of rows modified by server requests for a table.
Shown as row
gcp.bigtable.server.multi_cluster_failovers_count
(count)		Number of failovers during multi-cluster requests.
gcp.bigtable.server.received_bytes_count
(count)		Number of uncompressed bytes of request data received by servers for a table.
Shown as byte
gcp.bigtable.server.request_count
(count)		Number of server requests for a table.
Shown as request
gcp.bigtable.server.returned_rows_count
(count)		Number of rows returned by server requests for a table.
Shown as row
gcp.bigtable.server.sent_bytes_count
(count)		Number of uncompressed bytes of response data sent by servers for a table.
Shown as byte
gcp.bigtable.table.bytes_used
(gauge)		Amount of compressed data stored in a table.
Shown as byte

Events

The Google Bigtable integration does not include any events.

Service Checks

The Google Bigtable integration does not include any service checks.

Troubleshooting

Need help? Contact Datadog support.