For AI agents: A markdown version of this page is available at https://docs.datadoghq.com/integrations/google-cloud-bigtable.md. A documentation index is available at /llms.txt.

Google Cloud Bigtable

To find out if this integration is available in your organization, see your Datadog Integrations page or ask your organization administrator.

To initiate an exception request to enable this integration for your organization, email support@ddog-gov.com.

Overview

Bigtable is Google’s NoSQL Big Data database service. It’s the same database that powers many core Google services, including Search, Analytics, Maps, and Gmail.

Use the Datadog Google Cloud Platform integration to collect metrics from Google Bigtable.

Setup

Installation

If you haven’t already, set up the Google Cloud Platform integration first. There are no other installation steps.

Log collection

Google Bigtable logs are collected with Google Cloud Logging and sent to a Dataflow job through a Cloud Pub/Sub topic. If you haven’t already, set up logging with the Datadog Dataflow template.

Once this is done, export your Google Bigtable logs from Google Cloud Logging to the Pub/Sub topic:

  1. Go to the Google Cloud Logging page and filter the Google Bigtable logs.
  2. Click Create Export and name the sink.
  3. Choose “Cloud Pub/Sub” as the destination and select the Pub/Sub topic that was created for that purpose. Note: The Pub/Sub topic can be located in a different project.
  4. Click Create and wait for the confirmation message to show up.

Data Collected

Metrics

gcp.bigtable.backup.bytes_used
(gauge)
Backup storage used in bytes.
Shown as byte
gcp.bigtable.client.application_blocking_latencies.avg
(count)
The average the total latency introduced by your application when Cloud Bigtable has available response data but your application has not consumed it.
Shown as millisecond
gcp.bigtable.client.application_blocking_latencies.samplecount
(count)
The total latency introduced by your application when Cloud Bigtable has available response data but your application has not consumed it.
gcp.bigtable.client.application_blocking_latencies.sumsqdev
(count)
The total latency introduced by your application when Cloud Bigtable has available response data but your application has not consumed it.
gcp.bigtable.client.attempt_latencies.avg
(count)
The average client observed latency per RPC attempt.
Shown as millisecond
gcp.bigtable.client.attempt_latencies.samplecount
(count)
Client observed latency per RPC attempt.
gcp.bigtable.client.attempt_latencies.sumsqdev
(count)
Client observed latency per RPC attempt.
gcp.bigtable.client.client_blocking_latencies.avg
(count)
The average the latency introduced by the client by blocking on sending more requests to the server when there are too many pending requests in bulk operations.
Shown as millisecond
gcp.bigtable.client.client_blocking_latencies.samplecount
(count)
The latency introduced by the client by blocking on sending more requests to the server when there are too many pending requests in bulk operations.
gcp.bigtable.client.client_blocking_latencies.sumsqdev
(count)
The latency introduced by the client by blocking on sending more requests to the server when there are too many pending requests in bulk operations.
gcp.bigtable.client.connectivity_error_count
(count)
Number of requests that failed to reach the Google network. (Requests without google response headers).
gcp.bigtable.client.first_response_latencies.avg
(count)
The average latency from operation start until the response headers were received. The publishing of the measurement will be delayed until the attempt response has been received.
Shown as millisecond
gcp.bigtable.client.first_response_latencies.samplecount
(count)
Latency from operation start until the response headers were received. The publishing of the measurement will be delayed until the attempt response has been received.
gcp.bigtable.client.first_response_latencies.sumsqdev
(count)
Latency from operation start until the response headers were received. The publishing of the measurement will be delayed until the attempt response has been received.
gcp.bigtable.client.operation_latencies.avg
(count)
The average distribution of the total end-to-end latency across all RPC attempts associated with a Bigtable operation.
Shown as millisecond
gcp.bigtable.client.operation_latencies.samplecount
(count)
Distribution of the total end-to-end latency across all RPC attempts associated with a Bigtable operation.
gcp.bigtable.client.operation_latencies.sumsqdev
(count)
Distribution of the total end-to-end latency across all RPC attempts associated with a Bigtable operation.
gcp.bigtable.client.retry_count
(count)
The number of additional RPCs sent after the initial attempt.
gcp.bigtable.client.server_latencies.avg
(count)
The average the latency measured between the time when Google frontend receives an RPC and sending back the first byte of the response.
Shown as millisecond
gcp.bigtable.client.server_latencies.samplecount
(count)
The latency measured between the time when Google frontend receives an RPC and sending back the first byte of the response.
gcp.bigtable.client.server_latencies.sumsqdev
(count)
The latency measured between the time when Google frontend receives an RPC and sending back the first byte of the response.
gcp.bigtable.cluster.autoscaling.max_node_count
(gauge)
Maximum number of nodes in an autoscaled cluster.
gcp.bigtable.cluster.autoscaling.min_node_count
(gauge)
Minimum number of nodes in an autoscaled cluster.
gcp.bigtable.cluster.autoscaling.recommended_node_count_for_cpu
(gauge)
Recommended number of nodes in an autoscaled cluster based on CPU usage.
gcp.bigtable.cluster.autoscaling.recommended_node_count_for_storage
(gauge)
Recommended number of nodes in an autoscaled cluster based on storage usage.
gcp.bigtable.cluster.cpu_load
(gauge)
CPU load of a cluster.
gcp.bigtable.cluster.cpu_load_by_app_profile_by_method_by_table
(gauge)
CPU load of a cluster. Split by app profile, method, and table. Contains the same underlying data as bigtable.googleapis.com/cluster/cpu_load.
gcp.bigtable.cluster.cpu_load_hottest_node
(gauge)
CPU load of the busiest node in a cluster.
gcp.bigtable.cluster.cpu_load_hottest_node_high_granularity
(gauge)
CPU load of the busiest node in a cluster sampled at a high granularity.
gcp.bigtable.cluster.disk_load
(gauge)
Utilization of HDD disks in a cluster.
gcp.bigtable.cluster.node_count
(gauge)
Number of nodes in a cluster.
gcp.bigtable.cluster.storage_utilization
(gauge)
Storage used as a fraction of total storage capacity.
gcp.bigtable.disk.bytes_used
(gauge)
Amount of compressed data for tables stored in a cluster.
Shown as byte
gcp.bigtable.disk.per_node_storage_capacity
(gauge)
Capacity of compressed data for tables that can be stored per node in the cluster.
Shown as byte
gcp.bigtable.disk.storage_capacity
(gauge)
Capacity of compressed data for tables that can be stored in a cluster.
Shown as byte
gcp.bigtable.replication.latency.avg
(gauge)
Distribution of the replication latencies of mutations to a table, measured as the difference between the commit time at the source and destination clusters.
Shown as millisecond
gcp.bigtable.replication.latency.samplecount
(gauge)
Sample count for replication request latencies.
Shown as sample
gcp.bigtable.replication.latency.sumsqdev
(gauge)
Sum of squared deviation for replication request latencies.
Shown as second
gcp.bigtable.replication.max_delay
(gauge)
Upper bound for replication delay between clusters of a table. Indicates the time frame during which latency information may not be accurate.
Shown as second
gcp.bigtable.server.data_boost.eligibility_count
(count)
Current Bigtable requests that are eligible and ineligible for Data Boost.
gcp.bigtable.server.data_boost.ineligible_reasons
(gauge)
Reasons that current traffic is ineligible for Data Boost.
gcp.bigtable.server.data_boost.spu_usage
(gauge)
The Serverless-Processing-Units usage (in SPU-seconds) for Data Boost requests.
gcp.bigtable.server.error_count
(count)
Number of server requests for a table that failed with an error.
gcp.bigtable.server.latencies.avg
(gauge)
Distribution of server request latencies for a table. The latency is measured between the time when Cloud Bigtable (behind the Google frontend) receives an RPC and the time when it sends back the last byte of the response.
Shown as millisecond
gcp.bigtable.server.latencies.samplecount
(gauge)
Sample count for replication request latencies.
Shown as sample
gcp.bigtable.server.latencies.sumsqdev
(gauge)
Sum of squared deviation for replication request latencies.
Shown as second
gcp.bigtable.server.modified_rows_count
(count)
Number of rows modified by server requests for a table.
gcp.bigtable.server.multi_cluster_failovers_count
(count)
Number of failovers during multi-cluster requests.
gcp.bigtable.server.received_bytes_count
(count)
Number of bytes of request data received by servers for a table.
Shown as byte
gcp.bigtable.server.request_count
(count)
Number of server requests for a table.
gcp.bigtable.server.request_max_per_minute_count
(count)
Maximum number of requests received in a five-second span per minute.
gcp.bigtable.server.returned_rows_count
(count)
Number of rows returned by server requests for a table.
gcp.bigtable.server.sent_bytes_count
(count)
Number of bytes of response data sent by servers for a table.
Shown as byte
gcp.bigtable.table.bytes_used
(gauge)
Amount of compressed data stored in a table.
Shown as byte
gcp.bigtable.table.change_stream_log_used_bytes
(gauge)
Amount of disk storage used by the change stream logs.
Shown as byte

Events

The Google Bigtable integration does not include any events.

Service Checks

The Google Bigtable integration does not include any service checks.

Troubleshooting

Need help? Contact Datadog support.