概要

Google Cloud Storage は、ライブデータの提供からデータ分析/ML、データアーカイブまで、さまざまな機能を備えた開発者/企業向けの統合型オブジェクトストレージです。

Google Storage からメトリクスを取得して、以下のことができます。

  • Storage サービスのパフォーマンスを視覚化。
  • Storage サービスのパフォーマンスをアプリケーションと関連付け。

セットアップ

インストール

Google Cloud Platform インテグレーションをまだセットアップしていない場合は、最初にセットアップします。それ以上のインストール手順はありません。

構成

カスタム Cloud Storage ラベルをタグとして収集するには、Cloud Asset Inventory のアクセス権を有効にします。

収集データ

Google Cloud Storage のログは Google Cloud Logging で収集され、Cloud Pub/Sub トピックを通じて Dataflow ジョブに送信されます。まだの場合は、Datadog Dataflow テンプレートでロギングをセットアップしてください

これが完了したら、Google Cloud Storage のログを Google Cloud Logging から Pub/Sub トピックへエクスポートします。

  1. Google Cloud Logging のページに移動し、Google Cloud Storage のログを絞り込みます。
  2. Create Export をクリックし、シンクに名前を付けます。
  3. 宛先として “Cloud Pub/Sub” を選択し、その目的で作成された Pub/Sub トピックを選択します。: Pub/Sub トピックは別のプロジェクトに配置できます。
  4. 作成をクリックし、確認メッセージが表示されるまで待ちます。

収集データ

メトリクス

gcp.storage.api.lro_count
(count)
Count of completed long-running operations, since the last sample.
gcp.storage.api.request_count
(count)
The number of API calls.
Shown as request
gcp.storage.authn.authentication_count
(count)
The number of HMAC/RSA signed requests.
Shown as request
gcp.storage.authz.acl_based_object_access_count
(count)
The number of requests that result in an object being granted access solely due to object ACLs.
Shown as request
gcp.storage.authz.acl_operations_count
(count)
The usage of ACL operations.
Shown as operation
gcp.storage.authz.object_specific_acl_mutation_count
(count)
The number of changes made to object specific ACLs.
gcp.storage.autoclass.transition_operation_count
(count)
Total number of storage class transition operations initiated by Autoclass.
gcp.storage.autoclass.transitioned_bytes_count
(count)
Total bytes transitioned by Autoclass.
Shown as byte
gcp.storage.client.grpc.client.attempt.duration.avg
(gauge)
The average of the cumulative time taken to complete an RPC attempt including the time it takes to pick a subchannel.
Shown as second
gcp.storage.client.grpc.client.attempt.duration.samplecount
(gauge)
The sample count for the cumulative time taken to complete an RPC attempt including the time it takes to pick a subchannel.
Shown as second
gcp.storage.client.grpc.client.attempt.duration.sumsqdev
(gauge)
The sum of squared deviation for the cumulative time taken to complete an RPC attempt including the time it takes to pick a subchannel.
Shown as second
gcp.storage.client.grpc.client.attempt.rcvd_total_compressed_message_size.avg
(count)
The average cumulative bytes (compressed but not encrypted) received across all response messages (metadata excluded) per RPC attempt; does not include gRPC or transport framing bytes.
Shown as byte
gcp.storage.client.grpc.client.attempt.rcvd_total_compressed_message_size.samplecount
(count)
The sample count for the cumulative bytes (compressed but not encrypted) received across all response messages (metadata excluded) per RPC attempt; does not include gRPC or transport framing bytes.
Shown as byte
gcp.storage.client.grpc.client.attempt.rcvd_total_compressed_message_size.sumsqdev
(count)
The sum of squared deviation for the cumulative bytes (compressed but not encrypted) received across all response messages (metadata excluded) per RPC attempt; does not include gRPC or transport framing bytes.
Shown as byte
gcp.storage.client.grpc.client.attempt.sent_total_compressed_message_size.avg
(count)
The average cumulative bytes (compressed but not encrypted) sent across all request messages (metadata excluded) per RPC attempt; does not include gRPC or transport framing bytes.
Shown as byte
gcp.storage.client.grpc.client.attempt.sent_total_compressed_message_size.samplecount
(count)
The sample count for the cumulative bytes (compressed but not encrypted) sent across all request messages (metadata excluded) per RPC attempt; does not include gRPC or transport framing bytes.
Shown as byte
gcp.storage.client.grpc.client.attempt.sent_total_compressed_message_size.sumsqdev
(count)
The sum of squared deviation for the cumulative bytes (compressed but not encrypted) sent across all request messages (metadata excluded) per RPC attempt; does not include gRPC or transport framing bytes.
Shown as byte
gcp.storage.client.grpc.client.attempt.started
(count)
The cumulative number of RPC attempts started, including those that have not completed.
gcp.storage.client.grpc.client.call.duration.avg
(gauge)
The average of the cumulative time the gRPC library takes to complete an RPC from the application’s perspective.
Shown as second
gcp.storage.client.grpc.client.call.duration.samplecount
(gauge)
The sample count for the cumulative time the gRPC library takes to complete an RPC from the application’s perspective.
Shown as second
gcp.storage.client.grpc.client.call.duration.sumsqdev
(gauge)
The sum of squared deviation of the cumulative time the gRPC library takes to complete an RPC from the application’s perspective.
Shown as second
gcp.storage.client.grpc.lb.rls.cache_entries
(gauge)
Number of entries in the RLS cache.
Shown as entry
gcp.storage.client.grpc.lb.rls.cache_size
(gauge)
The current size of the RLS cache.
Shown as byte
gcp.storage.client.grpc.lb.rls.default_target_picks
(count)
Cumulative number of LB picks sent to the default target.
gcp.storage.client.grpc.lb.rls.failed_picks
(count)
Cumulative number of LB picks failed due to either a failed RLS request or the RLS channel being throttled.
gcp.storage.client.grpc.lb.rls.target_picks
(count)
Cumulative number of LB picks sent to each RLS target. Note that if the default target is also returned by the RLS server, RPCs sent to that target from the cache are counted in this metric, not in grpc.rls.default_target_picks.
gcp.storage.client.grpc.lb.wrr.endpoint_weight_not_yet_usable
(count)
Cumulative number of endpoints from each scheduler update that do not yet have usable weight information (either the load report has not yet been received, or it is within the blackout period).
gcp.storage.client.grpc.lb.wrr.endpoint_weight_stale
(count)
Cumulative number of endpoints from each scheduler update whose latest weight is older than the expiration period.
gcp.storage.client.grpc.lb.wrr.endpoint_weights.avg
(count)
The average of the cumulative bucket counts. The histogram buckets will be endpoint weight ranges. Each bucket will be a counter that is incremented once for every endpoint whose weight is within that range. Note that endpoints without usable weights will have weight 0.
gcp.storage.client.grpc.lb.wrr.endpoint_weights.samplecount
(count)
The sample count for the cumulative bucket counts. The histogram buckets will be endpoint weight ranges. Each bucket will be a counter that is incremented once for every endpoint whose weight is within that range. Note that endpoints without usable weights will have weight 0.
gcp.storage.client.grpc.lb.wrr.endpoint_weights.sumsqdev
(count)
The sum of squared deviation for the cumulative bucket counts. The histogram buckets will be endpoint weight ranges. Each bucket will be a counter that is incremented once for every endpoint whose weight is within that range. Note that endpoints without usable weights will have weight 0.
gcp.storage.client.grpc.lb.wrr.rr_fallback
(count)
Cumulative number of scheduler updates in which there were not enough endpoints with valid weight, which caused the WRR policy to fall back to RR behavior.
gcp.storage.client.grpc.xds_client.connected
(gauge)
Whether or not the xDS client currently has a working ADS stream to the xDS server. For a given server, this will be set to 1 when the stream is initially created. It will be set to 0 when we have a connectivity failure or when the ADS stream fails without seeing a response message, as per A57. Once set to 0, it will be reset to 1 when we receive the first response on an ADS stream.
gcp.storage.client.grpc.xds_client.resource_updates_invalid
(count)
Cumulative count of resources received that were considered invalid.
Shown as resource
gcp.storage.client.grpc.xds_client.resource_updates_valid
(count)
Cumulative count of resources received that were considered valid. The counter will be incremented even for resources that have not changed.
Shown as resource
gcp.storage.client.grpc.xds_client.resources
(gauge)
Number of xDS resources.
Shown as resource
gcp.storage.client.grpc.xds_client.server_failure
(count)
Cumulative count of xDS servers going from healthy to unhealthy. A server goes unhealthy when we have a connectivity failure or when the ADS stream fails without seeing a response message, as per gRFC A57.
gcp.storage.network.received_bytes_count
(count)
The number of bytes received over the network.
Shown as byte
gcp.storage.network.sent_bytes_count
(count)
The number of bytes sent over the network.
Shown as byte
gcp.storage.quota.dualregion_google_egress_bandwidth.limit
(gauge)
Current limit on quota metric storage.googleapis.com/dualregion_google_egress_bandwidth.
Shown as bit
gcp.storage.quota.dualregion_google_egress_bandwidth.usage
(count)
Current usage on quota metric storage.googleapis.com/dualregion_google_egress_bandwidth.
Shown as bit
gcp.storage.quota.dualregion_internet_egress_bandwidth.limit
(gauge)
Current limit on quota metric storage.googleapis.com/dualregion_internet_egress_bandwidth.
Shown as bit
gcp.storage.quota.dualregion_internet_egress_bandwidth.usage
(count)
Current usage on quota metric storage.googleapis.com/dualregion_internet_egress_bandwidth.
Shown as bit
gcp.storage.replication.meeting_rpo
(gauge)
A boolean value of whether objects are meeting RPO or not. For ASYNC_TURBO, RPO is 15 minutes; For DEFAULT, RPO is 12 hours.
gcp.storage.replication.missing_rpo_minutes_last_30d
(gauge)
Total number of minutes where the Recovery Point Objective (RPO) was missed, as measured over the most recent 30 days. The metric is delayed by storage.googleapis.com/replication/time_since_metrics_updated.
gcp.storage.replication.object_replications_last_30d
(gauge)
Total number of object replications, as measured over the most recent 30 days. The metric can be broken down by whether each object replication was meeting or missing Recovery Point Objective (RPO). The metric is delayed by storage.googleapis.com/replication/time_since_metrics_updated.
gcp.storage.replication.time_since_metrics_updated
(gauge)
Seconds since the last time when storage.googleapis.com/replication/* metrics were calculated.
Shown as second
gcp.storage.replication.turbo_max_delay
(gauge)
Delay represents the age in seconds of the oldest object that has not been replicated. Objects uploaded before this time have been replicated.
Shown as second
gcp.storage.replication.v2.object_replications_last_30d
(gauge)
Total number of object replications, as measured over the most recent 30 days. The metric can be broken down by whether each object replication was meeting or missing the design target. The metric is delayed by storage.googleapis.com/replication/v2/time_since_metrics_updated.
gcp.storage.replication.v2.time_since_metrics_updated
(gauge)
Seconds since the last time when metric storage.googleapis.com/replication/missing_rpo_minutes_last_30d and /replication/v2/object_replications_last_30d were calculated.
Shown as second
gcp.storage.storage.object_count
(gauge)
The total number of objects per bucket.
Shown as object
gcp.storage.storage.total_byte_seconds
(gauge)
The total daily storage in byte seconds used.
Shown as second
gcp.storage.storage.total_bytes
(gauge)
The total size of all objects in the bucket.
Shown as byte
gcp.storage.storage.v2.deleted_bytes
(count)
Delta count of deleted bytes per bucket, grouped by storage class. This value is measured and reported once per day, and there might be a delay after measuring before the value becomes available in Datadog. On days with no deletes, no datapoint is exported for this metric.
Shown as byte
gcp.storage.storage.v2.total_byte_seconds
(gauge)
Total daily storage in byte*seconds used by the bucket, grouped by storage class and type where type can be live-object, noncurrent-object, soft-deleted-object and multipart-upload. This value is measured once per day, and there might be a delay after measuring before the value becomes available in Datadog. Once available, the value is repeated at each sampling interval throughout the day.
gcp.storage.storage.v2.total_bytes
(gauge)
Total size of all objects and multipart-uploads in the bucket, grouped by storage class and type where type can be live-object, noncurrent-object, soft-deleted-object and multipart-upload. This value is measured once per day, and there might be a delay after measuring before the value becomes available in Datadog. Once available, the value is repeated at each sampling interval throughout the day.
Shown as byte
gcp.storage.storage.v2.total_count
(gauge)
Total number of objects and multipart-uploads per bucket, grouped by storage class and type where type can be live-object, noncurrent-object, soft-deleted-object and multipart-upload. This value is measured once per day, and there might be a delay after measuring before the value becomes available in Datadog. Once available, the value is repeated at each sampling interval throughout the day.

イベント

Google Cloud Storage インテグレーションには、イベントは含まれません。

サービスチェック

Google Cloud Storage インテグレーションには、サービスのチェック機能は含まれません。

トラブルシューティング

ご不明な点は、Datadog のサポートチームまでお問い合わせください。