Google Storage

Présentation

Google Cloud Storage est une solution unifiée de stockage d’objets destinée aux développeurs et aux entreprises. Elle associe des fonctionnalités de diffusion de données en temps réel, d’analyse et de machine learning, d’archivage, etc.

Recueillez des métriques de Google Storage pour :

  • Visualiser les performances de vos services Storage
  • Corréler les performances de vos services Storage avec vos applications

Configuration

Installation

Si vous ne l’avez pas déjà fait, configurez d’abord l’intégration Google Cloud Platform. Aucune autre procédure d’installation n’est requise.

Configuration

Pour recueillir des étiquettes Cloud Storage personnalisées sous forme de tags, activez l’autorisation Cloud Asset Inventory.

Collecte de logs

Les logs Google Cloud Storage sont recueillis avec Google Cloud Logging et envoyés à un Cloud Pub/Sub via un forwarder Push HTTP. Si vous ne l’avez pas déjà fait, configurez un Cloud Pub/Sub à l’aide d’un forwarder Push HTTP.

Une fois cette opération effectuée, exportez vos logs Google Cloud Storage depuis Google Cloud Logging vers le Pub/Sub :

  1. Accédez à la page Google Cloud Logging et filtrez les logs Google Cloud Storage.
  2. Cliquez sur Create Export et nommez le récepteur.
  3. Choisissez Cloud Pub/Sub comme destination et sélectionnez le Pub/Sub créé à cette fin. Remarque : le Pub/Sub peut se situer dans un autre projet.
  4. Cliquez sur Create et attendez que le message de confirmation s’affiche.

Données collectées

Métriques

gcp.storage.api.lro_count
(count)
Count of completed long-running operations, since the last sample.
gcp.storage.api.request_count
(count)
The number of API calls.
Shown as request
gcp.storage.authn.authentication_count
(count)
The number of HMAC/RSA signed requests.
Shown as request
gcp.storage.authz.acl_based_object_access_count
(count)
The number of requests that result in an object being granted access solely due to object ACLs.
Shown as request
gcp.storage.authz.acl_operations_count
(count)
The usage of ACL operations.
Shown as operation
gcp.storage.authz.object_specific_acl_mutation_count
(count)
The number of changes made to object specific ACLs.
gcp.storage.autoclass.transition_operation_count
(count)
Total number of storage class transition operations initiated by Autoclass.
gcp.storage.autoclass.transitioned_bytes_count
(count)
Total bytes transitioned by Autoclass.
Shown as byte
gcp.storage.client.grpc.client.attempt.duration.avg
(gauge)
The average of the cumulative time taken to complete an RPC attempt including the time it takes to pick a subchannel.
Shown as second
gcp.storage.client.grpc.client.attempt.duration.samplecount
(gauge)
The sample count for the cumulative time taken to complete an RPC attempt including the time it takes to pick a subchannel.
Shown as second
gcp.storage.client.grpc.client.attempt.duration.sumsqdev
(gauge)
The sum of squared deviation for the cumulative time taken to complete an RPC attempt including the time it takes to pick a subchannel.
Shown as second
gcp.storage.client.grpc.client.attempt.rcvd_total_compressed_message_size.avg
(count)
The average cumulative bytes (compressed but not encrypted) received across all response messages (metadata excluded) per RPC attempt; does not include gRPC or transport framing bytes.
Shown as byte
gcp.storage.client.grpc.client.attempt.rcvd_total_compressed_message_size.samplecount
(count)
The sample count for the cumulative bytes (compressed but not encrypted) received across all response messages (metadata excluded) per RPC attempt; does not include gRPC or transport framing bytes.
Shown as byte
gcp.storage.client.grpc.client.attempt.rcvd_total_compressed_message_size.sumsqdev
(count)
The sum of squared deviation for the cumulative bytes (compressed but not encrypted) received across all response messages (metadata excluded) per RPC attempt; does not include gRPC or transport framing bytes.
Shown as byte
gcp.storage.client.grpc.client.attempt.sent_total_compressed_message_size.avg
(count)
The average cumulative bytes (compressed but not encrypted) sent across all request messages (metadata excluded) per RPC attempt; does not include gRPC or transport framing bytes.
Shown as byte
gcp.storage.client.grpc.client.attempt.sent_total_compressed_message_size.samplecount
(count)
The sample count for the cumulative bytes (compressed but not encrypted) sent across all request messages (metadata excluded) per RPC attempt; does not include gRPC or transport framing bytes.
Shown as byte
gcp.storage.client.grpc.client.attempt.sent_total_compressed_message_size.sumsqdev
(count)
The sum of squared deviation for the cumulative bytes (compressed but not encrypted) sent across all request messages (metadata excluded) per RPC attempt; does not include gRPC or transport framing bytes.
Shown as byte
gcp.storage.client.grpc.client.attempt.started
(count)
The cumulative number of RPC attempts started, including those that have not completed.
gcp.storage.client.grpc.client.call.duration.avg
(gauge)
The average of the cumulative time the gRPC library takes to complete an RPC from the application’s perspective.
Shown as second
gcp.storage.client.grpc.client.call.duration.samplecount
(gauge)
The sample count for the cumulative time the gRPC library takes to complete an RPC from the application’s perspective.
Shown as second
gcp.storage.client.grpc.client.call.duration.sumsqdev
(gauge)
The sum of squared deviation of the cumulative time the gRPC library takes to complete an RPC from the application’s perspective.
Shown as second
gcp.storage.client.grpc.lb.rls.cache_entries
(gauge)
Number of entries in the RLS cache.
Shown as entry
gcp.storage.client.grpc.lb.rls.cache_size
(gauge)
The current size of the RLS cache.
Shown as byte
gcp.storage.client.grpc.lb.rls.default_target_picks
(count)
Cumulative number of LB picks sent to the default target.
gcp.storage.client.grpc.lb.rls.failed_picks
(count)
Cumulative number of LB picks failed due to either a failed RLS request or the RLS channel being throttled.
gcp.storage.client.grpc.lb.rls.target_picks
(count)
Cumulative number of LB picks sent to each RLS target. Note that if the default target is also returned by the RLS server, RPCs sent to that target from the cache are counted in this metric, not in grpc.rls.default_target_picks.
gcp.storage.client.grpc.lb.wrr.endpoint_weight_not_yet_usable
(count)
Cumulative number of endpoints from each scheduler update that do not yet have usable weight information (either the load report has not yet been received, or it is within the blackout period).
gcp.storage.client.grpc.lb.wrr.endpoint_weight_stale
(count)
Cumulative number of endpoints from each scheduler update whose latest weight is older than the expiration period.
gcp.storage.client.grpc.lb.wrr.endpoint_weights.avg
(count)
The average of the cumulative bucket counts. The histogram buckets will be endpoint weight ranges. Each bucket will be a counter that is incremented once for every endpoint whose weight is within that range. Note that endpoints without usable weights will have weight 0.
gcp.storage.client.grpc.lb.wrr.endpoint_weights.samplecount
(count)
The sample count for the cumulative bucket counts. The histogram buckets will be endpoint weight ranges. Each bucket will be a counter that is incremented once for every endpoint whose weight is within that range. Note that endpoints without usable weights will have weight 0.
gcp.storage.client.grpc.lb.wrr.endpoint_weights.sumsqdev
(count)
The sum of squared deviation for the cumulative bucket counts. The histogram buckets will be endpoint weight ranges. Each bucket will be a counter that is incremented once for every endpoint whose weight is within that range. Note that endpoints without usable weights will have weight 0.
gcp.storage.client.grpc.lb.wrr.rr_fallback
(count)
Cumulative number of scheduler updates in which there were not enough endpoints with valid weight, which caused the WRR policy to fall back to RR behavior.
gcp.storage.client.grpc.xds_client.connected
(gauge)
Whether or not the xDS client currently has a working ADS stream to the xDS server. For a given server, this will be set to 1 when the stream is initially created. It will be set to 0 when we have a connectivity failure or when the ADS stream fails without seeing a response message, as per A57. Once set to 0, it will be reset to 1 when we receive the first response on an ADS stream.
gcp.storage.client.grpc.xds_client.resource_updates_invalid
(count)
Cumulative count of resources received that were considered invalid.
Shown as resource
gcp.storage.client.grpc.xds_client.resource_updates_valid
(count)
Cumulative count of resources received that were considered valid. The counter will be incremented even for resources that have not changed.
Shown as resource
gcp.storage.client.grpc.xds_client.resources
(gauge)
Number of xDS resources.
Shown as resource
gcp.storage.client.grpc.xds_client.server_failure
(count)
Cumulative count of xDS servers going from healthy to unhealthy. A server goes unhealthy when we have a connectivity failure or when the ADS stream fails without seeing a response message, as per gRFC A57.
gcp.storage.network.received_bytes_count
(count)
The number of bytes received over the network.
Shown as byte
gcp.storage.network.sent_bytes_count
(count)
The number of bytes sent over the network.
Shown as byte
gcp.storage.quota.dualregion_google_egress_bandwidth.limit
(gauge)
Current limit on quota metric storage.googleapis.com/dualregion_google_egress_bandwidth.
Shown as bit
gcp.storage.quota.dualregion_google_egress_bandwidth.usage
(count)
Current usage on quota metric storage.googleapis.com/dualregion_google_egress_bandwidth.
Shown as bit
gcp.storage.quota.dualregion_internet_egress_bandwidth.limit
(gauge)
Current limit on quota metric storage.googleapis.com/dualregion_internet_egress_bandwidth.
Shown as bit
gcp.storage.quota.dualregion_internet_egress_bandwidth.usage
(count)
Current usage on quota metric storage.googleapis.com/dualregion_internet_egress_bandwidth.
Shown as bit
gcp.storage.replication.meeting_rpo
(gauge)
A boolean value of whether objects are meeting RPO or not. For ASYNC_TURBO, RPO is 15 minutes; For DEFAULT, RPO is 12 hours.
gcp.storage.replication.missing_rpo_minutes_last_30d
(gauge)
Total number of minutes where the Recovery Point Objective (RPO) was missed, as measured over the most recent 30 days. The metric is delayed by storage.googleapis.com/replication/time_since_metrics_updated.
gcp.storage.replication.object_replications_last_30d
(gauge)
Total number of object replications, as measured over the most recent 30 days. The metric can be broken down by whether each object replication was meeting or missing Recovery Point Objective (RPO). The metric is delayed by storage.googleapis.com/replication/time_since_metrics_updated.
gcp.storage.replication.time_since_metrics_updated
(gauge)
Seconds since the last time when storage.googleapis.com/replication/* metrics were calculated.
Shown as second
gcp.storage.replication.turbo_max_delay
(gauge)
Delay represents the age in seconds of the oldest object that has not been replicated. Objects uploaded before this time have been replicated.
Shown as second
gcp.storage.replication.v2.object_replications_last_30d
(gauge)
Total number of object replications, as measured over the most recent 30 days. The metric can be broken down by whether each object replication was meeting or missing the design target. The metric is delayed by storage.googleapis.com/replication/v2/time_since_metrics_updated.
gcp.storage.replication.v2.time_since_metrics_updated
(gauge)
Seconds since the last time when metric storage.googleapis.com/replication/missing_rpo_minutes_last_30d and /replication/v2/object_replications_last_30d were calculated.
Shown as second
gcp.storage.storage.object_count
(gauge)
The total number of objects per bucket.
Shown as object
gcp.storage.storage.total_byte_seconds
(gauge)
The total daily storage in byte seconds used.
Shown as second
gcp.storage.storage.total_bytes
(gauge)
The total size of all objects in the bucket.
Shown as byte
gcp.storage.storage.v2.deleted_bytes
(count)
Delta count of deleted bytes per bucket, grouped by storage class. This value is measured and reported once per day, and there might be a delay after measuring before the value becomes available in Datadog. On days with no deletes, no datapoint is exported for this metric.
Shown as byte
gcp.storage.storage.v2.total_byte_seconds
(gauge)
Total daily storage in byte*seconds used by the bucket, grouped by storage class and type where type can be live-object, noncurrent-object, soft-deleted-object and multipart-upload. This value is measured once per day, and there might be a delay after measuring before the value becomes available in Datadog. Once available, the value is repeated at each sampling interval throughout the day.
gcp.storage.storage.v2.total_bytes
(gauge)
Total size of all objects and multipart-uploads in the bucket, grouped by storage class and type where type can be live-object, noncurrent-object, soft-deleted-object and multipart-upload. This value is measured once per day, and there might be a delay after measuring before the value becomes available in Datadog. Once available, the value is repeated at each sampling interval throughout the day.
Shown as byte
gcp.storage.storage.v2.total_count
(gauge)
Total number of objects and multipart-uploads per bucket, grouped by storage class and type where type can be live-object, noncurrent-object, soft-deleted-object and multipart-upload. This value is measured once per day, and there might be a delay after measuring before the value becomes available in Datadog. Once available, the value is repeated at each sampling interval throughout the day.

Événements

L’intégration Google Cloud Storage n’inclut aucun événement.

Checks de service

L’intégration Google Cloud Storage n’inclut aucun check de service.

Dépannage

Besoin d’aide ? Contactez l’assistance Datadog.