Google Storage

개요

Google Cloud Storage는 실시간 데이터 제공부터 데이터 분석/ML, 데이터 보관까지 개발자 및 기업을 위한 서비스를 제공해 드리는 통합 오브젝트 스토리지입니다.

Google Storage 메트릭을 수집하면 다음을 할 수 있습니다.

  • Storage 서비스 성능을 시각화합니다.
  • Storage 서비스의 성능과 애플리케이션의 상관 관계를 파악합니다.

설정

설치

아직 설치하지 않았다면 먼저 Google 클라우드 플랫폼 통합을 설정합니다. 그 외 다른 설치가 필요하지 않습니다.

설정

커스텀 Cloud Storage 레이블을 태그로 수집하려면 클라우드 에셋 인벤토리 권한을 활성화합니다.

로그 수집

Google Cloud Storage 로그는 Google Cloud Logging으로 수집하여 클라우드 Pub/Sub 토픽을 통해 데이터 플로우 작업으로 전송됩니다. 아직 설정하지 않았다면 Datadog 데이터 플로우 템플릿으로 로깅을 설정하세요.

해당 작업이 완료되면 Google Cloud Logging에서 Google Cloud Storage 로그를 다음 Pub/Sub 주제로 내보냅니다.

  1. Google Cloud Logging 페이지로 이동해 Google Cloud Storage 로그를 필터링하세요.
  2. Create Export를 클릭하고 싱크 이름을 지정하세요.
  3. “Cloud Pub/Sub"를 대상으로 선택하고 해당 목적으로 생성된 Pub/Sub 주제를 선택합니다. 참고: Pub/Sub 주제는 다른 프로젝트에 있을 수 있습니다.
  4. Create를 클릭하고 확인 메시지가 나타날 때까지 기다립니다.

수집한 데이터

메트릭

gcp.storage.api.lro_count
(count)		Count of completed long-running operations, since the last sample.
gcp.storage.api.request_count
(count)		The number of API calls.
Shown as request
gcp.storage.authn.authentication_count
(count)		The number of HMAC/RSA signed requests.
Shown as request
gcp.storage.authz.acl_based_object_access_count
(count)		The number of requests that result in an object being granted access solely due to object ACLs.
Shown as request
gcp.storage.authz.acl_operations_count
(count)		The usage of ACL operations.
Shown as operation
gcp.storage.authz.object_specific_acl_mutation_count
(count)		The number of changes made to object specific ACLs.
gcp.storage.autoclass.transition_operation_count
(count)		Total number of storage class transition operations initiated by Autoclass.
gcp.storage.autoclass.transitioned_bytes_count
(count)		Total bytes transitioned by Autoclass.
Shown as byte
gcp.storage.client.grpc.client.attempt.duration.avg
(gauge)		The average of the cumulative time taken to complete an RPC attempt including the time it takes to pick a subchannel.
Shown as second
gcp.storage.client.grpc.client.attempt.duration.samplecount
(gauge)		The sample count for the cumulative time taken to complete an RPC attempt including the time it takes to pick a subchannel.
Shown as second
gcp.storage.client.grpc.client.attempt.duration.sumsqdev
(gauge)		The sum of squared deviation for the cumulative time taken to complete an RPC attempt including the time it takes to pick a subchannel.
Shown as second
gcp.storage.client.grpc.client.attempt.rcvd_total_compressed_message_size.avg
(count)		The average cumulative bytes (compressed but not encrypted) received across all response messages (metadata excluded) per RPC attempt; does not include gRPC or transport framing bytes.
Shown as byte
gcp.storage.client.grpc.client.attempt.rcvd_total_compressed_message_size.samplecount
(count)		The sample count for the cumulative bytes (compressed but not encrypted) received across all response messages (metadata excluded) per RPC attempt; does not include gRPC or transport framing bytes.
Shown as byte
gcp.storage.client.grpc.client.attempt.rcvd_total_compressed_message_size.sumsqdev
(count)		The sum of squared deviation for the cumulative bytes (compressed but not encrypted) received across all response messages (metadata excluded) per RPC attempt; does not include gRPC or transport framing bytes.
Shown as byte
gcp.storage.client.grpc.client.attempt.sent_total_compressed_message_size.avg
(count)		The average cumulative bytes (compressed but not encrypted) sent across all request messages (metadata excluded) per RPC attempt; does not include gRPC or transport framing bytes.
Shown as byte
gcp.storage.client.grpc.client.attempt.sent_total_compressed_message_size.samplecount
(count)		The sample count for the cumulative bytes (compressed but not encrypted) sent across all request messages (metadata excluded) per RPC attempt; does not include gRPC or transport framing bytes.
Shown as byte
gcp.storage.client.grpc.client.attempt.sent_total_compressed_message_size.sumsqdev
(count)		The sum of squared deviation for the cumulative bytes (compressed but not encrypted) sent across all request messages (metadata excluded) per RPC attempt; does not include gRPC or transport framing bytes.
Shown as byte
gcp.storage.client.grpc.client.attempt.started
(count)		The cumulative number of RPC attempts started, including those that have not completed.
gcp.storage.client.grpc.client.call.duration.avg
(gauge)		The average of the cumulative time the gRPC library takes to complete an RPC from the application’s perspective.
Shown as second
gcp.storage.client.grpc.client.call.duration.samplecount
(gauge)		The sample count for the cumulative time the gRPC library takes to complete an RPC from the application’s perspective.
Shown as second
gcp.storage.client.grpc.client.call.duration.sumsqdev
(gauge)		The sum of squared deviation of the cumulative time the gRPC library takes to complete an RPC from the application’s perspective.
Shown as second
gcp.storage.client.grpc.lb.rls.cache_entries
(gauge)		Number of entries in the RLS cache.
Shown as entry
gcp.storage.client.grpc.lb.rls.cache_size
(gauge)		The current size of the RLS cache.
Shown as byte
gcp.storage.client.grpc.lb.rls.default_target_picks
(count)		Cumulative number of LB picks sent to the default target.
gcp.storage.client.grpc.lb.rls.failed_picks
(count)		Cumulative number of LB picks failed due to either a failed RLS request or the RLS channel being throttled.
gcp.storage.client.grpc.lb.rls.target_picks
(count)		Cumulative number of LB picks sent to each RLS target. Note that if the default target is also returned by the RLS server, RPCs sent to that target from the cache are counted in this metric, not in grpc.rls.default_target_picks.
gcp.storage.client.grpc.lb.wrr.endpoint_weight_not_yet_usable
(count)		Cumulative number of endpoints from each scheduler update that do not yet have usable weight information (either the load report has not yet been received, or it is within the blackout period).
gcp.storage.client.grpc.lb.wrr.endpoint_weight_stale
(count)		Cumulative number of endpoints from each scheduler update whose latest weight is older than the expiration period.
gcp.storage.client.grpc.lb.wrr.endpoint_weights.avg
(count)		The average of the cumulative bucket counts. The histogram buckets will be endpoint weight ranges. Each bucket will be a counter that is incremented once for every endpoint whose weight is within that range. Note that endpoints without usable weights will have weight 0.
gcp.storage.client.grpc.lb.wrr.endpoint_weights.samplecount
(count)		The sample count for the cumulative bucket counts. The histogram buckets will be endpoint weight ranges. Each bucket will be a counter that is incremented once for every endpoint whose weight is within that range. Note that endpoints without usable weights will have weight 0.
gcp.storage.client.grpc.lb.wrr.endpoint_weights.sumsqdev
(count)		The sum of squared deviation for the cumulative bucket counts. The histogram buckets will be endpoint weight ranges. Each bucket will be a counter that is incremented once for every endpoint whose weight is within that range. Note that endpoints without usable weights will have weight 0.
gcp.storage.client.grpc.lb.wrr.rr_fallback
(count)		Cumulative number of scheduler updates in which there were not enough endpoints with valid weight, which caused the WRR policy to fall back to RR behavior.
gcp.storage.client.grpc.xds_client.connected
(gauge)		Whether or not the xDS client currently has a working ADS stream to the xDS server. For a given server, this will be set to 1 when the stream is initially created. It will be set to 0 when we have a connectivity failure or when the ADS stream fails without seeing a response message, as per A57. Once set to 0, it will be reset to 1 when we receive the first response on an ADS stream.
gcp.storage.client.grpc.xds_client.resource_updates_invalid
(count)		Cumulative count of resources received that were considered invalid.
Shown as resource
gcp.storage.client.grpc.xds_client.resource_updates_valid
(count)		Cumulative count of resources received that were considered valid. The counter will be incremented even for resources that have not changed.
Shown as resource
gcp.storage.client.grpc.xds_client.resources
(gauge)		Number of xDS resources.
Shown as resource
gcp.storage.client.grpc.xds_client.server_failure
(count)		Cumulative count of xDS servers going from healthy to unhealthy. A server goes unhealthy when we have a connectivity failure or when the ADS stream fails without seeing a response message, as per gRFC A57.
gcp.storage.network.received_bytes_count
(count)		The number of bytes received over the network.
Shown as byte
gcp.storage.network.sent_bytes_count
(count)		The number of bytes sent over the network.
Shown as byte
gcp.storage.quota.dualregion_google_egress_bandwidth.limit
(gauge)		Current limit on quota metric storage.googleapis.com/dualregion_google_egress_bandwidth.
Shown as bit
gcp.storage.quota.dualregion_google_egress_bandwidth.usage
(count)		Current usage on quota metric storage.googleapis.com/dualregion_google_egress_bandwidth.
Shown as bit
gcp.storage.quota.dualregion_internet_egress_bandwidth.limit
(gauge)		Current limit on quota metric storage.googleapis.com/dualregion_internet_egress_bandwidth.
Shown as bit
gcp.storage.quota.dualregion_internet_egress_bandwidth.usage
(count)		Current usage on quota metric storage.googleapis.com/dualregion_internet_egress_bandwidth.
Shown as bit
gcp.storage.replication.meeting_rpo
(gauge)		A boolean value of whether objects are meeting RPO or not. For ASYNC_TURBO, RPO is 15 minutes; For DEFAULT, RPO is 12 hours.
gcp.storage.replication.missing_rpo_minutes_last_30d
(gauge)		Total number of minutes where the Recovery Point Objective (RPO) was missed, as measured over the most recent 30 days. The metric is delayed by storage.googleapis.com/replication/time_since_metrics_updated.
gcp.storage.replication.object_replications_last_30d
(gauge)		Total number of object replications, as measured over the most recent 30 days. The metric can be broken down by whether each object replication was meeting or missing Recovery Point Objective (RPO). The metric is delayed by storage.googleapis.com/replication/time_since_metrics_updated.
gcp.storage.replication.time_since_metrics_updated
(gauge)		Seconds since the last time when storage.googleapis.com/replication/* metrics were calculated.
Shown as second
gcp.storage.replication.turbo_max_delay
(gauge)		Delay represents the age in seconds of the oldest object that has not been replicated. Objects uploaded before this time have been replicated.
Shown as second
gcp.storage.replication.v2.object_replications_last_30d
(gauge)		Total number of object replications, as measured over the most recent 30 days. The metric can be broken down by whether each object replication was meeting or missing the design target. The metric is delayed by storage.googleapis.com/replication/v2/time_since_metrics_updated.
gcp.storage.replication.v2.time_since_metrics_updated
(gauge)		Seconds since the last time when metric storage.googleapis.com/replication/missing_rpo_minutes_last_30d and /replication/v2/object_replications_last_30d were calculated.
Shown as second
gcp.storage.storage.object_count
(gauge)		The total number of objects per bucket.
Shown as object
gcp.storage.storage.total_byte_seconds
(gauge)		The total daily storage in byte seconds used.
Shown as second
gcp.storage.storage.total_bytes
(gauge)		The total size of all objects in the bucket.
Shown as byte
gcp.storage.storage.v2.deleted_bytes
(count)		Delta count of deleted bytes per bucket, grouped by storage class. This value is measured and reported once per day, and there might be a delay after measuring before the value becomes available in Datadog. On days with no deletes, no datapoint is exported for this metric.
Shown as byte
gcp.storage.storage.v2.total_byte_seconds
(gauge)		Total daily storage in byte*seconds used by the bucket, grouped by storage class and type where type can be live-object, noncurrent-object, soft-deleted-object and multipart-upload. This value is measured once per day, and there might be a delay after measuring before the value becomes available in Datadog. Once available, the value is repeated at each sampling interval throughout the day.
gcp.storage.storage.v2.total_bytes
(gauge)		Total size of all objects and multipart-uploads in the bucket, grouped by storage class and type where type can be live-object, noncurrent-object, soft-deleted-object and multipart-upload. This value is measured once per day, and there might be a delay after measuring before the value becomes available in Datadog. Once available, the value is repeated at each sampling interval throughout the day.
Shown as byte
gcp.storage.storage.v2.total_count
(gauge)		Total number of objects and multipart-uploads per bucket, grouped by storage class and type where type can be live-object, noncurrent-object, soft-deleted-object and multipart-upload. This value is measured once per day, and there might be a delay after measuring before the value becomes available in Datadog. Once available, the value is repeated at each sampling interval throughout the day.

이벤트

Google Cloud Storage 통합은 이벤트를 포함하지 않습니다.

서비스 점검

Google Cloud Storage 통합은 서비스 점검을 포함하지 않습니다.

트러블슈팅

도움이 필요하신가요? Datadog 지원팀에 문의하세요.