AuthZed Cloud

Supported OS Linux Windows Mac OS

Integration version1.0.0

Overview

Authzed Cloud is an open-core, Google Zanzibar-inspired database system for creating and managing security-critical application permissions.

Developers define a schema that models their permissions requirements. They then use any of the official or community-maintained client libraries to apply the schema and insert data into the database. They can query this data to efficiently check permissions within their applications.

Authzed Cloud metrics allow developers and SREs to monitor their deployments, including request latency, cache metrics (such as size and hit/miss rates), and datastore connection and query performance. These metrics help diagnose performance issues and fine-tune the performance of their SpiceDB clusters.

Sending these metrics to Datadog enables users to leverage their existing observability stack and correlate Authzed Cloud metrics with other system events.

Setup

The Datadog integration is available in the AuthZed Dashboard under the “Settings” tab on a Permission System.

  1. Go to the dashboard homepage.
  2. Select a Permission System for which to submit metrics.
  3. Click on the Settings tab.
  4. Scroll down to the Datadog Metrics block of the settings UI.
  5. Enter your Datadog account API key.
  6. Enter your Datadog site if different from the default.
  7. Click Save.

To ensure that the dashboard graph for latency correctly shows the p50, p95, and p99 latencies, you’ll also need to set the Percentiles setting for the authzed.grpc.server_handling metric in the Metrics Summary view to ON.

You should see metrics start to flow to Datadog shortly thereafter. If you don’t, contact our support.

Data Collected

Metrics

authzed_cloud.cockroachdb.capacity_available
(gauge)
Remaining available capacity
Shown as byte_in_decimal_bytes_family
authzed_cloud.cockroachdb.database_connections
(gauge)
Open connections
Shown as connection
authzed_cloud.cockroachdb.read_iops
(count)
Read IOPS
Shown as operation
authzed_cloud.cockroachdb.read_throughput
(count)
Read throughput
Shown as byte_in_decimal_bytes_family
authzed_cloud.cockroachdb.request_latency_sum
(count)
Request latency
Shown as nanosecond
authzed_cloud.cockroachdb.request_latency_bucket
(count)
Request latency
Shown as nanosecond
authzed_cloud.cockroachdb.request_latency_count
(count)
Request latency
Shown as nanosecond
authzed_cloud.cockroachdb.system_cpu_usage
(gauge)
CPU usage as a percentage of available CPU
Shown as percent
authzed_cloud.cockroachdb.write_iops
(count)
Write IOPS
Shown as operation
authzed_cloud.cockroachdb.write_throughput
(count)
Write throughput
Shown as byte_in_decimal_bytes_family
authzed_cloud.fgam.config_entities
(gauge)
The number of FGAM configurations currently active
Shown as item
authzed_cloud.grpc.server_handled
(count)
Total number of RPCs completed on the server, regardless of success or failure.
Shown as request
authzed_cloud.grpc.server_handling_sum
(count)
Distribution of response latency (seconds) of gRPC calls that had been handled by the server.
Shown as second
authzed_cloud.grpc.server_handling_bucket
(count)
Distribution of response latency (seconds) of gRPC calls that had been handled by the server.
Shown as second
authzed_cloud.grpc.server_handling_count
(count)
Distribution of response latency (seconds) of gRPC calls that had been handled by the server.
Shown as second
authzed_cloud.grpc.server_msg_received
(count)
Total number of RPC messages received on the server.
Shown as request
authzed_cloud.grpc.server_msg_sent
(count)
Total number of gRPC messages sent by the server.
Shown as request
authzed_cloud.grpc.server_started
(count)
Total number of RPCs started on the server.
Shown as request
authzed_cloud.materialize.hydrator.permission_hydrate
(count)
Number of permissions hydrated by the hydrator
Shown as item
authzed_cloud.materialize.snapshotter.relationship_snapshot_total
(count)
Number of relations captured by the snapshotter
Shown as item
authzed_cloud.materialize.updater.revision_processing_lag
(gauge)
An estimate of how long it takes a revision to become available in materialize
Shown as second
authzed_cloud.materialize.updater.update_duration_sum
(count)
How long the updater spends on each update
Shown as second
authzed_cloud.materialize.updater.update_duration_bucket
(count)
How long the updater spends on each update
Shown as second
authzed_cloud.materialize.updater.update_duration_count
(count)
How long the updater spends on each update
Shown as second
authzed_cloud.metrics.up
(gauge)
Indicates whether metrics are being sent by AuthZed
authzed_cloud.process.cpu
(gauge)
CPU usage estimate
Shown as core
authzed_cloud.process.virtual_memory
(gauge)
Virtual memory size in bytes
Shown as byte_in_decimal_bytes_family
authzed_cloud.rds.capacity_available
(gauge)
Capacity available in RDS
Shown as byte_in_decimal_bytes_family
authzed_cloud.rds.database_connections
(gauge)
Number of active DB connections
Shown as connection
authzed_cloud.rds.read_iops
(gauge)
Read I/O operations
Shown as operation
authzed_cloud.rds.read_latency
(gauge)
RDS read latency
Shown as millisecond
authzed_cloud.rds.read_throughput
(gauge)
Read throughput in bytes
Shown as byte_in_decimal_bytes_family
authzed_cloud.rds.system_cpu_usage
(gauge)
RDS CPU usage
Shown as percent
authzed_cloud.rds.write_iops
(gauge)
Write I/O operations
Shown as operation
authzed_cloud.rds.write_latency
(gauge)
RDS write latency
Shown as millisecond
authzed_cloud.rds.write_throughput
(gauge)
Write throughput in bytes
Shown as byte_in_decimal_bytes_family
authzed_cloud.spicedb.cache.cost_added
(gauge)
Cost of entries added to the cache
Shown as byte_in_decimal_bytes_family
authzed_cloud.spicedb.cache.cost_evicted
(gauge)
Cost of entries evicted from the cache
Shown as byte_in_decimal_bytes_family
authzed_cloud.spicedb.cache.hits
(count)
Number of cache hits
Shown as hit
authzed_cloud.spicedb.cache.misses
(count)
Number of cache misses
Shown as miss
authzed_cloud.spicedb.datastore.gc.duration_sum
(count)
How long a SpiceDB relation GC cycle lasts
Shown as second
authzed_cloud.spicedb.datastore.gc.duration_bucket
(count)
How long a SpiceDB relation GC cycle lasts
Shown as second
authzed_cloud.spicedb.datastore.gc.duration_count
(count)
How long a SpiceDB relation GC cycle lasts
Shown as second
authzed_cloud.spicedb.datastore.gc.expired_relationships
(count)
Number of SpiceDB relationships removed by GC
Shown as item
authzed_cloud.spicedb.datastore.gc.failure
(count)
Number of SpiceDB relation GC failures
Shown as attempt
authzed_cloud.spicedb.datastore.gc.namespaces
(count)
Number of SpiceDB relation GC namespaces
Shown as item
authzed_cloud.spicedb.datastore.gc.relationships
(count)
Number of SpiceDB relationships removed by GC
Shown as item
authzed_cloud.spicedb.datastore.gc.transactions
(count)
Number of transactions associated with SpiceDB relation GC
Shown as transaction
authzed_cloud.spicedb.datastore.pgx.acquired_connections
(gauge)
The number of DB connections acquired by PGX
Shown as connection
authzed_cloud.spicedb.datastore.pgx.constructing_connections
(gauge)
The number of DB connections being constructed by PGX
Shown as connection
authzed_cloud.spicedb.datastore.pgx.idle_connections
(gauge)
The number of DB connections sitting idle from PGX
Shown as connection
authzed_cloud.spicedb.datastore.pgx.total_connections
(gauge)
The total number of connections handled by PGX (sum of acquired, idle, constructing)
Shown as connection
authzed_cloud.spicedb.datastore.pgx.waited_connections
(gauge)
The number of acquired connections that had to wait for a connection to become available. If this is greater than zero, tune your SpiceDB and datastore so that it’s not saturated.
Shown as connection
authzed_cloud.spicedb.hooks.latency_sum
(count)
Time spent in SpiceDB middleware hooks (includes FGAM)
Shown as second
authzed_cloud.spicedb.hooks.latency_bucket
(count)
Time spent in SpiceDB middleware hooks (includes FGAM)
Shown as second
authzed_cloud.spicedb.hooks.latency_count
(count)
Time spent in SpiceDB middleware hooks (includes FGAM)
Shown as second
authzed_cloud.spicedb.middleware.consistency_assigned
(count)
Requests broken down by consistency level selected
Shown as request

Uninstallation

The Datadog integration is available in the AuthZed Dashboard under the Settings tab on a Permission System.

  1. Go to the dashboard homepage.
  2. Select a Permission System for which to submit metrics.
  3. Click on the Settings tab.
  4. Scroll down to the Datadog Metrics block of the settings UI.
  5. Click Remove.

This disables the Datadog integration in your AuthZed Cloud cluster. Note that this could take several minutes.

Support

Need help? Contact AuthZed support.