Supported OS Linux Windows Mac OS

インテグレーションバージョン2.3.1

概要

このチェックでは、Datadog Agent を通じて Weaviate を監視します。詳細については、Weaviate モニタリングを参照してください。

計画と使用

ホストで実行されている Agent 用にこのチェックをインストールおよび構成する場合は、以下の手順に従ってください。コンテナ環境の場合は、オートディスカバリーのインテグレーションテンプレートのガイドを参照してこの手順を行ってください。

インフラストラクチャーリスト

Agent リリース 7.47.0 から、Weaviate チェックは Datadog Agent パッケージに含まれています。

: この機能を使用するには、Agent v7.47.0 以上が必要です。

ブラウザトラブルシューティング

Weaviate は、Prometheus 形式のメトリクスを公開するように構成できます。Datadog Agent は、以下に説明するインテグレーションを使用して、これらのメトリクスを収集することができます。指示に従って、Weaviate インスタンスのデータ収集を構成してください。Prometheus メトリクスを公開するために必要な構成については、Weaviate ドキュメントの Monitoring ページを参照してください。

さらに、各種 API エンドポイントと通信することで、メトリクスの小さなサブセットを収集できます。具体的には

  • /v1/meta: バージョン情報
  • /v1/nodes: オブジェクトやシャードなどのノード固有のメトリクス
  • /v1/.well-known/live: HTTP レスポンスタイムとサービスの有効性

: このチェックでは、メトリクスの収集に OpenMetrics を使用しており、Python 3 が必要です。

コンテナ化

メトリクスの収集

Weaviate クラスターで Prometheus 形式のメトリクスが公開されていることを確認してください。Weaviate ドキュメントの Monitoring ページの指示に従って構成およびカスタマイズできます。Agent がメトリクスの収集を開始するには、Weaviate ポッドにアノテーションを付ける必要があります。アノテーションの詳細については、オートディスカバリーインテグレーションテンプレートを参照してください。その他の構成オプションについては、サンプル weaviate.d/conf.yaml を参照してください。

: リストされたメトリクスは、利用可能な場合にのみ収集できます。一部のメトリクスは、特定のアクションが実行されたときにのみ生成されます。例えば、オブジェクト削除メトリクスは、オブジェクトが削除されたときにのみ公開されます。

Weaviate チェックの構成で最も重要なパラメーターは以下の 2 つです。

  • openmetrics_endpoint: このパラメーターは、Prometheus 形式のメトリクスが公開される場所に設定する必要があります。デフォルトのポートは 2112 ですが、PROMETHEUS_MONITORING_PORT 環境変数を使用して構成することができます。コンテナ環境では、%%host%%ホストの自動検出に使用します。
  • weaviate_api_endpoint: このパラメーターはオプションです。デフォルトでは、このパラメーターは <hostname>:8080 に設定され、RESTful API の構成を指定します。

RESTful API エンドポイントに認証が必要な場合、リクエストヘッダーの一部として API キーを提供するようにチェックを構成できます。

apiVersion: v1
kind: Pod
# (...)
metadata:
  name: '<POD_NAME>'
  annotations:
    ad.datadoghq.com/weaviate.checks: |
      {
        "weaviate": {
          "init_config": {},
          "instances": [
            {
              "openmetrics_endpoint": "http://%%host%%:2112/metrics",
              "weaviate_api_endpoint": "http://%%host%%:8080",
              "headers": {'Authorization': 'Bearer if_needed_for_auth'}
            }
          ]
        }
      }      
    # (...)
spec:
  containers:
    - name: 'weaviate'
# (...)

: アノテーションは annotations キーを使用して、Weaviate Helm チャートに直接設定することができます。

検証

Agent の status サブコマンドを実行し、Checks セクションで weaviate を探します。

リアルユーザーモニタリング

データセキュリティ

weaviate.async.operations.running
(gauge)
The number of currently running async operations. The operation itself is defined through the operation label
Shown as operation
weaviate.backup.restore.class.ms.count
(count)
The number of samples used to calculate the duration of restoring a class
Shown as operation
weaviate.backup.restore.class.ms.sum
(count)
The duration of restoring a class
Shown as millisecond
weaviate.backup.restore.data.transferred
(count)
The total number of bytes transferred during a backup restore
Shown as byte
weaviate.backup.restore.from.backend.ms.count
(count)
The number of samples used to calculate the duration of the file transfer stage of a backup restore
Shown as operation
weaviate.backup.restore.from.backend.ms.sum
(count)
The duration of the file transfer stage of a backup restore
Shown as millisecond
weaviate.backup.restore.init.ms.count
(count)
The number of samples used to calculate the duration of the startup phase of a backup restore
Shown as operation
weaviate.backup.restore.init.ms.sum
(count)
The duration of the startup phase of a backup restore
Shown as millisecond
weaviate.backup.restore.ms.count
(count)
The number of samples used to calculate the duration of backup restores
Shown as operation
weaviate.backup.restore.ms.sum
(count)
The duration of backup restores
Shown as millisecond
weaviate.backup.store.data.transferred
(count)
The total number of bytes transferred during a backup store
Shown as byte
weaviate.backup.store.to.backend.ms.count
(count)
The number of samples used to calculate the duration of the file transfer stage of a backup store
Shown as operation
weaviate.backup.store.to.backend.ms.sum
(count)
The duration of the file transfer stage of a backup store
Shown as millisecond
weaviate.batch.delete.durations_ms.bucket
(count)
The number of operations observed for the batch delete duration histogram by upper_bound buckets
Shown as operation
weaviate.batch.delete.durations_ms.count
(count)
The count of operations observed in the batch delete duration histogram
Shown as operation
weaviate.batch.delete.durations_ms.sum
(count)
The sum of the duration of a batch delete in ms. The operation label further defines what operation as part of the batch delete is being measured. Granularity is a shard of a class
Shown as millisecond
weaviate.batch.durations_ms.bucket
(count)
The number of operations observed for the batch operation duration histogram by upper_bound buckets
Shown as operation
weaviate.batch.durations_ms.count
(count)
The count of operations observed in the batch operation duration histogram
Shown as operation
weaviate.batch.durations_ms.sum
(count)
The sum of the duration of a single batch operation in ms. The operation label further defines what operation as part of the batch (e.g. object, inverted, vector) is being used. Granularity is a shard of a class
Shown as millisecond
weaviate.bucket.pause.durations.ms.count
(count)
The number of samples used to calculate the duration of bucket pauses
Shown as operation
weaviate.bucket.pause.durations.ms.sum
(count)
The duration of bucket pauses
Shown as millisecond
weaviate.concurrent.goroutines
(gauge)
The number of concurrently running goroutines
weaviate.concurrent.queries
(gauge)
The number of concurrently running query operations
Shown as operation
weaviate.go.gc.duration.seconds.count
(count)
The summary count of garbage collection cycles in the Weaviate instance
Shown as second
weaviate.go.gc.duration.seconds.quantile
(gauge)
A summary of the pause duration of garbage collection cycles in the Weaviate instance
Shown as second
weaviate.go.gc.duration.seconds.sum
(count)
The sum of the pause duration of garbage collection cycles in the Weaviate instance
Shown as second
weaviate.go.goroutines
(gauge)
The number of goroutines that currently exist in the Weaviate instance
weaviate.go.info
(gauge)
Metric containing the Go version as a tag
weaviate.go.memstats.alloc_bytes
(gauge)
The number of bytes allocated and still in use in the Weaviate instance
Shown as byte
weaviate.go.memstats.alloc_bytes.count
(count)
The monotonic count of bytes allocated and still in use in the Weaviate instance
Shown as byte
weaviate.go.memstats.buck_hash.sys_bytes
(gauge)
The number of bytes used by the profiling bucket hash table in the Weaviate instance
Shown as byte
weaviate.go.memstats.frees.count
(count)
The total number of frees in the Weaviate instance
weaviate.go.memstats.gc.cpu_fraction
(gauge)
The fraction of this program's available CPU time used by the GC since the program started in the Weaviate instance
Shown as fraction
weaviate.go.memstats.gc.sys_bytes
(gauge)
The number of bytes used for garbage collection system metadata in the Weaviate instance
Shown as byte
weaviate.go.memstats.heap.alloc_bytes
(gauge)
The number of heap bytes allocated and still in use in the Weaviate instance
Shown as byte
weaviate.go.memstats.heap.idle_bytes
(gauge)
The number of heap bytes waiting to be used in the Weaviate instance
Shown as byte
weaviate.go.memstats.heap.inuse_bytes
(gauge)
The number of heap bytes that are in use in the Weaviate instance
Shown as byte
weaviate.go.memstats.heap.objects
(gauge)
The number of allocated objects in the Weaviate instance
Shown as object
weaviate.go.memstats.heap.released_bytes
(gauge)
The number of heap bytes released to the OS in the Weaviate instance
Shown as byte
weaviate.go.memstats.heap.sys_bytes
(gauge)
The number of heap bytes obtained from system in the Weaviate instance
Shown as byte
weaviate.go.memstats.lookups.count
(count)
The number of pointer lookups
weaviate.go.memstats.mallocs.count
(count)
The number of mallocs
weaviate.go.memstats.mcache.inuse_bytes
(gauge)
The number of bytes in use by mcache structures in the Weaviate instance
Shown as byte
weaviate.go.memstats.mcache.sys_bytes
(gauge)
The number of bytes used for mcache structures obtained from system in the Weaviate instance
Shown as byte
weaviate.go.memstats.mspan.inuse_bytes
(gauge)
The number of bytes in use by mspan structures in the Weaviate instance
Shown as byte
weaviate.go.memstats.mspan.sys_bytes
(gauge)
The number of bytes used for mspan structures obtained from system in the Weaviate instance
Shown as byte
weaviate.go.memstats.next.gc_bytes
(gauge)
The number of heap bytes when next garbage collection takes place in the Weaviate instance
Shown as byte
weaviate.go.memstats.other.sys_bytes
(gauge)
The number of bytes used for other system allocations in the Weaviate instance
Shown as byte
weaviate.go.memstats.stack.inuse_bytes
(gauge)
The number of bytes in use by the stack allocator in the Weaviate instance
Shown as byte
weaviate.go.memstats.stack.sys_bytes
(gauge)
The number of bytes obtained from system for stack allocator in the Weaviate instance
Shown as byte
weaviate.go.memstats.sys_bytes
(gauge)
The number of bytes obtained from system in the Weaviate instance
Shown as byte
weaviate.go.threads
(gauge)
The number of OS threads created in the Weaviate instance
Shown as thread
weaviate.http.latency_ms
(gauge)
The HTTP request response time latency in ms
Shown as millisecond
weaviate.lsm.active.segments
(gauge)
The number of currently present segments per shard. Granularity is shard of a class. Grouped by strategy
Shown as segment
weaviate.lsm.bloom.filters.duration_ms.count
(count)
The number of samples used to calculate the duration of a bloom operation
Shown as operation
weaviate.lsm.bloom.filters.duration_ms.sum
(count)
The duration of a bloom filter operation per shard in ms. Granularity is shard of a class. Grouped by strategy
Shown as millisecond
weaviate.lsm.memtable.durations_ms.count
(count)
The number of samples used to calculate the duration of a sync or async vector index maintenance operation
Shown as operation
weaviate.lsm.memtable.durations_ms.sum
(count)
The duration of a sync or async vector index maintenance operation. The operation itself is defined through the operation label
Shown as millisecond
weaviate.lsm.memtable.size
(gauge)
The size of memtable by path
weaviate.lsm.segment.objects
(gauge)
The number of entries per LSM segment by level. Granularity is shard of a class. Grouped by strategy and level
Shown as object
weaviate.lsm.segment.size
(gauge)
The size of LSM segment by level and unit
weaviate.lsm.segments
(gauge)
The number of segments by level
Shown as segment
weaviate.node.shard.objects
(gauge)
The number of objects inside a Weaviate shard
Shown as object
weaviate.node.stats.objects
(gauge)
The number of objects inside a Weaviate node
Shown as object
weaviate.node.stats.shards
(gauge)
The number of shard inside a Weaviate shard
Shown as shard
weaviate.node.status
(gauge)
The current status of a Weaviate Node. 0:HEALTHY, 1:UNHEALTHY, 2:UNAVAILABLE, and 3:UNKNOWN
weaviate.objects
(gauge)
The numbers of objects present. Granularity is a shard of a class
Shown as object
weaviate.objects.durations_ms.count
(count)
The number of samples used to calculate the duration of an individual object operation
Shown as operation
weaviate.objects.durations_ms.sum
(count)
The duration of an individual object operation, such as put, delete, etc. as indicated by the operation label, also as part of a batch. The step label adds additional precisions to each operation. Granularity is a shard of a class
Shown as millisecond
weaviate.process.cpu.seconds.count
(count)
The total user and system CPU time spent in seconds in the Weaviate instance
Shown as second
weaviate.process.max_fds
(gauge)
The maximum number of open file descriptors in the Weaviate instance
weaviate.process.open_fds
(gauge)
The number of open file descriptors in the Weaviate instance
weaviate.process.resident_memory.bytes
(gauge)
The resident memory size in bytes in the Weaviate instance
Shown as byte
weaviate.process.start_time.seconds
(gauge)
The start time of the process since unix epoch in seconds in the Weaviate instance
Shown as second
weaviate.process.virtual_memory.bytes
(gauge)
The virtual memory size in bytes in the Weaviate instance
Shown as byte
weaviate.process.virtual_memory.max_bytes
(gauge)
The maximum amount of virtual memory available in bytes in the Weaviate instance
Shown as byte
weaviate.promhttp.metric_handler.requests.count
(count)
The total number of scrapes by HTTP status code
Shown as request
weaviate.promhttp.metric_handler.requests_in_flight
(gauge)
The current number of scrapes being served
Shown as request
weaviate.queries.durations_ms.bucket
(count)
The number of operations observed in the query duration histogram by upper_bound buckets
Shown as operation
weaviate.queries.durations_ms.count
(count)
The count of operations observed in the query duration histogram
Shown as operation
weaviate.queries.durations_ms.sum
(count)
The sum of query durations in milliseconds
Shown as millisecond
weaviate.queries.filtered.vector.durations_ms.count
(count)
The count of the duration of queries summary
weaviate.queries.filtered.vector.durations_ms.sum
(count)
The duration of queries in milliseconds
Shown as millisecond
weaviate.query.dimensions.count
(count)
The vector dimensions used by any read-query that involves vectors
weaviate.requests
(gauge)
The number of requests tagged by a given status(ok, usererror, servererror). Available only on Weaviate version 1.20.0+
Shown as request
weaviate.startup.diskio.throughput.bucket
(count)
The number of operations observed for the the disk I/O throughput duration by upper_bound buckets
Shown as operation
weaviate.startup.diskio.throughput.count
(count)
The count of operations observed in the disk I/O throughput duration histogram
Shown as operation
weaviate.startup.diskio.throughput.sum
(count)
The sum of disk I/O throughput startup operations in bytes/s, such as reading back the HNSW index or recovering LSM segments. The operation itself is defined by the operation label
Shown as byte
weaviate.startup.durations_ms.count
(count)
The number of samples used to calculate the duration individual startup operation
Shown as operation
weaviate.startup.durations_ms.sum
(count)
The duration of individual startup operations in ms. The operation itself is defined through the operation label
Shown as millisecond
weaviate.startup.progress
(gauge)
The ratio (percentage) of startup progress for a particular component in a shard
weaviate.vector.dimensions.count
(count)
The total dimensions in a shard
weaviate.vector.index.durations_ms.count
(count)
The number of samples used to calculate the duration of a sync or async vector index operation
Shown as operation
weaviate.vector.index.durations_ms.sum
(count)
The duration of regular vector index operation, such as insert or delete. The operation itself is defined through the operation label. The step label adds more granularity to each operation
Shown as millisecond
weaviate.vector.index.maintenance.durations_ms.count
(count)
The number of samples used to calculate the duration of a sync or async vector index operation
Shown as operation
weaviate.vector.index.maintenance.durations_ms.sum
(count)
The duration of a sync or async vector index maintenance operation. The operation itself is defined through the operation label
Shown as millisecond
weaviate.vector.index.operations
(gauge)
The total number of mutating operations on the vector index. The operation itself is defined by the operation label
Shown as operation
weaviate.vector.index.size
(gauge)
The total capacity of the vector index. Typically larger than the number of vectors imported as it grows proactively
weaviate.vector.index.tombstone.cleaned.count
(count)
The total number of deleted and removed vectors after repair operations
weaviate.vector.index.tombstone.cleanup.threads
(gauge)
The number of currently active threads for repairing/cleaning up the vector index after deletes have occurred
weaviate.vector.index.tombstones
(gauge)
The number of active vector index tombstones

ヘルプ

Weaviate インテグレーションには、イベントは含まれません。

ヘルプ

weaviate.openmetrics.health
Returns CRITICAL if the Agent is unable to connect to the Weaviate OpenMetrics endpoint, otherwise returns OK.
Statuses: ok, critical

weaviate.node.status
Returns CRITICAL if the node is UNAVAILABLE, WARNING if UNHEALTHY and OK if HEALTHY.
Statuses: ok, warning, critical

weaviate.liveness.status
Returns OK if liveness endpoint returns a 200 response, otherwise returns CRITICAL.
Statuses: ok, critical

ヘルプ

ご不明な点は、Datadog のサポートチームまでお問合せください。