Milvus

Supported OS Linux Windows Mac OS

통합 버전1.2.0
이 페이지는 아직 영어로 제공되지 않습니다. 번역 작업 중입니다.
현재 번역 프로젝트에 대한 질문이나 피드백이 있으신 경우 언제든지 연락주시기 바랍니다.

Overview

This check monitors Milvus through the Datadog Agent. It provides insights into your Milvus deployment’s performance by collecting information about the latency and number of executions of individual operations. This integration also allows for monitoring the size and ressource allocation of your deployment.

Setup

Installation

The Milvus check is included in the Datadog Agent package. No additional installation is needed on your server.

Configuration

Host

Follow the instructions below to configure this check for an Agent running on a host. For containerized environments, see the Containerized section.

    1. Edit the milvus.d/conf.yaml file, in the conf.d/ folder at the root of your Agent’s configuration directory to start collecting your Milvus performance data. See the sample milvus.d/conf.yaml for all available configuration options.

    2. Restart the Agent.

    Containerized

    For containerized environments, see the Autodiscovery Integration Templates for guidance on applying these instructions.

    Logs

    The Milvus integration can collect logs from the Milvus pods or containers.

      Apply this if you want to collect logs from Milvus standalone containers.

      1. Collecting logs is disabled by default in the Datadog Agent. Enable it in your datadog.yaml file:

        logs_enabled: true

      2. Uncomment and edit the logs configuration block in your milvus.d/conf.yaml file. Here’s an example:

        logs:
  - type: docker
    source: milvus
    service: milvus

      Apply this if you want to collect logs from a Milvus Kubernetes cluster.

      Collecting logs is disabled by default in the Datadog Agent. To enable it, see Kubernetes Log Collection.

      Then, set Log Integrations as pod annotations. This can also be configured with a file, a configmap, or a key-value store. For more information, see the configuration section of Kubernetes Log Collection.

      Validation

      Run the Agent’s status subcommand and look for milvus under the Checks section.

      Data Collected

      Metrics

      milvus.bf_search_cnt.bucket
      (count)      		Histogram bucket for the number of brute-force searches per request
      milvus.bf_search_cnt.count
      (count)      		Count aggregation of brute-force searches per request
      milvus.bf_search_cnt.sum
      (count)      		Sum aggregation of brute-force searches per request
      milvus.bitset_ratio.bucket
      (count)      		Bitset ratio histogram bucket
      milvus.bitset_ratio.count
      (count)      		Count of bitset ratio calculations
      milvus.bitset_ratio.sum
      (count)      		Sum of bitset ratios
      milvus.build_info
      (gauge)      		Build information of milvus
      milvus.build_latency.bucket
      (count)      		Index build latency histogram bucket
      milvus.build_latency.count
      (count)      		Count of index builds
      milvus.build_latency.sum
      (count)      		Sum of index build latencies(s)
      Shown as second
      milvus.cache_hit_cnt.bucket
      (count)      		Histogram bucket for the cache hit count per request
      milvus.cache_hit_cnt.count
      (count)      		Count aggregation of cache hit count per request
      milvus.cache_hit_cnt.sum
      (count)      		Sum aggregation of cache hit count per request
      milvus.cgo.active_future_total
      (gauge)      		Total number of active futures.
      milvus.cgo.cgo_duration_seconds.bucket
      (count)      		Histogram bucket of cgo call duration in seconds
      milvus.cgo.cgo_duration_seconds.count
      (count)      		Count of cgo calls
      milvus.cgo.cgo_duration_seconds.sum
      (count)      		Sum of cgo call duration in seconds.
      Shown as second
      milvus.cgo.cgo_queue_duration_seconds.bucket
      (count)      		Histogram bucket for the duration of cgo calls in queue
      milvus.cgo.cgo_queue_duration_seconds.count
      (count)      		Counnt of cgo calls in queue
      milvus.cgo.cgo_queue_duration_seconds.sum
      (count)      		Sum of the duration of cgo calls in queue
      Shown as second
      milvus.cgo.running_cgo_call_total
      (gauge)      		Total number of running cgo calls.
      milvus.datacoord.channel_checkpoint_unix_seconds
      (gauge)      		Channel checkpoint timestamp in Unix seconds
      milvus.datacoord.collection_num
      (gauge)      		Number of collections
      milvus.datacoord.consume_datanode_tt_lag_ms
      (gauge)      		Now time minus time travel per physical channel
      Shown as millisecond
      milvus.datacoord.datanode_num
      (gauge)      		Number of data nodes
      milvus.datacoord.import_tasks
      (gauge)      		The import tasks grouping by type and state
      milvus.datacoord.index.node_num
      (gauge)      		Number of IndexNodes managed by IndexCoord
      milvus.datacoord.index.req.count
      (count)      		Number of building index requests
      milvus.datacoord.index.task
      (gauge)      		Number of index tasks of each type
      milvus.datacoord.segment_num
      (gauge)      		Number of segments
      milvus.datacoord.stored.binlog_size
      (gauge)      		Binlog size of healthy segments
      Shown as byte
      milvus.datacoord.stored.index_files_size
      (gauge)      		Index files size of the segments
      Shown as byte
      milvus.datacoord.stored.rows_num
      (gauge)      		Number of stored rows of healthy segment
      milvus.datacoord.task_execute_max_latency.bucket
      (count)      		Histogram bucket for the latency of task execute operation
      milvus.datacoord.task_execute_max_latency.count
      (count)      		Count of task execute operations
      milvus.datacoord.task_execute_max_latency.sum
      (count)      		Sum of the latency of task execute operations
      Shown as millisecond
      milvus.datacoord.watched_dml_chanel_num
      (gauge)      		The num of data manipulation language channel watched by datanode
      milvus.datanode.autoflush_buffer_op.count
      (count)      		Count of auto flush buffer operations
      milvus.datanode.consume.bytes.count
      (count)      		Count of consumed bytes
      milvus.datanode.consume.msg.count
      (count)      		Count of consumed msg
      milvus.datanode.consume.tt_lag_ms
      (gauge)      		Now time minus time travel per physical channel
      Shown as millisecond
      milvus.datanode.encode_buffer_latency.bucket
      (count)      		Histogram bucket for the latency of encode buffer data
      milvus.datanode.encode_buffer_latency.count
      (count)      		Count of encode buffer data operations
      milvus.datanode.encode_buffer_latency.sum
      (count)      		Sum of the latency of encode buffer data operations
      Shown as millisecond
      milvus.datanode.flowgraph_num
      (gauge)      		Number of flowgraphs
      milvus.datanode.flush.buffer_op.count
      (count)      		Count of flush buffer operations
      milvus.datanode.flush.req.count
      (count)      		Count of flush request
      milvus.datanode.flushed_data.rows.count
      (count)      		Num of rows flushed to storage
      milvus.datanode.flushed_data.size.count
      (count)      		Byte size of data flushed to storage
      Shown as byte
      milvus.datanode.msg.dispatcher_tt_lag_ms
      (gauge)      		time.Now() sub dispatcher's current consume time
      Shown as millisecond
      milvus.datanode.msg.rows.count
      (count)      		Count of rows consumed from msgStream
      milvus.datanode.save_latency.bucket
      (count)      		Histogram bucket for the latency of saving flush data to storage
      milvus.datanode.save_latency.count
      (count)      		Count of saving flush data to storage events
      milvus.datanode.save_latency.sum
      (count)      		Sum of the latency of saving flush data to storage events
      Shown as millisecond
      milvus.diskann.range_search_iters.bucket
      (count)      		Histogram bucket for DISKANN range search iterations
      milvus.diskann.range_search_iters.count
      (count)      		Count of DISKANN range search events
      milvus.diskann.range_search_iters.sum
      (count)      		Sum of DISKANN range search iterations
      milvus.diskann.search_hops.bucket
      (count)      		Histogram for DISKANN search hops
      milvus.diskann.search_hops.count
      (count)      		Count of DISKANN search hop events
      milvus.diskann.search_hops.sum
      (count)      		Sum of DISKANN search hops
      milvus.diskann_bitset_ratio.bucket
      (count)      		Histogram bucket for the DISKANN bitset ratio for search and range search
      milvus.diskann_bitset_ratio.count
      (count)      		Count of the DISKANN bitset ratio operations for search and range search
      milvus.diskann_bitset_ratio.sum
      (count)      		Sum of the DISKANN bitset ratio for search and range search
      milvus.exec_latency.bucket
      (count)      		Histogram bucket for the execute latency per request
      milvus.exec_latency.count
      (count)      		Count aggregation of the execute latency per request
      milvus.exec_latency.sum
      (count)      		Sum aggregation of the execute latency per request
      milvus.filter.connectivity_ratio.bucket
      (count)      		Histogram bucket for the average connectivity ratio set under filtering per request
      milvus.filter.connectivity_ratio.count
      (count)      		Count of the connectivity ratios set under filtering per request
      milvus.filter.connectivity_ratio.sum
      (count)      		Sum of the average connectivity ratios set under filtering per request
      milvus.filter.mv.activated_fields_cnt.bucket
      (count)      		Histogram bucket of the average materialized view activated fields per request
      milvus.filter.mv.activated_fields_cnt.count
      (count)      		Count of the requests activating materialized view fields
      milvus.filter.mv.activated_fields_cnt.sum
      (count)      		Sum of the average materialized view activated fields per request
      milvus.filter.mv.change_base_cnt.bucket
      (count)      		Histogram bucket for the materialized view change base count per request
      milvus.filter.mv.change_base_cnt.count
      (count)      		Count of requests triggering materialized view change base events
      milvus.filter.mv.change_base_cnt.sum
      (count)      		Sum of materialized view change base counts per request
      milvus.filter.mv.only_cnt.bucket
      (count)      		Histogram bucket for the materialized view only count per request
      milvus.filter.mv.only_cnt.count
      (count)      		Count of requests that trigger materialized view only events
      milvus.filter.mv.only_cnt.sum
      (count)      		Sum of materialized view only counts
      milvus.filter.mv.supplement_ep_bool_cnt.bucket
      (count)      		Histogram bucket for materialized view supplement entry point from bitset boolean count per request
      milvus.filter.mv.supplement_ep_bool_cnt.count
      (count)      		Count of requests triggering a materialized view supplement entry point from bitset boolean event
      milvus.filter.mv.supplement_ep_bool_cnt.sum
      (count)      		Sum of materialized view supplement entry point from bitset boolean events
      milvus.flushed_segment_file_num.bucket
      (count)      		Histogram bucket for the number of files for flushed segment
      milvus.flushed_segment_file_num.count
      (count)      		Count of flushed segment events
      milvus.flushed_segment_file_num.sum
      (count)      		Sum of the number of files for flushed segments
      milvus.go.gc_duration_seconds.count
      (count)      		A summary of the pause duration of garbage collection cycles.
      milvus.go.gc_duration_seconds.quantile
      (gauge)      		A summary of the pause duration of garbage collection cycles.
      milvus.go.gc_duration_seconds.sum
      (count)      		A summary of the pause duration of garbage collection cycles.
      milvus.go.goroutines
      (gauge)      		Number of goroutines that currently exist.
      milvus.go.info
      (gauge)      		Information about the Go environment.
      milvus.go.memstats.alloc_bytes
      (gauge)      		Number of bytes allocated and still in use.
      milvus.go.memstats.alloc_bytes.count
      (count)      		Total number of bytes allocated, even if freed.
      milvus.go.memstats.buck_hash_sys_bytes
      (gauge)      		Number of bytes used by the profiling bucket hash table.
      milvus.go.memstats.frees.count
      (count)      		Total number of frees.
      milvus.go.memstats.gc_sys_bytes
      (gauge)      		Number of bytes used for garbage collection system metadata.
      milvus.go.memstats.heap.alloc_bytes
      (gauge)      		Number of heap bytes allocated and still in use.
      milvus.go.memstats.heap.idle_bytes
      (gauge)      		Number of heap bytes waiting to be used.
      milvus.go.memstats.heap.inuse_bytes
      (gauge)      		Number of heap bytes that are in use.
      milvus.go.memstats.heap.objects
      (gauge)      		Number of allocated objects.
      milvus.go.memstats.heap.released_bytes
      (gauge)      		Number of heap bytes released to OS.
      milvus.go.memstats.heap.sys_bytes
      (gauge)      		Number of heap bytes obtained from system.
      milvus.go.memstats.last_gc_time_seconds
      (gauge)      		Number of seconds since 1970 of last garbage collection.
      milvus.go.memstats.lookups.count
      (count)      		Total number of pointer lookups.
      milvus.go.memstats.mallocs.count
      (count)      		Total number of mallocs.
      milvus.go.memstats.mcache.inuse_bytes
      (gauge)      		Number of bytes in use by mcache structures.
      milvus.go.memstats.mcache.sys_bytes
      (gauge)      		Number of bytes used for mcache structures obtained from system.
      milvus.go.memstats.mspan.inuse_bytes
      (gauge)      		Number of bytes in use by mspan structures.
      milvus.go.memstats.mspan.sys_bytes
      (gauge)      		Number of bytes used for mspan structures obtained from system.
      milvus.go.memstats.next_gc_bytes
      (gauge)      		Number of heap bytes when next garbage collection will take place.
      milvus.go.memstats.other_sys_bytes
      (gauge)      		Number of bytes used for other system allocations.
      milvus.go.memstats.stack.inuse_bytes
      (gauge)      		Number of bytes in use by the stack allocator.
      milvus.go.memstats.stack.sys_bytes
      (gauge)      		Number of bytes obtained from system for stack allocator.
      milvus.go.memstats.sys_bytes
      (gauge)      		Number of bytes obtained from system.
      milvus.go.threads
      (gauge)      		Number of OS threads created.
      milvus.graph_search_cnt.bucket
      (count)      		Histogram bucket for the number of graph search per request
      milvus.graph_search_cnt.count
      (count)      		Count of requests that trigger a graph search
      milvus.graph_search_cnt.sum
      (count)      		Sum of graph searches
      milvus.hnsw.bitset_ratio.bucket
      (count)      		Histogram bucket of the HNSW bitset ratio for search and range search
      milvus.hnsw.bitset_ratio.count
      (count)      		Count aggregation of the HNSW bitset ratio for search and range search
      milvus.hnsw.bitset_ratio.sum
      (count)      		Sum aggregation of the HNSW bitset ratio for search and range search
      milvus.hnsw.search_hops.bucket
      (count)      		Histogram bucket of the HNSW search hops in layer 0
      milvus.hnsw.search_hops.count
      (count)      		Count aggregation of the HNSW search hops in layer 0
      milvus.hnsw.search_hops.sum
      (count)      		Sum aggregation of the HNSW search hops in layer 0
      milvus.indexnode.build_index_latency.bucket
      (count)      		Histogram bucket of the latency of build index for segment
      milvus.indexnode.build_index_latency.count
      (count)      		Count of build index events
      milvus.indexnode.build_index_latency.sum
      (count)      		Sum of the latency of build index events
      Shown as millisecond
      milvus.indexnode.encode_index_latency.bucket
      (count)      		Histogram bucket of the latency of encoding the index file
      milvus.indexnode.encode_index_latency.count
      (count)      		Count of the index file encoding events
      milvus.indexnode.encode_index_latency.sum
      (count)      		Sum of the latency of index file encoding events
      Shown as millisecond
      milvus.indexnode.index.task.count
      (count)      		Number of tasks that index node received
      milvus.indexnode.index.task_latency_in_queue.bucket
      (count)      		Histogram bucket for the latency of index task in queue
      milvus.indexnode.index.task_latency_in_queue.count
      (count)      		Count of index task in queue
      milvus.indexnode.index.task_latency_in_queue.sum
      (count)      		Sum of the latency of index task in queue
      Shown as millisecond
      milvus.indexnode.knowhere_build_index_latency.bucket
      (count)      		Histogram bucket for the latency of building the index by knowhere
      milvus.indexnode.knowhere_build_index_latency.count
      (count)      		Count of index builds by knowhere
      milvus.indexnode.knowhere_build_index_latency.sum
      (count)      		Sum of the latency of building the index by knowhere
      Shown as millisecond
      milvus.indexnode.save_index_latency.bucket
      (count)      		Histogram bucket for the latency of saving the index file
      milvus.indexnode.save_index_latency.count
      (count)      		Count of index file save envents
      milvus.indexnode.save_index_latency.sum
      (count)      		Sum of the latency of saving the index file
      Shown as millisecond
      milvus.internal.core_search_latency.bucket
      (count)      		Histogram bucket for the [cpp]latency(us) of search on segment
      milvus.internal.core_search_latency.count
      (count)      		Count aggregation of [cpp]latency(us) of search on segment
      milvus.internal.core_search_latency.sum
      (count)      		Sum aggregation of [cpp]latency(us) of search on segment
      Shown as microsecond
      milvus.internal.mmap.allocated_space_bytes.bucket
      (count)      		Histogram bucket for the [cpp]mmap allocated space stats
      milvus.internal.mmap.allocated_space_bytes.count
      (count)      		Count aggregation of [cpp]mmap allocated space stats
      milvus.internal.mmap.allocated_space_bytes.sum
      (count)      		Sum aggregation of [cpp]mmap allocated space stats
      milvus.internal.mmap.in_used_space_bytes
      (gauge)      		[cpp]mmap in used space stats
      milvus.internal.storage.kv_size.bucket
      (count)      		Histogram bucket for the [cpp]key-value size stats
      milvus.internal.storage.kv_size.count
      (count)      		Count aggregation of [cpp]key-value size stats
      milvus.internal.storage.kv_size.sum
      (count)      		Sum aggregation of [cpp]key-value size stats
      milvus.internal.storage.load_duration.bucket
      (count)      		Histogram bucket for the [cpp]durations of load segment
      milvus.internal.storage.load_duration.count
      (count)      		Count aggregation of [cpp]durations of load segment
      milvus.internal.storage.load_duration.sum
      (count)      		Sum aggregation of [cpp]durations of load segment
      milvus.internal.storage.op.count
      (count)      		[cpp]count of persistent data operation
      milvus.internal.storage.request_latency.bucket
      (count)      		Histogram bucket for the [cpp]request latency(ms) on the client side
      milvus.internal.storage.request_latency.count
      (count)      		Count aggregation of [cpp]request latency(ms) on the client side
      milvus.internal.storage.request_latency.sum
      (count)      		Sum aggregation of [cpp]request latency(ms) on the client side
      Shown as millisecond
      milvus.io_cnt.bucket
      (count)      		Histogram bucket for the IO count per request
      milvus.io_cnt.count
      (count)      		Count of requests triggering IO operations
      milvus.io_cnt.sum
      (count)      		Sum of IO operation counts
      milvus.ivf_search_cnt.bucket
      (count)      		Histogram bucket for the number of inverted file search per request
      milvus.ivf_search_cnt.count
      (count)      		Count of requests trigerring an inverted file search
      milvus.ivf_search_cnt.sum
      (count)      		Sum of inverted file searches
      milvus.load_latency.bucket
      (count)      		Histogram bucket for the index load latency (ms)
      milvus.load_latency.count
      (count)      		Count of index load events
      milvus.load_latency.sum
      (count)      		Sum of index load latency (ms)
      Shown as millisecond
      milvus.meta.kv_size.bucket
      (count)      		Histogram for the key-value size stats
      milvus.meta.kv_size.count
      (count)      		Count aggregation of the key-value size stats
      milvus.meta.kv_size.sum
      (count)      		Sum aggregation of the key-value size stats
      milvus.meta.op.count
      (count)      		Count of meta operation
      milvus.meta.request_latency.bucket
      (count)      		Histogram bucket for the request latency on the client side
      milvus.meta.request_latency.count
      (count)      		Count aggregation of the request latency on the client side
      milvus.meta.request_latency.sum
      (count)      		Sum aggregation of the request latency on the client side
      Shown as millisecond
      milvus.msg_queue_consumer_num
      (gauge)      		Number of consumers
      milvus.msgstream.op.count
      (count)      		Count of stream message operation
      milvus.msgstream.request_latency.bucket
      (count)      		Histogram bucket for the request latency on the client side
      milvus.msgstream.request_latency.count
      (count)      		Count aggregation of the request latency on the client side
      milvus.msgstream.request_latency.sum
      (count)      		Sum aggregation of the request latency on the client side
      Shown as millisecond
      milvus.num_node
      (gauge)      		Number of nodes and coordinates
      milvus.process.cpu_seconds.count
      (count)      		Total user and system CPU time spent in seconds.
      milvus.process.max_fds
      (gauge)      		Maximum number of open file descriptors.
      milvus.process.open_fds
      (gauge)      		Number of open file descriptors.
      milvus.process.resident_memory_bytes
      (gauge)      		Resident memory size in bytes.
      milvus.process.start_time_seconds
      (gauge)      		Start time of the process since Unix epoch in seconds.
      milvus.process.virtual_memory.bytes
      (gauge)      		Virtual memory size in bytes.
      milvus.process.virtual_memory.max_bytes
      (gauge)      		Maximum amount of virtual memory available in bytes.
      milvus.proxy.apply.pk_latency.bucket
      (count)      		Histogram bucket for the latency of 'apply primary key' events
      milvus.proxy.apply.pk_latency.count
      (count)      		Count of 'apply primary key' events
      milvus.proxy.apply.pk_latency.sum
      (count)      		Sum of latencies for 'apply primary key' events
      Shown as millisecond
      milvus.proxy.apply.timestamp_latency.bucket
      (count)      		Histogram bucket for the latency of proxy 'apply timestamp' events
      milvus.proxy.apply.timestamp_latency.count
      (count)      		Count of proxy 'apply timestamp' events
      milvus.proxy.apply.timestamp_latency.sum
      (count)      		Sum of latencies of proxy 'apply timestamp' events
      Shown as millisecond
      milvus.proxy.assign_segmentID_latency.bucket
      (count)      		Histogram bucket for the latency of proxy 'get segmentID from dataCoord' events
      milvus.proxy.assign_segmentID_latency.count
      (count)      		Count of proxy 'get segmentID from dataCoord' events
      milvus.proxy.assign_segmentID_latency.sum
      (count)      		Sum of latencies for proxy 'get segmentID from dataCoord' events
      Shown as millisecond
      milvus.proxy.cache.hit.count
      (count)      		Count of cache hits/miss
      milvus.proxy.cache.update_latency.bucket
      (count)      		Histogram bucket for the latency of proxy 'update cache when cache miss' events
      milvus.proxy.cache.update_latency.count
      (count)      		Count of proxy 'update cache when cache miss' events
      milvus.proxy.cache.update_latency.sum
      (count)      		Sum of latencies of proxy 'update cache when cache miss'
      Shown as millisecond
      milvus.proxy.delete_vectors.count
      (count)      		Counter of vectors successfully deleted
      milvus.proxy.msgstream_obj_num
      (gauge)      		Number of MsgStream objects per physical channel
      milvus.proxy.mutation_send_latency.bucket
      (count)      		Histogram bucket for the latency of proxy 'send insert request to MsgStream' events
      milvus.proxy.mutation_send_latency.count
      (count)      		Count of proxy 'send insert request to MsgStream' events
      milvus.proxy.mutation_send_latency.sum
      (count)      		Sum of latencies of proxy 'send insert request to MsgStream' events
      Shown as millisecond
      milvus.proxy.rate_limit_req.count
      (count)      		Count of operation executed
      milvus.proxy.report_value.count
      (count)      		Report value about the request
      milvus.proxy.req.count
      (count)      		Count of operation executed
      milvus.proxy.req.in_queue_latency.bucket
      (count)      		Histogram bucket for the time in queue of request
      milvus.proxy.req.in_queue_latency.count
      (count)      		Count of request in queue
      milvus.proxy.req.in_queue_latency.sum
      (count)      		Sum of the time requests spend in queue
      Shown as millisecond
      milvus.proxy.req.latency.bucket
      (count)      		Histogram bucket for the latency of each request
      milvus.proxy.req.latency.count
      (count)      		Count of request latency events
      milvus.proxy.req.latency.sum
      (count)      		Sum of latencies of requests
      Shown as millisecond
      milvus.proxy.send_bytes.count
      (count)      		Count of bytes sent back to sdk
      milvus.proxy.sq.decode_result_latency.bucket
      (count)      		Histogram bucket for the latency of proxy 'decode the search result' events
      milvus.proxy.sq.decode_result_latency.count
      (count)      		Count of proxy 'decode the search result' events
      milvus.proxy.sq.decode_result_latency.sum
      (count)      		Sum of latencies for proxy 'decode the search result' events
      Shown as millisecond
      milvus.proxy.sq.reduce_result_latency.bucket
      (count)      		Histogram bucket for the latency of proxy 'reduces search result' events
      milvus.proxy.sq.reduce_result_latency.count
      (count)      		Count of proxy 'reduces search result' events
      milvus.proxy.sq.reduce_result_latency.sum
      (count)      		Sum of latencies for proxy 'reduces search result' events
      Shown as millisecond
      milvus.proxy.sq.wait_result_latency.bucket
      (count)      		Histogram bucket for the latency of proxy 'waits for the result' events
      milvus.proxy.sq.wait_result_latency.count
      (count)      		Count of proxy 'waits for the result' events
      milvus.proxy.sq.wait_result_latency.sum
      (count)      		Sum of latencies for proxy 'waits for the result' events
      Shown as millisecond
      milvus.proxy.sync_segment_request_length.bucket
      (count)      		Histogram bucket for the length of SegmentIDRequests when assigning segments for insert
      milvus.proxy.sync_segment_request_length.count
      (count)      		Count of assigning segments for insert events
      milvus.proxy.sync_segment_request_length.sum
      (count)      		Sum of the length of SegmentIDRequests when assigning segments for insert
      milvus.proxy.tt_lag_ms
      (gauge)      		Now time minus time travel per physical channel
      Shown as millisecond
      milvus.quant.compute_cnt.bucket
      (count)      		Histogram bucket for the quant compute count per request
      milvus.quant.compute_cnt.count
      (count)      		Count of requests triggering quant compute events
      milvus.quant.compute_cnt.sum
      (count)      		Sum of quant compute event counts
      milvus.querycoord.collection_num
      (gauge)      		Number of collections
      milvus.querycoord.current_target_checkpoint_unix_seconds
      (gauge)      		Current target checkpoint timestamp in Unix seconds
      milvus.querycoord.load.latency.bucket
      (count)      		Histogram bucket Latency of load the entire collection
      milvus.querycoord.load.latency.count
      (count)      		Latency of load the entire collection
      milvus.querycoord.load.latency.sum
      (count)      		Latency of load the entire collection
      Shown as millisecond
      milvus.querycoord.load.req.count
      (count)      		Count of load request
      milvus.querycoord.partition_num
      (gauge)      		Number of partitions
      milvus.querycoord.querynode_num
      (gauge)      		Number of QueryNodes managered by QueryCoord
      milvus.querycoord.release.latency.bucket
      (count)      		Histogram bucket for the latency of release request
      milvus.querycoord.release.latency.count
      (count)      		Count of release request events
      milvus.querycoord.release.latency.sum
      (count)      		Sum of the latencies of release request events
      Shown as millisecond
      milvus.querycoord.release.req.count
      (count)      		Count of release request
      milvus.querycoord_task_num
      (gauge)      		the number of tasks in QueryCoord's scheduler
      milvus.querynode.apply_bf_latency.bucket
      (count)      		Histogram bucket for the apply brute-force cost in ms
      milvus.querynode.apply_bf_latency.count
      (count)      		Count of apply brute-force events
      milvus.querynode.apply_bf_latency.sum
      (count)      		Sum of the apply brute-force cost in ms
      Shown as millisecond
      milvus.querynode.collection_num
      (gauge)      		Number of collections loaded
      milvus.querynode.consume.bytes_counter.count
      (count)      		Count of consumed bytes
      milvus.querynode.consume.msg.count
      (count)      		Count of consumed msg
      milvus.querynode.consume.tt_lag_ms
      (gauge)      		Now time minus time travel per physical channel
      Shown as millisecond
      milvus.querynode.disk.cache.evict.bytes.count
      (count)      		Number of bytes evicted from disk cache
      Shown as byte
      milvus.querynode.disk.cache.evict.count
      (count)      		Number of segments evicted from disk cache
      milvus.querynode.disk.cache.evict.duration.count
      (count)      		Total time cost of evicting segments from disk cache
      milvus.querynode.disk.cache.evict.global_duration.bucket
      (count)      		Histogram bucket for the global duration of evicting segments from disk cache
      milvus.querynode.disk.cache.evict.global_duration.count
      (count)      		Count of evicting segments from disk cache events
      milvus.querynode.disk.cache.evict.global_duration.sum
      (count)      		Sum of the global duration of evicting segments from disk cache
      Shown as millisecond
      milvus.querynode.disk.cache.load.bytes.count
      (count)      		Number of bytes loaded from disk cache
      Shown as byte
      milvus.querynode.disk.cache.load.count
      (count)      		Number of segments loaded from disk cache
      milvus.querynode.disk.cache.load.duration.count
      (count)      		Total time cost of loading segments from disk cache
      milvus.querynode.disk.cache.load.global_duration.bucket
      (count)      		Histogram bucket for the global duration of loading segments from disk cache
      milvus.querynode.disk.cache.load.global_duration.count
      (count)      		Count of loading segments from disk cache events
      milvus.querynode.disk.cache.load.global_duration.sum
      (count)      		Sum of the global duration of loading segments from disk cache
      Shown as millisecond
      milvus.querynode.disk.used_size
      (gauge)      		Disk used size(MB)
      Shown as mebibyte
      milvus.querynode.dml_vchannel_num
      (gauge)      		Number of data manipulation language channels watched
      milvus.querynode.entity.num
      (gauge)      		Number of entities which can be searched/queried, clustered by collection, partition and state
      milvus.querynode.entity.size
      (gauge)      		Entities' memory size, clustered by collection and state
      Shown as byte
      milvus.querynode.execute_bytes_counter.count
      (count)      		Count of execute bytes
      milvus.querynode.flowgraph_num
      (gauge)      		Number of flowgraphs
      milvus.querynode.forward_delete_latency.bucket
      (count)      		Histogram bucket for the forward delete cost in ms
      milvus.querynode.forward_delete_latency.count
      (count)      		Count of forward delete events
      milvus.querynode.forward_delete_latency.sum
      (count)      		Sum of the forward delete cost in ms
      Shown as millisecond
      milvus.querynode.load.index_latency.bucket
      (count)      		Histogram bucket for the latency of load per segment's index, in milliseconds
      milvus.querynode.load.index_latency.count
      (count)      		Count of load per segment's index events
      milvus.querynode.load.index_latency.sum
      (count)      		Sum of the latencies of load per segment's index events, in milliseconds
      Shown as millisecond
      milvus.querynode.load.segment.concurrency
      (gauge)      		Number of concurrent loading segments in QueryNode
      milvus.querynode.load.segment.latency.bucket
      (count)      		Histogram bucket for the latency of load per segment
      milvus.querynode.load.segment.latency.count
      (count)      		Count of load per segment events
      milvus.querynode.load.segment.latency.sum
      (count)      		Sum of the latencies of load per segment events
      Shown as millisecond
      milvus.querynode.msg_dispatcher_tt_lag_ms
      (gauge)      		time.Now() sub dispatcher's current consume time
      Shown as millisecond
      milvus.querynode.partition_num
      (gauge)      		Number of partitions loaded
      milvus.querynode.process_insert_or_delete_latency.bucket
      (count)      		Histogram bucket for the process insert or delete cost in ms
      milvus.querynode.process_insert_or_delete_latency.count
      (count)      		Count of process insert or delete events
      milvus.querynode.process_insert_or_delete_latency.sum
      (count)      		Sum of process insert or delete cost in ms
      Shown as millisecond
      milvus.querynode.read_task.concurrency
      (gauge)      		Number of concurrent executing read tasks in QueryNode
      milvus.querynode.read_task.ready_len
      (gauge)      		Number of ready read tasks in readyQueue
      milvus.querynode.read_task.unsolved_len
      (gauge)      		Number of unsolved read tasks in unsolvedQueue
      milvus.querynode.search.group.nq.bucket
      (count)      		Histogram bucket for the number of queries of each grouped search task
      milvus.querynode.search.group.nq.count
      (count)      		Count of grouped search tasks
      milvus.querynode.search.group.nq.sum
      (count)      		Sum of the number of queries of grouped search tasks
      milvus.querynode.search.group.size.bucket
      (count)      		Histogram bucket for the number of tasks of each grouped search task
      milvus.querynode.search.group.size.count
      (count)      		Count of grouped search tasks
      milvus.querynode.search.group.size.sum
      (count)      		Sum of the number of tasks of grouped search tasks
      milvus.querynode.search.group.topk.bucket
      (count)      		Histogram bucket for the topK of each grouped search task
      milvus.querynode.search.group.topk.count
      (count)      		Count of grouped search tasks
      milvus.querynode.search.group.topk.sum
      (count)      		Sum of the topK of grouped search tasks
      milvus.querynode.search.nq.bucket
      (count)      		Histogram bucket for the number of queries of each search task
      milvus.querynode.search.nq.count
      (count)      		Count of search tasks
      milvus.querynode.search.nq.sum
      (count)      		Sum of the number of queries of search tasks
      milvus.querynode.search.topk.bucket
      (count)      		Histogram bucket for the top of each search task
      milvus.querynode.search.topk.count
      (count)      		Count of search tasks
      milvus.querynode.search.topk.sum
      (count)      		Sum of the top of search tasks
      milvus.querynode.segment.access.count
      (count)      		Number of segments accessed
      milvus.querynode.segment.access.duration.count
      (count)      		Total time cost of accessing segments
      milvus.querynode.segment.access.global_duration.bucket
      (count)      		Histogram bucket for the global time cost of accessing segments
      milvus.querynode.segment.access.global_duration.count
      (count)      		Count of accessing segments events
      milvus.querynode.segment.access.global_duration.sum
      (count)      		Sum of the global time cost of accessing segments
      Shown as millisecond
      milvus.querynode.segment.access.wait_cache.count
      (count)      		Number of segments waiting for loading access
      milvus.querynode.segment.access.wait_cache.duration.count
      (count)      		Total time cost of waiting for loading access
      milvus.querynode.segment.access.wait_cache.global_duration.bucket
      (count)      		Histogram bucket for the global time cost of waiting for loading access
      milvus.querynode.segment.access.wait_cache.global_duration.count
      (count)      		Count of waiting for loading access events
      milvus.querynode.segment.access.wait_cache.global_duration.sum
      (count)      		Sum of the global time cost of waiting for loading access events
      Shown as millisecond
      milvus.querynode.segment.latency_per_vector.bucket
      (count)      		Histogram bucket for one vector's search latency per segment
      milvus.querynode.segment.latency_per_vector.count
      (count)      		Count aggregation of one vector's search latency per segment
      milvus.querynode.segment.latency_per_vector.sum
      (count)      		Sum aggregation of one vector's search latency per segment
      Shown as millisecond
      milvus.querynode.segment.num
      (gauge)      		Number of segments loaded, clustered by its collection, partition, state and # of indexed fields
      milvus.querynode.sq.core_latency.bucket
      (count)      		Histogram bucket for the latency of searches or queries in segcore
      milvus.querynode.sq.core_latency.count
      (count)      		Count of search or query events in segcore
      milvus.querynode.sq.core_latency.sum
      (count)      		Sum of latencies of searches or queries in segcore
      Shown as millisecond
      milvus.querynode.sq.queue.latency.bucket
      (count)      		Histogram bucket for the latency of searches or queries in queue
      milvus.querynode.sq.queue.latency.count
      (count)      		Count of search or query events
      milvus.querynode.sq.queue.latency.sum
      (count)      		Sum of latencies of searches or queries in queue
      Shown as millisecond
      milvus.querynode.sq.queue.user_latency.bucket
      (count)      		Histgram bucket for the latency per user of search or query in queue
      milvus.querynode.sq.queue.user_latency.count
      (count)      		Count of search or query in queue
      milvus.querynode.sq.queue.user_latency.sum
      (count)      		Sum of latencies per user of search or query in queue
      Shown as millisecond
      milvus.querynode.sq.reduce_latency.bucket
      (count)      		Histogram bucket for the latencies of reduce search or query result
      milvus.querynode.sq.reduce_latency.count
      (count)      		Count of reduce search or query result events
      milvus.querynode.sq.reduce_latency.sum
      (count)      		Sum of latencies of reduce search or query result
      Shown as millisecond
      milvus.querynode.sq.req.count
      (count)      		count of search / query request
      milvus.querynode.sq.req.latency.bucket
      (count)      		Histogram bucket for the latency of search or query requests
      milvus.querynode.sq.req.latency.count
      (count)      		Count of search or query requests
      milvus.querynode.sq.req.latency.sum
      (count)      		Sum of latencies of search or query requests
      Shown as millisecond
      milvus.querynode.sq.segment_latency.bucket
      (count)      		Histogram bucket for the latency of search or query per segment
      milvus.querynode.sq.segment_latency.count
      (count)      		Count of search or query per segment
      milvus.querynode.sq.segment_latency.sum
      (count)      		Sum of latencies of search or query per segment
      Shown as millisecond
      milvus.querynode.sq.wait_tsafe_latency.bucket
      (count)      		Histogram bucket for the time that search or query waits for tsafe
      milvus.querynode.sq.wait_tsafe_latency.count
      (count)      		Count of occurrences of search or query waiting for tsafe
      milvus.querynode.sq.wait_tsafe_latency.sum
      (count)      		Sum of the time that search or query wait for tsafe
      Shown as millisecond
      milvus.querynode.wait_processing_msg
      (gauge)      		Count of wait processing msg
      milvus.querynode.watch_dml_channel_latency.bucket
      (count)      		Histogram bucket for the latencies of watch data manipulation language channel
      milvus.querynode.watch_dml_channel_latency.count
      (count)      		Count of watch data manipulation language channel events
      milvus.querynode.watch_dml_channel_latency.sum
      (count)      		Sum of the latencies of watch data manipulation language channel events
      Shown as millisecond
      milvus.queue.latency.bucket
      (count)      		Histogram bucket for the queue latency per request
      milvus.queue.latency.count
      (count)      		Count of request with queue latency
      milvus.queue.latency.sum
      (count)      		Sum of queue latencies
      milvus.range_search_latency.bucket
      (count)      		Histogram bucket for range search latency (ms)
      milvus.range_search_latency.count
      (count)      		Count of range search operations
      milvus.range_search_latency.sum
      (count)      		Sum of range search latencies (ms)
      Shown as millisecond
      milvus.raw_compute_cnt.bucket
      (count)      		Histogram bucket for the raw compute count per request
      milvus.raw_compute_cnt.count
      (count)      		Count of request that trigger raw compute operations
      milvus.raw_compute_cnt.sum
      (count)      		Sum of raw compute operations
      milvus.re_search_cnt.bucket
      (count)      		Histogram bucket for the number of fallback search per request
      milvus.re_search_cnt.count
      (count)      		Count of requests triggering fallback search operations
      milvus.re_search_cnt.sum
      (count)      		Sum of fallback searches
      milvus.rootcoord.collection_num
      (gauge)      		Number of collections
      milvus.rootcoord.credential_num
      (gauge)      		Number of credentials
      milvus.rootcoord.ddl_req.count
      (count)      		Count of DDL operations
      milvus.rootcoord.ddl_req.latency.bucket
      (count)      		Histogram bucket for the latency of each DDL operation
      milvus.rootcoord.ddl_req.latency.count
      (count)      		Count of DDL operations
      milvus.rootcoord.ddl_req.latency.sum
      (count)      		Sum of the latencies of DDL operations
      Shown as millisecond
      milvus.rootcoord.ddl_req.latency_in_queue.bucket
      (count)      		Histogram bucket for the latencies of each DDL operation in queue
      milvus.rootcoord.ddl_req.latency_in_queue.count
      (count)      		Count of DDL operations in queue
      milvus.rootcoord.ddl_req.latency_in_queue.sum
      (count)      		Sum of the latencies of DDL operations in queue
      Shown as millisecond
      milvus.rootcoord.dml_channel_num
      (gauge)      		Number of DML channels
      milvus.rootcoord.entity_num
      (gauge)      		Number of entities, clustered by collection and their status(loaded/total)
      milvus.rootcoord.force_deny_writing_counter.count
      (count)      		The number of times milvus turns into force-deny-writing states
      milvus.rootcoord.id_alloc.count
      (count)      		Count of ID allocated
      milvus.rootcoord.indexed_entity_num
      (gauge)      		Indexed number of entities, clustered by collection, index name and whether it's a vector index
      milvus.rootcoord.msgstream_obj_num
      (gauge)      		Number of message streams
      milvus.rootcoord.num_of_roles
      (gauge)      		The number of roles
      milvus.rootcoord.partition_num
      (gauge)      		Number of partitions
      milvus.rootcoord.produce_tt_lag_ms
      (gauge)      		Now time minus time travel per physical channel
      Shown as millisecond
      milvus.rootcoord.proxy_num
      (gauge)      		Number of proxy nodes managered by rootcoord
      milvus.rootcoord.qn_mem_high_water_level
      (gauge)      		Querynode memory high water level
      milvus.rootcoord.sync_timetick_latency.bucket
      (count)      		Histogram bucket for the latencies of synchronizing timetick message
      milvus.rootcoord.sync_timetick_latency.count
      (count)      		Count of synchronizing timetick message events
      milvus.rootcoord.sync_timetick_latency.sum
      (count)      		Sum of latencies of synchronizing timetick message events
      Shown as millisecond
      milvus.rootcoord.timestamp
      (gauge)      		Latest timestamp allocated in memory
      milvus.rootcoord.timestamp_saved
      (gauge)      		Timestamp saved in meta storage
      milvus.runtime_info
      (gauge)      		Runtime information of Milvus
      milvus.search.latency.bucket
      (count)      		Histogram bucket for search latencies (ms)
      milvus.search.latency.count
      (count)      		Count of search events
      milvus.search.latency.sum
      (count)      		Sum of search latencies (ms)
      Shown as millisecond
      milvus.search.topk.bucket
      (count)      		Histogram bucket for search topk
      milvus.search.topk.count
      (count)      		Count aggregation of search topk
      milvus.search.topk.sum
      (count)      		Sum aggregation of search topk
      milvus.storage.kv_size
      (gauge)      		Key-value size stats
      milvus.storage.op.count
      (count)      		Count of persistent data operation
      milvus.storage.request_latency
      (gauge)      		Request latency on the client side

      Events

      The Milvus integration does not include any events.

      Service Checks

      The Milvus integration does not include any service checks.

      Troubleshooting

      Need help? Contact Datadog support.