Impala

Supported OS Linux Windows Mac OS

Integration version2.2.1

Overview

This check monitors Impala through the Datadog Agent.

Setup

Follow the instructions below to install and configure this check for an Agent running on a host. For containerized environments, see the Autodiscovery Integration Templates for guidance on applying these instructions.

Installation

The Impala check is included in the Datadog Agent package. No additional installation is needed on your server.

Configuration

  1. Edit the impala.d/conf.yaml file, in the conf.d/ folder at the root of your Agent’s configuration directory to start collecting your Impala performance data. See the sample impala.d/conf.yaml for all available configuration options.

Here is an example monitoring a daemon:

init_config:
   
instances:
  ## @param service_type - string - required
  ## The Impala service you want to monitor. Possible values are `daemon`, `statestore`, and `catalog`.
  #
- service_type: daemon

  ## @param openmetrics_endpoint - string - required
  ## The URL exposing metrics in the OpenMetrics format.
  ##
  ## The default port for the services are:
  ## - Daemon: 25000
  ## - Statestore: 25010
  ## - Catalog: 25020
  #
  openmetrics_endpoint: http://%%host%%:25000/metrics_prometheus

You can also monitor several services at the same time with the same agent:

init_config:
   
instances:

- service_type: daemon
  service: daemon-1
  openmetrics_endpoint: http://<DAEMON-1-IP>:25000/metrics_prometheus
- service_type: daemon
  service: daemon-2
  openmetrics_endpoint: http://<DAEMON-2-IP>:25000/metrics_prometheus
- service_type: statestore
  openmetrics_endpoint: http://<STATESTORE-IP>:25010/metrics_prometheus
- service_type: catalog
  openmetrics_endpoint: http://<CATALOG-IP>:25020/metrics_prometheus
  1. Restart the Agent.

Validation

Run the Agent’s status subcommand and look for impala under the Checks section.

Data Collected

Metrics

impala.catalog.events_processor.avg_events_fetch_duration
(gauge)
[Catalog] Average time taken to fetch a batch of metastore events.
Shown as second
impala.catalog.events_processor.avg_events_process_duration
(gauge)
[Catalog] Average time taken to process a batch of events received from metastore.
Shown as second
impala.catalog.events_processor.events_received.count
(count)
[Catalog] Total number of metastore events received.
impala.catalog.events_processor.events_received_15min_rate
(gauge)
[Catalog] Exponentially weighted moving average (EWMA) of number of events received in last 15 min.
impala.catalog.events_processor.events_received_1min_rate
(gauge)
[Catalog] Exponentially weighted moving average (EWMA) of number of events received in last 1 min.
impala.catalog.events_processor.events_received_5min_rate
(gauge)
[Catalog] Exponentially weighted moving average (EWMA) of number of events received in last 5 min.
impala.catalog.events_processor.events_skipped.count
(count)
[Catalog] Total number of metastore events skipped.
impala.catalog.events_processor.last_synced_event_id.count
(count)
[Catalog] Last metastore event id that the catalog server processed and synced to.
impala.catalog.jvm.code_cache.committed_usage
(gauge)
[Catalog] JVM code-cache Committed Usage Bytes.
Shown as byte
impala.catalog.jvm.code_cache.current_usage
(gauge)
[Catalog] JVM code-cache Current Usage Bytes.
Shown as byte
impala.catalog.jvm.code_cache.init_usage
(gauge)
[Catalog] JVM code-cache Init Usage Bytes.
Shown as byte
impala.catalog.jvm.code_cache.max_usage
(gauge)
[Catalog] JVM code-cache Max Usage Bytes.
Shown as byte
impala.catalog.jvm.code_cache.peak_committed_usage
(gauge)
[Catalog] JVM code-cache Peak Committed Usage Bytes.
Shown as byte
impala.catalog.jvm.code_cache.peak_current_usage
(gauge)
[Catalog] JVM code-cache Peak Current Usage Bytes.
Shown as byte
impala.catalog.jvm.code_cache.peak_init_usage
(gauge)
[Catalog] JVM code-cache Peak Init Usage Bytes.
Shown as byte
impala.catalog.jvm.code_cache.peak_max_usage
(gauge)
[Catalog] JVM code-cache Peak Max Usage Bytes.
Shown as byte
impala.catalog.jvm.compressed_class_space.committed_usage
(gauge)
[Catalog] JVM compressed-class-space Committed Usage Bytes.
Shown as byte
impala.catalog.jvm.compressed_class_space.current_usage
(gauge)
[Catalog] JVM compressed-class-space Current Usage Bytes.
Shown as byte
impala.catalog.jvm.compressed_class_space.init_usage
(gauge)
[Catalog] JVM compressed-class-space Init Usage Bytes.
Shown as byte
impala.catalog.jvm.compressed_class_space.max_usage
(gauge)
[Catalog] JVM compressed-class-space Max Usage Bytes.
Shown as byte
impala.catalog.jvm.compressed_class_space.peak_committed_usage
(gauge)
[Catalog] JVM compressed-class-space Peak Committed Usage Bytes.
Shown as byte
impala.catalog.jvm.compressed_class_space.peak_current_usage
(gauge)
[Catalog] JVM compressed-class-space Peak Current Usage Bytes.
Shown as byte
impala.catalog.jvm.compressed_class_space.peak_init_usage
(gauge)
[Catalog] JVM compressed-class-space Peak Init Usage Bytes.
Shown as byte
impala.catalog.jvm.compressed_class_space.peak_max_usage
(gauge)
[Catalog] JVM compressed-class-space Peak Max Usage Bytes.
Shown as byte
impala.catalog.jvm.gc.count
(count)
[Catalog] JVM Garbage Collection Count.
impala.catalog.jvm.gc.num_info_threshold_exceeded.count
(count)
[Catalog] JVM Pause Detection Info Threshold Exceeded.
impala.catalog.jvm.gc.num_warn_threshold_exceeded.count
(count)
[Catalog] JVM Pause Detection Warning Threshold Exceeded.
impala.catalog.jvm.gc.time_millis.count
(count)
[Catalog] JVM Garbage Collection Time.
Shown as millisecond
impala.catalog.jvm.gc.total_extra_sleep_time_millis.count
(count)
[Catalog] JVM Pause Detection Extra Sleep Time.
Shown as millisecond
impala.catalog.jvm.heap.committed_usage
(gauge)
[Catalog] JVM heap Committed Usage Bytes.
Shown as byte
impala.catalog.jvm.heap.current_usage
(gauge)
[Catalog] JVM heap Current Usage Bytes.
Shown as byte
impala.catalog.jvm.heap.init_usage
(gauge)
[Catalog] JVM heap Init Usage Bytes.
Shown as byte
impala.catalog.jvm.heap.max_usage
(gauge)
[Catalog] JVM heap Max Usage Bytes.
Shown as byte
impala.catalog.jvm.heap.peak_committed_usage
(gauge)
[Catalog] JVM heap Peak Committed Usage Bytes.
Shown as byte
impala.catalog.jvm.heap.peak_current_usage
(gauge)
[Catalog] JVM heap Peak Current Usage Bytes.
Shown as byte
impala.catalog.jvm.heap.peak_init_usage
(gauge)
[Catalog] JVM heap Peak Init Usage Bytes.
Shown as byte
impala.catalog.jvm.heap.peak_max_usage
(gauge)
[Catalog] JVM heap Peak Max Usage Bytes.
Shown as byte
impala.catalog.jvm.metaspace.committed_usage
(gauge)
[Catalog] JVM metaspace Committed Usage Bytes.
Shown as byte
impala.catalog.jvm.metaspace.current_usage
(gauge)
[Catalog] JVM metaspace Current Usage Bytes.
Shown as byte
impala.catalog.jvm.metaspace.init_usage
(gauge)
[Catalog] JVM metaspace Init Usage Bytes.
Shown as byte
impala.catalog.jvm.metaspace.max_usage
(gauge)
[Catalog] JVM metaspace Max Usage Bytes.
Shown as byte
impala.catalog.jvm.metaspace.peak_committed_usage
(gauge)
[Catalog] JVM metaspace Peak Committed Usage Bytes.
Shown as byte
impala.catalog.jvm.metaspace.peak_current_usage
(gauge)
[Catalog] JVM metaspace Peak Current Usage Bytes.
Shown as byte
impala.catalog.jvm.metaspace.peak_init_usage
(gauge)
[Catalog] JVM metaspace Peak Init Usage Bytes.
Shown as byte
impala.catalog.jvm.metaspace.peak_max_usage
(gauge)
[Catalog] JVM metaspace Peak Max Usage Bytes.
Shown as byte
impala.catalog.jvm.non_heap.committed_usage
(gauge)
[Catalog] JVM non-heap Committed Usage Bytes.
Shown as byte
impala.catalog.jvm.non_heap.current_usage
(gauge)
[Catalog] JVM non-heap Current Usage Bytes.
Shown as byte
impala.catalog.jvm.non_heap.init_usage
(gauge)
[Catalog] JVM non-heap Init Usage Bytes.
Shown as byte
impala.catalog.jvm.non_heap.max_usage
(gauge)
[Catalog] JVM non-heap Max Usage Bytes.
Shown as byte
impala.catalog.jvm.non_heap.peak_committed_usage
(gauge)
[Catalog] JVM non-heap Peak Committed Usage Bytes.
Shown as byte
impala.catalog.jvm.non_heap.peak_current_usage
(gauge)
[Catalog] JVM non-heap Peak Current Usage Bytes.
Shown as byte
impala.catalog.jvm.non_heap.peak_init_usage
(gauge)
[Catalog] JVM non-heap Peak Init Usage Bytes.
Shown as byte
impala.catalog.jvm.non_heap.peak_max_usage
(gauge)
[Catalog] JVM non-heap Peak Max Usage Bytes.
Shown as byte
impala.catalog.jvm.ps_eden_space.committed_usage
(gauge)
[Catalog] JVM ps-eden-space Committed Usage Bytes.
Shown as byte
impala.catalog.jvm.ps_eden_space.current_usage
(gauge)
[Catalog] JVM ps-eden-space Current Usage Bytes.
Shown as byte
impala.catalog.jvm.ps_eden_space.init_usage
(gauge)
[Catalog] JVM ps-eden-space Init Usage Bytes.
Shown as byte
impala.catalog.jvm.ps_eden_space.max_usage
(gauge)
[Catalog] JVM ps-eden-space Max Usage Bytes.
Shown as byte
impala.catalog.jvm.ps_eden_space.peak_committed_usage
(gauge)
[Catalog] JVM ps-eden-space Peak Committed Usage Bytes.
Shown as byte
impala.catalog.jvm.ps_eden_space.peak_current_usage
(gauge)
[Catalog] JVM ps-eden-space Peak Current Usage Bytes.
Shown as byte
impala.catalog.jvm.ps_eden_space.peak_init_usage
(gauge)
[Catalog] JVM ps-eden-space Peak Init Usage Bytes.
Shown as byte
impala.catalog.jvm.ps_eden_space.peak_max_usage
(gauge)
[Catalog] JVM ps-eden-space Peak Max Usage Bytes.
Shown as byte
impala.catalog.jvm.ps_old_gen.committed_usage
(gauge)
[Catalog] JVM ps-old-gen Committed Usage Bytes.
Shown as byte
impala.catalog.jvm.ps_old_gen.current_usage
(gauge)
[Catalog] JVM ps-old-gen Current Usage Bytes.
Shown as byte
impala.catalog.jvm.ps_old_gen.init_usage
(gauge)
[Catalog] JVM ps-old-gen Init Usage Bytes.
Shown as byte
impala.catalog.jvm.ps_old_gen.max_usage
(gauge)
[Catalog] JVM ps-old-gen Max Usage Bytes.
Shown as byte
impala.catalog.jvm.ps_old_gen.peak_committed_usage
(gauge)
[Catalog] JVM ps-old-gen Peak Committed Usage Bytes.
Shown as byte
impala.catalog.jvm.ps_old_gen.peak_current_usage
(gauge)
[Catalog] JVM ps-old-gen Peak Current Usage Bytes.
Shown as byte
impala.catalog.jvm.ps_old_gen.peak_init_usage
(gauge)
[Catalog] JVM ps-old-gen Peak Init Usage Bytes.
Shown as byte
impala.catalog.jvm.ps_old_gen.peak_max_usage
(gauge)
[Catalog] JVM ps-old-gen Peak Max Usage Bytes.
Shown as byte
impala.catalog.jvm.ps_survivor_space.committed_usage
(gauge)
[Catalog] JVM ps-survivor-space Committed Usage Bytes.
Shown as byte
impala.catalog.jvm.ps_survivor_space.current_usage
(gauge)
[Catalog] JVM ps-survivor-space Current Usage Bytes.
Shown as byte
impala.catalog.jvm.ps_survivor_space.init_usage
(gauge)
[Catalog] JVM ps-survivor-space Init Usage Bytes.
Shown as byte
impala.catalog.jvm.ps_survivor_space.max_usage
(gauge)
[Catalog] JVM ps-survivor-space Max Usage Bytes.
Shown as byte
impala.catalog.jvm.ps_survivor_space.peak_committed_usage
(gauge)
[Catalog] JVM ps-survivor-space Peak Committed Usage Bytes.
Shown as byte
impala.catalog.jvm.ps_survivor_space.peak_current_usage
(gauge)
[Catalog] JVM ps-survivor-space Peak Current Usage Bytes.
Shown as byte
impala.catalog.jvm.ps_survivor_space.peak_init_usage
(gauge)
[Catalog] JVM ps-survivor-space Peak Init Usage Bytes.
Shown as byte
impala.catalog.jvm.ps_survivor_space.peak_max_usage
(gauge)
[Catalog] JVM ps-survivor-space Peak Max Usage Bytes.
Shown as byte
impala.catalog.jvm.total_committed_usage
(gauge)
[Catalog] JVM total Committed Usage Bytes.
Shown as byte
impala.catalog.jvm.total_current_usage
(gauge)
[Catalog] JVM total Current Usage Bytes.
Shown as byte
impala.catalog.jvm.total_init_usage
(gauge)
[Catalog] JVM total Init Usage Bytes.
Shown as byte
impala.catalog.jvm.total_max_usage
(gauge)
[Catalog] JVM total Max Usage Bytes.
Shown as byte
impala.catalog.jvm.total_peak_committed_usage
(gauge)
[Catalog] JVM total Peak Committed Usage Bytes.
Shown as byte
impala.catalog.jvm.total_peak_current_usage
(gauge)
[Catalog] JVM total Peak Current Usage Bytes.
Shown as byte
impala.catalog.jvm.total_peak_init_usage
(gauge)
[Catalog] JVM total Peak Init Usage Bytes.
Shown as byte
impala.catalog.jvm.total_peak_max_usage
(gauge)
[Catalog] JVM total Peak Max Usage Bytes.
Shown as byte
impala.catalog.memory.mapped
(gauge)
[Catalog] Total bytes of memory mappings in this process (the virtual memory size)..
Shown as byte
impala.catalog.memory.rss
(gauge)
[Catalog] Resident set size (RSS) of this process, including TCMalloc, buffer pool and JVM.
Shown as byte
impala.catalog.memory.total_used
(gauge)
[Catalog] Total memory currently used by TCMalloc and buffer pool.
Shown as byte
impala.catalog.partial_fetch_rpc_queue_len
(gauge)
[Catalog] RPC queue length for partial object fetches.
impala.catalog.rpc_method_catalog_server.get_partial_catalog_object_call_duration.count
(count)
[Catalog] Count of duration the rpc method spent to get the catalog.
impala.catalog.rpc_method_catalog_server.get_partial_catalog_object_call_duration.quantile
(gauge)
[Catalog] Quantile of duration the rpc method spent to get the catalog.
impala.catalog.rpc_method_catalog_server.get_partial_catalog_object_call_duration.sum
(count)
[Catalog] Sum of duration the rpc method spent to get the catalog.
impala.catalog.rpc_method_statestore_subscriber.heartbeat_call_duration.count
(count)
[Catalog] Count of duration the heartbeat call lasted.
impala.catalog.rpc_method_statestore_subscriber.heartbeat_call_duration.quantile
(gauge)
[Catalog] Quantile of duration the heartbeat call lasted.
impala.catalog.rpc_method_statestore_subscriber.heartbeat_call_duration.sum
(count)
[Catalog] Sum of duration the heartbeat call lasted.
impala.catalog.rpc_method_statestore_subscriber.update_state_call_duration.count
(count)
[Catalog] Count of duration the rpc method spent to update the state.
impala.catalog.rpc_method_statestore_subscriber.update_state_call_duration.quantile
(gauge)
[Catalog] Quantile of duration the rpc method spent to update the state.
impala.catalog.rpc_method_statestore_subscriber.update_state_call_duration.sum
(count)
[Catalog] Sum of duration the rpc method spent to update the state.
impala.catalog.server_topic_processing_time_s_total.count
(count)
[Catalog] Catalog Server Topic Processing Time
Shown as second
impala.catalog.statestore_subscriber.heartbeat_interval_time_total.count
(count)
[Catalog] The time (in seconds) between Statestore heartbeats.
Shown as second
impala.catalog.statestore_subscriber.last_recovery_duration
(gauge)
[Catalog] The amount of time the StateStore subscriber took to recover the connection the last time it was lost.
Shown as second
impala.catalog.statestore_subscriber.num_connection_failures.count
(count)
[Catalog] Number of times that the Daemon detected a loss of connectivity to the StateStore.
impala.catalog.statestore_subscriber.processing_time.count
(count)
[Catalog] Statestore subscriber processing time for a given topic.
impala.catalog.statestore_subscriber.statestore_client_cache.clients_in_use
(gauge)
[Catalog] The number of active StateStore subscriber clients in this Impala Daemon's client cache. These clients are for communication from this role to the StateStore.
impala.catalog.statestore_subscriber.statestore_client_cache.total_clients
(gauge)
[Catalog] The total number of StateStore subscriber clients in this Impala Daemon's client cache. These clients are for communication from this role to the StateStore.
impala.catalog.statestore_subscriber.topic_update_duration_total.count
(count)
[Catalog] The time (in seconds) taken to process Statestore subscriber topic updates.
Shown as second
impala.catalog.statestore_subscriber.topic_update_interval_time_total.count
(count)
[Catalog] The time (in seconds) between Statestore subscriber topic updates.
Shown as second
impala.catalog.statestore_subscriber.update_interval.count
(count)
[Catalog] Interval between topic updates for a given topic.
impala.catalog.tcmalloc.in_use
(gauge)
[Catalog] Number of bytes used by the application. This will not typically match the memory use reported by the OS, because it does not include TCMalloc overhead or memory fragmentation.
Shown as byte
impala.catalog.tcmalloc.pageheap.free
(gauge)
[Catalog] Number of bytes in free, mapped pages in page heap. These bytes can be used to fulfill allocation requests. They always count towards virtual memory usage, and unless the underlying memory is swapped out by the OS, they also count towards physical memory usage.
Shown as byte
impala.catalog.tcmalloc.pageheap.unmapped
(gauge)
[Catalog] Number of bytes in free, mapped pages in page heap. These bytes can be used to fulfill allocation requests. They always count towards virtual memory usage, and unless the underlying memory is swapped out by the OS, they also count towards physical memory usage.
Shown as byte
impala.catalog.tcmalloc.physical_reserved
(gauge)
[Catalog] Derived metric computing the amount of physical memory (in bytes) used by the process, including that actually in use and free bytes reserved by tcmalloc. Does not include the tcmalloc metadata.
Shown as byte
impala.catalog.tcmalloc.total_reserved
(gauge)
[Catalog] Bytes of system memory reserved by TCMalloc.
Shown as byte
impala.catalog.thread_manager.running_threads
(gauge)
[Catalog] The number of running threads in this process.
impala.catalog.thread_manager.total_threads_created
(gauge)
[Catalog] Threads created over the lifetime of the process.
impala.catalog.thrift_server.connection.setup_queue_size
(gauge)
[Catalog] The number of connections to the Catalog Service that have been accepted and are waiting to be setup.
impala.catalog.thrift_server.connection.setup_time.count
(count)
[Catalog] Count of duration the clients of Catalog Service spent waiting for connection to be set up.
impala.catalog.thrift_server.connection.setup_time.quantile
(gauge)
[Catalog] Quantile of duration the clients of Catalog Service spent waiting for connection to be set up.
Shown as second
impala.catalog.thrift_server.connection.setup_time.sum
(count)
[Catalog] Sum of duration the clients of Catalog Service spent waiting for connection to be set up.
Shown as second
impala.catalog.thrift_server.connections.in_use
(gauge)
[Catalog] The number of active catalog service connections to this Catalog Server.
impala.catalog.thrift_server.svc_thread_wait_time.count
(count)
[Catalog] Count of duration the clients of Catalog Service spent waiting for service threads.
impala.catalog.thrift_server.svc_thread_wait_time.quantile
(gauge)
[Catalog] Quantile of duration the clients of Catalog Service spent waiting for service threads.
Shown as second
impala.catalog.thrift_server.svc_thread_wait_time.sum
(count)
[Catalog] Sum of duration the clients of Catalog Service spent waiting for service threads.
Shown as second
impala.catalog.thrift_server.timedout_cnxn_requests
(gauge)
[Catalog] The number of connection requests to the Catalog Service that have been timed out waiting to be setup.
impala.catalog.thrift_server.total_connections.count
(count)
[Catalog] The total number of connections made to this Catalog Server's catalog service over its lifetime.
impala.daemon.admission_controller.executor_group_num_queries_executing_default
(gauge)
[Daemon] Total number of queries running on executor group: default
impala.daemon.admission_controller.total_dequeue_failed_coordinator_limited.count
(count)
[Daemon] The number of times queries cannot be dequeued because of a resource limit on the coordinator.
impala.daemon.buffer_pool.arena.allocated_buffer_sizes.count
(count)
[Daemon] Count of for buffer sizes allocated from the system. Only a subset of allocations are counted in this metric to reduce overhead.
Shown as byte
impala.daemon.buffer_pool.arena.allocated_buffer_sizes.quantile
(gauge)
[Daemon] Quantile of buffer sizes allocated from the system. Only a subset of allocations are counted in this metric to reduce overhead.
Shown as byte
impala.daemon.buffer_pool.arena.allocated_buffer_sizes.sum
(count)
[Daemon] Sum of buffer sizes allocated from the system. Only a subset of allocations are counted in this metric to reduce overhead.
Shown as byte
impala.daemon.buffer_pool.arena.clean_page_hits_total.count
(count)
[Daemon] Number of times a clean page was evicted to fulfil an allocation.
impala.daemon.buffer_pool.arena.direct_alloc_count_total.count
(count)
[Daemon] Number of times a new buffer was directly allocated from the system allocator to fulfil an allocation.
impala.daemon.buffer_pool.arena.local_arena_free_buffer_hits_total.count
(count)
[Daemon] Number of times a free buffer was recycled from this core's arena to fulfil an allocation.
impala.daemon.buffer_pool.arena.num_final_scavenges_total.count
(count)
[Daemon] Number of times the allocator had to lock all arenas and scavenge to fulfil an allocation.
impala.daemon.buffer_pool.arena.num_scavenges_total.count
(count)
[Daemon] Number of times the allocator had to scavenge to fulfil an allocation.
impala.daemon.buffer_pool.arena.numa_arena_free_buffer_hits_total.count
(count)
[Daemon] Number of times that a recycled buffer within the same NUMA node was used to fulfil an allocation.
impala.daemon.buffer_pool.arena.system_alloc_time_total.count
(count)
[Daemon] Total time the buffer pool spent in the system allocator for this arena.
impala.daemon.buffer_pool.clean_page_bytes
(gauge)
[Daemon] Total bytes of clean page memory cached in the buffer pool.
Shown as byte
impala.daemon.buffer_pool.clean_pages
(gauge)
[Daemon] Total number of clean pages cached in the buffer pool.
impala.daemon.buffer_pool.clean_pages_limit
(gauge)
[Daemon] Limit on number of clean pages cached in the buffer pool.
impala.daemon.buffer_pool.free_buffer
(gauge)
[Daemon] Total bytes of free buffer memory cached in the buffer pool.
Shown as byte
impala.daemon.buffer_pool.free_buffers
(gauge)
[Daemon] Total number of free buffers cached in the buffer pool.
impala.daemon.buffer_pool.limit
(gauge)
[Daemon] Maximum allowed bytes allocated by the buffer pool.
Shown as byte
impala.daemon.buffer_pool.reserved
(gauge)
[Daemon] Total bytes of buffers reserved by Impala subsystems
Shown as byte
impala.daemon.buffer_pool.system_allocated
(gauge)
[Daemon] Total buffer memory currently allocated by the buffer pool.
impala.daemon.buffer_pool.unused_reservation
(gauge)
[Daemon] Total bytes of buffer reservations by Impala subsystems that are currently unused.
Shown as byte
impala.daemon.catalog.catalog_object_version_lower_bound
(gauge)
[Daemon] Lower bound of catalog object version in local catalog cache.
impala.daemon.catalog.curr_topic
(gauge)
[Daemon] Statestore topic update version.
impala.daemon.catalog.curr_version
(gauge)
[Daemon] Catalog topic update version.
impala.daemon.catalog.num_databases
(gauge)
[Daemon] The number of databases in the catalog.
impala.daemon.catalog.num_tables
(gauge)
[Daemon] The number of tables in the catalog.
impala.daemon.catalog.server_client_cache.clients_in_use
(gauge)
[Daemon] The number of clients currently in use by the Catalog Server client cache.
impala.daemon.catalog.server_client_cache.total_clients
(gauge)
[Daemon] The total number of clients in the Catalog Server client cache.
impala.daemon.catalog_cache.average_load_time
(gauge)
[Daemon] Average time spent loading new values into the Impalad Catalog Cache.
impala.daemon.catalog_cache.eviction.count
(count)
[Daemon] Total number of evictions from the Impalad Catalog Cache.
impala.daemon.catalog_cache.hit.count
(count)
[Daemon] Total number of Impalad Catalog cache hits.
impala.daemon.catalog_cache.hit_rate
(gauge)
[Daemon] Ratio of Impalad Catalog cache requests that were hits.
impala.daemon.catalog_cache.load.count
(count)
[Daemon] Total requests to Impalad Catalog cache requests that loaded new values.
impala.daemon.catalog_cache.load_exception.count
(count)
[Daemon] Total requests to Impalad Catalog cache requests that threw exceptions loading new values.
impala.daemon.catalog_cache.load_exception_rate
(gauge)
[Daemon] Ratio of Impalad Catalog cache requests that threw exceptions loading new values.
impala.daemon.catalog_cache.load_success.count
(count)
[Daemon] Number of Impalad Catalog cache requests that successfully loaded new values.
impala.daemon.catalog_cache.miss.count
(count)
[Daemon] Number of Impalad Catalog cache requests that returned uncached values.
impala.daemon.catalog_cache.miss_rate
(gauge)
[Daemon] Ratio of Impalad Catalog cache requests that were misses.
impala.daemon.catalog_cache.request.count
(count)
[Daemon] Total number of Impalad Catalog cache requests.
impala.daemon.catalog_cache.total_load_time.count
(count)
[Daemon] Total time spent in Impalad Catalog cache loading new values.
Shown as second
impala.daemon.cluster_membership.backends.count
(count)
[Daemon] Total number of backends registered with the statestore.
impala.daemon.cluster_membership.executor_groups.count
(count)
[Daemon] Total number of executor groups that have at least one executor.
impala.daemon.cluster_membership.executor_groups_total_healthy.count
(count)
[Daemon] Total number of executor groups that are in a healthy state, that is, have at least the configured minimum number of executors to be considered for admission.
impala.daemon.ddl_durations_ms.count
(count)
[Daemon] Count of DDL operation latencies.
Shown as millisecond
impala.daemon.ddl_durations_ms.quantile
(gauge)
[Daemon] Quantile of DDL operation latencies.
impala.daemon.ddl_durations_ms.sum
(count)
[Daemon] Sum of DDL operation latencies.
Shown as millisecond
impala.daemon.external_data_source_class_cache.hits.count
(count)
[Daemon] Number of cache hits in the External Data Source Class Cache.
impala.daemon.external_data_source_class_cache.misses.count
(count)
[Daemon] Number of cache misses in the External Data Source Class Cache.
impala.daemon.hedged_read_ops.count
(count)
[Daemon] The total number of hedged reads attempted over the life of the process.
impala.daemon.hedged_read_ops.win.count
(count)
[Daemon] The total number of times hedged reads were faster than regular read operations.
impala.daemon.io_mgr.bytes_read.count
(count)
[Daemon] The total number of bytes read by the IO manager.
Shown as byte
impala.daemon.io_mgr.bytes_written.count
(count)
[Daemon] Total number of bytes written to disk by the IO manager.
Shown as byte
impala.daemon.io_mgr.cached_bytes_read.count
(count)
[Daemon] Total number of cached bytes read by the IO manager.
Shown as byte
impala.daemon.io_mgr.cached_file_handles.hit
(gauge)
[Daemon] Number of cache hits for cached HDFS file handles
impala.daemon.io_mgr.cached_file_handles.hit_ratio_total.count
(count)
[Daemon] HDFS file handle cache hit ratio, between 0 and 1, where 1 means all reads were served from cached file handles.
Shown as byte
impala.daemon.io_mgr.cached_file_handles.miss
(gauge)
[Daemon] Number of cache hits for cached HDFS file handles.
Shown as byte
impala.daemon.io_mgr.cached_file_handles.miss.count
(count)
[Daemon] Number of cache misses for cached HDFS file handles
impala.daemon.io_mgr.cached_file_handles.reopened.count
(count)
[Daemon] Number of cached HDFS file handles reopened
impala.daemon.io_mgr.local_bytes_read.count
(count)
[Daemon] Total number of local bytes read by the IO manager.
Shown as byte
impala.daemon.io_mgr.num_cached_file_handles
(gauge)
[Daemon] Number of currently cached HDFS file handles in the IO manager.
impala.daemon.io_mgr.num_file_handles_outstanding
(gauge)
[Daemon] Number of HDFS file handles that are currently in use by readers.
impala.daemon.io_mgr.num_open_files
(gauge)
[Daemon] The current number of files opened by the IO Manager
impala.daemon.io_mgr.remote_data_cache.dropped.count
(count)
[Daemon] Total number of bytes not inserted in remote data cache due to concurrency limit.
Shown as byte
impala.daemon.io_mgr.remote_data_cache.dropped_entries.count
(count)
[Daemon] Total number of entries not inserted in remote data cache due to concurrency limit.
impala.daemon.io_mgr.remote_data_cache.hit.count
(count)
[Daemon] Total number of hits in the remote data cache.
impala.daemon.io_mgr.remote_data_cache.hit_bytes.count
(count)
[Daemon] Total number of bytes of hits in the remote data cache.
Shown as byte
impala.daemon.io_mgr.remote_data_cache.instant_evictions.count
(count)
[Daemon] Total number of instantaneous evictions from the remote data cache. An instantaneous eviction happens when the eviction policy rejects an entry during insert.
impala.daemon.io_mgr.remote_data_cache.miss.count
(count)
[Daemon] Total number of misses in the remote data cache.
impala.daemon.io_mgr.remote_data_cache.miss_bytes.count
(count)
[Daemon] Total number of bytes of misses in the remote data cache.
Shown as byte
impala.daemon.io_mgr.remote_data_cache.num_entries
(gauge)
[Daemon] Current number of entries in the remote data cache.
impala.daemon.io_mgr.remote_data_cache.num_writes.count
(count)
[Daemon] Total number of writes into the remote data cache.
impala.daemon.io_mgr.remote_data_cache.total
(gauge)
[Daemon] Current byte size of the remote data cache.
Shown as byte
impala.daemon.io_mgr.short_circuit.read.count
(count)
[Daemon] Total number of short-circuit bytes read by the IO manager.
Shown as byte
impala.daemon.io_mgr_queue.read_latency.count
(count)
[Daemon] Count of read operation times on disk.
impala.daemon.io_mgr_queue.read_latency.quantile
(gauge)
[Daemon] Quantile of read operation times on disk.
Shown as second
impala.daemon.io_mgr_queue.read_latency.sum
(count)
[Daemon] Sum of read operation times on disk.
Shown as second
impala.daemon.io_mgr_queue.read_size.count
(count)
[Daemon] Count of read operation sizes on disk.
impala.daemon.io_mgr_queue.read_size.quantile
(gauge)
[Daemon] Quantile of read operation sizes on disk.
Shown as byte
impala.daemon.io_mgr_queue.read_size.sum
(count)
[Daemon] Sum of read operation sizes on disk.
Shown as byte
impala.daemon.io_mgr_queue.write_io_error_total.count
(count)
[Daemon] The number of write io errors on disk.
impala.daemon.io_mgr_queue.write_latency.count
(count)
[Daemon] Count of write operation times on disk.
impala.daemon.io_mgr_queue.write_latency.quantile
(gauge)
[Daemon] Quantile of write operation times on disk.
Shown as second
impala.daemon.io_mgr_queue.write_latency.sum
(count)
[Daemon] Sum of write operation times on disk.
Shown as second
impala.daemon.io_mgr_queue.write_size.count
(count)
[Daemon] Count of write operation sizes on disk.
impala.daemon.io_mgr_queue.write_size.quantile
(gauge)
[Daemon] Quantile of write operation sizes on disk.
Shown as byte
impala.daemon.io_mgr_queue.write_size.sum
(count)
[Daemon] Sum of write operation sizes on disk.
Shown as byte
impala.daemon.jvm.code_cache.committed_usage
(gauge)
[Daemon] JVM code-cache Committed Usage Bytes.
Shown as byte
impala.daemon.jvm.code_cache.current_usage
(gauge)
[Daemon] JVM code-cache Current Usage Bytes.
Shown as byte
impala.daemon.jvm.code_cache.init_usage
(gauge)
[Daemon] JVM code-cache Init Usage Bytes.
Shown as byte
impala.daemon.jvm.code_cache.max_usage
(gauge)
[Daemon] JVM code-cache Max Usage Bytes.
Shown as byte
impala.daemon.jvm.code_cache.peak_committed_usage
(gauge)
[Daemon] JVM code-cache Peak Committed Usage Bytes.
Shown as byte
impala.daemon.jvm.code_cache.peak_current_usage
(gauge)
[Daemon] JVM code-cache Peak Current Usage Bytes.
Shown as byte
impala.daemon.jvm.code_cache.peak_init_usage
(gauge)
[Daemon] JVM code-cache Peak Init Usage Bytes.
Shown as byte
impala.daemon.jvm.code_cache.peak_max_usage
(gauge)
[Daemon] JVM code-cache Peak Max Usage Bytes.
Shown as byte
impala.daemon.jvm.compressed_class_space.committed_usage
(gauge)
[Daemon] JVM compressed-class-space Committed Usage Bytes.
Shown as byte
impala.daemon.jvm.compressed_class_space.current_usage
(gauge)
[Daemon] JVM compressed-class-space Current Usage Bytes.
Shown as byte
impala.daemon.jvm.compressed_class_space.init_usage
(gauge)
[Daemon] JVM compressed-class-space Init Usage Bytes.
Shown as byte
impala.daemon.jvm.compressed_class_space.max_usage
(gauge)
[Daemon] JVM compressed-class-space Max Usage Bytes.
Shown as byte
impala.daemon.jvm.compressed_class_space.peak_committed_usage
(gauge)
[Daemon] JVM compressed-class-space Peak Committed Usage Bytes.
Shown as byte
impala.daemon.jvm.compressed_class_space.peak_current_usage
(gauge)
[Daemon] JVM compressed-class-space Peak Current Usage Bytes.
Shown as byte
impala.daemon.jvm.compressed_class_space.peak_init_usage
(gauge)
[Daemon] JVM compressed-class-space Peak Init Usage Bytes.
Shown as byte
impala.daemon.jvm.compressed_class_space.peak_max_usage
(gauge)
[Daemon] JVM compressed-class-space Peak Max Usage Bytes.
Shown as byte
impala.daemon.jvm.gc.count
(count)
[Daemon] JVM Garbage Collection Count.
impala.daemon.jvm.gc.num_info_threshold_exceeded.count
(count)
[Daemon] JVM Pause Detection Info Threshold Exceeded.
impala.daemon.jvm.gc.num_warn_threshold_exceeded.count
(count)
[Daemon] JVM Pause Detection Warning Threshold Exceeded.
impala.daemon.jvm.gc.time_millis.count
(count)
[Daemon] JVM Garbage Collection Time.
Shown as millisecond
impala.daemon.jvm.gc.total_extra_sleep_time_millis.count
(count)
[Daemon] JVM Pause Detection Extra Sleep Time.
Shown as millisecond
impala.daemon.jvm.heap.committed_usage
(gauge)
[Daemon] JVM heap Committed Usage Bytes.
Shown as byte
impala.daemon.jvm.heap.current_usage
(gauge)
[Daemon] JVM heap Current Usage Bytes.
Shown as byte
impala.daemon.jvm.heap.init_usage
(gauge)
[Daemon] JVM heap Init Usage Bytes.
Shown as byte
impala.daemon.jvm.heap.max_usage
(gauge)
[Daemon] JVM heap Max Usage Bytes.
Shown as byte
impala.daemon.jvm.heap.peak_committed_usage
(gauge)
[Daemon] JVM heap Peak Committed Usage Bytes.
Shown as byte
impala.daemon.jvm.heap.peak_current_usage
(gauge)
[Daemon] JVM heap Peak Current Usage Bytes.
Shown as byte
impala.daemon.jvm.heap.peak_init_usage
(gauge)
[Daemon] JVM heap Peak Init Usage Bytes.
Shown as byte
impala.daemon.jvm.heap.peak_max_usage
(gauge)
[Daemon] JVM heap Peak Max Usage Bytes.
Shown as byte
impala.daemon.jvm.metaspace.committed_usage
(gauge)
[Daemon] JVM metaspace Committed Usage Bytes.
Shown as byte
impala.daemon.jvm.metaspace.current_usage
(gauge)
[Daemon] JVM metaspace Current Usage Bytes.
Shown as byte
impala.daemon.jvm.metaspace.init_usage
(gauge)
[Daemon] JVM metaspace Init Usage Bytes.
Shown as byte
impala.daemon.jvm.metaspace.max_usage
(gauge)
[Daemon] JVM metaspace Max Usage Bytes.
Shown as byte
impala.daemon.jvm.metaspace.peak_committed_usage
(gauge)
[Daemon] JVM metaspace Peak Committed Usage Bytes.
Shown as byte
impala.daemon.jvm.metaspace.peak_current_usage
(gauge)
[Daemon] JVM metaspace Peak Current Usage Bytes.
Shown as byte
impala.daemon.jvm.metaspace.peak_init_usage
(gauge)
[Daemon] JVM metaspace Peak Init Usage Bytes.
Shown as byte
impala.daemon.jvm.metaspace.peak_max_usage
(gauge)
[Daemon] JVM metaspace Peak Max Usage Bytes.
Shown as byte
impala.daemon.jvm.non_heap.committed_usage
(gauge)
[Daemon] JVM non-heap Committed Usage Bytes.
Shown as byte
impala.daemon.jvm.non_heap.current_usage
(gauge)
[Daemon] JVM non-heap Current Usage Bytes.
Shown as byte
impala.daemon.jvm.non_heap.init_usage
(gauge)
[Daemon] JVM non-heap Init Usage Bytes.
Shown as byte
impala.daemon.jvm.non_heap.max_usage
(gauge)
[Daemon] JVM non-heap Max Usage Bytes.
Shown as byte
impala.daemon.jvm.non_heap.peak_committed_usage
(gauge)
[Daemon] JVM non-heap Peak Committed Usage Bytes.
Shown as byte
impala.daemon.jvm.non_heap.peak_current_usage
(gauge)
[Daemon] JVM non-heap Peak Current Usage Bytes.
Shown as byte
impala.daemon.jvm.non_heap.peak_init_usage
(gauge)
[Daemon] JVM non-heap Peak Init Usage Bytes.
Shown as byte
impala.daemon.jvm.non_heap.peak_max_usage
(gauge)
[Daemon] JVM non-heap Peak Max Usage Bytes.
Shown as byte
impala.daemon.jvm.ps_eden_space.committed_usage
(gauge)
[Daemon] JVM ps-eden-space Committed Usage Bytes.
Shown as byte
impala.daemon.jvm.ps_eden_space.current_usage
(gauge)
[Daemon] JVM ps-eden-space Current Usage Bytes.
Shown as byte
impala.daemon.jvm.ps_eden_space.init_usage
(gauge)
[Daemon] JVM ps-eden-space Init Usage Bytes.
Shown as byte
impala.daemon.jvm.ps_eden_space.max_usage
(gauge)
[Daemon] JVM ps-eden-space Max Usage Bytes.
Shown as byte
impala.daemon.jvm.ps_eden_space.peak_committed_usage
(gauge)
[Daemon] JVM ps-eden-space Peak Committed Usage Bytes.
Shown as byte
impala.daemon.jvm.ps_eden_space.peak_current_usage
(gauge)
[Daemon] JVM ps-eden-space Peak Current Usage Bytes.
Shown as byte
impala.daemon.jvm.ps_eden_space.peak_init_usage
(gauge)
[Daemon] JVM ps-eden-space Peak Init Usage Bytes.
Shown as byte
impala.daemon.jvm.ps_eden_space.peak_max_usage
(gauge)
[Daemon] JVM ps-eden-space Peak Max Usage Bytes.
Shown as byte
impala.daemon.jvm.ps_old_gen.committed_usage
(gauge)
[Daemon] JVM ps-old-gen Committed Usage Bytes.
Shown as byte
impala.daemon.jvm.ps_old_gen.current_usage
(gauge)
[Daemon] JVM ps-old-gen Current Usage Bytes.
Shown as byte
impala.daemon.jvm.ps_old_gen.init_usage
(gauge)
[Daemon] JVM ps-old-gen Init Usage Bytes.
Shown as byte
impala.daemon.jvm.ps_old_gen.max_usage
(gauge)
[Daemon] JVM ps-old-gen Max Usage Bytes.
Shown as byte
impala.daemon.jvm.ps_old_gen.peak_committed_usage
(gauge)
[Daemon] JVM ps-old-gen Peak Committed Usage Bytes.
Shown as byte
impala.daemon.jvm.ps_old_gen.peak_current_usage
(gauge)
[Daemon] JVM ps-old-gen Peak Current Usage Bytes.
Shown as byte
impala.daemon.jvm.ps_old_gen.peak_init_usage
(gauge)
[Daemon] JVM ps-old-gen Peak Init Usage Bytes.
Shown as byte
impala.daemon.jvm.ps_old_gen.peak_max_usage
(gauge)
[Daemon] JVM ps-old-gen Peak Max Usage Bytes.
Shown as byte
impala.daemon.jvm.ps_survivor_space.committed_usage
(gauge)
[Daemon] JVM ps-survivor-space Committed Usage Bytes.
Shown as byte
impala.daemon.jvm.ps_survivor_space.current_usage
(gauge)
[Daemon] JVM ps-survivor-space Current Usage Bytes.
Shown as byte
impala.daemon.jvm.ps_survivor_space.init_usage
(gauge)
[Daemon] JVM ps-survivor-space Init Usage Bytes.
Shown as byte
impala.daemon.jvm.ps_survivor_space.max_usage
(gauge)
[Daemon] JVM ps-survivor-space Max Usage Bytes.
Shown as byte
impala.daemon.jvm.ps_survivor_space.peak_committed_usage
(gauge)
[Daemon] JVM ps-survivor-space Peak Committed Usage Bytes.
Shown as byte
impala.daemon.jvm.ps_survivor_space.peak_current_usage
(gauge)
[Daemon] JVM ps-survivor-space Peak Current Usage Bytes.
Shown as byte
impala.daemon.jvm.ps_survivor_space.peak_init_usage
(gauge)
[Daemon] JVM ps-survivor-space Peak Init Usage Bytes.
Shown as byte
impala.daemon.jvm.ps_survivor_space.peak_max_usage
(gauge)
[Daemon] JVM ps-survivor-space Peak Max Usage Bytes.
Shown as byte
impala.daemon.jvm.total_committed_usage
(gauge)
[Daemon] JVM total Committed Usage Bytes.
Shown as byte
impala.daemon.jvm.total_current_usage
(gauge)
[Daemon] JVM total Current Usage Bytes.
Shown as byte
impala.daemon.jvm.total_init_usage
(gauge)
[Daemon] JVM total Init Usage Bytes.
Shown as byte
impala.daemon.jvm.total_max_usage
(gauge)
[Daemon] JVM total Max Usage Bytes.
Shown as byte
impala.daemon.jvm.total_peak_committed_usage
(gauge)
[Daemon] JVM total Peak Committed Usage Bytes.
Shown as byte
impala.daemon.jvm.total_peak_current_usage
(gauge)
[Daemon] JVM total Peak Current Usage Bytes.
Shown as byte
impala.daemon.jvm.total_peak_init_usage
(gauge)
[Daemon] JVM total Peak Init Usage Bytes.
Shown as byte
impala.daemon.jvm.total_peak_max_usage
(gauge)
[Daemon] JVM total Peak Max Usage Bytes.
Shown as byte
impala.daemon.mem_tracker.process.bytes_freed_by_last_gc
(gauge)
[Daemon] The amount of memory freed by the last memory tracker garbage collection.
Shown as byte
impala.daemon.mem_tracker.process.bytes_over_limit
(gauge)
[Daemon] The amount of memory by which the process was over its memory limit the last time the memory limit was encountered.
Shown as byte
impala.daemon.mem_tracker.process.current_usage
(gauge)
[Daemon] Memtracker current usage for this service.
Shown as byte
impala.daemon.mem_tracker.process.limit
(gauge)
[Daemon] The process memory tracker limit.
impala.daemon.mem_tracker.process.num_gcs.count
(count)
[Daemon] The total number of garbage collections performed by the memory tracker over the life of the process.
impala.daemon.mem_tracker.process.peak_usage
(gauge)
[Daemon] Memtracker peak usage for this service.
Shown as byte
impala.daemon.memory.mapped
(gauge)
[Daemon] Total bytes of memory mappings in this process (the virtual memory size).
Shown as byte
impala.daemon.memory.rss
(gauge)
[Daemon] Resident set size (RSS) of this process, including TCMalloc, buffer pool and Jvm.
impala.daemon.memory.total_used
(gauge)
[Daemon] Total memory currently used by TCMalloc and buffer pool.
Shown as byte
impala.daemon.num_files_open_for_insert
(gauge)
[Daemon] The number of HDFS files currently open for writing.
impala.daemon.num_fragments.count
(count)
[Daemon] The total number of query fragments processed over the life of the process.
impala.daemon.num_fragments_in_flight
(gauge)
[Daemon] The number of query fragment instances currently executing.
impala.daemon.num_open_beeswax_sessions
(gauge)
[Daemon] The number of open Beeswax sessions.
impala.daemon.num_open_hiveserver2_sessions
(gauge)
[Daemon] The number of open HiveServer2 sessions.
impala.daemon.num_queries.count
(count)
[Daemon] The total number of queries processed over the life of the process.
impala.daemon.num_queries_executed.count
(count)
[Daemon] The total number of queries that executed on this backend over the life of the process.
impala.daemon.num_queries_executing
(gauge)
[Daemon] The number of queries currently executing on this backend.
impala.daemon.num_queries_expired.count
(count)
[Daemon] Number of queries expired due to inactivity.
impala.daemon.num_queries_registered
(gauge)
[Daemon] The total number of queries registered on this Impala server instance. Includes queries that are in flight and waiting to be closed
impala.daemon.num_queries_spilled.count
(count)
[Daemon] Number of queries for which any operator spilled.
impala.daemon.num_sessions_expired.count
(count)
[Daemon] Number of sessions expired due to inactivity.
impala.daemon.query_durations_ms.count
(count)
[Daemon] Count of query latencies.
impala.daemon.query_durations_ms.quantile
(gauge)
[Daemon] Quantile of query latencies.
Shown as millisecond
impala.daemon.query_durations_ms.sum
(count)
[Daemon] Sum of query latencies.
Shown as millisecond
impala.daemon.request_pool_service_resolve_pool_duration.count
(count)
[Daemon] Time (ms) spent resolving request request pools.
Shown as millisecond
impala.daemon.resultset_cache.total_bytes
(gauge)
[Daemon] Total number of bytes consumed for rows cached to support HS2 FETCH_FIRST.
Shown as byte
impala.daemon.resultset_cache.total_num_rows
(gauge)
[Daemon] Total number of rows cached to support HS2 FETCH_FIRST.
impala.daemon.rpcs_queue_overflow.count
(count)
[Daemon] Total number of incoming RPCs that were rejected due to overflow of the service queue for this service.
impala.daemon.scan_ranges.count
(count)
[Daemon] The total number of scan ranges read over the life of the process.
impala.daemon.scan_ranges_num_missing_volume_id.count
(count)
[Daemon] The total number of scan ranges read over the life of the process that did not have volume metadata.
impala.daemon.senders_blocked_on_recvr_creation
(gauge)
[Daemon] Number of senders waiting for receiving fragment to initialize.
impala.daemon.simple_scheduler.assignments.count
(count)
[Daemon] The number of assignments.
impala.daemon.simple_scheduler.local_assignments.count
(count)
[Daemon] Number of assignments operating on local data.
impala.daemon.statestore_subscriber.heartbeat_interval_time.count
(count)
[Daemon] The time (in seconds) between Statestore heartbeats.
impala.daemon.statestore_subscriber.last_recovery_duration
(gauge)
[Daemon] The amount of time the StateStore subscriber took to recover the connection the last time it was lost.
impala.daemon.statestore_subscriber.num_connection_failures.count
(count)
[Daemon] Number of times that the Daemon detected a loss of connectivity to the StateStore.
impala.daemon.statestore_subscriber.processing_time.count
(count)
[Daemon] Statestore subscriber processing time for a given topic.
impala.daemon.statestore_subscriber.statestore_client_cache.clients_in_use
(gauge)
[Daemon] The number of active StateStore subscriber clients in this Impala Daemon's client cache. These clients are for communication from this role to the StateStore.
impala.daemon.statestore_subscriber.statestore_client_cache.total_clients
(gauge)
[Daemon] The total number of StateStore subscriber clients in this Impala Daemon's client cache. These clients are for communication from this role to the StateStore.
impala.daemon.statestore_subscriber.topic.update_duration.count
(count)
[Daemon] The time (in seconds) taken to process Statestore subscriber topic updates.
Shown as second
impala.daemon.statestore_subscriber.topic.update_interval_time.count
(count)
[Daemon] The time (in seconds) between Statestore subscriber topic updates.
Shown as second
impala.daemon.statestore_subscriber.update_interval.count
(count)
[Daemon] Interval between topic updates for a given topic.
impala.daemon.tcmalloc.in_use
(gauge)
[Daemon] Number of bytes used by the application. This will not typically match the memory use reported by the OS, because it does not include TCMalloc overhead or memory fragmentation.
Shown as byte
impala.daemon.tcmalloc.pageheap.free
(gauge)
[Daemon] Number of bytes in free, mapped pages in page heap. These bytes can be used to fulfill allocation requests. They always count towards virtual memory usage, and unless the underlying memory is swapped out by the OS, they also count towards physical memory usage.
Shown as byte
impala.daemon.tcmalloc.pageheap.unmapped
(gauge)
[Daemon] Number of bytes in free, mapped pages in page heap. These bytes can be used to fulfill allocation requests. They always count towards virtual memory usage, and unless the underlying memory is swapped out by the OS, they also count towards physical memory usage.
Shown as byte
impala.daemon.tcmalloc.physical_reserved
(gauge)
[Daemon] Derived metric computing the amount of physical memory (in bytes) used by the process, including that actually in use and free bytes reserved by TCMalloc. Does not include the TCMalloc metadata.
Shown as byte
impala.daemon.tcmalloc.total_reserved
(gauge)
[Daemon] Bytes of system memory reserved by TCMalloc.
Shown as byte
impala.daemon.thread_manager.running_threads
(gauge)
[Daemon] The number of running threads in this process.
impala.daemon.thread_manager.total_threads_created
(gauge)
[Daemon] Threads created over the lifetime of the process.
impala.daemon.thrift_server.beeswax.frontend.connection_setup_queue_size
(gauge)
[Daemon] The number of Beeswax API connections to this Impala Daemon that have been accepted and are waiting to be setup.
impala.daemon.thrift_server.beeswax.frontend.connection_setup_time.count
(count)
[Daemon] Count of duration the time clients of Beeswax API spent waiting for connection to be set up.
impala.daemon.thrift_server.beeswax.frontend.connection_setup_time.quantile
(gauge)
[Daemon] Quantile of duration the time clients of Beeswax API spent waiting for connection to be set up.
Shown as second
impala.daemon.thrift_server.beeswax.frontend.connection_setup_time.sum
(count)
[Daemon] Sum of duration the time clients of Beeswax API spent waiting for connection to be set up.
Shown as second
impala.daemon.thrift_server.beeswax.frontend.connections_in_use
(gauge)
[Daemon] The number of active Beeswax API connections to this Impala Daemon.
impala.daemon.thrift_server.beeswax.frontend.svc_thread_wait_time.count
(count)
[Daemon] Count of duration the clients of Beeswax API spent waiting for service threads.
impala.daemon.thrift_server.beeswax.frontend.svc_thread_wait_time.quantile
(gauge)
[Daemon] Count of duration the clients of Beeswax API spent waiting for service threads.
Shown as second
impala.daemon.thrift_server.beeswax.frontend.svc_thread_wait_time.sum
(count)
[Daemon] Sum of duration the clients of Beeswax API spent waiting for service threads.
Shown as second
impala.daemon.thrift_server.beeswax.frontend.timedout_cnxn_requests
(gauge)
[Daemon] The number of Beeswax API connection requests to this Impala Daemon that have been timed out waiting to be setup.
impala.daemon.thrift_server.beeswax.frontend.total_connections.count
(count)
[Daemon] The total number of Beeswax API connections made to this Impala Daemon over its lifetime.
impala.daemon.thrift_server.hiveserver2.frontend.connection_setup_queue_size
(gauge)
[Daemon] The number of HiveServer2 API connections to this Impala Daemon that have been accepted and are waiting to be setup.
impala.daemon.thrift_server.hiveserver2.frontend.connection_setup_time.count
(count)
[Daemon] Count of duration the time clients of HiveServer2 API spent waiting for connection to be set up.
impala.daemon.thrift_server.hiveserver2.frontend.connection_setup_time.quantile
(gauge)
[Daemon] Quantile of duration the time clients of HiveServer2 API spent waiting for connection to be set up.
Shown as second
impala.daemon.thrift_server.hiveserver2.frontend.connection_setup_time.sum
(count)
[Daemon] Sum of duration the time clients of HiveServer2 API spent waiting for connection to be set up.
Shown as second
impala.daemon.thrift_server.hiveserver2.frontend.connections_in_use
(gauge)
[Daemon] The number of active Beeswax API HiveServer2 to this Impala Daemon.
impala.daemon.thrift_server.hiveserver2.frontend.svc_thread_wait_time.count
(count)
[Daemon] Count of duration the clients of HiveServer2 API spent waiting for service threads.
impala.daemon.thrift_server.hiveserver2.frontend.svc_thread_wait_time.quantile
(gauge)
[Daemon] Quantile of duration the clients of HiveServer2 API spent waiting for service threads.
Shown as second
impala.daemon.thrift_server.hiveserver2.frontend.svc_thread_wait_time.sum
(count)
[Daemon] Sum of duration the clients of HiveServer2 API spent waiting for service threads.
Shown as second
impala.daemon.thrift_server.hiveserver2.frontend.timedout_cnxn_requests
(gauge)
[Daemon] The number of Beeswax API connection HiveServer2 to this Impala Daemon that have been timed out waiting to be setup.
impala.daemon.thrift_server.hiveserver2.frontend.total_connections.count
(count)
[Daemon] The total number of Beeswax API HiveServer2 made to this Impala Daemon over its lifetime.
impala.daemon.thrift_server.hiveserver2.http_frontend.connection_setup_queue_size
(gauge)
[Daemon] The number of HiveServer2 HTTP API connections to this Impala Daemon that have been accepted and are waiting to be setup.
impala.daemon.thrift_server.hiveserver2.http_frontend.connection_setup_time.count
(count)
[Daemon] Count of duration the time clients of HiveServer2 HTTP API spent waiting for connection to be set up.
impala.daemon.thrift_server.hiveserver2.http_frontend.connection_setup_time.quantile
(gauge)
[Daemon] Quantile of duration the time clients of HiveServer2 HTTP API spent waiting for connection to be set up.
Shown as second
impala.daemon.thrift_server.hiveserver2.http_frontend.connection_setup_time.sum
(count)
[Daemon] Sum of duration the time clients of HiveServer2 HTTP API spent waiting for connection to be set up.
Shown as second
impala.daemon.thrift_server.hiveserver2.http_frontend.connections_in_use
(gauge)
[Daemon] The number of active Beeswax HTTP API HiveServer2 to this Impala Daemon.
impala.daemon.thrift_server.hiveserver2.http_frontend.svc_thread_wait_time.count
(count)
[Daemon] Count of duration the clients of HiveServer2 HTTP API spent waiting for service threads.
impala.daemon.thrift_server.hiveserver2.http_frontend.svc_thread_wait_time.quantile
(gauge)
[Daemon] Quantile of duration the clients of HiveServer2 HTTP API spent waiting for service threads.
Shown as second
impala.daemon.thrift_server.hiveserver2.http_frontend.svc_thread_wait_time.sum
(count)
[Daemon] Sum of duration the clients of HiveServer2 HTTP API spent waiting for service threads.
Shown as second
impala.daemon.thrift_server.hiveserver2.http_frontend.timedout_cnxn_requests
(gauge)
[Daemon] The number of Beeswax HTTP API connection HiveServer2 to this Impala Daemon that have been timed out waiting to be setup.
impala.daemon.thrift_server.hiveserver2.http_frontend.total_connections.count
(count)
[Daemon] The total number of Beeswax HTTP API HiveServer2 made to this Impala Daemon over its lifetime.
impala.daemon.tmp_file_mgr.active_scratch_dirs
(gauge)
[Daemon] The number of active scratch directories for spilling to disk.
impala.daemon.tmp_file_mgr.scratch_space_bytes_used
(gauge)
[Daemon] The current total spilled bytes across all scratch directories.
Shown as byte
impala.daemon.tmp_file_mgr.scratch_space_bytes_used_dir_0
(gauge)
[Daemon] The current total spilled bytes for a single scratch directory.
Shown as byte
impala.daemon.tmp_file_mgr.scratch_space_bytes_used_high_water_mark
(gauge)
[Daemon] The high water mark for spilled bytes across all scratch directories.
impala.daemon.total_senders_blocked_on_recvr_creation.count
(count)
[Daemon] Total number of senders that have been blocked waiting for receiving fragment to initialize.
impala.daemon.total_senders_timedout_waiting_for_recvr_creation.count
(count)
[Daemon] Total number of senders that timed-out waiting for receiving fragment to initialize.
impala.statestore.heartbeat_durations.count
(count)
[Statestore] The time (in seconds) spent sending heartbeat RPCs. Includes subscriber-side processing time and network transmission time.
Shown as second
impala.statestore.last_priority_topic_update_durations
(gauge)
[Statestore] The last duration (in seconds) spent sending priority topic update RPCs. Includes subscriber-side processing time and network transmission time.
Shown as second
impala.statestore.last_topic_update_durations
(gauge)
[Statestore] The last duration (in seconds) spent sending non-priority topic update RPCs. Includes subscriber-side processing time and network transmission time.
Shown as second
impala.statestore.live_backends
(gauge)
[Statestore] The number of registered Statestore subscribers.
impala.statestore.max_priority_topic_update_durations
(gauge)
[Statestore] The maximum duration (in seconds) spent sending priority topic update RPCs. Includes subscriber-side processing time and network transmission time.
Shown as second
impala.statestore.max_topic_update_durations
(gauge)
[Statestore] The maximum duration (in seconds) spent sending non-priority topic update RPCs. Includes subscriber-side processing time and network transmission time.
Shown as second
impala.statestore.mean_priority_topic_update_durations
(gauge)
[Statestore] The mean duration (in seconds) spent sending priority topic update RPCs. Includes subscriber-side processing time and network transmission time.
Shown as second
impala.statestore.mean_topic_update_durations
(gauge)
[Statestore] The mean (in seconds) spent sending non-priority topic update RPCs. Includes subscriber-side processing time and network transmission time.
Shown as second
impala.statestore.memory.mapped
(gauge)
[Statestore] Total bytes of memory mappings in this process (the virtual memory size).
Shown as byte
impala.statestore.memory.rss
(gauge)
[Statestore] Resident set size (RSS) of this process, including TCMalloc, buffer pool and JVM.
Shown as byte
impala.statestore.memory.total_used
(gauge)
[Statestore] Total memory currently used by TCMalloc and buffer pool.
Shown as byte
impala.statestore.min_priority_topic_update_durations
(gauge)
[Statestore] The minimum duration (in seconds) spent sending priority topic update RPCs. Includes subscriber-side processing time and network transmission time.
Shown as second
impala.statestore.min_topic_update_durations
(gauge)
[Statestore] The minimum duration (in seconds) spent sending non-priority topic update RPCs. Includes subscriber-side processing time and network transmission time.
Shown as second
impala.statestore.priority_topic_update_durations.count
(count)
[Statestore] The time (in seconds) spent sending priority topic update RPCs. Includes subscriber-side processing time and network transmission time.
Shown as second
impala.statestore.register_subscriber_call_duration.count
(count)
[Statestore] Count of duration the StateStore Service spent to register subscribers.
impala.statestore.register_subscriber_call_duration.quantile
(gauge)
[Statestore] Quantile of duration the StateStore Service spent to register subscribers.
Shown as second
impala.statestore.register_subscriber_call_duration.sum
(count)
[Statestore] Sum of duration the StateStore Service spent to register subscribers.
Shown as second
impala.statestore.subscriber.heartbeat.client_cache.clients_in_use
(gauge)
[Statestore] The number of clients in use by the Statestore heartbeat client cache.
impala.statestore.subscriber.heartbeat.client_cache.total_clients
(gauge)
[Statestore] The total number of clients in the Statestore heartbeat client cache.
impala.statestore.subscriber.update_state.client_cache.clients_in_use
(gauge)
[Statestore] The number of clients in use by the Statestore update state client cache.
impala.statestore.subscriber.update_state.client_cache.total_clients
(gauge)
[Statestore] The total number of clients in the Statestore update state client cache.
impala.statestore.tcmalloc.in_use
(gauge)
[Statestore] Number of bytes used by the application. This will not typically match the memory use reported by the OS, because it does not include TCMalloc overhead or memory fragmentation.
Shown as byte
impala.statestore.tcmalloc.pageheap.free
(gauge)
[Statestore] Number of bytes in free, mapped pages in the page heap. These bytes can be used to fulfill allocation requests. They always count towards virtual memory usage, and unless the underlying memory is swapped out by the OS, they also count towards physical memory usage.
Shown as byte
impala.statestore.tcmalloc.pageheap.unmapped
(gauge)
[Statestore] Number of bytes in free, mapped pages in the page heap. These bytes can be used to fulfill allocation requests. They always count towards virtual memory usage, and unless the underlying memory is swapped out by the OS, they also count towards physical memory usage.
Shown as byte
impala.statestore.tcmalloc.physical_reserved
(gauge)
[Statestore] Derived metric computing the amount of physical memory (in bytes) used by the process, including that actually in use and free bytes reserved by TCMalloc. Does not include the TCMalloc metadata.
Shown as byte
impala.statestore.tcmalloc.total_reserved
(gauge)
[Statestore] Bytes of system memory reserved by TCMalloc.
Shown as byte
impala.statestore.thread_manager.running_threads
(gauge)
[Statestore] The number of running threads in this process.
impala.statestore.thread_manager.total_threads_created
(gauge)
[Statestore] Threads created over the lifetime of the process.
impala.statestore.thrift_server.connection_setup_queue_size
(gauge)
[Statestore] The number of connections to the Statestore Service that have been accepted and are waiting to be setup.
impala.statestore.thrift_server.connection_setup_time.count
(count)
[Statestore] Count of duration the clients of StateStore Service spent waiting for connection to be set up.
impala.statestore.thrift_server.connection_setup_time.quantile
(gauge)
[Statestore] Quantile of duration the clients of StateStore Service spent waiting for connection to be set up.
Shown as second
impala.statestore.thrift_server.connection_setup_time.sum
(count)
[Statestore] Sum of duration the clients of the StateStore Service spent waiting for connection to be set up.
Shown as second
impala.statestore.thrift_server.connections_in_use
(gauge)
[Statestore] The number of active connections to this StateStore's service.
impala.statestore.thrift_server.svc_thread_wait_time.count
(count)
[Statestore] Count of duration the clients of StateStore Service spent waiting for service threads.
impala.statestore.thrift_server.svc_thread_wait_time.quantile
(gauge)
[Statestore] Quantile of duration the clients of StateStore Service spent waiting for service threads.
Shown as second
impala.statestore.thrift_server.svc_thread_wait_time.sum
(count)
[Statestore] Sum of duration the clients of StateStore Service spent waiting for service threads.
Shown as second
impala.statestore.thrift_server.timedout_cnxn_requests
(gauge)
[Statestore] The number of connection requests to the Statestore Service that have been timed out waiting to be setup.
impala.statestore.thrift_server.total_connections.count
(count)
[Statestore] The total number of connections made to this StateStore's StateStore service over its lifetime.
impala.statestore.topic_update_durations.count
(count)
[Statestore] The time (in seconds) spent sending non-priority topic update RPCs. Includes subscriber-side processing time and network transmission time.
Shown as second
impala.statestore.total_key_size
(gauge)
[Statestore] The sum of the size of all keys for all topics tracked by the StateStore.
Shown as byte
impala.statestore.total_topic_size
(gauge)
[Statestore] The sum of the size of all keys and all values for all topics tracked by the StateStore.
Shown as byte
impala.statestore.total_value_size
(gauge)
[Statestore] The sum of the size of all values for all topics tracked by the StateStore.
Shown as byte

Events

The Impala integration does not include any events.

Service Checks

impala.daemon.openmetrics.health
Returns CRITICAL if the check cannot access the Prometheus metrics endpoint of the Impala daemon instance.
Statuses: ok, critical

impala.statestore.openmetrics.health
Returns CRITICAL if the check cannot access the Prometheus metrics endpoint of the Impala statestore instance.
Statuses: ok, critical

impala.catalog.openmetrics.health
Returns CRITICAL if the check cannot access the Prometheus metrics endpoint of the Impala catalog instance.
Statuses: ok, critical

Logs

The Impala integration can collect logs from the Impala services and forward them to Datadog.

  1. Collecting logs is disabled by default in the Datadog Agent. Enable it in your datadog.yaml file:

    logs_enabled: true
    
  2. Uncomment and edit the logs configuration block in your impalad.d/conf.yaml file. Here’s an example with the daemon process:

    logs:
      - type: file
        path: /var/log/impala/impalad.INFO
        source: impala
        tags:
        - service_type:daemon
        log_processing_rules:
        - type: multi_line
          pattern: ^[IWEF]\d{4} (\d{2}:){2}\d{2}
          name: new_log_start_with_log_level_and_date
      - type: file
        path: /var/log/impala/impalad.WARNING
        source: impala
        tags:
        - service_type:daemon
        log_processing_rules:
        - type: multi_line
          pattern: ^[IWEF]\d{4} (\d{2}:){2}\d{2}
          name: new_log_start_with_log_level_and_date
      - type: file
        path: /var/log/impala/impalad.ERROR
        source: impala
        tags:
        - service_type:daemon
        log_processing_rules:
        - type: multi_line
          pattern: ^[IWEF]\d{4} (\d{2}:){2}\d{2}
          name: new_log_start_with_log_level_and_date
      - type: file
        path: /var/log/impala/impalad.FATAL
        source: impala
        tags:
        - service_type:daemon
        log_processing_rules:
        - type: multi_line
          pattern: ^[IWEF]\d{4} (\d{2}:){2}\d{2}
          name: new_log_start_with_log_level_and_date
    

See the example configuration file on how to collect all logs.

Troubleshooting

Need help? Contact Datadog support.