---
title: Hive
description: Gathers various JMX metrics from HiveServer2 and Hive MetaStore
breadcrumbs: Docs > Integrations > Hive
---
# Hive
Supported OS Integration version2.4.0
## Overview{% #overview %}
This check monitors two parts of [Hive](https://cwiki.apache.org/confluence/display/Hive/Home): Hive Metastore and HiveServer2.
**Minimum Agent version:** 6.11.3
## Setup{% #setup %}
### Installation{% #installation %}
The Hive check is included in the [Datadog Agent](https://app.datadoghq.com/account/settings/agent/latest) package. No additional installation is needed on your server.
### Configuration{% #configuration %}
#### Hive setup{% #hive-setup %}
1. Edit the Hive configuration file in [`HIVE_HOME/conf/hive-site.xml`](https://cwiki.apache.org/confluence/display/Hive/Configuration+Properties#ConfigurationProperties-Metrics) to enable the Hive Metastore and HiveServer2 metrics by adding these properties:
```xml
hive.metastore.metrics.enabled
true
hive.server2.metrics.enabled
true
```
1. Enable a JMX remote connection for the HiveServer2 and/or for the Hive Metastore. For example, set the `HADOOP_CLIENT_OPTS` environment variable:
```gdscript3
export HADOOP_CLIENT_OPTS="$HADOOP_CLIENT_OPTS -Dcom.sun.management.jmxremote \
-Dcom.sun.management.jmxremote.authenticate=false -Dcom.sun.management.jmxremote.ssl=false \
-Dcom.sun.management.jmxremote.port=8808"
```
Then restart the HiveServer2 or the Hive Metastore. Hive Metastore and HiveServer2 cannot share the same JMX connection.
{% tab title="Host" %}
#### Host{% #host %}
To configure this check for an Agent running on a host:
Follow the instructions below to configure this check for an Agent running on a host. For containerized environments, see the Containerized section.
##### Metric collection{% #metric-collection %}
1. Edit the `hive.d/conf.yaml` file, in the `conf.d/` folder at the root of your Agent's configuration directory to start collecting your hive performance data. See the [sample hive.d/conf.yaml](https://github.com/DataDog/integrations-core/blob/master/hive/datadog_checks/hive/data/conf.yaml.example) for all available configuration options.
This check has a limit of 350 metrics per instance. The number of returned metrics is indicated in [the status page](https://docs.datadoghq.com/agent/guide/agent-commands/#agent-status-and-information). You can specify the metrics you are interested in by editing the configuration below. To learn how to customize the metrics to collect, see the [JMX Checks documentation](https://docs.datadoghq.com/integrations/java/) for more detailed instructions. If you need to monitor more metrics, contact [Datadog support](https://docs.datadoghq.com/help/).
1. [Restart the Agent](https://docs.datadoghq.com/agent/guide/agent-commands/#start-stop-and-restart-the-agent).
##### Log collection{% #log-collection %}
*Available for Agent versions >6.0*
1. Collecting logs is disabled by default in the Datadog Agent, you need to enable it in `datadog.yaml`:
```yaml
logs_enabled: true
```
1. Add this configuration block to your `hive.d/conf.yaml` file to start collecting your Hive logs:
```yaml
logs:
- type: file
path: /tmp//hive.log
source: hive
service: ''
log_processing_rules:
- type: multi_line
name: new_log_start_with_date
pattern: \d{4}\-\d{2}\-\d{2}
```
Change the `path` and `service` parameter values and configure them for your environment. See the [sample hive.d/conf.yaml](https://github.com/DataDog/integrations-core/blob/master/hive/datadog_checks/hive/data/conf.yaml.example) for all available configuration options.
1. [Restart the Agent](https://docs.datadoghq.com/agent/guide/agent-commands/#start-stop-and-restart-the-agent).
{% /tab %}
{% tab title="Containerized" %}
#### Containerized{% #containerized %}
For containerized environments, see the [Autodiscovery Integration Templates](https://docs.datadoghq.com/agent/kubernetes/integrations/) for guidance on applying the parameters below.
##### Metric collection{% #metric-collection %}
To collect metrics with the Datadog-Hive integration, see the [Autodiscovery with JMX](https://docs.datadoghq.com/agent/guide/autodiscovery-with-jmx/?tab=containerizedagent) guide.
##### Log collection{% #log-collection %}
*Available for Agent versions >6.0*
Collecting logs is disabled by default in the Datadog Agent. To enable it, see [Kubernetes Log Collection](https://docs.datadoghq.com/agent/kubernetes/log/).
| Parameter | Value |
| -------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `` | `{"source": "hive", "service": "", "log_processing_rules":{"type":"multi_line","name":"new_log_start_with_date", "pattern":"\d{4}\-\d{2}\-\d{2}"}}` |
{% /tab %}
### Validation{% #validation %}
[Run the Agent's status subcommand](https://docs.datadoghq.com/agent/guide/agent-commands/#agent-status-and-information) and look for `Hive` under the Checks section.
## Data Collected{% #data-collected %}
### Metrics{% #metrics %}
| |
| |
| **hive.metastore.api.alter\_table**(rate) | API call to alter table.*Shown as task* |
| **hive.metastore.api.alter\_table.active\_call**(gauge) | Active API call to alter table.*Shown as unit* |
| **hive.metastore.api.create\_table**(rate) | API call to create a table.*Shown as task* |
| **hive.metastore.api.create\_table.active\_call**(gauge) | Active API call to create a table.*Shown as unit* |
| **hive.metastore.api.drop\_table**(rate) | API call to drop a table.*Shown as task* |
| **hive.metastore.api.drop\_table.active\_call**(gauge) | Active API call to drop a table.*Shown as unit* |
| **hive.metastore.api.flushcache**(rate) | API flushcache.*Shown as task* |
| **hive.metastore.api.flushcache.active\_call**(gauge) | Active API flushcache.*Shown as unit* |
| **hive.metastore.api.get\_all\_databases**(rate) | API call to get all databases.*Shown as task* |
| **hive.metastore.api.get\_all\_databases.active\_call**(gauge) | Active API call to get all databases.*Shown as unit* |
| **hive.metastore.api.get\_all\_functions**(rate) | API call to get all functions.*Shown as task* |
| **hive.metastore.api.get\_all\_functions.active\_call**(gauge) | Active API call to get all functions.*Shown as unit* |
| **hive.metastore.api.get\_all\_tables**(rate) | API call to get all tables.*Shown as task* |
| **hive.metastore.api.get\_all\_tables.active\_call**(gauge) | Active API call to get all tables.*Shown as unit* |
| **hive.metastore.api.get\_database**(rate) | API call to get a database.*Shown as task* |
| **hive.metastore.api.get\_database.active\_call**(gauge) | Active API call to get a database.*Shown as unit* |
| **hive.metastore.api.get\_table**(rate) | API call to get a table.*Shown as task* |
| **hive.metastore.api.get\_table.active\_call**(gauge) | Active API call to get a table.*Shown as unit* |
| **hive.metastore.api.get\_table\_req**(rate) | *Shown as task* |
| **hive.metastore.api.get\_table\_req.active\_call**(gauge) | *Shown as unit* |
| **hive.metastore.api.init**(rate) | API initialization.*Shown as task* |
| **hive.metastore.api.init.active\_call**(gauge) | Active API initialization.*Shown as unit* |
| **hive.metastore.api.shutdown**(rate) | API shutdown.*Shown as task* |
| **hive.metastore.api.shutdown.active\_call**(gauge) | Active API shutdown.*Shown as unit* |
| **hive.metastore.db.created**(rate) | Total number of created database.*Shown as item* |
| **hive.metastore.db.deleted**(rate) | Total number of deleted database.*Shown as item* |
| **hive.metastore.db.init**(rate) | Number of initialized database.*Shown as item* |
| **hive.metastore.directsql\_errors**(gauge) | Number of SQL error.*Shown as unit* |
| **hive.metastore.open\_connections**(gauge) | Number of connection opened.*Shown as connection* |
| **hive.metastore.partition.created**(rate) | Total number of created partition.*Shown as item* |
| **hive.metastore.partition.deleted**(rate) | Total number of deleted partition.*Shown as item* |
| **hive.metastore.partition.init**(rate) | Number of initialized partition.*Shown as item* |
| **hive.metastore.table.created**(rate) | Total number of created table.*Shown as table* |
| **hive.metastore.table.deleted**(rate) | Total number of deleted table.*Shown as table* |
| **hive.metastore.table.init**(rate) | Number of initialized table.*Shown as table* |
| **hive.server.api.operation.initialized.75percentile**(gauge) | P75 time to init an operation.*Shown as millisecond* |
| **hive.server.api.operation.initialized.95percentile**(gauge) | P95 time to init an operation.*Shown as millisecond* |
| **hive.server.api.operation.initialized.active\_call**(gauge) | Active initialized operation.*Shown as user* |
| **hive.server.api.operation.initialized.count**(rate) | Number of operation initialized.*Shown as operation* |
| **hive.server.api.operation.initialized.max**(gauge) | Max time to init an operation.*Shown as millisecond* |
| **hive.server.api.operation.initialized.mean**(gauge) | Mean time to init an operation.*Shown as millisecond* |
| **hive.server.api.operation.initialized.meanrate**(gauge) | Operation initialization rate.*Shown as operation* |
| **hive.server.api.operation.initialized.min**(gauge) | Min time to init an operation.*Shown as millisecond* |
| **hive.server.api.operation.pending.75percentile**(gauge) | P75 time in pending state for an sql operation.*Shown as millisecond* |
| **hive.server.api.operation.pending.95percentile**(gauge) | P95 time in pending state for an sql operation.*Shown as millisecond* |
| **hive.server.api.operation.pending.active\_call**(gauge) | Active pending operation.*Shown as operation* |
| **hive.server.api.operation.pending.count**(rate) | Number of operation in pending state.*Shown as operation* |
| **hive.server.api.operation.pending.max**(gauge) | Max time in pending state for an sql operation.*Shown as millisecond* |
| **hive.server.api.operation.pending.mean**(gauge) | Mean time in pending state for an sql operation.*Shown as millisecond* |
| **hive.server.api.operation.pending.meanrate**(gauge) | Operation pending rate.*Shown as operation* |
| **hive.server.api.operation.pending.min**(gauge) | Min time in pending state for an sql operation.*Shown as millisecond* |
| **hive.server.api.operation.running.75percentile**(gauge) | P75 time in running state for an sql operation.*Shown as millisecond* |
| **hive.server.api.operation.running.95percentile**(gauge) | P95 time in running state for an sql operation.*Shown as millisecond* |
| **hive.server.api.operation.running.active\_call**(gauge) | Active running operation.*Shown as operation* |
| **hive.server.api.operation.running.count**(rate) | Number of operation in running state.*Shown as millisecond* |
| **hive.server.api.operation.running.max**(gauge) | Max time in running state for an sql operation.*Shown as millisecond* |
| **hive.server.api.operation.running.mean**(gauge) | Mean time in running state for an sql operation.*Shown as millisecond* |
| **hive.server.api.operation.running.meanrate**(gauge) | Operation running rate.*Shown as operation* |
| **hive.server.api.operation.running.min**(gauge) | Min time in running state for an sql operation.*Shown as millisecond* |
| **hive.server.api.queries.executing.active\_call**(gauge) | Active executing queries.*Shown as query* |
| **hive.server.api.sql\_operation.pending.75percentile**(gauge) | P75 time in pending state for an sql operation.*Shown as millisecond* |
| **hive.server.api.sql\_operation.pending.95percentile**(gauge) | P95 time in pending state for an sql operation.*Shown as millisecond* |
| **hive.server.api.sql\_operation.pending.active\_call**(gauge) | Active pending sql operation.*Shown as operation* |
| **hive.server.api.sql\_operation.pending.count**(rate) | Number of sql operation in pending state.*Shown as operation* |
| **hive.server.api.sql\_operation.pending.max**(gauge) | Max time in pending state for an sql operation.*Shown as millisecond* |
| **hive.server.api.sql\_operation.pending.mean**(gauge) | Mean time in pending state for an sql operation.*Shown as millisecond* |
| **hive.server.api.sql\_operation.pending.meanrate**(gauge) | Pending sql operation rate.*Shown as operation* |
| **hive.server.api.sql\_operation.pending.min**(gauge) | Min time in pending state for an sql operation.*Shown as millisecond* |
| **hive.server.api.sql\_operation.running.75percentile**(gauge) | P75 time for running state for a sql operation.*Shown as millisecond* |
| **hive.server.api.sql\_operation.running.95percentile**(gauge) | P95 time for running state for a sql operation.*Shown as millisecond* |
| **hive.server.api.sql\_operation.running.active\_call**(gauge) | Active running sql operation.*Shown as operation* |
| **hive.server.api.sql\_operation.running.count**(rate) | Number of sql operation in running state.*Shown as operation* |
| **hive.server.api.sql\_operation.running.max**(gauge) | Max time for running state for a sql operation.*Shown as millisecond* |
| **hive.server.api.sql\_operation.running.mean**(gauge) | Mean time for running state for a sql operation.*Shown as millisecond* |
| **hive.server.api.sql\_operation.running.meanrate**(gauge) | Running sql operation rate.*Shown as operation* |
| **hive.server.api.sql\_operation.running.min**(gauge) | Min time for running state for a sql operation.*Shown as millisecond* |
| **hive.server.memory.heap.committed**(gauge) | Committed memory for the HiveServer2 or Metastore. Depends on the connected instance type*Shown as byte* |
| **hive.server.memory.heap.init**(gauge) | Memory used at the initialization by the HiveServer2 or Metastore. Depends on the connected instance type*Shown as byte* |
| **hive.server.memory.heap.max**(gauge) | Maximum memory that can be used by the HiveServer2 or Metastore. Depends on the connected instance type*Shown as byte* |
| **hive.server.memory.heap.used**(gauge) | Memory used by the HiveServer2 or Metastore. Depends on the connected instance type*Shown as byte* |
| **hive.server.memory.non\_heap.committed**(gauge) | Committed memory for the HiveServer2 or Metastore. Depends on the connected instance type*Shown as byte* |
| **hive.server.memory.non\_heap.init**(gauge) | Memory used at the initialization by the HiveServer2 or Metastore. Depends on the connected instance type*Shown as byte* |
| **hive.server.memory.non\_heap.max**(gauge) | Maximum memory that can be used by the HiveServer2 or Metastore. Depends on the connected instance type*Shown as byte* |
| **hive.server.memory.non\_heap.used**(gauge) | Memory used by the HiveServer2 or Metastore. Depends on the connected instance type*Shown as byte* |
| **hive.server.memory.total.committed**(gauge) | Total committed memory for the HiveServer2 or Metastore. Depends on the connected instance type*Shown as byte* |
| **hive.server.memory.total.init**(gauge) | Total memory at the initialization by the HiveServer2 or Metastore. Depends on the connected instance type*Shown as byte* |
| **hive.server.memory.total.max**(gauge) | Total maximum memory that can be used for the HiveServer2 or Metastore. Depends on the connected instance type*Shown as byte* |
| **hive.server.memory.total.used**(gauge) | Total memory used by the HiveServer2 or Metastore. Depends on the connected instance type*Shown as byte* |
| **hive.server.open\_operations**(gauge) | Operation opened in the HiveServer2.*Shown as operation* |
| **hive.server.operation.completed.closed**(rate) | Number of closed operation.*Shown as operation* |
| **hive.server.operation.completed.finished**(rate) | Number of finished operation.*Shown as operation* |
| **hive.server.queries.compiling.75percentile**(gauge) | P75 time for compiling a query.*Shown as millisecond* |
| **hive.server.queries.compiling.95percentile**(gauge) | P95 time for compiling a query.*Shown as millisecond* |
| **hive.server.queries.compiling.active\_call**(gauge) | Active compiling queries.*Shown as query* |
| **hive.server.queries.compiling.count**(rate) | Number of compiled query.*Shown as query* |
| **hive.server.queries.compiling.max**(gauge) | Max time for compiling a query.*Shown as millisecond* |
| **hive.server.queries.compiling.mean**(gauge) | Mean time for compiling a query.*Shown as millisecond* |
| **hive.server.queries.compiling.meanrate**(gauge) | Compiling query rate.*Shown as query* |
| **hive.server.queries.compiling.min**(gauge) | Min time for compiling a query.*Shown as millisecond* |
| **hive.server.queries.executing.75percentile**(gauge) | P75 time for executing a query.*Shown as millisecond* |
| **hive.server.queries.executing.95percentile**(gauge) | P95 time for executing a query.*Shown as millisecond* |
| **hive.server.queries.executing.count**(rate) | Number of executed queries.*Shown as query* |
| **hive.server.queries.executing.max**(gauge) | Max time for executing a query.*Shown as millisecond* |
| **hive.server.queries.executing.mean**(gauge) | Mean time for executing a query.*Shown as millisecond* |
| **hive.server.queries.executing.meanrate**(gauge) | Executing query rate.*Shown as query* |
| **hive.server.queries.executing.min**(gauge) | Min time for executing a query.*Shown as millisecond* |
| **hive.server.queries.submitted.75percentile**(gauge) | P75 time for a submitted query.*Shown as millisecond* |
| **hive.server.queries.submitted.95percentile**(gauge) | P95 time for a submitted query.*Shown as millisecond* |
| **hive.server.queries.submitted.active\_call**(gauge) | Active submitted queries.*Shown as query* |
| **hive.server.queries.submitted.count**(rate) | Number of submitted query.*Shown as query* |
| **hive.server.queries.submitted.max**(gauge) | Max time for a submitted query.*Shown as millisecond* |
| **hive.server.queries.submitted.mean**(gauge) | Mean time for a submitted query.*Shown as millisecond* |
| **hive.server.queries.submitted.meanrate**(gauge) | Submitted query rate.*Shown as query* |
| **hive.server.queries.submitted.min**(gauge) | Min time for a submitted query.*Shown as millisecond* |
| **hive.server.queries.succeeded.count**(rate) | Number of succeeded queries.*Shown as query* |
| **hive.server.queries.succeeded.meanrate**(gauge) | Succeeded queries rate.*Shown as query* |
| **hive.server.session.active**(gauge) | Number of active session.*Shown as session* |
| **hive.server.session.active.time\_mean**(gauge) | Average time a session has been active.*Shown as millisecond* |
| **hive.server.session.open**(gauge) | Number of opened session.*Shown as session* |
| **hive.server.session.open.time\_mean**(gauge) | Average time a session has been opened.*Shown as millisecond* |
| **hive.server.sql\_operation.completed.closed**(rate) | Number of closed sql operation.*Shown as operation* |
| **hive.server.sql\_operation.completed.finished**(rate) | Number of finished sql operation.*Shown as operation* |
| **hive.server.sql\_operation.user.active**(gauge) | Number of active user.*Shown as user* |
### Events{% #events %}
The Hive check does not include any events.
### Service Checks{% #service-checks %}
**hive.can\_connect**
Returns `CRITICAL` if the Agent is unable to connect to and collect metrics from the monitored HiveServer2/Hive Metastore instance, `WARNING` if no metrics are collected, and `OK` otherwise.
*Statuses: ok, critical, warning*
## Troubleshooting{% #troubleshooting %}
Need help? Contact [Datadog support](https://docs.datadoghq.com/help/).