- 重要な情報
- はじめに
- 用語集
- ガイド
- エージェント
- インテグレーション
- OpenTelemetry
- 開発者
- API
- CoScreen
- アプリ内
- Service Management
- インフラストラクチャー
- アプリケーションパフォーマンス
- 継続的インテグレーション
- ログ管理
- セキュリティ
- UX モニタリング
- 管理
Supported OS
このチェックは、Hive の Hive Metastore と HiveServer2 の 2 つを監視します。
Hive チェックは Datadog Agent パッケージに含まれています。サーバーに追加でインストールする必要はありません。
Hive Metastore と HiveServer2 のメトリクスを有効化するには、以下のプロパティを追加して、HIVE_HOME/conf/hive-site.xml
にある Hive コンフィギュレーションファイルを編集します。
<property>
<name>hive.metastore.metrics.enabled</name>
<value>true</value>
</property>
<property>
<name>hive.server2.metrics.enabled</name>
<value>true</value>
</property>
HiveServer2、Hive Metastore、またはその両方の JMX リモート接続を有効にします。たとえば、以下のように HADOOP_CLIENT_OPTS
環境変数を設定します。
export HADOOP_CLIENT_OPTS="$HADOOP_CLIENT_OPTS -Dcom.sun.management.jmxremote \
-Dcom.sun.management.jmxremote.authenticate=false -Dcom.sun.management.jmxremote.ssl=false \
-Dcom.sun.management.jmxremote.port=8808"
次に、HiveServer2 または Hive Metastore を再起動します。Hive Metastore と HiveServer2 で同じ JMX 接続を共有することはできません。
ホストで実行中の Agent に対してこのチェックを構成するには:
ホストで実行中の Agent でこのチェックを構成する場合は、以下の手順に従ってください。コンテナ環境の場合は、コンテナ化 セクションを参照してください。
Hive のパフォーマンスデータを収集するには、Agent のコンフィギュレーションディレクトリのルートにある conf.d/
フォルダーの hive.d/conf.yaml
ファイルを編集します。使用可能なすべてのコンフィギュレーションオプションについては、サンプル hive.d/conf.yaml
を参照してください。
このチェックでは、インスタンスあたりのメトリクス数が 350 に制限されています。返されたメトリクスの数は、情報ページに表示されます。以下で説明するコンフィギュレーションを編集することで、関心があるメトリクスを指定できます。 収集するメトリクスをカスタマイズする方法については、JMX チェックのドキュメント で詳細な手順を参照してください。制限数以上のメトリクスを監視する必要がある場合は、Datadog のサポートチーム までお問い合わせください。
Agent バージョン 6.0 以降で利用可能
Datadog Agent で、ログの収集はデフォルトで無効になっています。以下のように、datadog.yaml
でこれを有効にする必要があります。
logs_enabled: true
Hive のログの収集を開始するには、次の構成ブロックを hive.d/conf.yaml
ファイルに追加します。
logs:
- type: file
path: /tmp/<USER>/hive.log
source: hive
service: '<SERVICE_NAME>'
log_processing_rules:
- type: multi_line
name: new_log_start_with_date
pattern: \d{4}\-\d{2}\-\d{2}
path
パラメーターと service
パラメーターの値を変更し、環境に合わせて構成してください。使用可能なすべての構成オプションの詳細については、サンプル hive.d/conf.yaml
を参照してください。
コンテナ環境の場合は、オートディスカバリーのインテグレーションテンプレート のガイドを参照して、次のパラメーターを適用してください。
Datadog-Hive インテグレーションを使用してメトリクスを収集するには、JMX を使用したオートディスカバリー ガイドを参照してください。
Agent バージョン 6.0 以降で利用可能
Datadog Agent で、ログの収集はデフォルトで無効になっています。有効にする方法については、Kubernetes ログ収集 を参照してください。
パラメーター | 値 |
---|---|
<LOG_CONFIG> | {"source": "hive", "service": "<サービス名>", "log_processing_rules":{"type":"multi_line","name":"new_log_start_with_date", "pattern":"\d{4}\-\d{2}\-\d{2}"}} |
Agent の status サブコマンドを実行
し、Checks セクションで Hive
を探します。
hive.metastore.open_connections (gauge) | Number of connection opened. Shown as connection |
hive.metastore.db.created (rate) | Total number of created database. Shown as item |
hive.metastore.db.deleted (rate) | Total number of deleted database. Shown as item |
hive.metastore.db.init (rate) | Number of initialized database. Shown as item |
hive.metastore.table.created (rate) | Total number of created table. Shown as table |
hive.metastore.table.deleted (rate) | Total number of deleted table. Shown as table |
hive.metastore.table.init (rate) | Number of initialized table. Shown as table |
hive.metastore.partition.created (rate) | Total number of created partition. Shown as item |
hive.metastore.partition.deleted (rate) | Total number of deleted partition. Shown as item |
hive.metastore.partition.init (rate) | Number of initialized partition. Shown as item |
hive.metastore.directsql_errors (gauge) | Number of SQL error. Shown as unit |
hive.metastore.api.create_table (rate) | API call to create a table. Shown as task |
hive.metastore.api.create_table.active_call (gauge) | Active API call to create a table. Shown as unit |
hive.metastore.api.get_table (rate) | API call to get a table. Shown as task |
hive.metastore.api.get_table.active_call (gauge) | Active API call to get a table. Shown as unit |
hive.metastore.api.drop_table (rate) | API call to drop a table. Shown as task |
hive.metastore.api.drop_table.active_call (gauge) | Active API call to drop a table. Shown as unit |
hive.metastore.api.init (rate) | API initialization. Shown as task |
hive.metastore.api.init.active_call (gauge) | Active API initialization. Shown as unit |
hive.metastore.api.get_all_databases (rate) | API call to get all databases. Shown as task |
hive.metastore.api.get_all_databases.active_call (gauge) | Active API call to get all databases. Shown as unit |
hive.metastore.api.get_database (rate) | API call to get a database. Shown as task |
hive.metastore.api.get_database.active_call (gauge) | Active API call to get a database. Shown as unit |
hive.metastore.api.get_all_tables (rate) | API call to get all tables. Shown as task |
hive.metastore.api.get_all_tables.active_call (gauge) | Active API call to get all tables. Shown as unit |
hive.metastore.api.shutdown (rate) | API shutdown. Shown as task |
hive.metastore.api.shutdown.active_call (gauge) | Active API shutdown. Shown as unit |
hive.metastore.api.flushcache (rate) | API flushcache. Shown as task |
hive.metastore.api.flushcache.active_call (gauge) | Active API flushcache. Shown as unit |
hive.metastore.api.alter_table (rate) | API call to alter table. Shown as task |
hive.metastore.api.alter_table.active_call (gauge) | Active API call to alter table. Shown as unit |
hive.metastore.api.get_all_functions (rate) | API call to get all functions. Shown as task |
hive.metastore.api.get_all_functions.active_call (gauge) | Active API call to get all functions. Shown as unit |
hive.metastore.api.get_table_req (rate) | Shown as task |
hive.metastore.api.get_table_req.active_call (gauge) | Shown as unit |
hive.server.open_operations (gauge) | Operation opened in the HiveServer2. Shown as operation |
hive.server.session.active (gauge) | Number of active session. Shown as session |
hive.server.session.active.time_mean (gauge) | Average time a session has been active. Shown as millisecond |
hive.server.session.open (gauge) | Number of opened session. Shown as session |
hive.server.session.open.time_mean (gauge) | Average time a session has been opened. Shown as millisecond |
hive.server.queries.submitted.active_call (gauge) | Active submitted queries. Shown as query |
hive.server.queries.submitted.max (gauge) | Max time for a submitted query. Shown as millisecond |
hive.server.queries.submitted.mean (gauge) | Mean time for a submitted query. Shown as millisecond |
hive.server.queries.submitted.min (gauge) | Min time for a submitted query. Shown as millisecond |
hive.server.queries.submitted.75percentile (gauge) | P75 time for a submitted query. Shown as millisecond |
hive.server.queries.submitted.95percentile (gauge) | P95 time for a submitted query. Shown as millisecond |
hive.server.queries.submitted.count (rate) | Number of submitted query. Shown as query |
hive.server.queries.submitted.meanrate (gauge) | Submitted query rate. Shown as query |
hive.server.queries.compiling.active_call (gauge) | Active compiling queries. Shown as query |
hive.server.queries.compiling.max (gauge) | Max time for compiling a query. Shown as millisecond |
hive.server.queries.compiling.mean (gauge) | Mean time for compiling a query. Shown as millisecond |
hive.server.queries.compiling.min (gauge) | Min time for compiling a query. Shown as millisecond |
hive.server.queries.compiling.75percentile (gauge) | P75 time for compiling a query. Shown as millisecond |
hive.server.queries.compiling.95percentile (gauge) | P95 time for compiling a query. Shown as millisecond |
hive.server.queries.compiling.count (rate) | Number of compiled query. Shown as query |
hive.server.queries.compiling.meanrate (gauge) | Compiling query rate. Shown as query |
hive.server.api.queries.executing.active_call (gauge) | Active executing queries. Shown as query |
hive.server.queries.executing.max (gauge) | Max time for executing a query. Shown as millisecond |
hive.server.queries.executing.mean (gauge) | Mean time for executing a query. Shown as millisecond |
hive.server.queries.executing.min (gauge) | Min time for executing a query. Shown as millisecond |
hive.server.queries.executing.75percentile (gauge) | P75 time for executing a query. Shown as millisecond |
hive.server.queries.executing.95percentile (gauge) | P95 time for executing a query. Shown as millisecond |
hive.server.queries.executing.count (rate) | Number of executed queries. Shown as query |
hive.server.queries.executing.meanrate (gauge) | Executing query rate. Shown as query |
hive.server.queries.succeeded.count (rate) | Number of succeeded queries. Shown as query |
hive.server.queries.succeeded.meanrate (gauge) | Succeeded queries rate. Shown as query |
hive.server.api.sql_operation.pending.active_call (gauge) | Active pending sql operation. Shown as operation |
hive.server.api.sql_operation.pending.max (gauge) | Max time in pending state for an sql operation. Shown as millisecond |
hive.server.api.sql_operation.pending.mean (gauge) | Mean time in pending state for an sql operation. Shown as millisecond |
hive.server.api.sql_operation.pending.min (gauge) | Min time in pending state for an sql operation. Shown as millisecond |
hive.server.api.sql_operation.pending.75percentile (gauge) | P75 time in pending state for an sql operation. Shown as millisecond |
hive.server.api.sql_operation.pending.95percentile (gauge) | P95 time in pending state for an sql operation. Shown as millisecond |
hive.server.api.sql_operation.pending.count (rate) | Number of sql operation in pending state. Shown as operation |
hive.server.api.sql_operation.pending.meanrate (gauge) | Pending sql operation rate. Shown as operation |
hive.server.api.sql_operation.running.active_call (gauge) | Active running sql operation. Shown as operation |
hive.server.api.sql_operation.running.max (gauge) | Max time for running state for a sql operation. Shown as millisecond |
hive.server.api.sql_operation.running.mean (gauge) | Mean time for running state for a sql operation. Shown as millisecond |
hive.server.api.sql_operation.running.min (gauge) | Min time for running state for a sql operation. Shown as millisecond |
hive.server.api.sql_operation.running.75percentile (gauge) | P75 time for running state for a sql operation. Shown as millisecond |
hive.server.api.sql_operation.running.95percentile (gauge) | P95 time for running state for a sql operation. Shown as millisecond |
hive.server.api.sql_operation.running.count (rate) | Number of sql operation in running state. Shown as operation |
hive.server.api.sql_operation.running.meanrate (gauge) | Running sql operation rate. Shown as operation |
hive.server.sql_operation.completed.closed (rate) | Number of closed sql operation. Shown as operation |
hive.server.sql_operation.completed.finished (rate) | Number of finished sql operation. Shown as operation |
hive.server.sql_operation.user.active (gauge) | Number of active user. Shown as user |
hive.server.api.operation.initialized.active_call (gauge) | Active initialized operation. Shown as user |
hive.server.api.operation.initialized.max (gauge) | Max time to init an operation. Shown as millisecond |
hive.server.api.operation.initialized.mean (gauge) | Mean time to init an operation. Shown as millisecond |
hive.server.api.operation.initialized.min (gauge) | Min time to init an operation. Shown as millisecond |
hive.server.api.operation.initialized.75percentile (gauge) | P75 time to init an operation. Shown as millisecond |
hive.server.api.operation.initialized.95percentile (gauge) | P95 time to init an operation. Shown as millisecond |
hive.server.api.operation.initialized.count (rate) | Number of operation initialized. Shown as operation |
hive.server.api.operation.initialized.meanrate (gauge) | Operation initialization rate. Shown as operation |
hive.server.api.operation.pending.active_call (gauge) | Active pending operation. Shown as operation |
hive.server.api.operation.pending.max (gauge) | Max time in pending state for an sql operation. Shown as millisecond |
hive.server.api.operation.pending.mean (gauge) | Mean time in pending state for an sql operation. Shown as millisecond |
hive.server.api.operation.pending.min (gauge) | Min time in pending state for an sql operation. Shown as millisecond |
hive.server.api.operation.pending.75percentile (gauge) | P75 time in pending state for an sql operation. Shown as millisecond |
hive.server.api.operation.pending.95percentile (gauge) | P95 time in pending state for an sql operation. Shown as millisecond |
hive.server.api.operation.pending.count (rate) | Number of operation in pending state. Shown as operation |
hive.server.api.operation.pending.meanrate (gauge) | Operation pending rate. Shown as operation |
hive.server.api.operation.running.active_call (gauge) | Active running operation. Shown as operation |
hive.server.api.operation.running.max (gauge) | Max time in running state for an sql operation. Shown as millisecond |
hive.server.api.operation.running.mean (gauge) | Mean time in running state for an sql operation. Shown as millisecond |
hive.server.api.operation.running.min (gauge) | Min time in running state for an sql operation. Shown as millisecond |
hive.server.api.operation.running.75percentile (gauge) | P75 time in running state for an sql operation. Shown as millisecond |
hive.server.api.operation.running.95percentile (gauge) | P95 time in running state for an sql operation. Shown as millisecond |
hive.server.api.operation.running.count (rate) | Number of operation in running state. Shown as millisecond |
hive.server.api.operation.running.meanrate (gauge) | Operation running rate. Shown as operation |
hive.server.operation.completed.finished (rate) | Number of finished operation. Shown as operation |
hive.server.operation.completed.closed (rate) | Number of closed operation. Shown as operation |
hive.server.memory.heap.init (gauge) | Memory used at the initialization by the HiveServer2 or Metastore. Depends on the connected instance type Shown as byte |
hive.server.memory.heap.used (gauge) | Memory used by the HiveServer2 or Metastore. Depends on the connected instance type Shown as byte |
hive.server.memory.heap.max (gauge) | Maximum memory that can be used by the HiveServer2 or Metastore. Depends on the connected instance type Shown as byte |
hive.server.memory.heap.committed (gauge) | Committed memory for the HiveServer2 or Metastore. Depends on the connected instance type Shown as byte |
hive.server.memory.non_heap.init (gauge) | Memory used at the initialization by the HiveServer2 or Metastore. Depends on the connected instance type Shown as byte |
hive.server.memory.non_heap.used (gauge) | Memory used by the HiveServer2 or Metastore. Depends on the connected instance type Shown as byte |
hive.server.memory.non_heap.max (gauge) | Maximum memory that can be used by the HiveServer2 or Metastore. Depends on the connected instance type Shown as byte |
hive.server.memory.non_heap.committed (gauge) | Committed memory for the HiveServer2 or Metastore. Depends on the connected instance type Shown as byte |
hive.server.memory.total.init (gauge) | Total memory at the initialization by the HiveServer2 or Metastore. Depends on the connected instance type Shown as byte |
hive.server.memory.total.used (gauge) | Total memory used by the HiveServer2 or Metastore. Depends on the connected instance type Shown as byte |
hive.server.memory.total.max (gauge) | Total maximum memory that can be used for the HiveServer2 or Metastore. Depends on the connected instance type Shown as byte |
hive.server.memory.total.committed (gauge) | Total committed memory for the HiveServer2 or Metastore. Depends on the connected instance type Shown as byte |
Hive チェックには、イベントは含まれません。
hive.can_connect
Returns CRITICAL
if the Agent is unable to connect to and collect metrics from the monitored HiveServer2/Hive Metastore instance, WARNING
if no metrics are collected, and OK
otherwise.
Statuses: ok, critical, warning
ご不明な点は、Datadog のサポートチーム までお問合せください。