概要

BigQuery は、Google が提供するフルマネージド型の企業向け分析用データウェアハウスです。ペタバイト規模の処理を低コストで行うことができます。

Google BigQuery からメトリクスを取得して、以下のことができます。

  • BigQuery クエリのパフォーマンスを視覚化。
  • BigQuery クエリのパフォーマンスをアプリケーションと関連付け。

セットアップ

インストール

Google Cloud Platform インテグレーションをまだセットアップしていない場合は、最初にセットアップします。これ以外に必要なインストール手順はありません。

収集データ

Google BigQuery のログは Google Cloud Logging で収集され、Cloud Pub/Sub トピックを通じて Dataflow ジョブに送信されます。まだの場合は、Datadog Dataflow テンプレートでロギングをセットアップしてください

これが完了したら、Google BigQuery のログを Google Cloud Logging から Pub/Sub トピックへエクスポートします。

  1. Google Cloud Logging のページに移動し、Google BigQuery のログを絞り込みます。
  2. Create Export をクリックし、シンクに名前を付けます。
  3. 宛先として “Cloud Pub/Sub” を選択し、その目的で作成された Pub/Sub トピックを選択します。: Pub/Sub トピックは別のプロジェクトに配置できます。
  4. 作成をクリックし、確認メッセージが表示されるまで待ちます。

収集データ

メトリクス

gcp.bigquery.job.num_in_flight
(gauge)
Number of in flight jobs.
Shown as job
gcp.bigquery.query.biengine_fallback_count
(count)
The reasons that queries failed BI Engine execution.
Shown as query
gcp.bigquery.query.column_metadata_index_staleness.avg
(gauge)
The average distribution of staleness in milliseconds of the column metadata index for queries that successfully used the column metadata index in the last sampling interval.
Shown as millisecond
gcp.bigquery.query.column_metadata_index_staleness.samplecount
(gauge)
The sample count for distribution of staleness in milliseconds of the column metadata index for queries that successfully used the column metadata index in the last sampling interval.
Shown as millisecond
gcp.bigquery.query.column_metadata_index_staleness.sumsqdev
(gauge)
The sum of squared deviation for distribution of staleness in milliseconds of the column metadata index for queries that successfully used the column metadata index in the last sampling interval.
Shown as millisecond
gcp.bigquery.query.count
(gauge)
Queries in flight.
Shown as query
gcp.bigquery.query.execution_count
(count)
Number of queries executed.
Shown as query
gcp.bigquery.query.execution_times.avg
(gauge)
Average of query execution times.
Shown as second
gcp.bigquery.query.execution_times.samplecount
(count)
Sample Count of query execution times.
Shown as second
gcp.bigquery.query.execution_times.sumsqdev
(gauge)
Sum of Squared Deviation for query execution times.
Shown as second
gcp.bigquery.query.scanned_bytes
(rate)
Number of scanned bytes. Note: this metric is available with a six-hour delay.
Shown as byte
gcp.bigquery.query.scanned_bytes_billed
(rate)
Number of scanned bytes billed. Note: this metric is available with a six-hour delay.
Shown as byte
gcp.bigquery.query.statement_scanned_bytes
(count)
Scanned bytes broken down by statement type. Note: this metric is available with a six-hour delay.
Shown as byte
gcp.bigquery.query.statement_scanned_bytes_billed
(count)
Scanned bytes billed broken down by statement type. Note: this metric is available with a six-hour delay.
Shown as byte
gcp.bigquery.slots.allocated
(gauge)
Number of BigQuery slots currently allocated for project, slot allocation can be broken down based on reservation and job type.
gcp.bigquery.slots.allocated_for_project
(gauge)
Number of BigQuery slots currently allocated for the project.
gcp.bigquery.slots.allocated_for_project_and_job_type
(gauge)
Number of BigQuery slots currently allocated for the project and job type.
gcp.bigquery.slots.allocated_for_reservation
(gauge)
Number of BigQuery slots currently allocated for project in the reservation.
gcp.bigquery.slots.assigned
(gauge)
The number of slots assigned to the given project or organization.
gcp.bigquery.slots.capacity_committed
(gauge)
The total slot capacity commitments purchased through this administrator project or organization.
gcp.bigquery.slots.max_assigned
(gauge)
The maximum number of slots assigned to the given project or organization.
gcp.bigquery.slots.total_allocated_for_reservation
(gauge)
Number of BigQuery slots currently allocated across all projects in the reservation.
gcp.bigquery.storage.insertall_inserted_bytes
(count)
The number of bytes uploaded by the project using the InsertAll streaming API.
Shown as byte
gcp.bigquery.storage.insertall_inserted_rows
(count)
The number of rows uploaded by the project using the InsertAll streaming API.
Shown as row
gcp.bigquery.storage.stored_bytes
(gauge)
Number of bytes stored. Note: this metric is available with a three-hour delay.
Shown as byte
gcp.bigquery.storage.table_count
(gauge)
Number of tables. Note: this metric is available with a three-hour delay.
Shown as table
gcp.bigquery.storage.uploaded_bytes
(count)
Number of uploaded bytes. Note: this metric is available with a six-hour delay.
Shown as byte
gcp.bigquery.storage.uploaded_bytes_billed
(count)
Number of uploaded bytes billed. Note: this metric is available with a six-hour delay.
Shown as byte
gcp.bigquery.storage.uploaded_row_count
(count)
Number of uploaded rows. Note: this metric is available with a six-hour delay.
Shown as row

イベント

Google BigQuery インテグレーションには、イベントは含まれません。

サービスチェック

Google BigQuery インテグレーションには、サービスのチェック機能は含まれません。

トラブルシューティング

ご不明な点は、Datadog のサポートチームまでお問い合わせください。