Google Cloud Composer は、複数のクラウドやオンプレミスデータセンターにまたがるパイプラインの作成、スケジューリング、監視を可能にする、フルマネージド型のワークフローオーケストレーションサービスです。
Datadog Google Cloud Platform インテグレーションを使用して、Google Cloud Composer からメトリクスを収集できます。
Google Cloud Platform インテグレーションをまだセットアップしていない場合は、最初にセットアップします。それ以上のインストール手順はありません。
Google Cloud Composer のログは Google Cloud Logging で収集され、Cloud Pub/Sub トピックを通じて Dataflow ジョブに送信されます。まだの場合は、Datadog Dataflow テンプレートでロギングをセットアップしてください。
これが完了したら、Google Cloud Composer のログを Google Cloud Logging から Pub/Sub へエクスポートします。
| gcp.composer.environment.api.request_count (count) | Number of Composer API requests seen so far. Shown as request |
| gcp.composer.environment.api.request_latencies.avg (gauge) | Distribution of Composer API call latencies. Shown as millisecond |
| gcp.composer.environment.api.request_latencies.samplecount (count) | Sample count for API request latencies Shown as millisecond |
| gcp.composer.environment.api.request_latencies.sumsqdev (gauge) | Sum of squared deviation for API request latencies Shown as second |
| gcp.composer.environment.dag_processing.parse_error_count (count) | Number of errors raised during parsing DAG files Shown as error |
| gcp.composer.environment.dag_processing.processes (gauge) | Number of currently running DAG parsing processes Shown as process |
| gcp.composer.environment.dag_processing.processesor_timeout_count (count) | Number of file processors terminated due to processing timeout. Shown as timeout |
| gcp.composer.environment.dag_processing.total_parse_time (gauge) | Number of seconds taken to scan and import all DAG files once Shown as second |
| gcp.composer.environment.dagbag_size (gauge) | The current DAG bag size |
| gcp.composer.environment.database.cpu.reserved_cores (gauge) | Number of cores reserved for the database instance Shown as core |
| gcp.composer.environment.database.cpu.usage_time (count) | CPU usage time of the database instance, in seconds Shown as second |
| gcp.composer.environment.database.cpu.utilization (gauge) | CPU utilization ratio (from 0.0 to 1.0) of the database instance |
| gcp.composer.environment.database.disk.bytes_used (gauge) | Used disk space on the database instance, in bytes Shown as byte |
| gcp.composer.environment.database.disk.quota (gauge) | Maximum data disk size of the database instance, in bytes Shown as byte |
| gcp.composer.environment.database.disk.utilization (gauge) | Disk quota usage ratio (from 0.0 to 1.0) of the database instance |
| gcp.composer.environment.database.memory.bytes_used (gauge) | Memory usage of the database instance in bytes Shown as byte |
| gcp.composer.environment.database.memory.quota (gauge) | Maximum RAM size of the database instance, in bytes Shown as byte |
| gcp.composer.environment.database.memory.utilization (gauge) | Memory utilization ratio (from 0.0 to 1.0) of the database instance |
| gcp.composer.environment.database.network.received_bytes_count (count) | Number of bytes received by the database instance Shown as byte |
| gcp.composer.environment.database.network.sent_bytes_count (count) | Number of bytes sent by the database instance Shown as byte |
| gcp.composer.environment.database_health (gauge) | Healthiness of Composer Airflow database |
| gcp.composer.environment.executor.open_slots (gauge) | Number of open slots on executor |
| gcp.composer.environment.executor.queued_tasks (gauge) | Number of queued tasks on executor Shown as task |
| gcp.composer.environment.executor.running_tasks (gauge) | Number of running tasks on executor Shown as task |
| gcp.composer.environment.finished_task_instance_count (count) | Overall number of finished task instances Shown as instance |
| gcp.composer.environment.healthy (gauge) | Healthiness of Composer environment. |
| gcp.composer.environment.num_celery_workers (gauge) | Number of Celery workers. Shown as worker |
| gcp.composer.environment.num_workflows (gauge) | Number of workflows. |
| gcp.composer.environment.scheduler_heartbeat_count (count) | Scheduler heartbeats |
| gcp.composer.environment.task_queue_length (gauge) | Number of tasks in queue. Shown as task |
| gcp.composer.environment.unfinished_task_instances (gauge) | Overall task instances in not finished state Shown as instance |
| gcp.composer.environment.web_server.cpu.reserved_cores (gauge) | Number of cores reserved for the web server instance Shown as core |
| gcp.composer.environment.web_server.cpu.usage_time (count) | CPU usage time of the web server instance, in seconds Shown as second |
| gcp.composer.environment.web_server.health (gauge) | Healthiness of Airflow web server |
| gcp.composer.environment.web_server.memory.bytes_used (gauge) | Memory usage of the web server instance in bytes Shown as byte |
| gcp.composer.environment.web_server.memory.quota (gauge) | Maximum RAM size of the web server instance, in bytes Shown as byte |
| gcp.composer.environment.worker.max_workers (gauge) | Maximum number of Airflow workers Shown as worker |
| gcp.composer.environment.worker.min_workers (gauge) | Minimum number of Airflow workers Shown as worker |
| gcp.composer.environment.worker.pod_eviction_count (count) | Number of Airflow worker pods evictions Shown as eviction |
| gcp.composer.environment.worker.scale_factor_target (gauge) | Scale factor for Airflow workers count |
| gcp.composer.environment.zombie_task_killed_count (count) | Number of zombie tasks killed Shown as task |
| gcp.composer.workflow.run_count (count) | Number of workflow runs completed so far. |
| gcp.composer.workflow.run_duration (gauge) | Duration of workflow run completion. Shown as second |
| gcp.composer.workflow.task.run_count (count) | Number of workflow tasks completed so far. Shown as task |
| gcp.composer.workflow.task.run_duration (gauge) | Duration of task completion. Shown as second |
Google Cloud Composer インテグレーションには、イベントは含まれません。
Google Cloud Composer インテグレーションには、サービスのチェック機能は含まれません。
ご不明な点は、Datadog のサポートチームまでお問い合わせください。
