Alibaba Cloud Flink

Integration version1.0.0

To find out if this integration is available in your organization, see your Datadog Integrations page or ask your organization administrator.

To initiate an exception request to enable this integration for your organization, email support@ddog-gov.com.

Overview

Alibaba Cloud Flink is a fully managed service for building and running stream processing applications based on Apache Flink. The service supports real-time data pipelines, event-driven analytics, and continuous data processing at scale.

Use this integration to monitor job throughput, processing latency, checkpointing, and resource utilization for JobManagers and TaskManagers.

Setup

Installation

After you set up the Alibaba Cloud integration, ensure that the Realtime Compute for Apache Flink service is included in your Alibaba Cloud monitoring configuration.

Data Collected

Metrics

alibabacloud.flink.current_emit_event_time_lag.average
(gauge)
Lag between the event time of the records a Flink job emits and the time it emits them.
Shown as millisecond
alibabacloud.flink.current_emit_event_time_lag.maximum
(gauge)
Lag between the event time of the records a Flink job emits and the time it emits them.
Shown as millisecond
alibabacloud.flink.current_emit_event_time_lag.minimum
(gauge)
Lag between the event time of the records a Flink job emits and the time it emits them.
Shown as millisecond
alibabacloud.flink.current_fetch_event_time_lag.average
(gauge)
Data transmission delay of the Flink job.
Shown as millisecond
alibabacloud.flink.current_fetch_event_time_lag.maximum
(gauge)
Data transmission delay of the Flink job.
Shown as millisecond
alibabacloud.flink.current_fetch_event_time_lag.minimum
(gauge)
Data transmission delay of the Flink job.
Shown as millisecond
alibabacloud.flink.jmcpu_usage.average
(gauge)
CPU utilization of a single JobManager.
Shown as percent
alibabacloud.flink.jmcpu_usage.maximum
(gauge)
CPU utilization of a single JobManager.
Shown as percent
alibabacloud.flink.jmcpu_usage.minimum
(gauge)
CPU utilization of a single JobManager.
Shown as percent
alibabacloud.flink.jmgc_count_concurrent_mark_sweep.average
(rate)
Number of JobManager garbage collections per minute (ConcurrentMarkSweep).
Shown as operation
alibabacloud.flink.jmgc_count_concurrent_mark_sweep.maximum
(rate)
Number of JobManager garbage collections per minute (ConcurrentMarkSweep).
Shown as operation
alibabacloud.flink.jmgc_count_concurrent_mark_sweep.minimum
(rate)
Number of JobManager garbage collections per minute (ConcurrentMarkSweep).
Shown as operation
alibabacloud.flink.jmgc_count_par_new.value
(rate)
Number of JobManager garbage collections per minute (ParNew).
Shown as operation
alibabacloud.flink.jmgc_time_concurrent_mark_sweep.average
(gauge)
Time taken per JobManager garbage collection (ConcurrentMarkSweep).
Shown as millisecond
alibabacloud.flink.jmgc_time_concurrent_mark_sweep.maximum
(gauge)
Time taken per JobManager garbage collection (ConcurrentMarkSweep).
Shown as millisecond
alibabacloud.flink.jmgc_time_concurrent_mark_sweep.minimum
(gauge)
Time taken per JobManager garbage collection (ConcurrentMarkSweep).
Shown as millisecond
alibabacloud.flink.jmgc_time_par_new.value
(gauge)
Time taken per JobManager garbage collection (ParNew).
Shown as millisecond
alibabacloud.flink.jm_heap_memory_committed.average
(gauge)
Committed heap memory of the JobManager.
Shown as mebibyte
alibabacloud.flink.jm_heap_memory_committed.maximum
(gauge)
Committed heap memory of the JobManager.
Shown as mebibyte
alibabacloud.flink.jm_heap_memory_committed.minimum
(gauge)
Committed heap memory of the JobManager.
Shown as mebibyte
alibabacloud.flink.jm_heap_memory_max.average
(gauge)
Maximum heap memory of the JobManager.
Shown as mebibyte
alibabacloud.flink.jm_heap_memory_max.maximum
(gauge)
Maximum heap memory of the JobManager.
Shown as mebibyte
alibabacloud.flink.jm_heap_memory_max.minimum
(gauge)
Maximum heap memory of the JobManager.
Shown as mebibyte
alibabacloud.flink.jm_heap_memory_used.average
(gauge)
Used heap memory of the JobManager.
Shown as mebibyte
alibabacloud.flink.jm_heap_memory_used.maximum
(gauge)
Used heap memory of the JobManager.
Shown as mebibyte
alibabacloud.flink.jm_heap_memory_used.minimum
(gauge)
Used heap memory of the JobManager.
Shown as mebibyte
alibabacloud.flink.num_of_checkpoints.value
(gauge)
Number of checkpoints.
Shown as operation
alibabacloud.flink.num_of_pending_records.average
(gauge)
Number of records not yet read from the source.
Shown as record
alibabacloud.flink.num_of_pending_records.maximum
(gauge)
Number of records not yet read from the source.
Shown as record
alibabacloud.flink.num_of_pending_records.minimum
(gauge)
Number of records not yet read from the source.
Shown as record
alibabacloud.flink.num_of_records_in_from_source.value
(rate)
Number of records received from the source per minute.
Shown as record
alibabacloud.flink.num_of_records_in_from_source_per_second.value
(rate)
Number of records received from the source per second.
Shown as record
alibabacloud.flink.num_of_records_out_to_sink.value
(rate)
Number of records output to the sink per minute.
Shown as record
alibabacloud.flink.num_of_records_out_to_sink_per_second.value
(rate)
Number of records output to the sink per second.
Shown as record
alibabacloud.flink.num_of_restart.value
(rate)
Number of job error recoveries (restarts) per minute.
Shown as operation
alibabacloud.flink.source_idle_time.average
(gauge)
Idle time of the source with no data to process.
Shown as millisecond
alibabacloud.flink.source_idle_time.maximum
(gauge)
Idle time of the source with no data to process.
Shown as millisecond
alibabacloud.flink.source_idle_time.minimum
(gauge)
Idle time of the source with no data to process.
Shown as millisecond
alibabacloud.flink.tmcpu_usage.average
(gauge)
CPU utilization of a single TaskManager.
Shown as percent
alibabacloud.flink.tmcpu_usage.maximum
(gauge)
CPU utilization of a single TaskManager.
Shown as percent
alibabacloud.flink.tmcpu_usage.minimum
(gauge)
CPU utilization of a single TaskManager.
Shown as percent
alibabacloud.flink.tmgc_count_concurrent_mark_sweep.average
(rate)
Number of TaskManager garbage collections per minute (ConcurrentMarkSweep).
Shown as operation
alibabacloud.flink.tmgc_count_concurrent_mark_sweep.maximum
(rate)
Number of TaskManager garbage collections per minute (ConcurrentMarkSweep).
Shown as operation
alibabacloud.flink.tmgc_count_concurrent_mark_sweep.minimum
(rate)
Number of TaskManager garbage collections per minute (ConcurrentMarkSweep).
Shown as operation
alibabacloud.flink.tmgc_count_par_new.value
(rate)
Number of TaskManager garbage collections per minute (ParNew).
Shown as operation
alibabacloud.flink.tmgc_time_concurrent_mark_sweep.average
(gauge)
Time taken per TaskManager garbage collection (ConcurrentMarkSweep).
Shown as millisecond
alibabacloud.flink.tmgc_time_concurrent_mark_sweep.maximum
(gauge)
Time taken per TaskManager garbage collection (ConcurrentMarkSweep).
Shown as millisecond
alibabacloud.flink.tmgc_time_concurrent_mark_sweep.minimum
(gauge)
Time taken per TaskManager garbage collection (ConcurrentMarkSweep).
Shown as millisecond
alibabacloud.flink.tmgc_time_par_new.value
(gauge)
Time taken per TaskManager garbage collection (ParNew).
Shown as millisecond
alibabacloud.flink.tm_heap_memory_committed.average
(gauge)
Committed heap memory of the TaskManager.
Shown as mebibyte
alibabacloud.flink.tm_heap_memory_committed.maximum
(gauge)
Committed heap memory of the TaskManager.
Shown as mebibyte
alibabacloud.flink.tm_heap_memory_committed.minimum
(gauge)
Committed heap memory of the TaskManager.
Shown as mebibyte
alibabacloud.flink.tm_heap_memory_max.average
(gauge)
Maximum heap memory of the TaskManager.
Shown as mebibyte
alibabacloud.flink.tm_heap_memory_max.maximum
(gauge)
Maximum heap memory of the TaskManager.
Shown as mebibyte
alibabacloud.flink.tm_heap_memory_max.minimum
(gauge)
Maximum heap memory of the TaskManager.
Shown as mebibyte
alibabacloud.flink.tm_heap_memory_used.average
(gauge)
Used heap memory of the TaskManager.
Shown as mebibyte
alibabacloud.flink.tm_heap_memory_used.maximum
(gauge)
Used heap memory of the TaskManager.
Shown as mebibyte
alibabacloud.flink.tm_heap_memory_used.minimum
(gauge)
Used heap memory of the TaskManager.
Shown as mebibyte
alibabacloud.flink.tm_memory.average
(gauge)
Total process memory of the TaskManager as reported by Linux.
Shown as mebibyte
alibabacloud.flink.tm_memory.maximum
(gauge)
Total process memory of the TaskManager as reported by Linux.
Shown as mebibyte
alibabacloud.flink.tm_memory.minimum
(gauge)
Total process memory of the TaskManager as reported by Linux.
Shown as mebibyte

Events

Alibaba Cloud Flink does not include any events.

Service Checks

Alibaba Cloud Flink does not include any service checks.

Troubleshooting

Need help? Contact Datadog support.