aws.mwaa.celery_task_timeout_error
(count) | Number of Celery task timeout errors raised when publishing a task to the Celery broker.
Shown as unit |
aws.mwaa.celery_worker_heartbeat
(count) | Number of heartbeats emitted by Celery workers.
Shown as unit |
aws.mwaa.collect_dbdags
(gauge) | Average milliseconds taken for fetching all Serialized Dags from DB. Available in both Airflow v1 and v2.
Shown as millisecond |
aws.mwaa.collect_dbdags.maximum
(gauge) | Maximum milliseconds taken for fetching all Serialized Dags from DB. Available in both Airflow v1 and v2.
Shown as millisecond |
aws.mwaa.collect_dbdags.minimum
(gauge) | Minimum milliseconds taken for fetching all Serialized Dags from DB. Available in both Airflow v1 and v2.
Shown as millisecond |
aws.mwaa.critical_section_busy
(count) | Count of times a scheduler process tried to get a lock on the critical section (needed to send tasks to the executor) and found it locked by another process. Only available in Airflow v2.
Shown as unit |
aws.mwaa.critical_section_duration
(gauge) | Average milliseconds spent in the critical section of scheduler loop - only a single scheduler can enter this loop at a time. Only available in Airflow v2.
Shown as millisecond |
aws.mwaa.critical_section_duration.maximum
(gauge) | Maximum milliseconds spent in the critical section of scheduler loop - only a single scheduler can enter this loop at a time. Only available in Airflow v2.
Shown as millisecond |
aws.mwaa.critical_section_duration.minimum
(gauge) | Minimum milliseconds spent in the critical section of scheduler loop - only a single scheduler can enter this loop at a time. Only available in Airflow v2.
Shown as millisecond |
aws.mwaa.critical_section_query_duration
(gauge) | Average milliseconds spent querying the database within the critical section of the scheduler loop. Available for Airflow v2.5.1 and later.
Shown as millisecond |
aws.mwaa.dag_bag_size
(count) | Number of DAGs found when the scheduler ran a scan based on it’s configuration. Available in both Airflow v1 and v2.
Shown as unit |
aws.mwaa.dagcallback_exceptions
(count) | Number of exceptions raised from DAG callbacks. When this happens, it means DAG callback is not working. Only available in Airflow v2.
Shown as unit |
aws.mwaa.dag_runs_running
(gauge) | Number of DAG runs currently in the running state.
Shown as unit |
aws.mwaa.dagdependency_check
(gauge) | Average milliseconds taken to check DAG dependencies. Available in both Airflow v1 and v2.
Shown as millisecond |
aws.mwaa.dagdependency_check.maximum
(gauge) | Maximum milliseconds taken to check DAG dependencies. Available in both Airflow v1 and v2.
Shown as millisecond |
aws.mwaa.dagdependency_check.minimum
(gauge) | Minimum milliseconds taken to check DAG dependencies. Available in both Airflow v1 and v2.
Shown as millisecond |
aws.mwaa.dagduration_failed
(gauge) | Milliseconds taken for a DagRun to reach failed state. Available in both Airflow v1 and v2.
Shown as millisecond |
aws.mwaa.dagduration_success
(gauge) | Milliseconds taken for a DagRun to reach success state. Available in both Airflow v1 and v2.
Shown as millisecond |
aws.mwaa.dagfile_processing_last_duration
(gauge) | Average milliseconds taken to load the given DAG file. Available in both Airflow v1 and v2.
Shown as millisecond |
aws.mwaa.dagfile_processing_last_duration.maximum
(gauge) | Maximum milliseconds taken to load the given DAG file. Available in both Airflow v1 and v2.
Shown as millisecond |
aws.mwaa.dagfile_processing_last_duration.minimum
(gauge) | Minimum milliseconds taken to load the given DAG file. Available in both Airflow v1 and v2.
Shown as millisecond |
aws.mwaa.dagfile_processing_last_num_of_db_queries
(gauge) | Number of database queries executed the last time the given DAG file was processed. Available for Airflow v2.10.1 and later.
Shown as query |
aws.mwaa.dagfile_processing_last_run_seconds_ago
(gauge) | Seconds since <dag_file> was last processed. Available in both Airflow v1 and v2.
Shown as second |
aws.mwaa.dagfile_refresh_error
(count) | Number of failures loading any DAG files. Available in both Airflow v1 and v2.
Shown as unit |
aws.mwaa.dagschedule_delay
(gauge) | Milliseconds of delay between the scheduled DagRun start date and the actual DagRun start date. Available in both Airflow v1 and v2.
Shown as millisecond |
aws.mwaa.ephemeral_storage_reserved
(gauge) | Total amount of ephemeral storage reserved for the given Airflow component container.
Shown as megabyte |
aws.mwaa.ephemeral_storage_utilized
(gauge) | Amount of ephemeral storage utilized by the given Airflow component container.
Shown as megabyte |
aws.mwaa.exception_failures
(count) | Number of failures caused by exception in the previous smart sensor poking loop. Only available in Airflow v2.
Shown as unit |
aws.mwaa.export_metrics_heartbeat
(count) | Heartbeats emitted by the metrics export process.
Shown as unit |
aws.mwaa.failed_celery_task_execution
(count) | Number of Celery task command executions that failed. Available for Airflow v2.4.3 and later.
Shown as unit |
aws.mwaa.failed_dag_count
(count) | Number of DAGs currently failing to import or parse.
Shown as unit |
aws.mwaa.failed_slacallback
(count) | Number of failed SLA miss callback notification attempts. Only available for Airflow v2.4.3 to v2.10.3.
Shown as unit |
aws.mwaa.failed_slaemail_attempts
(count) | Number of failed SLA miss email notification attempts. Only available in Airflow v2.
Shown as unit |
aws.mwaa.file_path_queue_size
(gauge) | Number of DAG files currently queued for processing by the DAG file processor.
Shown as unit |
aws.mwaa.file_path_queue_update_count
(count) | Number of times the DAG file path processing queue was updated. Available for Airflow v2.6.3 and later.
Shown as unit |
aws.mwaa.finished_task_instances
(count) | Number of task instances that finished, across all end states.
Shown as unit |
aws.mwaa.first_task_scheduling_delay
(gauge) | Milliseconds elapsed between first task start_date and dagrun expected start. Only available in Airflow v2.
Shown as millisecond |
aws.mwaa.import_errors
(count) | Number of errors from trying to parse DAG files. Available in both Airflow v1 and v2.
Shown as unit |
aws.mwaa.infra_failures
(count) | Number of infrastructure failures in the previous smart sensor poking loop. Only available in Airflow v2.
Shown as unit |
aws.mwaa.job_end
(count) | Number of ended <job_name> job, ex. SchedulerJob, LocalTaskJob. Available in both Airflow v1 and v2.
Shown as unit |
aws.mwaa.job_heartbeat_failure
(count) | Number of failed Heartbeats for a <job_name> job, ex. SchedulerJob, LocalTaskJob. Available in both Airflow v1 and v2.
Shown as unit |
aws.mwaa.job_start
(count) | Number of started <job_name> job, ex. SchedulerJob, LocalTaskJob. Available in both Airflow v1 and v2.
Shown as unit |
aws.mwaa.manager_stalls
(count) | Number of stalled DagFileProcessorManager. Only available in Airflow v2.
Shown as unit |
aws.mwaa.network_rx_bytes
(rate) | Rate of bytes received by the given Airflow component container.
Shown as byte |
aws.mwaa.network_tx_bytes
(rate) | Rate of bytes sent by the given Airflow component container.
Shown as byte |
aws.mwaa.open_slots
(count) | Number of open slots on executor. Available in both Airflow v1 and v2.
Shown as unit |
aws.mwaa.operator_failures
(count) | Operator <operator_name> failures. Available in both Airflow v1 and v2.
Shown as unit |
aws.mwaa.operator_failures.sum
(count) | Total number of operator <operator_name> failures. Available in both Airflow v1 and v2.
Shown as unit |
aws.mwaa.operator_successes
(count) | Operator <operator_name> successes. Available in both Airflow v1 and v2.
Shown as unit |
aws.mwaa.operator_successes.sum
(count) | Total number of operator <operator_name> successes. Available in both Airflow v1 and v2.
Shown as unit |
aws.mwaa.orphaned
(count) | Number of datasets marked as orphaned. Available for Airflow v2.6.3 and later.
Shown as unit |
aws.mwaa.orphaned_tasks_adopted
(count) | Number of Orphaned tasks adopted by the Scheduler. Only available in Airflow v2.
Shown as unit |
aws.mwaa.orphaned_tasks_cleared
(count) | Number of Orphaned tasks cleared by the Scheduler. Only available in Airflow v2.
Shown as unit |
aws.mwaa.other_callback_count
(count) | Number of callbacks processed by the DAG processor that are not DAG or task callbacks. Available for Airflow v2.6.3 and later.
Shown as unit |
aws.mwaa.poked_exceptions
(count) | Number of exceptions in the previous smart sensor poking loop. Only available in Airflow v2.
Shown as unit |
aws.mwaa.poked_success
(count) | Number of newly succeeded tasks poked by the smart sensor in the previous poking loop. Only available in Airflow v2.
Shown as unit |
aws.mwaa.poked_tasks
(count) | Number of tasks poked by the smart sensor in the previous poking loop. Only available in Airflow v2.
Shown as unit |
aws.mwaa.pool_deferred_slots
(count) | Number of deferred slots in the pool. Available for Airflow v2.7.2 and later.
Shown as unit |
aws.mwaa.pool_open_slots
(count) | Number of open slots in the pool. Only available in Airflow v2.
Shown as unit |
aws.mwaa.pool_queued_slots
(count) | Number of queued slots in the pool. Only available in Airflow v2.
Shown as unit |
aws.mwaa.pool_running_slots
(count) | Number of running slots in the pool. Only available in Airflow v2.
Shown as unit |
aws.mwaa.pool_scheduled_slots
(count) | Number of scheduled slots in the pool. Available for Airflow v2.10.1 and later.
Shown as unit |
aws.mwaa.pool_starving_tasks
(count) | Number of starving tasks in the pool. Available in both Airflow v1 and v2.
Shown as unit |
aws.mwaa.pool_used_slots
(count) | Number of used slots in the pool. Only available in Airflow v1.
Shown as unit |
aws.mwaa.processes
(count) | Number of DAG file processor subprocesses currently running.
Shown as unit |
aws.mwaa.processor_timeouts
(count) | Number of file processors that have been killed due to taking too long. Available in both Airflow v1 and v2.
Shown as unit |
aws.mwaa.queued_tasks
(count) | Sum number of queued tasks on executor. Available in both Airflow v1 and v2.
Shown as unit |
aws.mwaa.queued_tasks.average
(gauge) | Average number of queued tasks on executor. Available in both Airflow v1 and v2.
Shown as unit |
aws.mwaa.running_tasks
(count) | Sum number of running tasks on executor. Available in both Airflow v1 and v2.
Shown as unit |
aws.mwaa.running_tasks.average
(gauge) | Average number of running tasks on executor. Available in both Airflow v1 and v2.
Shown as unit |
aws.mwaa.s3_sync_errors
(count) | Number of errors syncing DAGs or plugins from the environment’s S3 bucket.
Shown as unit |
aws.mwaa.scheduler_heartbeat
(count) | Scheduler heartbeats. Available in both Airflow v1 and v2.
Shown as unit |
aws.mwaa.scheduler_loop_duration
(gauge) | Average milliseconds spent in a single scheduler loop iteration. Available for Airflow v2.5.1 and later.
Shown as millisecond |
aws.mwaa.sla_callback_count
(count) | Number of SLA callbacks processed by the DAG processor. Available for Airflow v2.6.3 and later.
Shown as unit |
aws.mwaa.slamissed
(count) | Number of SLA misses recorded for scheduled tasks. Only available for Airflow v2.4.3 to v2.10.3.
Shown as unit |
aws.mwaa.task_cpu_usage
(gauge) | Percentage of CPU utilized by the given task. Available for Airflow v2.10.1 and later.
Shown as percent |
aws.mwaa.task_executed
(count) | Number of tasks executed by a Celery worker.
Shown as unit |
aws.mwaa.task_failures
(count) | Number of task failures.
Shown as unit |
aws.mwaa.task_instance_created_using_operator
(count) | Number of tasks instances created for a given Operator. Available in both Airflow v1 and v2.
Shown as unit |
aws.mwaa.task_instance_duration
(gauge) | Milliseconds taken to finish a task. Available in both Airflow v1 and v2.
Shown as millisecond |
aws.mwaa.task_instance_failures
(count) | Overall task instances failures. Available in both Airflow v1 and v2.
Shown as unit |
aws.mwaa.task_instance_finished
(count) | Number of completed task in a given dag. Similar to <job_name>_end but for task. Only available in Airflow v2.
Shown as unit |
aws.mwaa.task_instance_previously_succeeded
(count) | Number of previously succeeded task instances. Only available in Airflow v2.
Shown as unit |
aws.mwaa.task_instance_queued_duration
(gauge) | Milliseconds a task instance spent in the queued state before running. Available for Airflow v2.7.2 and later.
Shown as millisecond |
aws.mwaa.task_instance_running
(gauge) | Number of task instances currently running.
Shown as unit |
aws.mwaa.task_instance_scheduled_duration
(gauge) | Milliseconds a task instance spent in the scheduled state before running. Available for Airflow v2.7.2 and later.
Shown as millisecond |
aws.mwaa.task_instance_started
(count) | Number of started task in a given dag. Similar to <job_name>_start but for task. Only available in Airflow v2.
Shown as unit |
aws.mwaa.task_instance_successes
(count) | Overall task instances successes. Available in both Airflow v1 and v2.
Shown as unit |
aws.mwaa.task_memory_usage
(gauge) | Percentage of memory utilized by the given task. Available for Airflow v2.10.1 and later.
Shown as percent |
aws.mwaa.task_pulled
(count) | Number of tasks pulled from the Celery broker by a worker.
Shown as unit |
aws.mwaa.task_queued
(count) | Number of tasks queued to the Celery broker for execution.
Shown as unit |
aws.mwaa.task_removed_from_dag
(count) | Number of tasks removed for a given dag (i.e. task no longer exists in DAG). Available in both Airflow v1 and v2.
Shown as unit |
aws.mwaa.task_restored_to_dag
(count) | Number of tasks restored for a given dag (i.e. task instance which was previously in REMOVED state in the DB is added to DAG file). Available in both Airflow v1 and v2.
Shown as unit |
aws.mwaa.task_returned
(count) | Number of task results returned by a Celery worker.
Shown as unit |
aws.mwaa.task_timeout_error
(count) | Number of AirflowTaskTimeout errors raised when publishing Task to Celery Broker. Only available in Airflow v2.
Shown as unit |
aws.mwaa.tasks_executable
(count) | Sum number of tasks that are ready for execution (set to queued) with respect to pool limits, DAG concurrency, executor state, and priority. Available in both Airflow v1 and v2.
Shown as unit |
aws.mwaa.tasks_executable.average
(gauge) | Average number of tasks that are ready for execution (set to queued) with respect to pool limits, DAG concurrency, executor state, and priority. Available in both Airflow v1 and v2.
Shown as unit |
aws.mwaa.tasks_killed_externally
(count) | Sum number of tasks killed externally. Available in both Airflow v1 and v2.
Shown as unit |
aws.mwaa.tasks_killed_externally.average
(count) | Average number of tasks killed externally. Available in both Airflow v1 and v2.
Shown as unit |
aws.mwaa.tasks_pending
(count) | Sum number of tasks pending. Available in Airflow v1.
Shown as unit |
aws.mwaa.tasks_pending.average
(gauge) | Average number of tasks pending. Available in both Airflow v1.
Shown as unit |
aws.mwaa.tasks_running
(count) | Sum number of tasks running in executor. Available in both Airflow v1 and v2.
Shown as unit |
aws.mwaa.tasks_running.average
(gauge) | Average number of tasks running in executor. Available in both Airflow v1 and v2.
Shown as unit |
aws.mwaa.tasks_starving
(count) | Sum number of tasks that cannot be scheduled because of no open slot in pool. Available in both Airflow v1 and v2.
Shown as unit |
aws.mwaa.tasks_starving.average
(gauge) | Average number of tasks that cannot be scheduled because of no open slot in pool. Available in both Airflow v1 and v2.
Shown as unit |
aws.mwaa.tasks_without_dag_run
(count) | Number of tasks without DagRuns or with DagRuns not in Running state. Available in both Airflow v1 and v2.
Shown as unit |
aws.mwaa.total_parse_time
(gauge) | Average seconds taken to scan and import all DAG files once. Available in both Airflow v1 and v2.
Shown as second |
aws.mwaa.total_parse_time.maximum
(gauge) | Maximum seconds taken to scan and import all DAG files once. Available in both Airflow v1 and v2.
Shown as second |
aws.mwaa.total_parse_time.minimum
(gauge) | Minimum seconds taken to scan and import all DAG files once. Available in both Airflow v1 and v2.
Shown as second |
aws.mwaa.triggered_dag_runs
(gauge) | Number of DAG runs triggered by dataset updates. Available for Airflow v2.6.3 and later.
Shown as unit |
aws.mwaa.triggerer_capacity
(gauge) | Number of open trigger slots available on a triggerer process.
Shown as unit |
aws.mwaa.triggerer_heartbeat
(count) | Number of heartbeats emitted by a triggerer process. Available for Airflow v2.8.1 and later.
Shown as unit |
aws.mwaa.triggers_blocked_main_thread
(count) | Number of times a trigger blocked a triggerer’s main event loop thread. Available for Airflow v2.7.2 and later.
Shown as unit |
aws.mwaa.triggers_failed
(count) | Number of deferred tasks that failed through a triggerer. Available for Airflow v2.7.2 and later.
Shown as unit |
aws.mwaa.triggers_running
(gauge) | Number of triggers currently running on a triggerer process. Available for Airflow v2.7.2 and later.
Shown as unit |
aws.mwaa.triggers_succeeded
(count) | Number of deferred tasks that completed successfully through a triggerer. Available for Airflow v2.7.2 and later.
Shown as unit |
aws.mwaa.updates
(count) | Number of dataset update events emitted. Available for Airflow v2.6.3 and later.
Shown as unit |
aws.mwaa.zombies_killed
(count) | Zombie tasks killed. Available in both Airflow v1 and v2.
Shown as unit |
aws.mwaa.active_connection_count
(count) | Total number of concurrent TCP connections active from clients to the Amazon MWAA web servers.
Shown as connection |
aws.mwaa.approximate_age_of_oldest_task
(gauge) | Approximate age of the oldest task waiting to be picked up by a worker.
Shown as second |
aws.mwaa.cpuutilization
(gauge) | Average percentage of CPU utilization for a scheduler, worker, or web server container, or for the metadatabase instance.
Shown as percent |
aws.mwaa.cpuutilization.maximum
(gauge) | Maximum percentage of CPU utilization for a scheduler, worker, or web server container, or for the metadatabase instance.
Shown as percent |
aws.mwaa.cpuutilization.samplecount
(count) | Number of CPUUtilization datapoints reported for the scheduler, worker, or web server container in the period. On the Cluster dimension, this count also represents the number of running tasks for that component, such as the number of additional workers or web servers. |
aws.mwaa.database_connections
(gauge) | Number of connections in use on the Amazon MWAA metadatabase.
Shown as connection |
aws.mwaa.disk_queue_depth
(gauge) | Number of outstanding read and write requests waiting to access the metadatabase disk.
Shown as request |
aws.mwaa.environment_queued_tasks
(gauge) | Approximate number of tasks available for processing that have not yet been picked up by a worker. Analogous to the Amazon SQS ApproximateNumberOfMessagesVisible metric, not the Airflow-native aws.mwaa.queued_tasks metric from the AmazonMWAA namespace.
Shown as unit |
aws.mwaa.environment_running_tasks
(gauge) | Approximate number of tasks currently in flight, picked up by a worker but not yet complete. Analogous to the Amazon SQS ApproximateNumberOfMessagesNotVisible metric, not the Airflow-native aws.mwaa.running_tasks metric from the AmazonMWAA namespace.
Shown as unit |
aws.mwaa.environment_ephemeral_storage_reserved
(gauge) | Total amount of ephemeral storage reserved for the scheduler, worker, or web server container. Distinct from the Airflow-native aws.mwaa.ephemeral_storage_reserved metric from the AmazonMWAA namespace, which is reported per AirflowComponent.
Shown as megabyte |
aws.mwaa.environment_ephemeral_storage_utilized
(gauge) | Amount of ephemeral storage utilized by the scheduler, worker, or web server container. Distinct from the Airflow-native aws.mwaa.ephemeral_storage_utilized metric from the AmazonMWAA namespace, which is reported per AirflowComponent.
Shown as megabyte |
aws.mwaa.freeable_memory
(gauge) | Amount of available random access memory on the metadatabase instance.
Shown as byte |
aws.mwaa.memory_utilization
(gauge) | Average percentage of memory used by the scheduler, worker, or web server container.
Shown as percent |
aws.mwaa.memory_utilization.maximum
(gauge) | Maximum percentage of memory used by the scheduler, worker, or web server container.
Shown as percent |
aws.mwaa.memory_utilization.samplecount
(count) | Number of MemoryUtilization datapoints reported for the scheduler, worker, or web server container in the period. On the Cluster dimension, this count also represents the number of running tasks for that component, such as the number of additional workers or web servers. |
aws.mwaa.network_receive_throughput
(rate) | Rate of incoming network traffic on the metadatabase instance.
Shown as byte |
aws.mwaa.environment_network_rx_bytes
(rate) | Rate of bytes received by the scheduler, worker, or web server container. Distinct from the Airflow-native aws.mwaa.network_rx_bytes metric from the AmazonMWAA namespace, which is reported per AirflowComponent.
Shown as byte |
aws.mwaa.network_transmit_throughput
(rate) | Rate of outgoing network traffic on the metadatabase instance.
Shown as byte |
aws.mwaa.environment_network_tx_bytes
(rate) | Rate of bytes sent by the scheduler, worker, or web server container. Distinct from the Airflow-native aws.mwaa.network_tx_bytes metric from the AmazonMWAA namespace, which is reported per AirflowComponent.
Shown as byte |
aws.mwaa.read_iops
(rate) | Average number of disk read I/O operations per second on the metadatabase instance.
Shown as operation |
aws.mwaa.read_latency
(gauge) | Average amount of time taken per disk read I/O operation on the metadatabase instance.
Shown as second |
aws.mwaa.read_throughput
(rate) | Average number of bytes read from disk per second on the metadatabase instance.
Shown as byte |
aws.mwaa.write_iops
(rate) | Average number of disk write I/O operations per second on the metadatabase instance.
Shown as operation |
aws.mwaa.write_latency
(gauge) | Average amount of time taken per disk write I/O operation on the metadatabase instance.
Shown as second |
aws.mwaa.write_throughput
(rate) | Average number of bytes written to disk per second on the metadatabase instance.
Shown as byte |
