aws.glue.all.disk.available_gb
(gauge) | The available disk space aggregated across all Spark executors.
Shown as gigabyte |
aws.glue.all.disk.used_gb
(gauge) | The disk space used across all Spark executors.
Shown as gigabyte |
aws.glue.all.disk.used_percentage
(gauge) | The percentage of disk space used across all Spark executors.
Shown as percent |
aws.glue.all.jvm.heap_usage
(gauge) | The fraction (0 to 1, where 1 represents 100%) of JVM heap memory used across all Spark executors.
Shown as fraction |
aws.glue.all.jvm.heap_used
(gauge) | The JVM heap memory used across all Spark executors.
Shown as byte |
aws.glue.all.memory.heap_available
(gauge) | The available heap memory aggregated across all Spark executors.
Shown as byte |
aws.glue.all.memory.heap_used
(gauge) | The heap memory used across all Spark executors.
Shown as byte |
aws.glue.all.memory.heap_used_percentage
(gauge) | The percentage of heap memory used across all Spark executors.
Shown as percent |
aws.glue.all.memory.non_heap_available
(gauge) | The available non-heap memory aggregated across all Spark executors.
Shown as byte |
aws.glue.all.memory.non_heap_used
(gauge) | The non-heap memory used across all Spark executors.
Shown as byte |
aws.glue.all.memory.non_heap_used_percentage
(gauge) | The percentage of non-heap memory used across all Spark executors.
Shown as percent |
aws.glue.all.memory.total_available
(gauge) | The available total memory aggregated across all Spark executors.
Shown as byte |
aws.glue.all.memory.total_used
(gauge) | The total memory used across all Spark executors.
Shown as byte |
aws.glue.all.memory.total_used_percentage
(gauge) | The percentage of total memory used across all Spark executors.
Shown as percent |
aws.glue.all.s3.filesystem.read_bytes
(count) | The bytes read from Amazon S3 since the previous report.
Shown as byte |
aws.glue.all.s3.filesystem.write_bytes
(count) | The bytes written to Amazon S3 since the previous report.
Shown as byte |
aws.glue.all.system.cpu_system_load
(gauge) | The fraction (0 to 1, where 1 represents 100%) of CPU system load used across all Spark executors.
Shown as fraction |
aws.glue.driver.aggregate.bytes_read
(count) | The number of bytes read from all data sources by completed Spark tasks.
Shown as byte |
aws.glue.driver.aggregate.elapsed_time
(count) | The ETL elapsed time, excluding job bootstrap time.
Shown as millisecond |
aws.glue.driver.aggregate.jvm_gc_time
(gauge) | The JVM garbage collection time reported by the Spark driver.
Shown as millisecond |
aws.glue.driver.aggregate.num_completed_stages
(count) | The number of completed stages in the job. |
aws.glue.driver.aggregate.num_completed_tasks
(count) | The number of completed tasks in the job. |
aws.glue.driver.aggregate.num_failed_tasks
(count) | The number of failed tasks in the job. |
aws.glue.driver.aggregate.num_killed_tasks
(count) | The number of killed tasks in the job. |
aws.glue.driver.aggregate.records_read
(count) | The number of records read from all data sources by completed Spark tasks. |
aws.glue.driver.aggregate.shuffle_bytes_written
(count) | The number of bytes written by executors while shuffling data.
Shown as byte |
aws.glue.driver.aggregate.shuffle_local_bytes_read
(count) | The number of local bytes read by executors while shuffling data.
Shown as byte |
aws.glue.driver.block_manager.disk.disk_space_used_mb
(gauge) | The disk space used for blocks across all executors.
Shown as megabyte |
aws.glue.driver.bytes_read
(gauge) | The number of bytes read per input source in the job run.
Shown as byte |
aws.glue.driver.bytes_written
(gauge) | The number of bytes written per output sink in the job run.
Shown as byte |
aws.glue.driver.disk.available_gb
(gauge) | The available disk space for the Spark driver.
Shown as gigabyte |
aws.glue.driver.disk.used_gb
(gauge) | The disk space used by the Spark driver.
Shown as gigabyte |
aws.glue.driver.disk.used_percentage
(gauge) | The percentage of disk space used by the Spark driver.
Shown as percent |
aws.glue.driver.error.out_of_memory_error
(count) | The number of out-of-memory errors reported for the Spark driver. |
aws.glue.driver.executor_allocation_manager.executors.number_all_executors
(gauge) | The number of actively running job executors. |
aws.glue.driver.executor_allocation_manager.executors.number_max_needed_executors
(gauge) | The maximum number of executors needed to satisfy the current load. |
aws.glue.driver.files_read
(gauge) | The number of files read per input source in the job run. |
aws.glue.driver.files_written
(gauge) | The number of files written per output sink in the job run. |
aws.glue.driver.jvm.heap_usage
(gauge) | The fraction (0 to 1, where 1 represents 100%) of JVM heap memory used by the Spark driver.
Shown as fraction |
aws.glue.driver.jvm.heap_used
(gauge) | The JVM heap memory used by the Spark driver.
Shown as byte |
aws.glue.driver.memory.heap_available
(gauge) | The available heap memory for the Spark driver.
Shown as byte |
aws.glue.driver.memory.heap_used
(gauge) | The heap memory used by the Spark driver.
Shown as byte |
aws.glue.driver.memory.heap_used_percentage
(gauge) | The percentage of heap memory used by the Spark driver.
Shown as percent |
aws.glue.driver.memory.non_heap_available
(gauge) | The available non-heap memory for the Spark driver.
Shown as byte |
aws.glue.driver.memory.non_heap_used
(gauge) | The non-heap memory used by the Spark driver.
Shown as byte |
aws.glue.driver.memory.non_heap_used_percentage
(gauge) | The percentage of non-heap memory used by the Spark driver.
Shown as percent |
aws.glue.driver.memory.total_available
(gauge) | The available total memory for the Spark driver.
Shown as byte |
aws.glue.driver.memory.total_used
(gauge) | The total memory used by the Spark driver.
Shown as byte |
aws.glue.driver.memory.total_used_percentage
(gauge) | The percentage of total memory used by the Spark driver.
Shown as percent |
aws.glue.driver.partitions_read
(gauge) | The number of partitions read per Amazon S3 input source. |
aws.glue.driver.records_read
(gauge) | The number of records read per input source in the job run. |
aws.glue.driver.records_written
(gauge) | The number of records written per output sink in the job run. |
aws.glue.driver.s3.filesystem_read_bytes
(count) | The bytes read from Amazon S3 since the previous report.
Shown as byte |
aws.glue.driver.s3.filesystem_write_bytes
(count) | The bytes written to Amazon S3 since the previous report.
Shown as byte |
aws.glue.driver.skewness.job
(gauge) | The maximum duration-weighted skewness of the job’s stages. |
aws.glue.driver.skewness.stage
(gauge) | The execution skewness of Spark tasks in a stage. |
aws.glue.driver.streaming.batch_processing_time_in_ms
(gauge) | The time required to process a streaming micro-batch.
Shown as millisecond |
aws.glue.driver.streaming.max_consumer_lag_in_ms
(gauge) | The maximum streaming consumer lag, in milliseconds.
Shown as millisecond |
aws.glue.driver.streaming.num_records
(gauge) | The number of records received in a streaming micro-batch. |
aws.glue.driver.system.cpu_system_load
(gauge) | The fraction (0 to 1, where 1 represents 100%) of CPU system load used by the Spark driver.
Shown as fraction |
aws.glue.driver.worker_utilization
(gauge) | The percentage of allocated workers that are in use.
Shown as percent |
aws.glue.error.all.sum
(count) | The total number of failed job runs. |
aws.glue.error.compilation_error
(count) | The number of failed job runs reported with the AWS Glue error category COMPILATION_ERROR. |
aws.glue.error.connection_error
(count) | The number of failed job runs reported with the AWS Glue error category CONNECTION_ERROR. |
aws.glue.error.data_lake_framework_error
(count) | The number of failed job runs reported with the AWS Glue error category DATA_LAKE_FRAMEWORK_ERROR. |
aws.glue.error.disk_no_space_error
(count) | The number of failed job runs reported with the AWS Glue error category DISK_NO_SPACE_ERROR. |
aws.glue.error.dynamodb_error
(count) | The number of failed job runs reported with the AWS Glue error category DYNAMODB_ERROR. |
aws.glue.error.glue_error
(count) | The number of failed job runs reported with the AWS Glue error category GLUE_ERROR. |
aws.glue.error.glue_internal_service_error
(count) | The number of failed job runs reported with the AWS Glue error category GLUE_INTERNAL_SERVICE_ERROR. |
aws.glue.error.glue_job_bookmark_version_mismatch_error
(count) | The number of failed job runs reported with the AWS Glue error category GLUE_JOB_BOOKMARK_VERSION_MISMATCH_ERROR. |
aws.glue.error.glue_operation_timeout_error
(count) | The number of failed job runs reported with the AWS Glue error category GLUE_OPERATION_TIMEOUT_ERROR. |
aws.glue.error.glue_validation_error
(count) | The number of failed job runs reported with the AWS Glue error category GLUE_VALIDATION_ERROR. |
aws.glue.error.import_error
(count) | The number of failed job runs reported with the AWS Glue error category IMPORT_ERROR. |
aws.glue.error.invalid_argument_error
(count) | The number of failed job runs reported with the AWS Glue error category INVALID_ARGUMENT_ERROR. |
aws.glue.error.lakeformation_error
(count) | The number of failed job runs reported with the AWS Glue error category LAKEFORMATION_ERROR. |
aws.glue.error.launch_error
(count) | The number of failed job runs reported with the AWS Glue error category LAUNCH_ERROR. |
aws.glue.error.out_of_memory_error
(count) | The number of failed job runs reported with the AWS Glue error category OUT_OF_MEMORY_ERROR. |
aws.glue.error.permission_error
(count) | The number of failed job runs reported with the AWS Glue error category PERMISSION_ERROR. |
aws.glue.error.query_error
(count) | The number of failed job runs reported with the AWS Glue error category QUERY_ERROR. |
aws.glue.error.redshift_error
(count) | The number of failed job runs reported with the AWS Glue error category REDSHIFT_ERROR. |
aws.glue.error.resource_not_found_error
(count) | The number of failed job runs reported with the AWS Glue error category RESOURCE_NOT_FOUND_ERROR. |
aws.glue.error.resources_already_exists_error
(count) | The number of failed job runs reported with the AWS Glue error category RESOURCES_ALREADY_EXISTS_ERROR. |
aws.glue.error.s3_error
(count) | The number of failed job runs reported with the AWS Glue error category S3_ERROR. |
aws.glue.error.syntax_error
(count) | The number of failed job runs reported with the AWS Glue error category SYNTAX_ERROR. |
aws.glue.error.system_exit_error
(count) | The number of failed job runs reported with the AWS Glue error category SYSTEM_EXIT_ERROR. |
aws.glue.error.throttling_error
(count) | The number of failed job runs reported with the AWS Glue error category THROTTLING_ERROR. |
aws.glue.error.timeout_error
(count) | The number of failed job runs reported with the AWS Glue error category TIMEOUT_ERROR. |
aws.glue.error.unclassified_error
(count) | The number of failed job runs reported with the AWS Glue error category UNCLASSIFIED_ERROR. |
aws.glue.error.unclassified_spark_error
(count) | The number of failed job runs reported with the AWS Glue error category UNCLASSIFIED_SPARK_ERROR. |
aws.glue.error.unsupported_operation_error
(count) | The number of failed job runs reported with the AWS Glue error category UNSUPPORTED_OPERATION_ERROR. |
aws.glue.executor.error.out_of_memory_error
(count) | The number of out-of-memory errors reported for Spark executors. |
aws.glue.executor.jvm.heap_usage
(gauge) | The fraction (0 to 1, where 1 represents 100%) of JVM heap memory used for a Spark executor.
Shown as fraction |
aws.glue.executor.jvm.heap_used
(gauge) | The JVM heap memory used for a Spark executor.
Shown as byte |
aws.glue.executor.s3.filesystem.read_bytes
(count) | The bytes read from Amazon S3 since the previous report.
Shown as byte |
aws.glue.executor.s3.filesystem.write_bytes
(count) | The bytes written to Amazon S3 since the previous report.
Shown as byte |
aws.glue.executor.system.cpu_system_load
(gauge) | The fraction (0 to 1, where 1 represents 100%) of CPU system load used for a Spark executor.
Shown as fraction |
aws.glue.glue_alldisk_used_gb
(gauge) | The disk space used across all Spark executors.
Shown as gigabyte |
aws.glue.glue_alldisk_used_percentage
(gauge) | The percentage of disk space used across all Spark executors.
Shown as percent |
aws.glue.glue_alljvm_heap_usage
(gauge) | The fraction (0 to 1, where 1 represents 100%) of JVM heap memory used across all Spark executors.
Shown as fraction |
aws.glue.glue_alljvm_heap_used
(gauge) | The JVM heap memory used across all Spark executors.
Shown as byte |
aws.glue.glue_alljvmheapusage
(gauge) | The fraction (0 to 1, where 1 represents 100%) of JVM heap memory used across all Spark executors.
Shown as fraction |
aws.glue.glue_allmemory_heap_used_percentage
(gauge) | The percentage of heap memory used across all Spark executors.
Shown as percent |
aws.glue.glue_allmemory_total_used
(gauge) | The total memory used across all Spark executors.
Shown as byte |
aws.glue.glue_allmemory_total_used_percentage
(gauge) | The percentage of total memory used across all Spark executors.
Shown as percent |
aws.glue.glue_alls_3filesystem_readbytes
(count) | The bytes read from Amazon S3 since the previous report.
Shown as byte |
aws.glue.glue_alls_3filesystem_writebytes
(count) | The bytes written to Amazon S3 since the previous report.
Shown as byte |
aws.glue.glue_alls_3filesystemreadbytes
(count) | The bytes read from Amazon S3 since the previous report.
Shown as byte |
aws.glue.glue_allsystem_cpu_system_load
(gauge) | The fraction (0 to 1, where 1 represents 100%) of CPU system load used across all Spark executors.
Shown as fraction |
aws.glue.glue_allsystemcpu_system_load
(gauge) | The fraction (0 to 1, where 1 represents 100%) of CPU system load used across all Spark executors.
Shown as fraction |
aws.glue.glue_driver_aggregate_bytes_read
(count) | The number of bytes read from all data sources by completed Spark tasks.
Shown as byte |
aws.glue.glue_driver_aggregate_elapsed_time
(count) | The ETL elapsed time, excluding job bootstrap time.
Shown as millisecond |
aws.glue.glue_driver_aggregate_jvm_gctime
(gauge) | The JVM garbage collection time reported by the Spark driver.
Shown as millisecond |
aws.glue.glue_driver_aggregate_num_completed_stages
(count) | The number of completed stages in the job. |
aws.glue.glue_driver_aggregate_num_completed_tasks
(count) | The number of completed tasks in the job. |
aws.glue.glue_driver_aggregate_num_failed_tasks
(count) | The number of failed tasks in the job. |
aws.glue.glue_driver_aggregate_num_killed_tasks
(count) | The number of killed tasks in the job. |
aws.glue.glue_driver_aggregate_records_read
(count) | The number of records read from all data sources by completed Spark tasks. |
aws.glue.glue_driver_aggregate_shuffle_bytes_written
(count) | The number of bytes written by executors while shuffling data.
Shown as byte |
aws.glue.glue_driver_block_manager_disk_disk_space_used_mb
(gauge) | The disk space used for blocks across all executors.
Shown as megabyte |
aws.glue.glue_driver_disk_used_percentage
(gauge) | The percentage of disk space used by the Spark driver.
Shown as percent |
aws.glue.glue_driver_executor_allocation_manager_executors_number_all_executors
(gauge) | The number of actively running job executors. |
aws.glue.glue_driver_executor_allocation_manager_executors_number_max_needed_executors
(gauge) | The maximum number of executors needed to satisfy the current load. |
aws.glue.glue_driver_files_read
(gauge) | The number of files read per input source in the job run. |
aws.glue.glue_driver_jvm_heap_usage
(gauge) | The fraction (0 to 1, where 1 represents 100%) of JVM heap memory used by the Spark driver.
Shown as fraction |
aws.glue.glue_driver_memory_heap_used_percentage
(gauge) | The percentage of heap memory used by the Spark driver.
Shown as percent |
aws.glue.glue_driver_memory_total_used
(gauge) | The total memory used by the Spark driver.
Shown as byte |
aws.glue.glue_driver_records_read
(gauge) | The number of records read per input source in the job run. |
aws.glue.glue_driver_s3_filesystem_readbytes
(count) | The bytes read from Amazon S3 since the previous report.
Shown as byte |
aws.glue.glue_driver_s3_filesystem_writebytes
(count) | The bytes written to Amazon S3 since the previous report.
Shown as byte |
aws.glue.glue_driver_skewness_job
(gauge) | The maximum duration-weighted skewness of the job’s stages. |
aws.glue.glue_driver_streaming_batch_processing_time_in_ms
(gauge) | The time required to process a streaming micro-batch.
Shown as millisecond |
aws.glue.glue_driver_streaming_num_records
(gauge) | The number of records received in a streaming micro-batch. |
aws.glue.glue_driver_system_cpu_system_load
(gauge) | The fraction (0 to 1, where 1 represents 100%) of CPU system load used by the Spark driver.
Shown as fraction |
aws.glue.glue_driver_worker_utilization
(gauge) | The percentage of allocated workers that are in use.
Shown as percent |
aws.glue.glue_error_all
(count) | The total number of failed job runs. |
aws.glue.glue_succeed_all
(count) | The total number of successful job runs. |
aws.glue.gluedriveraggregateelapsed_time
(count) | The ETL elapsed time, excluding job bootstrap time.
Shown as millisecond |
aws.glue.gluedriveraggregatenum_completed_stages
(count) | The number of completed stages in the job. |
aws.glue.gluedriveraggregatenum_completed_tasks
(count) | The number of completed tasks in the job. |
aws.glue.gluedriveraggregatenum_failed_tasks
(count) | The number of failed tasks in the job. |
aws.glue.gluedriveraggregatenum_killed_tasks
(count) | The number of killed tasks in the job. |
aws.glue.glueerror_all
(count) | The total number of failed job runs. |
aws.glue.gluesucceed_all
(count) | The total number of successful job runs. |
aws.glue.succeed.all.sum
(count) | The total number of successful job runs. |
aws.glue.zeroetl.delete_count
(count) | The number of records deleted from the target Iceberg table.
Shown as record |
aws.glue.zeroetl.ingestion_completed
(count) | The number of times ingestion completed successfully for the integration.
Shown as event |
aws.glue.zeroetl.ingestion_failed
(count) | The number of times ingestion failed for the integration, reported as 1 per failed run.
Shown as event |
aws.glue.zeroetl.insert_count
(count) | The number of records inserted in the target Iceberg table.
Shown as record |
aws.glue.zeroetl.last_synced_timestamp
(gauge) | The timestamp until which the source has been synced to the target.
Shown as unix millisecond |
aws.glue.zeroetl.source_delete_count
(count) | The number of records deleted at the source during ingestion.
Shown as record |
aws.glue.zeroetl.source_ingestion_size_in_bytes
(count) | The amount of source data ingested.
Shown as byte |
aws.glue.zeroetl.source_ingestion_succeeded
(count) | The number of successful source ingestion operations.
Shown as event |
aws.glue.zeroetl.source_insert_count
(count) | The number of records inserted at the source during ingestion.
Shown as record |
aws.glue.zeroetl.source_update_count
(count) | The number of records updated at the source during ingestion.
Shown as record |
aws.glue.zeroetl.update_count
(count) | The number of records updated in the target Iceberg table.
Shown as record |
aws.glue.dataquality.rules_failed
(count) | The number of rules that failed in a Data Quality evaluation. |
aws.glue.dataquality.rules_passed
(count) | The number of rules that passed in a Data Quality evaluation. |
aws.glue.dpu_hours_of_a_compaction_job
(count) | The number of DPU hours consumed by a compaction job.
Shown as hour |
aws.glue.duration_of_job_hours
(gauge) | The duration of a table optimizer job.
Shown as hour |
aws.glue.iceberg_table_compaction_failure
(count) | The number of failed Iceberg table compaction jobs.
Shown as job |
aws.glue.iceberg_table_compaction_success
(count) | The number of successful Iceberg table compaction jobs.
Shown as job |
aws.glue.iceberg_table_orphan_file_deletion_failure
(count) | The number of failed Iceberg table orphan-file deletion jobs.
Shown as job |
aws.glue.iceberg_table_orphan_file_deletion_success
(count) | The number of successful Iceberg table orphan-file deletion jobs.
Shown as job |
aws.glue.iceberg_table_retention_failure
(count) | The number of failed Iceberg table snapshot-retention jobs.
Shown as job |
aws.glue.iceberg_table_retention_success
(count) | The number of successful Iceberg table snapshot-retention jobs.
Shown as job |
aws.glue.number_of_bytes_compacted
(count) | The number of bytes compacted by the compaction job.
Shown as byte |
aws.glue.number_of_data_file_bytes_removed
(count) | The number of data-file bytes removed by the compaction job.
Shown as byte |
aws.glue.number_of_data_files_deleted
(count) | The number of data files deleted by the snapshot-retention job.
Shown as file |
aws.glue.number_of_data_files_removed
(count) | The number of data files removed by the compaction job.
Shown as file |
aws.glue.number_of_delete_file_bytes_removed
(count) | The number of delete-file bytes removed by the compaction job.
Shown as byte |
aws.glue.number_of_delete_files_removed
(count) | The number of delete files removed by the compaction job.
Shown as file |
aws.glue.number_of_dpus_allocated_to_compaction_job
(gauge) | The number of DPUs allocated to a compaction job. |
aws.glue.number_of_dpus_allocated_to_orphan_file_deletion_job
(gauge) | The number of DPUs allocated to an orphan-file deletion job. |
aws.glue.number_of_dpus_allocated_to_retention_job
(gauge) | The number of DPUs allocated to a snapshot-retention job. |
aws.glue.number_of_files_compacted
(count) | The number of files compacted by the compaction job.
Shown as file |
aws.glue.number_of_manifest_files_deleted
(count) | The number of manifest files deleted by the snapshot-retention job.
Shown as file |
aws.glue.number_of_manifest_lists_deleted
(count) | The number of manifest lists deleted by the snapshot-retention job.
Shown as file |
aws.glue.number_of_orphan_files_deleted
(count) | The number of orphan files deleted by the orphan-file deletion job.
Shown as file |
aws.glue.resource_usage
(gauge) | The percentage of the applicable AWS Glue service quota currently in use.
Shown as percent |
