Azure Data Factory

개요

Azure Data Factory는 데이터 저장, 이동, 처리 서비스를 자동화된 파이프라인으로 구성할 수 있도록 도와주는 클라우드 데이터 통합 서비스입니다.

Datadog Azure 통합을 사용해 Data Factory 메트릭을 수집할 수 있습니다.

설정

설치

If you haven’t already, set up the Microsoft Azure integration. No additional installation steps are required.

수집한 데이터

메트릭

azure.datafactory_factories.activity_cancelled_runs
(count)		Cancelled activity runs metrics.
azure.datafactory_factories.activity_failed_runs
(count)		Failed activity runs metrics.
azure.datafactory_factories.activity_succeeded_runs
(count)		Succeeded activity runs metrics.
azure.datafactory_factories.airflow_integration_runtime_celery_task_timeout_error
(count)		Airflow integration runtime celery task timeout error.
azure.datafactory_factories.airflow_integration_runtime_collect_db_dags
(gauge)		Airflow integration runtime collect DB dags.
Shown as millisecond
azure.datafactory_factories.airflow_integration_runtime_cpu_percentage
(gauge)		Airflow integration runtime cpu percentage.
Shown as percent
azure.datafactory_factories.airflow_integration_runtime_cpu_usage
(gauge)		Airflow integration runtime memory usage.
azure.datafactory_factories.airflow_integration_runtime_dag_bag_size
(count)		Airflow integration runtime DAG bag size.
azure.datafactory_factories.airflow_integration_runtime_dag_callback_exceptions
(count)		Airflow integration runtime DAG callback exceptions.
azure.datafactory_factories.airflow_integration_runtime_dag_file_refresh_error
(count)		Airflow integration runtime DAG file refresh error.
azure.datafactory_factories.airflow_integration_runtime_dag_processing_import_errors
(count)		Airflow integration runtime DAG processing import errors.
azure.datafactory_factories.airflow_integration_runtime_dag_processing_last_duration
(gauge)		Airflow integration runtime DAG processing last duration.
Shown as millisecond
azure.datafactory_factories.airflow_integration_runtime_dag_processing_last_run_seconds_ago
(gauge)		Airflow integration runtime DAG processing last run seconds ago.
Shown as second
azure.datafactory_factories.airflow_integration_runtime_dag_processing_manager_stalls
(count)		Airflow integration runtime DAG processingmanager stalls.
azure.datafactory_factories.airflow_integration_runtime_dag_processing_processes
(count)		Airflow integration runtime DAG processing processes.
azure.datafactory_factories.airflow_integration_runtime_dag_processing_processor_timeouts
(gauge)		Airflow integration runtime DAG processing processor timeouts.
Shown as second
azure.datafactory_factories.airflow_integration_runtime_dag_processing_total_parse_time
(gauge)		Airflow integration runtime DAG processing total parse time.
Shown as second
azure.datafactory_factories.airflow_integration_runtime_dag_run_dependency_check
(gauge)		Airflow integration runtime DAG run dependency check.
Shown as millisecond
azure.datafactory_factories.airflow_integration_runtime_dag_run_duration_failed
(gauge)		Airflow integration runtime DAG run duration failed.
Shown as millisecond
azure.datafactory_factories.airflow_integration_runtime_dag_run_duration_success
(gauge)		Airflow integration runtime DAG run duration success.
Shown as millisecond
azure.datafactory_factories.airflow_integration_runtime_dag_run_first_task_scheduling_delay
(gauge)		Airflow integration runtime DAG run first task scheduling delay.
Shown as millisecond
azure.datafactory_factories.airflow_integration_runtime_dag_run_schedule_delay
(gauge)		Airflow integration runtime DAG run schedule delay.
Shown as millisecond
azure.datafactory_factories.airflow_integration_runtime_executor_open_slots
(count)		Airflow integration runtime executor open slots.
azure.datafactory_factories.airflow_integration_runtime_executor_queued_tasks
(count)		Airflow integration runtime executor queued tasks.
azure.datafactory_factories.airflow_integration_runtime_executor_running_tasks
(count)		Airflow integration runtime executor running tasks.
azure.datafactory_factories.airflow_integration_runtime_job_end
(count)		Airflow integration runtime job end.
azure.datafactory_factories.airflow_integration_runtime_job_heartbeat_failure
(count)		Airflow integration runtime heartbeat failure.
azure.datafactory_factories.airflow_integration_runtime_job_start
(count)		Airflow integration runtime job start.
azure.datafactory_factories.airflow_integration_runtime_memory_percentage
(gauge)		Airflow integration runtime memory percentage.
Shown as percent
azure.datafactory_factories.airflow_integration_runtime_operator_failures
(count)		Airflow integration runtime operator failures.
azure.datafactory_factories.airflow_integration_runtime_operator_successes
(count)		Airflow integration runtime operator successes.
azure.datafactory_factories.airflow_integration_runtime_pool_open_slots
(count)		Airflow integration runtime pool open slots.
azure.datafactory_factories.airflow_integration_runtime_pool_queued_slots
(count)		Airflow integration runtime pool queued slots.
azure.datafactory_factories.airflow_integration_runtime_pool_running_slots
(count)		Airflow integration runtime pool running slots.
azure.datafactory_factories.airflow_integration_runtime_pool_starving_tasks
(count)		Airflow integration runtime pool starving tasks.
azure.datafactory_factories.airflow_integration_runtime_scheduler_critical_section_busy
(count)		Airflow integration runtime scheduler critical section busy.
azure.datafactory_factories.airflow_integration_runtime_scheduler_critical_section_duration
(gauge)		Airflow integration runtime scheduler critical section duration.
Shown as millisecond
azure.datafactory_factories.airflow_integration_runtime_scheduler_failed_sla_email_attempts
(count)		Airflow integration runtime scheduler failed SLA email attempts.
azure.datafactory_factories.airflow_integration_runtime_scheduler_heartbeat
(count)		Airflow integration runtime scheduler heartbeats.
azure.datafactory_factories.airflow_integration_runtime_scheduler_orphaned_tasks_adopted
(count)		Airflow integration runtime scheduler orphaned tasks adopted.
azure.datafactory_factories.airflow_integration_runtime_scheduler_orphaned_tasks_cleared
(count)		Airflow integration runtime scheduler orphaned tasks cleared.
azure.datafactory_factories.airflow_integration_runtime_scheduler_tasks_executable
(count)		Airflow integration runtime scheduler tasks executable.
azure.datafactory_factories.airflow_integration_runtime_scheduler_tasks_killed_externally
(count)		Airflow integration runtime scheduler tasks killed externally.
azure.datafactory_factories.airflow_integration_runtime_scheduler_tasks_running
(count)		Airflow integration runtime scheduler tasks running.
azure.datafactory_factories.airflow_integration_runtime_scheduler_tasks_starving
(count)		Airflow integration runtime scheduler tasks starving.
azure.datafactory_factories.airflow_integration_runtime_started_task_instances
(count)		Airflow integration runtime started task instances.
azure.datafactory_factories.airflow_integration_runtime_task_instance_created_using_operator
(count)		Airflow integration runtime task instance created using operator.
azure.datafactory_factories.airflow_integration_runtime_task_instance_duration
(gauge)		Airflow integration runtime task instance duration.
Shown as millisecond
azure.datafactory_factories.airflow_integration_runtime_task_instance_failures
(count)		Airflow integration runtime task instance failures.
azure.datafactory_factories.airflow_integration_runtime_task_instance_finished
(count)		Airflow integration runtime task instance finished.
azure.datafactory_factories.airflow_integration_runtime_task_instance_previously_succeeded
(count)		Airflow integration runtime task instance previously succeeded.
azure.datafactory_factories.airflow_integration_runtime_task_instance_successes
(count)		Airflow integration runtime task instance successes.
azure.datafactory_factories.airflow_integration_runtime_task_removed_from_dag
(count)		Airflow integration runtime task removed from DAG.
azure.datafactory_factories.airflow_integration_runtime_task_restored_to_dag
(count)		Airflow integration runtime task restored to DAG.
azure.datafactory_factories.airflow_integration_runtime_triggers_blocked_main_thread
(count)		Airflow integration runtime triggers blocked main thread.
azure.datafactory_factories.airflow_integration_runtime_triggers_failed
(count)		Airflow integration runtime triggers failed.
azure.datafactory_factories.airflow_integration_runtime_triggers_running
(count)		Airflow integration runtime triggers running.
azure.datafactory_factories.airflow_integration_runtime_triggers_succeeded
(count)		Airflow integration runtime triggers succeeded.
azure.datafactory_factories.airflow_integration_runtime_zombies_killed
(count)		Airflow integration runtime zombie tasks killed.
azure.datafactory_factories.factory_size_in_gb_units
(gauge)		Total factory size (GB unit).
azure.datafactory_factories.integration_runtime_available_memory
(gauge)		Integration runtime available memory.
Shown as byte
azure.datafactory_factories.integration_runtime_available_node_number
(gauge)		Integration runtime available node count.
azure.datafactory_factories.integration_runtime_average_task_pickup_delay
(gauge)		Integration runtime queue duration.
Shown as second
azure.datafactory_factories.integration_runtime_cpu_percentage
(gauge)		Integration runtime CPU utilization.
Shown as percent
azure.datafactory_factories.integration_runtime_queue_length
(gauge)		Integration runtime queue length.
azure.datafactory_factories.max_allowed_factory_size_in_gb_units
(gauge)		Maximum allowed factory size (GB unit).
azure.datafactory_factories.max_allowed_resource_count
(gauge)		Maximum allowed entities count.
azure.datafactory_factories.pipeline_cancelled_runs
(count)		Cancelled pipeline runs metrics.
azure.datafactory_factories.pipeline_elapsed_time_runs
(count)		Elapsed time pipeline runs metrics.
azure.datafactory_factories.pipeline_failed_runs
(count)		Failed pipeline runs metrics.
azure.datafactory_factories.pipeline_succeeded_runs
(count)		Succeeded pipeline runs metrics.
azure.datafactory_factories.resource_count
(gauge)		Total entities count.
azure.datafactory_factories.ssis_integration_runtime_start_cancel
(count)		Cancelled SSIS integration runtime start metrics.
azure.datafactory_factories.ssis_integration_runtime_start_failed
(count)		Failed SSIS integration runtime start metrics.
azure.datafactory_factories.ssis_integration_runtime_start_succeeded
(count)		Succeeded SSIS integration runtime start metrics.
azure.datafactory_factories.ssis_integration_runtime_stop_stuck
(count)		Stuck SSIS integration runtime stop metrics.
azure.datafactory_factories.ssis_integration_runtime_stop_succeeded
(count)		Succeeded SSIS integration runtime stop metrics.
azure.datafactory_factories.ssis_package_execution_cancel
(count)		Cancelled SSIS package execution metrics.
azure.datafactory_factories.ssis_package_execution_failed
(count)		Failed SSIS package execution metrics.
azure.datafactory_factories.ssis_package_execution_succeeded
(count)		Succeeded SSIS package execution metrics.
azure.datafactory_factories.trigger_cancelled_runs
(count)		Cancelled trigger runs metrics.
azure.datafactory_factories.trigger_failed_runs
(count)		Failed trigger runs metrics.
azure.datafactory_factories.trigger_succeeded_runs
(count)		Succeeded trigger runs metrics.
azure.datafactory_factories.count
(gauge)		Count of DataFactory factories.

이벤트

Azure Data Factory 통합에는 이벤트가 포함되어 있지 않습니다.

서비스 점검

Azure Data Factory 통합에는 서비스 점검이 포함되어 있지 않습니다.

트러블슈팅

도움이 필요하신가요? Datadog 지원팀에 문의하세요.