---
title: Jobs Monitoring for AWS Glue
description: 'Enable Data Observability: Jobs Monitoring for AWS Glue jobs with Datadog.'
breadcrumbs: >-
  Docs > Data Observability Overview > Data Observability: Jobs Monitoring >
  Jobs Monitoring for AWS Glue
---

# Jobs Monitoring for AWS Glue

{% callout %}
# Important note for users on the following Datadog sites: app.ddog-gov.com

{% alert level="danger" %}
This product is not supported for your selected [Datadog site](https://docs.datadoghq.com/getting_started/site). ().
{% /alert %}

{% /callout %}

## Overview{% #overview %}

[Data Observability: Jobs Monitoring](https://app.datadoghq.com/data-jobs/) gives visibility into the performance and reliability of your AWS Glue jobs.

## Prerequisites{% #prerequisites %}

Before you begin, make sure you have:

- An AWS account with Glue jobs you want to monitor.
- The [Datadog AWS integration](https://docs.datadoghq.com/integrations/amazon-web-services/) configured for the account.
- IAM permissions to modify the Datadog role's policies.

## Configure the AWS account{% #configure-the-aws-account %}

1. Navigate to [**Datadog Data Observability** > **Settings**](https://app.datadoghq.com/datasets/settings/integrations).

1. Click **Configure** next to AWS Glue.

   {% image
      source="https://datadog-docs.imgix.net/images/data_observability/aws_glue/settings-configure-button.89c6340c4a9809eaf5de6e335b1be3f7.png?auto=format"
      alt="AWS Glue configuration option in the Data Observability Settings page" /%}

1. Select an existing AWS account that is already connected to Datadog, or add a new one. For help adding a new account, see the [AWS Integration documentation](https://docs.datadoghq.com/integrations/amazon-web-services/).

   {% image
      source="https://datadog-docs.imgix.net/images/data_observability/aws_glue/account-selection.075b327ec679d7d21d2793c447293894.png?auto=format"
      alt="AWS account selection dropdown in the configuration flow" /%}

## Add required IAM permissions{% #add-required-iam-permissions %}

The Data Observability crawler requires additional permissions to monitor Glue jobs. Attach the following policy to the Datadog IAM role configured for your AWS integration:

```json
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": [
        "glue:GetCatalog",
        "glue:GetDatabase",
        "glue:GetDatabases",
        "glue:GetJobRun",
        "glue:GetJobRuns",
        "glue:GetJob",
        "glue:GetJobs",
        "glue:GetTable",
        "glue:GetTables",
        "glue:ListJobs",
        "s3:ListBucket",
        "kms:Decrypt",
        "lakeformation:GetDataAccess"
      ],
      "Resource": ["*"]
    },
    {
      "Sid": "AllowIcebergMetadataOnly",
      "Effect": "Allow",
      "Action": [
        "s3:GetObject",
        "s3:GetObjectVersion"
      ],
      "Resource": [
        "arn:aws:s3:::*/metadata/*"
      ]
    }
  ]
}
```

Some of these permissions are related to monitoring Iceberg tables in Glue. For more details on dataset-related IAM permissions, see the [AWS Glue Data Quality Monitoring documentation](https://docs.datadoghq.com/data_observability/quality_monitoring/data_lakes/aws_glue/?tab=specificdatabases#optional-restrict-access-to-specific-databases-and-tables).

## Configure the crawler{% #configure-the-crawler %}

1. Select the AWS regions where your Glue jobs are located.

1. Enable the **Job Monitoring** toggle.

   {% image
      source="https://datadog-docs.imgix.net/images/data_observability/aws_glue/crawler-configuration.a6bc7496b9cab08d39f11048c6bb4e8a.png?auto=format"
      alt="Crawler configuration showing region selection and sync frequency options" /%}

1. Click **Save**.

## (Optional) Configure Glue jobs logs{% #optional-configure-glue-jobs-logs %}

1. Follow [these steps](https://docs.datadoghq.com/logs/guide/send-aws-services-logs-with-the-datadog-lambda-function/?tab=awsconsole) to send AWS logs from CloudWatch to Datadog.

1. Manually configure triggers in **AWS CloudWatch** to capture AWS Glue logs. By default, Glue logs are stored in the following log groups:

   - `/aws-glue/jobs/error`
   - `/aws-glue/jobs/output`
   - `/aws-glue/jobs/logs-v2`

1. **Note**: After logs are ingested into Datadog, the CloudWatch log group name maps to the `host` attribute in Datadog Logs.

1. Create a Log Index that includes logs where the `host` attribute matches:

   - `/aws-glue/jobs/error`
   - `/aws-glue/jobs/output`
   - `/aws-glue/jobs/logs-v2`

This helps ensure the logs are searchable and available under the **Glue** tab in **Data Observability: Jobs Monitoring**.

## (Optional) Configure Glue metrics{% #optional-configure-glue-metrics %}

Enable the [Glue Integration](https://docs.datadoghq.com/integrations/amazon-glue/) tile for Glue metrics collection. Metrics should be available under the **Glue** job tab in **Data Observability: Jobs Monitoring**.

## Next steps{% #next-steps %}

The crawler runs every few minutes. In Datadog, view the [Data Observability: Jobs Monitoring](https://app.datadoghq.com/data-jobs/) page to see a list of your Glue job runs after setup.

## Further reading{% #further-reading %}

- [Data Observability: Jobs Monitoring](https://docs.datadoghq.com/data_jobs)
- [AWS Integration](https://docs.datadoghq.com/integrations/amazon-web-services/)