EMR EC2 instance profile is a IAM role assigned to every EC2 instance in an Amazon EMR cluster when the instance launches. Follow the Amazon guide to prepare this role based on your application’s need to interact with other AWS services. The following additional permissions may be required for Data Jobs Monitoring.
Save the following script to an S3 bucket that your EMR cluster can read. Take note of the path to this script.
#!/bin/bash
# Set required parameter DD_SITEDD_SITE=datadoghq.com# Set required parameter DD_API_KEY with Datadog API key.# The commands below assumes the API key is stored in AWS Secrets Manager, with the secret name as datadog/dd_api_key and the key as dd_api_key.# IMPORTANT: Modify if you choose to manage and retrieve your secret differently.SECRET_NAME=datadog/dd_api_key
DD_API_KEY=$(aws secretsmanager get-secret-value --secret-id $SECRET_NAME| jq -r .SecretString | jq -r '.["dd_api_key"]')# Optional parameters# Uncomment the following line to allow adding init script logs when reporting a failure back to Datadog. A failure is reported when the init script fails to start the Datadog Agent successfully.# export DD_DJM_ADD_LOGS_TO_FAILURE_REPORT=true# Download and run the latest init scriptDD_SITE=$DD_SITEDD_API_KEY=$DD_API_KEY bash -c "$(curl -L https://dd-data-jobs-monitoring-setup.s3.amazonaws.com/scripts/emr/emr_init_latest.sh)"||true
The script above sets the required parameters, and downloads and runs the latest init script for Data Jobs Monitoring in EMR. If you want to pin your script to a specific version, you can replace the file name in the URL with emr_init_1.4.0.sh to use the last stable version.
On the Create Cluster page, find the Identity and Access Management (IAM) roles section. For instance profile dropdown, select the IAM role you have granted permissions in Grant permissions to EMR EC2 instance profile.
