- 필수 기능
- 앱 내
- 서비스 관리
- 인프라스트럭처
- 애플리케이션 성능
- 디지털 경험
- 소프트웨어 제공
- 보안
- 로그 관리
- 관리
- 인프라스트럭처
- ci
- containers
- csm
- ndm
- otel_guides
- overview
- slos
- synthetics
- tests
- 워크플로
Data Jobs Monitoring gives visibility into the performance and reliability of Apache Spark applications on Amazon EMR.
Amazon EMR Release 6.6.0 or later is required.
Follow these steps to enable Data Jobs Monitoring for Amazon EMR.
dd_api_key
.datadog/dd_api_key
. Then, click Next.When you create a new EMR cluster in the Amazon EMR console, add a bootstrap action on the Create Cluster page:
Save the following script to an S3 bucket that your EMR cluster can read. Take note of the path to this script.
#!/bin/bash
# Set required parameter DD_SITE
DD_SITE=
# Set required parameter DD_API_KEY with Datadog API key.
# The commands below assumes the API key is stored in AWS Secrets Manager, with the secret name as datadog/dd_api_key and the key as dd_api_key.
# IMPORTANT: Modify if you choose to manage and retrieve your secret differently.
SECRET_NAME=datadog/dd_api_key
DD_API_KEY=$(aws secretsmanager get-secret-value --secret-id $SECRET_NAME | jq -r .SecretString | jq -r '.["dd_api_key"]')
# Optional parameters
# Uncomment the following line to allow adding init script logs when reporting a failure back to Datadog. A failure is reported when the init script fails to start the Datadog Agent successfully.
# export DD_DJM_ADD_LOGS_TO_FAILURE_REPORT=true
# Download and run the latest init script
DD_SITE=$DD_SITE DD_API_KEY=$DD_API_KEY bash -c "$(curl -L https://dd-data-jobs-monitoring-setup.s3.amazonaws.com/scripts/emr/emr_init_latest.sh)" || true
The script above sets the required parameters, downloads and runs the latest init script for Data Jobs Monitoring in EMR. If you want to pin your script to a specific version, you can replace the file name in the URL with emr_init_1.2.0.sh
to use the last stable version.
On the Create Cluster page, find the Bootstrap actions section. Click Add to bring up the Add bootstrap action dialog.
datadog_agent
.When your cluster is created, this bootstrap action installs the Datadog Agent and downloads the Java tracer on each node of the cluster.
In your Amazon EMR console, open the summary page for your newly created cluster. Take note of your cluster’s IAM role for instance profile.
Alternatively, you can also find this value by running:
aws emr describe-cluster --cluster-id <YOUR_CLUSTER_ID>
Look for Ec2InstanceAttributes.IamInstanceProfile
in the output.
In your AWS IAM console, click on Access management > Roles in the left navigation bar.
Click on the instance profile you saw in the previous step.
On the next page, under the Permissions tab, find the Permissions policies section. Click on Add permissions > Create inline policy.
On the Specify permissions page, find the Select a service section. Under Service, select Secrets Manager.
GetSecretValue
. This is a Read action.On the next page, give your policy a name. Then, click Create policy.
Tagging enables you to better filter, aggregate, and compare your telemetry in Datadog. You can configure tags by passing -Ddd.service
, -Ddd.env
, -Ddd.version
, and -Ddd.tags
options to your Spark driver and executor extraJavaOptions
properties.
In Datadog, each job’s name corresponds to the value you set for -Ddd.service
.
spark-submit \
--conf spark.driver.extraJavaOptions="-Ddd.service=<JOB_NAME> -Ddd.env=<ENV> -Ddd.version=<VERSION> -Ddd.tags=<KEY_1>:<VALUE_1>,<KEY_2:VALUE_2>" \
--conf spark.executor.extraJavaOptions="-Ddd.service=<JOB_NAME> -Ddd.env=<ENV> -Ddd.version=<VERSION> -Ddd.tags=<KEY_1>:<VALUE_1>,<KEY_2:VALUE_2>" \
application.jar
In Datadog, view the Data Jobs Monitoring page to see a list of all your data processing jobs.
You can set tags on Spark spans at runtime. These tags are applied only to spans that start after the tag is added.
// Add tag for all next Spark computations
sparkContext.setLocalProperty("spark.datadog.tags.key", "value")
spark.read.parquet(...)
To remove a runtime tag:
// Remove tag for all next Spark computations
sparkContext.setLocalProperty("spark.datadog.tags.key", null)
Additional helpful documentation, links, and articles: