Overview

Follow this guide to deploy the Datadog Distribution of OpenTelemetry (DDOT) Collector alongside the Datadog Agent on Amazon ECS Fargate. Because ECS Fargate does not support host-based deployments, the Datadog Agent runs as a sidecar container in the same ECS task as your application.

Requirements

To complete this guide, you need the following:

Datadog account:

1. Create a Datadog account if you don’t have one.
2. Find or create your Datadog API key.

AWS:

• An AWS account with access to Amazon ECS Fargate.
• The AWS CLI configured with permissions to create and manage ECS task definitions and services.

Network: When using the Datadog SDK with OpenTelemetry API support, telemetry is routed to different components depending on the signal source. Ensure the following ports are accessible on your Datadog Agent or Collector:

Install the Datadog Agent with OpenTelemetry Collector

Create an ECS task definition

On ECS Fargate, the Datadog Agent and your application run as containers within the same ECS task. Create an ECS task definition that includes both containers.

Use the following task definition as a starting point:

{
    "family": "<TASK_FAMILY>",
    "containerDefinitions": [
        {
            "name": "datadog-agent",
            "image": "public.ecr.aws/datadog/agent:latest-full",
            "essential": true,
            "cpu": 0,
            "environment": [
                {
                    "name": "ECS_FARGATE",
                    "value": "true"
                },
                {
                    "name": "DD_LOGS_ENABLED",
                    "value": "true"
                },
                {
                    "name": "DD_OTELCOLLECTOR_ENABLED",
                    "value": "true"
                },
                {
                    "name": "DD_SITE",
                    "value": "<DATADOG_SITE>"
                },
                {
                    "name": "DD_LOGS_CONFIG_CONTAINER_COLLECT_ALL",
                    "value": "true"
                }
            ],
            "secrets": [
                {
                    "name": "DD_API_KEY",
                    "valueFrom": "<DD_API_KEY_SECRET_ARN>"
                }
            ],
            "logConfiguration": {
                "logDriver": "awslogs",
                "options": {
                    "awslogs-group": "/ecs/<TASK_FAMILY>",
                    "awslogs-create-group": "true",
                    "awslogs-region": "<AWS_REGION>",
                    "awslogs-stream-prefix": "ecs"
                }
            },
            "mountPoints": [],
            "portMappings": [],
            "volumesFrom": []
        },
        {
            "name": "<APP_CONTAINER_NAME>",
            "image": "<APP_IMAGE>",
            "essential": true,
            "cpu": 0,
            "environment": [
                {
                    "name": "OTEL_SERVICE_NAME",
                    "value": "<SERVICE_NAME>"
                },
                {
                    "name": "OTEL_RESOURCE_ATTRIBUTES",
                    "value": "service.version=<VERSION>,deployment.environment.name=<ENV>"
                }
            ],
            "logConfiguration": {
                "logDriver": "awslogs",
                "options": {
                    "awslogs-group": "/ecs/<TASK_FAMILY>",
                    "awslogs-create-group": "true",
                    "awslogs-region": "<AWS_REGION>",
                    "awslogs-stream-prefix": "ecs"
                }
            },
            "mountPoints": [],
            "portMappings": [],
            "volumesFrom": []
        }
    ],
    "executionRoleArn": "<EXECUTION_ROLE_ARN>",
    "networkMode": "awsvpc",
    "volumes": [],
    "requiresCompatibilities": [
        "FARGATE"
    ],
    "cpu": "256",
    "memory": "512"
}

Replace the following placeholders:

• <TASK_FAMILY>: A name for your ECS task definition family.
• <DD_API_KEY_SECRET_ARN>: The ARN of the AWS Secrets Manager secret or SSM Parameter Store parameter that contains your Datadog API key. The task execution role must have secretsmanager:GetSecretValue (Secrets Manager) or ssm:GetParameters (SSM) permission to retrieve the secret.
• <DATADOG_SITE>: Your Datadog site. Your site is . (Ensure the correct DATADOG SITE is selected on the right.)
• <AWS_REGION>: The AWS region where your ECS tasks run (for example, us-east-1).
• <APP_CONTAINER_NAME>: The name of your application container.
• <APP_IMAGE>: Your application container image.
• <SERVICE_NAME>: The name of your service.
• <VERSION>: The version of your service.
• <ENV>: Your deployment environment (for example, production).
• <EXECUTION_ROLE_ARN>: The ARN of your ECS task execution IAM role.

Configure ECS resource detection

The DDOT Collector’s infraattributes processor enriches your telemetry with ECS infrastructure tags when the aws.ecs.task.arn resource attribute is present. To provide this attribute, add the ECS resource detector to your OpenTelemetry SDK.

The ECS resource detector automatically populates the following attributes:

• aws.ecs.task.arn
• aws.ecs.launchtype (set to fargate)
• cloud.provider

Refer to your language’s OpenTelemetry SDK documentation to add the ECS resource detector. For example:

• Go: go.opentelemetry.io/contrib/detectors/aws/ecs
• All languages: See the ECS resource detector documentation in the OpenTelemetry Collector contrib repository.

Send your telemetry to Datadog

To send your telemetry data to Datadog:

1. Instrument your application
2. Configure the application
3. Correlate observability data
4. Run your application

Instrument the application

Instrument your application using the OpenTelemetry API.

@WithSpan(kind = SpanKind.CLIENT)
public String getDate() {
    Span span = Span.current();
    span.setAttribute("peer.service", "random-date-service");
    ...
}

Configure the application

Because the DDOT Collector runs as a sidecar container in the same ECS task, your application sends OTLP data to localhost. Add the following environment variables to your application container in the task definition:

{
    "name": "OTEL_EXPORTER_OTLP_ENDPOINT",
    "value": "http://localhost:4317"
},
{
    "name": "OTEL_EXPORTER_OTLP_PROTOCOL",
    "value": "grpc"
}

Correlate observability data

Unified service tagging ties observability data together in Datadog so you can navigate across metrics, traces, and logs with consistent tags.

The OTEL_SERVICE_NAME and OTEL_RESOURCE_ATTRIBUTES environment variables in the task definition above configure unified service tagging.

Run the application

1. Register the task definition with AWS:

   Copy

   aws ecs register-task-definition --cli-input-json file://task-definition.json

2. If you don’t have an ECS cluster, create one:

   Copy

   aws ecs create-cluster --cluster-name <CLUSTER_NAME>

3. Create an ECS service to run the task:

   Copy

   aws ecs create-service \
     --cluster <CLUSTER_NAME> \
     --service-name <SERVICE_NAME> \
     --task-definition <TASK_FAMILY> \
     --desired-count 1 \
     --launch-type FARGATE \
     --network-configuration "awsvpcConfiguration={subnets=[<SUBNET_IDS>],securityGroups=[<SECURITY_GROUP_IDS>],assignPublicIp=ENABLED}"

   Replace <CLUSTER_NAME> with your ECS cluster name, <SERVICE_NAME> with a name for your ECS service, <SUBNET_IDS> with your subnet IDs (comma-separated), and <SECURITY_GROUP_IDS> with your security group IDs.

   If you are updating an existing service to a new task definition revision, run:

   Copy

   aws ecs update-service \
     --cluster <CLUSTER_NAME> \
     --service <SERVICE_NAME> \
     --task-definition <TASK_FAMILY>

After the task starts, unified service tagging is fully enabled for your metrics, traces, and logs.

Validate the deployment

1. Verify that your ECS task is running and both containers are in RUNNING state:

   aws ecs describe-tasks \
     --cluster <CLUSTER_NAME> \
     --tasks <TASK_ID> \
     --query "tasks[0].containers[*].{name:name,status:lastStatus}" \
     --output table
   

Use Datadog to explore the observability data for your application.

Logs

View logs to monitor and troubleshoot application and system operations.

Traces

View traces and spans to observe the status and performance of requests processed by your application, with infrastructure metrics correlated in the same trace.

Runtime metrics

Monitor your runtime (JVM) metrics for your applications.

Collector health metrics

View metrics from the DDOT Collector to monitor the Collector health.

Further reading

Documentation, liens et articles supplémentaires utiles:

Use Custom OpenTelemetry Components with Datadog AgentDOCUMENTATION