Container Cost Allocation

Overview

Datadog Cloud Cost Management (CCM) automatically allocates costs of Kubernetes and AWS ECS clusters to individual pods and tasks running in those clusters. Use cost metrics enriched with tags from pods, nodes, containers, and tasks to visualize container workload cost in the context of your entire cloud bill.

For Kubernetes clusters, CCM allocates costs of AWS and Google host instances, and non-local AWS EBS volumes. For AWS ECS clusters, CCM allocates costs of AWS EC2 instances.

Prerequisites

1. Set up and configure the AWS or Google Cloud Cost integration.
2. For Kubernetes support, install the Datadog Agent in a Kubernetes environment.
   • Ensure that you enable the Orchestrator Explorer in your Agent configuration.
   • Container cost allocation requires Agent version >= 7.27.0 and Cluster Agent version >= 1.11.0.
   • AWS Persistent Volume allocation requires Agent version >= 7.46.0
3. For AWS ECS support, set up Datadog Container Monitoring in ECS tasks.
   • Optionally, enable AWS Split Cost Allocation for usage-based ECS allocation.

Cost allocation

Cost allocation divides host compute and AWS EBS volume costs from the cloud provider into associated individual tasks or pods. These divided costs are then enriched with tags from nodes, pods, tasks, and volumes (AWS only). This lets you break down costs by any associated dimensions.

Kubernetes

Compute

For Kubernetes compute allocation, a Kubernetes node is joined with its associated host instance costs. The node’s cluster name and all node tags are added to the entire compute cost for the node. This allows you to associate cluster-level dimensions with the cost of the instance, without considering the pods scheduled to the node.

Next, Datadog looks at all of the pods running on that node for the day. The cost of the node is allocated to the pod based on the resources it has used and the length of time it ran. This calculated cost is enriched with all of the pod’s tags.

Note: Only tags from pods and nodes are added to cost metrics. To include labels, enable labels as tags for nodes and pods.

Persistent volume storage (AWS only)

For Kubernetes Persistent Volume storage allocation, Persistent Volumes (PV), Persistent Volume Claims (PVC), nodes, and pods are joined with their associated EBS volume costs. All associated PV, PVC, node, and pod tags are added to the EBS volume cost line items.

Next, Datadog looks at all of the pods that claimed the volume on that day. The cost of the volume is allocated to a pod based on the resources it used and the length of time it ran. These resources include the provisioned capacity for storage, IOPS, and throughput. This allocated cost is enriched with all of the pod’s tags.

Agentless Kubernetes costs (Google Kubernetes Engine only)

To view the costs of GKE clusters without enabling Datadog infrastructure monitoring, use GKE cost allocation. Enable GKE cost allocation on unmonitored GKE clusters to access this feature set.

Limitations and differences from the Datadog agent

• There is no support for tracking workload idle costs
• The cost of individual pods are not tracked, only the aggregated cost of a workload and the namespace. No pod_name tag.
• GKE enriches data using only pod labels and ignores any Datadog tags you add
• The full list of limitations can be found on the GKE docs

GKE cost allocation can be enabled by following the instructions in the GKE docs

AWS ECS on EC2

For ECS allocation, Datadog determines which tasks ran on each EC2 instance used for ECS. If you enable AWS Split Cost Allocation, the metrics allocate ECS costs by usage instead of reservation, providing more granular detail.

Based on resources the task has used, Datadog assigns the appropriate portion of the instance’s compute cost to that task. The calculated cost is enriched with all of the task’s tags and all of the container tags (except container names) running in the task.

AWS ECS on Fargate

ECS tasks that run on Fargate are already fully allocated in the CUR. CCM enriches that data by adding out-of-the-box tags and container tags to the AWS Fargate cost.

Everything else

The information on this page pertains to compute and AWS EBS costs for host instances running Kubernetes pods or ECS tasks. All other costs are given the same value and tags as the source metric, aws.cost.amortized or gcp.cost.amortized.

Understanding spend

Using the allocated_spend_type tag, you can visualize the spend category associated with your cost at the Kubernetes node, AWS ECS host, AWS EBS volume, or cluster level. Costs are allocated into different spend types:

Compute

The cost of a host instance is split into two components: 60% for the CPU and 40% for the memory. Each component is allocated to individual workloads based on their resource reservations and usage.

• Usage: Cost of memory and CPU used by workloads, based on the average usage on that day.
• Workload idle: Cost of memory and CPU that is being reserved and allocated but not used. This is the difference between the total resources requested and the average usage.
• Cluster idle: Cost of memory and CPU not reserved by workloads in a cluster. This is the difference between the total cost of the resources and what is allocated to workloads.

Persistent volumes (AWS only)

The cost of an EBS volume has three components: IOPS, throughput, and storage. Each is allocated according to a pod’s usage when the volume is mounted.

• Usage: Cost of provisioned IOPS, throughput, or storage being used by workloads. Storage cost is based on the maximum amount of volume storage used that day. IOPS and throughput costs are based on the average amount used that day.
• Workload idle: Cost of provisioned IOPS, throughput, or storage not being used by workloads. Storage cost is based on the maximum amount of storage used that day. IOPS and throughput costs are based on the average amount used that day. Note: This tag is only available if you have enabled Resource Collection in your AWS Integration. To prevent being charged for Cloud Security Posture Management, ensure that during the Resource Collection setup, the Cloud Security Posture Management box is unchecked.
• Cluster idle: Cost of provisioned IOPS, throughput, or storage for volumes not claimed by any pods that day.

Note: Persistent volume allocation is only supported in Kubernetes clusters, and is only available for pods that are part of a Kubernetes StatefulSet.

Cost metrics

When the prerequisites are met, new cost metrics automatically appear.

These new cost metrics include all of your cloud costs. This allows you to continue visualizing all of your cloud costs at one time, with added visibility into the costs of Kubernetes pods and AWS ECS tasks.

For example, say you have the tag team on a storage bucket, a cloud provider managed database, and Kubernetes pods. You can use these new metrics to group costs by team, which includes the costs for all three.

Tags

Datadog consolidates and applies additional tags from various sources to cost metrics. Conflicts are resolved by favoring higher-specificity tags such as pod tags over lower-specificity tags such as host tags. For example, a Kubernetes pod tagged service:datadog-agent running on a node tagged service:aws-node results in a final tag service:datadog-agent.

Containers

Kubernetes

In addition to Kubernetes pod and Kubernetes node tags, the following non exhaustive list of out-of-the-box tags are applied to cost metrics:

AWS ECS

In addition to ECS task tags, the following out-of-the-box tags are applied to cost metrics. Note: most tags from ECS containers (excluding container_name) are applied.

Persistent volumes (AWS only)

In addition to Kubernetes pod and Kubernetes node tags, the following out-of-the-box tags are applied to cost metrics.

Further reading

Additional helpful documentation, links, and articles:

Cloud Cost ManagementDOCUMENTATION