Cloud Cost Management

To use AWS Cloud Cost Management, you must have an AWS account with access to Cost and Usage Reports (CURs), and have the AWS integration installed in Datadog. To set up Cloud Cost Management in Datadog, you need to generate a Cost and Usage report.

Prerequisite: generate a Cost and Usage Report

Follow AWS instructions for Creating Cost and Usage Reports, and select the following content options for use with Datadog Cloud Cost Management:

• Include resource IDs
• Check the box for “Automatically refresh your Cost & Usage Report when charges are detected for previous months with closed bills.”

Select the following Delivery options:

• Time granularity: Hourly
• Report versioning: Create new report version
• Compression type: GZIP or Parquet
• Format: text/csv or Parquet

Configure the AWS integration

Navigate to Setup & Configuration and select your AWS management account from the dropdown menu, allowing Datadog to display tags associated with this account. If you have multiple similarly-named management accounts, view the tags associated with a selected account to ensure you have selected the specific account you want.

Note: Datadog recommends sending a Cost and Usage Report from an AWS management account for cost visibility into related member accounts. If you send a Cost and Usage report from an AWS member account, ensure that you have selected the following options in your management account’s preferences:

• Linked Account Access
• Linked Account Refunds and Credits
• Linked Account Discounts

This ensures complete cost accuracy by allowing periodic cost calculations against the AWS Cost Explorer.

Locate the Cost and Usage Report

If you have navigated away from the report that you created in the setup prerequisites section, follow AWS documentation to find and view your Cost and Usage Reports details.

To enable Datadog to locate the Cost and Usage Report, complete the fields with their corresponding details:

• Region: This is the region your bucket is located. For example, us-east-1.
• Bucket Name: This is the name of the s3 bucket that the CUR is saved to.
• Report Path Prefix: This is the folder name. If viewing Report path prefix from the AWS details page, this is the first section of the path. For example, if Report path prefix is displayed as cur-report-dir/cost-report, you would enter cur-report-dir.
• Report Name: This is the name you entered when you generated the report in the prerequisite section. If viewing the Report path prefix from the AWS details page, this is the second half of the path. For example, if Report path prefix is displayed as cur-report-dir/cost-report, you would enter cost-report.

Configure access to the Cost and Usage Report

Configure AWS to ensure Datadog has permissions to access the CUR and the s3 bucket it is stored in by creating a policy using the following JSON:

{
  "Version": "2012-10-17",
  "Statement": [
      {
          "Sid": "DDCloudCostReadBucket",
          "Effect": "Allow",
          "Action": [
              "s3:ListBucket"
          ],
          "Resource": "arn:aws:s3:::BUCKETNAME"
      },
      {
          "Sid": "DDCloudCostGetBill",
          "Effect": "Allow",
          "Action": [
              "s3:GetObject"
          ],
          "Resource": "arn:aws:s3:::BUCKETNAME/REPORT_PREFIX/REPORT_NAME/*"
      },
      {
          "Sid": "DDCloudCostCheckAccuracy",
          "Effect": "Allow",
          "Action": [
              "ce:Get*"
          ],
          "Resource": "*"
      },
      {
          "Sid": "DDCloudCostListCURs",
          "Action": [
              "cur:DescribeReportDefinitions"
          ],
          "Effect": "Allow",
          "Resource": "*"
      },
      {
          "Sid": "DDCloudCostListOrganizations",
          "Action": [
              "organizations:Describe*",
              "organizations:List*"
          ],
          "Effect": "Allow",
          "Resource": "*"
      }
  ]
}

Tip: Make note of the name you created for this policy for next steps.

Attach the policy to the Datadog integration role

Attach the new S3 policy to the Datadog integration role.

1. Navigate to Roles in the AWS IAM console.
2. Locate the role used by the Datadog integration. By default it is named DatadogIntegrationRole, but the name may vary if your organization has renamed it. Click the role name to open the role summary page.
3. Click Attach policies.
4. Enter the name of the S3 bucket policy created above.
5. Click Attach policy.

Note: Data can take up to 48 to 72 hours after setup to stabilize in Datadog.

Cost types

You can visualize your ingested data using the following cost types:

Tag enrichment

Datadog adds out-of-the-box tags to the ingested cost data to help you further break down and allocate your costs. These tags are derived from your Cost and Usage Report (CUR).

The following out-of-the-box tags are also available for filtering and grouping data:

Tag pipelines (beta)

You can use tag pipelines to create tag rules to help fix missing or incorrect tags on your Cloud bill, or to create new, inferred tags that align with business logic.

There are two types of rules supported: Create new tag, and Alias existing tag keys. You can keep your rules organized by leveraging rules-sets, which act as folders for your rules. The rules are executed in order (from top to bottom), to keep the execution order deterministic. You can organize rules and rulesets to ensure the order of execution matches your business logic.

Rule types

Create new tag - This allows you to create a new tag (key + value) based on the presence of existing tags. For example, you can create a rule to tag all resources that are part of team A, B, or C, and also run a specified application, with a new cost-center:webstore tag.

Alias existing tag keys - This allows you to use values from an existing tag, to map to a more standardized tag key. For example, if you’re looking to standardize across your organization to use the application tag key, but several teams have a variation of that tag (like app, web-app, or apps), you can alias apps to application. Each alias tag rule allows you to alias a maximum of 25 tag keys to a new tag.

The rule stops executing for each resource, once a first match is found. For example, if a resource already has a web-app tag, then the rule no longer attempts to identify an apps or service tag.

To use Azure Cloud Cost Management in Datadog, you must set up the Datadog Azure integration and set up amortized and actual exports. Additionally, Datadog must have permissions to read the exports from the container.

Note: If you are a US3 customer, you likely used the recommended Datadog Resource method through the Azure portal to set up the integration with Datadog. To support Cloud Cost Management, you need to create an App Registration.

Generate cost exports

1. Navigate to Exports under Azure portal’s Cost Management + Billing.
2. Select the export scope. Note: The scope must be subscription or resource group.
3. Once the scope is selected, click Add.

4. Select the following Export details:
   • Metric: Actual Cost (usage and purchases)
   • Export type: Daily export of month-to-date costs
   • File Partitioning: On

5. Choose a storage account, container, and directory for the exports. The billing exports must be stored in the subscription the export is for.
6. Select Create.

Repeat steps one to six for Metric: Amortized Cost (usage and purchases). Datadog recommends using the same storage container for both exports.

Provide Datadog access to your exports

1. In the Exports tab, click on the export’s Storage Account to navigate to it.
2. Click the Containers tab.
3. Choose the storage container your bills are in.
4. Select the Access Control (IAM) tab.
5. Click Add.
6. Choose Storage Blob Data Reader, then click Next.
7. Assign these permissions to one of the App Registrations you have connected with Datadog.
   • Click Select members, pick the name of the App Registration, and click Select.
   • Select review + assign.

If your exports are in different storage containers, repeat steps one to seven for the other storage container.

Configure Cost Management Reader access

1. Navigate to your subscriptions and click your subscription’s name.
2. Select the Access Control (IAM) tab.
3. Click Add.
4. Choose Cost Management Reader, then click Next.
5. Assign these permissions to the subscription.

This ensures complete cost accuracy by allowing periodic cost calculations against Azure Cost Management.