Beta - check Databricks cluster Azure attribute best practices

Ce produit n'est pas pris en charge par le site Datadog que vous avez sélectionné. ().
Cette page n'est pas encore disponible en français, sa traduction est en cours.
Si vous avez des questions ou des retours sur notre projet de traduction actuel, n'hésitez pas à nous contacter.

Metadata

Id: 38028698-e663-4ef7-aa92-773fef0ca86f

Cloud Provider: Databricks

Platform: Terraform

Severity: Low

Category: Best Practices

Learn More

Description

One or more Azure attribute best practices are not followed for this Databricks cluster. The rule flags clusters when:

  • azure_attributes.availability is set to SPOT or SPOT_AZURE,
  • azure_attributes.first_on_demand is 0, or
  • azure_attributes.first_on_demand is missing.

These settings must ensure the use of at least one on-demand instance and avoid exclusive reliance on spot VMs.

Compliant Code Examples

resource "databricks_cluster" "negative" {
  cluster_name            = "Shared Autoscaling"
  spark_version           = data.databricks_spark_version.latest.id
  node_type_id            = data.databricks_node_type.smallest.id
  autotermination_minutes = 20
  autoscale {
    min_workers = 1
    max_workers = 50
  }
  azure_attributes {
    availability           = "SPOT_WITH_FALLBACK_AZURE"
    first_on_demand        = 1
    spot_bid_price_percent = 100
  }
}

Non-Compliant Code Examples

resource "databricks_cluster" "positive2" {
  cluster_name            = "data"
  spark_version           = data.databricks_spark_version.latest.id
  node_type_id            = data.databricks_node_type.smallest.id
  autotermination_minutes = 20
  autoscale {
    min_workers = 1
    max_workers = 50
  }
  azure_attributes {
    availability           = "SPOT_WITH_FALLBACK_AZURE"
    first_on_demand        = 0
    spot_bid_price_percent = 100
  }
}

resource "databricks_cluster" "positive3" {
  cluster_name            = "data"
  spark_version           = data.databricks_spark_version.latest.id
  node_type_id            = data.databricks_node_type.smallest.id
  autotermination_minutes = 20
  autoscale {
    min_workers = 1
    max_workers = 50
  }
  azure_attributes {
    availability           = "SPOT_WITH_FALLBACK_AZURE"
    zone_id                = "auto"
    spot_bid_price_percent = 100
  }
}

resource "databricks_cluster" "positive1" {
  cluster_name            = "data"
  spark_version           = data.databricks_spark_version.latest.id
  node_type_id            = data.databricks_node_type.smallest.id
  autotermination_minutes = 20
  autoscale {
    min_workers = 1
    max_workers = 50
  }
  azure_attributes {
    availability           = "SPOT_AZURE"
    first_on_demand        = 1
    spot_bid_price_percent = 100
  }
}