---
title: DatadogBYOCCluster CRD
description: >-
  Deploy and manage BYOC clusters and Observability Pipelines Workers using the
  DatadogBYOCCluster custom resource definition with the Datadog Operator.
breadcrumbs: Docs > Containers > Datadog Operator > DatadogBYOCCluster CRD
---

> For the complete documentation index, see [llms.txt](https://docs.datadoghq.com/llms.txt).

# DatadogBYOCCluster CRD

To deploy and manage a BYOC Logs cluster and its Observability Pipelines Workers, use the Datadog Operator and `DatadogBYOCCluster` custom resource definition (CRD).

## Prerequisites{% #prerequisites %}

- [Helm](https://helm.sh/)
- [`kubectl` CLI](https://kubernetes.io/docs/tasks/tools/install-kubectl/)
- [Datadog Operator](https://docs.datadoghq.com/byoc-logs/install/datadog_operator.md) v1.33.0 or later, with the BYOC controller enabled and both the `DatadogBYOCCluster` and `DatadogObservabilityPipelinesWorker` CRDs installed
- An Amazon EKS cluster, S3 bucket, PostgreSQL database, IAM role, namespace, and Secrets configured as described in the [installation guide](https://docs.datadoghq.com/byoc-logs/install/datadog_operator.md)
- An Observability Pipeline created in Datadog, with a source and BYOC destination configured

The Operator deploys Workers for existing pipelines. It does not create pipeline definitions, the S3 bucket, PostgreSQL database, or IAM role.

## Setup{% #setup %}

1. Create a file with the spec of your `DatadogBYOCCluster` deployment configuration.

**Example**: A BYOC cluster with one pipeline.

Replace the placeholders with your AWS account, IAM role, S3 bucket, and pipeline ID. Set the AWS region and Datadog site for your deployment. Choose the source port name and numeric port from Pipelines and ports. The example uses the namespace and Secrets from the [installation guide](https://docs.datadoghq.com/byoc-logs/install/datadog_operator.md).

In the `byoc-cluster.yaml` file:

   ```yaml
      apiVersion: datadoghq.com/v1alpha1
      kind: DatadogBYOCCluster
      metadata:
        name: byoc-demo
        namespace: byoc
      spec:
        datadog:
          site: datadoghq.com
          apiKeySecretRef:
            name: datadog-secret
            key: api-key
        provider:
          aws:
            accountID: "<AWS_ACCOUNT_ID>"
            region: us-east-1
            irsaRoleARN: "arn:aws:iam::<AWS_ACCOUNT_ID>:role/<BYOC_ROLE>"
        components:
          metastore:
            replicas: 1
            database:
              uriSecretRef:
                name: byoc-postgres
                key: primary-uri
          indexer:
            replicas: 1
          searcher:
            replicas: 1
          compactor:
            replicas: 1
            resources:
              requests:
                cpu: "4"
                memory: 16Gi
              limits:
                memory: 16Gi
          controlPlane: {}
          janitor: {}
          pipelines:
            - name: default
              pipelineID: "<PIPELINE_ID>"
              replicas: 1
              ports:
                - name: "<SOURCE_PORT_NAME>"
                  port: <SOURCE_PORT>
        nodeConfigOverrides:
          default_index_root_uri: "s3://<S3_BUCKET>/indexes"
      
```

See the configuration fields for additional options. Size resources and replicas for your workload using the [sizing guide](https://docs.datadoghq.com/byoc-logs/operate/sizing.md#operator-resource-configuration).

1. Deploy your `DatadogBYOCCluster`:

   ```shell
   kubectl apply -f byoc-cluster.yaml
   ```

The Operator creates and manages the engine workloads and a `DatadogObservabilityPipelinesWorker` resource for each entry in `spec.components.pipelines`. See Status and reconciliation to inspect the deployment.

## Additional examples{% #additional-examples %}

### Image overrides{% #image-overrides %}

The Operator supplies default images for the BYOC engine and Workers. An Operator upgrade also updates those default images. To pin an image, merge an explicit tag or digest into `spec.imageOverrides`:

In the `byoc-cluster.yaml` file:

```yaml
spec:
  imageOverrides:
    byoc:
      tag: "<BYOC_IMAGE_TAG>"
    observabilityPipelinesWorker:
      digest: "sha256:<OPW_IMAGE_DIGEST>"
```

Specify a tag or a digest for each image you pin. An Operator upgrade does not automatically update an explicitly pinned image. Pinning one image does not pin the other. A repository-only override does not pin a version.

Use `repository` to mirror workload images and `imagePullSecrets` to reference registry credentials in the resource's namespace. Check engine and Worker compatibility before overriding versions.

## Configuration fields{% #configuration-fields %}

The following fields are set under `spec`. To inspect all available fields in the installed CRD, run:

```shell
kubectl explain datadogbyoccluster.spec --recursive
```

{% dl %}

{% dt %}
`datadog.site`
{% /dt %}

{% dd %}
*string* Datadog site shared by the engine and Workers. Defaults to `datadoghq.com` (US1) if omitted.
{% /dd %}

{% dt %}
`datadog.apiKeySecretRef`
{% /dt %}

{% dd %}
**required** - *object* Name and key of the Datadog API key Secret in the cluster's namespace.
{% /dd %}

{% dt %}
`provider.aws`
{% /dt %}

{% dd %}
*object* AWS account ID, region, and optional IRSA role ARN. The Operator supports AWS deployments using the reverse connection to Datadog.
{% /dd %}

{% dt %}
`identity.serviceAccountName`
{% /dt %}

{% dd %}
*string* Existing service account for engine workloads. If omitted, the Operator creates a service account named after the cluster.
{% /dd %}

{% dt %}
`components`
{% /dt %}

{% dd %}
**required** - *object* Engine components and pipeline Worker configurations. See Engine components and Pipelines and ports.
{% /dd %}

{% dt %}
`global`
{% /dt %}

{% dd %}
*object* Shared labels, annotations, environment variables, volumes, volume mounts, tolerations, affinity, topology spread constraints, and Pod disruption budget settings.
{% /dd %}

{% dt %}
`nodeConfigOverrides`
{% /dt %}

{% dd %}
*object* Overrides merged into the engine's node configuration, including `default_index_root_uri`.
{% /dd %}

{% dt %}
`imageOverrides`
{% /dt %}

{% dd %}
*object* Image repositories, tags, digests, pull policies, and workload image pull secrets. See Image overrides.
{% /dd %}

{% /dl %}

`provider.aws.irsaRoleARN` applies to the service account created by the Operator. For an existing service account, configure its IRSA annotation and IAM trust policy separately. Workers use their own service accounts; set a pipeline's `identity.serviceAccountName` to use an existing identity.

### Engine components{% #engine-components %}

The required engine component entries are `metastore`, `indexer`, `searcher`, `controlPlane`, and `janitor`. The optional `compactor` and `readOnlyMetastore` entries enable those workloads when configured. Each component supports replica and resource configuration, with additional settings for storage and scheduling.

Datadog strongly recommends enabling `compactor` to isolate compaction from indexing and scale compactors independently. Set its resource requests and limits using the [sizing guide](https://docs.datadoghq.com/byoc-logs/operate/sizing.md#operator-resource-configuration).

The primary and read-only metastores require a database connection Secret reference under `database.uriSecretRef`. The Operator deploys the metastore workloads; you provision and operate their PostgreSQL databases. See [Metastore read replicas](https://docs.datadoghq.com/byoc-logs/configure/metastore_read_replicas.md).

### Pipelines and ports{% #pipelines-and-ports %}

Set each existing pipeline's ID in `spec.components.pipelines[].pipelineID`. Each entry creates a managed `DatadogObservabilityPipelinesWorker` resource and Service named `<CLUSTER_NAME>-pipeline-<PIPELINE_NAME>`.

The Operator supplies the Worker's BYOC destination endpoint, Datadog site, and shared API key. If engine REST TLS is configured, the destination URL uses HTTPS. Configure certificates and trust settings for that connection separately.

Port names control automatic source-address configuration. Use an exact port name from the following table in `spec.components.pipelines[].ports[].name`. The Operator sets the corresponding environment variable to `0.0.0.0:<PORT>`, using that entry's `port` value, and creates the container and Service ports.

| Port name        | Source-address environment variable            |
| ---------------- | ---------------------------------------------- |
| `otlp-grpc`      | `DD_OP_SOURCE_OTEL_GRPC_ADDRESS`               |
| `otlp-http`      | `DD_OP_SOURCE_OTEL_HTTP_ADDRESS`               |
| `datadog-agent`  | `DD_OP_SOURCE_DATADOG_AGENT_ADDRESS`           |
| `splunk-tcp`     | `DD_OP_SOURCE_SPLUNK_TCP_ADDRESS`              |
| `splunk-hec`     | `DD_OP_SOURCE_SPLUNK_HEC_ADDRESS`              |
| `http-server`    | `DD_OP_SOURCE_HTTP_SERVER_ADDRESS`             |
| `fluent`         | `DD_OP_SOURCE_FLUENT_ADDRESS`                  |
| `logstash`       | `DD_OP_SOURCE_LOGSTASH_ADDRESS`                |
| `syslog`         | `DD_OP_SOURCE_SYSLOG_ADDRESS`                  |
| `socket`         | `DD_OP_SOURCE_SOCKET_ADDRESS`                  |
| `aws-firehose`   | `DD_OP_SOURCE_AWS_DATA_FIREHOSE_ADDRESS`       |
| `sumo-logic`     | `DD_OP_SOURCE_SUMO_LOGIC_ADDRESS`              |
| `prom-pushgw`    | `DD_OP_SOURCE_PROMETHEUS_PUSHGATEWAY_ADDRESS`  |
| `prom-rem-write` | `DD_OP_SOURCE_PROMETHEUS_REMOTE_WRITE_ADDRESS` |

If you use a port name outside this table, set the source's `DD_OP_SOURCE_*_ADDRESS` environment variable explicitly in `spec.components.pipelines[].env`. Declaring the port alone does not configure the Worker's source listener.

For example:

| Source             | Port name       | Example port |
| ------------------ | --------------- | ------------ |
| OpenTelemetry gRPC | `otlp-grpc`     | `4317`       |
| OpenTelemetry HTTP | `otlp-http`     | `4318`       |
| Datadog Agent      | `datadog-agent` | `8282`       |

Adding a port does not create or change the pipeline's source definition in Datadog. Match the pipeline source, Worker port, and sender protocol. Worker Services use cluster-internal connectivity; log sources need network access to those Services.

Add more entries to deploy multiple pipelines. Keep each entry's `name` unique. Renaming an entry creates a new Worker resource and deletes the old one, changing its endpoint and buffer life cycle.

### Resources and storage{% #resources-and-storage %}

Set resources under `spec.components.<COMPONENT>.resources` or an entry in `spec.components.pipelines`. The Operator derives engine cache and queue settings from resource and storage configuration. Choose capacity, replicas, storage, and node placement for your workload. See [Operator resource configuration](https://docs.datadoghq.com/byoc-logs/operate/sizing.md#operator-resource-configuration).

Configure persistent storage for indexers, searchers, or Workers under `storage.volumeClaimTemplate`. Choose the StorageClass and capacity before creating the workload. For Worker buffering, configure both the Observability Pipeline buffer and the Worker's persistent storage. See [Disk buffer durability](https://docs.datadoghq.com/byoc-logs/operate/disk_buffer_durability.md).

## Status and reconciliation{% #status-and-reconciliation %}

Inspect the cluster and its Worker resources:

```shell
kubectl get datadogbyoccluster <CLUSTER_NAME> -n <NAMESPACE_NAME> -o yaml
kubectl get datadogobservabilitypipelinesworkers -n <NAMESPACE_NAME>
```

Check `status.conditions` for:

- `Reconciled=True`: managed resources match the desired configuration.
- `Available=True`: the configured engine workloads and managed Workers are available.

Check each condition's `observedGeneration`, `reason`, and `message`. Configuration errors can be reported during reconciliation after Kubernetes accepts the resource. Also verify the Datadog connection and send a test log; workload availability alone does not verify ingestion.

Make changes to the parent `DatadogBYOCCluster` manifest. The Operator reverts direct edits to managed resources, including child Workers, and reports conflicts with resources it does not own. See [BYOC troubleshooting](https://docs.datadoghq.com/byoc-logs/operate/troubleshooting.md).

## Further reading{% #further-reading %}

Additional helpful documentation, links, and articles:

- [Install BYOC with the Datadog Operator](https://docs.datadoghq.com/byoc-logs/install/datadog_operator.md)
