---
title: Secrets Management
description: Learn how to set up the Worker to retrieve secrets from your secrets manager.
breadcrumbs: Docs > Observability Pipelines > Configuration > Secrets Management
---

# Secrets Management

{% callout %}
# Important note for users on the following Datadog sites: app.ddog-gov.com

{% alert level="danger" %}
This product is not supported for your selected [Datadog site](https://docs.datadoghq.com/getting_started/site). ().
{% /alert %}

{% /callout %}

## Overview{% #overview %}

The Observability Pipelines Worker helps you securely manage your secrets by integrating with the following secrets management solution:

- AWS Secrets Manager
- AWS Systems Manager
- Azure Key Vault
- HashiCorp Vault
- JSON File
- YAML File

**Notes**:

- If you want to use Kubernetes Secrets, follow the instructions in [Configure all key-value pairs in a Secret as container environment variables](https://kubernetes.io/docs/tasks/inject-data-application/distribute-credentials-secure/#configure-all-key-value-pairs-in-a-secret-as-container-environment-variables) and set up the secret as an environment variable for the Observability Pipelines Worker.
- When you set up a pipeline and install the Worker, if you enter secret identifiers and then choose to use environment variables, the environment variable is the identifier entered and prepended with `DD_OP`. For example, if you entered `PASSWORD_1` for a password identifier, the environment variable for that password is `DD_OP_PASSWORD_1`.

## Configure the Worker to retrieve secrets{% #configure-the-worker-to-retrieve-secrets %}

{% collapsible-section #aws-secrets-manager %}
#### AWS Secrets Manager

#### Set up an AWS instance profile{% #set-up-an-aws-instance-profile %}

Datadog recommends using the [instance profile method](https://docs.aws.amazon.com/IAM/latest/UserGuide/id_roles_use_switch-role-ec2_instance-profiles.html) of retrieving secrets because AWS handles all environment variables and session profiles for you. See the official [AWS Secrets Manager documentation](https://docs.aws.amazon.com/secretsmanager/latest/userguide/intro.html) for setup instructions.

#### Configure the Worker to use AWS Secrets Manager{% #configure-the-worker-to-use-aws-secrets-manager %}

{% tab title="Docker or Linux" %}
After you [install the Worker](https://docs.datadoghq.com/observability_pipelines/configuration/install_the_worker/?tab=docker#install-the-worker), configure the Worker's [bootstrap file](https://docs.datadoghq.com/observability_pipelines/configuration/install_the_worker/advanced_worker_configurations/) to resolve secrets using AWS Secrets Manager:

```yaml
backend_type: aws.secrets
backend_config:
  aws_session:
    aws_region: <region_name>
```

{% /tab %}

{% tab title="Kubernetes" %}
Before you [install the Worker](https://docs.datadoghq.com/observability_pipelines/configuration/install_the_worker/?tab=kubernetes#install-the-worker), add the bootstrap configuration to the [`datadog.bootstrap.config`](https://github.com/DataDog/helm-charts/blob/main/charts/observability-pipelines-worker/values.yaml#L46) section of the Helm chart `values.yaml` file:

```yaml
bootstrap:
  config:
    secret:
      backend_type: aws.secrets
      backend_config:
        aws_session:
          aws_region: <region_name>
```

{% /tab %}

The following `aws_session` settings are available:

| Setting               | Description                    |
| --------------------- | ------------------------------ |
| aws_region            | AWS Region                     |
| aws_profile           | AWS Session Profile            |
| aws_role_arn          | AWS sts:AssumeRole ARN         |
| aws_external_id       | AWS sts:AssumeRole ExternalId  |
| aws_access_key_id     | AWS IAM User Access Key ID     |
| aws_secret_access_key | AWS IAM User Access Key Secret |

In most cases, you need to specify `aws_region` to correspond to the region hosting the Secrets Manager secret.

When handling single strings, the backend configuration setting `force_string: true` coerces the secret as a string value.
{% /collapsible-section %}

{% collapsible-section #aws-systems-manager %}
#### AWS Systems Manager

#### Set up an AWS instance profile{% #set-up-an-aws-instance-profile-1 %}

Datadog recommends using the [instance profile method](https://docs.aws.amazon.com/IAM/latest/UserGuide/id_roles_use_switch-role-ec2_instance-profiles.html) of retrieving secrets because AWS handles all environment variables and session profiles for you. See the official [AWS Systems Manager Parameter Store](https://docs.aws.amazon.com/systems-manager/latest/userguide/systems-manager-parameter-store.html) documentation for setup instructions.

#### Configure the Worker to use AWS Systems Manager{% #configure-the-worker-to-use-aws-systems-manager %}

{% tab title="Docker or Linux" %}
After you [install the Worker](https://docs.datadoghq.com/observability_pipelines/configuration/install_the_worker/?tab=docker#install-the-worker), configure the Worker's [bootstrap file](https://docs.datadoghq.com/observability_pipelines/configuration/install_the_worker/advanced_worker_configurations/) to resolve secrets using AWS Systems Manager:

```yaml
secret:
  backend_type: aws_parameter_store
  backend_config:
    aws_session:
      aws_region: <region_name>
```

{% /tab %}

{% tab title="Kubernetes" %}
Before you [install the Worker](https://docs.datadoghq.com/observability_pipelines/configuration/install_the_worker/?tab=kubernetes#install-the-worker), add the bootstrap configuration to the [`datadog.bootstrap.config`](https://github.com/DataDog/helm-charts/blob/main/charts/observability-pipelines-worker/values.yaml#L46) section of the Helm chart `values.yaml` file:

```yaml
bootstrap:
  config:
    secret:
      backend_type: aws_parameter_store
      backend_config:
        aws_session:
          aws_region: <region_name>
```

{% /tab %}

{% /collapsible-section %}

{% collapsible-section #azure-key-vault %}
#### Azure Key Vault

#### Set up Azure authentication{% #set-up-azure-authentication %}

Datadog recommends using Managed Identities to authenticate with Azure. This allows you to associate cloud resources with AMI accounts and removes the need to put sensitive information in your Worker configuration file.

#### Create a managed identity{% #create-a-managed-identity %}

To access your Key Vault, create a Managed Identity and assign it to your VM. Then, configure the appropriate role assignment on the Key Vault to allow the managed identity to access the secrets.

#### Configure the Worker to use Azure Key Vault{% #configure-the-worker-to-use-azure-key-vault %}

{% tab title="Docker or Linux" %}
After you [install the Worker](https://docs.datadoghq.com/observability_pipelines/configuration/install_the_worker/?tab=docker#install-the-worker), configure the Worker's [bootstrap file](https://docs.datadoghq.com/observability_pipelines/configuration/install_the_worker/advanced_worker_configurations/) to resolve secrets using Azure Key Vault:

```yaml
backend_type: azure.keyvault
backend_config:
  keyvaulturl: <key_vault_url>
```

{% /tab %}

{% tab title="Kubernetes" %}
Before you [install the Worker](https://docs.datadoghq.com/observability_pipelines/configuration/install_the_worker/?tab=kubernetes#install-the-worker), add the bootstrap configuration to the [`datadog.bootstrap.config`](https://github.com/DataDog/helm-charts/blob/main/charts/observability-pipelines-worker/values.yaml#L46) section of the Helm chart `values.yaml` file:

```yaml
bootstrap:
  config:
    secret:
      backend_type: azure.keyvault
      backend_config:
        keyvaulturl: <key_vault_url>
```

{% /tab %}

{% /collapsible-section %}

{% collapsible-section #hashicorp-vault %}
#### HashiCorp Vault

#### Set up HashiCorp Vault{% #set-up-hashicorp-vault %}

1. Run your HashiCorp Vault. See the official [HashiCorp Vault documentation](https://developer.hashicorp.com/) for more information.

1. Write a policy that gives the permission to pull secrets from your vault. Create a `*.hcl` file, and include the following permission if you are using Secrets Engine Version 1:

   ```
   path "<your mount path>/<additional subpath>" {
     capabilities = ["read"]
   }
   ```

If you are using Secrets Engine Version 2, the following permissions are needed:

   ```
   path "<your_mount_path>/data/<additional_subpath>" {
     capabilities = ["read"]
   }
   
   /*
   Datadog needs access to mount information to check the Secrets Engine version
   number. If access isn't granted, version 1 is assumed.
   */
   path "sys/mounts" {
     capabilities = ["read"]
   }
   ```

1. Run `vault policy write <policy_name> <path_to_*.hcl_file>`.

1. Choose the method of authenticating to your vault. If using the AWS instance profile method, run `vault auth enable aws`.

#### Set up an AWS instance profile for an AWS-connected machine{% #set-up-an-aws-instance-profile-for-an-aws-connected-machine %}

Datadog recommends that you authenticate using the [instance profile method](https://docs.aws.amazon.com/IAM/latest/UserGuide/id_roles_use_switch-role-ec2_instance-profiles.html) if you are running your HashiCorp Vault from an AWS-connected machine.

#### Configure the Worker to use HashiCorp Vault{% #configure-the-worker-to-use-hashicorp-vault %}

{% tab title="Docker or Linux" %}
After you [install the Worker](https://docs.datadoghq.com/observability_pipelines/configuration/install_the_worker/?tab=docker#install-the-worker), configure the Worker's [bootstrap file](https://docs.datadoghq.com/observability_pipelines/configuration/install_the_worker/advanced_worker_configurations/) to resolve secrets with HashiCorp Vault:

```yaml
secret:
  backend_type: vault
  backend_config:
    vault_address: http://myvaultaddress.net
    vault_tls_config:
      # ... TLS settings if applicable
    vault_session:
      vault_auth_type: aws
      # ... additional session settings
```

{% /tab %}

{% tab title="Kubernetes" %}
Before you [install the Worker](https://docs.datadoghq.com/observability_pipelines/configuration/install_the_worker/?tab=kubernetes#install-the-worker), add the bootstrap configuration to the [`datadog.bootstrap.config`](https://github.com/DataDog/helm-charts/blob/main/charts/observability-pipelines-worker/values.yaml#L46) section of the Helm chart `values.yaml` file:

```yaml
bootstrap:
  config:
    secret:
      backend_type: vault
      backend_config:
        vault_address: http://myvaultaddress.net
        vault_tls_config:
          # ... TLS settings if applicable
        vault_session:
          vault_auth_type: aws
          # ... additional session settings
```

{% /tab %}

{% /collapsible-section %}

{% collapsible-section #json-file %}
#### JSON File

#### File permissions required{% #file-permissions-required %}

The file backend only requires **read** permissions for the configured JSON files. These permissions must be granted to the local Worker user.

#### Configure the Worker to use a JSON file secret backend{% #configure-the-worker-to-use-a-json-file-secret-backend %}

{% tab title="Docker or Linux" %}
After you [install the Worker](https://docs.datadoghq.com/observability_pipelines/configuration/install_the_worker/?tab=docker#install-the-worker), configure the Worker's [bootstrap file](https://docs.datadoghq.com/observability_pipelines/configuration/install_the_worker/advanced_worker_configurations/) to resolve secrets with a JSON file:

```yaml
secret:
  backend_type: json
  backend_config:
    file_path: /path/to/json/file.json
```

#### Create a JSON secrets file{% #create-a-json-secrets-file %}

Create the file `/path/to/json/file.json` to store the identifiers and their secret values:

```json
{
  "us1_api": "<api_key>",
  "secret_identifier1": "<secret1>"
}
```

{% /tab %}

{% tab title="Kubernetes" %}
You can't pass in a file to configure `secretFileContents`, but you can add your secrets and identifier mapping to the [`datadog.bootstrap.secretFileContents`](https://github.com/DataDog/helm-charts/blob/main/charts/observability-pipelines-worker/values.yaml#L46) section of the Helm chart `values.yaml` file:

```yaml
bootstrap:
  secretFileContents:
    us1_api: "<api_key>"
    secret_identifier1: "<secret1>"
```

{% /tab %}

{% /collapsible-section %}

{% collapsible-section #yaml-file %}
#### YAML File

#### File permissions required{% #file-permissions-required-1 %}

The file backend only requires **read** permissions for the configured YAML files. These permissions must be granted to the local Worker user.

#### Configure the Worker to use a YAML file secret backend{% #configure-the-worker-to-use-a-yaml-file-secret-backend %}

{% tab title="Docker or Linux" %}
After you [install the Worker](https://docs.datadoghq.com/observability_pipelines/configuration/install_the_worker/?tab=docker#install-the-worker), configure the Worker's [bootstrap file](https://docs.datadoghq.com/observability_pipelines/configuration/install_the_worker/advanced_worker_configurations/) to resolve secrets with a YAML file:

```
secret:
  backend_type: yaml
  backend_config:
    file_path: /path/to/yaml/file.yaml
```

#### Create a YAML secrets file{% #create-a-yaml-secrets-file %}

Create the file `/path/to/yaml/file.yaml` to store the identifiers and their secret values:

```
# /path/to/yaml/file.yaml

us1_api: "<api_key>"
secret_identifier1: "<secret1>"
```

{% /tab %}

{% tab title="Kubernetes" %}
You can't pass in a file to configure `secretFileContents`, but you can add your secrets and identifier mapping to the [`datadog.bootstrap.secretFileContents`](https://github.com/DataDog/helm-charts/blob/main/charts/observability-pipelines-worker/values.yaml#L46) section of the Helm chart `values.yaml` file:

```yaml
bootstrap:
  secretFileContents:
    us1_api: "<api_key>"
    secret_identifier1: "<secret1>"
```

{% /tab %}

{% /collapsible-section %}

## Further reading{% #further-reading %}

- [Set up pipelines](https://docs.datadoghq.com/observability_pipelines/configuration/set_up_pipelines/%22)
- [Install the Worker](https://docs.datadoghq.com/observability_pipelines/configuration/install_the_worker%22)