---
title: Amazon S3 Source
description: Datadog, the leading service for cloud-scale monitoring.
breadcrumbs: Docs > Observability Pipelines > Sources > Amazon S3 Source
---

# Amazon S3 Source

{% callout %}
# Important note for users on the following Datadog sites: app.ddog-gov.com

{% alert level="danger" %}
This product is not supported for your selected [Datadog site](https://docs.datadoghq.com/getting_started/site). ().
{% /alert %}

{% /callout %}
Available for:
{% icon name="icon-logs" /%}
 Logs 
## Overview{% #overview %}

Use Observability Pipelines' Amazon S3 source to receive logs from Amazon S3.

## Prerequisites{% #prerequisites %}

To use Observability Pipelines' Amazon S3 source, you must configure a SQS queue to receive your S3 bucket notifications.

## Setup{% #setup %}

Set up this source when you [set up a pipeline](https://docs.datadoghq.com/observability_pipelines/configuration/set_up_pipelines/). You can set up a pipeline in the [UI](https://app.datadoghq.com/observability-pipelines), using the [API](https://docs.datadoghq.com/api/latest/observability-pipelines/), or with [Terraform](https://registry.terraform.io/providers/datadog/datadog/latest/docs/resources/observability_pipeline). The instructions in this section are for setting up the source in the UI.

{% alert level="danger" %}
Only enter the identifiers for the Amazon S3 URL and, if applicable, the TLS key pass. Do not enter the actual values.
{% /alert %}

1. Enter the identifier for your Amazon S3 URL. If you leave it blank, the default is used.
1. Enter the AWS region.

### Optional settings{% #optional-settings %}

#### AWS authentication{% #aws-authentication %}

Select an **AWS authentication** option. If you select **Assume role**:

1. Enter the ARN of the IAM role you want to assume.
1. Optionally, enter the assumed role session name and external ID.

#### Enable TLS{% #enable-tls %}

Toggle the switch to **Enable TLS**. If you enable TLS, the following certificate and key files are required.**Note**: All file paths are made relative to the configuration data directory, which is `/var/lib/observability-pipelines-worker/config/` by default. See [Advanced Worker Configurations](https://docs.datadoghq.com/observability_pipelines/configuration/install_the_worker/advanced_worker_configurations/) for more information. The file must be owned by the `observability-pipelines-worker group` and `observability-pipelines-worker` user, or at least readable by the group or user.

- Enter the identifier for your Amazon S3 key pass. If you leave it blank, the default is used.
- `Server Certificate Path`: The path to the certificate file that has been signed by your Certificate Authority (CA) root file in DER or PEM (X.509).
- `CA Certificate Path`: The path to the certificate file that is your Certificate Authority (CA) root file in DER or PEM (X.509).
- `Private Key Path`: The path to the `.key` private key file that belongs to your Server Certificate Path in DER or PEM (PKCS#8) format.

## Set secrets{% #set-secrets %}

These are the defaults used for secret identifiers and environment variables.

**Note**: If you enter secret identifiers and then choose to use environment variables, the environment variable is the identifier entered and prepended with `DD_OP`. For example, if you entered `PASSWORD_1` for a password identifier, the environment variable for that password is `DD_OP_PASSWORD_1`.

{% tab title="Secrets Management" %}

- Amazon S3 URL identifier:
  - References the URL of the SQS queue to which the S3 bucket sends the notification events.
  - The default identifier is `SOURCE_AWS_S3_SQS_URL`.
- Amazon S3 TLS passphrase identifier (when TLS is enabled):
  - The default identifier is `SOURCE_AWS_S3_KEY_PASS`.

{% /tab %}

{% tab title="Environment Variables" %}

- Amazon S3 SQS URL:
  - The URL of the SQS queue to which the S3 bucket sends the notification events.
  - The default environment variable is `DD_OP_SOURCE_AWS_S3_SQS_URL`
- AWS_CONFIG_FILE path:
  - The path to the AWS configuration file local to this node.
  - The default environment variable is `AWS_CONFIG_FILE`.
- AWS_PROFILE name:
  - The name of the profile to use within these files.
  - The default environment variable is `AWS_PROFILE`.
- AWS S3 TLS passphrase (when enabled):
  - The default environment variable is `DD_OP_SOURCE_AWS_S3_KEY_PASS`.

{% /tab %}

## AWS Authentication{% #aws-authentication-1 %}

The Observability Pipelines Worker uses the standard AWS credential provider chain for authentication. See [AWS SDKs and Tools standardized credential providers](https://docs.aws.amazon.com/sdkref/latest/guide/standardized-credentials.html) for more information.

### Permissions{% #permissions %}

For Observability Pipelines to collect logs from Amazon S3, the following policy permissions are required:

- `s3:GetObject`
- `sqs:ReceiveMessage`
- `sqs:DeleteMessage`