For AI agents: A markdown version of this page is available at https://docs.datadoghq.com/ddsql_reference/data_directory/aws/aws.datasync_location_hdfs.dataset.md. A documentation index is available at /llms.txt.

DataSync Location HDFS

DataSync HDFS Location in AWS represents a connection point to a Hadoop Distributed File System (HDFS) cluster. It allows AWS DataSync to transfer data between HDFS and AWS storage services such as Amazon S3, Amazon EFS, or Amazon FSx. The resource stores configuration details like the HDFS NameNode addresses, authentication type, and security settings, enabling secure and efficient data migration or synchronization from on-premises or self-managed Hadoop environments into AWS.

aws.datasync_location_hdfs

Fields

TitleIDTypeData TypeDescription
_keycorestring
account_idcorestring
agent_arnscorearray<string>The ARNs of the DataSync agents that can connect with your HDFS cluster.
authentication_typecorestringThe type of authentication used to determine the identity of the user.
block_sizecoreint64The size of the data blocks to write into the HDFS cluster.
cmk_secret_configcorejsonDescribes configuration information for a DataSync-managed secret, such as a KerberosKeytab that DataSync uses to access a specific storage location, with a customer-managed KMS key.
creation_timecoretimestampThe time that the HDFS location was created.
custom_secret_configcorejsonDescribes configuration information for a customer-managed secret, such as a KerberosKeytab that DataSync uses to access a specific storage location, with a customer-managed Identity and Access Management (IAM) role that provides access to the secret.
kerberos_principalcorestringThe Kerberos principal with access to the files and folders on the HDFS cluster. This parameter is used if the AuthenticationType is defined as KERBEROS.
kms_key_provider_uricorestringThe URI of the HDFS cluster's Key Management Server (KMS).
location_arncorestringThe ARN of the HDFS location.
location_uricorestringThe URI of the HDFS location.
managed_secret_configcorejsonDescribes configuration information for a DataSync-managed secret, such as a KerberosKeytab that DataSync uses to access a specific storage location. DataSync uses the default Amazon Web Services-managed KMS key to encrypt this secret in Secrets Manager.
name_nodescorejsonThe NameNode that manages the HDFS namespace.
qop_configurationcorejsonThe Quality of Protection (QOP) configuration, which specifies the Remote Procedure Call (RPC) and data transfer protection settings configured on the HDFS cluster.
replication_factorcoreint64The number of DataNodes to replicate the data to when writing to the HDFS cluster.
simple_usercorestringThe user name to identify the client on the host operating system. This parameter is used if the AuthenticationType is defined as SIMPLE.
tagscorehstore_csv