개요

Observability Pipelines의 Amazon Security Lake 목적지를 사용하여 Amazon Security Lake로 로그를 전송하세요.

전제 조건

Amazon Security Lake 목적지를 설정하기 전에 다음을 수행해야 합니다.

1. Follow the Getting Started with Amazon Security Lake to set up Amazon Security Lake, and make sure to:
   • Enable Amazon Security Lake for the AWS account.
   • Select the AWS regions where S3 buckets will be created for OCSF data.
2. Follow Collecting data from custom sources in Security Lake to create a custom source in Amazon Security Lake.
   • When you configure a custom log source in Security Lake in the AWS console:
     • Enter a source name.
     • Select the OCSF event class for the log source and type.
     • Enter the account details for the AWS account that will write logs to Amazon Security Lake:
       • AWS account ID
       • External ID
   • Select Create and use a new service for service access.
   • Take note of the name of the bucket that is created because you need it when you set up the Amazon Security Lake destination later on.
     • To find the bucket name, navigate to Custom Sources. The bucket name is in the location for your custom source. For example, if the location is s3://aws-security-data-lake-us-east-2-qjh9pr8hy/ext/op-api-activity-test, the bucket name is aws-security-data-lake-us-east-2-qjh9pr8hy.

설정

파이프라인을 설정할 때 Amazon Security Lake 목적지를 구성하세요. 파이프라인은 UI에서 설정할 수 있으며, API 또는 Terraform을 사용하여 설정할 수 있습니다. 이 섹션에서 설명하는 단계는 UI에서 설정합니다.

참고:

• Amazon Security Lake 목적지를 추가하면 OCSF 프로세서가 자동으로 추가되어 로그를 Amazon Security Lake로 전송하기 전에 Parquet로 변환할 수 있습니다. 설정 지침은 OCSF로 리매핑 설명서를 참조하세요.
• OCSF 프로세서에 의해 형식이 지정된 로그만 Parquet로 변환됩니다.

파이프라인 UI에서 Amazon Security Lake 목적지를 선택한 후 다음 단계를 따르세요.

1. S3 버킷 이름을 입력합니다.
2. AWS 리전을 입력합니다.
3. 사용자 지정 소스 이름을 입력합니다.

선택적 설정

AWS 인증

1. AWS 인증 옵션을 선택하세요.
2. 맡으려는 IAM 역할의 ARN을 입력합니다.
3. 필요시 맡은 역할의 세션 이름과 외부 ID를 입력합니다.

TLS 활성화

Toggle the switch to Enable TLS.

• If you are using Secrets Management, enter the identifier for the key pass. See Secret defaults for the default used if the field is left blank.
• Enter the following certificate and key files:
  • Server Certificate Path: The path to the certificate file that has been signed by your Certificate Authority (CA) root file in DER, PEM, or CRT (X.509).
  • (Optional) CA Certificate Path: The path to the certificate file that is your Certificate Authority (CA) root file in DER, PEM, or CRT (X.509).
  • (Optional) Private Key Path: The path to the .key private key file that belongs to your Server Certificate Path in DER, PEM, or CRT (PKCS #8) format.
  • Notes:
    • The configuration data directory /var/lib/observability-pipelines-worker/config/ is automatically appended to the file paths. See Advanced Worker Configurations for more information.
    • The file must be readable by the observability-pipelines-worker group and user.

버퍼링

Toggle the switch to enable Buffering Options. Enable a configurable buffer on your destination to ensure intermittent latency or an outage at the destination doesn’t create immediate backpressure, and allow events to continue to be ingested from your source. Disk buffers can also increase pipeline durability by writing data to disk, ensuring buffered data persists through a Worker restart. See Destination buffers for more information.

• If left unconfigured, your destination uses a memory buffer with a capacity of 500 events.
• To configure a buffer on your destination:
  1. Select the buffer type you want to set (Memory or Disk).
  2. Enter the buffer size and select the unit.
     1. Maximum memory buffer size is 128 GB.
     2. Maximum disk buffer size is 5 TB.
        • Note: For Worker versions 2.20.x and older, the maximum disk buffer size is 500 GB.
  3. In the Behavior on full buffer dropdown menu, select whether you want to block events or drop new events when the buffer is full.

시크릿 기본값

These are the defaults used for secret identifiers and environment variables.

목적지의 작동 방식

AWS 인증

The Observability Pipelines Worker uses the standard AWS credential provider chain for authentication. See AWS SDKs and Tools standardized credential providers for more information.

권한

For Observability Pipelines to send logs to Amazon Security Lake, the following policy permissions are required:

• s3:ListBucket
• s3:PutObject

이벤트 배치 처리

이벤트 배치는 다음 중 하나의 파라미터를 충족하면 플러시됩니다. 자세한 내용은 목적지 이벤트 배치 처리를 참조하세요.