Enable logging

When you enable CloudFront logging for a distribution, specify the Amazon S3 bucket that you want CloudFront to store log files in. If you’re using Amazon S3 as your origin, Datadog recommends that you do not use the same bucket for your log files; using a separate bucket simplifies maintenance.

Note: Datadog recommends storing the log files for multiple distributions in the same bucket so that the log forwarder only has to subscribe to one bucket.

Send logs to Datadog

1. If you haven’t already, set up the Datadog Forwarder Lambda function in your AWS account.
2. Once set up, go to the Datadog Forwarder Lambda function. In the Function Overview section, click Add Trigger.
3. Select the S3 trigger for the Trigger Configuration.
4. Select the S3 bucket that contains your CloudFront logs.
5. Leave the event type as All object create events.
6. Click Add to add the trigger to your Lambda.

Go to the Log Explorer to start exploring your logs.

For more information on collecting AWS Services logs, see Send AWS Services Logs with the Datadog Lambda Function.

Enable logging

Create a specific configuration

When creating a real-time log configuration, you can specify which log fields you want to receive. By default, all of the available fields are selected.

Datadog recommends that you keep this default configuration and add the following custom parsing rule to automatically process logs with all fields enabled.

Navigate to the Pipelines page, search for Amazon CloudFront, [create or edit a grok parser processor][7], and add the following helper rules under Advanced Settings:

      real_time_logs (%{number:timestamp:scale(1000)}|%{number:timestamp})\s+%{_client_ip}\s+%{_time_to_first_byte}\s+%{_status_code}\s+%{_bytes_write}\s+%{_method}\s+%{regex("[a-z]*"):http.url_details.scheme}\s+%{notSpace:http.url_details.host:nullIf("-")}\s+%{notSpace:http.url_details.path:nullIf("-")}\s+%{_bytes_read}\s+%{notSpace:cloudfront.edge-location:nullIf("-")}\s+%{_request_id}\s+%{_ident}\s+%{_duration}\s+%{_version}\s+IPv%{integer:network.client.ip_version}\s+%{_user_agent}\s+%{_referer}\s+%{notSpace:cloudfront.cookie}\s+(%{notSpace:http.url_details.queryString:querystring}|%{notSpace:http.url_details.queryString:nullIf("-")})\s+%{notSpace:cloudfront.edge-response-result-type:nullIf("-")}\s+%{_x_forwarded_for}\s+%{_ssl_protocol}\s+%{_ssl_cipher}\s+%{notSpace:cloudfront.edge-result-type:nullIf("-")}\s+%{_fle_encrypted_fields}\s+%{_fle_status}\s+%{_sc_content_type}\s+%{_sc_content_len}\s+%{_sc_range_start}\s+%{_sc_range_end}\s+%{_client_port}\s+%{_x_edge_detailed_result_type}\s+%{notSpace:network.client.country:nullIf("-")}\s+%{notSpace:accept-encoding:nullIf("-")}\s+%{notSpace:accept:nullIf("-")}\s+%{notSpace:cache-behavior-path-pattern:nullIf("-")}\s+%{notSpace:headers:nullIf("-")}\s+%{notSpace:header-names:nullIf("-")}\s+%{integer:headers-count}.*

Send logs to Datadog

Real-time logs are delivered to the Kinesis Data Stream of your choice and can be directly forwarded to Datadog with the Amazon Data Firehose integration.

You can also configure a consumer, such as Amazon Data Firehose, to send real-time logs to an S3 bucket and use the Datadog Lambda forwarder to ship logs to Datadog.