Amazon CloudFront

Enable Cloudfront Logging

When you enable logging for a distribution, specify the Amazon S3 bucket that you want CloudFront to store log files in. If you’re using Amazon S3 as your origin, we recommend that you do not use the same bucket for your log files; using a separate bucket simplifies maintenance.

Important note: Store the log files for multiple distributions in the same bucket. When enabling logging, specify cloudfront as prefix for the file names, [to keep track of which log files are associated with which distributions][4].

Send Logs to Datadog

1. If you haven’t already, set up the [Datadog log collection AWS Lambda function][5].
2. Once the lambda function is installed, manually add a trigger on the S3 bucket that contains your CloudFront logs by clicking on S3 in the trigger list for your Lambda in the AWS console:

Configure your trigger by choosing the S3 bucket that contains your Cloudfront logs and change the event type to Object Created (All) then click on the add button.

Once done, go in your Datadog Log section to start exploring your logs!

Enable Cloudfront Logging

Create a specific configuration

When creating a real-time log configuration, you can specify which log fields you want to receive. By default, all of the available fields are selected.

It is recommended that you keep this default configuration and add the following custom parsing rule to automatically process logs with all fields enabled:

      real_time_logs (%{number:timestamp:scale(1000)}|%{number:timestamp})\s+%{_client_ip}\s+%{_time_to_first_byte}\s+%{_status_code}\s+%{_bytes_write}\s+%{_method}\s+%{regex("[a-z]*"):http.url_details.scheme}\s+%{notSpace:http.url_details.host:nullIf("-")}\s+%{notSpace:http.url_details.path:nullIf("-")}\s+%{_bytes_read}\s+%{notSpace:cloudfront.edge-location:nullIf("-")}\s+%{_request_id}\s+%{_ident}\s+%{_duration}\s+%{_version}\s+IPv%{integer:network.client.ip_version}\s+%{_user_agent}\s+%{_referer}\s+%{notSpace:cloudfront.cookie}\s+(%{notSpace:http.url_details.queryString:querystring}|%{notSpace:http.url_details.queryString:nullIf("-")})\s+%{notSpace:cloudfront.edge-response-result-type:nullIf("-")}\s+%{_x_forwarded_for}\s+%{_ssl_protocol}\s+%{_ssl_cipher}\s+%{notSpace:cloudfront.edge-result-type:nullIf("-")}\s+%{_fle_encrypted_fields}\s+%{_fle_status}\s+%{_sc_content_type}\s+%{_sc_content_len}\s+%{_sc_range_start}\s+%{_sc_range_end}\s+%{_client_port}\s+%{_x_edge_detailed_result_type}\s+%{notSpace:network.client.country:nullIf("-")}\s+%{notSpace:accept-encoding:nullIf("-")}\s+%{notSpace:accept:nullIf("-")}\s+%{notSpace:cache-behavior-path-pattern:nullIf("-")}\s+%{notSpace:headers:nullIf("-")}\s+%{notSpace:header-names:nullIf("-")}\s+%{integer:headers-count}.*

Send Logs to Datadog

Real-time logs are delivered to the Kinesis Data Stream of your choice and can be directly forwarded to Datadog with our Kinesis Firehose integration.

You can also configure a consumer, such as Amazon Kinesis Data Firehose, to send Real-time logs to an S3 bucket and use the Datadog Lambda forwarder to ship logs to Datadog.