---
title: Ensure journald is configured to write log files to persistent disk
description: Datadog, the leading service for cloud-scale monitoring.
breadcrumbs: >-
  Docs > Datadog Security > OOTB Rules > Ensure journald is configured to write
  log files to persistent disk
---

> For the complete documentation index, see [llms.txt](https://docs.datadoghq.com/llms.txt).

# Ensure journald is configured to write log files to persistent disk
 
## Description{% #description %}

The journald system may store log files in volatile memory or locally on disk. If the logs are only stored in volatile memory they will be lost upon reboot.

## Rationale{% #rationale %}

Log files contain valuable data and need to be persistent to aid in possible investigations.

## Remediation{% #remediation %}

### Shell script{% #shell-script %}

The following script can be run on the host to remediate the issue.

```bash
#!/bin/bash

# Remediation is applicable only in certain platforms
if dpkg-query --show --showformat='${db:Status-Status}' 'linux-base' 2>/dev/null | grep -q '^installed$'; then

for f in /run/systemd/journald.conf.d/*.conf /usr/local/lib/systemd/journald.conf.d/*.conf /usr/lib/systemd/journald.conf.d/*.conf; do
    [ -e "$f" ] || continue
    name=$(basename "$f")
    for d in /etc/systemd/journald.conf.d /run/systemd/journald.conf.d /usr/local/lib/systemd/journald.conf.d /usr/lib/systemd/journald.conf.d; do
        [ -e "$d/$name" ] && break
    done
    [ "$d/$name" = "$f" ] || continue

    wrong=$(sed -n "/^[[:space:]]*\[Journal\]/,/^[[:space:]]*\[/{/^[[:space:]]*Storage[[:blank:]]*=/{ /=[[:blank:]]*persistent"'[[:space:]]*$/!p }}' "$f")

    if [ -n "$wrong" ]; then
        mkdir -p "/etc/systemd/journald.conf.d"
        cp -p "$f" "/etc/systemd/journald.conf.d/$name"
    fi
done
found=false

# set value in all files if they contain section or key
for f in $(echo -n "/etc/systemd/journald.conf.d/complianceascode_hardening.conf /etc/systemd/journald.conf.d/*.conf /etc/systemd/journald.conf"); do
    if [ ! -e "$f" ]; then
        continue
    fi

    # find key in section and change value
    if grep -qzosP "(?m)^[[:space:]]*\[Journal\]([^\n\[]*\n+)+?[[:space:]]*Storage" "$f"; then
        if ! grep -qzosP "(?m)^[[:space:]]*Storage[[:space:]]*=[[:space:]]*persistent"'[[:space:]]*$' "$f"; then

            sed -i "/^[[:space:]]*Storage/s/\([[:blank:]]*=[[:blank:]]*\).*/\1persistent/" "$f"

        fi

        found=true

    # find section and add key = value to it
    elif grep -qs "^[[:space:]]*\[Journal\]" "$f"; then

            sed -i "/^[[:space:]]*\[Journal\]/a Storage=persistent" "$f"

            found=true
    fi


    # within [Journal], remove entries where Storage has wrong value

    sed -i "/^[[:space:]]*\[Journal\]/,/^[[:space:]]*\[/{/^[[:space:]]*Storage[[:blank:]]*=/{ /=[[:blank:]]*persistent"'[[:space:]]*$/!d }}' "$f"


done

# if section not in any file, append section with key = value to FIRST file in files parameter
if ! $found ; then
    file=$(echo "/etc/systemd/journald.conf.d/complianceascode_hardening.conf /etc/systemd/journald.conf.d/*.conf /etc/systemd/journald.conf" | cut -f1 -d ' ')
    mkdir -p "$(dirname "$file")"
    chmod 0755 "$(dirname "$file")"

    echo -e "[Journal]\nStorage=persistent" >> "$file"

fi

else
    >&2 echo 'Remediation is not applicable, nothing was done'
fi
```

### Ansible playbook{% #ansible-playbook %}

The following playbook can be run with Ansible to remediate the issue.

```go
- name: Gather the package facts
  package_facts:
    manager: auto
  tags:
  - journald_storage
  - low_complexity
  - low_disruption
  - medium_severity
  - no_reboot_needed
  - restrict_strategy

- name: Ensure journald is configured to write log files to persistent disk - Find
    the dropins of each dropin directory
  ansible.builtin.find:
    paths: '{{ item }}'
    patterns: '*.conf'
    file_type: any
  register: systemd_dropins_by_dir
  loop:
  - /etc/systemd/journald.conf.d
  - /run/systemd/journald.conf.d
  - /usr/local/lib/systemd/journald.conf.d
  - /usr/lib/systemd/journald.conf.d
  when: '"linux-base" in ansible_facts.packages'
  tags:
  - journald_storage
  - low_complexity
  - low_disruption
  - medium_severity
  - no_reboot_needed
  - restrict_strategy

- name: Ensure journald is configured to write log files to persistent disk - Find
    the vendor dropins that set Storage to another value
  ansible.builtin.find:
    paths:
    - /run/systemd/journald.conf.d
    - /usr/local/lib/systemd/journald.conf.d
    - /usr/lib/systemd/journald.conf.d
    patterns: '*.conf'
    contains: (?m)^[ \t]*\[Journal\].*(?:\n(?![ \t]*\[).*)*\n[ \t]*Storage[ \t]*=(?![
      \t]*persistent[ \t]*$)
    read_whole_file: true
  register: systemd_vendor_dropins_wrong
  when: '"linux-base" in ansible_facts.packages'
  tags:
  - journald_storage
  - low_complexity
  - low_disruption
  - medium_severity
  - no_reboot_needed
  - restrict_strategy

- name: Ensure journald is configured to write log files to persistent disk - Ensure
    /etc/systemd/journald.conf.d exists
  ansible.builtin.file:
    path: /etc/systemd/journald.conf.d
    state: directory
    mode: '0755'
  when:
  - '"linux-base" in ansible_facts.packages'
  - systemd_vendor_dropins_wrong.matched | int > 0
  tags:
  - journald_storage
  - low_complexity
  - low_disruption
  - medium_severity
  - no_reboot_needed
  - restrict_strategy

- name: Ensure journald is configured to write log files to persistent disk - Copy
    into /etc/systemd/journald.conf.d the vendor dropins in effect that set Storage
    to another value
  ansible.builtin.copy:
    src: '{{ item }}'
    dest: /etc/systemd/journald.conf.d/{{ item | basename }}
    remote_src: true
    force: false
    mode: '0644'
  register: systemd_vendor_dropin_copies
  loop: '{{ systemd_vendor_dropins_wrong.files | map(attribute=''path'') | list }}'
  when:
  - '"linux-base" in ansible_facts.packages'
  - item == (systemd_dropins_by_dir.results | map(attribute='files') | flatten | map(attribute='path')
    | select('search', '/' ~ (item | basename | regex_escape) ~ '$') | first)
  tags:
  - journald_storage
  - low_complexity
  - low_disruption
  - medium_severity
  - no_reboot_needed
  - restrict_strategy

- name: Ensure journald is configured to write log files to persistent disk - Set
    Storage in the copies of the vendor dropins
  community.general.ini_file:
    path: '{{ item }}'
    section: Journal
    option: Storage
    value: persistent
    state: present
    no_extra_spaces: true
    create: false
  loop: '{{ systemd_vendor_dropin_copies.results | selectattr(''changed'') | map(attribute=''dest'')
    | list }}'
  when: '"linux-base" in ansible_facts.packages'
  tags:
  - journald_storage
  - low_complexity
  - low_disruption
  - medium_severity
  - no_reboot_needed
  - restrict_strategy

- name: Ensure journald is configured to write log files to persistent disk - Search
    for a section in files
  ansible.builtin.find:
    paths: '{{item.path}}'
    patterns: '{{item.pattern}}'
    contains: ^\s*\[Journal\]
    read_whole_file: true
    use_regex: true
  register: systemd_dropin_files_with_section
  loop:
  - path: '{{ ''/etc/systemd/journald.conf'' | dirname }}'
    pattern: '{{ ''/etc/systemd/journald.conf'' | basename | regex_escape }}'
  - path: /etc/systemd/journald.conf.d
    pattern: .*\.conf
  when: '"linux-base" in ansible_facts.packages'
  tags:
  - journald_storage
  - low_complexity
  - low_disruption
  - medium_severity
  - no_reboot_needed
  - restrict_strategy

- name: Ensure journald is configured to write log files to persistent disk - Count
    number of files which contain the correct section
  ansible.builtin.set_fact:
    count_of_systemd_dropin_files_with_section: '{{systemd_dropin_files_with_section.results
      | map(attribute=''matched'') | list | map(''int'') | sum}}'
  when: '"linux-base" in ansible_facts.packages'
  tags:
  - journald_storage
  - low_complexity
  - low_disruption
  - medium_severity
  - no_reboot_needed
  - restrict_strategy

- name: Ensure journald is configured to write log files to persistent disk - Collect
    current permissions of existing drop-in files
  ansible.builtin.stat:
    path: '{{item}}'
  register: systemd_dropin_existing_modes
  when:
  - '"linux-base" in ansible_facts.packages'
  - count_of_systemd_dropin_files_with_section | int > 0
  loop: '{{systemd_dropin_files_with_section.results | sum(attribute=''files'', start=[])
    | map(attribute=''path'') | list }}'
  tags:
  - journald_storage
  - low_complexity
  - low_disruption
  - medium_severity
  - no_reboot_needed
  - restrict_strategy

- name: Ensure journald is configured to write log files to persistent disk - Add
    missing configuration to correct section
  community.general.ini_file:
    path: '{{item}}'
    section: Journal
    option: Storage
    value: persistent
    state: present
    no_extra_spaces: true
    mode: '{{ (systemd_dropin_existing_modes.results | selectattr(''item'', ''equalto'',
      item) | map(attribute=''stat.mode'') | first) | default(''0644'', true) }}'
  when:
  - '"linux-base" in ansible_facts.packages'
  - count_of_systemd_dropin_files_with_section | int > 0
  loop: '{{systemd_dropin_files_with_section.results | sum(attribute=''files'', start=[])
    | map(attribute=''path'') | list }}'
  tags:
  - journald_storage
  - low_complexity
  - low_disruption
  - medium_severity
  - no_reboot_needed
  - restrict_strategy

- name: Ensure journald is configured to write log files to persistent disk - Add
    configuration to new remediation file
  community.general.ini_file:
    path: /etc/systemd/journald.conf.d/complianceascode_hardening.conf
    section: Journal
    option: Storage
    value: persistent
    state: present
    no_extra_spaces: true
    create: true
    mode: '0644'
  when:
  - '"linux-base" in ansible_facts.packages'
  - count_of_systemd_dropin_files_with_section | int == 0
  tags:
  - journald_storage
  - low_complexity
  - low_disruption
  - medium_severity
  - no_reboot_needed
  - restrict_strategy
```