Linux

Log Management with journald

·1 min read

systemd's journald is the logging daemon for every modern Linux distribution. It collects every process's stdout/stderr, kernel messages, and syslog calls, indexes them, and lets you query by service, priority, time range, and almost anything else. If you're still tailing /var/log/syslog by hand, you're doing it the hard way.

journalctl in five queries

journalctl -u nginx.service           # one service
journalctl -u nginx -f                # follow (like tail -f)
journalctl --since "1 hour ago"
journalctl --since "2026-03-18 09:00" --until "2026-03-18 10:00"
journalctl -p err..emerg              # priority range
journalctl _PID=1234                  # one process

Priorities

journald uses syslog priority levels — emerg(0), alert(1), crit(2), err(3), warning(4), notice(5), info(6), debug(7). -p err means "err or higher" by default; use ranges (err..emerg) for clarity.

Per-boot logs

journalctl --list-boots               # boot IDs
journalctl -b                         # current boot
journalctl -b -1                      # previous boot
journalctl -b -1 -k                   # kernel messages from previous boot

Persistence

By default, some distributions store journals only in /run/log/journal/ — RAM-backed, wiped on reboot. To persist:

mkdir -p /var/log/journal
systemd-tmpfiles --create --prefix /var/log/journal
systemctl restart systemd-journald

Once /var/log/journal/ exists, journald uses it. Configure retention in /etc/systemd/journald.conf:

[Journal]
Storage=persistent
SystemMaxUse=2G
SystemKeepFree=500M
MaxFileSec=1week
ForwardToSyslog=no

Structured fields

Unlike text logs, journald entries are key-value records. journalctl -o json-pretty shows the full structure. You can query any field:

journalctl _SYSTEMD_UNIT=ssh.service _PID=1234
journalctl PRIORITY=3                  # err
journalctl SYSLOG_IDENTIFIER=cron

Sending from your own programs

# Bash
echo "deployment finished" | systemd-cat -t my-deploy -p info

# Python
import systemd.journal
systemd.journal.send("deploy done", PRIORITY=6, DEPLOY_ID="abc123")

The Python snippet writes a structured entry queryable later with journalctl DEPLOY_ID=abc123.

Forwarding to rsyslog or a remote collector

For centralized logging, journald can forward to a local syslog daemon (ForwardToSyslog=yes) or push remotely via systemd-journal-remote. For most setups, a Vector/Fluentd/Promtail sidecar reading the journal is the simpler path.

Cleaning up

journalctl --disk-usage
journalctl --vacuum-time=2weeks
journalctl --vacuum-size=500M
journalctl --rotate

Related