Log rotation that does not lose data

Log rotation looks simple and has one failure mode that is completely silent: the rotated file keeps growing and the new file stays empty, and nothing reports an error.

Why it happens

A process opens its log file and holds the descriptor. Rotation renames the file. On Unix, renaming does not affect an open descriptor — the process is writing to an inode, not a name. It carries on writing to the renamed file quite happily.

The new file is created, stays at zero bytes, and the “rotated” file grows forever. You discover it when the disk fills, or when you go looking for today’s logs and find them empty.

The two correct approaches

Signal the process to reopen. Most daemons reopen their logs on SIGHUP or SIGUSR1. This is clean and loses nothing:

postrotate
    systemctl kill -s USR1 --kill-who=main myapp.service 2>/dev/null || true
endscript

copytruncate. Copy the file, then truncate the original in place. The descriptor stays valid. Simple, requires no cooperation from the program, and can lose the lines written between the copy and the truncate. Acceptable for logs where a few lines do not matter; not for audit trails.

delaycompress, and why

Without it, the rotated file is compressed immediately. Anything reading that file — a log shipper, an intrusion detection agent — finds it has become a .gz mid-read. delaycompress leaves it uncompressed for one cycle, which removes the race entirely.

Verify rather than assume

The failure is silent, so test it:

logrotate -d /etc/logrotate.d/myapp     # dry run, changes nothing
logrotate -f /etc/logrotate.d/myapp     # force it
# then generate activity and confirm the NEW file grows
wc -l /var/log/myapp.log
stat -c %s /var/log/myapp.log.1         # must not increase

That last check — confirming the rotated file has stopped growing — is the one that actually proves it worked.