[frahlg]

Problem

On 13 October 2026 systemd stopped Core on the home box while a full backup was copying. The process exited before its deferred cleanup ran. It left /dev/shm/ftw-backup-4218622654.raw (1.37 GB) and its -journal, and nothing removes them before a reboot. On 16 October backupScratchFile found too little free space in /dev/shm, so the raw copy went to the SD card. The backup slowed to about 2,000 rows/s and hit the two-hour liveBackupTimeout. #1670 explains that failure and logs it; this change removes the cause.

Change

At start, after it resolves the backups folder and before it can accept a backup, Core removes scratch that stopped backups left:

• in /dev/shm: ftw-backup-<digits>.raw and ftw-backup-<digits>.raw-journal, regular files only;
• in the backups folder: .ftw-backup-stage-<digits>/ and .ftw-backup-verify-<digits>/ folders, and unfinished .ftw-full-backup-*.ftwbak.tmp archives.

A run stopped while packing or verifying leaves the last two on the data disk, often larger than the raw copy, so they are included. Nothing else matches: not published .ftwbak archives, their .verified.json, .ftw-pre-restore-* safety copies, symlinks, or names with other suffixes. Core logs each removed path with its size in bytes.

How Core tells a stopped backup from a running one

A backup holds a shared flock on each folder where it keeps scratch: the folder of the raw copy (/dev/shm, or the stage folder when it falls back), the output folder for the whole of backup.Create, and the folder a verification works in. The kernel drops a flock when the process exits, also on SIGKILL. At start Core asks for the same lock exclusively and does not wait. If it gets it, no backup is running in that folder, so every matching entry is a leftover. If not, a running backup may own them, and Core keeps all of them and says so in the log.

That protects an ftw-backup create run or a second Core on the same host. The lock is on the folder itself, so no lock file is left anywhere. It is never on a SQLite file: flock and SQLite's fcntl locks are independent on local Linux file systems, but NFS emulates one with the other.

Where Core cannot take the exclusive lock, it keeps everything; Windows, which has no flock, never removes anything. A backup that cannot take its shared lock still runs.

Overlap with #1670

#1670 edits backupScratchFile and adds backupShmDir. This change stays out of both. Its only line in store.go is two lines below #1670's hunk, with unchanged lines between them. RemoveStaleBackupScratchInShm names /dev/shm itself. Whichever PR lands second should point it at backupShmDir; that is a one-line follow-up.

#1615 also edits archive.go, in Inspect; the hunks do not touch.

Tests

• TestKilledBackupScratchIsRemovedAndLiveScratchKept starts a helper process that holds a folder and writes ftw-backup-2.raw. While it runs, the cleanup keeps its file and journal. After SIGKILL, the cleanup removes both, reports 4,608 bytes and logs the path with bytes=4096. Five decoys with similar names survive.
• TestRawCopyFolderIsHeldWhileCopying finds the backup's own raw copy (beside the destination, or in /dev/shm through /proc/self/fd) at each progress report and checks that its folder is locked.
• TestRunningBackupKeepsItsScratch runs the cleanup at every progress step of a real Create and then of a verification. It removes nothing, and both finish.
• TestStaleScratchIsRemovedAndArchivesKept removes a stage folder, a verification folder and an archive temp (6,000 bytes) and keeps archives, their verification records, a restore safety copy and unrelated files.
• Removing any of the three holds makes its test fail; I checked each.
• On Linux (golang:1.26 container, arm64) the four tests pass with a 512 MB /dev/shm, where the raw copy goes to tmpfs, and with 64 MB, where it falls back to the stage folder. With the raw-copy hold removed, the Linux run fails with nothing holds /dev/shm while /dev/shm/ftw-backup-3714948555.raw is written.
• make verify passes on macOS arm64. GOOS=windows builds and vets state and backup.

No command ran on the home box. Its leftover goes away when it first starts a Core with this change.

🤖 Generated with Claude Code

[chatgpt-codex-connector]

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review	Status	Commit	Review trigger
📝 Code Review	✅ Completed 2026-10-16T17:23:39.144941Z	823509f	PR opened

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.