Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
17 commits
Select commit Hold shift + click to select a range
bfa2211
Retention: default-on age bound on the File source .processed archive
wshallwshall Oct 27, 2026
778d85e
test: type the processed-archive store fixture so the tests mypy leg …
wshallwshall Oct 27, 2026
ad2ef8f
processed sweep: guard every configured inbound, fold case, delete by…
wshallwshall Oct 27, 2026
261ac03
processed sweep: identity guard over every protected dir, ctime keying
wshallwshall Oct 27, 2026
2539a23
docs: keep PHI.md's spill-dir sentence and section 8 row bound to the…
wshallwshall Oct 27, 2026
e758317
processed sweep: refuse an archive whose opened identity is unreadable
wshallwshall Oct 27, 2026
eda938a
docs: say what moves ctime, and the Windows failed-stamp case, for .p…
wshallwshall Oct 27, 2026
454b8d5
Merge branch 'main' into b447-1188-processed-age-bound
wshallwshall Oct 27, 2026
6c775f6
processed sweep: guard every shard's inbounds, File outbounds, fail c…
wshallwshall Oct 28, 2026
c83ff49
docs: the .processed sweep guards every shard, File outbounds, unread…
wshallwshall Oct 28, 2026
ded823e
processed sweep: re-read an unreadable guarded dir per archive, resol…
wshallwshall Oct 28, 2026
90b1b6e
processed sweep: inode 0 is unresolved, no futile re-read, undeclared…
wshallwshall Oct 28, 2026
e51835e
processed sweep: memoise only a guarded-walk dir that has an identity
wshallwshall Oct 28, 2026
22733c7
Merge remote-tracking branch 'origin/main' into b447-1188-repair
wshallwshall Oct 28, 2026
5d93184
processed sweep: pin only directory settings for sibling shards, not …
wshallwshall Oct 28, 2026
46a449c
processed sweep: guard the engine's own directories; claim a shared d…
wshallwshall Oct 28, 2026
4cda7e1
processed sweep: guard the [logging].file directory too
wshallwshall Oct 28, 2026
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 4 additions & 2 deletions docs/CONFIGURATION.md
Original file line number Diff line number Diff line change
Expand Up @@ -1102,8 +1102,9 @@ disposition, and the audit trail stay intact — the Mirth Data-Pruner pattern);
— `messages.metadata` is nulled in the same statement as the body (ASVS 14.2.7). The raw `[retention]` windows still default to
`0`/`""` = keep/off (at least three keys ship on, and none of them purges anything: the `min_free_disk_mb` free-space floor, the `audit_log_warn_rows` growth warning and the hourly `wal_checkpoint_seconds` WAL truncate), **but `serve` applies a posture gate on top of them, so retention is *not*
opt-in on a PHI instance**: each *unset* window that carries an auto-bound —
`[security].delete_message_bodies_after_days`, `[retention].dead_letter_days` and
`[retention].reference_snapshot_days` — is **defaulted to 30 days** at startup, under **both**
`[security].delete_message_bodies_after_days`, `[retention].dead_letter_days`,
`[retention].reference_snapshot_days` and `[retention].processed_archive_days` —
is **defaulted to 30 days** at startup, under **both**
`[security].enforcement` dials, and the defaulted settings are named on stderr. A window set
**explicitly to `0`** is not defaulted: that **refuses to start (exit 2)** under `enforce`, and warns
under `warn`. This paragraph used to state the opposite split — refusal under `enforce`, auto-bound
Expand Down Expand Up @@ -1147,6 +1148,7 @@ entry a Handler still reads. That stays true until state has an eviction key tha
|---|---|---|---|
| `messages_days` | | | **→ moved to `[security].delete_message_bodies_after_days`** (ADR 0118) — set it there; no longer accepted in `[retention]`. |
| `dead_letter_days` | int | `0` | past N days, null the bodies of **dead-lettered** rows at **every stage**. A dead `ingress` or `routed` row carries the whole raw body, so the purge reaches it as well as a dead outbound row (BACKLOG #1188). This is their own window, because a dead row stays replayable until purged. Unset or `0`, this global window meets the startup posture gate described above this table; `0` = keep only where that gate allows it. An outbound's own `dead_letter_days` overrides it for that outbound's dead rows only (see *Per-connection overrides* below). An override of `0` meets its own gate, described there. A dead row at any other stage always takes this global window. *Corrected 2026-09-28 (BACKLOG #1186):* this row used to scope the purge to outbound rows, which BACKLOG #1188 made false. |
| `processed_archive_days` | int | `0` | past N days, **delete** the archived copies a File inbound left in its `.processed` directory (`processed_subdir`, written when `after_read = "move"`, the default). Each is a clear-text copy of an inbound payload, so this is the fourth window the startup posture gate described above this table auto-bounds, at the body window (owner ruling A2 of 2026-10-08, BACKLOG #1188). Keyed on **archive** time: the File source sets each copy's mtime as it archives it, so a partner's preserved old mtime cannot age a fresh copy out early; on POSIX the sweep keys on the later of mtime and ctime. The archive move sets ctime, but so does any later change to the copy's metadata (a `chmod`, `chown` or ACL change), and that restarts the copy's window. On Windows a copy whose stamp failed keeps the drop's own mtime and can age out early; the failure is logged as a warning. Regular files one level deep only, never through a link or junction; file content is never read and no file name is logged. Only a File inbound whose listener is bound, and whose `after_read` is `"move"`, is swept, so a stopped or undeployed inbound's archive waits until it runs again. An archive directory that is, or contains, the watch or `.error` directory of any inbound in the configuration (on every engine shard under `serve --shard`) or the target directory of any File outbound, bound or not, or one of the engine's own directories (at least the directories holding the store, key and TLS files the settings name, the config directory, and any `*_dir` setting or the backup `destination`, an archive inside one of those last two counting too), is skipped with a warning on every pass. The sweep deletes nothing in at least these further cases: for every connection, when one connection's directory settings cannot be resolved (a missing `env()` value); and for one connection, when a directory it must not reach exists, or may, but its identity cannot be read under that connection's identity (an unreachable share, a refused `stat`). Each is logged as a warning on every pass. Every regular file in the directory counts as an archive, so give `processed_subdir` a directory of its own. `.error` is never swept. `0` = keep only where the gate allows it |
| `allow_unbounded_phi` | | | **→ moved to `[security].allow_keeping_phi_indefinitely`** (ADR 0118) — set it there; no longer accepted in `[retention]`. |
| `state_max_age_days` | int | `0` | past N days, **delete** transform-state entries (ADR 0005) last written before the cutoff — keeps the in-memory state cache + table bounded. A simple global age purge (by `set_at`); per-namespace policy is a follow-up. `0` = keep |
| `connection_event_retention_hours` | int | `0` | past N **hours**, **delete** `connection_event` rows (the `[diagnostics]` #46 transport/lifecycle log — high-volume under a connect-per-message sender or a probe storm, so its own short window in **hours**, not days). `0` = inherit the `messages_days` body window (the ADR 0021 §7.5 default). |
Expand Down
Loading
Loading