[wshallwshall]

Merge danger

Medium. This adds a default-on retention control that deletes files from File inbound .processed archive directories once they pass the age bound. It is a security and PHI control, so it gets its own PR. It shares files with several open engine PRs: at least 2564, 2565 and 2562 (docs/PHI.md, docs/CONFIGURATION.md, config/settings.py), and others touching settings.py, engine.py and wiring_runner.py. git merge-tree against origin/main at open time was clean. A covering code review of the last two commits (261ac03, 2539a23) is running. The qa label goes on once it reports.

Evidence

mypy is clean on all four legs at the head:

• linux: 364 files;
• win32: 345 files;
• tests: 1430 files;
• scripts/asvs: 6 files.

ruff is clean. Targeted pytest runs:

• the retention and file-surface sets: 708 passed, 19 skipped;
• -k "file or retention or wiring_runner or engine_start": 3281 passed. Its one failure (test_phi_at_rest_inventory) was fixed in 2539a23, and that file now passes;
• -k "phi or doc_drift or configuration or docs": 2140 passed, 93 skipped.

The full suite was not run. Mutation checks: removing the stamp, removing the overlap guard, the read-only-delete test, and the identity-guard test each turned tests red.

BACKLOG #1188: research an honest pass for ASVS 14.2.7 -- the three classified PHI tiers no sweep reaches, under the 2026-07-30 no-auto-bound ruling

This implements owner ruling A2 of 2026-10-08: a default-on age bound on .processed, added as a fourth auto-bound window at the body window and keyed on archive time. .error deletion is out of scope; it was never ruled. Batch 447. Source branch b447-1188-processed-age-bound, head 2539a23f6a5da8596ca5b412f1b33eb403281980.

The setting and the default:

• New setting [retention].processed_archive_days, added as a fourth window in PHI_RETENTION_WINDOWS with auto_bound_days=30, the same as the body window.
• An unset value becomes 30 on a PHI instance through the existing gate. An explicit 0 is refused under enforce unless allow_keeping_phi_indefinitely is set. MIN_PHI_RETENTION_WINDOWS goes from 10 to 11.
• Manager decisions: the default is a fixed 30 days rather than tracking an operator-set body window, and stopped or undeployed File inbounds are not swept (a documented residual).

Archive-time keying:

• FileSource._move(stamp=True) sets the pinned file's mtime to now just before the move into .processed.
• POSIX does this with UTIME_NOW on the directory fd, without following links, and the sweep keys on the later of mtime and ctime. Windows sets FILE_BASIC_INFO through ReOpenFile.
• A failed stamp logs a warning and archives anyway.

The sweep:

• FileSource.prune_processed_archive runs off the event loop through _run_fs. It deletes regular files one level deep, never following a link, and deletes through a handle on Windows.
• The directory is opened through _open_dest and checked by identity.
• It refuses an archive directory that is, or contains, any configured inbound's watch or .error directory. The check compares file identity over every protected directory and its parents, and folds case.
• A pass is bounded by max_pass_seconds, cancels between files, and logs counts only, never file names.
• The audit row gains processed_archive_days and processed_archives_deleted.

Docs: docs/PHI.md (section 2 row split, a section 8 row, the purge table) and docs/CONFIGURATION.md (retention section only).

Legs to read: the linux pytest leg, because test_posix_ages_a_copy_from_its_ctime and the POSIX dir_fd paths only ran their types on Windows. Also the mypy legs and windows-service-smoke.

Proposed ledger banner: "ENGINE BUILT 2026-10-27: owner ruling A2's .processed age bound; a vault re-score of the cell is owed".

Review findings declined, with reasons:

• (a) A long sweep on a credentialed source holds its single credential thread. max_pass_seconds bounds it, and batching it would be a redesign.
• (b) The stamp happens before the claim, so a failed claim leaves a new mtime. That is an ordering cost, not lost data.
• (c) An archive inside another inbound's recursive watch tree is not refused. That setup is already broken.
• (d) File outbound directories are not in the guard list.
• The POSIX unlink keeps a two-system-call window, the same residual _pin_confined documents.

Unfiled follow-ups, for the Lander to file as rows: (c) and (d) above.

Confidence Score: 4/5

Fix the cross-engine-shard directory guard before merging; the new sweep could delete another inbound's unread files.

Findings

1.  Other engine shards lose protection ▶

Fix with agent prompt

### Issue 1
messagefoundry/pipeline/wiring_runner.py:3062-3070
`archive_guard_dirs()` only sees this engine shard's inbounds because `filter_registry_for_shard()` removes the others before the runner receives the registry. If inbound A's `processed_subdir` equals inbound B's watch or `.error` directory on another engine shard, B is missing from both guard lists. A's new sweep would then delete B's aged files, including drops B has not read.

Preserve the full configuration's guarded directories before filtering, and test this case across two engine shards.

---

For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.

Summary

Adds a fourth default-on, 30-day retention window for File inbound .processed archives.

• File inbounds remove old .processed archives using an archive-time age limit.

Acknowledged scope and trade-offs from wshallwshall:

• A long sweep holds the source's credential thread; the stated reason is the pass time limit and avoiding a batching redesign.
• Stamping before a failed archive claim changes the drop's mtime; this was accepted as an ordering cost.
• Archives inside another inbound's recursive watch tree were declined because that setup was described as already broken.
• File outbound directories were explicitly left outside the guard list.
• The POSIX two-call unlink window was accepted as the existing documented limit.
• Stopped or undeployed inbounds are intentionally not swept.

Diagram

%%{init: {'theme': 'neutral'}}%%
flowchart TD
  Config[Full inbound configuration] --> Filter[Filter for this engine shard]
  Filter --> Runner[RegistryRunner]
  Runner --> Archives[Bound processed archives]
  Runner --> Guard[Guard directories from filtered inbounds]
  Archives --> Sweep[Delete aged archive files]
  Guard --> Sweep
  Other[Other engine shards' watch and error directories] -. Missing from guard .-> Guard

Loading

_{Reviews (1) · Last reviewed commit: "docs: keep PHI.md's spill-dir sentence a..." · Reviewed by Greptile}

[wshallwshall]

QA (BACKLOG #1188), rounds 1-2: code-review skill at level xhigh. Round 1 nonce round-a418729a, round 2 nonce round-7dbeb8c6; korus review_tag.py exit 0 both; tag verbatim both rounds: "xhigh effort -> 10 inline angles -> dedup (no verify) -> sweep -> <=15 findings". Inline review with no verifier pass, per the tag. Round-2 repairs (261ac03) and the PHI.md fix (2539a23) are not yet covered; a covering review is running and its QA line will follow. qa label withheld until then.

[greptile-apps]

Other engine shards lose protection

archive_guard_dirs() only sees this engine shard's inbounds because filter_registry_for_shard() removes the others before the runner receives the registry. If inbound A's processed_subdir equals inbound B's watch or .error directory on another engine shard, B is missing from both guard lists. A's new sweep would then delete B's aged files, including drops B has not read.

Preserve the full configuration's guarded directories before filtering, and test this case across two engine shards.

Prompt To Fix With AI

This is a comment left during a code review.
Path: messagefoundry/pipeline/wiring_runner.py
Line: 3062-3070

Comment:
**Other engine shards lose protection**

`archive_guard_dirs()` only sees this engine shard's inbounds because `filter_registry_for_shard()` removes the others before the runner receives the registry. If inbound A's `processed_subdir` equals inbound B's watch or `.error` directory on another engine shard, B is missing from both guard lists. A's new sweep would then delete B's aged files, including drops B has not read.

Preserve the full configuration's guarded directories before filtering, and test this case across two engine shards.

---

For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.

[wshallwshall]

QA (BACKLOG #1188), covering review of 261ac03 and 2539a23: code-review skill at level xhigh, nonce round-77530dba, korus review_tag.py exit 0, tag verbatim: "xhigh effort -> 10 inline angles -> dedup (no verify) -> sweep -> <=15 findings". 15 findings; one real regression fixed (identity check fail-closed again) plus doc/test fixes in e758317 and eda938a. Those fix commits reviewed in round-73a302db, review_tag.py exit 0, same tag. Every commit on the branch now has review coverage. Known defects shipped as critic notes (round cap): the ino-0 refusal stops the sweep on a volume with no file IDs, or on an ACCESS_DENIED stat fallback, and the docs do not list that skip (Manager decision: keep fail-closed, the strictly safer option); an unstat-able guarded dir adds no identity; the inventory pin is name-only; PHI.md still says 'cannot be resolved' skips the whole sweep and the section 8 row lacks the Windows caveat. Declined findings and reasons are in the Builder reports; unfiled follow-ups: POSIX ctime can defer deletion under recurring metadata touches; outbound dirs unguarded; archive inside a recursive inbound tree. Linux pytest leg must be read (POSIX ctime test skipped on Windows).

[wshallwshall]

Lander (local_728989d9), 2026-10-28. HELD for author fixes (adversarial review: the strictly safer option). Main never deletes from .processed; this PR adds a default-on 30-day sweep, so each guard gap is a NEW irreversible-deletion risk:

1. Cross-engine-shard guard (Greptile P1 on 454b8d5): archive_guard_dirs() reads self.registry.inbound, which under serve --shard is filtered to this shard's inbounds (pipeline/sharding.py:194 filter_registry_for_shard). An archive dir equal to another shard's watch dir would have that shard's aged drops deleted, and sharding is a supported topology. Guard against the full registry or refuse.
2. (d) File outbound directories are not guarded: an overlapping processed_subdir would delete files awaiting partner pickup. Guard them.
3. An unstat-able guarded dir adds no identity and fails open on the alias check; make it fail closed, consistent with the ino-0 refusal.
   Filed as rows, not holding: (c) archive inside another inbound's recursive watch tree, and the POSIX ctime deferral (fails toward keeping files). Each fix only widens what the sweep refuses. The new head needs its own review proof.

[wshallwshall]

Manager 784448: Lander hold addressed. New head e51835e, a fast-forward from 454b8d5, checked with ls-remote.

Fixes (each one only widens what the sweep refuses):

1. Engine shards: the guard reads every inbound in the full config, not just this shard's, so a sibling shard's watch and .error directories are protected.
2. File outbounds: each File outbound's target directory is in the guard set. The guard resolves only the directory settings, so a missing env() value in another setting does not switch the sweep off.
3. A guarded directory that cannot be read, or whose stat reports inode 0, is unresolved and the sweep refuses (fail closed). Windows network-path errors 53 and 67 are not treated as absent.

Tests: 8 new tests seen failing on 454b8d5 and passing after. The inode-0 ancestor test was seen failing on 90b1b6e.

QA (covers the final head): code-review skill at level xhigh, nonce round-e6339e2c, on e51835e against origin/main. korus review_tag.py exit 0. Tag verbatim: "xhigh effort -> 10 inline angles -> dedup (no verify) -> sweep -> <=15 findings". 15 findings. Inline review, no verifier pass, per the tag. Earlier rounds: round-ea84af76, round-674132d1, round-a64cbb92 (that last one found the inode-0 regression fixed in e51835e).
Checks: ruff check and format clean; mypy linux, win32 and tests legs clean; targeted pytest 174 passed, 2 skipped (retention, sharding, file confinement). The full suite was not run locally.

Open findings that change behaviour or weaken a control, none introduced by the repairs. For the Lander to judge and file:

• Fails toward keeping files: a guarded directory readable only under another connection's credential stops the sweep for every uncredentialed archive, while startup still reports it as bounded. A shared archive is marked swept before its sweep runs. A future mtime never ages out. The 30-day default does not follow a shorter body window.
• Could delete or alias: an archive inside another inbound's recursive watch tree (already filed as row c). POSIX autofs ENOENT, or two mounts of one export with different device numbers, can pass the alias check. The store, log, backup, TLS-key and config directories are not guarded. A leave/delete inbound's own processed_subdir is not guarded. archive_guard_dirs reads the registry without the reload lock.
• Other: on Windows a failed stamp lets the next pass copy the file; the drop is stamped before its move succeeds; a long sweep blocks intake for a credentialed inbound; non-OSError exceptions escape the retention pass and lose its audit row.