Config loads run one snapshot of the directory; sandbox workers get its bytes (BACKLOG #2596, steps 1 partly, 2, 5)

changelog.d/2596.security.md

@@ -0,0 +1,19 @@
+- **A config load reads the directory once and runs only the bytes it read.** Each start and
+  reload reads its config into one snapshot after the config source's trust check, and the loader
+  compiles every config module, and each `_` helper imported with an `import` statement, from it,
+  so neither runs from `__pycache__`. A
+  released reload checks its approval's digest against that same snapshot before any module runs,
+  so an edit to a module, helper or table made while it loads is neither run nor recorded, and the
+  second digest check after the load is gone. Where the environment value file sits in the config
+  directory's `environments/` folder, `serve` parses the values a reload applies from the same
+  snapshot, so an edit to it during a release never applies under its digest, and a value file
+  created or removed there since the snapshot is refused.
+  Where the values are read from disk instead, a release takes a second digest after that read. See
+  [SECURITY.md](../docs/SECURITY.md). (vault `BACKLOG #2596`)
+- **BREAKING: an unreadable `_` helper refuses a start or reload, even when no module imports
+  it.** Every file the load reads must be readable. (vault `BACKLOG #2596`)
+- **A release that also loosens a config module's permissions is audited as `invalid_config`.**
+  The trust check runs before the digest check, so where it refuses, such a tamper is refused
+  there, not as `digest_mismatch`. A release refused as a `WiringError` before the digest check now
+  writes a `config_reload_failed` row carrying `approved_fingerprint` and `approval_digest_check`
+  set to `not_reached`. (vault `BACKLOG #2596`)

docs/CONFIGURATION.md

@@ -1740,8 +1740,7 @@ started together, can each raise it; the alert list folds them into one instance
   still compared, on the fingerprint taken after the load. When both cover the same files and still
   differ, `GET /config/provenance` also has no loaded fingerprint to vouch for until the next
   reload, and the flag toggle rows that process writes are marked `baseline_unchecked`. The two
-  reads miss at least an edit that lands and is reverted between them, and a module the loader runs
-  from a still-valid `__pycache__` bytecode file rather than from its source.
+  reads miss at least an edit that lands and is reverted between them.
 - The check is alert-only. A baseline read that fails or takes over five seconds is logged at
   WARNING, and the start goes on.
 - A start that never checked its config, because its read failed or it took no fingerprint,

docs/DANGEROUS-FUNCTIONALITY.md

@@ -96,8 +96,10 @@ break that Handler rather than protect it.
 
 ## 2. The config loader imports by file path
 
-`config/wiring.py` builds an import spec from a path and executes the module. Two sites do this: the
-loader's own finder, and the direct module load.
+`config/wiring.py` reads the config directory once, compiles each module from those bytes, and
+executes it. Two sites do this: the loader's helper importer, and the direct module load. Neither
+reads a `__pycache__` file, and a sandbox worker runs the bytes the engine read, not the directory
+(vault BACKLOG #2596).
 
 **What it means.** Every `.py` file in your config directory that does not start with `_` gets
 imported at startup, and its top-level code runs. A file you dropped there to look at later is not

docs/SECURITY.md

@@ -2328,31 +2328,55 @@ it and does not compare against it. Only a committed reload or a flag toggle tha
 fingerprint counts. A later reload that fails and rolls back does not mark the row. The check
 runs just before the write, so a change after it is not caught.
 
-**A released reload loads only the config its approval named (vault BACKLOG #2596, step 3).** When
+**A released reload loads only the config its approval named (vault BACKLOG #2596).** When
 dual control holds a reload, the hold's `params` carry `fingerprint`: the ADR 0041 D1 content
-digest of the target directory, taken at request time. An approver sees it in the queue. The
-release checks the directory against it before any config module runs. After a load that
-succeeds, it checks again, before the checks that refuse an invalid graph. It first takes the
-fail-closed digest. Then it compares the digest the `config_reload` row records, if the engine
-took one. After a load that fails as an invalid config, it checks the fail-closed digest once
-more. So an edit that also breaks the graph is recorded as an edit. If any check differs from the
-approved digest, nothing loads past that point and nothing swaps. The running graph keeps the
-environment values it had. A reload applies the values it re-read only with the swap, and a
+digest of the target directory, taken at request time. An approver sees it in the queue. At
+release the config source's trust check runs before the directory is read. It stats paths and
+reads no file's contents. The release then reads the directory once, into a snapshot, and checks
+that snapshot's digest against the approved one before any config module runs. The load compiles
+and runs that same snapshot, and the `config_reload` row records its digest. So the config
+modules, helpers and tables checked are the ones the loader runs, and an edit to them on disk
+during the load is neither run nor recorded. The limits below name at least one import that runs
+from disk instead. Where the environment value file sits in the config directory's
+`environments/` folder, `serve` parses it from the same snapshot, so an edit to it during the load
+does not apply, and a value file created or removed there since the snapshot is refused. Otherwise,
+at least where the value file sits elsewhere or an embedded engine has another value provider, the
+release reads the values from disk and then takes the digest again, which refuses a change made in
+between to a file the digest covers. If the digest differs, nothing loads and nothing swaps. The running graph keeps
+the environment values it had. A reload applies the values it re-read only with the swap, and a
 rolled-back swap puts the old ones back. A dry run checks the values and never applies them. The
 approve answers **422**, the request moves to `failed`, and a `config_reload_failed` row records
 reason `digest_mismatch` with `approved_fingerprint` and `current_fingerprint`. A hold whose
 params carry no digest is refused the same way, with reason `no_approved_digest`. A release that
 cannot take the digest now is refused with reason `digest_unavailable`, and the engine logs the
 cause at WARNING. The requester asks again, and a second approver reviews the new bytes.
 
+**A release refused before the digest check never reaches it.** At least the settings preflight
+and the config source's trust check run before it. So where the trust check refuses rather than
+warns, a tamper that also loosens the permissions on a file it covers, such as a config module,
+is refused as `invalid_config`, not `digest_mismatch`. A `WiringError` refusal made before the
+check, by those two or another step, writes a row carrying `approved_fingerprint`, the approval's
+digest, and `approval_digest_check` set to `not_reached`, and no `current_fingerprint`, because no
+digest was taken. A missing, unlistable or disallowed directory is refused before the check too,
+with its own reason and neither field. A refusal after the digest check passed carries neither
+field either, whether the approved bytes caused it or an input the digest does not cover, such as
+an environment value file outside the config directory or a connection's CA file. The second
+digest above is the exception: it refuses as `digest_mismatch` with both fields, or as at least
+`digest_unavailable` or `not_found`, as the first check does.
+
+**Every start and reload refuses a config directory holding a file the load reads that the engine
+cannot read, a `_` helper no module imports included.** An unreadable file or folder under
+`environments/` is left out, because the load never reads it. A released reload reads strictly, so
+it refuses that too, as `digest_unavailable`.
+
 This digest fails closed, unlike the provenance fingerprint, which skips what it cannot read. A
 file or folder it covers that cannot be read gives no digest at all, never the digest of the rest.
 A directory that is gone when the digest first lists it is refused as missing. So is a name or
 path too long for the filesystem, or a name Windows refuses. A directory removed after that first
 listing is not refused as missing. The digest covers the files it still finds, down to an empty
 bundle. That digest does not match an approved bundle that held files, so the release is refused
 as `digest_mismatch`. A file that goes between its stat and its read gives `digest_unavailable`
-instead. After the load the release checks this digest first, and then the one the row records.
+instead.
 
 At least these requests file no hold, because their release could only be refused. Each is
 answered and audited at once, as an inline reload's refusal is:
@@ -2371,15 +2395,21 @@ unchanged. An ungated reload takes no request-time digest. At least these limits
   integrity seal, so a writer to the `pending_approvals` table who replaces the digest leaves a
   request the gate refuses. The seal does not stop an older sealed copy of the row being put back,
   and a keyless store checks nothing (vault BACKLOG #2592);
-- the digest covers what `config/fingerprint.py` hashes and no more. A planted `__pycache__` file
-  runs under an unchanged digest. So does an edit to an environment value file outside the config
-  directory, or to a `MEFOR_VALUE_*` variable. These gaps are closing step 1 of vault BACKLOG #2596;
-- each check samples the files. An edit that lands after the first check runs its module code
-  before the second check refuses it. An edit reverted while the release loads is not seen at all.
-  Hashing, compiling and running the same bytes is that same step 1;
+- the digest covers what `config/fingerprint.py` hashes and no more. The release reads a value
+  file outside the config directory's `environments/` folder after the digest check, and a
+  `MEFOR_VALUE_*` variable too, so an edit to either changes the values a release applies under an
+  unchanged digest. The second digest samples the files, so where an embedded engine's other value
+  provider reads a value file under the config directory, an edit to it reverted before that digest
+  is not seen. The second digest also refuses an edit to any covered file made after the first,
+  though the load runs the snapshot the first one checked.
+  The loader compiles each config module, and each `_` helper a config module imports with an
+  `import` statement, from the snapshot. A module reached through the ordinary import system runs
+  from disk, a matching `__pycache__` file included, so the bytes that run are not the bytes the
+  digest covers. At least `importlib.import_module`, and a sibling import while the config directory
+  is on `sys.path`, reach it that way;
 - the request-time digest is taken before the route checks its step-up proof;
 - the digest checks that each folder it reads can be listed, and then lists it again. A folder
-  that becomes unlistable between the two is left out of the digest;
+  removed between the two is left out of the digest;
 - a directory moved away during a release is recorded as `not_found`, not as an edit, unless the
   move lands while a digest is reading it;
 - a swap that fails while reconciling outbound connections does not roll those connections back.

docs/SERVICE.md

@@ -713,9 +713,11 @@ trust boundary: anyone who can write a `.py` file there can run code as the serv
       run `namei -l` on every file there that is a link. Each directory those commands list must be owned by root, or by the engine's own
       account for the config directory itself, with no group or other write bit. The shipped
       image's `/config` sits directly under `/`, which is root-owned.
-  - **The cache folder is checked because it runs.** Python's source loader runs a matching
-    `__pycache__\*.pyc` in place of the `.py` beside it and never compares the two. A cache file
-    the check did not read could run under a source it passed (vault BACKLOG #2596).
+  - **The cache folder is checked because a cache file there can still run.** The loader compiles
+    each config module, and each `_` helper imported with an `import` statement, from the bytes it
+    read, so neither runs from `__pycache__`. A module reached through the ordinary import system
+    still can, at least through `importlib.import_module` or a sibling import while the config
+    directory is on `sys.path`, and so can tools such as `python -m` (vault BACKLOG #2596).
   - **At start, `serve` checks two more files the same way, each with the folder it sits in**: the
     service settings file it read (`--service-config`, else `.\messagefoundry.toml`) and the value
     file `environments\<env>.toml`. It refuses to start when another account can write either one

docs/adr/0041-load-path-attestation-and-change-attribution.md

@@ -105,6 +105,20 @@ advisory provenance — the content digest is the integrity anchor.
 (`.IB_OLD.py`, an AppleDouble `._X.py`) is no longer hashed, because the loader no longer runs one; the
 `*.py` set is `config.wiring.config_py_files`, the one rule the loader and ADR 0036's guard share. The
 `v1` text above is kept as the record of what was decided on its date.*
+*Amended 2026-10-27 (vault BACKLOG #2596, step 1): the digest is now of the bytes that ran, and the
+scheme is `mefor-cfg-fp:v3`.* A load reads the directory once, into a snapshot
+([`config/snapshot.py`](../../messagefoundry/config/snapshot.py)), and runs only that: each module and
+`_` helper is compiled by the loader from its snapshot bytes, so a `__pycache__` file is never consulted,
+and the code sets and `connections.toml` are parsed from the same snapshot. The fingerprint folds that
+snapshot, so a reload's audit digest names the bytes it ran, and a dual-control release checks the
+snapshot it then loads (D2). A `[sandbox].mode=subprocess` worker is sent the snapshot and never reads
+the directory. What `v3` changes in what is hashed: a code-set table is matched by its suffix in lower
+case, as the loader matches it, so `codesets/x.CSV` is hashed on POSIX; and each path is hashed, so a
+path that resolves to a file another path names is no longer folded once. **Not changed, and still
+open:** `environments/*.toml` is hashed under the config directory, as before. That is not where a
+serve start reads its value file, which sits under `[environments].base_dir`; binding the value file
+by its resolved path is the remainder of #2596 step 1. The `v2` text above is kept as the record of
+what was decided on its date.
 
 This binds "reviewed-and-merged in git" to "what actually loaded," makes two reloads with different
 on-disk code distinguishable, defeats attribution-laundering (the audit now shows *which code* a given