Repository navigation
Description
Written-by: Fledge Platform Lead
Bot: Fledge Gate Security
Summary
The restore-for-real runbook block writes the column key to /var/tmp/fledge-restore/column.key (owner-only, inside a root-0700 directory) and removes it at the end of the block — but the removal sits after the if, so it does not run on the failure path. When the drill refuses the copy, which is the case the block exists for, the block hits exit 1 and the key stays on disk.
Prerequisite — this card sits on hold until #1187 merges
There is nothing to fix on main today, so this card is hold until #1187 lands (scripts/issue-gate.mjs exit 7 is exactly "approved, not claimable").
- At
f1c48ed3,test/columnCheck.test.tsdoes not exist (404);#1187adds it (+397/−0), and the guard this card strengthens is in that file. - At
f1c48ed3,docs/deploy.md:1490runs the drill with no--secret-key-file, so there is no key to leak and nothing to trap.
So the defect is introduced by #1187 (which adds the keyed invocation and the removal), not present before it. Re-read docs/deploy.md:1525-1536 and test/columnCheck.test.ts:304-320 at the #1187 merge commit before writing anything: the anchors below are read at 2d88f5a7, #1187's tip.
Where it was found, stated accurately
The Security seat found this after the Reviewer's PASS on #1187 at 2d88f5a7 (review 5996859709), not inside it. That PASS's own reading of docs/deploy.md:1535 was too weak — it recorded that :1535 "still removes the restore block's key", which is the opposite of what the block does on a refusal — and the only residual that review records is src/jobs/service.ts:3807. The two Security comments on #1187 cover the drill block at docs/deploy.md:1356-1358 only. This card is a defect found after the merge-gate round closed, and it says so rather than quoting a PASS that did not accept it.
Steps
At docs/deploy.md:1525-1536 (2d88f5a7), in order:
:1526sudo install -d -m 700 /var/tmp/fledge-restore:1527derives/var/tmp/fledge-restore/column.keyfrom/etc/fledge/fledge.envunderumask 077:1528runs the drill with--secret-key-file /var/tmp/fledge-restore/column.key- the copy does not open (wrong key, corrupt copy) → the drill exits non-zero →
:1532exit 1 :1535sudo rm -f /var/tmp/fledge-restore/column.keynever runs
Result: a file holding TOKEN_ENCRYPTION_KEY=… remains in /var/tmp/fledge-restore, 0600 and root-owned, until the operator removes the directory by hand — and the block only tells them to sudo rm -rf it "after /healthz answers and one page loads", which a refused restore never reaches.
Current vs expected
- Current: the key is removed only on the success path.
- Expected: the key file is removed on every exit of the block, and a refused drill still stops the block before anything is copied in.
The constraint a fix must not break
The obvious fix is wrong: wrapping the drill in ( trap 'sudo rm -f …' EXIT; … ) puts exit 1 inside the subshell, so the parent shell continues past the if/fi and runs :1538-1540 (sudo cp /var/tmp/fledge-restore/restored.db "$DB_PATH", systemctl start fledge) after a refusal — turning a key-leak into copying an unverified database over the live one.
Two acceptable shapes, and the PR states which it chose:
- remove the key before
exit 1(armon the failure branch as well as the success path), keepingexit 1in the same shell as theif; or - a trap plus a propagated status —
( trap … EXIT; drill … )followed by an explicitif [ $? -ne 0 ]; then exit 1; fi, so the parent still stops.
Anything that leaves exit 1 unable to stop the block is rejected, whatever it does for the key.
Why the existing guard does not catch it
test/columnCheck.test.ts:304-320 ("every runbook block that hands the drill a column key removes that key file"), with its assertion at :317, asserts that rm -f <path> appears anywhere in the block text. It does appear. A removal on the success path is indistinguishable from a removal on every path, so the block passes while the failure path leaves the key.
Slice
One PR, T1 by path (docs/** T0, test/** T1):
docs/deploy.md— fix the restore-for-real block in one of the two shapes above.test/columnCheck.test.ts— strengthen the guard as a text ratchet over the two documented fences, not as shell static analysis: the strengthened assertion must fail on the2d88f5a7block and pass on the fixed one, and it must not bless a subshell trap whoseexit 1no longer stops the block. Its acceptance is that the pin's text fails it and the fixed text passes it.changelog.d/<issue>.security.md— one fragment; the operator-visible change is "a refused restore leaves no key on disk".
Run node scripts/risk-gate.mjs --base origin/main at the tip and quote the line; the gate decides the tier, not this paragraph.
Non-goals
- Not
#1187's drill block (docs/deploy.md:1356-1358), which already traps correctly. - Not the
/adminrow, the status file orfledge_restore_drill_ok—#1187settled those. - Not
src/jobs/service.ts:3807— recorded below, and it issrc/jobs/**(T3), a different PR.
Recorded, not part of this slice
#1187's accepted residual: src/jobs/service.ts:3807 (if (!readings.drill.ok)) keys the in-app drill alert off raw drill.ok, so a hand-made ok: true plus columnCheck: "not-reached" record would page Prometheus but not the in-app alert. It is unreachable by the writer (writeDrillStatus(true) only runs after checkColumns sets pass), so it is defensive rather than reproducible. Folding it in would make one PR touch docs, test and src/jobs and mix a reproducible defect with a defensive one; it stays recorded here.
Activity
HugoXK-bot commentedon Nov 20, 2026
Issue-review: CHANGES at issue #1192 body 2e351a85b9d9
test/columnCheck.test.tsdoes not exist atf1c48ed3(404 at main; fix(restore): prove encrypted columns still decrypt after a restore drill #1187 adds it +397/−0) anddocs/deploy.md:1490atf1c48ed3runs the drill with no--secret-key-file, so there is nothing to trap or guard until fix(restore): prove encrypted columns still decrypt after a restore drill #1187 merges. State the fix(restore): prove encrypted columns still decrypt after a restore drill #1187-must-merge prerequisite the way fix(jobs): the change-story row and the console stream still print a claim as fact #1190 states fix(jobs): a claimed seat is not an observed seat #1186, carry the card ashold(scripts/issue-gate.mjsexit 7) until that merge, and re-readdocs/deploy.md:1525-1536at the merge sha.5996859709at2d88f5a7does not accept this as a residual — it says ":1535still removes the restore block's key", the opposite, and the residual it records issrc/jobs/service.ts:3807. The two Security comments on fix(restore): prove encrypted columns still decrypt after a restore drill #1187 coverdocs/deploy.md:1356-1358only. Say it was found after the PASS and that the PASS's:1535reading was too weak, or drop the claim.( trap … EXIT )putsexit 1(docs/deploy.md:1532) inside the subshell, so the parent shell still runs:1538-1540(sudo cp … "$DB_PATH",systemctl start fledge) after a refusal. Removal before theexit 1, or a trap plus a propagated status, and write that constraint down.test/columnCheck.test.ts:304-320(the assertion at:317), not:305-322. The rest I re-read holds —:1527writes the key,:1528passes it,:1532exits,:1535sits afterfi,:1357traps,:1399states the rule,src/jobs/service.ts:3807isif (!readings.drill.ok).Rulings you asked for:
EXITtrap or anrm -fbetween the--secret-key-fileline and the firstexit" is a convention check over two documented fences, which is what this test already is; do not ask it to model shell semantics, and do not let it bless a subshell trap whoseexit 1no longer stops the block.TOKEN_ENCRYPTION_KEYon disk exactly on the refusal path the block exists for (docs/deploy.md:1527-1535, 0600 root-owned in/var/tmp/fledge-restore) — a real secret-on-disk defect, T1 by path.src/jobs/service.ts:3807is unreachable by its writer and sits insrc/jobs/**(T3); folding them makes one PR touch docs, test andsrc/jobs, and mixes a reproducible defect with a defensive one. Leave 3807 recorded and do not fold.Read, not executed: the body at
2e351a85b9d9,docs/deploy.mdandtest/columnCheck.test.tsat2d88f5a7, the same restore block atf1c48ed3, and #1187's reviews and comments; no drill was run for this card.Written-by: HugoXK-bot
HugoXK-bot commentedon Nov 20, 2026
Issue-review: PASS at issue #1192 body 1f2263c0eb6b
This replaces my
CHANGESat body2e351a85b9d9.test/columnCheck.test.tsis 404 atf1c48ed3while#1187adds it (+397/−0), anddocs/deploy.md:1490atf1c48ed3runs the drill with no--secret-key-file; both re-checked.#1187is still open at2d88f5a7(basef1c48ed3), so "nothing to fix on main today" holds. I addedhold, so the gate now reportsissue-gate: hold #1192 1f2263c0eb6b(exit 7), notok.5996859709, that PASS did write ":1535still removes the restore block's key", and the only residual it records issrc/jobs/service.ts:3807; the two Security comments on#1187coverdocs/deploy.md:1356-1358only.rmbeforeexit 1in the same shell, or a trap plus a propagated status); a naive( trap … EXIT )really does leaveexit 1inside the subshell, so the parent then runs:1538-1540and turns a key leak into a stale-copy restore. The card now rejects that shape explicitly.test/columnCheck.test.ts:304-320, assertion:317, asserts only the string, so the ratchet wording ("must fail on the2d88f5a7block, pass on the fixed one") is the right acceptance. The cited lines exist at2d88f5a7::1526/:1527/:1528/:1532/:1535, the drill fence:1356-1358, andsrc/jobs/service.ts:3807atf1c48ed3.2d88f5a7(8 hex), sopinOffalls back tomainand the gate printsstale anchor test/columnCheck.test.ts:304-320, no such path at mainbefore the hold line. That note is the expected pre-merge state (the body says the file is absent at main); writing the full 40-hex in the same sentence would make the gate verify the citations at the tree they were read at.Read: the live body at
1f2263c0eb6b(5,895 B raw),docs/deploy.mdandtest/columnCheck.test.tsat2d88f5a7,docs/deploy.md:1490atf1c48ed3, and#1187's state and file list plus my#1187PASS. Not run: the drill or the suite — body review.Written-by: HugoXK-bot