Repository navigation
Conversation
#1193) The deploy-lock case proved ownership with a `spawnSync` probe per iteration under a hard 5 s deadline, so a loaded runner could spend the whole budget without ever seeing the holder; the give-up then said only that the holder "never took the lock", a conclusion drawn from an absence. `ci-ok` went red twice on the same sha for this case and green on a third attempt of the same sha. Readiness now arrives on the holder's own stdout, which costs no process per iteration. The deadline is the named constant `LOCK_READY_TIMEOUT_MS`, and the give-up probes once so its message names elapsed ms, poll iterations, the last probe status, the holder's pid and whether it was alive. Every existing assertion keeps its meaning, and `deploy/fledge-deploy-tag.sh` is untouched.
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Written-by: Fledge Platform Dev-1
Fixes #1193
What changed
test/deployAutomation.test.tsonly: one readiness wait, one new case,+159/−13.deploy/fledge-deploy-tag.shis untouched, so production semantics are unchanged —LOCK=${FLEDGE_DEPLOY_LOCK:-/tmp/fledge-deploy-tag.lock}and theflock -nearlyexit 0still stand, and the case still asserts both (:349,:371-372).The readiness wait no longer spawns a process per iteration (
#1193). The holder isflock -n <lock> sh -c 'printf "ready\n"; exec sleep 30': it prints one readiness line only afterflockowns the lock, and the wait resolves on that child's'data'event, so observing readiness costs no fork. The deadline is the named constantLOCK_READY_TIMEOUT_MS(:272, 20 s) whose comment says what it is for;LOCK_READY_TICK_MS(:273) is the state-observation tick, not a probe interval.On give-up the message names what it saw instead of concluding from an absence — elapsed ms, poll iterations, the last probe status, the holder pid and alive/exited:
The probe exists only on that path (
probeLock,:283): zero probes when the holder is ready, one when it gives up.Red evidence, assertion by assertion
The new case cannot run at all against the literal
f1c48ed3file — there is no seam to call — so the red was taken against the base mechanism expressed through the same seam: a throwaway copy of the base file plus awaitForLockReadythat runs the oldspawnSyncprobe loop, 5 s budget and bare message, with the three new assertions as three separate cases. All three red, each on its own assertion:the wait must outlast the old 5 s budget, reported 1000 ms. The old loop measures against its own budget and ends as soon as one probe is refused, so it cannot run past the budget it is given.The input did not match the regular expression /after \d+ poll\(s\) in \d+ ms; last probe status \d+, holder pid \d+ (alive|exited \d+)/. Input: 'the lock holder never took the lock'— the literal defect message from the two CI failures.a readiness wait on a ready holder must not spawn a probe— the old loop probes once per iteration by construction.Green at this tip: the same three assertions pass inside the new case (all three run, none skipped), and the whole file is 15/15.
Assertions kept
The old case is now
asyncand keeps every assertion with its meaning::349(production lock line),:351,:353(two hosts do not share a lock),:371-372(a lock-blocked tick exits 0 and logs nothing),:380-382(another host still reaches its refusal). The restore and rollback cases are untouched (:241-263,:461-467). Noskip, no retry wrapper, noprocess.env.CI/NODE_ENV/version branch, no second attempt inside the case, andtest/deployAutomation.test.ts:3is untouched (spawnSyncis still used bytickand by the give-up probe).One consequence worth naming: each holder in the new case gets its own lock path (
ready.lock), becausekill()on a holder does not release its lock within the same process tick on this host — the next case must not be decided by a kill that has not landed yet.Ran:
npm run checkat1fbb525— exit 0,# tests 3265 / # pass 3265 / # fail 0, lint 11 warnings at the baseline,tsc --noEmitandbuildclean.npx tsx --test test/deployAutomation.test.tsat this tip — 15/15, duration ~34 s.node scripts/risk-gate.mjs --base origin/main—risk-tier: T1; basis: test/deployAutomation.test.ts.Not run:
docs/change and no changelog fragment: this is a test-determinism fix with no user-visible behaviour (the#992exemption).Docs-not-needed: test-only change, the deploy runbook's behaviour is unchanged.Changelog-not-needed: no user-visible change.#1193), Docker image, and any production host.Risk
T1, test-only. Residual: the deadline is still a wall-clock budget, so a runner stalled for longer than 20 s inside the wait still fails — but it now fails with a message that separates "lock never seen" from "probe saw a holder it never heard from", which is the part that cost three CI attempts to learn nothing.
Tip
1fbb525729ef3434506334f051f9fde85a9f14d7, tree6b366ed2b78bb8806c901814fe0170a4729f7e88, basef1c48ed3513a8180b870fae19fb95d32222da44b.