review round and refactor - #46
Merged
Merged
Conversation
Reconcile re-enters the desire switch regardless of Failed, and the RetryRequested event announces it; the comment restated that.
The three ShortCircuitsNoOp tests asserted only err == nil, which a version without the guards also returns against the fake client. One interceptor-counted test replaces them and fails when a guard is removed.
…Failed retry to the code
The pseudo-version carries meta.VMLive and the toolbox-name bound; the tag bump follows the cocoon-common release.
The two predicates were the same test with the operands swapped; the shared form now lives in cocoon-common.
This was referenced Sep 21, 2026
…acity advanceMigration polled Migrating silently when the restored main could not be scheduled on the target node. The arm now mirrors the released-seat wake: podUnschedulable surfaces the scheduler message as a Warning Event and bumps cocoon_operator_migrate_unschedulable_total; the set keeps polling.
…ered A pod parked at the current generation left the CocoonSet reading Running/Ready with observedGeneration caught up, so nothing in the status said a pod still ran the old spec. buildStatus now collects the parked pods and the Progressing condition reports them; the Ready condition and the phase are unchanged because the parked pod may well be Ready.
spec.suspend on a set whose hibernate vk-cocoon refuses (a macOS agent) sat in Suspending forever with no Event and no terminal phase. The suspend gate and the release drain now share pollSuspend: the entry into Suspending is stamped on the cocoonset.cocoonstack.io/suspending-since annotation, after three minutes the set reports Failed with a SuspendTimedOut Warning, and the next pass re-enters Suspending with a fresh deadline, the cadence the CR path uses for HibernateTimedOut. The stamp is cleared when the set settles Suspended.
This was referenced Sep 21, 2026
allOwnedPodsHibernated returned the probe error before pollSuspend ran, so an unreachable registry left a set in Suspending behind the requeue backoff with the deadline never evaluated. pollSuspend now takes the probe error: before the deadline it is returned as before, past it the set reports Failed with a SuspendTimedOut Event that carries the error, as the CR path does.
The event said the pod was unschedulable on the current spec.nodeName, but a main recreated mid-migration is pinned by the affinity it was built with, so a retarget while it is still Pending leaves it waiting on the old node. The scheduler message already names the node; the docs no longer claim that a spec.nodeName change moves a pending pod.
…the probe error returns With the registry still failing after a SuspendTimedOut, the set stayed in Failed and every pass reset the deadline stamp, so the timeout could never fire again. pollSuspend now persists Suspending before returning the probe error; the status patch is a no-op once the phase is already Suspending.
Enter Hibernating before returning a registry probe error so initial requests and retries from Failed receive a deadline. Preserve the existing clock during an active attempt and the completed-snapshot fast path.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Whole-repo review round, 2026-09-21. Every production file read in full; the hibernate/CocoonSet flow lens and a test lens run by reader agents, every finding adjudicated against the source. No production behaviour changes on purpose: every behaviour question went to an issue.
Commits
ddd2c16review: drop the stray sentence above markFailed15dd392test: the podpatch short-circuits must never reach the client9a2bb21docs: match the suspend gate, wake short-circuit, sub-agent bind and Failed retry to the codecc7c090build: pin cocoon-common at 5d8d95e and import its CRDs575a9a9cut: vmLive and vmClonedAndRunning give way to meta.VMLiveb85d37fcut: drop the two godocs the predicate hoist left behind1c41772build: bump the docker actions and the golang base imagee450febfix: a migration stuck on an unschedulable target raises MigrateNoCapacity68841b2fix: a dead-lettered pod turns Progressing False with reason DeadLetteredc1abedffix: a suspend that cannot complete reports Failed after suspendTimeout8878375fix: the suspend deadline counts while the registry probe keeps failing3209bc4fix: MigrateNoCapacity names no target nodef82de99fix: a suspend retried after its timeout re-enters Suspending before the probe error returnsc96f256docs: the node a pending migration waits on is read from the pod's affinity5351ba9fix: start hibernation deadlines before returning probe errors2ffce73docs: the hibernate timeout starts on the first pass and counts failing probesFindings
podpatch/podpatch_test.gointerceptor.FuncscountingPatch, including the clear-on-a-clean-pod direction; mutation-checkedhibernation/reconciler.gomarkFailedrestated the recovery pathcocoonset/suspend.goPod.Phase=Failedonly, which vk-cocoon never publishes for a live pod; a macOS agent parks the set inSuspendingforeverpollSuspend; the entry intoSuspendingis stamped oncocoonset.cocoonstack.io/suspending-since, aftersuspendTimeout(3 min) the set reportsFailedwith aSuspendTimedOutWarning and the next pass re-entersSuspendingwith a fresh deadline, theHibernateTimedOutcadence of the CR path; the stamp is cleared onSuspended; the gate still skips only kubelet-terminal pods since a lifecycle-failed VM may be live;TestReconcileSuspendTimesOutAfterTheDeadline, mutation-checked; Codex round 6 then showed a persistently failing registry probe bypassed the deadline, sopollSuspendtakes the probe error and reportsFailedpast the deadline while returning the error before it; round 7 then showed the retry after a timeout stayed inFailedand reset the stamp every pass, sopollSuspendpersistsSuspendingbefore returning the probe error and the test now covers the second timeout (TestReconcileSuspendTimesOutWhenTheRegistryProbeKeepsFailing)cocoonset/migrate.godocs/cocoonset.mdstep 12 states; rebuilding sub-agents after a migration is a mechanism without an observed needcocoonset/rebuild.go,cocoonset/status.goProgressingcondition readsFalse/DeadLetteredand names the pods; phase andReadyare unchanged because a parked pod may still be Ready;TestBuildStatusReportsDeadLetteredPods, mutation-checkedcocoonset/migrate.go!VMLivearm reusespodUnschedulableand emitsMigrateNoCapacitywith the scheduler message pluscocoon_operator_migrate_unschedulable_total, mirroring the released-seat wake;TestMigrationReportsAnUnschedulableTarget; Codex round 6 showed the first cut named the currentspec.nodeNamealthough a pending pod stays pinned to the node it was built for, so the event carries the scheduler message alone and the docs no longer claim a retarget moves a pending podhibernationFailed phasecocoonset/restore.gopre-marked restore vs image conflicthibernation/hibernate.goFailed, returned beforeHibernatingwas entered, sohibernateTimeoutnever started and the phase starved behind the requeue backoff (found by the owner)setPhase(Hibernating)before the probe error is returned; a no-op during an active attempt, so the running clock is preserved, and the completed-snapshot fast path is untouched;TestReconcileHibernateProbeErrorsKeepDeadlineAcrossRetries(fails on the parent tree withphase = "", want Hibernating);docs/hibernation.mdstates when the timeout starts and that failing probes countDocs:
docs/cocoonset.md(suspend gate scope, wake engage/fall-through conditions, sub-agent bind at creation),docs/hibernation.md(what retries fromFailed, what staysPending),docs/observability.md(which paths treatlifecycle-state=failedas terminal) corrected to the code, including two wording residuals Codex raised;docs/observability.mdanddocs/cocoonset.mdnow carryMigrateNoCapacity,SuspendTimedOut, the new metric and theDeadLetteredcondition.Cut-list (report only, nothing applied)
cocoonset/migrate.govmLiveandhibernation/wake.govmClonedAndRunningare the same predicatemeta.VMLive, which landed on cocoon-common's round branch; this PR pins cocoon-common at that commit's pseudo-version (v0.3.4-0.20260921020807-5d8d95e98bca, fetchable through proxy.golang.org, verified in a Linux container with an empty module cache) and imports its CRDs, so the toolbox-name bound ships too; swap the pin for the tag once cocoon-common v0.3.4 is releaseddelete.goparseVMNamesAnnotationduplicates cocoon-net'ssplitTrimLOC (same counting on both ends)
Per commit (prod adds/dels · test adds/dels): review 0/1 prod (−1 comment); test 37/33; docs 0/0; pin + CRD import (go.mod/go.sum only); VMLive cut −6; build 0/0; MigrateNoCapacity 18/0 · 36/0; DeadLettered 25/6 · 33/2; suspendTimeout 49/7 · 48/0; probe-error deadline 12/7 · 34/0; event wording 1/1 · 2/2; retry re-entry 4/1 · 25/0; CR hibernate deadline 3/0 · 110/0. The seven fix commits add zero comment lines.
Gates
GOWORK=off make fmt-check,GOWORK=off make lintonGOOS=darwinandGOOS=linux: 0 issues.asl ./...on both GOOS: 0 findings (the two advisory forwardershasPhaseDeadlineandvmClonedAndRunningstay KEPT per the 2026-09-12 ledger).go test -race -count=1 ./...: green (five packages).golang:1.27container onc1abedfand again on5351ba9:go build ./... && go vet ./... && go test -race -count=1 ./...green (five packages).docker build --target buildon the new golang digest (linux/arm64) succeeds.golang:1.27container:go build ./... && go vet ./... && go test -race -count=1 ./...green at 2b07c2b (the only change since is a docs sentence).Codex
Rounds 1 to 3 on the committed diff raised five docs wording residuals (Failed-retry scope, missing-pod phase, wake engage and fall-through conditions, the recreated-main case); all reworded. Round 4 verdict on 9a2bb21: "No blockers: this diff converges." Round 5 on the pin and the VMLive switch: one P3 (two godocs left behind by the deleted predicates, removed in the follow-up commit) and "No blockers: this diff converges." Round 6 on c1abedf (the Dependabot fold and the three owner-approved fixes) raised two P2s, the probe-error bypass of the suspend deadline and the misleading target node in
MigrateNoCapacity, both fixed in their own commits. Round 7 on 3209bc4 raised two more: the retry after a timeout never re-enteredSuspending(fixed inf82de99) and the docs told readers to find the waited-on node in a scheduler message that aggregates reasons (fixed inc96f256). Round 8 on c96f256: "No blockers: this diff converges." The owner's5351ba9was verified by a full read, the gates and the container without a Codex round, by the owner's instruction.Follow-ups filed
Closes #42, #44, #45; #43 closed by decision. Dependabot #47, #48, #49 and #50 are folded into
1c41772and closed. The committed CRD copy underconfig/crd/basesalready carries cocoon-common's toolbox-name bound (imported incc7c090); once cocoon-common v0.3.4 is tagged, the next dependency-bump commit swaps the pseudo-version for the tag and re-runsmake import-crds.