From 7ab632e095fd194a7d6dca83715fd3e63b9393a3 Mon Sep 17 00:00:00 2001 From: Aaron Trowbridge Date: Fri, 23 Oct 2026 06:57:14 -0400 Subject: [PATCH] docs(skills): loop-methodology laws from the diraq-esr-realism campaign - debugging: the Vacuous Solve (silent control-drop: bit-identical objective across options = optimizer decoupled from dynamics; seed-objective-must-move check), the Silent Edit (six measured instances of no-op transformations; assert-and-grep discipline), the Silent OOM ((2d^2N)^2 Hessian memory, death-after-init signature), background reaping (setsid) - research: wiring-gate-per-axis in the anti-gaming contract (three measured instances + the telegraph free-validator idiom; director-verifies-analyzer), failure modes: the silent no-op edit + unattended-run laws - verification: edit-application and options-reached rows in Common Failures (an unverified edit is an unverified claim) Provenance: harmoniqs/diraq-esr-realism campaign ledger loops 3-10. User-directed application (session 2026-10-23). --- packages/extension/skills/debugging/SKILL.md | 41 +++++++++++++++++++ packages/extension/skills/research/SKILL.md | 19 +++++++++ .../extension/skills/verification/SKILL.md | 8 ++++ 3 files changed, 68 insertions(+) diff --git a/packages/extension/skills/debugging/SKILL.md b/packages/extension/skills/debugging/SKILL.md index ee5476f7c..d6dc9aeef 100644 --- a/packages/extension/skills/debugging/SKILL.md +++ b/packages/extension/skills/debugging/SKILL.md @@ -79,6 +79,47 @@ Before diving into debugging quantum optimal control code, verify these fundamen - **Common causes:** Timestep too large for operator norms, stiff dynamics, numerical overflow in matrix exponential. - **Investigation:** Reduce $\Delta t$. Check $\|H\| \cdot \Delta t$. Examine which timestep first produces NaN. Verify no division by zero in custom objectives. +### The Vacuous Solve (silent control-drop) — 2026-10, measured +- **Symptoms:** The solver consumes thousands of iterations, the variables MOVE (extracted + knots differ from the seed), but the internal objective/fidelity is **bit-identical to the + seed value across every option, setting, and formulation change** (max_iter, mu_strategy, + member count). Post-solve quality is entirely explained by regularizer drift. +- **Common causes:** The problem's dynamics integrator never received the controls — template + integrator auto-resolution silently resolving to a control-free path (Piccolo 2.2.0 + measured: `SplineIntegrator` auto-wiring left the objective constant); or reading a STALE + `fidelity(qtraj)` (the construction-time ODE solution) that masks movement in the other + direction. +- **Investigation:** Evaluate the objective at the SEED and assert the post-solve value has + MOVED (a descent method cannot end where it started, bit-identical). Pass integrators + explicitly. Verify extracted solutions through independent scorers, never the problem's + internal readouts. Provenance: harmoniqs/diraq-esr-realism 2026-10-23 overnight §4. + +### The Silent Edit (no-op transformations) — six measured instances +- **Symptoms:** An edit reports success but behavior never changes; archaeology shows the + pattern never matched (trailing comma, already-replaced form, reverted block). Worst case: + stacked no-ops make the file's edit history untrustworthy and force a clean rewrite. +- **Common causes:** `str.replace`/`sed` succeed on no-match by design; kwargs that miss + their consumer get swallowed; later edits revert earlier ones through shared patterns; + LLM memory of what landed is not evidence. +- **Investigation/Prevention:** Every programmatic edit ASSERTS its pattern matched + (`assert old in s`) and the critical line is grep-verified after the write; multi-edit + confusion resolves by rewriting clean with git as arbiter; numbers are never hand-copied + between files when a formula can be evaluated (machine-pinned literals, rtol ≤ 1e-9). + Provenance: six instances in harmoniqs/diraq-esr-realism loops 4–10. + +### The Silent OOM (death right after initialization) +- **Symptoms:** A process dies immediately after "optimizer initialization complete" with no + error in any log — the OOM killer SIGKILLs without a traceback. +- **Common causes:** Dense interior-point Hessians over unitary trajectories scale as + (2·d²·N)² — a 36-dim member block at N=21 needs ~25 GB. Member counts are memory decisions. +- **Investigation:** Compute the Hessian memory before launching; prefer small-d blocks. + Check `dmesg`/journalctl for oom-kill records where readable. + +### Background process death at harness boundaries +- **Symptoms:** `nohup … &` jobs are gone by the next interaction; logs end mid-init. +- **Cause:** Sandboxed tool environments reap the process group when the call returns. +- **Fix:** `setsid nohup … &` detaches the session (measured both ways, 2026-10-23). + ### Infeasibility (conflicting constraints) - **Symptoms:** Solver reports "infeasible" or "locally infeasible"; no solution found. - **Common causes:** Control bounds too tight for the target gate, conflicting time and fidelity constraints, target gate unreachable in given time. diff --git a/packages/extension/skills/research/SKILL.md b/packages/extension/skills/research/SKILL.md index f70c64034..262b302cb 100644 --- a/packages/extension/skills/research/SKILL.md +++ b/packages/extension/skills/research/SKILL.md @@ -232,6 +232,16 @@ invariants; impossibility checks on noise denominators; FD sanity gates on deriv quantities; raw artifacts are the evidence, prose is not. Promotion of any result to catalog/status is human-only, always. +**Every measured axis carries its own wiring gate** (2026-10, three measured instances: +loop-3 playback-unwired, loop-6 detuning-axis-unwired, loop-10's S2-objective probe) — an +unwired axis returns 0s (or a unit-slip returns ~0.5) and banks a wrong verdict with every +other gate green. A spec whose axis lacks a positive wiring check (loader reproduction, +monotonicity, a closed-form anchor cell) is an incomplete spec; critics reject on this +class. Bonus idiom: a symmetric ±σ telegraph arm's MEAN equals the closed-form single-node +cost exactly — a free wiring validator for detuning axes. Analyzer findings are verified by +the director before being applied (2026-10, a factually-wrong receipts finding was caught); +a critic's arithmetic is never trusted over the test file's machine-pinned literals. + ## Failure modes to watch - **The thin brief** — a subagent with a fresh context and a vague brief rediscovers @@ -242,3 +252,12 @@ catalog/status is human-only, always. that's an open loop in §5, never an outcome in §2. - **Summary drift** — after each compaction, the audit in §9 is the check that the summary still points HERE rather than becoming a competing source of truth. +- **The silent no-op edit** — programmatic edits that report success but never matched + (six measured instances, 2026-10): every edit asserts its pattern and grep-verifies the + critical line; before `git add -A`, inspect status and account for every path (silent + subagent failures leave partial state that a blind add commits); after multi-edit + confusion, rewrite clean and let git arbitrate. +- **Unattended runs** — plain `nohup` backgrounds die at tool-call exit in sandboxed + harnesses; `setsid` survives (measured both ways). Chain unattended cells through a + driver script; commit the state to the ledger BEFORE the night starts so any compaction + is safe. diff --git a/packages/extension/skills/verification/SKILL.md b/packages/extension/skills/verification/SKILL.md index 2d0453375..bab96757d 100644 --- a/packages/extension/skills/verification/SKILL.md +++ b/packages/extension/skills/verification/SKILL.md @@ -50,6 +50,14 @@ Skip any step = lying, not verifying | Regression test works | Red-green cycle verified | Test passes once | | Agent completed | VCS diff shows changes | Agent reports "success" | | Requirements met | Line-by-line checklist | Tests passing | +| Edit applied | The edit's pattern ASSERTED a match and the critical line grep-verifies post-write | The edit command returning, "it should be in there" | +| Options/kwargs reached their consumer | A probe shows behavior change (tiny `max_iter` moves the reported count) | No error raised on pass | +| Numeric literal in a test/spec | Machine-pinned from the computing formula, rtol ≤ 1e-9 | Hand-copied digits, tolerance-sized rtol that masks drift | + +The edit row is the 2026-10 law (six measured instances in one campaign): `str.replace`, +`sed`, and kwarg passes all **succeed on no-match by design** — an unverified edit is an +unverified claim, and stacked silent no-ops leave a file whose edit history must be +rewritten clean with git as arbiter. ## Quantum-Specific Verifications