Arya Somu, 2026. Two completed, independently audited studies of reward variation in Qwen3-4B expression repair. This version adds the nine-worker, three-seed mechanism study; earlier development evidence remains separate.
Nine fresh A40 policies crossed success-only (A), full S-C/1024 penalty (B), and the same penalty with a complete homogeneous-success optimizer skip (C) with seeds 103/211/307. Each ran 64 groups of four trajectories and evaluated 96 previously untouched tasks twice. The primary B-C point difference was -29.564236 tokens (-1.804218% relative to C), with descriptive 95% interval [-76.662413,+15.480469]. Accuracy was +3.993056 percentage points, interval [-0.520833,+8.506944]. All three seed token differences were negative, but the registered 10% magnitude and entirely negative interval conditions failed. No promising cost response was established; a smaller effect is not ruled out.
Total optimizer updates across three seeds were A 96, B 174, C 95. All 365 active updates were clipped; C masked 97 homogeneous groups. The intervention changes Adam history and future trajectories, not just a single isolated gradient component. Three seeds, synthetic tasks, frequent token ceilings and descriptive intervals limit interpretation.
- Paper PDF, editable LaTeX, shorter summary, and full derivations.
- Executed method, latest results, complete scientific evidence, and derived chart/inspector data.
- Interactive visualizer and repository companion.
- Public scope, paper reference map, and file manifest.
Python 3.12 and NumPy 2.1.2 reconstruct the new study without a model or credentials:
python3.12 verify_public_release.py
python3.12 -m pip install numpy==2.1.2
python3.12 analyze_reward_mechanism_public.py --output results/mechanism-reproduced.jsonThe program hashes every original decompressed scientific member, independently reconstructs recorded scoring, rewards/advantages/masks, update support, replay/loss gates, pairing, endpoints and the frozen crossed bootstrap. Qualification and actual tensor hashes are recorded receipts from the private audit, not newly rerun CUDA/tensor tests. Optional plot reproduction uses requirements-analysis.txt and python3.12 build_mechanism_figures.py.
The penalty solved 54/64 at 1,567.890625 mean tokens versus the reused control's 52/64 at 1,556.6875 (+0.719677% tokens); principal intervals span zero. Its 59 versus 30 updates and all raw records remain under historical evidence. Its development tasks and runtime difference are not pooled into the fresh mechanism cohort.
python3.12 analyze_reward_continuation.py --evidence baselines/reward-continuation-2026-10-10 --output results/historical-reproduced.json
python3.12 -m unittest tests.test_reward_retrospectiveThe paper and visualizer link to research-v1.1.0. Earlier immutable releases are preserved. Original code is Apache 2.0; original text/tasks/measurements are CC BY 4.0. Third-party notices apply. No weights, private Git history, credentials or provider account/control records are redistributed. Archived training source does not authorize a new paid run. An adaptive-controller advantage, token-saving guarantee or general coding claim is not established.