Repository navigation
fix(examples): reserve exits 2/10/11 and make falsifiers reach behavioral checks - #479
Merged
Merged
Conversation
…oral checks Exit codes are how smoke and the falsifiers tell which check fired. Twenty examples spent argparse's 2 or the render gates' 10/11 on their own checks, so those contracts could never be falsifier targets or were misattributed; gltf-skin-roundtrip's gates even returned the same 10/11 as its armature and bone checks. exit-pre-sidecar's --force-run returned 2 on both its expected and its unexpected branch (now 3 and 4). check_exit_code_readme now rejects a literal 2/10/11 at an example's exit site and any 2/10/11 table row that is not the reserved meaning or adds "also" (#473). Twelve falsifiers stopped at a read-back of the value their own flag had just written, so the behavioral check was never reached and could be deleted with smoke still green. Each read-back is gone or runs after the behavioral check as a diagnostic; shader-node-group gains a tiny Cycles probe in a throwaway scene and text-version-stamp compares evaluated glyphs against a reference TextCurve. Every row was re-proven by deleting its behavioral check: the falsifier then misses its expected exit (#469). png-exr-alpha no longer sets scene image_settings that Image.save() ignores, inside a silent except. gn-sdf-remesh gains --no-set-material, swatch-grid gains --invert-engine-id and --no-specular-shim and asserts the specular value. Bulk per-element writes become foreach_set / foreach_get, and check_example_rules covers .loops/.edges/.data on mesh and attribute layers (#474). Signed-off-by: TMHSDigital <tmhospitalitystrategies@gmail.com> Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
TMHSDigital
force-pushed
the
fix/example-exit-codes-falsifiers
branch
from
October 7, 2026 17:45
83a88fc to
98fbd22
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #473
Closes #469
Closes #474
Binaries (local,
.scratch/): Blender 5.2.1 LTS (blender-5.2.1-windows-x64, reports5.2.1 LTS) and Blender 4.5.11 LTS (blender-4.5.11-windows-x64, reports4.5.11 LTS). Every run was--background.#473: 2, 10 and 11 are reserved in examples/
--inject-partnow goes throughp.error(argparse's own 2).EXIT_FRAMING.--force-run: exits 3 whenexit_preis absent (the expected 4.5 outcome) and 4 when it is unexpectedly present. Both used to be 2. "Sidecar env unset" moved from 1 (FATAL) to 5.tests/check_exit_code_readme.py, examples/ only:tests/test_exit_code_readme.py::ReservedCodes. The rule is documented in CONTRIBUTING § Exit codes.#469: falsifiers now land on a behavioral check
Each read-back of a value the falsifier itself wrote is either removed or moved after the behavioral check as a diagnostic.
--muteaim dot=0.320863 (71.28°)--threshold-highhalo+1px=0.000--same-heightZ-extent 1.000000 != height 2.0--no-capsuse_fill_capsverts=1020 faces=1008 != 1044/1028--point-domainattr.domainsized 1009, expected loop count 3960--skip-linkreceiver_collectionhero/decoy 1.10x (min 3.0x)--unlink-skyzenith luma 0.0400 < 0.05--no-contouruse_contourstrokes=234 points=1277(gates 240/1320)--same-tintmin_pairwise_sep=0.001(happy path 0.139)--wrong-bodybody893/873 != reference 403/387--same-basedistinct_regions=4--same-axispreset_z=2.4600 zup_z=2.4600Notes on individual rows:
--outputstill is untouched. It is a 240×80 Cycles render at 16 samples, about 2 s.same_axisbranches are gone fromcheck(). The orientations check now runs right after the preset-stands check.How the deletion column was produced: each mutant replaced the behavioral check's
return Nwithpass, built by.scratchscriptmutate.py. In every case the falsifier exits something other than its catalogexpect_exit, sorun_catalog.pywould go red.#474
save_and_reload(img, "OPEN_EXR"). A comment explains thatImage.save()writes the image's ownfile_formatand ignores sceneimage_settings, and that section 4'ssave_renderis the path that honors them. The silentexcept TypeError: passis removed.--no-set-materiallands on 6 (new catalog falsifier).--invert-engine-id(5) and--no-specular-shim(7). The specular socket and its value are now asserted. The README no longer says smoke skips--output.--zero-uvnow usesuv.data.foreach_set.local_extentnow usesforeach_get.me.edgesnow usesforeach_get/foreach_set. The widened lint found it.tests/check_example_rules.py: a new separate function,check_widened_bulk_writes, covers.loops/.edges/.datawhen the iterated expression is a Mesh collection or an attribute/UV/color layer, with bmesh excluded.:606 uv.data).--same-axisdocstring reference is gone, and the same wording is fixed in the other rewritten docstrings.Evidence
run_catalog.pyover the 32 touched rows (happy paths plus 48 falsifiers): all PASS. swatch-grid needed an absolute--out; a relative--outfails its happy path locally, and CI passes an absolute path.--force-runstays manual, since there is no catalog key for a falsifier that should fire only below a version.exit-pre-sidecar --force-runexits 3 withAttributeError: ... no attribute 'exit_pre'.python tests/run_all.pyreports 31 passed, 0 failed. The examples index and gallery pages were regenerated, since the README exit tables render intodocs/gallery/*/index.html.--outputrender was re-run. The render paths changed only in exit numbers and the gate un-remaps, plus the shader-node-group probe, which uses a separate scene.🤖 Generated with Claude Code