Skip to content

fix(examples): reserve exits 2/10/11 and make falsifiers reach behavioral checks - #479

Merged
TMHSDigital merged 1 commit into
mainfrom
fix/example-exit-codes-falsifiers
Oct 7, 2026
Merged

TMHSDigital merged 1 commit into
mainfrom
fix/example-exit-codes-falsifiers

Conversation

@TMHSDigital

Copy link
Copy Markdown
Owner

Closes #473
Closes #469
Closes #474

Binaries (local, .scratch/): Blender 5.2.1 LTS (blender-5.2.1-windows-x64, reports 5.2.1 LTS) and Blender 4.5.11 LTS (blender-4.5.11-windows-x64, reports 4.5.11 LTS). Every run was --background.

#473: 2, 10 and 11 are reserved in examples/

  • Exit 2 is now argparse only. coincident-vert-weld 2→5, unapplied-scale-gltf 2→5, cross-version-property-delete 2→9, usd-export-evaluation-mode 2→7, gn-zone-iterate 2→7, gp-lineart-contour 2→9, export-preset-axis 2→13, png-exr-alpha 2→14. mesh-hygiene-audit's bad --inject-part now goes through p.error (argparse's own 2).
  • 10 and 11 now mean only the framing and asset-quality gates.
    • car-mirror-symmetry: contract 10/11→17/18, and the gates return their own codes instead of being remapped.
    • gltf-export-roundtrip: 10/11→22/23, same un-remap.
    • gltf-skin-roundtrip: 10/11→21/22. Before this, its gates returned the same 10/11 as the armature and bone checks.
    • soccer-ball-goldberg: 10/11→14/15, framing back on 10.
    • uv-layer-grid: 11→16. gn-modifier-inputs: 11→13. modular-kit-snap: 11→12, and the literal 10 is now EXIT_FRAMING.
    • Other overloads split off: curve-bevel-arc 10/11→12/13, export-preset-axis contract 11 removed and no-file→14, png-exr-alpha 10/11→15/16, vse-cut-list 10/11→13/14. lightmap-uv-channel, triangulate-tangents and vertex-weight-limit: "no file" 10→12.
  • exit-pre-sidecar --force-run: exits 3 when exit_pre is absent (the expected 4.5 outcome) and 4 when it is unexpectedly present. Both used to be 2. "Sidecar env unset" moved from 1 (FATAL) to 5.
  • New lint in tests/check_exit_code_readme.py, examples/ only:
    • It rejects a literal 2/10/11 at an exit site. Gates must return the helper's constant or check call.
    • It rejects an Exit codes row for 2/10/11 that is not the reserved meaning or that contains "also".
    • Unit tests are in tests/test_exit_code_readme.py::ReservedCodes. The rule is documented in CONTRIBUTING § Exit codes.
    • Showcase is out of scope: it keeps its documented collider-11 / remap-20 convention.

#469: falsifiers now land on a behavioral check

Each read-back of a value the falsifier itself wrote is either removed or moved after the behavioral check as a diagnostic.

Example Falsifier Old exit (read-back) New exit (behavioral check) Measured 5.2.1 LTS Check deleted → falsifier (5.2.1 / 4.5.11)
damped-track-aim --mute 7 mute flag 9 evaluated aim dot exit 9, aim dot=0.320863 (71.28°) 7 / 7 (FAIL vs 9)
compositor-glare --threshold-high 3 socket value 4 halo pixels exit 4, halo+1px=0.000 0 / 0
gn-modifier-inputs --same-height 7 readback 8 evaluated Z-extent exit 8, Z-extent 1.000000 != height 2.0 7 / 7
curve-bevel-arc --no-caps 6 use_fill_caps 7 evaluated tessellation exit 7, verts=1020 faces=1008 != 1044/1028 0 / 0
color-attribute-wheel --point-domain 5 attr.domain 6 layer storage size exit 6, sized 1009, expected loop count 3960 1 / 1 (the probe read crashes)
light-link-studio --skip-link 5 receiver_collection 6 linked render ratio exit 6, hero/decoy 1.10x (min 3.0x) 7 / 7
sky-texture-sun-elevation --unlink-sky 6 link check 7 zenith render exit 7, zenith luma 0.0400 < 0.05 8 / 8
gp-lineart-contour --no-contour 4 use_contour 5 evaluated strokes exit 5, strokes=234 points=1277 (gates 240/1320) 7 / 7
shader-node-group --same-tint 6 socket values 8 rendered probe (new) exit 8, min_pairwise_sep=0.001 (happy path 0.139) 6 / 6
text-version-stamp --wrong-body 4 body 4 evaluated glyphs vs a reference TextCurve exit 4, 893/873 != reference 403/387 0 / 0
swatch-grid --same-base 3 material colors 6 rendered regions exit 6, distinct_regions=4 3 / 3
export-preset-axis --same-axis 9 flag branch 9 measured re-import spans exit 9, preset_z=2.4600 zup_z=2.4600 8 / 8

Notes on individual rows:

  • compositor-glare: the falsifier threshold went from 10 to 1000. The rings emit up to strength 18, so a threshold of 10 still bloomed (halo 0.110, which passed the gate).
  • shader-node-group: the probe renders in its own scene, so the --output still is untouched. It is a 240×80 Cycles render at 16 samples, about 2 s.
  • export-preset-axis: the same_axis branches are gone from check(). The orientations check now runs right after the preset-stands check.

How the deletion column was produced: each mutant replaced the behavioral check's return N with pass, built by .scratch script mutate.py. In every case the falsifier exits something other than its catalog expect_exit, so run_catalog.py would go red.

#474

  • png-exr-alpha: the EXR round-trip now uses save_and_reload(img, "OPEN_EXR"). A comment explains that Image.save() writes the image's own file_format and ignores scene image_settings, and that section 4's save_render is the path that honors them. The silent except TypeError: pass is removed.
  • gn-sdf-remesh: --no-set-material lands on 6 (new catalog falsifier).
  • swatch-grid: two new catalog falsifiers, --invert-engine-id (5) and --no-specular-shim (7). The specular socket and its value are now asserted. The README no longer says smoke skips --output.
  • Bulk writes:
    • triangulate-tangents --zero-uv now uses uv.data.foreach_set.
    • unapplied-scale-gltf local_extent now uses foreach_get.
    • gn-instance-grid's sharp-edge loop over me.edges now uses foreach_get/foreach_set. The widened lint found it.
  • tests/check_example_rules.py: a new separate function, check_widened_bulk_writes, covers .loops/.edges/.data when the iterated expression is a Mesh collection or an attribute/UV/color layer, with bmesh excluded.
    • It flags the pre-fix triangulate-tangents (:606 uv.data).
    • It now passes over all 142 example and showcase scripts.
  • text-version-stamp: the stray --same-axis docstring reference is gone, and the same wording is fixed in the other rewritten docstrings.

Evidence

  • Live run, 5.2.1 LTS: run_catalog.py over the 32 touched rows (happy paths plus 48 falsifiers): all PASS. swatch-grid needed an absolute --out; a relative --out fails its happy path locally, and CI passes an absolute path. --force-run stays manual, since there is no catalog key for a falsifier that should fire only below a version.
  • Live run, 4.5.11 LTS: the same 32 rows, 46 falsifiers (version-gated ones SKIP): all PASS. exit-pre-sidecar --force-run exits 3 with AttributeError: ... no attribute 'exit_pre'.
  • Live run, both versions: all 12 deletion mutants above.
  • Local validators: python tests/run_all.py reports 31 passed, 0 failed. The examples index and gallery pages were regenerated, since the README exit tables render into docs/gallery/*/index.html.
  • By inspection only: 5.1 (not run locally). No --output render was re-run. The render paths changed only in exit numbers and the gate un-remaps, plus the shader-node-group probe, which uses a separate scene.

🤖 Generated with Claude Code

@github-actions github-actions Bot added skills examples Runnable smoke-gated examples under examples/ documentation Improvements or additions to documentation labels Oct 7, 2026
…oral checks

Exit codes are how smoke and the falsifiers tell which check fired. Twenty
examples spent argparse's 2 or the render gates' 10/11 on their own checks,
so those contracts could never be falsifier targets or were misattributed;
gltf-skin-roundtrip's gates even returned the same 10/11 as its armature
and bone checks. exit-pre-sidecar's --force-run returned 2 on both its
expected and its unexpected branch (now 3 and 4). check_exit_code_readme
now rejects a literal 2/10/11 at an example's exit site and any 2/10/11
table row that is not the reserved meaning or adds "also" (#473).

Twelve falsifiers stopped at a read-back of the value their own flag had
just written, so the behavioral check was never reached and could be
deleted with smoke still green. Each read-back is gone or runs after the
behavioral check as a diagnostic; shader-node-group gains a tiny Cycles
probe in a throwaway scene and text-version-stamp compares evaluated
glyphs against a reference TextCurve. Every row was re-proven by deleting
its behavioral check: the falsifier then misses its expected exit (#469).

png-exr-alpha no longer sets scene image_settings that Image.save()
ignores, inside a silent except. gn-sdf-remesh gains --no-set-material,
swatch-grid gains --invert-engine-id and --no-specular-shim and asserts
the specular value. Bulk per-element writes become foreach_set /
foreach_get, and check_example_rules covers .loops/.edges/.data on mesh
and attribute layers (#474).

Signed-off-by: TMHSDigital <tmhospitalitystrategies@gmail.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@TMHSDigital
TMHSDigital force-pushed the fix/example-exit-codes-falsifiers branch from 83a88fc to 98fbd22 Compare October 7, 2026 17:45
@TMHSDigital
TMHSDigital merged commit dccca5d into main Oct 7, 2026
13 checks passed
@TMHSDigital
TMHSDigital deleted the fix/example-exit-codes-falsifiers branch October 7, 2026 17:48
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

documentation Improvements or additions to documentation examples Runnable smoke-gated examples under examples/ skills

Projects

None yet

1 participant