Skip to content

fix(ci): run every documented example falsifier in smoke - #443

Merged
TMHSDigital merged 1 commit into
mainfrom
fix/example-falsifiers-all
Oct 5, 2026
Merged

TMHSDigital merged 1 commit into
mainfrom
fix/example-falsifiers-all

Conversation

@TMHSDigital

Copy link
Copy Markdown
Owner

Closes #362.

What changed

  • Catalog: tests/smoke/catalog.json gains 19 falsifiers on 10 existing rows. These are every non-render falsifier the example READMEs documented but no job ran. The rows' falsifiers lists already supported several entries, so no new key or cron job is needed. They run on every smoke leg, which also satisfies "at least weekly".

    Row Added (expected exit)
    cross-version-property-delete --unset-instead (7)
    gn-zone-iterate --no-offset (4), --unpair-foreach (5), --foreach-main (5)
    gn-sim-fountain --euler (4), --prebake-trap (5), --calc-to-frame (6)
    gn-bundle-roundtrip --mismatch (3), --bypass (4), --mismatch-mark (5)
    exit-pre-sidecar --no-handler, --silent-handler, --wrong-text, --write-in-main (sidecar must fail)
    ngon-triangulate / unapplied-scale-gltf / coincident-vert-weld --skip-triangulate / --bake / --weld (4)
    usd-export-evaluation-mode --evaluation-mode VIEWPORT (4)
    gn-modifier-inputs --api dict (5; min_version 5.2)
  • READMEs: the 10 "Smoke does not pass …" sentences now name these as catalog falsifiers. check_smoke_catalog rejected every stale one until it was fixed. Gallery pages are regenerated.

  • ROADMAP: removed the item that deferred this decision.

  • Left out of smoke on purpose:

    • render-path diagnostics: --falsify (writes a PNG), --close-camera, --probe;
    • path and mode switches: --force-run, --obj, --api rna.

Evidence

Live run: tests/smoke/run_catalog.py over a 10-row subset catalog, on local binaries:

Binary reports PASS SKIP FAIL
Blender 5.2.1 LTS 40 0 0
Blender 5.1.2 39 1 (--api dict, from 5.2) 0
Blender 4.5.11 LTS 27 13 (bundle rows < 5.0, exit_pre < 5.1, --api dict, --subscript-registered) 0
  • --api dict on 5.1.2: run directly, it exits 0 (api=dict blender=(5, 1, 2), gn-modifier-inputs OK). That is why it has min_version 5.2, and the README says so.
  • Local checks: check_smoke_catalog.py and check_exit_code_readme.py pass.

🤖 Generated with Claude Code

Example READMEs documented falsifiers that no job ran. Only one flag per
row was in the catalog, and the weekly sweep reads only showcase/. Any of
those checks could stop failing unnoticed.

Add the 19 remaining non-render falsifiers to their catalog rows:
- cross-version-property-delete --unset-instead (7)
- gn-zone-iterate --no-offset (4), --unpair-foreach (5), --foreach-main (5)
- gn-sim-fountain --euler (4), --prebake-trap (5), --calc-to-frame (6)
- gn-bundle-roundtrip --mismatch (3), --bypass (4), --mismatch-mark (5)
- exit-pre-sidecar --no-handler, --silent-handler, --wrong-text,
  --write-in-main (sidecar must fail)
- ngon-triangulate --skip-triangulate (4), unapplied-scale-gltf --bake (4),
  coincident-vert-weld --weld (4)
- usd-export-evaluation-mode --evaluation-mode VIEWPORT (4)
- gn-modifier-inputs --api dict (5, from 5.2; exits 0 on 5.1)

They run on every smoke leg, so they run at least weekly. The READMEs'
"Smoke does not pass" sentences now list them as catalog falsifiers, which
check_smoke_catalog enforces. The ROADMAP item that deferred this is
removed. Render-path diagnostics (--falsify, --close-camera, --probe) and
mode switches (--force-run, --obj) stay out of smoke.

Closes #362

Signed-off-by: TMHSDigital <154358121+TMHSDigital@users.noreply.github.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@github-actions github-actions Bot added examples Runnable smoke-gated examples under examples/ documentation Improvements or additions to documentation labels Oct 5, 2026
@TMHSDigital

Copy link
Copy Markdown
Owner Author

CI evidence before merge: all checks pass ( 13 pass ). Blender Smoke on PR head: Blender 5.2.2 LTS / Blender 4.5.14 LTS. Summaries (happy path; falsifiers) per leg: 143 passed, 1 skipped, 0 failed;162 passed, 0 skipped, 0 failed;141 passed, 3 skipped, 0 failed;147 passed, 15 skipped, 0 failed;

@TMHSDigital
TMHSDigital merged commit 0b8097d into main Oct 5, 2026
13 checks passed
@TMHSDigital
TMHSDigital deleted the fix/example-falsifiers-all branch October 5, 2026 02:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

documentation Improvements or additions to documentation examples Runnable smoke-gated examples under examples/

Projects

None yet

Development

Successfully merging this pull request may close these issues.

testing: secondary example falsifiers are documented but never run anywhere (cron sweep covers showcase only)

1 participant