From 8c868178b5c2310ad0e34ade086070978b19c659 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?=C3=82ngelo=20Mois=C3=A9s=20Alves?= Date: Sat, 24 Oct 2026 20:26:46 -0300 Subject: [PATCH] fix(tools): the flash budget gate fails when it cannot see the firmware.bin tools/check_flash_budget.py holds the image's firmware.bin, signed, to the OTA ceiling and to the size recorded for it ("bin"). When .pio/build//firmware.bin was not there, it printed a NOTE and exited 0, so a run in a tree whose .pio/build had been cleaned passed without reading any .bin. Found in the review of S10c D5 (2026-10-23, on the UI-drivers branch). It did the same when OTA_APP_SAFE_MAX_SIZE could not be read out of src/ota/ota_layout.h, a condition --table already fails on. Both are failures now, and the message names what was not checked. An image under "ota_exempt" (pico_w_test_https) still needs no .bin. CI is unaffected: its firmware job builds the image and runs the gate in the same tree. The NOTE said "no firmware.bin next to the log" while the gate looks under .pio/build//; the message names that path now. Reproduction, on pico_w_release's log and .bin built from main at a8ccf4c3 (#288, the one PR since, changed only pico_w_test_https): - with .pio/build/pico_w_release/firmware.bin in place, main's gate and this one both pass ("firmware.bin signed 1035181 B, 5203 B under the OTA ceiling"); - with .pio/build cleaned, main's gate prints the NOTE and exits 0; this one fails and exits 1. tools/test_flash_budget.py gains five cases that run a copy of the gate the way CI does, in a temporary tree: no .pio/build fails (A), a .bin of its recorded size passes (B), one 1 B over its record fails (C), an exempt image needs no .bin (D), and an ota_layout.h the gate cannot read fails (E). Against main's gate, A and E fail and B, C and D pass (22 ok, 2 failed); with the fix, 24 ok. The CI gates, run locally in a depth-1 clone (static analysis included; native suites, sanitizers and fuzz skipped): 36 of 36 steps pass. Host only: nothing here runs on a board. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01LyLm7Q1o7kCHngE63BnoTx --- tools/README.md | 4 +-- tools/check_flash_budget.py | 28 +++++++++------- tools/test_flash_budget.py | 66 +++++++++++++++++++++++++++++++++++++ 3 files changed, 84 insertions(+), 14 deletions(-) diff --git a/tools/README.md b/tools/README.md index af288b14..327e5313 100644 --- a/tools/README.md +++ b/tools/README.md @@ -32,8 +32,8 @@ or a release. | `check_license.py` | A licença diz a mesma coisa nas **três** cópias (LICENSE, página `/license` e a string do firmware que a tela Licença do painel desenha, o texto MIT palavra por palavra nas duas telas), nenhum pacote de idioma volta a trazer `@LICENSE`, e os **dois** scripts de release levam o `LICENSE`, o `THIRD_PARTY_NOTICES.md` e a pasta `LICENSES`. Nasceu de dois achados de 21/09: a página web dizia 2025 e o zip do Arduino IDE saía sem `LICENSE`. | 2026-10-02 | | `gen_notices.py` | Escreve a lista de software de terceiros, de `tools/third_party.toml`, nos três lugares que a mostram: `THIRD_PARTY_NOTICES.md` (com os textos em `LICENSES/`), o bloco *Third-Party Notices* da página `/license` e o fim da tela Licença do painel; e copia para o firmware o texto em inglês que abre essa tela, tirado da própria página. A página e o painel de uma imagem listam o que a família dela carrega: as sete com rádio ou os dois SIMUT Wire (13/10). `--check` no CI. Nasceu de 02/10: conferida contra os símbolos de cada `firmware.elf`, a lista não citava seis componentes que toda imagem leva e errava os termos do BTstack. | 2026-10-13 | | `release_manifest.py` | The manifest a fleet manager downloads before an OTA: version, per-image size and sha256. Written by `release-ota.yml` next to the assets. Desde 04/10 traz o Pico 2 W (`releasetwo`): o `.bin` é a atualização marcada para teste (recusa uma sem a marca) e o `.uf2` é a imagem de fábrica, a tabela de partições e a atualização assinada no slot A com a marca apagada, escrita por `rp2350/uf2.py`. | 2026-10-04 | -| `check_flash_budget.py` | Fails the build when a firmware image grows past its budget in tools/flash_budget.json, or when its firmware.bin, signed, passes the OTA ceiling or the size recorded for it (`"bin"`; an image under `ota_exempt` is held to neither). `--table` holds the OTA headroom table of `docs/analysis/PLANO_STABLE.md` §3.1 to those records. | 2026-10-02 | -| `test_flash_budget.py` | The two rules that keep that table true: a `.bin` that grew past its record fails, and the table must say, row for row, what the record makes of it (an exempt image's row says so). | 2026-10-02 | +| `check_flash_budget.py` | Fails the build when a firmware image grows past its budget in tools/flash_budget.json, or when its firmware.bin, signed, passes the OTA ceiling or the size recorded for it (`"bin"`; an image under `ota_exempt` is held to neither). Also fails when it cannot look: no firmware.bin under `.pio/build//`, or an `ota_layout.h` it cannot read (until 2026-10-24 both passed with a note). `--table` holds the OTA headroom table of `docs/analysis/PLANO_STABLE.md` §3.1 to those records. | 2026-10-24 | +| `test_flash_budget.py` | The two rules that keep that table true: a `.bin` that grew past its record fails, and the table must say, row for row, what the record makes of it (an exempt image's row says so). And that the gate, run as CI runs it on a copy in a temporary tree, fails when it cannot look: no firmware.bin, or an `ota_layout.h` it cannot read. | 2026-10-24 | | `check_fsguard.py` | check_fsguard.py — the /config filesystem guards, pinned so they cannot silently rot. | 2026-08-29 | | `check_readme_numbers.py` | The counts the three READMEs quote are the counts the tree measures: host test cases and suites (the `RUN_TEST` lines of each `native*` suite), the release image's share of the slot (`flash_budget.json`), event codes (`logcodes.tsv`) and routes (`check_authz.py`). A reworded sentence fails too, so the gate cannot pass by finding nothing. Born on 2026-10-01: the READMEs still had the v2.7.1 flash figures and two test counts behind, and an outside review quoted them back. Mutation-tested, 9 of 9. | 2026-10-01 | | `check_angulo.py` | The Ângulo interface standard where a machine can see it: the token copy pinned by sha256, the site's CSS (the configurator's included) on tokens and the 4 px grid, the brand on token colours, the inline brand of the standalone pages equal to the generated one, the configurator's app manifest on token colours, no emoji in the READMEs, on the site or in a Living document, badges flat-square. Mutation-tested, 19 of 19 and then 4 of 4 for the configurator (AGENTS.md §7). | 2026-09-26 | diff --git a/tools/check_flash_budget.py b/tools/check_flash_budget.py index 9a6f8b80..395dc7d5 100755 --- a/tools/check_flash_budget.py +++ b/tools/check_flash_budget.py @@ -53,8 +53,12 @@ # the OTA headroom table against the record, no build needed python3 tools/check_flash_budget.py --table -Exit status is 0 when the image is within budget, 1 when it is over or when the -log carries no size line to check. +Exit status is 0 when the image is within budget, 1 when it is over, and 1 when +the gate cannot look: a log with no size line, no firmware.bin under +.pio/build// to hold to the OTA ceiling, or an ota_layout.h it cannot read. +Until 2026-10-24 the last two were a NOTE and exit 0, so a run after .pio/build +was cleaned passed without reading any .bin (found in S10c D5's review, +2026-10-23). An image under "ota_exempt" needs no .bin. Project: SIMUT License: MIT @@ -83,7 +87,7 @@ def ota_safe_max(): here, for the reason check_lang_packs gives about the pack limits: a gate holding its own copy of a number is a gate that can disagree with the code it guards. Returns None if the header stops looking like this, and the - caller then says so instead of pretending to check.""" + caller then fails instead of pretending to check.""" with open(os.path.join(ROOT, "src", "ota", "ota_layout.h"), encoding="utf-8") as fh: src = fh.read() @@ -270,9 +274,10 @@ def check_ota_bin(env, ceiling, exempt=None, recorded_bin=None): return path = os.path.join(ROOT, ".pio", "build", env, "firmware.bin") if not os.path.exists(path): - print(f"[flash-budget] NOTE {env}: no firmware.bin next to the log, " - f"so the OTA ceiling was not checked.") - return + fail(f"{env}: no {os.path.relpath(path, ROOT)}, so neither the OTA " + f"ceiling nor the recorded size (\"bin\") was checked. The log " + f"does not carry the .bin: run this in the tree that built the " + f"image, before .pio/build is cleaned.") # What goes over the air is the .bin with the signature trailer after it # (docs/analysis/OTA_ASSINADA.md), so that is what has to fit. trailer = sig_trailer_len() @@ -397,12 +402,11 @@ def main(): safe_max = ota_safe_max() if safe_max is None: - print(f"[flash-budget] NOTE {env}: could not read OTA_APP_SAFE_MAX_SIZE " - f"out of src/ota/ota_layout.h — the OTA ceiling was not checked.") - else: - check_ota_bin(env, ota_ceiling(cfg["envs"][env], safe_max), - cfg.get("ota_exempt", {}).get(env), - cfg["envs"][env].get("bin")) + fail(f"{env}: could not read OTA_APP_SAFE_MAX_SIZE out of " + f"src/ota/ota_layout.h, so the OTA ceiling cannot be checked.") + check_ota_bin(env, ota_ceiling(cfg["envs"][env], safe_max), + cfg.get("ota_exempt", {}).get(env), + cfg["envs"][env].get("bin")) if __name__ == "__main__": diff --git a/tools/test_flash_budget.py b/tools/test_flash_budget.py index 61774d92..d880174b 100644 --- a/tools/test_flash_budget.py +++ b/tools/test_flash_budget.py @@ -13,9 +13,17 @@ tools/flash_budget.json) fails — an image that grows records it; * the table says, row for row, what the record makes of it — so neither can move without the other. + +And one rule of the gate itself: when it cannot look (no firmware.bin, or an +ota_layout.h it cannot read), it fails instead of passing. """ +import json import os +import re +import shutil +import subprocess import sys +import tempfile sys.path.insert(0, os.path.dirname(os.path.abspath(__file__))) import check_flash_budget as cfb # noqa: E402 @@ -143,5 +151,63 @@ def doc(*rows): check("the OTA ceiling is the slot when there is one, the Pico W's otherwise", cfb.ota_ceiling({"slot": SLOT}, SAFE) == SLOT and cfb.ota_ceiling({}, SAFE) == SAFE) +# The gate as CI runs it (the env and its build log), on a copy of it in a +# temporary tree. Until 2026-10-24 an image whose firmware.bin was not there +# got a NOTE and exit 0, so a run after .pio/build was cleaned passed without +# reading any .bin (found in S10c D5's review, 2026-10-23); an ota_layout.h it +# could not read did the same. Not looking is a failure, not a pass. +GATE = os.path.abspath(cfb.__file__) +BIN = 1028252 +LOG = "Flash: [==========] 95.7% (used 1000000 bytes from 1044480 bytes)\n" + + +def run_gate(env, bin_size=None, layout_edit=None): + """(exit, output) of the copied gate for @p env, with a firmware.bin of + @p bin_size bytes under .pio/build// unless it is None.""" + with tempfile.TemporaryDirectory() as tmp: + os.makedirs(os.path.join(tmp, "tools")) + os.makedirs(os.path.join(tmp, "src", "ota")) + shutil.copy(GATE, os.path.join(tmp, "tools")) + for h in ("ota_layout.h", "signature.h"): + shutil.copy(os.path.join(cfb.ROOT, "src", "ota", h), os.path.join(tmp, "src", "ota")) + if layout_edit: + path = os.path.join(tmp, "src", "ota", "ota_layout.h") + with open(path, encoding="utf-8") as fh: + text = fh.read() + with open(path, "w", encoding="utf-8") as fh: + fh.write(layout_edit(text)) + with open(os.path.join(tmp, "tools", "flash_budget.json"), "w", encoding="utf-8") as fh: + json.dump({"ceiling": 1044480, + "envs": {"pico_w_release": {"budget": 1000000, "measured": 1000000, "bin": BIN}, + "pico_w_test_https": {"budget": 1000000, "measured": 1000000}}, + "ota_exempt": {"pico_w_test_https": "bench image, flashed over USB"}}, fh) + log = os.path.join(tmp, "build.log") + with open(log, "w", encoding="utf-8") as fh: + fh.write(LOG) + if bin_size is not None: + os.makedirs(os.path.join(tmp, ".pio", "build", env)) + with open(os.path.join(tmp, ".pio", "build", env, "firmware.bin"), "wb") as fh: + fh.write(b"\xff" * bin_size) + r = subprocess.run([sys.executable, os.path.join(tmp, "tools", "check_flash_budget.py"), env, log], + capture_output=True, text=True) + return r.returncode, (r.stdout + r.stderr).strip() + + +code, out = run_gate("pico_w_release") +check("an image whose firmware.bin is not there fails, naming where it looked", + code == 1 and "FAIL" in out and ".pio/build/pico_w_release/firmware.bin" in out, f"exit {code}: {out}") +code, out = run_gate("pico_w_release", BIN) +check("a firmware.bin of its recorded size passes", + code == 0 and "OK pico_w_release: firmware.bin signed" in out, f"exit {code}: {out}") +code, out = run_gate("pico_w_release", BIN + 1) +check("a firmware.bin 1 B over its record fails", code == 1, f"exit {code}: {out}") +code, out = run_gate("pico_w_test_https") +check("an image exempt from the OTA ceiling needs no firmware.bin", + code == 0 and "SKIP pico_w_test_https" in out, f"exit {code}: {out}") +code, out = run_gate("pico_w_release", BIN, + lambda t: re.sub(r"#define\s+OTA_FLASH_TOTAL\b", "#define OTA_FLASH_TOTAL_GONE", t)) +check("an ota_layout.h the gate cannot read fails, naming OTA_APP_SAFE_MAX_SIZE", + code == 1 and "FAIL" in out and "OTA_APP_SAFE_MAX_SIZE" in out, f"exit {code}: {out}") + print(f"{len(RAN) - len(FAILS)} ok, {len(FAILS)} falha(s)") sys.exit(1 if FAILS else 0)