Repository navigation
fix(tools): the savings floor measures past the BTstack bank's fill - #285
Merged
Merged
Conversation
The nightly savings watermark (savings.yml, measure_savings.py --check) failed every night from 2026-10-02 to 2026-10-22, 21 in a row, and no pull request showed it: the job runs on none. On the last night five switches were under their floor: tel_tls by 10,096 B, bluetooth by 2,776, cli_full by 80 of flash and 2,048 of RAM, web_export_api by 40 and sound_buzzer by 24. None was a leak. bluetooth's was the instrument alone; the others were real changes nobody recorded, tel_tls's with the instrument on top. The instrument. The framework keeps BTstack's flash bank in 8 KB of .rodata aligned to a 4 KB sector (__bluetooth_tlv, btstack_flash_bank.cpp), in every image with Bluetooth: the alpha and the Air among the bases. The bank is the last thing in .rodata, the linker fills up to 4,095 B in front of it, and PlatformIO's `used` counts the fill. On a base with the bank, a byte more of .rodata anywhere in the image vanished into the fill while the variant without Bluetooth counted it, so bluetooth's saving fell with every change and would have jumped 4,096 B when the fill ran out. On the Air both sides have the bank, and tel_tls's saving jumped 4,096 B either way. Measured here with one toolchain: the symbols the Bluetooth switch removes are 132,474 B on 2721eb2 and 132,482 B on 25e169d, while its saving fell 2,912 B; the ones the tel_tls switch removes are the same 69,879 B on 2721eb2 and on 2690c0a (#220), while its saving rose 4,096 B. The fix. The floor (--check, --update) compares `used` minus the *fill* that ends at the bank's address, read from the linker map each build now asks for (-Wl,-Map through PLATFORMIO_BUILD_FLAGS). The five bases come out byte-identical with and without the flag (firmware.bin sha256: pico_w_test b4074dd0, pico_w_release e971dfe8, pico_w_asserts abd422b5, pico_w_alpha 93001b45, pico_w_air e5935de2). A build that leaves no map, that ran without zopfli, or whose map the reader cannot follow is refused rather than measured. --matrix keeps the whole `used`: there the question is fitting the slot, and the fill takes slot. Past the fill, the two savings follow the code. bluetooth: 136,792 B on 2721eb2, 136,816 on 2690c0a (#220), 136,808 on 6109ddb (#221), 136,808 on 25e169d. tel_tls: 69,968, 69,976, 58,120, 58,080. With the fill they were 140,268, 139,572, 138,892, 137,356 and 67,936, 72,032, 57,880, 57,840. The real drops, each with its cause: - tel_tls, -11,856 B at #221 (016daca, the stage and the apply take only signed images). The image verifies every update's ECDSA P-256 signature with BearSSL's verifier, now always on: 52 symbols, 12,884 B, that the switch removed on 2721eb2 and does not on 11d3822 (mul_f256, square_f256, p256_*, br_ecdsa_i31_vrfy_raw, br_i31_*). - cli_full, RAM 0 -> -2,048 B at #223 (3335d11). The panel's License screen reads the @webdict through a stack chunk, not lazyRead( ). The full CLI's `help` never reads the pack, so the test image lost lazyRead( ) and its 2,048 B static buffer; the image without the full CLI keeps both, for the emergency console's @HELP. - cli_full, flash -80 B. #214 (ab1e5b3) put `set time` in every image, for the unit with no network that asks for the time: cmdHandleSetTime( ) and parse_3ints( ), 412 B, left what the switch removes. What it removes grew around them (executeCommand( ) +396, cmdHandleSensorField( ) +124, renderMetrics( ) +88), and its .rodata shrank 128. - sound_buzzer, -24 B. #234 (f24e1b7) moved SoundManager::fillConfig( )'s body (218 -> 22 B) into soundStateToConfig( ) (186 B), which the commit_all dry run also uses; what the switch removes from commit_all came out 12 B smaller. - web_export_api, -40 B. The compiler now emits std::vector<String>'s growth path (_M_check_len( ), 60 B) out of line in /api/history_days, which every image has; on 2721eb2 the variant without the export had no such path out of line. The floors, recorded with --update on 25e169d's tree, past the fill: bluetooth (pico_w_alpha): 140,132 -> 136,808 tel_tls (pico_w_air): 67,936 -> 58,080 cli_full (pico_w_test): 48,232 -> 48,152, RAM 0 -> -2,048 web_https (pico_w_release): 24,520 -> 24,568, RAM 24 -> 16 web_history (pico_w_air): 21,768 -> 23,872 tft_graph (pico_w_release): 20,312 -> 20,432, RAM 2,236 -> 2,232 sensor_bme280 (pico_w_release): 17,724 -> 17,804 mdns (pico_w_release): 16,568 -> 16,632 web_export_api (pico_w_release): 12,336 -> 12,296 tel_mqtt (pico_w_air): 7,704 -> 9,344 sensor_ds18b20 (pico_w_release): 5,432 -> 5,520 sound_buzzer (pico_w_release): 5,008 -> 4,984 web_metrics (pico_w_release): 4,904 -> 4,968 sensor_dht22 (pico_w_release): 3,400 -> 3,456 concurrency_asserts (pico_w_asserts): 2,176 -> 2,240 syslog (pico_w_release): 1,760 -> 1,768 web_https's and tft_graph's RAM floors move 8 and 4 B, inside the 64 B the gate allows for the linker's alignment. docs/analysis/ECONOMIA_DE_RECURSOS.md (Living) gains the section, with the table; AGENTS.md, tools/README.md and the savings.yml header say what the floor compares, that it builds 21 images, and that no pull request sees it fail. The gates job runs tools/test_measure_savings.py, 21 cases: the fill read from a real map (548 B, pico_w_alpha on 25e169d), the bank flush against the section before it, absent, or only under "Discarded input sections", the zero-size entries skipped, a fill further up not taken for the bank's, an unreadable map refused rather than read as 0, and so is a map with the bank's object (btstack_flash_bank.cpp.o) and no bank under the bank's name, as a framework that renamed it would leave; the build asking for the map and refused without it, without zopfli or with a map it cannot follow, and each side's fill taken off. Mutation-tested: 10 of 10 one-line changes to the tool are caught, each by the case written for it (the reader's refusal also by the build's). On the twelve real maps of the measurements above the reader gives the fill each build printed, and 0, with no refusal, on the variants without Bluetooth. No image changes and src/ is untouched, so CHANGELOG.md is too, as with #279. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013C6NHM2fJi3M7EMr47WyQ3
angeloINTJ
pushed a commit
that referenced
this pull request
Oct 22, 2026
#286 also appended to tools/flash_budget.json's measured_on and measured_at_commit, so the two appends conflicted. Resolved from main's file plus this branch's edits: the pico_w_test_https budget at 1,028,148 B and this change's entries after main's, now naming a8ccf4c as the base. Measured again on the merged tree, all nine from scratch, against main a8ccf4c built from scratch: pico_w_test_https alone differs (used 1,028,148 B against 1,032,404, .bin 1,040,180 against 1,044,436, the last LOAD ending at 0x100FDF34: 4,300 B of room); the other eight are byte for byte main's. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LyLm7Q1o7kCHngE63BnoTx
angeloINTJ
pushed a commit
that referenced
this pull request
Oct 23, 2026
Takes main from 25e169d to 2075b84: #285 (tools: the savings floor measures past the BTstack bank's fill, with a new tools/test_measure_savings.py), #286 (src: the console and syslog name the event again outside English on alpha and wire: LogManager.cpp and .h, DisplayManager_Alpha.cpp, AppManager_Boot.cpp), #287 (docs: the bench of 2026-10-22, TRL( ), OTA and the packs) and #288 (the licence page of pico_w_test_https is served from LittleFS, which also merged #285 and #286 into its own branch). There is no conflict in src/: the branch also changed src/AppManager_Boot.cpp since 25e169d, and git merges it as text. The five conflicts are three texts and two measurements; the measurements are measured again on the merged tree, never merged as text. Conflicts, one line each: - CLAUDE.md: main's rewritten end of "Language packs" (TRL( ) is a ledger the device never reads), then the branch's bullet "The Ângulo's texts". - tools/README.md: every row kept: the branch's three UI-gate rows, main's measure_savings.py row and new test_measure_savings.py row (#285), then the branch's four rows main had left as they were; test_build_custom.py keeps the branch's 72 hostile profiles (the script prints 72 on the merged tree; main's 54 is the old count). - tools/check_lang_packs.py: one message, "a missing @WEBDICT/@angulo entry silently leaves that one line in English. (@TRL is a ledger the device never reads.)"; tools/test_lang_gate.py passes (52 of 52). - docs/analysis/PLANO_STABLE.md section 3.1: the eleven OTA rows written from the re-measured records (check_flash_budget.py --table passes: 11 images, 1 exempt); only the pico_wire row moves. - tools/flash_budget.json: budget = measured and the .bin of all eleven images from a build from scratch of the merged tree (list below); measured_on and measured_at_commit keep the branch's text, then main's sentences (#286, #288), then one for this merge. The READMEs need nothing: check_readme_numbers.py passes on the merged tree (750 cases in 11 suites, release 97.9 % of the slot, 170 event codes, 62 routes (52 gated, 10 public), in all three READMEs). CLAUDE.md and tools/README.md: the "scripts" figure follows tools/README.md's row count (198 at ecb527e, 199 with #285's test_measure_savings.py row). Flash used and .bin, ecb527e -> this merge, both built from scratch with the default core (ecb527e reproduces every record of its flash_budget.json, its eleven sha256 equal S10c B6-b's, and the 948 pt-BR frames of the six host screen groups are unchanged on the merge): - pico_w_release: 1,022,900 -> 1,022,900 B used (+0), 1,034,932 -> 1,034,932 .bin (+0) - pico_w_test: 1,027,348 -> 1,027,348 B used (+0), 1,039,380 -> 1,039,380 .bin (+0) - pico_w_test_https: 1,032,412 -> 1,028,148 B used (-4264), 1,044,444 -> 1,040,180 .bin (-4264) - pico_w_asserts: 1,025,204 -> 1,025,204 B used (+0), 1,037,236 -> 1,037,236 .bin (+0) - pico_w_alpha: 976,236 -> 976,228 B used (-8), 990,380 -> 990,380 .bin (+0) - pico_w_air: 1,019,480 -> 1,019,480 B used (+0), 1,035,340 -> 1,035,340 .bin (+0) - pico2_w_release: 998,928 -> 998,928 B used (+0), 998,952 -> 998,952 .bin (+0) - pico_wire: 592,268 -> 592,260 B used (-8), 604,304 -> 604,296 .bin (-8) - pico_wiretft: 774,596 -> 774,596 B used (+0), 786,632 -> 786,632 .bin (+0) - pico_wiretft_angulo: 814,716 -> 814,716 B used (+0), 826,752 -> 826,752 .bin (+0) - pico2_w_angulo: 1,035,608 -> 1,035,608 B used (+0), 1,035,632 -> 1,035,632 .bin (+0) Main's own change (#285 to #288), built the same way on its own trees, 25e169d -> 2075b84: - pico_w_release: 1,022,908 -> 1,022,908 B used (+0), 1,034,940 -> 1,034,940 .bin (+0) - pico_w_test: 1,027,348 -> 1,027,348 B used (+0), 1,039,380 -> 1,039,380 .bin (+0) - pico_w_test_https: 1,032,404 -> 1,028,148 B used (-4256), 1,044,436 -> 1,040,180 .bin (-4256) - pico_w_asserts: 1,025,212 -> 1,025,212 B used (+0), 1,037,244 -> 1,037,244 .bin (+0) - pico_w_alpha: 976,236 -> 976,228 B used (-8), 990,380 -> 990,380 .bin (+0) - pico_w_air: 1,019,480 -> 1,019,480 B used (+0), 1,035,340 -> 1,035,340 .bin (+0) - pico2_w_release: 999,016 -> 999,016 B used (+0), 999,040 -> 999,040 .bin (+0) - pico_wire: 592,308 -> 592,300 B used (-8), 604,344 -> 604,336 .bin (-8) - pico_wiretft: 774,612 -> 774,612 B used (+0), 786,648 -> 786,648 .bin (+0) The merge's delta equals main's on 8 of the nine images; it differs on pico_w_test_https. Host only: nothing of this merge has run on a board. Proof, host only (stages/sync5_c1 of the UI-drivers scratchpad): - Main's four PRs change 29 files and the merge changes the same 29 against ecb527e; every patch is identical to main's except the two measured files, check_lang_packs.py's resolved message and the one-line scripts figure of CLAUDE.md and tools/README.md. Main's 5 hunks in src/ (#286: AppManager_Boot.cpp, DisplayManager_Alpha.cpp x2, LogManager.cpp, LogManager.h) are each in the merge once; #288 touches neither src/ nor WebUI.h. - check_flash_budget.py passes for the eleven images, --table passes (11 images, 1 exempt), test_flash_budget.py 19 ok. Against ecb527e only pico_w_alpha, pico_wire and pico_w_test_https change; the other eight are byte-identical (sha256). - ci_gates_local.py --raso (depth-1 clone): 63 of 63 steps passed; 1 skipped (own env, CI only) - tools/elf_funcoes.py and reloc_only.py, ecb527e against the merge: pico_w_alpha and pico_wire change DisplayManager::trlLookup( ) and logcodeLookup( ) (#286; REAL, 3 -> 2 instructions); pico_w_test_https changes WebManager::handleLicense( ) (#288; REAL, 9 -> 6), drops WebUI_GZ::LICENSE_PAGE_GZ, gains "/web/license.html.gz", and 12 callers move literal words only; nothing in the other six products or the two Angulo images changes. Main's own pairs (25e169d -> 2075b84) give the same lists but for three items, all decoded with objdump: SensorManager::SensorManager( ) (UNPROVEN on the merge side only: its 4-byte constant 0xFFFFFFFF is equal, and the table beside it points at the same SoundManager::MEL_* arrays, 8 B further on in both pairs), and the text fragments 'd;' and 'jT6' (printable bytes inside the removed licence blob, 4,267 B on ecb527e and 4,260 B on 25e169d). - pico_w_test_https: the last LOAD ends at 0x100fdf34, 4,300 B before 0x100FF000 (ecb527e: 0x100fefdc, 36 B; main 2075b84: 0x100fdf34). - The Angulo: its two ELFs change no function or datum; 948 of 948 pt-BR host frames of the six groups equal ecb527e's; test_idiomas (121 checks), test_textos (105), test_temas (174 cases) and check_ui_texts pass. - pico_wire and pico_w_test_https rebuilt from scratch on this commit's tree give the reference sha256. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LyLm7Q1o7kCHngE63BnoTx
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What was wrong
The nightly savings watermark (
savings.yml,tools/measure_savings.py --check) failed every night from 2026-10-02 to 2026-10-22, 21 in a row. No pull request showed it, because the job runs on none. On the last night, five switches were under their floor:tel_tlsbluetoothcli_fullweb_export_apisound_buzzerNone was a leak.
bluetooth's drop was the instrument alone. The others were real changes nobody recorded in the floor, andtel_tls's had the instrument on top.The instrument
The framework keeps BTstack's flash bank in 8 KB of
.rodata, aligned to a 4 KB sector (__bluetooth_tlv,cores/rp2040/sdkoverride/btstack_flash_bank.cpp). Every image with Bluetooth has it, and among the bases those are the alpha and the Air. The bank is the last thing in.rodata, and the linker fills up to 4,095 B in front of it. PlatformIO'susedcounts that fill..rodataanywhere in the image vanished into the fill. The variant without Bluetooth counted it. Sobluetooth's saving fell with every change, and it would have jumped 4,096 B when the fill ran out.tel_tls's saving jumped 4,096 B either way.Measured here with one toolchain and one framework on all four commits:
2721eb2e(10-01)2690c0a2(#220)6109ddb3(#221)25e169df(10-22)bluetooth,usedbluetooth, past the filltel_tls,usedtel_tls, past the fillbluetooth: its switch removes 132,474 B of symbols on2721eb2eand 132,482 B on25e169df. The code changed 8 B, whileusedreported -2,912 B.tel_tls: its switch removes the same 69,879 B of symbols on2721eb2eand on2690c0a2, whileusedreported +4,096 B.This framework already carries October's patches. On
2721eb2e, the one of the first three commits the nightly job measured,bluetooth'susedcomes out 8 B from the job's number andtel_tls's equal to it. On25e169df, all six switches above equal the job's numbers byte for byte.The fix
The floor (
--check,--update) comparesusedminus the*fill*that ends at the bank's address. It reads that fill from the linker map each build now asks for (-Wl,-Map, throughPLATFORMIO_BUILD_FLAGS).firmware.binsha256:pico_w_testb4074dd0,pico_w_releasee971dfe8,pico_w_assertsabd422b5,pico_w_alpha93001b45,pico_w_aire5935de2.--matrixkeeps the wholeused. There the question is fitting the slot, and the fill takes slot.The real drops
Measured past the fill, each with its cause:
tel_tls, -11,856 B, at feat: the stage and the apply take only signed images #221 (016dacab: the stage and the apply take only signed images).2721eb2eand does not on11d38223(mul_f256,square_f256,p256_*,br_ecdsa_i31_vrfy_raw,br_i31_*).cli_full, RAM 0 → -2,048 B, at Packs keep only @DICT in RAM; the License screen matches /license, with complete third-party notices #223 (3335d115).@WEBDICTthrough a stack chunk, notlazyRead( ).helpnever reads the pack, so the test image lostlazyRead( )and its 2,048 B static buffer.@HELP.cli_full, flash -80 B.ab1e5b3b) putset timein every image, for the unit with no network that asks for the time. SocmdHandleSetTime( )andparse_3ints( ), 412 B, left what the switch removes.executeCommand( )+396,cmdHandleSensorField( )+124,renderMetrics( )+88..rodatashrank 128.sound_buzzer, -24 B.f24e1b71) movedSoundManager::fillConfig( )'s body (218 → 22 B) intosoundStateToConfig( )(186 B), which thecommit_alldry run also uses.commit_allcame out 12 B smaller.web_export_api, -40 B.std::vector<String>'s growth path (_M_check_len( ), 60 B) out of line in/api/history_days, which every image has.2721eb2e, the variant without the export had no such path out of line.The floors
Recorded with
--updateon25e169df's tree, past the fill:bluetoothpico_w_alphatel_tlspico_w_aircli_fullpico_w_testweb_httpspico_w_releaseweb_historypico_w_airtft_graphpico_w_releasesensor_bme280pico_w_releasemdnspico_w_releaseweb_export_apipico_w_releasetel_mqttpico_w_airsensor_ds18b20pico_w_releasesound_buzzerpico_w_releaseweb_metricspico_w_releasesensor_dht22pico_w_releaseconcurrency_assertspico_w_assertssyslogpico_w_releaseweb_historyandtel_mqttmove because the fill is now out, asbluetoothandtel_tlsdo.web_https's andtft_graph's RAM floors move 8 and 4 B, inside the 64 B the gate allows for the linker's alignment.Tests
tools/test_measure_savings.pyhas 21 cases. It is a new step of thegatesjob, so every pull request now runs the arithmetic the floor does. The cases:pico_w_alphaon25e169df);btstack_flash_bank.cpp.o) but no bank under the bank's name, which is what a framework that renamed the bank would leave;Mutations: 10 of 10 one-line changes to the tool are caught. Each fails the case written for it, and the reader's refusal also fails the build's. On the twelve real maps of the measurements above (alpha, Air and their variants, on three commits), the reader gives the fill each build printed, and 0 without a refusal on the variants without Bluetooth.
Gates
Locally (
ci_gates_local.py), thegates,static-analysisandfuzzjobs pass: 48 of 48 steps. The sanitizer pass runs in CI only. After the review's changes, the quick set passed again, 36 of 36, with this test at 21/21.Review
A clean-context review returned COM RESSALVAS (with reservations), nothing blocking. Taken:
AGENTS.md's figure: 140 kB became 137 kB past the fill.build.ymlcomment put the whole red on the fill. Now it says the fill was all ofbluetooth's drop and part oftel_tls's.Its other point, that the real path had never run on CI, is the run below.
The real path on CI
The savings workflow, dispatched on this branch: run 39788868842, green:
check_savings: 16 recursos, nenhum abaixo do piso de economia.tel_tls,web_historyandtel_mqttvariants.Docs
docs/analysis/ECONOMIA_DE_RECURSOS.md(Living) gains a section with the evidence, the causes and the table.AGENTS.md,tools/README.mdand thesavings.ymlheader now say:gh run list --workflow "SIMUT savings watermark".src/is untouched, soCHANGELOG.mdis unchanged too, as with fix(tools): patch.sh 2b-ter greps text its patch adds; a gate holds every guard to its patch #279.Not in this pull request
src/. That is about 16 to 21 minutes of CI time.helpholds a 2,048 B static buffer (_lazyReadBuf) in every image without the full CLI, the release included. It exists only to read@HELPin one piece. Reading it line by line from flash would give that RAM back. I measured nothing for it.🤖 Generated with Claude Code
https://claude.ai/code/session_013C6NHM2fJi3M7EMr47WyQ3