Repository navigation
build: link natively on arm64 (Apple Silicon); compile-time profile - #213
Conversation
fc vendored an SSE-only equihash that nothing in VIZ calls, and linked it unconditionally, so no executable could link on arm64 (Apple Silicon, Graviton, Ampere); amd64 CI never saw it. The fc submodule now drops it (VIZ-Blockchain/fc#4, which also qualifies two fc::forward calls Apple clang 21 reports as ambiguous). The unreferenced tests/consensus_sim/fc-arm64-portability.patch patched the same dead files and goes with them. On macOS, Homebrew's static Boost.Iostreams config lists its dependencies as bare names (bz2, lzma, zstd, icu*); the Apple linker does not search the Homebrew prefix and icu4c is keg-only, so the link failed with "library 'zstd' not found". The APPLE block now adds the brew prefix lib dirs to the link path. docker-pr-build.yml gains a label-gated (check-arm64) build on ubuntu-24.04-arm, mirroring the Boost 1.83 range check, so the arm64 end of the platform range stays covered. docs/development/building.md records the result and the -ftime-trace profile: a cold vizd is ~10 min natively on an 8-core M-series laptop vs ~1 h under amd64 emulation; five TUs are a third of the compile time and the optimizer, not the parser, is ~2/3 of a typical unit, which is why PCH alone cannot fix it.
|
Full native Wall time is I/O-bound (5 % CPU): every simulated node opens a fresh 64 MB chainbase mmap, and |
On1x
left a comment
There was a problem hiding this comment.
All checks are green on the current head d5ca0bae, including the new arm64 job. The job is meaningful rather than decorative: it builds on ubuntu-24.04-arm and runs vizd itself, and it carries the same Dockerfile and flag set as the other build jobs, so a green run says the arm64 link actually works. The patch we merged today in VIZ-Blockchain/fc (#4, dropping the unused equihash vendor library) was the blocker for that link; I verified no caller of fc::equihash exists in the node tree, so removing it is dead-code removal. Deleting the obsolete arm64 portability patch is consistent with the same change.
Summary
Deep-dive into build time and runtime hot paths. The actionable finding: the tree could not link on arm64 at all, which is why every local build on this project's maintainer laptop has gone through amd64 emulation in Docker (~1 h cold) instead of running natively (~10 min). The cause was dead code.
thirdparty/fcvendored an SSE-onlyequihash(Steem proof-of-work) that nothing in VIZ calls (zero references inlibraries/,plugins/,programs/,tests/), and linked it unconditionally. Removed in build: drop the unused equihash vendor library; qualify fc::forward fc#4 (merged,bd855b7); this PR bumps the submodule to fc master.forward<T>()calls invariant_object.hppthat Apple clang 21 reports as ambiguous.bz2;lzma;zstd;icu*as bare names; the Apple linker does not search the Homebrew prefix and icu4c is keg-only, so the link died withlibrary 'zstd' not found. TheAPPLEblock now adds the brew prefix lib dirs to the link path.tests/consensus_sim/fc-arm64-portability.patchwas unreferenced and patched the same dead files; removed.build_arm64(ubuntu-24.04-arm, free for public repos), same shape asbuild_boost_183, so the arm64 end of the platform range stays covered. Thecheck-arm64label is on this PR.docs/development/building.md: Apple Silicon note and the compile-time profile below.What the analysis found (and did not change)
CI wall clock is tests, not compile: with the warm ccache from #202 the build step is ~2 min; the serial
consensus_simrun is 18–20 min. #203 already parallelises it, so nothing here touches tests.yml.Compile-time profile (clang
-ftime-trace, Release, native arm64, 74 project TUs ≈ 43 CPU-min):libraries/chain/database.cpplibraries/wallet/wallet.cpplibraries/network/dlt_p2p_node.cpplibraries/protocol/operations.cppplugins/snapshot/plugin.cppIn a typical unit (
block_log.cpp: 10 s frontend, 23 s backend) two thirds of the time is the optimizer: every TU that serializes asigned_blockinstantiates ~1,500fc::rawpack/unpack visitor functions (one per field per operation) and-O3optimizes each again. SoENABLE_PCHcan only shave the parsing third; the structural fix would be compiling the block/transaction serializers once (explicit instantiation), which fc'sinlinetemplates make unreliable — left as a follow-up, not attempted here.Per-block runtime path (
_apply_blockand the 25 processors it calls) was read for scans that grow with chain size ahead of 1 s slots: every loop is bounded by an ordered index (by_expiration,by_status,by_next_time, …) or byCHAIN_MAX_VALIDATORS/CHAIN_MAX_BLOCK_POST_VALIDATION_COUNT; the only full-table scan (update_bandwidth_reserve_candidates) is hourly and already scaled byblock_interval(). Nothing to change.Verified
vizdandcli_walletlink;vizd --versionprints 4.2.0. Coldvizdat-j8: ~10 min.ctest -E '^consensus_sim$': 13/13 pass.consensus_sim_testsrun: see the comment below (in progress when this was opened).