You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Measure and reduce frontend build and spec time (tsc, lint, vitest, build, story tests) #453
Frontend build and spec speed: the JavaScript Build workflow's slowest jobs take about 5–8 minutes each. This issue tracks measuring and reducing the time for compile (tsc -b), lint, vitest, the Rollup/Vite build, and Storybook story tests, both locally and in CI.
Compile, lint and vitest run inside one CI step, so their separate times are not visible. The first task is to time them separately.
Candidates (evidence-based; savings not yet measured)
Source compiles twice: g:build runs inside g:ci, and again after typecheck-stories.
The conformance and adapter package builds run in both the tooling and Storybook jobs.
Story tests are the largest single step. Check whether run-story-matrix.mjs runs its matrix in parallel or sharded.
The renderer-adapter and package-manager matrix jobs have no yarn cache (installs take 21–37 s vs 2–3 s in the cached core jobs).
Local loop: document and time yarn compile / tsc -b, yarn lint, vitest run and the build per workspace; check that incremental tsc -b and eslint --cache hit on a no-change rerun.
Done when
Per-step local and CI timings are recorded (compile, lint, vitest, build, story tests).
State: Open, partly delivered (narrowed by triage 2026-11-15) Delivered: Sharding, caching, reuse and aggregate checks (#455, #456, #473, #474, #476, #491). Remaining: Report compile, lint, vitest, build and story timings separately, locally and in CI (ci:static still runs as one step), then measure the remaining duplicate-build and cache experiments (each Storybook shard still runs its own package and static build) with controlled before/after evidence and every check kept. Evidence: Published CI totals after #491 include concurrent changes, different job sets and unmatched cache warmth; they do not establish isolated savings for the remaining experiments.
Current behaviour — The issue’s performance plan is partly delivered. Source specs now run in three shards (workflow); TypeScript consumers run separately for compilers 6 and 7 (workflow); Storybook also has three cached shards (workflow). However, Source static validation remains one combined CI command, and its script still invokes both build and g:build (script). The first performs TypeScript compilation; the latter performs build-mode compilation, Rollup and CSS copying (definitions, shared build).
Root cause / gap — The status block understates delivery, while aggregate steps still obscure individual compile, lint and bundle costs. Sharding does not establish completion of the local-loop measurements or before/after acceptance criteria. Redundant compilation remains a candidate, not a measured saving.
Proposed change — Reconcile the status with the two merged PRs. Add per-command elapsed timings while preserving command order, exit codes and every validation. Record comparable cold/warm local results and CI wall time, queue time and runner-minutes. Then optimize one measured bottleneck per change; do not remove either compilation until its outputs and consumers are accounted for. Add timing-wrapper examples for successful and failing commands, with should-preserve-exit-status and should-execute-once assertions, following the existing Node tooling-test convention. Retain Storybook’s existing partition specifications.
Size, risk, release intent — M; moderate risk from command ordering and artifact reuse; no-release. Sharding savings still need measured confirmation rather than estimates.
Dependencies — Delivered prerequisites: #455 and #456.
TypeScript consumers (6/7) no longer repeat the public-types check that verify-public-types (6/7) runs from the same build recipe. verify-public-types is now in Build's needs.
verify-no-pixi-consumer and verify-spatial-consumer are removed: TypeScript consumers (7) already runs both checks, and they still run for compilers 6 and 7. That takes the workflow from 54 jobs to 52, matrix legs counted.
Measured, job durations in seconds:
Job
Before (main runs 40892593269, 40860861536, 40845583404)
After (PR run 40963541146, main run 40964386171)
TypeScript consumers (6)
275, 243, 255
85, 91
TypeScript consumers (7)
152, 85, 103
103, 77
verify-no-pixi-consumer + verify-spatial-consumer
99, 87, 170
removed
Total runner time per run
4848 (run 40892593269)
4215, 3695
Remaining
The heaviest jobs are now Storybook (about 4 min per shard), Source specs (about 2 min per shard) and the renderer and package-manager matrices. The next measured step is to profile Storybook shard time.
Moving heavy jobs to the free self-hosted cratis-arc runners would cut hosted minutes further. That needs a decision on runner use and cost before anyone implements it.
Slice 2 merged in #474 (no release). The renderer-adapter matrix verifies all three package managers per adapter and boundary from one build and pack: 21 legs became 9, with all 21 verifications still logged. Renderer runner time fell from a main median of 1227 s to 650 s, and total jobs per run from 52 to 40. Measurements are in #474 (comment). Next candidates, one measured bottleneck per PR: the Storybook shards (about 80 s of duplicated package and static builds per shard, but sharing them adds critical-path time), and adding the renderer, package-manager and Arc-consumer jobs to the Build aggregator's needs.
Slice 3 merged in #476 (no release): the Build check now requires every job in the JavaScript Build workflow, so a failed or cancelled leg in any of them fails Build. main has no branch protection or ruleset yet; marking Build as a required check makes this block merges. Remaining candidate: the duplicated package and static builds in the Storybook shards (about 80 s per shard).
CI measurement after levers 1+2, at the train head of #491 (ee256d9). That head also carries #176 and other changes merged from main since the baseline, so the difference below is not attributable to levers 1+2 alone.
JavaScript Build used 3,842 runner-seconds across 37 jobs, compared with a baseline of 4,401.5 seconds across 41 jobs (the average of two earlier runs, 41431195012 and 41429970261). That is 559.5 seconds, or 12.7%, lower. The job set changed:
Six release-verification jobs are gone, 530 s in the baseline: release-evidence, verify-published-exports, verify-package-graph, verify-provider-context-identity and verify-public-types for TypeScript 6 and 7.
Two Published artifact jobs are new: 339 s for TypeScript 6 and 7.
Of the jobs present in both runs, most got shorter, but there is noticeable run-to-run variation, for example ±30 s on single jobs.
Verify Fresh Dependencies used 64 s and Markdown Verification used 14 s. The baseline did not include those two workflows.
Method: the sum over jobs of completed minus started time, so time spent waiting for a runner is excluded. There is one sample after the change and two before, and cache warmth was not compared.
source job (109 s in total): Yarn install 29 s, Lint Source 18 s, Build published Source artifact 15 s, Verify parts manifest 7 s, Typecheck stories 5 s, Report public API changes 5 s. Every other static step takes 4 s or less.
Source specs shards: tests take 83–101 s per shard, plus about 18–24 s for Yarn install.
Storybook shards (252–305 s each):
Step
Shard 1
Shard 2
Shard 3
Build packages the composed Storybook imports
31 s
27 s
24 s
Build composed renderer Storybook
65 s
60 s
51 s
Test assigned Storybook units
156 s
134 s
123 s
Each Storybook shard rebuilds the same packages and the same composed Storybook, which takes about 75–96 s per shard and about 4.5 runner-minutes per run. The static checks are already fast. The next step in this issue is a measured experiment: build Storybook once and share it with the shards. It only lands if the time saved is still larger once the artifact upload and download time is counted.
Sharing the build saves about 114 runner-seconds per run. The download took 3 s, so the artifact transfer cost almost nothing. The cost is about 60–70 s more wall time, because the shards wait for the producer job and each still runs its own install and browser setup. The repository is public, so hosted runner time is free, and every pull request would wait about a minute longer for its result. That is a net loss, so #567 is closed. This is one observation per variant; the size of the wall-time penalty matches the plan's prediction.
Still to do here: local per-phase timings (compile, lint, Vitest, build, story tests). The README now explains how to take them.
These are local per-phase timings on main at e77a8e2, measured on an Apple M4 Pro (12 cores) with Node 26.8.1 and Yarn 4.17.1. Dependencies were already installed. Each script was run once in ci:static order, then the main phases were rerun without cleaning.
Source (yarn workspace @cratis/components run <script>):
Phase
First run
Rerun without cleaning
Lint (lint:ci)
2.9 s
2.1 s
Compile (build, including parts generation)
2.1 s
1.1 s
Package build (g:build, Rollup)
6.7 s
6.5 s
Specs (test, Vitest)
69.7 s
57.9 s
Remaining 9 ci:static scripts
19.5 s in total; the slowest is verify-parts-manifest at 6.7 s
–
clean does not clear the ESLint or TypeScript incremental caches, so lint and compile stay fast even after it runs.
Storybook:
Phase
Seconds
Builds of the five packages Storybook imports
14.8
Preflight (ci:preflight)
3.7
Composed Storybook build
27.1
Index verification
0.7
Story tests, all three partitions in one run
116.9
Vitest and the story tests are about 90% of the local time. CI spreads them over three shards each. Locally, the full Source chain plus specs takes about 1.5 minutes, and the full Storybook sequence about 2.7 minutes.
Activity
woksin commentedon Oct 29, 2026
Analysis and proposed solution
Current behaviour — The issue’s performance plan is partly delivered. Source specs now run in three shards (workflow); TypeScript consumers run separately for compilers 6 and 7 (workflow); Storybook also has three cached shards (workflow). However, Source static validation remains one combined CI command, and its script still invokes both
buildandg:build(script). The first performs TypeScript compilation; the latter performs build-mode compilation, Rollup and CSS copying (definitions, shared build).Root cause / gap — The status block understates delivery, while aggregate steps still obscure individual compile, lint and bundle costs. Sharding does not establish completion of the local-loop measurements or before/after acceptance criteria. Redundant compilation remains a candidate, not a measured saving.
Proposed change — Reconcile the status with the two merged PRs. Add per-command elapsed timings while preserving command order, exit codes and every validation. Record comparable cold/warm local results and CI wall time, queue time and runner-minutes. Then optimize one measured bottleneck per change; do not remove either compilation until its outputs and consumers are accounted for. Add timing-wrapper examples for successful and failing commands, with should-preserve-exit-status and should-execute-once assertions, following the existing Node tooling-test convention. Retain Storybook’s existing partition specifications.
Size, risk, release intent — M; moderate risk from command ordering and artifact reuse; no-release. Sharding savings still need measured confirmation rather than estimates.
Dependencies — Delivered prerequisites: #455 and #456.
woksin commentedon Oct 29, 2026
Slice 1 delivered: duplicate packed-consumer type checks removed
Merged in #473 (no-release).
TypeScript consumers (6/7)no longer repeat the public-types check thatverify-public-types (6/7)runs from the same build recipe.verify-public-typesis now inBuild'sneeds.verify-no-pixi-consumerandverify-spatial-consumerare removed:TypeScript consumers (7)already runs both checks, and they still run for compilers 6 and 7. That takes the workflow from 54 jobs to 52, matrix legs counted.Measured, job durations in seconds:
Remaining
cratis-arcrunners would cut hosted minutes further. That needs a decision on runner use and cost before anyone implements it.woksin commentedon Oct 31, 2026
Slice 2 merged in #474 (no release). The renderer-adapter matrix verifies all three package managers per adapter and boundary from one build and pack: 21 legs became 9, with all 21 verifications still logged. Renderer runner time fell from a main median of 1227 s to 650 s, and total jobs per run from 52 to 40. Measurements are in #474 (comment). Next candidates, one measured bottleneck per PR: the Storybook shards (about 80 s of duplicated package and static builds per shard, but sharing them adds critical-path time), and adding the renderer, package-manager and Arc-consumer jobs to the Build aggregator's needs.
woksin commentedon Oct 31, 2026
Slice 3 merged in #476 (no release): the
Buildcheck now requires every job in the JavaScript Build workflow, so a failed or cancelled leg in any of them failsBuild.mainhas no branch protection or ruleset yet; markingBuildas a required check makes this block merges. Remaining candidate: the duplicated package and static builds in the Storybook shards (about 80 s per shard).woksin commentedon Nov 3, 2026
CI measurement after levers 1+2, at the train head of #491 (ee256d9). That head also carries #176 and other changes merged from main since the baseline, so the difference below is not attributable to levers 1+2 alone.
JavaScript Build used 3,842 runner-seconds across 37 jobs, compared with a baseline of 4,401.5 seconds across 41 jobs (the average of two earlier runs, 41431195012 and 41429970261). That is 559.5 seconds, or 12.7%, lower. The job set changed:
release-evidence,verify-published-exports,verify-package-graph,verify-provider-context-identityandverify-public-typesfor TypeScript 6 and 7.Of the jobs present in both runs, most got shorter, but there is noticeable run-to-run variation, for example ±30 s on single jobs.
Verify Fresh Dependencies used 64 s and Markdown Verification used 14 s. The baseline did not include those two workflows.
Runs:
Method: the sum over jobs of completed minus started time, so time spent waiting for a runner is excluded. There is one sample after the change and two before, and cache warmth was not compared.
This issue stays open for the remaining levers.
woksin commentedon Nov 21, 2026
#566 is merged. CI now runs the Source static checks as named steps, so each phase has its own time. The numbers below are from the PR's JavaScript build run (https://github.com/Cratis/Components/actions/runs/44935521153).
sourcejob (109 s in total): Yarn install 29 s, Lint Source 18 s, Build published Source artifact 15 s, Verify parts manifest 7 s, Typecheck stories 5 s, Report public API changes 5 s. Every other static step takes 4 s or less.Source specs shards: tests take 83–101 s per shard, plus about 18–24 s for Yarn install.
Storybook shards (252–305 s each):
Each Storybook shard rebuilds the same packages and the same composed Storybook, which takes about 75–96 s per shard and about 4.5 runner-minutes per run. The static checks are already fast. The next step in this issue is a measured experiment: build Storybook once and share it with the shards. It only lands if the time saved is still larger once the artifact upload and download time is counted.
woksin commentedon Nov 21, 2026
The experiment in #567 is measured and not merged. It shares one Storybook build with the three story-test shards.
All checks passed in both runs. The table compares the PR run (https://github.com/Cratis/Components/actions/runs/44938476948) with the baseline from #566 (https://github.com/Cratis/Components/actions/runs/44935521153).
Sharing the build saves about 114 runner-seconds per run. The download took 3 s, so the artifact transfer cost almost nothing. The cost is about 60–70 s more wall time, because the shards wait for the producer job and each still runs its own install and browser setup. The repository is public, so hosted runner time is free, and every pull request would wait about a minute longer for its result. That is a net loss, so #567 is closed. This is one observation per variant; the size of the wall-time penalty matches the plan's prediction.
Still to do here: local per-phase timings (compile, lint, Vitest, build, story tests). The README now explains how to take them.
woksin commentedon Nov 21, 2026
These are local per-phase timings on main at e77a8e2, measured on an Apple M4 Pro (12 cores) with Node 26.8.1 and Yarn 4.17.1. Dependencies were already installed. Each script was run once in
ci:staticorder, then the main phases were rerun without cleaning.Source (
yarn workspace @cratis/components run <script>):lint:ci)build, including parts generation)g:build, Rollup)test, Vitest)ci:staticscriptsverify-parts-manifestat 6.7 scleandoes not clear the ESLint or TypeScript incremental caches, so lint and compile stay fast even after it runs.Storybook:
ci:preflight)Vitest and the story tests are about 90% of the local time. CI spreads them over three shards each. Locally, the full Source chain plus specs takes about 1.5 minutes, and the full Storybook sequence about 2.7 minutes.
This completes the scope of this issue: