Skip to content

Measure and reduce frontend build and spec time (tsc, lint, vitest, build, story tests) #453

Closed

Description

@woksin

Frontend build and spec speed: the JavaScript Build workflow's slowest jobs take about 5–8 minutes each. This issue tracks measuring and reducing the time for compile (tsc -b), lint, vitest, the Rollup/Vite build, and Storybook story tests, both locally and in CI.

Measured (CI, last 3 green JavaScript Build runs on main: 39866936034, 39827741030, 39508052707)

Step Command Median s (min–max)
Source build/lint/spec/API checks yarn workspace @cratis/components run ci (generate-parts, verify-*, g:ci = clean + eslint + tsc -b + rollup + vitest, typecheck-stories, g:build again) 298 (245–330)
Root tooling, policy, workspace checks tooling tests + migrator/eslint-plugin ci + 4 adapter/conformance builds + consumer typecheck 274 (222–301)
Storybook story tests yarn test-storybook (story matrix) 321 (226–351)
Storybook build build-storybook 45 (33–49)
Build imported packages (Storybook job) 4 adapter/conformance builds 25 (18–28)
Playwright install playwright install --with-deps chromium 18 (16–19)
Job totals source / tooling / storybook 323 / 317 / 442

Compile, lint and vitest run inside one CI step, so their separate times are not visible. The first task is to time them separately.

Candidates (evidence-based; savings not yet measured)

  • Source compiles twice: g:build runs inside g:ci, and again after typecheck-stories.
  • The conformance and adapter package builds run in both the tooling and Storybook jobs.
  • Story tests are the largest single step. Check whether run-story-matrix.mjs runs its matrix in parallel or sharded.
  • The renderer-adapter and package-manager matrix jobs have no yarn cache (installs take 21–37 s vs 2–3 s in the cached core jobs).
  • Local loop: document and time yarn compile / tsc -b, yarn lint, vitest run and the build per workspace; check that incremental tsc -b and eslint --cache hit on a no-change rerun.

Done when

  • Per-step local and CI timings are recorded (compile, lint, vitest, build, story tests).
  • Each change lands with before/after numbers.
  • No check is dropped from CI.

Part of the cross-repository build-speed effort (spec speed for Studio: Cratis/StudioIssues#1187, Cratis/StudioIssues#1188, Cratis/StudioIssues#1189).

Status

State: Open, partly delivered (narrowed by triage 2026-11-15)
Delivered: Sharding, caching, reuse and aggregate checks (#455, #456, #473, #474, #476, #491).
Remaining: Report compile, lint, vitest, build and story timings separately, locally and in CI (ci:static still runs as one step), then measure the remaining duplicate-build and cache experiments (each Storybook shard still runs its own package and static build) with controlled before/after evidence and every check kept.
Evidence: Published CI totals after #491 include concurrent changes, different job sets and unmatched cache warmth; they do not establish isolated savings for the remaining experiments.

Activity

  1. self-assigned this
    on Oct 23, 2026
  2. woksin commented on Oct 29, 2026

    @woksin
    ContributorAuthor

    Analysis and proposed solution

    Current behaviour — The issue’s performance plan is partly delivered. Source specs now run in three shards (workflow); TypeScript consumers run separately for compilers 6 and 7 (workflow); Storybook also has three cached shards (workflow). However, Source static validation remains one combined CI command, and its script still invokes both build and g:build (script). The first performs TypeScript compilation; the latter performs build-mode compilation, Rollup and CSS copying (definitions, shared build).

    Root cause / gap — The status block understates delivery, while aggregate steps still obscure individual compile, lint and bundle costs. Sharding does not establish completion of the local-loop measurements or before/after acceptance criteria. Redundant compilation remains a candidate, not a measured saving.

    Proposed change — Reconcile the status with the two merged PRs. Add per-command elapsed timings while preserving command order, exit codes and every validation. Record comparable cold/warm local results and CI wall time, queue time and runner-minutes. Then optimize one measured bottleneck per change; do not remove either compilation until its outputs and consumers are accounted for. Add timing-wrapper examples for successful and failing commands, with should-preserve-exit-status and should-execute-once assertions, following the existing Node tooling-test convention. Retain Storybook’s existing partition specifications.

    Size, risk, release intent — M; moderate risk from command ordering and artifact reuse; no-release. Sharding savings still need measured confirmation rather than estimates.

    Dependencies — Delivered prerequisites: #455 and #456.

  3. woksin commented on Oct 29, 2026

    @woksin
    ContributorAuthor

    Slice 1 delivered: duplicate packed-consumer type checks removed

    Merged in #473 (no-release).

    • TypeScript consumers (6/7) no longer repeat the public-types check that verify-public-types (6/7) runs from the same build recipe. verify-public-types is now in Build's needs.
    • verify-no-pixi-consumer and verify-spatial-consumer are removed: TypeScript consumers (7) already runs both checks, and they still run for compilers 6 and 7. That takes the workflow from 54 jobs to 52, matrix legs counted.

    Measured, job durations in seconds:

    Job Before (main runs 40892593269, 40860861536, 40845583404) After (PR run 40963541146, main run 40964386171)
    TypeScript consumers (6) 275, 243, 255 85, 91
    TypeScript consumers (7) 152, 85, 103 103, 77
    verify-no-pixi-consumer + verify-spatial-consumer 99, 87, 170 removed
    Total runner time per run 4848 (run 40892593269) 4215, 3695

    Remaining

    • The heaviest jobs are now Storybook (about 4 min per shard), Source specs (about 2 min per shard) and the renderer and package-manager matrices. The next measured step is to profile Storybook shard time.
    • Moving heavy jobs to the free self-hosted cratis-arc runners would cut hosted minutes further. That needs a decision on runner use and cost before anyone implements it.
  4. woksin commented on Oct 31, 2026

    @woksin
    ContributorAuthor

    Slice 2 merged in #474 (no release). The renderer-adapter matrix verifies all three package managers per adapter and boundary from one build and pack: 21 legs became 9, with all 21 verifications still logged. Renderer runner time fell from a main median of 1227 s to 650 s, and total jobs per run from 52 to 40. Measurements are in #474 (comment). Next candidates, one measured bottleneck per PR: the Storybook shards (about 80 s of duplicated package and static builds per shard, but sharing them adds critical-path time), and adding the renderer, package-manager and Arc-consumer jobs to the Build aggregator's needs.

  5. woksin commented on Oct 31, 2026

    @woksin
    ContributorAuthor

    Slice 3 merged in #476 (no release): the Build check now requires every job in the JavaScript Build workflow, so a failed or cancelled leg in any of them fails Build. main has no branch protection or ruleset yet; marking Build as a required check makes this block merges. Remaining candidate: the duplicated package and static builds in the Storybook shards (about 80 s per shard).

  6. woksin commented on Nov 3, 2026

    @woksin
    ContributorAuthor

    CI measurement after levers 1+2, at the train head of #491 (ee256d9). That head also carries #176 and other changes merged from main since the baseline, so the difference below is not attributable to levers 1+2 alone.

    JavaScript Build used 3,842 runner-seconds across 37 jobs, compared with a baseline of 4,401.5 seconds across 41 jobs (the average of two earlier runs, 41431195012 and 41429970261). That is 559.5 seconds, or 12.7%, lower. The job set changed:

    • Six release-verification jobs are gone, 530 s in the baseline: release-evidence, verify-published-exports, verify-package-graph, verify-provider-context-identity and verify-public-types for TypeScript 6 and 7.
    • Two Published artifact jobs are new: 339 s for TypeScript 6 and 7.

    Of the jobs present in both runs, most got shorter, but there is noticeable run-to-run variation, for example ±30 s on single jobs.

    Verify Fresh Dependencies used 64 s and Markdown Verification used 14 s. The baseline did not include those two workflows.

    Runs:

    Method: the sum over jobs of completed minus started time, so time spent waiting for a runner is excluded. There is one sample after the change and two before, and cache warmth was not compared.

    This issue stays open for the remaining levers.

  7. removed their assignment
    on Nov 4, 2026
  8. self-assigned this
    on Nov 21, 2026
  9. woksin commented on Nov 21, 2026

    @woksin
    ContributorAuthor

    #566 is merged. CI now runs the Source static checks as named steps, so each phase has its own time. The numbers below are from the PR's JavaScript build run (https://github.com/Cratis/Components/actions/runs/44935521153).

    source job (109 s in total): Yarn install 29 s, Lint Source 18 s, Build published Source artifact 15 s, Verify parts manifest 7 s, Typecheck stories 5 s, Report public API changes 5 s. Every other static step takes 4 s or less.

    Source specs shards: tests take 83–101 s per shard, plus about 18–24 s for Yarn install.

    Storybook shards (252–305 s each):

    Step Shard 1 Shard 2 Shard 3
    Build packages the composed Storybook imports 31 s 27 s 24 s
    Build composed renderer Storybook 65 s 60 s 51 s
    Test assigned Storybook units 156 s 134 s 123 s

    Each Storybook shard rebuilds the same packages and the same composed Storybook, which takes about 75–96 s per shard and about 4.5 runner-minutes per run. The static checks are already fast. The next step in this issue is a measured experiment: build Storybook once and share it with the shards. It only lands if the time saved is still larger once the artifact upload and download time is counted.

  10. woksin commented on Nov 21, 2026

    @woksin
    ContributorAuthor

    The experiment in #567 is measured and not merged. It shares one Storybook build with the three story-test shards.

    All checks passed in both runs. The table compares the PR run (https://github.com/Cratis/Components/actions/runs/44938476948) with the baseline from #566 (https://github.com/Cratis/Components/actions/runs/44935521153).

    Baseline Shared build
    Storybook runner time (producer + 3 shards) 827 s (305 / 270 / 252) 713 s (147 + 176 / 181 / 209)
    Whole workflow runner time 4393 s 4340 s
    Workflow wall time 334 s 396 s
    Storybook path, start to last shard 305 s 374 s

    Sharing the build saves about 114 runner-seconds per run. The download took 3 s, so the artifact transfer cost almost nothing. The cost is about 60–70 s more wall time, because the shards wait for the producer job and each still runs its own install and browser setup. The repository is public, so hosted runner time is free, and every pull request would wait about a minute longer for its result. That is a net loss, so #567 is closed. This is one observation per variant; the size of the wall-time penalty matches the plan's prediction.

    Still to do here: local per-phase timings (compile, lint, Vitest, build, story tests). The README now explains how to take them.

  11. woksin commented on Nov 21, 2026

    @woksin
    ContributorAuthor

    These are local per-phase timings on main at e77a8e2, measured on an Apple M4 Pro (12 cores) with Node 26.8.1 and Yarn 4.17.1. Dependencies were already installed. Each script was run once in ci:static order, then the main phases were rerun without cleaning.

    Source (yarn workspace @cratis/components run <script>):

    Phase First run Rerun without cleaning
    Lint (lint:ci) 2.9 s 2.1 s
    Compile (build, including parts generation) 2.1 s 1.1 s
    Package build (g:build, Rollup) 6.7 s 6.5 s
    Specs (test, Vitest) 69.7 s 57.9 s
    Remaining 9 ci:static scripts 19.5 s in total; the slowest is verify-parts-manifest at 6.7 s –

    clean does not clear the ESLint or TypeScript incremental caches, so lint and compile stay fast even after it runs.

    Storybook:

    Phase Seconds
    Builds of the five packages Storybook imports 14.8
    Preflight (ci:preflight) 3.7
    Composed Storybook build 27.1
    Index verification 0.7
    Story tests, all three partitions in one run 116.9

    Vitest and the story tests are about 90% of the local time. CI spreads them over three shards each. Locally, the full Source chain plus specs takes about 1.5 minutes, and the full Storybook sequence about 2.7 minutes.

    This completes the scope of this issue:

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions