Skip to content

perf(ci): cut the E2E critical path by 51% and the macOS unit leg by 15% - #1283

Merged
jeff-r2026 merged 6 commits into
Tencent:mainfrom
SaulMoro:perf/ci-shard-e2e-coverage
Oct 23, 2026
Merged

jeff-r2026 merged 6 commits into
Tencent:mainfrom
SaulMoro:perf/ci-shard-e2e-coverage

Conversation

@SaulMoro

@SaulMoro SaulMoro commented Oct 22, 2026 •

Copy link
Copy Markdown
Collaborator

Summary

CI is slow from work, not from queueing: runners start in 3–13 s. Two jobs set the wall time. On main and internal PRs it was the credentialed E2E job, which ran every E2E file on one runner under the fixture-repo lock. On every PR it is the macOS unit leg, whose suite spends 43% of its test time in 44.6k git processes.

 e2e  (fixture token, one runner, e2e-fixture-repo lock)
-  every E2E file                        913 s median on main
+  src/__tests__/e2e/e2e.test.ts only    the one file that uses the fixture repo
 fork-e2e-shard × 3  (no credentials)
-  fork PRs only
+  every PR and push                     slowest shard 374–449 s
 lint-and-test × 6
-  --coverage on every leg
+  --coverage on Ubuntu Node 20 only     the one leg that uploads it

Work the unit suite did for nothing:

 locateExclude(dir)       a config dir not created yet (.codex, .kiro/settings)
-  start git there, fail, answer null, uncached: again on every call
+  answer null without git                 same answer, 35% fewer git processes in mcp-reconcile and uninstall
 updateFileLocked         tests: _setExcludeLockRetryMs(1)   mcp-reconcile
 runLocalDriver           tests: _setDriverPollMs(20)        4 workflow driver files

Production keeps the lock retry and the poll interval.

Evidence

  • Before: E2E (GitHub provider, full surface): 913 s median over 40 main pushes.
    After: slowest credential-free shard: 449 s, 398 s and 419 s in this PR's runs (-51%).
  • Before: slow macOS unit legs: 838 s and 849 s job time (Vitest 807–814 s). Run.
    After: 713 s and 729 s (Vitest 675–698 s), -15%. Run.
  • Per file on macOS, before → after: mcp-reconcile.test.ts 136–145 s → 95–102 s; workflow-workspace.test.ts, touched only by the coverage change, 160–172 s → 123–133 s.
  • Coverage alone, paired on one macOS runner with the order swapped: about 67 s per leg.
  • Locally: mcp-reconcile.test.ts + uninstall.test.ts start 10918 → 7115 git processes; 51–52 s → 44–45 s and 17–18 s → 16 s, three runs each. Unit suite 12180 passed; tsc and lint clean.
  • Real CLI (npm run build, then npm run test:e2e): git-exclude-worktrees-915, git-exclude-delivered-915, claude-mcp-local-915: 73 passed.

Not measured: the credentialed e2e job, which fork PRs skip, so its first run is the merge to main. macOS runners vary: the same commit takes 300 s on some and 700–850 s on others; the numbers above compare the slow ones.

Ruled out: raising Vitest workers on the 3-vCPU macOS runner. 6 workers halved the leg, but 8 timed out submodule tests in workflow-workspace.test.ts, so the headroom is too thin to rely on.

Merge Danger

Door: two-way

Blast Radius: CI, and teamai pull / teamai uninstall when they look up a git exclude file for an agent directory that does not exist (same answer, without starting git).

If the narrowed e2e job fails on main, a revert restores the single-runner job.

Tencent#1210 pinned the POSIX `bash -lc` rendering in codex-team-hooks.test.ts,
and Tencent#1221 put that file on the Windows test list. On Windows the
command renders the Git Bash path, and the Windows list guard rejects a
pinned POSIX rendering in a file that does not pin the platform. Every
Lint & Test leg and the Windows job on main fail.
The full-surface E2E job ran every E2E file on one runner under the
fixture-repo lock, so it set the pace of every main push and internal
PR. Only e2e.test.ts uses the fixture repo: run that file alone under
the lock, and run the rest in the three credential-free shards on every
event.
@SaulMoro
SaulMoro force-pushed the perf/ci-shard-e2e-coverage branch from 04aae97 to 2df819d Compare October 22, 2026 07:15
@SaulMoro
SaulMoro marked this pull request as draft October 22, 2026 07:15
@SaulMoro
SaulMoro force-pushed the perf/ci-shard-e2e-coverage branch from c44449a to 2df819d Compare October 22, 2026 07:46
Unit tests that hold the exclude lock wait out updateFileLocked's whole
retry budget, and the workflow driver tests wait out its 2 s poll.
Production keeps both values; the test files that pay for them shorten
them through _setExcludeLockRetryMs and _setDriverPollMs.
@SaulMoro SaulMoro changed the title perf(ci): shard the internal E2E run and limit coverage to one leg perf(ci): cut the E2E critical path by 51% and shorten test lock waits Oct 22, 2026
@SaulMoro
SaulMoro marked this pull request as ready for review October 22, 2026 10:20
@github-actions

Copy link
Copy Markdown

No confident findings in the specified PR diff.

The description documents sufficient testing: it records two credential-free E2E shard runs and links the CI run, alongside unit-test results. The explicitly deferred credentialed E2E run is not blocking; the review rules require one representative real-CLI run, not the full provider × agent matrix.

Review was read-only; I did not run, build, or install PR code.

findMcpGitExcludes asks where each agent's config directory is excluded,
including directories not created yet (.codex, .kiro/settings, .trae).
For each one locateExclude started git in a missing cwd, which always
failed, and the null answer is not cached, so every call paid for it
again. The answer is the same null; it now comes without the process.

mcp-reconcile.test.ts and uninstall.test.ts start 35% fewer git
processes (10918 to 7115) and run 13% and 9% faster locally.
Five of the six unit legs ran with --coverage and discarded the report:
only Ubuntu on Node 20 uploads it, and no threshold reads it. In a paired
run on one macOS runner, coverage cost about 67 s of a 550-810 s leg.
@github-actions

Copy link
Copy Markdown

No confident findings in the specified PR diff. The earlier review raised no findings, so none require a resolution update.

The description documents sufficient testing: two credential-free E2E shard runs, a linked CI run, and unit-test results. The explicitly deferred credentialed E2E run is not blocking; one representative real-CLI run suffices under the review rules.

Review was read-only. I did not run, build, or install PR code.

@SaulMoro SaulMoro changed the title perf(ci): cut the E2E critical path by 51% and shorten test lock waits perf(ci): cut the E2E critical path by 51% and the macOS unit leg by 15% Oct 22, 2026

@jeff-r2026 jeff-r2026 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. CI/test-only change; approving per maintainer decision.

@jeff-r2026
jeff-r2026 merged commit decdf5b into Tencent:main Oct 23, 2026
18 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants