Skip to content

test(sdk): cut SDK test time and skip build output in docs:check - #369

Merged
axonstone merged 5 commits into
mainfrom
test/faster-sdk-suite
Oct 30, 2026
Merged

axonstone merged 5 commits into
mainfrom
test/faster-sdk-suite

Conversation

@axonstone

@axonstone axonstone commented Oct 30, 2026 •

Copy link
Copy Markdown
Contributor

Summary

The SDK test task on CI drops from 92 to 100 seconds after #368 to 59 to 67 seconds, and pnpm docs:check no longer fails after a local VS Code build. These are the follow-ups listed in #368.

What changes:

  • Two capability tests, Hub enrollment and Compute Optimizer region failure, used to sit through the SDK's real retry backoff (0.5 + 1 + 2 + 4 + 8 seconds). They now run it on fake timers, like the existing status-probe retry tests, and still assert all six attempts. On CI they went from 33.7s to 19.8s and from 24.7s to 8.9s for their files.
  • The SDK suite runs 1.5 times as many Vitest workers as CPUs, six on the 4-vCPU runners instead of three. The discovery HTTP suites mostly wait on admission timers, so the other files no longer queue behind them. Vitest caps percentage values at the CPU count, so the config computes the number.
  • docs:check skips the Git-ignored dist, .turbo, and coverage folders. Before, a VS Code build left a staged README in packages/vscode/dist whose links only resolve in the published extension, and the check failed on it.

One risk to watch: more workers than CPUs could stretch the timer-paced tests, which is why Turbo already runs at --concurrency=2. Two CI runs showed no timeouts, and per-file times stayed within about 2 seconds.

Diagram

n/a, test and checker changes only.

Scope

  • @cloudburn/sdk (tests and test config only)
  • docs/community files (documentation checker)

Release Notes

  • No user-facing package changes in this PR

Verification

  • pnpm verify passes locally at the head commit. The new docs:test case failed before the checker fix with local link target does not exist and passes after it.
  • The two fake-timer tests passed 8 of 8 repeated local runs.
  • CI: SDK test task 79.6s with the backoff change alone (run 41061023875), then 59.0s and 66.8s across two attempts with six workers (run 41061694337).
  • --affected from ci: run only affected package checks and parallelize the SDK suite #368 now works on real PRs: the first run here selected 9 of 30 tasks and Validation took 105s.

Boundary Checks

  • No engine/parser/provider logic added to @cloudburn/rules
  • CLI delegates scan logic to SDK

Documentation

  • Owning pages and whether each was updated: updated docs/TESTING.md with the skipped output. docs/README.md and docs/reference/commands.md stay true: they list what docs:check validates, which is unchanged.

Related Issues

Refs #368

The Hub enrollment and Compute Optimizer region-failure cases waited through
the SDK's real 0.5+1+2+4+8s retry backoff, about 16s each. Run them on fake
timers like the status probe retry tests: waitFor advances the admission waits
before the first request, then the test advances past the backoff. Assertions,
including all six attempts, are unchanged. Locally 15.5s -> 0.3s and
16.6s -> 1.4s.
docs:check walked every directory except .git and node_modules, so after a
VS Code build it failed on packages/vscode/dist/vscode/media/README.md, a
staged copy whose links resolve only in the published extension. Skip the
Git-ignored dist, .turbo and coverage directories as well.
@axonstone axonstone added the impact:limited Two-way door: a revert fully undoes it, including anything already deployed or published label Oct 30, 2026

@devin-ai-integration devin-ai-integration Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ Devin Review: No Issues Found

Devin Review analyzed this PR and found no bugs or issues to report.

Devin Review

@dannysteenman
dannysteenman marked this pull request as ready for review October 30, 2026 13:12
@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Oct 30, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-10-30T13:27:25.957613Z c98c71b New commits
🔒 Security Review ✅ Completed 2026-10-30T13:27:55.464890Z c98c71b New commits
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

Vitest defaults to one worker fewer than the CPU count, three on the
4-vCPU CI runners. The discovery HTTP suites spend most of their time waiting
on admission timers, so the CPU-bound files queue behind them. Use 1.5x the
CPU count; Vitest caps percentage values at the CPU count, so compute it.
@axonstone axonstone changed the title test(sdk): stop waiting on real retry backoff in SDK tests test(sdk): cut SDK test time and skip build output in docs:check Oct 30, 2026
@towardsthecloud-ci

Copy link
Copy Markdown

roborev: Combined Review (c98c71b)

Verdict: No findings at or above medium severity.


Reviewers: claude-code | Synthesis: claude-code | Total: 20s

@axonstone
axonstone merged commit e300a34 into main Oct 30, 2026
5 of 6 checks passed
@axonstone
axonstone deleted the test/faster-sdk-suite branch October 30, 2026 13:38
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

impact:limited Two-way door: a revert fully undoes it, including anything already deployed or published

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant