Skip to content

ci: shard backend tests and run test suites with compose run - #413

Merged
linearcombination merged 1 commit into
WycliffeAssociates:doc-dev.walink.orgfrom
linearcombination:improve-github-actions-ci-behavior-for-backend-tests
Nov 18, 2026
Merged

linearcombination merged 1 commit into
WycliffeAssociates:doc-dev.walink.orgfrom
linearcombination:improve-github-actions-ci-behavior-for-backend-tests

Conversation

@linearcombination

Copy link
Copy Markdown
Contributor

Closes: #411, Closes: #412

The API test job (~15 min) had become the longest job in every run, and a frontend shard repeatedly hit its step timeout after Playwright had already exited 0, because up --exit-code-from hung while stopping the stack.

Backend sharding:

  • Add pytest-split and run the backend suite across 3 matrix groups, passing PYTEST_SPLITS/PYTEST_GROUP into docker-compose.api-test.yml (--splits/--group). Defaults to 1/1 so local runs execute the full suite. xdist (-n auto) still runs within each group.

Test execution:

  • Replace up --exit-code-from with docker compose run --rm -T in both the API and frontend test jobs. run starts the runner's dependencies (honoring health checks) and returns the runner's exit code without tearing down the stack.

Timeouts and logs:

  • Set limits from observed durations (frontend shards ~3-6 min, API groups expected ~5-7 min), keeping inner < outer so failures surface cleanly: Playwright globalTimeout 15 min < test steps 20 min < jobs 25 min.
  • Cap the log-dump steps at 3 min and --tail=500 so they can't use up the remaining job time.

The API test job (~15 min) had become the longest job in every run, and
a frontend shard repeatedly hit its step timeout after Playwright had
already exited 0, because `up --exit-code-from` hung while stopping
the stack.

Backend sharding:
- Add pytest-split and run the backend suite across 3 matrix groups,
  passing PYTEST_SPLITS/PYTEST_GROUP into docker-compose.api-test.yml
  (--splits/--group). Defaults to 1/1 so local runs execute the full
  suite. xdist (-n auto) still runs within each group.

Test execution:
- Replace `up --exit-code-from` with `docker compose run --rm -T` in
  both the API and frontend test jobs. `run` starts the runner's
  dependencies (honoring health checks) and returns the runner's exit
  code without tearing down the stack.

Timeouts and logs:
- Set limits from observed durations (frontend shards ~3-6 min, API
  groups expected ~5-7 min), keeping inner < outer so failures surface
  cleanly: Playwright globalTimeout 15 min < test steps 20 min < jobs
  25 min.
- Cap the log-dump steps at 3 min and --tail=500 so they can't use up
  the remaining job time.
@linearcombination
linearcombination merged commit 50de21c into WycliffeAssociates:doc-dev.walink.org Nov 18, 2026
15 checks passed
@github-project-automation github-project-automation Bot moved this from In progress to Done in DOC Project Management Nov 18, 2026
@linearcombination
linearcombination deleted the improve-github-actions-ci-behavior-for-backend-tests branch November 18, 2026 20:45
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Development

Successfully merging this pull request may close these issues.

2 participants