Repository navigation
ci(stellar): make testnet snippet validation blocking (#187) - #208
Conversation
…l#187) Separate deterministic content checks from live provider availability so that real doc regressions block merges while Horizon outages do not. Exit-code contract (check-stellar-testnet-snippets.ts): 0 – all checks passed 1 – content regression (always blocks): file unreadable, snippet count below threshold, reachable endpoint returned wrong shape or 4xx 2 – provider outage (never blocks): DNS/TCP failure, timeout, HTTP 429/5xx from Horizon testnet MODE env-var selects which checks run: content – deterministic file checks only (blocking CI step) provider – live Horizon calls only (non-blocking CI step) all – both (default, used locally) Workflow changes (snippets.yml): - Remove top-level continue-on-error: true from stellar-testnet-snippets - Add blocking 'content' step (MODE=content, no continue-on-error) - Add non-blocking 'provider' step (MODE=provider, continue-on-error: true) emits ::warning:: annotation on outage, ::error:: on content regression - Add guard step that re-fails the job if the provider step found a content regression that continue-on-error would otherwise swallow Closes wraith-protocol#187
|
@biafra877-web Great news! 🎉 Based on an automated assessment of this PR, the linked Wave issue(s) no longer count against your application limits. You can now already apply to more issues while waiting for a review of this PR. Keep up the great work! 🚀 |
…r9p-f939-6h3c) Two high-severity advisories in follow-redirects affect the transitive path @stellar/stellar-sdk > axios > follow-redirects: GHSA-q3jh-wrc6-jgq8 – array-form headers bypass cross-origin credential stripping (<1.16.1) GHSA-8r9p-f939-6h3c – sensitiveHeaders regex escaping can expose credential headers (<1.16.1) @stellar/stellar-sdk@^13.3.0 has not yet released a version that pulls in a patched axios; add a pnpm.overrides entry to force follow-redirects to >=1.16.1 across the whole install until the parent ships a fix. Update DEPENDENCY-POLICY.md to record the override.
The previous commit added follow-redirects@<1.16.1 to pnpm.overrides in package.json but did not update pnpm-lock.yaml, causing frozen-lockfile to fail with ERR_PNPM_LOCKFILE_CONFIG_MISMATCH. Surgically update the lockfile: - Add follow-redirects@<1.16.1 to the overrides block - Bump resolution from 1.16.0 to 1.16.1 (updated integrity hash) - Update axios snapshot dependency reference to 1.16.1 - Update follow-redirects snapshot entry key to 1.16.1
truthixify
left a comment
truthixify
left a comment
There was a problem hiding this comment.
Blocking: .github/workflows/snippets.yml:208 cannot distinguish exit 1 from exit 2. With continue-on-error, a failed provider step has outcome=failure but conclusion=success, so the condition requiring both values to be failure is never true. A reachable Horizon 4xx or response-shape regression therefore exits 1 and still leaves the job green, contrary to #187. Capture the command exit status explicitly and fail only for status 1, while treating status 2 as the non-blocking outage case; add a workflow-level regression test or equivalent status-classification test.
With continue-on-error, GitHub Actions always sets conclusion=success, making the old outcome==failure && conclusion==failure condition permanently false. Exit 1 (content regression) was silently swallowed just like exit 2 (provider outage), leaving the job green on regressions. Fix: capture the raw exit status explicitly via $GITHUB_OUTPUT and branch on it in a separate classify step (if: always()): exit 0 → pass exit 1 → hard fail, blocks merge exit 2 → ::warning:: annotation, non-blocking other → hard fail Also add scripts/check-stellar-testnet-snippets.test.mjs with six node:test cases covering all classification paths, the core exit-1 vs exit-2 invariant, and a MODE=content integration smoke test against the real stellar.mdx. Wired as test:stellar-testnet in package.json and as a workflow step after classification. Closes wraith-protocol#187
It has been addressed, Please review |
truthixify
left a comment
truthixify
left a comment
There was a problem hiding this comment.
The raw provider exit status is now captured before continue-on-error can normalize the step result, and the classifier fails exit 1/unexpected values while allowing only exit 2 as a provider outage. Exact-head CI is green; locally, all 6 classifier/content tests and MODE=content validation pass.
…llar-testnet-snippet-validation-blocking ci(stellar): make testnet snippet validation blocking (wraith-protocol#187)
Problem
stellar-testnet-snippetsusedcontinue-on-error: trueon the whole job, solive Horizon incompatibilities silently passed and never blocked a merge.
Solution
Split the single validation step into two concerns with distinct exit codes and
different CI blocking semantics.
Exit-code contract (
check-stellar-testnet-snippets.ts)012A
MODEenv-var selects which checks run (content/provider/all),so CI can invoke them separately from the same
check:stellar-testnetnpm script.Workflow changes (
snippets.yml)continue-on-error: truefromstellar-testnet-snippetsMODE=content): deterministic file/snippet-count checks; nocontinue-on-errorMODE=provider): live Horizon calls;continue-on-error: true;emits
::warning::on outage and::error::on content regressionthat
continue-on-errorwould otherwise swallow)Done when
Closes #187