Skip to content

fix: add shared network resilience seam (tablassert.net) - #163

Merged
SkyeAv merged 1 commit into
mainfrom
net/seam
Sep 15, 2026
Merged

SkyeAv merged 1 commit into
mainfrom
net/seam

Conversation

@SkyeAv

@SkyeAv SkyeAv commented Sep 15, 2026

Copy link
Copy Markdown
Owner

Adds src/tablassert/net.py, a stdlib-only network-resilience seam that classifies each failure as transient or permanent and retries transient ones with bounded jittered backoff — and rewires BABEL downloads onto it. Fleet motivation: a 16-worker run over 42,981 PMC articles lost 2,194 of 2,507 queue failures (87.5%) to DNS-shaped errors ([Errno -2] Name or service not known) during a resolver-outage burst; single-attempt fetches turned a recoverable blip into a terminal skip.

Classification

  • One ordered table, net.is_transient: DNS (URLError wrapping socket.gaierror), connection loss, timeouts, HTTP 408/425/429 and every 5xx, bare OSErrors with network errnos, and optional-library exceptions (openai/litellm/httpx) matched by name — never imported — are transient. ValueError/TypeError/KeyError, non-CC-license PermissionError, no-OA-files FileNotFoundError, other 4xx, and local-errno OSErrors (ENOSPC, EACCES) fail fast.
  • Never raises, walks the chain: hostile exceptions overriding __bool__/__cause__/errno cannot break classification; __cause__/__context__ are walked (≤ 3 links) so library wrappers keep their code.

Retry driver

  • net.retry_transient: 4 attempts, 1.0 s base doubling to 20.0 s, sleeps summing to ≤ 60.0 s; Retry-After headers and (reset after …) quota hints set the wait (max(hint, jittered step)), capped at 60 s. Exhaustion raises a coded NetworkTransientError (network-transient).

BABEL rewire

  • cli.download_babel_file drops its hand-rolled predicate for net.is_transient. Three intentional, test-pinned deltas: 425 now retried; permanent local OSError (ENOSPC on .part) fails fast instead of burning 5 attempts / 75 s; bare network-errno OSError keeps the 5-attempt guarantee. Range resume, streaming, and the retries=5 schedule unchanged.
  • Loguru pickling fix: retry warnings bound error=str(e) — the enqueue=True sink silently dropped lines binding raw HTTPError objects (unpicklable BufferedReader).

Design

  • Stdlib-only: the seam must be importable from tablassert.cli, which installs without the [agent] extra — tenacity/urllib3.util.retry would land in the base install.
  • Deferred: no docs/api/net.md page (the errors.DOCS_URL remediation is tracked as a separate project in 18.0.0's changelog).

Testing

  • uv run pytest tests/test_net.py -q --no-cov -n 029 passed in 0.08s
  • uv run pytest tests/test_cover_cli.py -q --no-cov -n 0 → all pass (BABEL classification deltas pinned)
  • Full gate on the stack tip: uv run pytest -q1565 passed, 15 skipped; uv run ruff check . / ruff format --check . → clean; uv run pyright0 errors; make test-rust + cargo clippy --manifest-path rust/Cargo.toml --all-targets -- -D warnings → clean
  • All tests offline: no sockets, no real sleeps (--durations=10 max 0.17 s); independent Tier-2 review passed after two repair rounds (regex classification, loguru pickling).

@coderabbitai

coderabbitai Bot commented Sep 15, 2026

Copy link
Copy Markdown

Important

  • 🔍 Trigger review

This repository does not receive automatic reviews because it has fewer than 10 stars.

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Advanced

Run ID: c821b1d9-9dfc-4599-adad-61138c630053


Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@SkyeAv
SkyeAv added this pull request to stack #170 September 15, 2026 17:40
@SkyeAv SkyeAv changed the title net/seam fix: add shared network resilience seam (tablassert.net) Sep 15, 2026
@SkyeAv
SkyeAv marked this pull request as ready for review September 15, 2026 18:07
Add a stdlib-only transient classifier and bounded jittered retry driver for DNS, socket, HTTP, and gateway failures. Reuse the classification table for BABEL downloads, preserve network errno retries, make retry logs safe for the enqueueing logger, and document the new module. Add offline coverage for fleet failure classes, retry budgets, and download behavior.
@SkyeAv
SkyeAv merged commit cab9bff into main Sep 15, 2026
5 checks passed
SkyeAv added a commit that referenced this pull request Sep 15, 2026
Cut 19.0.0 and bump the package version in pyproject.toml, CITATION.cff,
and uv.lock.

Major: five breaking changes ship. The `tablassert` console command now
requires the optional `[cli]` extra, since `cyclopts` and `rich` left the
base install (#187). Level-one normalization redefines fullmap keys as
cleaned Unicode-lowercase, Porter2-normalized, byte-ordered token sets and
moves the database to schema v6, so every schema-v5 fullmap is rejected and
must be rebuilt (#171, #172, #174, #175, #176). `build-fullmap --aria2c`
is gone -- aria2c is used automatically whenever the `[aria2]` extra is
installed (#178) -- and `--taxon-allowlist` is gone because the built-in
top-100 experimental-taxon allowlist now applies to every build, guarding
both reuse paths by `META.taxon_allowlist` identity (#153). Logging now
requires the `[log]` extra and is fully disabled without it (#152).

Features: `tablassert.net` is a new stdlib-only transient/permanent
classification and retry seam shared by the agent and the BABEL downloader
(#163, #164), with one bounded jittered retry layer across the inner agent,
reflexion, and judge (#167) and machine-readable `error_code` values on
skipped checkpoint records (#166). `build-kg` gained an automatic ephemeral
shared-prefix TCode cache keyed by content-addressed XXH64 op digests
(#155-#159, #161). The `--distill` corpus became schema-uniform v2 with a
sibling `outcomes.ndjson`, a deterministic `RewardConfig`-tunable reward,
and the new zero-dependency `tablassert distill-weigh` command (#180-#183).
`CLASS_FIELD_OVERRIDES` grants the DAKP sparse qualifier stack on the pinned
association classes (#188).

Fixes: `distill-export` partitions its corpus by content and unions the
schema across record files, so a mixed v1/v2 directory no longer CastErrors
or silently stringifies a column (#184); BABEL retry warnings survive the
loguru sink; PMC downloads are idempotent, atomic, and bounded-parallel
(#165); reflexion and judge prompts are hard-bounded (#168).

Changelog:
- versioned the Unreleased section as 19.0.0 - 2026-09-15
- merged three duplicate `### Added` blocks into one
- moved the two `BREAKING:` entries from `### Changed` into
  `### Breaking Changes` and added a `**Migration:**` note to all five
- added PR links to all 17 entries, which carried none
- added the missing entries: the distill v2 corpus/reward/distill-weigh
  stack (#180-#183), the distill-export schema-drift fix (#184), the
  TCode run-cache detail (#155-#161), and the README badge removal (#154)

Docs: none needed here. Every shipped doc change landed with its own PR,
and the docs source-of-truth gate passes against the bumped tree.

Testing:
- make check -> exit 0
- uv run ruff check . -> All checks passed!
- uv run pyright -> 0 errors, 0 warnings, 0 informations
- uv run pytest -> 1622 passed, 52 skipped
- cargo test --manifest-path rust/Cargo.toml -> 156 passed, 0 failed
- cargo clippy --all-targets -- -D warnings -> clean
- uv lock --check -> resolved 169 packages, lock current
- uv run mkdocs build --strict -> exit 0
- docs SSOT + CLI coverage after the changelog edit -> 204 passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant