Skip to content

feat: integrate CANCER-SEEK ultra-sensitive pipeline + methylation cfRNA module (v2.3.0) - #7

Open
rollroyces wants to merge 25 commits into
mainfrom
feat/cancer-seek-integration
Open

rollroyces wants to merge 25 commits into
mainfrom
feat/cancer-seek-integration

Conversation

@rollroyces

Copy link
Copy Markdown
Owner

Summary

This PR consolidates the CANCER-SEEK ultra-sensitive screening scaffold from three sibling repos into the rollroyces/deepcatch umbrella as v2.3.0:

  • rollroyces/cancer-screening (archived) → src/ultrasensitive/ + src/screening_policy/
  • rollroyces/deepcatch-methylation (archived) → src/methylation/
  • rollroyces/cfDNA-Fragmentomics (archived) → no port; redirected to src/fragmentomics/ + sister pipeline

The 189-test DeepCatch fast-path gate is not affected: the new modules are gated behind the optional [ultra] extras.

What ships in v2.3.0

New code (12 modules)

  • src/ultrasensitive/umi.py — UMI-based error correction (duplex consensus)
  • src/ultrasensitive/variant_calling.py — low-VAF variant caller with CHIP filter
  • src/ultrasensitive/cnv_analyzer.py — CNV CBS segmentation (bug fix: __Logger → __name__)
  • src/ultrasensitive/methylation.py — beta-value methylation processor with DMR detection
  • src/ultrasensitive/xgboost_classifier.py — XGBoost + SHAP explanations (gated behind [ultra])
  • src/ultrasensitive/methylation_encoder.py — Transformer encoder (architecture-only; real-data validation deferred to Phase 4)
  • src/ultrasensitive/fusion_net.py — 4-modality FusionNet (reference; future routing through multimodal_fusion/)
  • src/ultrasensitive/vae_anomaly.py — β-VAE anomaly detector
  • src/ultrasensitive/spike_in_simulator.py — spike-in LOD validator (CLI: deepcatch-lod)
  • src/screening_policy/rl_agent.py — SAC-like RL agent + 10,000-patient simulated cohort
  • src/methylation/cfrna_coupling.py — read-side companion to mrnavax-rare
  • src/methylation/methylation_reader.py — methylation-array → persistence-input normalizer

New docs

  • docs/CANCER_SEEK_ARCHITECTURE.md — system architecture for ultra-sensitive pipeline
  • docs/ROADMAP_CANCER_SEEK.md — 12-month integration roadmap with workstream statuses
  • docs/legacy/DATA_SCIENCE_ML.md — historical CANCER-SEEK ML reference (byte-for-byte port)
  • docs/ultrasensitive/PHASE0_PLAN.md — methylation Phase 0 smoke validation plan

Build updates

  • Bumped version 2.2.0 → 2.3.0
  • Added [ultra] extras: xgboost, shap
  • Added [ultra-rl] extras: stable-baselines3, gymnasium
  • New deepcatch-lod console script entry point
  • New pytest marker ultra
  • Registered new packages: src.ultrasensitive, src.screening_policy, src.methylation

Doc updates

  • MODEL.md bumped to v2.3.0 with v2.3.0 changelog, new modality list, new limitations 11–12
  • README.md bumped to v2.3.0 with new CANCER-SEEK callout, new module reference table row, new repo structure block, new CLI examples

Tests (new)

  • test/test_ultrasensitive/__init__.py — 8 smoke-test stubs (UMI, CNV, variant caller, methylation, spike-in, XGBoost deps, ...), @pytest.mark.ultra
  • test/test_methylation/__init__.py — 7 smoke tests for cfRNA coupling + reader contract

Why this consolidation

Splitting the research across four sibling repos (this one, cancer-screening, deepcatch-methylation, cfDNA-Fragmentomics, the production cfdna-fragmentomics-pipeline) made cross-cutting changes expensive and obscured the unified headline benchmark. Consolidating under deepcatch:

  • Keeps the test surface honest (189 passing tests on a single pip install -e .)
  • Single-source documentation for cross-cutting claims
  • Removes redundant maintenance (LICENSE audits, branch-protection, README provenance)
  • Resolves the scope conflict in deepcatch-methylation (cfRNA vs CpG / FinaleMe) by adopting the CpG track — which is also the one documented in METHYLATION_PROJECT.md §9

Test plan

# Verify the existing 189-test fast-path gate is unbroken
env -u PYTHONPATH ./.venv/bin/python -m pytest test/ src/foundation/test_integration.py -m "not slow" -q
# Expected: 189 passed, 12 deselected

# Verify the new modules import without breaking default install
python -c "import src.ultrasensitive.umi, src.ultrasensitive.cnv_analyzer; from src.screening_policy.rl_agent import SoftActorCriticAgent; from src.methylation.cfrna_coupling import couple_cfrna_to_cdm; print('all imports OK')"

# Run the new smoke tests
env -u PYTHONPATH ./.venv/bin/python -m pytest test/test_ultrasensitive/ test/test_methylation/ -v -m "not slow"

# Optional [ultra] extras
pip install -e ".[ultra]"
deepcatch-lod --vaf_range "[1e-2, 1e-3, 1e-4]" --n_samples 5

Risk assessment

  • Low: the new modules are import-isolated from the existing 189-test surface (no cross-imports). Verified by inspection.
  • Low: the new tests are gated by @pytest.mark.ultra so the fast-path gate is not affected. The CI badge in the README continues to describe 189 passed, 12 deselected.
  • Medium: fusion_net.py duplicates src/multimodal_fusion/advanced_fusion.py (already migrated from sklearn-LR to PyTorch in Audit-2). Module docstring calls out the deprecation path.
  • Medium: methylation_encoder.py is architecture-only — reuses Python's transformer_layer without a real-data checkpoint. Real-data validation is Phase 4 of METHYLATION_PROJECT.md.
  • Honest disclosure: the CANCER-SEEK ultra-sensitive modules have no real-cohort benchmark. Any LOD or sens@spec number from these modules is a synthetic-fixture simulation output, NOT a clinical-grade result.

Migration / rollback

This is the only PR between v2.2.0 and v2.3.0. There are no breaking changes to the public API of any existing DeepCatch module — only additions under three new namespaces (src.ultrasensitive, src.screening_policy, src.methylation) and the optional [ultra] extras dependency.

To roll back: git revert this commit (or the merge commit) and re-tag v2.2.0 on main. The sibling repos remain on main with their README archived-banners pointing back here.

Companion-repo follow-ups

After merge:

  1. Mark rollroyces/cancer-screening, rollroyces/deepcatch-methylation, rollroyces/cfDNA-Fragmentomics as archived via the GitHub admin panel (README redirects are already in place).
  2. Close any open PRs on those three repos with a redirect to this PR.
  3. Update the rollroyces profile repo (README.md) to point to the consolidated umbrella.
  4. Cut a v2.3.0 Zenodo release once the existing fast-path gate passes on the merge commit.

🤖 This PR was assembled by MiniMax Code on Royce's behalf with full provenance in the commit messages of feat/cancer-seek-integration and the workspace MIGRATION_PLAN.md.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant