Repository navigation
Carve out campus-core: phased separation of shared layers from service imports #970
Description
Goal
Carve the campus package so that shared layers (common / config / model / storage / webauth / services) are importable and installable without the service implementations, Flask, or database drivers — enabling a future campus-common / campus-core distribution split while campus-suite keeps deploying the full monolith.
Why
campus-suite is one distribution shipping every service plus flask, psycopg2-binary, pymongo, boto3, bcrypt, requests, gunicorn. The downstream Flask apps (campus-classroom, campus-admin, campus-profile) import only campus.common, campus.model, campus.audit.middleware, and the flask glue — yet must install three database drivers. At runtime, import weight is dominated by two avoidable module-level imports (requests, flask-in-model) that every campus.common / campus.model consumer pays for.
Recon (2026-10-27)
Layering as-is (arrows = "imports"):
L4 deploy
L3 auth (10.6k lines) api (3.4k) audit (6.3k) <- services
L2 yapper (655) flask_campus (824) <- bus + flask glue
L1 storage (4.0k) webauth (920) services.email (127)
L0.5 model (2.6k) <- domain models, all services mixed
L0 common (3.6k) config (146) <- leaves
Measured fresh-interpreter import weight (working set via GetProcessMemoryInfo):
| Import | Delta | Modules |
|---|---|---|
| baseline | 13.5 MB | 55 |
| bcrypt | +0.3 MB | +2 |
| requests | +15.3 MB | +341 |
| flask | +18.4 MB | +308 |
campus.common (today) |
30.4 MB total | 428 |
Findings:
campus/common/__init__.pyeagerly importshttp->requestsat module level (campus/common/http/default.py). Only campus.audit uses it. Also the only in-common path tobcrypt.campus/model/emailotp/template.pyimportsflaskat module level forrender_template_string— the only reason the model layer drags Flask+Jinja.campus.storage -> campus.modelinversion: interfaces/backends validate againstModel/InternalModel/constraints. Movable part is small and stdlib-only:model/base.py(221 lines) +model/constraints.py(20 lines).yapper <-> authcycle:campus/yapper/__init__.pylazily importscampus.auth.resourcesfor vault credentials;campus.authlazily imports yapper. Function-level laziness makes it work today but blocks any package split.campus.modelmixes models across services (audit: TraceSpan/APIKey; auth: Client/Grant/Vault; api: Booking/Timetable; emailotp) — importing any model loads all.
Good news: campus/__init__.py is already a pkgutil namespace package documented to allow extension by separate distributions — co-installing multiple distributions under campus.* needs no import-path changes. Storage DB drivers are already lazily imported per backend.
Phases
Phase 0 — import-weight cleanup (no packaging change)
- perf(common): load campus.common.http lazily so importing campus.common stops pulling requests #967 lazy-load
campus.common.http(drops requests + bcrypt fromimport campus.common) - perf(model): load flask lazily in emailotp.template so importing campus.model stays Flask-free #968 lazy flask in
campus.model.emailotp.template(drops Flask stack fromimport campus.model) - chore(common): bcrypt import-weight evaluation - verdict keep eager import (measured negligible) #969 bcrypt evaluation — verdict: keep eager (measured +0.3 MB, negligible)
Delivered as a stacked PR series base->tip: lazy-http -> lazy-flask.
Phase 1 — model kernel into common (design discussion ongoing)
Move model/base.py + model/constraints.py (stdlib-only, ~240 lines) into campus.common; campus.storage then depends only on common; campus.model re-exports Model/InternalModel/constraints for compatibility.
Phase 2 — break the yapper/auth cycle
Inject a credentials-provider callable into yapper.create() (auth passes its vault accessor) so yapper no longer imports campus.auth even lazily.
Phase 3 — distribution split (revised 2026-10-28: 5-dist layout)
The single-repo question was resolved as separate repos (campus-api-python precedent); each dist ships a PEP 420 namespace portion and is consumed as a version-pinned release wheel. campus.model left campus-core in the design correction (comment below): it is the churny, consumer-facing layer, so apps install common + model + flask-glue and never core.
| Distribution | Contents | Deps | Status |
|---|---|---|---|
campus-common |
campus.common |
bcrypt only | SHIPPED — v0.2.1; suite #987/#988; sdk#121/#122, classroom#66/#67, admin#31/#32, profile#43/#44, nyxchange#110/#111 |
campus-model |
campus.model (incl. http.py header value classes, #982) |
campus-common | SHIPPED — v0.1.0; suite #990; sdk#123, profile#45 (classroom/admin/nyxchange/cli have no direct campus.model imports → transitive, deliberate) |
campus-core |
campus.storage, campus.webauth, campus.services |
common + model; DB drivers as extras [postgres]/[mongo]/[s3] — never installed downstream |
pending |
flask-campus |
consumer layer: flask_campus + audit middleware/client/http (folds #980's module set) |
common + model + requests + flask + campus_python | pending |
campus-suite |
campus.auth, campus.api, campus.audit, campus.yapper, campus.deploy |
common + model (+ core/flask as they land); flask, gunicorn | ships the full monolith meanwhile |
Namespace mechanics landed in #986 (init-less portions + scripts/audit_wheel_namespace.py + PACKAGING.md rules). Interface kit per dist: closed-world __all__ tests + checked-in manifest + strict-0.x versioning (patch = additive, minor = breaking).
Ground rules
- Everything lands via stacked PRs into
weekly(worktree-first, one branch per PR). - No import-path changes for consumers; any moved symbol keeps a compatibility re-export.
- Contract suite stays green (blocking CI gate); each phase keeps
ruff check .green.
Activity
nycomp commentedon Oct 27, 2026
Revised phase plan (2026-10-27, agreed in review of phase 0)
Phase 0 merged (7515b6f). Two decisions from that review reshape what follows:
1. No base-model move.
campus.storage → campus.modelis a legitimate one-way dependency (model is stdlib-only post-phase-0), so the original phase 1 — movingmodel/base.py+constraints.pyinto campus.common — is dropped. It would splitcampus.modelacross two distributions with re-export shims for zero dependency benefit.2.
campus.common.httpmoves tocampus.audit. Usage audit shows its only production consumers are the audit service's client + web auth, plus the test harness seam (tests/flask_test/json_client.pymonkeypatchesDefaultClient). It is domain infrastructure (env-credential conventions, campus error envelopes), not common material. With it gone,requestsleaves campus-common's dependency list entirely.campus.common membership test (decided): domain-free infrastructure only — env, errors, introspect, schema, utils, validation, config. "Would a generic CLI tool import it?" Domain vocabulary (Model, HttpHeader, the HTTP client) never enters.
Phase 1 — boundary tidy-up (next)
campus.common.http→campus.audit.http; harness seam + TESTING-GUIDE routing section follow. Mechanical: ~3 production files, ~4 test files, shim removal incampus/common/__init__.py, import-weight test update.campus.commonimports nocampus.*;model/storage/webauth/servicesimport no flask/werkzeug and none ofauth/api/audit/yapper/flask_campus; storage drivers stay lazily imported.Phase 2 — yapper↔auth cycle (demoted, off the critical path)
Both yapper and auth land in campus-suite, so the lazy cycle is intra-suite and no longer blocks the carve. Still worth doing as a quality refactor (inject a credentials-provider callable into
yapper.create()).Phase 3 — distribution split (the main event)
campus-commoncampus.common(+ config)campus-corecampus.model,campus.storage,campus.webauth,campus.servicesrequests(webauth); extras[postgres]/[mongo]/[s3]campus-suitecampus.auth,campus.api,campus.audit,campus.yapper,campus.flask_campus,campus.deploycampus-suite→campus-common+campus-core; declare flask explicitly (currently transitive); stop installing psycopg2/pymongo/boto3.campus_python.json_client(changes the test seam; own issue if pursued).refactor(audit): move campus.common.http to campus.audit.http (#970)
test: package boundary contracts for the campus-core carve (#970)
nycomp commentedon Oct 27, 2026
Phase 1 delivered and merged 2026-10-27 (5909847, stack #975 → #976 → #977):
TestCampusRequest.authorizationfix — contract suite red onweeklywith current sdk (sdk PR to test sync-branches workflow #119 drift, second bug(devops): worktree bootstrap propagates stale locally-built campus_python - import campus_python broken in fresh worktrees and main venv #971 incident); green again everywhere.campus.common.http→campus.audit.http(harness seam + docs moved).requestsis out of campus-common's dependency list;import campus.commonnow loads zero third-party modules (bcrypt included — it left the eager chain with http/default).tests/unit/test_package_boundaries.py) — shared-layer allowed-deps matrix, no service-layer imports (even lazy), no module-level flask/werkzeug, common third-party = bcrypt only, storage drivers stay lazy. Mutation-verified; green on today's tree.Phase 2 (yapper) and phase 3 (distributions) remain HELD per plan; middleware consolidation parked in #980. Next natural checkpoint: phase 3 packaging decisions (config home, single-repo vs separate repos) when campus-core work resumes.
docs(yapper): consumer guide for the injected-credentials shape (#970)
nycomp commentedon Oct 27, 2026
Phase 2 delivered and merged 2026-10-28 (c6f034b, PR #981):
campus.yapperis self-contained — it imports no campus package at all. The lazyyapper ↔ authcycle is gone at the source.create()resolves the event DB URI in priority order:db_uriargument →db_uri_provider()callable →YAPPERDB_URIenv → remote vault via campus_python. TheDEPLOY=campus.authspecial-case branch is deleted (auth injects its in-process vault reader inget_yapper()); test fixtures export the seeded URI as env for barecreate()calls while the vault-delivery contract stays intact.SHARED_ALLOWEDwith zero allowed campus deps (statically enforced); newtests/unit/yapper/test_factory.pycovers resolution order and assertscampus.authstays unimported at runtime.docs/yapper.mdconsumer guide (URI resolution, per-caller recipes, migration note for the removed DEPLOY branch); corrected three package-tree listings and recorded the dependency rule in PACKAGING.md; module docstring names normalized repo-wide (172/172 match their paths).Verified: weekly CI green; main venv unit + contract green post-merge; dev deploy SUCCESS for campus.auth with clean boot logs and a live endpoint smoke (standard error envelope on the POST-only root route).
Remaining: phase 3 (distribution split; open decisions: config home, single-repo vs separate repos) and the parked #980 middleware consolidation.
nycomp commentedon Oct 28, 2026
Sweep status update 2026-10-28 — progress since the phase table was written:
Merged: Phase 0 (7515b6f: lazy common.http + lazy flask in emailotp template), Phase 1 (5909847: model kernel into common + boundary contracts incl. #977), Phase 2 (c6f034b: yapper/auth cycle broken via injected credentials provider; #981). Both phase-3 prep work items are done: emailotp template rendering moved into campus.api (#983, e1adc3a — campus.model is now Flask-free at call time) and the webauth header models remigrated into
campus.model(#984 + #985, closes #982 —campus.model.HttpHeaderrestored,campus.webauth.modelsdeleted, SDK flipped in campus-api-python#120, downstream relocked).Design correction (supersedes the phase-3 section above): after evidence review,
campus.modelleaves campus-core and becomes its own distributioncampus-model— it is the churny, consumer-facing layer (classroom: Question/Assignment/ClassroomLink; profile: User/Integration) while common stays stable. Revised 5-dist layout:campus-common(common, bcrypt only) /campus-model(pure dataclasses, deps common) /campus-core(storage+webauth+services, DB drivers as extras, never installed downstream) /flask-campus(consumer layer: flask_campus + audit middleware/client, deps common+model+requests+flask+campus_python — folds in #980's module set) /campus-suite(auth+api+audit+yapper+deploy). Apps install common+model+flask-campus, never core — making "downstream must not touch storage/webauth" physically true. The phase-3 table in this body still reflects the older model-under-core split; remaining work = pyproject surgery, unified versioning policy, per-dist install-smoke CI, downstream pyproject bumps (flask declared explicitly), PACKAGING.md rewrite.nycomp commentedon Oct 28, 2026
Phase-3 namespace groundwork landed: PR #986 merged
campus/__init__.pyaway —campusis now a native PEP 420 namespace portion (the deleted init was a pkgutilextend_pathshim with zero attribute consumers, repo-wide-grep verified; the SDK'scampus_pythonis a separate top-level namespace and unaffected).What shipped:
scripts/audit_wheel_namespace.py— durable guard: fails any wheel shippingcampus/__init__.py, a.py-bearing subpackage dir without__init__.py, or an empty portion. It caught a real pre-existing gap on its first run (campus/yapper/backends/shipped without an init; fixed in the same PR).docs/PACKAGING.md— "Namespace Package Rules (PEP 420)" section: init-less rule per contributing dist, the audit command, and rebuild-not-layer venv advice.Verification: 872 unit + full contract suite green;
poetry buildwheel passes the namespace audit; post-merge dev smokes clean (api root 200 / emailotp 401 envelope, auth standard 405 envelope; both deploys SUCCESS). Consumer import syntax is unchanged — this is pure packaging groundwork for the phase-3 dist split; downstream feels it only at their next relock.nycomp commentedon Oct 28, 2026
Phase 3, dist 1 of 5 landed: campus.common is now the campus-common distribution, and the monorepo plus all downstream consumers have been migrated onto it.
Shipped:
mainonly, branch protection requiring Ruff / Unit 3.11–3.13 / Build & Audit, labels mirrored):campus/commoncarved verbatim from weekly — verified stdlib + bcrypt only, zero campus-sibling imports, flask/werkzeug lazy-only per chore(common): campus.common still imports flask at import time via campus/common/utils/url.py #861 (enforced by its own AST boundary guardtests/unit/test_boundaries.py, which took over the common-specific rule from the monorepo's package-boundary contract).ab674bd): declarescampus-commonas a git dependency, ships nocampus/common;SHARED_ALLOWEDdrops the in-repo common entry (its AST rules still govern importingcampus.commonfrom the in-tree layers); tests split (10 suites moved;test_deploy/test_import_weight/test_openapi_specstay;test_url.pyreduced to the codespace-deploy test it uniquely covers). Also fixed ruff's src-based first-party detection for the now-cross-repo namespace (known-first-party = ["campus"]).campus.common), classroom Circles API #66, admin OpenAPI schema #31, profile Merge main #43 — all merged CI-green; nyxchange Comprehensive documentation #110 open pending its required review (checks green). cli carries zerocampus.commonimports → transitively satisfied, deliberately not relocked.Verified end-to-end: Railway dev deploys of campus.api + campus.auth built and smoke-clean against the carved wheel (build resolves the new public repo; namespace portions merge in containers). Import surface for consumers is unchanged; the remaining phase-3 dists (model, core, flask) repeat this exact pattern.
nycomp commentedon Oct 28, 2026
Phase 3, dist 2 of 5 landed:
campus.modelis now the campus-model distribution, and the suite plus its direct consumers are migrated onto it. (Phase-3 table in the body updated to the ratified 5-dist layout with per-dist status.)Shipped:
mainonly, ruleset requiring Ruff / Unit 3.11–3.13 / Build & Audit, labels + topics mirrored from campus):campus/modelcarved verbatim from weekly 3f04e68 — verified stdlib + campus-common only, zero campus-sibling imports, flask-free even at call time. Interface locked from day one: closed-world__all__tests, leaf-surface consistency (audit/classroom/emailotp/http__all__⊆ package__all__), checked-ininterface_manifest.json, strict-0.x versioning policy (README/CHANGELOG).campus-modelas a wheel dep, ships nocampus/model(wheel verified);SHARED_ALLOWEDdrops the in-repo model entry — the AST rules now govern what in-repo layers import FROM the externalcampus.model(storage/webauth allowed sets unchanged); model behavior tests moved to the campus-model repo, the suite keeps a consumer-side flask-free import check.campus.modelat runtime — profile'sUser/Integration, the SDK'sHttpHeader+ resource model types) — both merged CI-green. classroom/admin/nyxchange/cli have zero directcampus.modelimports → transitive via campus-suite, deliberately not relocked (same call as cli in dist 1); they pick the carved suite up at their next relock.One accuracy note baked into campus-model's README: a plain
import campus.modelloads bcrypt (viacredentials' use ofcampus.common.utils.secret) — pre-existing monorepo behavior, documented rather than smuggled into the carve.Verified end-to-end: worktree unit + contract + integration ALL GREEN; campus-model CI green incl. fresh-venv wheel smoke (pip resolves campus-common from its release, namespace merges, full 38-symbol surface imports); Railway dev campus.api + campus.auth deploys SUCCESS on e8eab29 and smoke-clean (api root/health 200, emailotp POST → standard 401 envelope behind the auth gate; auth GET landing 200, POST → standard 405 envelope,
healthz404 as expected).Remaining dists: campus-core (storage/webauth/services, DB drivers as extras) and flask-campus (consumer layer, folds #980). Open decision from the design: config's home (lean option: fold into common at the campus-core step).
nycomp commentedon Oct 28, 2026
Housekeeping note (2026-10-28, post dist 2): sdk#124 dropped campus-suite from campus-api-python — the SDK's campus.* surface is fully served by the campus-common + campus-model wheels, so the mutual branch pin (sdk pinned campus-suite@weekly while campus pinned campus-api-python@main) is severed. Future SDK relocks are independent of suite relocks; the suite side keeps its SDK pin (it genuinely consumes campus_python).
Remaining campus-suite consumers (2026-10-28 usage audit, feeds the flask-campus dist-4 plan — detail on #980): classroom/admin/profile (flask_campus + audit.middleware) and nyxchange (flask_campus + campus.deploy — deploy's home is the open wrinkle). cli declares no campus-suite at all.
Also merged today: #991 untracked requirements-freeze.txt (worktree-bootstrap artifact, accidentally committed in e7b025e — now gitignored).